Introducing Substructure
By the Substructure Team
We’re building Substructure because working with LLMs in production data pipelines is harder than it should be.
The Problem
Every team building LLM-powered data processing ends up solving the same problems:
- Prompt iteration is expensive. Change one prompt, re-run the entire pipeline. No caching means you pay for every upstream node again, even if nothing changed.
- LLM output is unreliable. Malformed JSON, broken escape sequences in LaTeX, trailing commas, freeform text wrapping structured output. Every team builds ad-hoc parsing and retry logic.
- There’s no standard workflow format. Pipelines live as scripts, notebooks, or bespoke DAG code. Visual editing, versioning, and sharing are afterthoughts.
- Observability is bolted on. When something fails five nodes deep, you trace through logs to find which node got bad input. There’s no first-class way to inspect data at every step.
What We Built
Substructure is a workflow platform you author visually or in Python. Nodes are LLM prompts, code blocks, HTTP calls, and data transforms — connected by typed edges with schema contracts.
The key pieces:
Visual DAG editor — An infinite graph canvas with drag-and-drop nodes. Input, Output, LLM Prompt, Code, Branch, Map, Fan-out/Fan-in, HTTP Request, Schema Transform, and Dataset Sink. Edges enforce schema compatibility at design time.
Python SDK — Define the same workflows in Python. Node factories produce the same workflow document the editor saves, and Pydantic models work as output schemas.
Content-addressed caching — Every node result is cached by a hash of its config and input. Change a prompt and only that node re-executes. Resume failed runs from their last checkpoint instead of starting over.
Response repair — Markdown fence stripping, escape repair, JSON extraction from freeform text, structural fixes. Runs automatically before schema validation. When repair fails, edit the raw response and revalidate without making another LLM call.
Run inspector — Nodes on the canvas light up by execution state (running, completed, failed, cached). Click any node to see its actual input, output, timing, and cost.
Dataset sink — Write structured output to persistent, queryable datasets. Full provenance on every record — trace any output back to its source input and workflow version.
Architecture
The system runs as two isolated processes: an API server and an execution worker. They communicate through a task queue and shared database. Workers can crash without affecting the editor. Everything deploys with a single command.
GPT, Claude, Gemini, and more are supported behind a common interface with structured output, fallback chains, and per-node model selection.
What’s Next
We’re opening early access for the self-hosted version. If you’re building LLM data pipelines and want a better workflow, join the waitlist.