« Phase 29 README · Warmup · Hitchhiker's · Principal Deep Dive · Core Contributor · Staff Notes

Phase 29 — Deep Dive: LangChain Core

The whole of LangChain core reduces to one abstract base class and one algebraic property. The class is Runnable. The property is closure under composition: the composition of Runnables is itself a Runnable. Everything else — streaming, batching, fallbacks, retries, tracing, RAG, the agent loop — is a consequence. If you understand the mechanism at that level, you can read any LCEL expression as a data-flow graph and predict its behaviour without running it.

The load-bearing abstraction: Runnable

A Runnable is a component exposing a fixed interface: invoke(input) -> output, batch(inputs) -> outputs, stream(input) -> Iterator[chunk], plus their async twins ainvoke/abatch/astream. That is the entire contract. A prompt template is a Runnable (dict -> PromptValue). A chat model is a Runnable (messages -> AIMessage). An output parser is a Runnable (AIMessage -> str/obj). A retriever is a Runnable (str -> list[Document]). A bare Python function becomes a Runnable via coercion. Because the type signature of each differs but the method surface is identical, they can be dispatched uniformly.

The critical design move: the three verbs are not independent. There is an invariant tying them together — invoke(x) must equal reduce(operator.add, stream(x)). Streaming produces chunks; folding those chunks with + must reconstruct exactly what invoke returns. For text that is string concatenation; for chat messages it is AIMessageChunk.__add__, which concatenates content and merges partial tool_calls by index. Hold this invariant in your head: it is why a chain can offer both a token-by-token UI stream and a single blocking call from the same code path, and it is the property a correct custom Runnable must preserve.

Composition and coercion: how | builds a graph

prompt | model | parser does not "call" anything. The __or__ operator constructs a RunnableSequence — a Runnable holding an ordered list of steps. Nothing executes until you call invoke/stream. So the pipe is graph construction, not evaluation; the object you get back is inert and reusable.

Two coercion rules make LCEL feel like Python instead of ceremony:

  • A dict literal in a pipe position becomes a RunnableParallel. {"a": r1, "b": r2} runs r1 and r2 on the same input (concurrently, on a thread pool) and assembles {"a": r1(x), "b": r2(x)}. This is fan-out.
  • A plain callable becomes a RunnableLambda. That is why you can drop format_docs straight into a chain.

Add three structural primitives and you have the whole algebra: RunnablePassthrough (identity — pass input through untouched), RunnablePassthrough.assign(k=...) (compute new key k, merge it into the input dict, keep everything else), and RunnableBranch (evaluate predicates in order, run the first matching branch — routing). Resilience is bolted on as wrapper Runnables: .with_fallbacks([...]), .with_retry(), .with_config(...), .bind(...). Each returns a new Runnable wrapping the original, so they compose too.

The streaming mechanism (where naive implementations fail)

Here is the part most tutorials get wrong. Streaming through a sequence is not "call each step, stream the last one." It is implemented with a fourth method, transform(iterator_of_inputs) -> iterator_of_outputs. A RunnableSequence streams by threading iterators: step N's output iterator becomes step N+1's input iterator. A model yields token chunks; StrOutputParser implements transform to pass each chunk through as it arrives; so model | StrOutputParser() streams token-by-token end-to-end.

But the default transform on any Runnable that doesn't override it does the only safe thing: it buffers the entire input iterator, joins it, then calls stream once. So the instant you insert a step that needs the whole value — RunnableLambda(lambda s: s.strip()), a json.loads, any post-processing of the complete string — that step buffers, and everything downstream loses incrementality. The UI freezes until the full text is assembled, then dumps it. This is why the rule is "the model plus a stream-capable parser go LAST." Streaming is not a feature you turn on; it is a property of pipeline shape, and a single innocent trailing lambda silently destroys it. The mechanism (buffering transform) explains the symptom exactly.

Worked trace: the RAG chain

Take the canonical Lab 02 chain:

{"context": retriever | format_docs, "question": RunnablePassthrough()}
  | prompt | model | StrOutputParser()

Invoke it on the string "what is our refund window?":

  1. The dict coerces to RunnableParallel. Both branches receive the same input string. Branch context runs retriever.invoke(q) producing list[Document], then format_docs flattens them to a string. Branch question is RunnablePassthrough — it returns the input verbatim. The two run concurrently; the step yields {"context": "...", "question": "what is our refund window?"}.
  2. prompt (a ChatPromptTemplate) consumes that dict, filling {context} and {question} slots, emitting a PromptValue (list of messages).
  3. model consumes the messages, returns an AIMessage.
  4. StrOutputParser extracts .content.

Citations are pure plumbing, not a model capability. To attach sources you don't consume the documents — you carry them: replace the tail with RunnablePassthrough.assign(answer=(prompt | model | parser)) so the retrieved Document list survives alongside the generated answer in the output dict. The empty-retrieval failure mode is also mechanical: if retriever returns [], format_docs yields an empty context and — unless the prompt explicitly instructs refusal-on-empty — the model freestyles a confident hallucination. The bug lives in retrieval; it presents as a model bug.

Worked trace: the AgentExecutor loop

The classic AgentExecutor (Lab 03) is a while loop over an internal intermediate_steps: list[tuple[AgentAction, observation]]:

  1. Render intermediate_steps into a scratchpad and feed it to the agent — itself a Runnable (prompt | llm.bind_tools(tools) | output_parser) that returns either an AgentAction (tool name + input) or an AgentFinish (final answer).
  2. If AgentFinish: return, optionally with intermediate_steps when return_intermediate_steps=True.
  3. If AgentAction: look the tool up by name, execute it, and turn the result — including any exception — into an observation string appended to intermediate_steps. An unknown tool or a raised error becomes text the model reads next turn; that is the repair loop.
  4. Loop until finish or max_iterations (default 15), at which point it force-stops with the literal "Agent stopped due to iteration limit.".

The load-bearing insight: the control flow — the loop, the decision to continue, the pause point — is buried inside a class method. You cannot address it, checkpoint it, or wedge a human approval between "decide" and "act," because there is no external state to inspect. That opacity is not a bug to route around; it is precisely the constraint that motivates LangGraph, which lifts this exact loop into explicit, persisted graph state.

Why the naive design fails

Build this without the Runnable closure and every capability multiplies. Streaming, batching, retry, fallback, and tracing must be reimplemented for each component type, and a five-step chain has no single seam to attach them to. LCEL's answer is to write those behaviours once against the interface — because a RunnableSequence is a Runnable, the same stream/batch/with_retry machinery that works on a model works on the whole pipeline. The abstraction earns its keep the moment the chain has more than one step. Its limits are equally mechanical: RunnableSequence is a DAG with no back-edges, and RunnableConfig carries no persisted state between invocations — so the instant your control flow needs a cycle or durable memory, you have left LCEL's design envelope and entered LangGraph's.