Trace where a metric comes from¶

lineage() walks the dependency graph resolve() already builds and hands back the part of it your node touches — the source columns that feed a metric, or everything a column change would reach.

from bloomery import Direction, Node, NodeKind, lineage, load_catalog, load_project, resolve

resolution = resolve(load_project(sources), load_catalog(catalog_text))

walk = lineage(
    resolution.graph,
    Node(kind=NodeKind.METRIC, name="metric.gross_revenue"),
    Direction.UPSTREAM,
)
for edge in walk.edges:
    print(f"{edge.src.name} --{edge.label}--> {edge.dst.name}")

It reads no data, opens no connection, and touches no warehouse. This is lineage of the spec — what your documents connect to what.

It answers one instant. For the other question a reader has about a node — how its definition has moved across a series of spec sets — see trace a definition over time.

The three questions it answers¶

Where does this number come from? Walk UPSTREAM from a metric and you get the chain back to the source columns, with the label on each edge saying how — a direct mapping, a catalog recipe by id, a macro by ref and version.

What breaks if I change this column? Walk DOWNSTREAM from a source column before a migration and you have the blast radius. plan() tells you what changed after an edit; this tells you what an edit would reach. Where the project declares exposures, the walk carries on past the metrics into the dashboards and syncs that read them.

Why is this metric unreachable, in full? SpecEvidence.unreachable names the missing leaves. The upstream walk shows the whole structure they sit in.

From the command line¶

$ bloomery lineage specs/ --node metric.average_order_value
metric.average_order_value  (upstream)
  canonical.quantity                   --requires-->           metric.gross_revenue
  canonical.unit_price                 --requires-->           metric.gross_revenue
  metric.gross_revenue                 --requires_metrics-->   metric.average_order_value
  metric.order_count                   --requires_metrics-->   metric.average_order_value
  order_item.quantity                  --canonical-->          canonical.quantity
  order_item.unit_price                --canonical-->          canonical.unit_price
  source.shopify__order_lines.$.qty    --direct-->             order_item.quantity
  source.shopify__order_lines.$.qty    --recipe:from_total-->  order_item.unit_price
  source.shopify__order_lines.$.total  --recipe:from_total-->  order_item.unit_price

--direction downstream walks the other way, --direction both merges the two, and --format json emits the same value the Python call returns.

Naming a node¶

A node id is the name you have already seen in a CircularDerivation message. Every kind but one carries its kind as a prefix:

Entity fields are the exception, so order_item.unit_price is a whole id rather than a suffix. Mistype one and the refusal suggests the spelling it thinks you meant:

$ bloomery lineage specs/ --node metric.gross_revenu
no node named 'metric.gross_revenu' in this project's dependency graph. did you mean: metric.gross_revenue

A mart's columns are not edges. A metric the mart carries as a measure draws one, and so does a rollup of it, but the entity columns a mart flattens do not — those come from the flattener, which runs a stage after the graph is built, and waiting for it would cost the walk its ability to answer on a project that does not yet compile. So a change to a mart dimension that no metric reads — order.status, flattened and grouped by — reaches the mart in fact and not in this graph. plan() still reports it as a mart change.

An exposure is the odd one out for a different reason: it is the graph's sink, it carries no SQL, and only dbt gets an artifact for one. Cube and SQLMesh have no such concept and emit nothing — without refusing anything, so declaring a dashboard never restricts what you can compile.

Identity, when the name is not it¶

metric.<name> makes the name the identity, so renaming a metric deletes one node and adds another: the history stops, and every citation of the old id points at nothing.

Give the node an id: and the name becomes a label:

metrics:
  gross_revenue:
    id: mtr_7f3a9c        # minted once, never edited

The node is now metric.mtr_7f3a9c whatever the metric is called, and a rename relabels a vertex instead of replacing one. The value is opaque — compared, never parsed — so any string that does not change will do; a generated-looking one is a convention, not a requirement.

Three things are worth knowing before you adopt it:

• It is write-once. Editing an id: is a delete and an add, and nothing can tell that apart from an actual delete and add — that distinction is exactly what the id was carrying. The compiler cannot enforce this; you can.
• Two nodes of one kind may not mint the same id. Copying a spec file and leaving the id: behind is refused, and so is an id: equal to another node's name — during a partial rollout, that is the easier of the two to write by accident. Both are refused as the document is read, before anything builds a graph from it.
• Adopting nothing costs nothing. A project with no id: anywhere gets exactly the ids and exactly the artifacts it gets today.

You ask by the id and read the name. --node takes metric.mtr_7f3a9c, because that is what the graph calls the node; the printed walk shows metric.gross_revenue, because an opaque key is not what a name is for. --format json carries both — the nodes keep their ids and a labels object maps each to its readable spelling, so a script that keys on ids is untouched and one that renders a graph has something to render:

$ bloomery lineage specs/ --node metric.mtr_7f3a9c --format json
{
  "labels": {"metric.mtr_7f3a9c": "metric.gross_revenue"},
  "nodes": [{"kind": "metric", "name": "metric.mtr_7f3a9c"}, ...]
}

A did-you-mean is the exception and deliberately so: a suggestion exists to be retyped, so it comes back as the id --node accepts rather than as the name the walk prints.

Where the ids pay is across versions: renaming a metric becomes one change in plan() instead of a deletion and an addition, and a definition's timeline keeps one node across the boundary instead of showing a delete and an add.

It returns a sub-DAG, not paths¶

Lineage.nodes carries each node once, however many ways it is reachable, and Lineage.edges carries every connection between them. A node fed by two sources appears once in nodes and twice in edges.

That is deliberate. Enumerating root-to-leaf paths is exponential in the graph's width — a metric over three canonical fields, each mapped from four sources, has 64 paths through twelve edges — so the size of the answer would not be something you could predict from the size of your project. The sub-DAG is bounded by the graph. If you want paths, walk the sub-DAG yourself, where you can put a budget on it.

The command line prints an edge list for the same reason: a tree cannot draw a DAG without either repeating a shared node or dropping one of its edges.

An empty answer is an answer¶

A source column has no upstream. A metric nobody composes has no downstream. Neither is an error, and both come back as a one-node Lineage rather than a raise — nodes always contains the root.

$ bloomery lineage specs/ --node metric.order_count
metric.order_count  (upstream)
  no upstream lineage — this node is a leaf in that direction

metric.order_count is agg: count with no requires:, so it genuinely depends on no canonical field. Nothing is missing.

Bounding the walk¶

--max-depth N stops the walk N edges out; the root is depth 0. A bounded result sets truncated and the command line says so, because a partial answer that cannot say it is partial is worse than no answer:

$ bloomery lineage specs/ --node metric.average_order_value --max-depth 1
metric.average_order_value  (upstream)
  metric.gross_revenue  --requires_metrics-->  metric.average_order_value
  metric.order_count    --requires_metrics-->  metric.average_order_value

  truncated: --max-depth stopped the walk; there is more beyond this

Bounding to nothing and finding nothing are different facts, and each says so in its own words. --max-depth 0 on a node that has lineage shows none of it and says why:

$ bloomery lineage specs/ --node metric.average_order_value --max-depth 0
metric.average_order_value  (upstream)
  --max-depth stopped the walk before its first edge — this node has upstream lineage, and none of it is shown

That is not the leaf message above, and it must not be: "this node is a leaf" is the half a reader acts on, and here it would be false. --max-depth 0 on a genuine leaf is not truncated, because there was nothing to cut.

What it does not do¶

It is not column-level lineage through SQL. A catalog recipe's body is arbitrary SQL and a step is an opaque implementation. The edge says recipe:from_total fed order_item.unit_price; it does not say which expression inside that recipe used which input. bloomery does not parse recipe bodies for this, and a page that implied otherwise would be promising an analysis it does not run.

It is not runtime lineage. This is what your specs connect, not what actually ran. SQLMesh and dbt both keep their own record of that.

It is not a claim about correctness. A clean chain from source to metric says the spec is wired the way you wrote it. Whether the source column holds what it claims is the quality system's question.