The wide-mart gold layer¶

This page explains the doctrine behind bloomery's serving layer: gold is one wide, pre-joined mart per (grain × subject area), with every dimension flattened in at build time, so the common query path contains no joins at all. It is a constraint on the physical layer, chosen deliberately — and it is what turns the classic semantic-layer failure modes from runtime hazards into compile-time errors.

Why no query-time joins¶

Every classic semantic-layer failure is, at root, a query-time-join problem. Because bloomery's operator controls the physical layer, the join is moved to build time, where the compiler can verify it once against declared relationship cardinalities instead of re-planning it per query:

Declaring a mart¶

Marts are the fifth spec kind. Each mart names a base entity, flattens related entities in through declared relationships, and expands date roles:

marts_version: 1
marts:
  order_items:
    grain: order_item            # must equal the base entity's grain
    base: order_item
    flatten:
      - {via: item_of_order,     prefix: order_}
      - {via: order_of_customer, prefix: customer_}
      - {date: order_date, role: ordered}    # → ordered_day … ordered_year
      - {date: ship_date,  role: shipped}
    measures: [gross_revenue, discount, net_revenue, quantity]
    partition_by: [days(ordered_day)]
    cost_hint: 3                 # relative scan cost; tie-breaking only

The compiler validates every line of this at compile time: a measure whose grain differs from the mart's grain is a GrainViolation; a flatten step through anything other than a many_to_one or one_to_one relationship is FanoutRisk; post-prefix column collisions are errors, never auto-renames; and a mart that carries measures must declare at least one date role (MartMissingTimeDimension). Fan-out is refused where it would be built, not detected where it would be summed — the guardrails page shows the exact messages.

Historical dimensions need an anchor¶

An entity declared scd: type2 keeps its history: the silver relation holds one row per version per key. A mart can flatten one, and must say as of when:

flatten:
  - {via: order_of_customer, prefix: customer_, as_of: order_date}

as_of names a date or timestamp column of the mart's base entity — the fact's own date. That is the anchor the dimension is read at, so each order carries the customer segment that was current when the order was placed, not the segment as it is today. Two orders of the same customer can legitimately disagree about that customer's attributes, and that disagreement is the feature: it is what point-in-time attribution means.

The emitted join gains the validity predicate:

LEFT JOIN silver.customer AS customer_
  ON  "order".customer_id = customer_.customer_id
  AND "order".order_date >= customer_.valid_from
  AND (customer_.valid_to IS NULL OR "order".order_date < customer_.valid_to)

The interval is half-open, so a version ending exactly when the next begins matches one of them rather than both, and valid_to IS NULL is the current version. Both targets are configured to spell that interval valid_from/valid_to — SQLMesh's own default names, and a rename on dbt, whose snapshots would otherwise call them dbt_valid_from and dbt_valid_to. Bloomery owning the two names is what lets one join serve both targets.

An order whose date precedes every version of its customer matches nothing and its flattened columns are NULL, which is what a LEFT JOIN does everywhere else in a mart.

Without an anchor, it is refused¶

Omit as_of and the flatten is HistoricalFanout. The join would be an equality on the relationship's columns and nothing else, so it matches every version of each key and multiplies the base grain by the version count. Nothing downstream notices: the declared cardinality: many_to_one is a claim about the domain and it stays perfectly true, so FanoutRisk sees nothing wrong, the row counts look plausible, and the number is silently multiplied. Filtering to the current version automatically was considered and rejected for the same reason — it silently converts a historical dimension into a current-view one, so a mart whose whole purpose is point-in-time attribution returns today's segment for a two-year-old order.

An as_of on a dimension that is not scd: type2 is refused too: there is no version to choose between, and the validity columns it would join against do not exist.

Reading the current version¶

When the question is about the dimension as it is now, declare that instead of an anchor:

flatten:
  - {via: order_of_customer, prefix: customer_, reading: current}

reading: current reads only the current version — valid_to IS NULL — and it is a declaration, not a default: omit both keys and the flatten is still HistoricalFanout. On a flatten hop the predicate joins the ON clause, not a WHERE, so an order whose customer has no current version keeps its row with NULL customer columns:

LEFT JOIN silver.customer AS customer_
  ON  "order".customer_id = customer_.customer_id
  AND customer_.valid_to IS NULL

It is the same predicate the silver audits scope to the current version with, so a mart and an audit cannot disagree about which version is current.

Two declarations of it are refused as HistoricalFanout:

• reading: current on an entity that is not scd: type2. It holds one row per key, so there is no current version to choose and no valid_to column to read.
• reading: current beside as_of: on one step. One reads the version current now, the other the version current at the anchor, and a hop reads one version.

A base: is historical only when read current¶

A mart based on a type2 entity is refused, and an anchor cannot rescue it. There is no join to qualify. The mart declares one row per entity while the relation holds one per entity per version, so a count returns revisions — a grain lie no join predicate addresses. Declare reading: current on the mart and the base is filtered to one row per key:

marts:
  customer_current:
    grain: customer
    base: customer
    reading: current

which lowers to WHERE customer.valid_to IS NULL. The other route is unchanged: base the mart on a type1 current-view entity built from the historical one.

A current-reading mart is built whole¶

A mart that reads current versions — on its base or on any flatten step — must be materialization: full; declaring it incremental is refused. A version stops being current after the partition holding it was written, and no incremental run revisits that partition, so the mart would keep reading it as current. A mart with a partition_by defaults to incremental_by_partition, so a partitioned current-reading mart has to say materialization: full explicitly.

scd: type2 itself is unchanged. It remains fully supported as a silver target on both SQLMesh and dbt.

Role-playing dimensions¶

An order has an order date and a ship date — the same date dimension playing two roles. This is the concept that breaks naive semantic layers (joining one dimension table twice under different foreign keys). Bloomery models it once, as a dimension reference carrying an optional role, and lowers it per consumer. In the mart, each date: role expands at build time into bucket columns — ordered_day, ordered_week, ordered_month, ordered_quarter, ordered_year, and likewise shipped_* — so grouping revenue by ordered_month and by shipped_month reads two different physical columns and gives two different, correct answers, with no join and no aliasing anywhere near the query.

An unqualified reference to a dimension with multiple roles is refused with the roles named:

AmbiguousDimension: 'date' has roles [ordered, shipped]. Use 'ordered_date' or 'shipped_date'.

One definition, two consumers¶

A mart definition is compiled once into the IR and read by two consumers that must never disagree:

The SQLMesh emitter lowers the mart into the gold-layer model that builds the wide table — the base entity joined once per flatten step and once per date role. That model is the only place joins are ever emitted for a mart. The planner then treats the built table as a joinless catalog to select from. Because both read the same IR object, the thing that builds the table and the thing that queries it cannot disagree about its grain or its columns.

Marts are what the semantic manifest is made of¶

The MetricFlow manifest emitter maps one mart to exactly one semantic model: the mart's grain entity becomes the primary entity, its flattened columns become dimensions (each date role a time dimension), its measures carry their aggregation and time dimension, and the catalog's date dimension becomes the declared time spine — pointing at the same gold.dim_date relation the SQLMesh emitter builds. The emitter never produces a semantic model for a non-materialized entity, because that would reintroduce the query-time joins this whole design exists to prevent.

Cross-grain requests are aggregated first, or refused¶

A request whose metrics live on different grains has no correct answer as a single joined scan — summing across grains double-counts. It does have a correct answer as two aggregates joined afterwards, and that is what the planner builds when it can prove the join is safe: each measure is aggregated on the mart that owns it, and the results are joined on the key they share.

One thing must hold, and it is checked before anything is planned: every requested dimension must be present on every branch, and be the same dimension on each — same source entity and same source column, not merely the same name.

A filter and a row policy are held to that rule too, and to it alone. Each is placed on every branch, resolved to that branch's own spelling of the dimension, and a restriction one branch cannot evaluate refuses the whole request rather than being applied to part of it — half a filter does not give a narrower answer, it gives a restricted number beside an unrestricted one at the same key. An ordering and a limit apply to the joined result, never inside a branch, so they need no rule: a limit inside a branch would answer from a prefix of that branch.

A metric's own filter: is different, and composes. It restricts one measure by declaration rather than the whole request, so it rides the branch that owns that measure and no other: eu_order_count beside an unfiltered line_discount is planned, and the explanation says what the filter restricts on that measure's line — [additive — COUNT (restricted to region eq ['EU'])]. The joined domain is still every key any branch produced, so a group the filter emptied reads NULL for that metric; the restriction named on the column is how a reader tells that NULL from a group with no rows at all.

A metric whose own components live on different marts — a ratio, or a derived: expression — is answered too. Each component is aggregated by the mart that owns it and the expression is evaluated once, over the join. revenue / quantity across two marts is SUM(revenue) / SUM(quantity) at the requested grain, which is not the same number as a row-level revenue / quantity summed afterwards. Its inputs' own filters are named on its column, as above. Three shapes stay refused: a component that itself needs two marts, an input read at a time offset, which names a grain no branch produced, and a derived: metric whose inputs are restricted differently — two different own filters, or a filter on one input and none on another. That last one is refused on one mart as well as across two, because one number folded from two row sets is about neither:

RatioOperandsDisagree: derived metric 'orders_outside_eu': eu_shipping_count is
restricted by region eq ['EU'], shipping_count is restricted by nothing (R019
operands_disagree) — required: eu_shipping_count and shipping_count restricted to one
row set. Fix: restrict every input the same way, or none

A derived: metric whose inputs agree — all unfiltered, or all under the same filter — composes as they do.

Where any of that fails, the request is refused with the conflict named:

UnreachableAtGrain: metrics {shipping_cost, line_discount} live on different grains
  shipping_cost   → grain: order      (mart: gold.mart_orders)
  line_discount   → grain: order_item (mart: gold.mart_order_items)
  Summing across grains would double-count. Request them separately,
  or define a mart at the shared grain.

A name both marts carry and disagree about is refused on its own terms, because it is the one that looks answerable:

UnreachableAtGrain: dimension 'order_id' is carried by every mart this request needs,
and they do not mean the same column by it:
  order_items          → order_id (from order_item.order_id)
  orders               → order_id (from order.order_id)
  Two columns are the same dimension when they come from the same source column, not
  when they share a name.

And a request filter or row policy that does not reach every branch names itself and the branches that lack it, because the fix is a mart change rather than a request change:

UnreachableAtGrain: filter `region = 'EU'` does not reach every branch: customers cannot
apply it.
  filter dimension 'region' is not carried by every mart this request needs:
  customers            → not carried
  orders               → region
  A restriction placed on some branches and not others narrows one measure and not the
  other, and the join reports the two side by side as though one restriction applied
  throughout.
  Request the measures separately, or flatten 'region' onto every mart above.

The embedded MetricFlow engine would happily plan a multi-hop join across semantic models — so bloomery runs a coverage precheck before MetricFlow sees the request: all measures of one branch on one mart, all requested dimensions resolvable on it, and among multiple covering marts the cheapest cost_hint wins with ties broken lexicographically. Refuse-don't-guess is thereby enforced twice — once by the precheck, once by MetricFlow's own resolver behind it. Belt and braces is correct here: the product rule is that the system may not know the answer, but may not return a wrong one without warning.

No join happens before an aggregate. The join a cross-grain request needs is over relations that are already one row per key, so it cannot fan out; a join of raw rows at query time stays refused, and the mart is still where a join between entities belongs — proven once at build time instead of per request.

Marts sit at the end of the pipeline described in The compile pipeline; the additivity classes their measures carry are explained in Specs and the catalog.