Recon AnalyticsAs our research / analytics stack scaled, we kept loading in more heterogeneous data - panels, survey codes, public series like FCC, BLS, and Census data, competitive feeds, and each one was structured and curated differently. Giving an LLM enough context to query all of it correctly, without just dumping the entire catalog into every prompt, was becoming a real risk to both speed and accuracy as we on-boarded new sources.
I prototyped, and the team built a governed metadata layer that sits under the AI engine: same governed layer under interactive reporting and AI query. For every new or reprocessed data source, a human sets the rules before any model runs against it: what the source is, what questions it can answer, how it joins to the rest of the warehouse, its trust tier, and its known gotchas. That becomes a confirmed instruction pack. Documentation can be auto-drafted, but automation never promotes itself into the live path; a person has to confirm it first.
At query time, a deterministic step resolves which sources are actually relevant to the question, then loads only those sources' confirmed instructions. The model never sees the full catalog. It writes SQL inside that narrow frame, and we keep the SQL so anyone can check exactly what ran. This keeps prompts small (so queries stay fast) and keeps the model from ever inventing rules for a source we haven't vetted (so answers stay accurate).
The system now governs over 250 independent data sources spanning more than 14 billion rows. Using deterministic tool calling alongside on-demand canonical query examples for each source cut model calls by up to 50% compared to blind queries against the same data.
