Many teams building text‑to‑SQL tools hit the same wall: a query can be syntactically perfect yet return the wrong answer because the model never saw how the data actually looks. Schemas tell you tables and column types but they do not reveal whether a county is stored as “Alameda”, “Alameda County”, or “ALAMEDA”, which join creates duplicates, or what literal values a flag column uses. When the model guesses, it can join the wrong table, filter on a missing value, or mis‑read an ambiguous column, and none of these mistakes throws an error, so execution alone never catches them.
The BIRD benchmark makes this problem measurable by scoring execution accuracy against real‑world databases that contain imperfect values and ambiguous relationships. Feyn’s SQRL family changes the workflow from pure translation to inspection first. For each natural‑language question the model receives the schema and, if the schema alone is enough, it emits a SQL statement immediately. When ambiguity remains, SQRL issues read‑only probe queries, observes the returned rows, and only then crafts the final answer. The interaction uses two actions: an observation request that fetches live data and an answer block that commits the query. Most questions finish in one or two steps; the model is allowed up to five probes, keeping latency low while gaining the context needed to avoid silent errors.
On the BIRD Dev set SQRL‑35B‑A3B reaches 70.6 % execution accuracy, outperforming Claude Opus 4.6 at 68.8 % under the same harness. Three checkpoints—SQRL‑4B, SQRL‑9B, and SQRL‑35B‑A3B—are freely available on Hugging Face, letting teams plug an inspection‑capable model into their pipelines without rebuilding from scratch. By letting the model look at the data before it writes SQL, practitioners eliminate the class of wrong‑but‑valid queries that have plagued text‑to‑SQL systems for years.
#AI #Product #DataScience #TextToSQL #MachineLearning #Innovation