In short: We asked Claude models 21 finance questions on NetSuite data, once over a raw SQL connection and once over Data to AI. The cheapest model on Data to AI matched the most capable model on raw SQL at 24x lower cost per question. In the webinar, Radim Marek showed the three ways the models went wrong without it: counting rows twice, mixing currencies, and answering a question the data could not answer.
On 19 November, Petr Nemeth (Dataddo's founder) and Radim Marek, who leads Data to AI, walked through our latest benchmark and the mistakes behind its numbers. Here is what they covered.
Watch the full recording, or download the benchmark whitepaper for the full methodology.
Petr explained the choice simply: ask for EBITDA and there is exactly one right answer. That makes finance the cleanest place to tell a correct answer from a wrong one.
The setup:
Petr's takeaway: if you don't plan to run a frontier model on every question, money spent on the data layer can get you comparable answers for a fraction of the price.
A confidently wrong answer is one you can't tell is wrong without significant effort: precise, well explained, and off. A crash is easy to spot. A plausible number gets built on. (We explain the idea in more depth in What is a confidently wrong answer? blog post)
Radim walked through three questions from the benchmark where models connected straight to the database got it wrong, and what Data to AI did differently.
Clip: What is a confidently wrong answer?
1. Duplicate rows. How many units did we invoice in 2025? The data held 13 invoice lines twice. On raw SQL, 12 runs in a row answered 16,836 instead of 16,810, and the most capable model overcorrected to 15,092 by dropping freight and assembly lines. Data to AI blocks a plain sum on invoice lines, removes duplicates by line ID, leaves out voided and credited units, and returns 16,810.
2. Currency IDs. What share of invoiced value falls in each currency? NetSuite stores currencies as numeric IDs. One model gave the right maths with unreadable labels ("Currency 4"); another applied the exchange rate backwards and put 88.47% of value in Czech koruna, whose real share is 1.34%. The same model on Data to AI got it right, because each ID maps to a verified ISO code and conversion runs in the base currency by fixed rules.
3. A question the data can't answer. What was our best-selling product in 2025, by product name? There are no product names in the data. On raw SQL, models picked a subproject column instead and named three different "best sellers" across six runs. Every Data to AI run declined, said why, and gave the top item by ID instead.
The key difference from text-to-SQL: the business rules are not loaded into the AI's prompt in the hope it follows them. They sit in a separate, predictable layer, and that layer builds the query. The AI never writes free-form SQL; it chooses from defined entities.
Three frameworks cover the failure modes:
And every number comes with a receipt. A SQL query hands the AI what is effectively a spreadsheet. Data to AI hands it the table plus what is known about it: where it came from, how current it is, what it includes. That lets the model make better decisions without guessing.
Radim's summary: accuracy needs semantic guardrails, and the guardrails have to be enforced, not suggested.
You don't need a data team to see whether your AI guesses. Section 6 of the benchmark whitepaper is a 20-minute test: six questions to ask the assistant your team already uses on business numbers.
Then try the same questions through Data to AI. Start free, learn more on the Data to AI page, or watch the full recording.