Alongside chatbots that can write texts and code, a second category is growing more quietly. They are called decision models, and they do the exact opposite of what we are used to from AI. They do not write. They receive a question, examine the data, and return one answer from a predefined list, plus a number indicating how confident they are. This field has its own ranking maintained by the analytics firm Artificial Analysis, with more than twenty models from Microsoft, Google, OpenAI, Perplexity, Cloudflare, and others competing in it. In mid-November, Liquid AI took the top spot with its d1.2 and d1.2-ultra models.
How it works in a simple example
This is best illustrated with a customer complaint. A message arrives saying that payments have not worked for three days, the customer has to pay rent tomorrow, and the situation is supposedly unacceptable. The company needs to determine three things from it: who to forward it to, whether it is urgent, and how upset the person is.
A language model responds with text that someone then has to analyze and convert into data. A decision model answers all three questions at once in a single call and assigns a probability to each option. In other words, it indicates the probability that the issue belongs with the accounting department, the probability that it belongs with technical support, how urgent it is, and that it believes the customer is angry.
That is precisely why they are cheap and fast. They select from a fixed set of options, so they do not have to generate anything, and only input tokens are billed, whereas with language models you pay for both input and output. Liquid AI’s d1.2 model costs two cents per million input tokens, while the ultra version costs just over four cents.
Confidence that can be verified
Everything depends on that confidence score. When a model says eighty percent, it should be correct in roughly eight cases out of ten. If that holds true, a simple rule can be built around it. Anything below ninety percent goes to a human for review, while the rest is processed automatically.
Technically, this is called calibration, and it is measured as the difference between stated confidence and actual accuracy. The smaller the number, the better. In its own test covering thirteen task sets and more than twenty-two thousand questions, Liquid AI reports an error of 0.8 points for the ultra version and 1.2 points for the base model. According to the company, the competing Jev 1.14 scored 2.7 points, Perplexity’s Decider 3.7, and OpenAI’s GPT-6 Luna 7.2.
This is where some caution is needed, because the company conducted the test itself and no outside party has yet repeated it using the same questions. The independent Artificial Analysis ranking also measures calibration using the so-called Brier score, where the d1.2-ultra model achieved 75 points and d1.2 scored 69. The ultra version leads the previously best decision model by two points.
This is exactly where the difference between a chatbot and a decision model lies. A chatbot will answer confidently even when it is making things up, and you have to recognize that yourself. A decision model will say that it is uncertain, and the system knows how to handle that.
What the ranking actually measures
The Artificial Analysis Decision Index v2.1 combines results from six different tests with different weightings. The largest share, thirty percent, is assigned to a set called AA-Paperclip. In it, models receive fifteen hundred résumés, receipts, invoices, and academic articles. For each one, they must make a decision and extract something from the text, with the entire document counted as a single decision.
Fifteen percent is assigned to the evaluation of workplace documents, another fifteen to knowledge and the tendency to make things up, fifteen to reasoning on questions from Humanity’s Last Exam with the option to decline to answer, and ten percent is taken up by an Nvidia test of when the model should use a tool and when it should not.
The final fifteen percent is quite specific to this category and is called AA-Symmetry. It checks whether a decision remains consistent when the question is rephrased, the options are swapped, or the question is turned on its head. A model that answers differently depending on the order of the answers will fail this test.
What the new models can do and what remains unknown
Liquid AI released the d1.2 and d1.2-ultra models on November 16 and added audio support alongside text and images. The company discontinued the older d1, d1.1, and d1-ultra models, which do not support audio.
Five new response types have been added. The model can extract a specific value from a document, such as an amount on an invoice, and if it cannot find it, it states that it is not there instead of making up a number. It can find all passages in a text that match the instructions, such as all personal data in a note. In an image, it can mark one or more objects with bounding boxes. It can estimate a number along with a range in which it is confident. And from a provided list, it selects all matching labels, each with its own probability.
In its own tests, the company reports ninety-eight percent accuracy for the ultra version when reading values from receipts and ninety-four percent when finding objects in images. For personal data in text, it reports ninety-four percent precision and ninety-three percent recall. The ultra version’s context window can handle sixty-six thousand tokens, and according to the company, the average time per task has fallen by twenty-eight percent.
Several things are still missing. No outside party has verified the calibration using the same questions, the company has not disclosed speed specifications or limits, and it is unclear whether the new response types work equally well in languages other than English. Moreover, the models are available only through Liquid AI’s interface, so no one can download and run them locally.
Source: artificialanalysis.ai



