In today’s world of artificial intelligence, there is a lot of talk about giant models that break benchmark records and promise a revolution. But in corporate environments, where the real work gets done, things look different. Small language models (SLMs) are the ones quietly and efficiently handling everyday tasks, reducing costs, and increasing productivity. Let’s examine how this works in practice, based on companies’ experiences and research.
The Paradox of Large and Small Models
An article in the Wall Street Journal describes a paradox: While large language models (LLMs), such as those from OpenAI or Anthropic, make headlines for passing bar exams or winning mathematical olympiads, the situation is different in real-world business. These giant models are expensive and slow, so companies use them only for complex planning. Instead, they rely on small models that are faster, cheaper, and specialized. For example, Kyle Lo of the nonprofit Allen Institute for AI says that large models are not necessary at all for most of the operations we need today. Companies such as Meta use large models only to transfer knowledge to smaller ones, which then run in production because the large ones are too expensive.
This approach is reflected in how companies build their AI systems as assembly lines. Data goes in and passes through a series of small models, each performing its part of the task, with a useful output emerging at the end. This makes it possible to process large amounts of data quickly and cheaply, which is crucial for competitiveness.
How Small Models Work
Let’s look at some specific companies. Aurelian, a Seattle startup, uses generative AI to automate responses to non-emergency calls to 911 centers. New York-based Hark Audio uses AI to identify and edit interesting moments from half a million active podcasts. San Francisco-based Gong analyzes thousands of recorded sales calls to help companies such as Google and Cisco sell more. And Airbnb automatically handles a significant portion of customer requests using open models from China’s Alibaba, which is faster than relying on human operators.
At Gong, it works like this: When a customer asks a question such as "Why am I losing deals?", the system first sends the request to an advanced model from Anthropic or OpenAI, which creates a general plan. The data then passes through a series of small models—one searches thousands of calls, another summarizes them, and another analyzes the summaries. Finally, everything returns to the large model, which creates the final report. Eilon Reshef, Gong’s co-founder, explains that the least expensive model is used to determine the relevance of a conversation, a moderately priced one extracts information, and the most expensive one is used only for final processing. This saves both time and money.
Don MacKinnon of Hark Audio adds that while giant models are intelligent, they do not make it possible to use a company’s proprietary data efficiently. His team created a library of tens of thousands of human-selected clips and used them to fine-tune its own small model, which now automates the entire process.
Economic Benefits of Small Models
Cost plays a major role. According to the industry average, OpenAI’s smallest model, GPT-5 Nano, costs about 10 cents per million tokens (approximately CZK 2.30), while the full-fledged GPT-5 costs $3.44 per million tokens (about CZK 79). Large models also consume thousands of times more tokens because of their internal reasoning "monologue." Small models are not only cheaper but also faster, making them ideal for real-time applications.
Research by Nvidia and the Georgia Institute of Technology, mentioned in the article, confirms that small models are powerful enough for repetitive tasks in agentic systems, where models specialize in narrow areas. They are more economical, more flexible, and better suited for deployment on standard devices.
Applications Across Industries
Small models are also gaining ground in other areas. In customer service, they process routine inquiries faster, as can be seen in finance, manufacturing, insurance, and retail. They are used to analyze sentiment in reviews, social media posts, and surveys, which helps with marketing. In data processing, they parse documents and extract metadata for regulatory compliance.
In edge applications (edge AI), they run on mobile devices or IoT systems where privacy protection is important—for example, in healthcare for diagnostics or in finance for fraud detection. In industry, they help optimize clinical trials in pharmaceuticals, perform quality control in manufacturing, and process insurance claims. Companies such as Nvidia and Meta deploy them in products where they achieve performance comparable to large models but with lower costs and greater flexibility.
Limitations of Small Models
Nevertheless, small models have their limitations. They work best on narrowly defined tasks and may not match large models in general understanding. They can fall behind in large-scale applications with highly variable data. Even so, their speed, low cost, and ease of customization make them ideal for many business scenarios where general-purpose intelligence is not required.
This trend suggests that the future of AI lies in a combination—large models for complex reasoning and small ones for everyday work. Companies that understand this will gain an advantage in efficiency and innovation.
