Try to remember the last time you thought about how much space modern AI models take up. Probably never, right? But for the companies operating these systems, this is a daily headache. And Google has just come up with a solution that could significantly reduce memory demand.
What exactly is the problem?
AI models work with enormous sets of numbers called vectors. You can think of a vector as a detailed description—for example, what a photo looks like, what a particular word means, or what properties a dataset has. The more detailed the description, the more numbers it contains and the more memory it requires. And memory, especially fast memory, costs money and has its limits.
One of the biggest bottlenecks in today's AI models is the so-called KV cache. Put simply, it is a fast scratchpad where the model continuously stores information so that it does not have to keep searching through slower storage. But this memory gets clogged. Vectors are simply large and fill it up quickly.
Vector quantization is a traditional technique that addresses this problem by rounding and reducing the size of numbers. However, traditional methods have a catch: for each block of data, they also need to store so-called quantization constants at full precision. This adds an extra 1 to 2 bits to each number, suddenly cutting the original savings in half. It is as if you wanted to save space in your suitcase, but the list of things you packed took up half the suitcase by itself.
Three algorithms and one goal
Google researchers Amir Zandieh and Vahab Mirrokni have published a paper that tackles this problem with three algorithms at once. They will be presented at the prestigious ICLR 2026 and AISTATS 2026 conferences.
TurboQuant is the main algorithm that brings the whole approach together. It works in two steps.
First comes PolarQuant. It takes a data vector and converts it from conventional coordinates (X, Y, Z) into so-called polar coordinates. Instead of saying "go 3 blocks east and 4 north," it says "go 5 blocks at an angle of 37 degrees." This produces two things: a radius (how strong the signal is) and an angle (what the signal means). And because the distribution of angles is predictable, the model does not need to store any auxiliary constants. The memory overhead disappears.
Then QJL (Quantized Johnson-Lindenstrauss) steps in. It reduces each number to a single bit—specifically, just its sign (+1 or -1). This sounds crazy, but the mathematics behind it, the so-called Johnson-Lindenstrauss transform, guarantees that the important relationships between the data are preserved. QJL is applied only to the small residual error left behind by PolarQuant. It serves as a mathematical patch that eliminates distortion and makes the result more accurate.
What can it do in practice?
The test results are quite compelling. The researchers tested all three algorithms on the open language models Gemma and Mistral across a wide range of demanding tests focused on working with long texts. TurboQuant can compress the KV cache to just 3 bits without any model fine-tuning and without any noticeable loss of accuracy. This is crucial because with other compression methods, the trade-off is usually lower response quality.
Speed? 4-bit TurboQuant is up to 8 times faster than the original uncompressed version on NVIDIA H100 graphics cards. Eight times. On the same hardware.
And what about searching through millions of vectors at once—the so-called vector search that powers major search engines? TurboQuant outperforms competing PQ and RabbiQ methods, even though those competing methods use large auxiliary tables and are specially tuned for specific data. TurboQuant does not need any of that; it works in general.
More than just a technical trick
Google apparently has more in mind than just academic results. TurboQuant is aimed directly at practical use, specifically at operating its own Gemini model. Less memory means lower operating costs and the ability to serve more users at once.
The entire approach is built on solid mathematical foundations, not just an empirical "we tried it and it worked." The authors prove that the algorithms operate close to the theoretical optimum. This is important for reliability in production environments, where failures cost real money.
The research was conducted in collaboration with experts from Google DeepMind, New York University (NYU), and the Korea Advanced Institute of Science and Technology (KAIST). It is therefore not the work of a single team, but the result of broader collaboration across the research community. At this year's ICLR and AISTATS conferences, we will see how the rest of the community responds to these results. I am betting the response will not be silence.
