Russians create AI model as interesting as DeepSeek R1

A recent advance in artificial intelligence (AI) model development has emerged from a collaboration between the Yandex Research Laboratory in Russia and esteemed institutions such as the Massachusetts Institute of Technology (MIT), the Institute of Science and Technology Austria (ISTA), and the Johannes Gutenberg University of Science and Technology in Germany.

The new method, named Hadamard Incoherence with Gaussian MSE-optimal GridS (HIGGS), facilitates the compression of neural networks without the need for extra data or intricate parameter optimization. This innovation is particularly beneficial in contexts where suitable data for further training of AI models is scarce. HIGGS strikes a balance between model quality, scalability, and quantization complexity, allowing AI models to operate effectively across various devices.

One of the significant advantages of HIGGS is its ability to expedite the testing and deployment of AI solutions, making the process more economical. Users can now utilize AI models simply on a smartphone or laptop, eliminating the necessity for expensive servers and graphics accelerators. Traditionally, quantizing an AI model on personal devices could take from hours to weeks, whereas HIGGS reduces this time to mere minutes.

The effectiveness of HIGGS has been demonstrated through its application on well-known AI models such as Llama 3 and Qwen 2.5, showing it to be the most efficient quantization method regarding quality-to-model-size ratio when compared to existing data-free alternatives.

Developers and researchers can access the HIGGS method on the Hugging Face platform and GitHub. A comprehensive paper outlining the methodology has been published on arXiv and has been accepted for presentation at the North American Chapter of the Association for Computational Linguistics (NAACL), the largest AI conference, which will be held in Albuquerque, New Mexico, from April 29 to May 4. The paper has garnered interest from various institutions, including Red Hat AI, Peking University, and the Hong Kong University of Science and Technology.