Fast Facts
- Subquadratic proposes a new approach that could dramatically boost the speed and reduce the cost of certain language tasks, though it won’t replace top models universally.
- Traditional LLMs rely on dense attention within transformers, which become computationally intensive as text length increases, due to quadratic growth in calculations.
- The company’s breakthrough uses sparse attention to limit the number of token relationships processed, significantly cutting down the needed computations.
- Ultimately, this innovation could revolutionize how large language models are built, making them more efficient and shifting away from the transformer architecture in the future.
Breaking a Bottleneck in Large Language Models
A new startup claims it has overcome a major obstacle facing large language models (LLMs). Their breakthrough could make certain tasks faster and cheaper. While they say it won’t replace top models everywhere, it might change the way LLMs work in the future. The CEO of the company believes we are on the verge of a new era of efficiency. They suggest that traditional models built on transformers could become less common in just a few years.
Understanding How Most LLMs Work
Most large language models rely on a process called dense attention. This process helps the model understand the meaning of text. Today’s LLMs combine many transformer units to analyze text. Dense attention works by multiplying each word’s encoding with every other word’s encoding. For example, a 10,000-word document requires nearly 50 million multiplications. This heavy computation explains why LLMs use so much power. As text gets longer, the amount of work grows rapidly, making these models expensive to run.
The Impact of Subquadratic Technology
The startup’s new approach replaces dense attention with sparse attention. Instead of multiplying every pair of words, it focuses only on important connections. This reduces the number of calculations needed. The idea is that not all word relationships are equally important. By skipping unnecessary multiplications, the process speeds up and costs less. This innovation could lead to faster, more affordable language models. However, it may take time before the technology is widely adopted, and some traditional models will still be needed for complex tasks.
Continue Your Tech Journey
Learn how the Internet of Things (IoT) is transforming everyday life.
Discover archived knowledge and digital history on the Internet Archive.
AITechV1
