Liquid AI LFM2-24B-A2B: Hybrid Architecture for Efficient Edge-Capable AI

Liquid AI’s New LFM2-24B-A2B Hybrid Architecture Blends Attention with Convolutions to Solve the Scaling Bottlenecks of Modern LLMs

Liquid AI has released LFM2-24B-A2B, a hybrid model that integrates gated short convolution blocks with Grouped Query Attention (GQA). Despite its 24-billion parameter scale, it activates only 2.3 billion parameters per token to optimize for consumer-grade hardware.

Why This Matters

Traditional Transformers suffer from quadratic scaling bottlenecks and massive VRAM consumption due to the KV cache required by Softmax Attention. By replacing 75% of attention layers with linear-complexity gated convolutions, Liquid AI addresses the technical reality of power and memory limits, allowing a model with the knowledge density of 24B parameters to run on a 2.3B active parameter budget. This shift from raw parameter counts to architectural efficiency enables high-throughput inference on hardware previously limited to much smaller, less capable models.

Key Insights

• The A2B architecture utilizes a 1:3 ratio, featuring 30 gated short convolution blocks and 10 Grouped Query Attention (GQA) blocks across 40 total layers (Liquid AI, 2026).
• A Sparse Mixture of Experts (MoE) design allows the 24B model to run on a 2.3B parameter budget per token, enabling deployment on 32GB RAM hardware like consumer laptops.
• Performance testing on NVIDIA H100 via vLLM demonstrates throughput of 26.8K total tokens per second at 1,024 concurrent requests.
• The model outperforms larger rivals such as Snowflake gpt-oss-20b and Qwen3-30B-A3B in logic and reasoning benchmarks like GSM8K and MATH-500.
• Training was conducted on 17 trillion tokens, supporting a 32,768 token context window optimized for local RAG pipelines and privacy-sensitive document analysis.

Practical Applications

• Local Document Analysis: Deploying 32k context RAG pipelines on consumer-grade NPUs or integrated GPUs to maintain data privacy without data-center infrastructure. Pitfall: Over-allocating VRAM for KV caches in all-attention models typically leads to OOM errors, which A2B avoids through convolution-based base layers.
• High-Throughput Edge Inference: Utilizing vLLM or SGLang for high-concurrency request handling in power-constrained environments. Pitfall: Using dense models with high active parameter counts results in high latency and energy drain, whereas this MoE design provides 2B-level speed with 24B-level intelligence.

References:

• https://www.marktechpost.com/2026/02/25/liquid-ais-new-lfm2-24b-a2b-hybrid-architecture-blends-attention-with-convolutions-to-solve-the-scaling-bottlenecks-of-modern-llms/