LFM2.5-2.6B: Deploy Powerful AI Agents on Edge Devices

Unlocking On-Device Agentic AI

You can now deploy sophisticated AI agents directly on edge devices, thanks to the LFM2.5-2.6B model. This compact model delivers top-tier performance without hefty resource demands, topping every instruction-following benchmark and nearly every tool-use benchmark.

LFM2.5-2.6B's capabilities stem from an extensive pre-training regimen on approximately 34 trillion tokens. A crucial mid-training phase extended its context window to an impressive 128K tokens. Following this, the base model transforms into an agent through a meticulous four-stage post-training process.

Key Features and Specs

The core of its agentic behavior lies in a specialized Agentic RL pipeline. This system intelligently separates model optimization, inference generation, and environment execution. You can utilize a Sandbox Service to execute actions, where a Blackbox Harness hosts your agent and manages interactions with its task environment.

• Instruction Following & Tool Use: LFM2.5-2.6B excels in these areas, leading every instruction-following benchmark and nearly all tool-use benchmarks.
• Agentic Tasks: It outperforms both Gemma models and maintains parity with the Qwen series.
• Knowledge & Math: Your model will benefit from its leadership in knowledge tasks and strong performance in math.
• Coding: For highly complex coding tasks, larger models currently maintain a clearer lead.

Unmatched Inference Speed

LFM2.5-2.6B offers day-one support across major inference ecosystems, including llama.cpp, MLX, vLLM, SGLang, and ONNX, ensuring broad compatibility for your deployments.

You can achieve decode speeds of 220 tokens/s on an M5 Max and 113 tokens/s on a Ryzen AI Max+ 395. On a single H100 GPU, you can reach almost 15,000 output tokens per second at high concurrency.

Getting Started

To integrate LFM2.5-2.6B into your workflow, ensure you have the latest version of the Transformers library installed. You can load and run the model with simple Python commands.

pip install "transformers>=5.0.0"

Then, use the following code to load and run the model:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("your_org/LFM2.5-2.6B")
model = AutoModelForCausalLM.from_pretrained("your_org/LFM2.5-2.6B", torch_dtype=torch.bfloat16)
prompt = "Your detailed instruction or task for the agent goes here."
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100) 
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

What This Means For You

LFM2.5-2.6B is ideal for scenarios where you need robust instruction following and reliable tool use on constrained hardware. You can explore the browser demo of LFM2.5-2.6B powering a research agent, which assists with specific questions and generates summaries.

The Bottom Line for Developers

With LFM2.5-2.6B, you can deploy local agents everywhere, bringing advanced AI capabilities to your high-volume workloads. Its unmatched inference speed and compact size make it an ideal choice for edge AI applications.