1 month ago
TokenSpeed-Kernel: Master Multi-Silicon LLM Inference
Unlock high-performance LLM inference with the TokenSpeed-kernel. Learn how this open-source subsyst...
Tech News
29 articles in this category
1 month ago
PyTorch Monarch Arrives on AMD ROCm for LLM Training
Discover how PyTorch Monarch now enables single-controller distributed training for LLMs on AMD Inst...
3 months ago
Run vLLM Server on HF Jobs in One Command
Learn how to quickly deploy a vLLM server on Hugging Face Jobs with a single command. Optimize your...
3 months ago
Your LLM Checkpoints: How olmo-eval Standardizes Iterative Evaluation
If you're building LLMs, olmo-eval offers a Python-based workbench to manage your iterative evaluati...
4 months ago
Your aarch64 GPU Dependency Headaches on NVIDIA Arm Platforms Just Ended
PyTorch 2.11.0 now provides aarch64 GPU wheels on PyPI, directly solving a two-year dependency heada...
4 months ago
Running PyTorch Models on Apple Silicon GPUs with the ExecuTorch MLX Delegate
Unlocking Faster Generative AI WorkloadsIf you are deploying PyTorch models on Apple Silicon, your g...
4 months ago
Your OCR Workflows Just Got a Transformers Upgrade with PaddleOCR 3.5
PaddleOCR 3.5 brings Transformers-centered workflows to your OCR and document parsing tasks. Underst...
4 months ago
Your vLLM Migration: Fix Core Correctness Before Adding RL Corrections
When migrating vLLM from V0 to V1, prioritize backend correctness. Learn why issues in processed rol...
4 months ago
Your LLM Inference: How IBM Research Built RITS on vLLM
IBM Research launched the RITS Platform in Nov 2024, using vLLM for LLM inference. Understand the ar...
5 months ago
Your LLM Serving Bottleneck: Why Disaggregating CPU from GPU is Critical
If you're operating LLM inference, you're likely bottlenecked. Discover how Shepherd Model Gateway's...
5 months ago
Your Infrastructure's Next LLM? Dissecting IBM's Granite 4.1 Build
Examine the technical architecture, training regimen, and implications of IBM's Granite 4.1 LLMs for...
5 months ago
NVIDIA's Nemotron 3 Nano Omni: What Its Multimodal Architecture Means For Your AI Deployment
NVIDIA's Nemotron 3 Nano Omni offers a unified architecture for multimodal AI. Understand its Mamba,...