Microsoft

RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference

The growing context lengths of large language models (LLMs) pose significant challenges for efficient inference, primarily due to GPU memory and bandwidth constraints. We present RetroInfer, a novel ...
IEEE

Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference

Abstract: The increasing adoption of large language models (LLMs) with extended context windows necessitates efficient Key-Value Cache (KVC) management to optimize inference performance. Inference ...

From DeepSeek-R1 to Ghibli art: 5 technical breakthroughs that changed AI in 2025

Here are the highlights of the year’s AI breakthroughs and discoveries that could set the stage for an even more ...

Report: Nvidia could acquire LLM startup AI21 Labs for $3B

The acquisition comes less than a week after Nvidia inked a $20 billion deal to license the technology of Groq Inc., a ...
Tech Xplore

New AI model accurately grades messy handwritten math answers and explains student errors

A research team affiliated with UNIST has unveiled a novel AI system capable of grading and providing detailed feedback on ...
Security Boulevard

Real-time threat detection for post-quantum AI inference environments.

Explore real-time threat detection in post-quantum AI inference environments. Learn how to protect against evolving threats and secure model context protocol (mcp) deployments with future-proof ...
Microsoft

TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edge

Ternary quantization has emerged as a powerful technique for reducing both computational and memory footprint of large language models (LLM), enabling efficient real-time inference deployment without ...

AMD: The Physics Of Inference Favors The Underdog

AMD (AMD) is rated a 'Buy' based on its architectural strengths and plausible 3-5 year EPS growth framework. AMD’s higher ...