What is inference? Explaining the massive new shift in AI computing

The focus of artificial-intelligence spending has gone from training models to using them. Here’s how to understand the ...

Phison Rescales Local AI Inferencing with Flash Memory Expansion

Phison Electronics (8299TT), a global leader in NAND flash controllers and storage solutions, today announced its GTC ...

New memory architecture targets AI inference bottlenecks

Lightbits Labs Ltd. today is introducing a new architecture aimed at addressing one of the most stubborn bottlenecks in large-scale artificial intelligence inference: the growing mismatch between the ...

Nvidia GTC 2026: Jensen Huang’s Groq ‘Mellanox moment’ and the inference land grab

Ahead of Nvidia Corp.’s GTC 2026 this week, we reiterate our thesis that the center of gravity in artificial intelligence is ...

Nvidia shrinks LLM memory 20x without changing model weights

Nvidia's KV Cache Transform Coding (KVTC) compresses LLM key-value cache by 20x without model changes, cutting GPU memory costs and time-to-first-token by up to 8x for multi-turn AI applications.
EDN

Analog in-memory compute tackles the AI inference conundrum

An analog in-memory compute chip claims to solve the power/performance conundrum facing artificial intelligence (AI) inference applications by facilitating energy efficiency and cost reductions ...

NVIDIA Enters Production With Dynamo, the Broadly Adopted Inference Operating System for AI Factories

NVIDIA Dynamo 1.0 provides a production-grade, open source foundation for inference at scale.Dynamo and NVIDIA TensorRT-LLM ...
Verdict on MSN

Nvidia launches Dynamo 1.0 AI inference operating system

Dynamo 1.0 manages AI inference workloads across data centres, offering integration with major cloud and open source platforms.
Business Wire

Credo Unveils Industry’s First Memory Fanout Gearbox for Scalable, High-Bandwidth AI Inference

SAN JOSE, Calif.--(BUSINESS WIRE)--Credo Technology Group Holding Ltd (Credo) (NASDAQ: CRDO), an innovator in providing secure, high-speed connectivity solutions that deliver improved reliability and ...
TMCnet

Penguin Solutions' OriginAI Factory Platform Delivers Optimized Performance for AI Inference

OriginAI inference solutions are designed leveraging Penguin Solutions 3.3+ billion hours of GPU runtime experience and more ...
Semiconductor Engineering

Pooling CPU Memory for LLM Inference With Lower Latency and Higher Throughput (UC Berkeley)

“The rapid growth of LLMs has revolutionized natural language processing and AI analysis, but their increasing size and memory demands present significant challenges. A common solution is to spill ...