Investing.com — Artificial intelligence is creating a growing memory bottleneck, with Bernstein analysts saying demand is spreading beyond high-bandwidth memory into conventional DRAM, NAND flash and storage.
The analysts said the investment opportunity across memory will increasingly depend on the AI workload, as training, inference, retrieval-augmented generation and agentic AI place different demands on the memory stack.
Training remains highly compute-intensive, with memory bandwidth limiting model size and computing speed. HBM plays a central role, but large-scale training also requires system DRAM, local SSDs and networked storage for datasets, caching and checkpoints.
Inference presents a different challenge. The initial “prefill” stage, when an AI model processes a prompt and generates its first token, is largely compute-bound, making GPUs and HBM more significant.
The subsequent “decode” stage is memory-bound. Models store previous tokens in a key-value, or KV, cache, with requirements rising alongside context length and concurrent users. Bernstein said KV caches could require more memory than model weights in large deployments, potentially limiting how many users an AI service can support.
That pressure is driving new memory tiers, including CXL memory, ’s “Storage Next” initiative and CMX context storage, alongside HBM, conventional DRAM and SSDs. The emerging technologies seek to balance performance, capacity and cost as AI systems require larger pools of memory.
Retrieval-augmented generation could broaden demand further. Bernstein said building RAG databases requires substantial SSD or HDD capacity alongside system DRAM, with DRAM playing a larger role when those databases are searched.
Agentic AI could intensify requirements further as autonomous systems retain intermediate results, interact with external tools and pass outputs between agents. This can rapidly increase KV-cache demand while creating greater requirements for CPUs and conventional memory in traditional servers.
Memory manufacturers are also developing technologies such as high-bandwidth flash, which seeks to combine HBM-like bandwidth with NAND’s greater capacity and lower cost. Bernstein said the technical hurdles remain high.
Bernstein rates , , , , and Outperform, while is rated Underperform.