Hot Chips 2026: d-Matrix Raptor stacks TSMC N4 logic on custom DRAM for 100 TB/s AI inference

A memory-wall announcement, not just another accelerator

At Hot Chips 2026, the attention around d-Matrix’s Raptor accelerator came from a simple inversion of the usual AI package. Instead of placing high-bandwidth memory beside a compute die and routing data across an interposer, Raptor places the compute logic directly on top of a custom DRAM die. The logic is manufactured on TSMC’s N4-class 4nm process, and the two dies are bonded face-to-face at a 36-micron pitch [1].

That physical choice is the product story. d-Matrix says Raptor delivers 100 TB/s of bandwidth from 32GB per card, with vertical I/O energy of about 0.37 picojoules per bit [1]. TechTimes, reporting on the same Hot Chips disclosure, described the device as working silicon and said the stacked implementation uses vertical microbumps between the logic and DRAM dies rather than the longer electrical path associated with HBM packages [2].

The important correction is that this is not a TSMC accelerator announcement in the sense of a TSMC-branded AI chip. It is d-Matrix’s Raptor accelerator, built with TSMC N4 logic and a custom DRAM die whose manufacturer has not been disclosed [1]. That distinction matters because the technical credibility comes partly from TSMC’s process node, while the architectural risk remains d-Matrix’s own.

Why 100 TB/s matters for inference

The timing of the announcement is not accidental. Generative AI inference is increasingly limited by the movement of weights and key-value cache data rather than by raw matrix multiplication. During decode, a model produces output one token at a time, repeatedly touching model weights and cache. Serve-class inference therefore turns into a bandwidth and energy problem.

Raptor targets that specific phase. Tom’s Hardware reports that d-Matrix framed the part as a 3D-DRAM accelerator for generative inference and showed a TSMC 4nm compute die stacked over a custom DRAM die to deliver the 100 TB/s figure from 32GB per card [1]. XenoSpectrum, taking a more cautious reading of the accompanying technical material, says roughly 105 TB/s of card-level bandwidth and about 2.5 ns average flit latency were measured on real silicon, while the larger LLM performance multiples are architectural projections rather than end-to-end product benchmarks [3].

That distinction is central. Bandwidth, latency, I/O energy and refresh behavior can be discussed as measured silicon data, according to the XenoSpectrum analysis [3]. Claims about tokens per second, throughput per card or time per output token depend on modeling assumptions that pair the memory system with a given compute engine and workload [3]. In other words, Raptor’s memory subsystem appears real; its full datacenter value still has to be proven in deployed systems.

The HBM comparison: faster, smaller, riskier

d-Matrix’s comparison point is HBM, the dominant memory technology for AI accelerators. HBM provides large capacity and high bandwidth, but it still requires physical interfaces, interposer routing and package edge space. Raptor tries to remove much of that distance by stacking logic and DRAM face-to-face.

The claimed result is a lopsided trade: 32GB of 3D DRAM per Raptor card at more than 100 TB/s versus a cited 192GB HBM4 configuration at roughly 18 TB/s [5]. Hardware Busters summarized the comparison as about 5.6 times the bandwidth and five to eight times better energy per bit, while also stressing that the gain is bought with one-sixth the memory capacity [5].

That capacity tradeoff is not a footnote. A 32GB card cannot simply replace a large HBM GPU in every workload. If the working set fits, the extra bandwidth may be decisive. If it does not, the system has to partition the model and cache across multiple cards, adding networking, scheduling and software complexity. Hardware Busters therefore called the design an architectural wager rather than a spec-sheet victory [5].

XenoSpectrum makes the same point from another angle: the 100 TB/s figure is aggregated across the whole card, not per stack or per chiplet [3]. Its analysis describes a baseline configuration of 32GB and 100 TB/s per card, compared with an HBM configuration offering more capacity but much lower aggregate bandwidth [3]. That makes Raptor a specialized answer to a specific inference bottleneck, not a universal HBM replacement.

The energy claim is the headline

The most commercially important number may be 0.37 pJ/bit. Tom’s Hardware reports that d-Matrix co-founder and CTO Sudeep Bhoja presented that as a measured number from working silicon, contrasting it with about 2.4 pJ/bit for moving data into an HBM4 base die [1]. TechTimes likewise reports more than 100 TB/s at 0.37 pJ/bit for the Raptor card and frames the advantage as the result of eliminating much of the long-distance PHY burden [2].

XenoSpectrum adds nuance: it reports an I/O energy figure of 0.376 pJ/bit under specific measured conditions and notes that comparisons against “one-tenth of HBM” depend on which HBM reference and which switching assumptions are used [3]. Its reading is that a roughly sixfold reduction versus a representative HBM3 comparison is better supported than the most aggressive marketing shorthand [3].

Even with that caution, the direction is significant. At 100 TB/s, every fraction of a picojoule per bit becomes a large power number. If the data path can be made shorter and lower energy, AI inference systems can spend more of their thermal and electrical budget on useful work rather than memory movement.

Thermal design: logic on top for a reason

Stacking active logic and DRAM creates a thermal problem: DRAM is temperature-sensitive, while AI logic can dissipate hundreds of watts. Raptor’s unusual “logic-on-top” layout is meant to address that. Tom’s Hardware reports that the cold plate can sit directly on the compute silicon, with the DRAM underneath serving as part of the vertical interconnect path [1]. TechTimes says the orientation lets liquid cooling contact the logic die directly and notes that d-Matrix is constraining stack power density to below 0.5 W/mm² to keep the DRAM within a manageable thermal range [2].

That packaging choice also shows why this is more than a memory spec. Raptor is an exercise in co-design: memory topology, power delivery, bank mapping, cooling, packaging yield and software all interact. If any one piece fails, the headline bandwidth may not translate into usable inference economics.

The broader Hot Chips context reinforces the point. Forbes reported that the 2026 conference focused heavily on memory and storage as AI’s hardware center of gravity, with talks on HBM evolution, Samsung’s zHBM and d-Matrix’s 3D DRAM approach [4]. In that setting, Raptor is part of a larger industry search for ways to put compute closer to memory without creating unsolvable heat and yield problems.

What remains unproven

Raptor is compelling because it attacks a real bottleneck with measured silicon, not only a slide-deck concept. But the open questions are substantial.

First, d-Matrix has not disclosed who manufactures the custom DRAM die [1]. Second, Tom’s Hardware reports that the company gave no firm volume, pricing or production schedule at Hot Chips, although it also noted that d-Matrix’s CEO had previously pointed to a 2027 launch window [1]. Third, the most exciting application-level numbers remain projections built from early silicon rather than public results from a broadly available card [1].

Fourth, capacity may be a commercial constraint. A 32GB card with extraordinary bandwidth may be ideal for decode-heavy inference if the model, cache or partition fits. For workloads that need large local memory pools, HBM’s capacity advantage remains relevant. The market question is therefore not “does Raptor beat HBM?” but “which inference systems can be redesigned around many small, very fast memory pools?”

The strategic read

The most important implication from Hot Chips 2026 is that AI accelerator competition is moving into physical data movement. GPUs, in-memory compute startups, HBM vendors and packaging specialists are all converging on the same problem: arithmetic is abundant, but feeding the arithmetic is expensive.

d-Matrix’s answer is radical but understandable. By collapsing the distance between compute and DRAM, Raptor tries to make DRAM behave more like an on-package near-memory fabric than an external memory subsystem. The 100 TB/s claim is impressive; the 0.37 pJ/bit claim is potentially more disruptive; the 32GB capacity limit is the price.

For now, Raptor should be treated as promising early silicon rather than a proven datacenter platform. But it is also a serious signpost. If d-Matrix can turn its Hot Chips demonstration into reliable, manufacturable and programmable systems, the AI inference market may gain a new class of accelerator optimized less for peak FLOPS and more for the shortest possible path between weights, cache and compute.

Sources from the last 72 hours

1. [1]Hot Chips 2026: d-Matrix stacks AI accelerator directly on custom DRAM for 100 TB/s per card — TSMC 4nm compute die bonded face-to-face at a 36-micron pitch on top of a custom-designed dieAug 26, 2026, 12:00 AM UTC
2. [2]d-Matrix Raptor Delivers 100 TB/s From Stacked DRAM at Fraction of HBM Energy CostAug 24, 2026, 2:47 PM UTC
3. [3]d-Matrix's Raptor Chip Hits 105 TB/s in Real Silicon Tests, But It's Not an HBM Replacement YetAug 24, 2026, 12:00 AM UTC
4. [4]Hardware Is Hot Again At The 2026 IEEE Hot Chips ConferenceAug 26, 2026, 2:24 AM UTC
5. [5]d-Matrix Raptor Drops the Memory PHY: 32GB at 100 TB/s Against HBM4’s 192GB at 18Aug 24, 2026, 12:00 AM UTC

AI-generated article based on recent web research, then preserved as a dated editorial snapshot.