Every new AI chip company goes through the same rite of passage: someone runs its hardware against a GPU cluster on a widely cited benchmark and publishes the result. If the number is favorable, the startup puts it in the deck. If it is unfavorable, the founder writes a blog post explaining why the benchmark doesn't capture real-world performance. In neither case does the exercise illuminate what is actually different about the architecture being evaluated.
Reconfigurable dataflow is the architecture category that suffers most from this comparison problem. The headline debate — dataflow chip vs. GPU — frames the question as a speed competition when the underlying architectural distinction is about something more fundamental: how compute is organized relative to data movement. Getting that distinction wrong leads to poor purchasing decisions, bad investment theses, and, eventually, companies building the wrong thing.
The GPU Is a SIMD Engine with a Very Good Memory Subsystem
The canonical GPU architecture is built around the Single Instruction, Multiple Data (SIMD) execution model. Thousands of simpler cores execute the same instruction across a wide vector of data simultaneously. This is extremely well-suited to the operations that dominate neural network training: matrix multiplications, element-wise activations, and reductions across batch dimensions. The modern GPU data center accelerator is essentially an extremely optimized dense linear algebra engine, supplemented by a high-bandwidth memory (HBM) subsystem that can saturate compute with data.
The reason GPUs dominate AI training is not that they are architecturally ideal for the task — it is that they are architecturally adequate for the task, they have existed for twenty years, and the software ecosystem built on top of them (compiler infrastructure, automatic differentiation frameworks, distributed training libraries) represents an enormous accumulated investment. The GPU is winning partly on merit and largely on switching cost.
The memory bandwidth story is where GPU limitations become visible. For a typical large language model training run at the 70-billion-parameter scale, the ratio of compute operations to memory bytes accessed — the arithmetic intensity — is high enough that GPU HBM bandwidth is not the primary bottleneck. But as models scale past a trillion parameters, the arithmetic intensity drops: the weight tensors become so large that they cannot reside in on-chip SRAM, and the ratio of useful compute to data movement shifts unfavorably. The GPU's memory hierarchy — SRAM on-chip, then HBM on-package, then cross-chip interconnect — was designed for a model size regime that is now several generations old.
What Dataflow Actually Means
A dataflow architecture inverts the relationship between computation and data. Rather than bringing data to a centralized execution unit, computation is distributed across a fabric in which data flows between stationary processing elements. Each node in the fabric executes a specific operation; the output of one node flows directly to the input of the next without touching a shared memory hierarchy. The entire computation is expressed as a directed graph of operations, and the hardware executes that graph by routing values through a network of functional units.
The practical implication is that a reconfigurable dataflow chip has no bottleneck at the memory bus in the traditional sense — because there is no shared memory bus. Weights and activations are staged across a distributed scratchpad that is physically adjacent to the computation that uses them. The "memory bandwidth" of the system is the aggregate bandwidth of the on-chip interconnect between processing elements, which scales with the number of tiles and can be designed to match the specific communication pattern of the workload.
The right question is not whether a dataflow chip runs a given benchmark faster than a GPU. It is whether the memory access pattern of the workload exposes the GPU's memory hierarchy as a bottleneck — and whether the dataflow chip's topology matches the workload's communication graph.
SambaNova Systems, a Coexin Fund I portfolio company, built the Reconfigurable Dataflow Unit (RDU) architecture precisely around this principle. The RDU expresses computation as a spatial dataflow graph compiled to a physical array of processing elements and on-chip memory banks. For workloads with highly regular, predictable memory access patterns — which describes most of the core operations in transformer inference — this eliminates the DRAM access latency that limits GPU performance on memory-bound operations.
Where the Architecture Diverges in Practice
The benchmark comparisons that circulate in the industry typically use dense matrix multiply throughput as the primary metric, sometimes augmented by end-to-end training throughput on standard model architectures. This comparison tends to favor GPUs for two reasons: dense matrix multiply is the workload for which GPU tensor cores were specifically optimized, and the benchmark models are architecturally similar to the models that GPU vendors used to tune their compilers and runtime libraries over the previous decade.
The comparison diverges significantly in three scenarios. The first is large-batch inference at scale: when running a large model for inference across thousands of concurrent requests, the memory access pattern becomes dominated by loading model weights from DRAM — a workload known as memory-bound inference — and the GPU's HBM bandwidth is the limiting factor, not FLOP throughput. Dataflow architectures that keep weights resident on-chip across many inference requests amortize that bandwidth cost differently.
The second scenario is models with sparse or irregular structure: mixture-of-experts architectures, retrieval-augmented models, and graph neural networks all have activation patterns that vary by input. GPU execution on these workloads involves significant control flow overhead and memory gather operations that the SIMD model handles poorly. Reconfigurable routing between processing elements handles irregular communication more naturally.
The third scenario is custom and novel layer operations: the rate at which the research community proposes new attention variants, normalization schemes, and activation functions has accelerated. A GPU-centric workflow requires writing and optimizing a new CUDA kernel for each new operator. A reconfigurable dataflow compiler, if it can lower arbitrary operator graphs efficiently, makes operator innovation cheaper. Whether any specific dataflow compiler actually achieves this in practice is an empirical question that benchmarks on standard models cannot answer.
The Software Stack Is the Real Moat — and the Real Risk
The architectural advantages of reconfigurable dataflow are real, but the reason GPUs still dominate AI training budgets is almost entirely software. PyTorch, JAX, and their associated compilation infrastructure represent a decade of engineering investment in automatic differentiation, kernel fusion, distributed sharding, and memory management. The implicit contract of these frameworks is "write your model once, run it on any hardware." The actual situation is that this contract is fulfilled almost exclusively for GPU hardware, and fulfilling it for alternative architectures requires a compilation chain that is extremely difficult to build correctly.
The moat for any dataflow architecture company, then, is compiler quality. A chip that delivers superior efficiency on the ideal workload but requires significant user-side effort to run non-standard models is a research instrument, not a product. The companies in this category that have made real commercial progress are those that have invested heavily in making the software experience invisible — where the user writes standard framework code and the compiler handles the translation to the physical dataflow graph without requiring hand-tuning.
This is where the investment thesis actually lives. The question is not whether reconfigurable dataflow is architecturally superior for specific workloads — it often is. The question is whether the team can build the compilation infrastructure to expose that advantage at commercial scale, and whether they can do it before the GPU ecosystem's next generation of on-chip interconnect closes the memory bandwidth gap that currently creates the opening.
This memo represents the views of the author. It does not constitute investment advice or a recommendation to buy or sell any security. References to portfolio companies are for illustrative purposes only.