Modern AI systems like ChatGPT and Claude depend on the rapid exchange of massive amounts of data between memory, where information is stored, and processors, where it is analyzed. On conventional 2D chips, these components are spread across a single flat surface with only limited memory close to the computing units. As a result, data must move through a few long, crowded pathways. While processors can operate at extremely high speeds, data transfer struggles to keep pace, and the lack of nearby memory causes frequent slowdowns. Engineers describe this bottleneck as the “memory wall,” a fundamental barrier where processing power outstrips the chip’s ability to deliver data efficiently.

Figure 1. 3D Chip Breaks Through AI’s Memory Wall.

For years, chipmakers tried to overcome the memory wall by shrinking transistors—the microscopic switches that perform calculations and store data—and packing ever more of them onto a single chip. But this approach is now nearing fundamental physical limits, often called the “miniaturization wall.” Figure 1 shows 3D Chip Breaks Through AI’s Memory Wall.

The new chip tackles both barriers by quite literally moving upward. By integrating memory and computation vertically, far more data can move much faster. As Tathagata Srimani, assistant professor of electrical and computer engineering at Carnegie Mellon University and senior author of the study, explains, the approach works much like elevator banks in a skyscraper, allowing large volumes of information to travel efficiently between layers.

Together, the memory wall and the miniaturization wall create what co-author Robert M. Radway of the University of Pennsylvania calls a “deadly combination.” The researchers addressed this head-on by tightly fusing memory and logic at extremely high density and then stacking them vertically. The result, Radway says, is “the Manhattan of computing”—more functionality packed into less space.

How the new 3D chip is made

Most previous efforts to build 3D chips have relied on stacking separate chips on top of one another. While effective to a degree, this method creates relatively sparse, coarse connections between layers, which can become new bottlenecks.

Instead, the team used a “monolithic” approach, fabricating each layer directly on top of the previous one in a single, continuous process. By operating at low enough temperatures to protect the underlying circuitry, the method allows much denser stacking and far more tightly connected layers.

Crucially, the entire process was carried out in a domestic commercial silicon foundry. According to co-author Mark Nelson of SkyWater Technology, translating an advanced academic design into something a commercial fab can manufacture is exceptionally difficult. Demonstrating that these architectures can be produced domestically—and at scale—shows they are not just laboratory curiosities, but viable technologies for real-world deployment.

The chip’s performance and potential

Initial hardware tests indicate that the prototype already delivers roughly four times the performance of comparable 2D chips. Simulations of taller versions with additional stacked layers suggest even greater improvements. Designs with more tiers show performance gains of up to twelve times on real AI workloads, including those based on Meta’s open-source LLaMA model.

Most notably, the researchers believe the architecture offers a realistic path toward 100- to 1,000-fold improvements in energy-delay product (EDP), a critical measure balancing speed and energy efficiency. By drastically reducing data movement distances and adding many more vertical data pathways, the chip achieves higher throughput while consuming less energy per operation—an outcome long considered unattainable with flat chip designs.

Beyond performance gains, the team stresses the broader implications of the work. Demonstrating that monolithic 3D chips can be designed and built in the United States establishes a roadmap for a new era of domestic semiconductor innovation. Much like the integrated-circuit revolution of the 1980s, this shift will depend on training a new generation of engineers fluent in vertical chip design.

Through initiatives such as the Microelectronics Commons California-Pacific-Northwest AI Hardware Hub, students and researchers are already being prepared to push this technology forward. As H.-S. Philip Wong of Stanford University notes, breakthroughs like this are about more than raw speed. They represent a growing national capability to innovate quickly, respond to emerging challenges, and shape the future of AI hardware.

Source: SciTECHDaily

Cite this article:

Priyadharshini S (2025), A New 3D Chip Aims to Overcome the “Memory Wall” Slowing AI, AnaTechMaz, pp.917