Cerebras: an Engineering Marvel to Rival NVIDIA
or bigness for bigness’s sake?
It's a compelling story about how a successful exit can empower an innovative itch. Months of brainstorming led to the ambitious decision to tackle a seemingly insurmountable challenge, ultimately resulting in a groundbreaking product. Cerebras Systems not only dreams big but also acts big – their AI chip is so large it could be compared to a dinner plate or a pizza box, making it the largest single piece of silicon ever produced. And it works.
Cerebras Systems, an eight-year-old company, recently introduced the third version of their wafer-scale engine (WSE-3), a massive 5nm-based chip boasting 4 trillion transistors and 900,000 AI-optimized compute cores, which powers the CS-3 AI supercomputer. Last week, they also announced a collaboration with Dell Technologies to address the growing AI workload demands.
We will discuss what it all means, why they are less known than NVIDIA despite continuously claiming to outpace NVIDIA's chips, and how their valuation recently reached over $4 billion (with the next stop being an IPO?) in our AI Infra Unicorn series.
In today’s episode:
Starting point of Cerebras Systems: daunting challenge and metaphors from Andrew Feldman
Becoming a unicorn - financial situation
But what exactly does Cerebras offer?
Mission
Training capabilities and inference challenges
Cerebras vs. NVIDIA: another analogy and key differences
Can Cerebras’s chips Replace NVIDIA GPUs?
How does the company make money?
Conclusion
Starting point of Cerebras: daunting challenge and metaphors from Andrew Feldman
It was a tremendous success. In five years, Andrew Feldman and Gary Lauterbach built SeaMicro, a novel power-efficient computer server for data processing, and sold it to AMD for $334 million. Serving on the board for another two years, in 2014 they finally decided to get some rest and quit. But once an entrepreneur, always an entrepreneur, especially when tremendous talent still floats around you, ready to follow. Feldman and Lauterbach stayed in touch with three other colleagues from SeaMicro: Michael James, J.P. Fricker, and Sean Lie, and gradually started to brainstorm, each bringing unique expertise in software, hardware, and systems architecture. All five of them shared the ambition to create something big, not just another incremental improvement in the tech world.
The idea of building a new type of server, optimized for Intel's groundbreaking 3D XPoint memory, initially captivated them. This technology promised to transform computing with its unprecedented speed and durability. However, the team quickly realized the limitations imposed by Intel's dominance over the technology. They shifted their focus to an even bolder vision: creating a computer optimized for artificial intelligence.
Feldman envisioned a machine solely dedicated to AI tasks, eschewing all other functionalities. This concept involved constructing a wafer-scale chip, a colossal 60 times larger than any existing chip, with unparalleled compute power.
“All we put on our chip is stuff for A.I. For now, progress will come through specialization.”
Andrew Feldman at The New Yorker
So when they said they wanted something big, they meant it literally. But it was a daunting challenge, reminiscent of the failed efforts of Trilogy Systems decades earlier, who were also trying to build a wafer-scale systems. And, because it was a daunting challenge, it became so inspiring: solving such a complex problem would grant them a unique market advantage with few competitors.