OpenAI Details Jalapeño Inference Chip Test Results
The company plans to deploy the custom ASIC in its data centers by year end.
OpenAI released early test results for Jalapeño, its first custom inference chip. The company stated it will begin deploying the chip in its own compute infrastructure by year end as the first step in a multigenerational roadmap, with Gen 2 already in development. The presentation occurred at the Hot Chips conference and used open source models as the benchmark focus. Multiple independent analysts posted observations on the efficiency comparisons shown in the shared materials.
No MTP, No PD disaggregation, Pure TP, still beats NVIDIA's Vera Rubin NVL72 on a third-party model, with A0 stepping. And B0 is 25% better. NVIDIA GPUs become HBM wrappers.
Reactions
22 postsZE 
Zephyr@zephyr_z9HOLY SHEEET I don't think anyone can beat this chip at perf/Watt or perf/$ Congrats @itsclivetime @thehiphopswami
- likes: 1.2K
- replies: 36
- bookmarks: 361
- reposts: 84
ZE CH Chubby♨️@kimmonismusNgl, this is actually bigger than „another“ model release. This chup looks really damn good for inference
- likes: 61
- replies: 2
- bookmarks: 6
- reposts: 1
GB Gavin Baker@GavinSBakerImpressive that Jalapeño outperforms the comparable TPU and is in the mix with Rubin. Credit where credit is due - first good ASIC outside of TPU/Trainium. However, will likely significantly underperform a disaggregated GPU/Trainum plus SRAM accelerator setup. Especially with…
- likes: 1.6K
- replies: 92
- bookmarks: 388
- reposts: 60
LF Liam Fedus@LiamFedusData centers are power-limited and OpenAI's Jalapeño sets new frontiers for tokens per second per watt. Impressive and fast work from OpenAI's ASIC program.
- likes: 110
- replies: 6
- bookmarks: 13
- reposts: 3
YJ
