Meta’s AI Runs on Google Chips at 700 Tokens Per Second

Google’s custom AI chips, known as tensor processing units, are now actively running production workloads for Meta. This marks a significant shift in how the social media giant handles its artificial intelligence tasks, as it moves away from relying solely on Nvidia hardware. The change was enabled by a new version of the SGLang inference engine, which is now native to the PyTorch framework that Meta uses extensively. Alongside this, an open-source version of vLLM, another popular inference engine, was also released to support the same hardware transition.

Previously, running these engines on Google’s chips required using a different framework called JAX. The new versions allow PyTorch to run natively on the hardware, meaning engineers do not have to rewrite their existing code. Zongwei Zhou, the lead for PyTorch-TPU at Meta, confirmed that the system is already live. He described the setup as a "remarkably mature, PyTorch-native serving stack" that resulted from a tight collaboration between Meta, Google, and the SGLang team, which is based at the startup RadixArk.

The performance metrics behind this new setup are substantial. According to Inferact, a startup involved in building the software, a tuned version of vLLM can process more than 700 tokens per second for a single user. This test used Google’s seventh-generation Ironwood TPU to run Moonshot AI’s Kimi K3 model, a massive system with 2.8 trillion parameters. For comparison, Nvidia’s GB200 chips achieved 452 tokens per second in a similar published test. The new setup also showed throughput improvements of 1.4 to 2 times compared to the Nvidia baseline when handling smaller batches without speculative decoding.

Jana van Greunen, who directs PyTorch engineering at Meta, emphasized the strategic importance of this move. She stated that Meta "cannot be constrained by a single hardware architecture" and that this new tool enables the company to deploy its workloads directly onto Google’s TPUs. Google confirmed that it co-developed this technology with Meta’s PyTorch team, highlighting the depth of the partnership. This is not the first time these two companies have worked together on this front, as Google has previously described an older bridge called PyTorch/XLA as a predecessor to the current system.

Meta has a history of experimenting with Google’s hardware. Reports from last year indicated that Meta’s research teams used TPUs through the older bridge between 2020 and 2023 before canceling those initial contracts. However, recent discussions suggested a renewed interest, with potential rentals of Google Cloud TPUs in 2026 and purchases planned for 2027. It is currently unclear where Meta’s production chips are physically located, but the company reportedly signed a multi-year rental agreement to secure access to the infrastructure.

This announcement landed just one day after Amazon revealed it was open-sourcing its own native PyTorch backend, TorchNeuron, for its Trainium chips. Both moves appear to be part of a broader industry effort to reduce dependence on Nvidia’s software ecosystem. By making their hardware more accessible to the dominant PyTorch framework, Google and Amazon are challenging Nvidia’s grip on the AI infrastructure market. This could give companies like Meta more leverage when negotiating deals with chipmakers in the future.

Looking ahead, the SGLang team plans to expand its model support, starting with DeepSeek V4.1 Flash and Reflection Beam. RadixArk aims to provide day-zero support for new open models by the first quarter of 2027, ensuring that developers can use the chips as soon as new models are released. Google, Meta, and RadixArk are scheduled to share more details about this collaboration at the SGLang Summit 2026, which will be held in San Francisco on November 12 and 13.

For readers in the Southeast and beyond, this development signals a maturing AI hardware market. It suggests that while Nvidia remains a dominant force, alternatives are becoming viable for large-scale production workloads. The ability to switch between different chip architectures without rewriting code gives tech companies greater flexibility and potentially better pricing power as they scale their AI operations.