Google ‘perfectly comfortable’ with Tensor not winning benchmarks and vertical integration benefiting Pixel

Tensor G2 is this week’s topic on the Made by Google Podcast, and the fourth episode provides an overview of the Pixel’s approach to chips, as well as benchmarks. Monika Gupta is senior director of product management for Google Silicon Teams and gets “to focus on what [Google] need[s] five years from now” for its chips. The interview emphasized how this in-house approach to chips involves the silicon team talking with Google’s AI researchers to “know exactly where machine learning models are trending in five years.”

I’m not making decisions based on where machine learning is today, and I can say that because I work at Google. Same with the software that our software team is doing. I know where the software team wants to take the user experiences five years from now. That’s the benefit of not being a merchant silicon supplier, but an in-house silicon supplier. So those trade-off decisions are very tough, but I think they get a little easier when you’re vertically integrated.

Meanwhile, there was a section on the Tensor team’s outlook on benchmarks and what the Pixel is prioritizing instead:

I think classical benchmarks served a purpose at some moment in time, but I think the industry has evolved since then. And if you look at what Google is trying to do by pushing AI innovations into a smartphone, because we feel like this is the approach that will deliver helpful experiences like some of the ones i just mentioned, classical benchmarks were authored in a time where AI and phones didn’t even exist. They may tell some story, but we don’t feel like they tell the complete story. And so for us what we benchmark are the actual software workloads that we are running on our chip and then we strive with every generation of tensor chip to make them better, whether it’s better quality, better performance, lower power.

Google says it is “perfectly comfortable” with not winning or looking as great on benchmarks as it prioritizes the end-user experiences that result from the chip decisions it made.

Like on Pixel 6 and Pixel 7, you can see all the amazing innovations that we have landed, and a lot of them were like the first on Pixel. So we’re very comfortable with that approach

That approach does reflect what Google is valuing (i.e. AI-powered features) — though AI benchmarks do exist — but it would be interesting to see whether their philosophy changes as the company’s silicon matures. For example, there’s an off-the-shelf nature to the first two generations that could eventually turn into custom cores, like Apple, with more experience in the field. When asked about the silicon roadmap, and Gupta repeated a sentiment from last year about how the end goal is to have Tensor support ambient computing:

…overall vision for us and for Tensor family is really all about ambient computing. Ambient computing means that the technology is making your life easier. I think we have a lot of evidence of this that we talked about today, whether it’s making photography easier, whether it’s making phone calls and how you use your phone, like your day-to-day tasks, easier. I would say we build upon that vision of ambient computing and figure out how to do super complex, nuanced things in the chip in a power-efficient way that are going to unlock some of those ambient computing experiences.