Coral is Google’s quiet initiative to enable AI without the cloud

AI allows machines to carry out all sorts of tasks that used to be the domain of humans alone. Need to run quality control on a factory production line? Set up an AI-powered camera to spot defects. How about interpreting medical data? Machine learning can identify potential tumors from scans and flag them to a doctor.

But applications like this are useful only so long as they’re fast and secure. An AI camera that takes minutes to process images isn’t much use in a factory, and no patient wants to risk the exposure of their medical data if it’s sent to the cloud for analysis.

These are the sorts of problems Google is trying to solve through a little-known initiative called Coral.

“Traditionally, data from [AI] devices was sent to large compute instances, housed in centralized data centers where machine learning models could operate at speed,” Vikram Tank, product manager at Coral, explained to The Verge over email. “Coral is a platform of hardware and software components from Google that help you build devices with local AI — providing hardware acceleration for neural networks … right on the edge device.”

You might not have heard of Coral before (it only “graduated” out of beta last October), but it’s part of a fast-growing AI sector. Market analysts predict that more than 750 million edge AI chips and computers will be sold in 2020, rising to 1.5 billion by 2024. And while most of these will be installed in consumer devices like phones, a great deal are destined for enterprise customers in industries like automotive and health care.

To meet customers’ needs Coral offers two main types of products: accelerators and dev boards meant for prototyping new ideas, and modules that are destined to power the AI brains of production devices like smart cameras and sensors. In both cases, the heart of the hardware is Google’s Edge TPU, an ASIC chip optimized to run lightweight machine learning algorithms — a (very) little brother to the water-cooled TPU used in Google’s cloud servers.

While its hardware can be used by lone engineers to create fun projects (Coral offers guides on how to build an AI marshmallow-sorting machine and smart bird feeder, for example), the long-term focus, says Tank, is on enterprise customers in industries like the automotive world and health care.

As an example of the type of problem Coral is targeting, Tank gives the scenario of a self-driving car that’s using machine vision to identify objects on the street.

“A car moving at 65 mph would traverse almost 10 feet in 100 milliseconds,” he says, so any “delays in processing” — caused by a slow mobile connection, for example — “add risk to critical use cases.” It’s much safer to do that analysis on-device rather than waiting on a slow connection to find out whether that’s a stop sign or a street light up ahead.

Tank says similar benefits exist with regard to improved privacy. “Consider a medical device manufacturer that wants to do real time analysis of ultrasound images using image recognition,” he says. Sending those images to the cloud creates a potential weak link for hackers to target, but analyzing images on-device allows patients and doctors to “have confidence that data processed on the device doesn’t go out of their control.”

Although Coral is targeting the world of enterprise, the project actually has its roots in Google’s “AIY” range of do-it-yourself machine learning kits, says Tank. Launched in 2017 and powered by Raspberry Pi computers, AIY kits let anyone build their own smart speakers and smart cameras, and they were a big success in the STEM toys and maker markets.

Tank says the AIY team quickly noticed that while some customers just wanted to follow the instructions and build the toys, others wanted to cannibalize the hardware to prototype their own devices. Coral was created to cater to these customers.

The problem for Google is that there are dozens of companies with similar pitches to Coral. These run the gamut from startups like Seattle-based Xnor, which makes AI cameras efficient enough to run on solar power, to powerful incumbents like Intel, which unveiled one of the first USB accelerators for enterprise in 2017 and paid $2 billion last December for the chipmaker Habana Labs to improve its edge AI offerings (among other things).

Given the large number of competitors out there, the Coral team says it differentiates itself by tightly integrating its hardware with Google’s ecosystem of AI services.

This stack of products — which covers chips, cloud training, dev tools, and more — has long been a key strength of Google’s AI work. In Coral’s case, there’s a library of AI models specifically compiled for its hardware, as well as AI services on Google Cloud that integrate directly with individual Coral modules like its environment sensors.

In fact, Coral is so tightly integrated with Google’s AI ecosystem that its Edge TPU-powered hardware only works with Google’s machine learning framework, TensorFlow, a fact that rivals in the AI edge market The Verge spoke to said was potentially a limiting factor.

“Coral products process specifically for their platform [while] our products support all the major AI frameworks and models in the market,” a spokesperson for AI edge firm Kneron told The Verge. (Kneron said there was “no negativity” in its assessment and that Google’s entry into the market was welcome as it “validates and drives innovation in the space.”)

But exactly how much business Coral is doing right now is impossible to say. Google is certainly not pushing Coral with anywhere near as much intensity as its cloud AI services, and the company wouldn’t share any sales figures or targets for the group. A source familiar with the matter, though, did tell The Verge that the majority of Coral’s orders are for single units (e.g. AI accelerators and dev boards), while only a few customers are making enterprise purchases on the order of 10,000 units.

For Google, the attraction of Coral may not necessarily be revenue, but simply learning more about how its AI is being applied in the places that matter. In the world of practical machine learning right now, all roads lead, inexorably, to the edge.