AI startup SiMa.ai debuts ‘purpose-built’ AI chip for edge computing

Within the very broad landscape of artificial intelligence computer chips, products to serve the “edge” market, from drones to internet of things devices to phones to low-power server environments, are a fertile area for vendors because it is one of the less-developed parts of the market compared to data center technology. 

As ZDNet reported earlier this year, dozens of startups have been getting tens of millions in venture funding to make chips for AI in mobile and other embedded computing uses. Because the edge market is less settled, there are plenty of different ways that vendors will approach the problem.

On Tuesday, AI chip startup SiMa dot ai formally unveiled what it calls its MLSoC, a system-on-chip for speeding up neural networks with lower power consumption. The company argues the new chip, which has begun shipping to customers, is the only part that is “purpose-built” to handle tasks with a heavy emphasis on computer vision tasks, such as detecting the presence of an object in a scene. 

“Everybody is building a machine learning accelerator, and just that alone,” said Krishna Rangasayee, co-founder and CEO of SiMa.ai, in an interview with ZDNet. 

“What is very different about the embedded edge market,” said Rangasayee, versus cloud computing, is that “people are looking for end-to-end application problem solvers,” rather than just a chip for machine learning functions.

“They're looking for a system-on-a-chip experience where you can run the entire application on a chip.” 

Competitors, said Rangasayee, “handle a narrow slice of the problem” by only performing the neural net function of machine learning. 

“Everybody needs ML, but it's one portion of the overall problem, not the entire problem,” said Rangasayee.

Also: The AI edge chip market is on fire, kindled by ‘staggering' VC funding

Built with Taiwan Semiconductor's 16-nanometer fabrication process, the SiMa.ai chip has multiple parts fashioned as a single chip. They include a machine learning accelerator, code-named “Mosaic,” which is dedicated to matrix multiplications that are the foundation of neural net processing.

Also onboard is an ARM A65 processor core, often found in automobiles, and a variety of functional units to aid in the specific task of vision applications, including a standalone computer vision processor, a video encoder and a decoder, 4 megabytes of on-chip memory, and a multitude of communications and memory access chips, including an interface to 32-bit LPDDR4 memory circuits. 

The chip hardware comes with SiMa.ai software to make it much easier to tune for performance, and to handle many more workloads. 

More details on the MLSoC are available on SiMa.ai's Web site. 

SiMa.ai's product is aimed at a variety of markets, including robots, drones, autonomous vehicles, and industrial automation, and applications in the healthcare and government markets. 

The government market has been a particularly swift adopter of the technology, said Rangasayee

“I'm surprised how fast the government sector is moving,” he said. The typical impression, noted Rangasayee, is that it takes governments five to seven years to procure new technology, but things are happening much faster than that. Applications the government is particularly interested in are things such as use of ML onboard tanks and for detectors that search for improvised explosive devices. Satellites are a promising application as well, he said. 

“It's a multi trillion-dollar-market still using decades-old technology,” observed Rangasayee of the various civilian and government applications. 

Many of today's computer vision systems for autonomous craft and other applications are using “traditional load-store architectures, Von Neumann architectures,” said Rangasayee, referring to the basic design of most computer chips on the market. 

The means, he said, that chips being used for machine learning and for computer have not advanced in terms of how they hand compute, bandwidth and data combined with one another. 

Also: To proliferate AI tasks, a starter kit from Xilinx, little programming required

“We have a unique ML SoC, the first system-on-a chip that comprehends ML, and so people can do classic computer vision, and solve legacy problems, in addition to ML, in one single architecture,” said Rangasayee.

SiMa.ai has received $150 million in venture capital over multiple rounds from mutual fund giant Fidelity and Dell Technologies, among others. Longtime chip industry insider Lip-Bu Tan, formerly head of chip design software firm Cadence Design, is on SiMa.ai's board of directors.

The word “sima” is a transliteration of the Sanskrit word for “edge.” 

In addition to Rangasayee, Moshe Gavrielov, formerly the CEO of Xilinx, is a co-founder. 

The term “Edge AI” has become a blanket term to refer to everything that is not in a data center, though it may include servers on the fringes of data centers. It ranges from smartphones to embedded devices that suck micro-watts of power using the TinyML framework for mobile AI from Google. 

SiMa.ai goes up against a raft of mobile and embedded competitors. In the edge market, competitors include AMD, now the parent of Xilinx, intellectual property giant ARM, Qualcomm, Intel, and Nvidia. However, those companies have been traditionally focused on larger chips running at far greater power, on the order of tens of watts.

The SiMa.ai chip boasts what its creators say is one of the lowest power budgets of any chip on the market to complete typical tasks such as ResNet-50, the most common neural net for processing the ImageNet tasks of labeling pictures.

The company says the part can perform 50 trillion operations per second, or “teraoperations,” at a power draw of 10 teraoperations per second per watt. That means the part will consume 5 watts when doing neural network tasks, though it may go as higher with other functions engaged.

That class of chip running at a few watts puts SiMa.ai in the company of a host of startups, including Hailo Technologies, Mythic, AlphaICs, Recogni, EdgeCortix, Flex Logix, Roviero, BrainChip, Syntiant, Untether AI, Expedera, Deep AI, Andes, and Plumerai, to name just the most obvious ones.

The only companies in that are “in our line of sight,” said Rangasayee, are Hailo and Mythic, but, “our large differentiation is they are building ML accelerators only, we are building full ML SoCs.”

By building in ARM cores and dedicated image circuitry along with the Mosaic neural network code, customers will have a greater ability to run existing programs while adding code from popular ML frameworks such as PyTorch and TensorFlow. 

Also: To measure ultra-low power AI, MLPerf gets a TinyML benchmark

“The interesting thing to me is the pent-up demand for a purpose-built platform to support legacy is pretty high,”  Rangasayee told ZDNet. “They can run their application almost from day one — that's a huge advantage we have.” 

“We are the first company to crack the code on solving any computer vision problem, because we don't care for the code base, it can be in C++, it can be in Python, or any ML framework,” explained Rangasayee. The broad support for programs, he said, inclines the company to view itself as the “Ellis Island” of chips. “Give us your poor, give us your tired, we'll take ‘Em all!” he said. 

That broad support means the company has a larger audience of tens of thousands of customers rather than just a niche, asserted Rangasayee.

Another point in the chip's favor, according to Rangasayee, is that it has ten times the performance of any comparable part. 

“The thing our customers care about is frames per second per watt” in terms of the image frames in for every watt of power, said Rangasayee. “We are minimum 10x of anyone,” he said. “We are demonstrating that day in and day out to every one of our customers.” 

The company doesn't yet offer benchmark specs according to the widely cited MLPerf benchmark scores, but Rangasayee said the company intends to do so farther down the road. 

“Right now, the priority is to make money,” said Rangasayee. “We are a very small company” at 120 employees, “we can't dedicate a team to do ML perf alone.” 

“You could do a lot of tweaking around benchmarks, but people care about end-to-end performance, not just an MLPerf benchmark.

“Yes, we have numbers and yes we do better than anybody else, but at the same time, we don't want to spend our time building in benchmarks, we only want to solve customer problems.” 

Although Tuesday's announcement is about a chip, SiMa.ai places special emphasis on its software capability, including what it calls “novel compiler optimization techniques.” The software makes it possible to support “a wide range of frameworks,” including TensorFlow, PyTorch, and ONNX, the dominant programming libraries that machine learning uses to develop and train neural networks.

The company says its software allows users to “run any computer vision application, any network, any model, any framework, any sensor, any resolution.” 

Said Rangasayee, “You could spend a lot of time on one application, but how do you get thousands of customers across the finish line? That's really the harder problem.”

Toward that goal, the company's software effort, said Rangasayee, consists of two things: compiler innovations in the “front end” and automation in the “back end.”

The compiler will support “120-plus interactions,” that affords the “flexibility and scalability” of bringing many more kinds of applications into the chip than would ordinarily be the case.

The back end portion of the software means that more applications can be “mapped into your performance” rather than “waiting months for results.”

“Most companies are putting a human in the loop to get the right performance,” said Rangasayee. “We knew we had to automate in a clever way to get a better experience in minutes.”

That software innovation is designed to make the use of the MLSoC “push-button,” he said, because “everyone wants ML, nobody wants the learning curve.” That is an approach that Rangasayee's former employer, Xilinx, has also taken in trying to make its embedded AI chips more user-friendly.

“I learned at my previous company how important software is, and this is really going to be dependent on the strength of our software,” said Rangasayee. “Yes, our silicon is great, and we are very proud of it, and without silicon you are not a company,” he said. 

“But, to me, that's the necessary function, not the sufficient; the sufficient function is to provide an effortless ML experience.”