Key takeaways
- Match the chip to the exact task the product must run on-device. Most products only need a fraction of the compute a flagship module provides.
- On-device AI hardware splits into four practical levels: reflex-level microcontrollers, dedicated perception accelerators, multi-sensor edge AI boards, and robot-brain modules built for autonomy.
- TOPS alone does not predict real-world performance. Power budget, onboard memory, software support, and the specific model you plan to run matter just as much.
- Published dev-kit prices span roughly $10 for a basic microcontroller board to several thousand dollars for a robotics-grade module, so the chip choice is also a bill-of-materials decision.
Choose an AI chip by naming the exact task the product must run on-device first (a keyword trigger, a single-camera detection model, multi-sensor perception, or full autonomy), then picking the lowest-power, lowest-cost part that covers that tier. On-device AI hardware breaks into four practical levels, from microcontrollers that react to a single trigger up to robotics-grade modules running several neural networks at once. Settle on the right level first; it keeps a design from being underpowered or overpriced.
Confirm the product needs on-device AI first
Confirm that on-device AI is the right call for this product before comparing chips. That decision involves cost, cloud dependency, latency, and privacy tradeoffs covered in ai in iot devices. This guide assumes that call is already made and focuses on which chip fits once AI is going on the board, one layer inside the broader question of what physical AI means for a hardware product.
Free tool: check if your invention is ready to build, in 3 minutes →The four levels of on-device AI compute
The table groups the chips below into four tiers by what they are built to do. Use it as a first filter, then read the matching section for sourced specs and prices.
| Level | Typical job | Example hardware | Typical AI performance | Typical power |
|---|---|---|---|---|
| 1. Reflex | Wake word, gesture, vibration trigger | ESP32-S3 class MCU; NPU-equipped MCUs such as STM32N6 | No published TOPS up to roughly 0.6 TOPS | Milliwatts to about 1W |
| 2. Perception accelerator | One camera or mic, continuous inference | Hailo-8L, Hailo-8, Google Coral Edge TPU | 4 to 26 TOPS | Coral ~2W, Hailo-8 2.5W typical; Hailo-8L unpublished |
| 3. Multi-sensor edge AI | Several sensors or a heavier local model | Raspberry Pi 5 with AI HAT+ or AI HAT+ 2 | 13 to 40 TOPS | A few watts plus the host board |
| 4. Robot brain | Multi-sensor fusion, planning, autonomy | Jetson Orin Nano Super, AGX Orin, AGX Thor; Qualcomm Dragonwing/RB3 Gen 2 | 12 to 275 TOPS; Jetson Thor is quoted in FP4 TFLOPS instead | 7W to 130W |
Level 1: reflex chips for simple on-device triggers
Level 1 covers products that recognize one thing and react: a wake word, a knock pattern, a fall, a vibration signature. A general-purpose microcontroller with a few kilobytes of model storage is usually enough.
- Espressif ESP32-S3: a dual-core Xtensa LX7 MCU at up to 240 MHz with vector instructions for neural-network math, used through Espressif’s ESP-DSP and ESP-NN libraries (ESP-WHO for vision, ESP-Skainet for voice build on these). It has no dedicated NPU, and Espressif’s own page gives no TOPS figure (Espressif, ESP32-S3 product page, checked 4 Nov 2026). Basic dev boards commonly sell for $10 to $15 at electronics distributors.
- STMicroelectronics STM32N6: adds a true dedicated NPU, ST’s Neural-ART engine, alongside an Arm Cortex-M55 core at up to 800 MHz. ST’s own Neural-ART accelerator materials and trade coverage of the announcement both cite roughly 600 giga-operations per second, about 0.6 TOPS, at 3 TOPS per watt, aimed at tinyML tasks like on-sensor person detection (STMicroelectronics; CNX Software, 13 Dec 2024).
A TOPS number rarely matters here. The real question is whether the always-on model fits in SRAM within the power budget.
Level 2: perception accelerators for vision and audio
Level 2 fits products needing one AI task running continuously: detection, classification, or voice recognition from a single camera or mic, paired with a small host that handles everything else.
- Hailo-8L: 13 TOPS, sold as an M.2 module over a 2-lane PCIe Gen 3 interface. Hailo markets it for “entry-level edge devices” and publishes no wattage figure (checked 4 Nov 2026).
- Hailo-8: 26 TOPS at a typical 2.5W, also sold as an M.2 or PCIe module. The original “Commercial Version” die is now end of life, replaced by an “Industrial Version” (checked 4 Nov 2026).
- Google Coral Edge TPU (USB Accelerator): 4 TOPS (INT8) at roughly 2 TOPS per watt. Google’s current Coral developer page no longer lists detailed specs or pricing (checked 4 Nov 2026); one UK distributor had it in stock at £112.70 including VAT the same day, well above its original launch price, worth confirming before use.
Most Level 2 designs buy these chips on a carrier board rather than as a bare module, which is what Level 3 builds on.
Level 3: multi-sensor edge AI boards
Level 3 fits products running several sensors or a heavier model: two or more camera feeds, a local vision-language model, or a small generative model answering prompts offline.
- Raspberry Pi AI HAT+: an add-on for the Raspberry Pi 5, built around the same Hailo-8L or Hailo-8 chips above, offering 13 or 26 TOPS. Raspberry Pi lists it as “available now from $70” (checked 4 Nov 2026); trade press reported the 26 TOPS variant at $110 at its October 2024 launch.
- Raspberry Pi AI HAT+ 2: launched 15 January 2026, built around a Hailo-10H accelerator rated at 40 TOPS (INT4) with its own 8GB of onboard memory for small generative and vision-language models. Raspberry Pi lists it “available now for $200” (checked 4 Nov 2026).
The jump from Level 2 to Level 3 is less about raw TOPS and more about memory: dedicated onboard RAM, as on the AI HAT+ 2, matters more for a larger model than another 10 TOPS would.
Level 4: robot-brain modules for autonomy
Level 4 fits products that fuse multiple sensor streams, run planning loops, or support several neural networks at once: mobile robots, delivery drones, and autonomous equipment rather than a single smart sensor.
- Jetson Orin Nano Super: 67 TOPS (INT8), power from 7W to 25W. NVIDIA’s blog announced the kit at $249 on 17 December 2024; NVIDIA’s own marketplace listing now shows $399, marked out of stock there with partner retailers as the buying path, so confirm current pricing directly (checked 4 Nov 2026).
- Jetson AGX Orin: up to 275 TOPS, power from 15W to 60W, per NVIDIA’s product page (no price listed, checked 4 Nov 2026).
- Jetson AGX Thor: NVIDIA quotes 2070 TFLOPS in FP4 (sparse) rather than TOPS, power from 40W to 130W. Trade press reported a $3,499 developer-kit price around its August 2025 launch; NVIDIA’s page does not list one (checked 4 Nov 2026).
- Qualcomm Dragonwing QCS6490: the processor behind the Robotics RB3 Gen 2 kit, up to 12 dense TOPS from its Hexagon 770 engine per Qualcomm’s own product brief, aimed at robotics, drones, and industrial handhelds. Qualcomm’s kit page lists no price (checked 4 Nov 2026).
Qualcomm also announced a far more powerful robotics chip, the Dragonwing IQ10, in January 2026 for humanoid and industrial robots. Reported TOPS figures vary by source and no public developer-kit price exists yet, so treat it as a sourcing note, not a part to spec today.
NPU vs. GPU vs. MCU: what is actually doing the math
“AI chip” covers at least three approaches to the same math; the right one depends on how fixed or varied the product’s models will be.
| Approach | What it is | Where it shows up here | Tradeoff |
|---|---|---|---|
| Vector-accelerated MCU | A microcontroller core with instructions that speed up small-model math | ESP32-S3 | Cheapest, lowest power; limited to small, fixed models |
| Dedicated NPU | A separate block built only to run neural-network math | STM32N6, Hailo-8/8L, Coral Edge TPU, Qualcomm Hexagon | Best power efficiency per inference; narrower software flexibility |
| GPU with tensor cores | A graphics processor with specialized tensor-core instructions | NVIDIA Jetson family | Most flexible for many model types; higher power draw per inference |
A dedicated NPU generally wins on TOPS per watt for one well-defined model; a GPU-based platform trades some efficiency for running more model types.
Match power budget, TOPS, and cost to your product
Three numbers decide fit: the power budget, the TOPS the target model needs at its required latency, and the unit cost the price point can absorb.
The right AI chip is usually the cheapest, lowest-power part that still hits the accuracy and latency the product needs, not the one with the largest TOPS number on the spec sheet.
| Power budget | Sensor load | Likely level |
|---|---|---|
| Coin cell or small battery, months between charges | Single trigger, no continuous inference | Level 1 |
| Rechargeable battery or low-power mains | Single camera or mic, continuous inference | Level 2 |
| Larger battery or mains power | Multiple sensors or a heavier local model | Level 3 |
| Mains or a large battery pack | Multi-sensor fusion, planning, or control loops | Level 4 |
A developer-kit price is not a production price: the bare chip inside a $249 or $3,499 kit usually costs a fraction of that at volume. TOPS figures also are not directly comparable across vendors, since some use INT8, some INT4, and NVIDIA’s newest platform uses FP4 sparse teraflops instead. Treat any single TOPS number as a rough filter, then confirm with a real benchmark.
A practical decision checklist
- Write the on-device task in one sentence: the input (camera, mic, accelerometer) and the output (a classification, a transcript, a control signal).
- Estimate compute need from benchmarks for that task, not a generic TOPS target from a spec sheet.
- Set the power budget from battery size and runtime, or available continuous current on mains power.
- Check that the chip’s software stack supports the model format the team already uses.
- Price the developer kit and the production chip separately; a $3,499 kit can still use a part that costs far less in volume.
- Run the actual target model on at least one candidate chip before locking the board design.
Frequently asked questions
What is the difference between an NPU and a GPU for edge AI?
An NPU is purpose-built for neural-network math and is usually more power-efficient per inference than a GPU running the same model. A GPU-based platform, like the Tensor Cores in NVIDIA’s Jetson family, trades some of that efficiency for flexibility across more model types. A fixed, single-task product usually favors a dedicated NPU; a product running several model types over its life usually favors a GPU-based module.
How many TOPS do I actually need for my product?
Start from the model, not the chip. A wake-word model needs no NPU at all. A single-camera detection model at a few frames per second typically needs single-digit to low-teens TOPS. Larger vision-language models push the requirement into the tens of TOPS, and real-time multi-sensor fusion usually pushes a design past 100 TOPS.
Can a microcontroller like the ESP32-S3 run real AI models?
Yes, within limits. It has no dedicated NPU, but Espressif’s ESP-DSP and ESP-NN libraries use its vector instructions to accelerate small models, and projects such as ESP-WHO and ESP-Skainet run on it today. These are narrow models measured in kilobytes, built for one task such as a wake word, not general-purpose vision or language models.
Is the Raspberry Pi AI HAT+ enough for computer vision?
For many single- or dual-camera products, yes. The original AI HAT+ delivers 13 or 26 TOPS depending on variant, enough for real-time detection on one or two camera streams. The AI HAT+ 2 adds a Hailo-10H accelerator at 40 TOPS with 8GB of its own memory for small generative and vision-language models. A product with many simultaneous feeds or real-time SLAM usually outgrows this tier.
When do I need an NVIDIA Jetson or a Qualcomm robotics chip instead?
Reach for a Jetson-class or Qualcomm robotics-class module when the product fuses multiple sensor streams in real time or supports several neural networks at once, the profile of mobile robots and autonomous equipment. Jetson Orin Nano Super sits at the low end at 67 TOPS; Jetson AGX Thor and Qualcomm’s Dragonwing chips target humanoid and industrial robotics with far higher budgets.
How much should I budget for the AI chip or accelerator itself?
Published dev-kit prices, checked 4 Nov 2026, range from roughly $10 to $15 for a microcontroller board, $70 to $200 for a Raspberry Pi AI HAT+ accelerator, $249 to around $400 for a Jetson Orin Nano Super kit, and into the thousands for a Jetson AGX Thor or robotics-class Qualcomm platform. Production unit cost is usually lower than dev-kit price at volume.
Where Inventornest fits
The chip is one decision inside a larger set of tradeoffs: sensor selection, enclosure thermal design, battery sizing, firmware architecture, and the certification path for a product that transmits data or uses RF. A hardware partner should be able to show how that choice ripples through the rest of the PCB and embedded system design, not just quote a part number off a spec sheet. Look for a team that tests more than one candidate chip against the real target model before the board is finalized, a question that matters as much for an IoT product design services engagement as for a robotics build.
physical AI product development work of this kind typically starts with defining the on-device task, benchmarking candidate chips against it, and only then locking the hardware design. A portfolio of hardware projects is one way to check whether a team has handled sensor-heavy or AI-accelerated products before, and a structured prototype test plan is a useful companion once a chip is chosen and the first board comes back. The next step is usually to book a consultation and walk through the four levels against the product’s own requirements.
