The State of On-Device Voice AI: Benchmarking Local Dictation and TTS with Superwhisper and Kokoro
The Shift Toward On-Device Speech Processing
For years, speech-to-text (STT) and text-to-speech (TTS) were dominated by cloud APIs (Google Cloud Speech, Amazon Polly, ElevenLabs). While cloud models offer high accuracy, they present distinct drawbacks: 1. Latency Overhead: Network round-trips add 200ms to 800ms of lag before audio streams begin playing or transcribing. 2. Data Privacy Risks: Dictating confidential legal documents, medical notes, or proprietary code into a cloud microphone stream exposes sensitive audio to third-party servers. 3. Usage Costs: Streaming transcription and synthesis across millions of spoken words introduces recurring per-character or per-minute SaaS charges.
With advancements in quantized neural networks and unified Apple Silicon / NPU architectures, on-device voice AI has achieved parity with commercial cloud endpoints.
---
1. Local Voice Dictation: Superwhisper
Superwhisper provides system-wide voice-to-text dictation across macOS, Windows, iOS, and Android.
Architectural Highlights - Local Whisper Execution: Runs optimized Whisper models directly on local hardware (utilizing Apple Silicon Neural Engine or Windows DirectML). Audio data never leaves the user's workstation. - Custom Vocabulary Conditioning: Engineers and doctors can inject project dictionaries, package names, and technical terms into local speech models. Learn how to configure this in our Superwhisper Developer Vocabulary Skill. - Hybrid Flexibility: Provides optional cloud fallback models for lightweight Intel hardware configurations where local model weights would incur excessive RAM overhead.
Pricing Structure - Free Plan: Unlimited dictation with local Whisper models, 2 modes, and custom vocabulary (after a 3,000-word Pro trial). - Pro Monthly: Exact unrounded price of $8.49/month. - Pro Annual: $84.99/year (saves ~17%). - Lifetime: $249.99 one-time purchase.
---
2. Ultra-Lightweight Voice Synthesis: Kokoro-82M
While Superwhisper processes incoming speech, Kokoro TTS (developed by Hexgrad) handles outgoing speech generation.
Why Kokoro-82M is a Breakthrough - Compact 82M Parameter Footprint: Unlike massive multi-gigabyte models, Kokoro-82M's entire weight package is approximately 300MB, allowing it to run at real-time speeds on standard consumer CPUs without requiring a dedicated discrete GPU. - True Multilingual Support: In its official v1.0 release, Kokoro supports 9 language variants across 54 distinct voices: - American English (20 voices) - British English (8 voices) - Japanese (5 voices) - Mandarin Chinese (8 voices) - Spanish (3 voices) - French (1 voice) - Hindi (4 voices) - Italian (2 voices) - Brazilian Portuguese (3 voices) - Apache-2.0 License: Permissive open-weight distribution allowing unconstrained commercial use and private self-hosting.
[!CAUTION] > As noted in the official model card, beware of third-party domains containing "kokoro" in their URL. Official weights and code are distributed exclusively via Hugging Face (hexgrad/Kokoro-82M) and GitHub (hexgrad/kokoro).
---
On-Device vs. Cloud Performance Comparison
| Attribute | On-Device (Superwhisper / Kokoro) | Cloud APIs (ElevenLabs / Whisper Cloud) | | :--- | :--- | :--- | | Inference Latency | Instant local response (0 network lag) | 250ms – 900ms network round-trip | | Data Privacy | 100% on-device; zero remote telemetry | Audio transmitted to third-party servers | | Offline Capability | Operates without internet connectivity | Completely non-functional offline | | Hardware Requirement | 4GB – 8GB free RAM; modern CPU/NPU | Negligible local compute | | Operating Cost | Zero per-word API billing | $0.006 – $0.06 per minute or 1M chars |
---