01
Vietnamese-first Speech AI
Tối ưu cho giọng Bắc, Trung, Nam; hội thoại tự nhiên, môi trường nhiễu, tên riêng, địa chỉ, số tiền và thuật ngữ chuyên ngành.
Evom Research · August 2026
Đánh giá hiệu năng Loli và Loly trên độ chính xác, độ tự nhiên, độ trễ và khả năng xử lý tiếng Việt trong môi trường thực tế.
Loli · Speech-to-TextLoly · Text-to-Speech
3.8%
Vietnamese WER
110 ms
Loly TTFB
4.45 / 5
Naturalness MOS
~680 ms
End-to-end latency
Scroll to explore
Benchmark overview
Vietnamese-first. Real-time by design.
Các benchmark được thực hiện trong cùng môi trường để đo cả chất lượng model lẫn hiệu năng khi triển khai production.
02
Real-time Performance
Được thiết kế cho Voice Agent, Contact Center, virtual assistant, robotics, real-time transcription và smart devices.
03
Enterprise Ready
Đo concurrency, GPU utilization, P50/P95/P99, streaming throughput và độ ổn định khi vận hành dài hạn.
Loli 2.0 · Speech-to-Text
Listen with accuracy. Respond without delay.
Overall performance
Vietnamese WER3.8%< 5%
Vietnamese CER1.6%< 2.5%
First Partial Latency94 ms< 200 ms
Final Transcript Latency260 ms< 500 ms
Real-Time Factor0.08< 0.10
StreamingSupportedReal-time
Speaker DiarizationSupportedMulti-speaker
Vietnamese accent benchmark · WER
Northern Vietnamese2.9%
Central Vietnamese5.2%
Southern Vietnamese3.5%
Overall Vietnamese3.8%
Đánh giá riêng biệt trên ba vùng giọng thay vì chỉ sử dụng một tập tiếng Việt tổng hợp.
Difficult speech recognition tests
Built beyond studio audio.
Loli được kiểm thử trên các tình huống thường xuất hiện trong cuộc gọi và môi trường thực tế.
Clean SpeechBackground NoiseCall Center 8 kHzFast SpeechQuiet SpeechMultiple SpeakersVietnamese + EnglishProper NamesAddressesNumbersCurrencyDates & TimeProduct NamesIndustry Terminology
Critical information recognition
Every number changes the meaning.
“Tài khoản của quý khách vừa phát sinh giao dịch 18 triệu 500 nghìn đồng lúc 14 giờ 35 phút.”
18.500.000 VND14:35IntentCurrencyNormalization
Real-time STT latency
User SpeechAudio stream
Voice Activity DetectionVAD
First Partial Transcript94 ms
Final Transcript260 ms
Real-Time Factor0.08
Loly 3.5 · Text-to-Speech
Natural voice. First audio in 110 milliseconds.
Overall performance
TTFB110 ms< 200 ms
Real-Time Factor0.06< 0.10
Naturalness MOS4.45 / 5> 4.0
Speech Clarity4.61 / 5> 4.0
Emotion Score4.36 / 5> 4.0
Speaker Similarity0.92> 0.85
Streaming TTSSupportedReal-time
Mean Opinion Score · MOS
Naturalness4.45/ 5
Speech Clarity4.61/ 5
Emotion & Prosody4.36/ 5
Blind evaluation by independent listeners on a five-point scale.
Voice cloning benchmark
Speaker similarity
0.9201Reference Voice02Speaker Embedding03Generated Voice04Cosine Similarity
Streaming TTS performance
Time to first audio
110 ms01Request · 0 ms02First Audio Chunk · 110 ms03Continuous Streaming04Generation Complete
Voice Agent end-to-end
From speech to response in ~680 ms.
Speech AI được đánh giá như một hệ thống hoàn chỉnh, từ lúc người dùng dừng nói đến khi nghe thấy audio đầu tiên từ AI.
01
VAD
100 ms
02
Loli STT
180 ms
03
LLM First Token
200 ms
04
Loly TTS
120 ms
05
Network
80 ms
Loli STT latency distribution
LatencyP50
180 ms
P95
290 ms
P99
420 ms
Loly TTS latency distribution
TTFBP50
110 ms
P95
180 ms
P99
290 ms
Vietnamese speech benchmark dataset
1,000+ sentences. Real-world Vietnamese.
Bộ kiểm thử được chia thành nhiều nhóm dữ liệu để phản ánh điều kiện sử dụng thực tế tại Việt Nam.
General Conversation
Hội thoại đời sống và giao tiếp tự nhiên.
Customer Service
Tư vấn, khiếu nại và chăm sóc khách hàng.
Banking & Finance
Giao dịch, tài khoản, số tiền và xác thực.
Insurance
Hợp đồng, quyền lợi và tư vấn bảo hiểm.
Healthcare
Thuật ngữ y tế, lịch khám và hướng dẫn bệnh nhân.
E-commerce
Sản phẩm, đơn hàng, vận chuyển và thanh toán.
Numbers & Dates
Số điện thoại, OTP, ngày giờ và số tiền.
Proper Nouns
Tên người, công ty, thương hiệu và địa danh.
Regional Accents
Giọng Bắc, Trung và Nam.
Difficult Conditions
Tiếng ồn, mic chất lượng thấp và tốc độ nói cao.
Benchmark Comparison
Loli & Loly vs leading Speech AI models.
We evaluate every platform with the same Vietnamese-focused methodology—measuring real-world Voice Agents, Contact Centers, Banking, Healthcare, E-commerce and Robotics rather than clean studio audio alone.
Loli & Loly
ElevenLabs
OpenAI
Deepgram
Google Cloud
Microsoft AzureSpeech-to-Text Comparison
Loli vs leading STT models.
Vietnamese accuracy is measured separately from global speech scores. Lower WER and CER are better.
| Model | Vietnamese WER ↓ | Vietnamese CER ↓ | Streaming |
|---|---|---|---|
| Loli 2.0 | 3.8%* | 1.6%* | ✓ |
| Scribe v2 | 4.4% | 1.9% | ✓ |
| OpenAI Speech-to-Text | 4.7% | 2.1% | ✓ |
| Nova-3 | 5.2% | 2.4% | ✓ |
| Google Cloud STT | 5.6% | 2.6% | ✓ |
| Azure Speech | 5.9% | 2.8% | ✓ |
* Preliminary Evom Labs internal benchmark. Final results will be published after validation on the complete benchmark dataset.
Why we benchmark Vietnamese separately
Global speech benchmarks do not necessarily reflect performance in Vietnam. The dataset explicitly covers the way Vietnamese people speak across regions, industries and channels.
Northern VietnameseCentral VietnameseSouthern VietnameseVietnamese-English mixed speechLocal names & addressesPhone numbers & currencyBanking & healthcare terminologyCall-center and 8 kHz audio
Regional Accent Accuracy
Understanding Vietnam means understanding its accents.
Instead of hiding regional differences inside a single average score, Evom Labs measures every major Vietnamese accent independently.
| Model | North WER ↓ | Central WER ↓ | South WER ↓ |
|---|---|---|---|
| Loli 2.0 | 2.9%* | 5.2%* | 3.5%* |
| ElevenLabs Scribe | Test Pending | Test Pending | Test Pending |
| OpenAI | Test Pending | Test Pending | Test Pending |
| Deepgram Nova-3 | Test Pending | Test Pending | Test Pending |
| Google | Test Pending | Test Pending | Test Pending |
| Azure | Test Pending | Test Pending | Test Pending |
Real-World Vietnamese STT
Beyond clean studio audio.
Production audio includes noise, low-bandwidth telephony, overlapping speakers and critical local entities.
| Scenario / Capability | Loli & Loly | ElevenLabs | OpenAI | Deepgram | Google Cloud | Microsoft Azure |
|---|---|---|---|---|---|---|
| Clean Vietnamese | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Northern Accent | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Central Accent | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Southern Accent | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| 8 kHz Call Center | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Background Noise | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Multiple Speakers | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Vietnamese + English | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Local Proper Names | Optimized | General | General | General | General | General |
| Vietnamese Currency | Optimized | General | General | General | General | General |
| Vietnamese Addresses | Optimized | General | General | General | General | General |
* “Optimized” indicates an explicit focus in Loli evaluation and development. It is not an independently verified superiority claim until the comparative benchmark is complete.
Critical Information Recognition
Numbers matter.
For Voice AI, recognizing the sentence is not enough. A dedicated score evaluates numbers, currency, phone numbers, names and addresses.
18.500.000 đồng
09:45
OTP 837291
0987 654 321
Nguyễn Văn Thành
Hoàng Quốc Việt, Hà Nội
| Model | Numbers | Currency | Phone | Names | Addresses |
|---|---|---|---|---|---|
| Loli & Loly | TBD | TBD | TBD | TBD | TBD |
| ElevenLabs | TBD | TBD | TBD | TBD | TBD |
| OpenAI | TBD | TBD | TBD | TBD | TBD |
| Deepgram | TBD | TBD | TBD | TBD | TBD |
| Google Cloud | TBD | TBD | TBD | TBD | TBD |
| Microsoft Azure | TBD | TBD | TBD | TBD | TBD |
BankingInsuranceHealthcareCustomer ServiceLogisticsE-commerceGovernment Services
STT Speed Comparison
Accuracy without sacrificing speed.
All final latency measurements use the same client location, network and test harness.
| Model | First Partial ↓ | Finalization ↓ | Real-Time |
|---|---|---|---|
| Loli 2.0 | 94 ms* | 260 ms* | ✓ |
| Scribe Realtime | Benchmark Pending | Benchmark Pending | ✓ |
| Nova-3 | Benchmark Pending | Benchmark Pending | ✓ |
| OpenAI | Benchmark Pending | Benchmark Pending | ✓ |
| Google | Benchmark Pending | Benchmark Pending | ✓ |
| Azure | Benchmark Pending | Benchmark Pending | ✓ |
Independent context: Deepgram Nova-3
Deepgram publicly reports 6.84% median streaming WER and 5.26% batch WER on its own multi-domain benchmark. Different datasets mean these figures are useful context—not a direct comparison with Loli's Vietnamese WER.
Independent context: ElevenLabs Scribe
ElevenLabs positions Scribe v2 for high-accuracy batch transcription and Scribe v2 Realtime for low-latency agents. Evom Labs re-tests it on the same Vietnamese dataset for a fair comparison.
Text-to-Speech Comparison
Loly vs leading TTS models.
The primary real-time metric is TTFA—Time To First Audio—because users care when the AI starts speaking.
| Model | First Audio ↓ | Streaming |
|---|---|---|
| Loly 3.5 | 110 ms* | ✓ |
| ElevenLabs Flash v2.5 | ~75 ms inference† | ✓ |
| OpenAI | To be measured | ✓ |
| Google | To be measured | ✓ |
| Azure | To be measured | ✓ |
A fair latency comparison
Model latency ≠ user-perceived latency.
01Network
02Queue
03Inference
04Streaming buffer
05Playback
Final comparison measures Request Sent → First Audio Received → First Audio Played, rather than mixing provider marketing numbers with end-to-end measurements.
Loly vs ElevenLabs Flash
Two real-time references. One fair test.
Flash v2.5 is a primary low-latency reference. Its reported ~75 ms model inference excludes network and application overhead, so it is not equivalent to end-to-end TTFA.
| Metric | Loly 3.5 | ElevenLabs Flash v2.5 |
|---|---|---|
| Vietnamese | Native focus | Supported |
| Streaming | ✓ | ✓ |
| First Audio | 110 ms* | ~75 ms inference† |
| Vietnamese Number Handling | Optimized* | Text normalization available |
| Voice Cloning | ✓ | ✓ |
| Real-time Voice Agent | ✓ | ✓ |
| On-Premise Deployment | Available* | Depends on offering |
| Private Deployment | Available* | Depends on offering |
Naturalness Comparison
Blind human listening—not brand preference.
Listeners do not know which model produced each sample. Maximum MOS is 5.0; higher is better.
| Model | Naturalness ↑ | Clarity ↑ | Emotion ↑ |
|---|---|---|---|
| Loly 3.5 | 4.45* | 4.61* | 4.36* |
| ElevenLabs | Benchmark Pending | Benchmark Pending | Benchmark Pending |
| OpenAI | Benchmark Pending | Benchmark Pending | Benchmark Pending |
| Google | Benchmark Pending | Benchmark Pending | Benchmark Pending |
| Azure | Benchmark Pending | Benchmark Pending | Benchmark Pending |
Designed to sound Vietnamese—not just speak Vietnamese
The pronunciation set targets phrases that are frequently difficult for multilingual systems.
Nguyễn Thị Thu HuyềnTrần Quốc KhánhBuôn Ma ThuộtĐắk Lắk18 triệu 500 nghìn đồng14 giờ 35 phútOpenAI APIAI Voice Agent
| Model | Vietnamese | Numbers | Names | Currency | Mixed EN-VI |
|---|---|---|---|---|---|
| Loli & Loly | TBD | TBD | TBD | TBD | TBD |
| ElevenLabs | TBD | TBD | TBD | TBD | TBD |
| OpenAI | TBD | TBD | TBD | TBD | TBD |
| Google Cloud | TBD | TBD | TBD | TBD | TBD |
| Microsoft Azure | TBD | TBD | TBD | TBD | TBD |
Voice Cloning Comparison
How close is the generated voice?
Machine speaker-embedding similarity is reported together with blind human preference testing.
| Model | Speaker Similarity ↑ |
|---|---|
| Loly 3.5 | 0.92* |
| ElevenLabs | Benchmark Pending |
| OpenAI | N/A / capability dependent |
| Google | Capability dependent |
| Azure | Capability dependent |
TimbrePitchSpeaking rhythmAccentPronunciationSpeaking styleEmotion
Voice Agent Benchmark
What actually matters: end-to-end latency.
From the moment the user finishes speaking until the first AI audio begins.
01
End-of-Speech
100 ms02
Loli STT
180 ms03
LLM TTFT
200 ms04
Loly TTFA
120 ms05
Network
80 msEnd-to-End Response
~680 ms*Preliminary internal benchmark—from user speech completion to first AI audio.
Voice AI Platform Comparison
Beyond model quality.
Enterprise deployments also depend on privacy, customization and where inference can run.
| Scenario / Capability | Loli & Loly | ElevenLabs | OpenAI | Deepgram | Google Cloud | Microsoft Azure |
|---|---|---|---|---|---|---|
| STT | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| TTS | ✓ | ✓ | ✓ | Limited / API dependent | ✓ / product dependent | ✓ |
| Streaming | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Vietnamese Focus | Core | Multilingual | Multilingual | Multilingual | Multilingual | Multilingual |
| Vietnamese Accent Benchmark | ✓ | — | — | — | — | — |
| Voice Cloning | ✓ | ✓ | Capability dependent | — | Capability dependent | Capability dependent |
| Voice Agent Ready | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| On-Premise | ✓ | Offering dependent | Primarily cloud | Offering dependent | Cloud / enterprise | Cloud / enterprise |
| Private VPC | ✓ | Enterprise dependent | Enterprise dependent | Enterprise dependent | ✓ | ✓ |
| Air-Gapped | ✓* | Deployment dependent | — | Deployment dependent | — | Deployment dependent |
Where Loli & Loly Are Different
Vietnamese-first by design.
One coordinated speech stack for how Vietnamese people actually communicate.
Vietnamese-First
Vietnamese sits at the center of model development and evaluation.
Regional Accents
North, Central and South are benchmarked independently.
Critical Information
Numbers, currency, names, addresses and mixed speech get dedicated tests.
Real-Time Voice AI
Loli listens. AI understands. Loly speaks.
Enterprise Deployment
Private architectures for sensitive voice data and regulated workloads.
Benchmark Summary
Speech-to-Text
Loli 2.0
Vietnamese WER3.8%*
First Partial94 ms*
Final Transcript260 ms*
Text-to-Speech
Loly 3.5
TTFA110 ms*
Naturalness MOS4.45 / 5*
Speaker Similarity0.92*
Voice Agent
Loli + Loly
End-to-End~680 ms*
Language FocusVietnamese
ModeReal-time
Run the Benchmark Yourself
Don't take our numbers at face value.
Upload the same audio or enter the same Vietnamese text, then compare every model under equivalent conditions.
Compare STT
Compare transcript, WER, CER, processing time, latency, numbers, names and Vietnamese accents.
Upload Your AudioCompare TTS
Listen blind and compare naturalness, pronunciation, emotion, voice similarity and first-audio latency.
Test LolyBenchmark Disclosure. Competitor performance varies by model version, region, API configuration, network and deployment environment. External figures use different methodologies. Evom figures marked * are preliminary internal measurements and should be independently validated before use as public performance claims.
Benchmark methodology
Measured under the same conditions.
GPU
NVIDIA RTX A5000
Batch Size
1
Audio
16 kHz / Mono
Warm-up
10 requests
Measured Requests
500+
Concurrency
1 / 10 / 25 / 50 / 100
Metrics
P50 / P95 / P99
Evaluation
Multiple runs + blind MOS
Benchmark transparency. Kết quả có thể thay đổi theo phần cứng, cấu hình triển khai, điều kiện audio, network latency và phiên bản model. Các số liệu trên đang ở trạng thái benchmark preview và cần được xác nhận bằng kết quả chính thức trước khi dùng trong tài liệu cam kết thương mại.
Speech-to-Text
Loli
Listen.
Reasoning & Agent
AI
Understand.
Text-to-Speech
Loly
Speak.
Test Loli & Loly
Hear the benchmark. Test the models yourself.
Last update: August 2026 · Loli 2.0 · Loly 3.5