Dograh
Experiments  /  TTS verbalization benchmark  /  chatterbox
Model · EXP-001

chatterbox

Ranked 5 of 7 on the TTS verbalization benchmark. Strongest on initialism or acronym, weakest on mathematical expression.

At a glance
Said right44%share of all 1,080 sentences
Rank5 of 7among the seven models
Cost per clip3.48scomputing time per audio clip
Likely range41%–47%where the true score most likely sits
Strengths and weak spots

Where chatterbox does best and worst

Its five strongest and five weakest types of text, by the share of sentences it said right.

Best at
Initialism or Acronym
0.97
Ordinal
0.88
Vehicle or Product Code
0.88
Stock Ticker
0.82
License Plate or Serial Numbers
0.78
Breaks on
Mathematical Expression
0.00
Hashtag or Mention
0.00
Geographic Coordinates
0.00
URL or Email
0.15
Fractions
0.15
A real miss

Mathematical Expression: what chatterbox actually said

The text
On the whiteboard she wrote 164 + 400 = 564 and underlined it.
Should be said as
On the whiteboard she wrote one hundred sixty four plus four hundred equals five hundred sixty four and underlined it.
chatterbox said
"on the whiteboard she wrote one hundred four plus four hundred fifty six four and underlined it"
All 27 types

Every type of text, best to worst

The share of sentences chatterbox said right on each of the 27 types. Each type has 40 test sentences.

Initialism or Acronym
0.97
Ordinal
0.88
Vehicle or Product Code
0.88
Stock Ticker
0.82
License Plate or Serial Numbers
0.78
Chemical Formula
0.62
Abbreviation
0.60
Legal Reference
0.60
Phone Number
0.60
Roman Numeral
0.57
Currency
0.47
Biological Classification
0.45
Sports score
0.45
Cardinal
0.42
Unit
0.42
ISBN
0.40
Musical Notation
0.40
Decimal
0.33
Time
0.33
Date
0.23
Version Numbers
0.20
Address
0.17
Fractions
0.15
URL or Email
0.15
Geographic Coordinates
0.00
Hashtag or Mention
0.00
Mathematical Expression
0.00