DevPik Logo
speech to texttranscriptionaiopenaimultilingual

Whisper: OpenAI's Free Speech-to-Text AI You Can Run on Your Own Computer

Whisper is OpenAI's own speech-to-text AI, and you can run it free on your own computer. Give it any audio file and it writes out everything that was said, in dozens of languages, and it can translate other languages straight into English. It has 113,000+ stars on GitHub, it is MIT licensed, and the smallest models need only about 1 GB of graphics memory.

ByMuhammad Tayyab8 min read
All open source picks
openai/whisper
The official repository — this write-up is not affiliated with the project.
113.2kPythonMIT

What Whisper Is: One Model That Listens, Translates, and Identifies Languages

Whisper is a general-purpose speech recognition model from OpenAI, and it is free and open source under the MIT license, with 113,221 stars on GitHub. It is trained on a large dataset of diverse audio, and it is a multitasking model: the same model does multilingual speech recognition, speech translation, spoken language identification, and voice activity detection. Under the hood it is a Transformer sequence-to-sequence model. The training format represents all of those tasks as a sequence of tokens for the decoder to predict, with special tokens that tell the model which task it is doing, so a single model replaces the many stages of a traditional speech pipeline.

Six Model Sizes: From Tiny on a Laptop to Large on a GPU

There are six model sizes, and picking one is a trade between speed, accuracy, and memory. Tiny has 39 million parameters, needs about 1 GB of graphics memory, and runs about 10 times faster than large. Base has 74 million parameters, also needs about 1 GB, and runs about 7 times faster. Small has 244 million parameters, needs about 2 GB, and runs about 4 times faster. Medium has 769 million parameters, needs about 5 GB, and runs about 2 times faster. Large has 1.55 billion parameters, needs about 10 GB, and is the reference speed. Turbo has 809 million parameters, needs about 6 GB, and runs about 8 times faster than large. Four of the sizes also come in English-only versions, which tend to do better for English-only work, especially tiny and base.

The Turbo Catch: The Default Model Does Not Translate

This is the one thing that trips people up. The default model is turbo, which is an optimized version of large-v3 that transcribes much faster with only a minimal drop in accuracy, and it works well for transcribing English. But turbo is not trained for translation tasks. If you ask it to translate non-English speech into English, it will return the original language instead, even if you pass the translate flag. So if you need translation, do not use turbo. Use one of the multilingual models instead, medium or large for the best results, with the translate task and the language set, like whisper japanese.wav --model medium --language Japanese --task translate.

Languages: Dozens, With Honest Accuracy Numbers

Whisper works in dozens of languages, and it detects which language is spoken on its own, so you do not have to tell it in advance. Its accuracy varies widely by language, and the project is honest about that: the README shows a word error rate breakdown by language for the large models, measured on the Common Voice 15 and Fleurs datasets, with character error rates for languages where that is the fairer measure. More detailed scores live in the paper, including translation quality scores. The practical takeaway is that English and other high-resource languages transcribe very well, while rarer languages are rougher, and the larger models close more of that gap.

Installing Whisper and Transcribing Your First File

Installing Whisper takes two steps. First install ffmpeg, which Whisper needs to read audio, from your package manager. Then run pip install -U openai-whisper, which pulls the latest release. The codebase works with Python 3.8 to 3.11 and recent PyTorch versions. After that, transcription is one command: whisper audio.mp3 --model turbo transcribes a file with the default model. You can pass several files at once, set the language with --language, or pick a different model with --model. If you would rather use it from Python, load a model with whisper.load_model("turbo") and call model.transcribe("audio.mp3"). Internally the transcribe method reads the audio with a sliding 30-second window and predicts the text a window at a time.

Running It Locally: Your Audio Stays on Your Machine

The part that matters most for many people is that Whisper runs locally. Your audio never has to leave your computer, which is a big deal for interviews, meetings, and voice notes you would rather not upload to a cloud service. The smallest models run on about 1 GB of graphics memory, so even a modest machine can transcribe. Once you have a transcript, it is just text, so the usual text tools apply. Whisper can also write its output as JSON with word-level timestamps, which is what caption tools want. If your timestamps live in a spreadsheet instead, a quick CSV to JSON pass turns them into JSON without retyping anything. And when a transcript is ready to share, a Text to HTML pass turns it into a clean web page you can send to anyone.

Frequently Asked Questions

What is Whisper?

Whisper is OpenAI's free open-source speech recognition model. It transcribes audio in dozens of languages, translates other languages into English, and identifies the spoken language, all with one model. It has 113,000+ stars on GitHub.

Is Whisper really free?

Yes. Whisper's code and model weights are released under the MIT license. You install it with pip, run it on your own computer, and there is nothing to pay.

Which Whisper model should I use?

Use tiny or base for quick drafts on a modest machine, small or medium for a good balance, and large for the best accuracy. Turbo is the fastest good option for transcribing English, but it cannot translate. The English-only .en versions do better for English-only work.

Why does turbo not translate?

Turbo is an optimized version of large-v3 built for fast transcription, and it was not trained for translation tasks. It returns the original language even when you pass the translate flag. For translation, use medium or large instead.

How many languages does Whisper support?

Whisper supports dozens of languages and detects the spoken language automatically. Accuracy varies by language, and the README publishes word error rates by language on the Common Voice 15 and Fleurs datasets.

Do I need a GPU to run Whisper?

A GPU helps, but it is not required. The tiny and base models need only about 1 GB of graphics memory and run on modest hardware. Larger models need more memory and run slower without a GPU, so match the model to your machine.

Related DevPik tools

Muhammad Tayyab

Written by

Muhammad Tayyab

Founder of DevPik

Muhammad Tayyab founded DevPik and builds its free developer tools. He writes about developer tools, AI and web technology.

More open source picks