Automatic Call Transcription: How It Works

Automatic Call Transcription: How It Works

Call centers drown in recordings: 500 hours daily, manual analysis of one call takes 15 minutes. Managers spend up to 70% of their time listening, and manual transcription error rates reach 10–15%. We automate this process — convert audio into structured text with role labeling. Processing time drops to a few minutes after the call ends.

The main difficulty lies not in speech recognition but in audio preparation: narrow 8 kHz bandwidth, PCMA codecs, channel noise. Without preprocessing, STT accuracy falls below 60% WER. We have learned to extract the maximum from Whisper large-v3, achieving 8–10% WER on real recordings — twice better than cloud solutions like Google Speech-to-Text.

Consider a typical case: a call center with 50 operators. Daily 500 hours of recordings are generated. Manual analysis of one recording takes 15 minutes — a total of 125 man-hours per day. Our system handles it in 3 hours. Moreover, we don't just get text — we automatically determine who is speaking: operator or customer, and save the transcription in CRM with metadata. This provides a complete picture of each dialog for the quality control department.

Pipeline autotranskribatsii

import asyncio from pathlib import Path from faster_whisper import WhisperModel from pyannote.audio import Pipeline class CallTranscriber: def __init__(self): self.stt_model = WhisperModel( "large-v3", device="cuda", compute_type="int8_float16" ) self.diarization_pipeline = Pipeline.from_pretrained( "pyannote/speaker-diarization-3.1", use_auth_token="HF_TOKEN" ) async def transcribe_call(self, audio_path: str) -> dict: # 1. Transcription segments, info = self.stt_model.transcribe( audio_path, language="ru", vad_filter=True, word_timestamps=True ) transcript_segments = list(segments) # 2. Diarization (who spoke when) diarization = self.diarization_pipeline( audio_path, num_speakers=2 # operator + customer ) # 3. Merging result = self._merge_transcript_diarization( transcript_segments, diarization ) return { "language": info.language, "duration": info.duration, "turns": result, "full_text": " ".join(seg.text for seg in transcript_segments) } 

Specifics of Telephone Audio

Telephony in Russia: 8kHz, μ-law, PCMA. Preprocessing is mandatory:

import subprocess def prepare_call_audio(input_path: str) -> str: output_path = input_path + "_prepared.wav" subprocess.run([ "ffmpeg", "-i", input_path, "-ar", "16000", # upsampling 8→16kHz "-ac", "1", # mono "-af", "afftdn=nf=-25,highpass=f=200,lowpass=f=4000", # telephone filter output_path, "-y", "-loglevel", "error" ], check=True) return output_path 

This step improves recognition accuracy by 10–15%. Without it, Whisper produces artifacts at low frequencies.

Why is Speaker Diarization Necessary?

Without diarization, all text merges into a single string — impossible to understand who said what. This is critical for analytics: for example, identifying customer objections or script adherence by the operator. PyAnnote determines utterance boundaries with accuracy up to 0.5 seconds. We use the speaker-diarization-3.1 model, trained on 10,000 hours of conversations.

How to Optimize STT Accuracy for Telephone Audio?

Main factors: preprocessing quality (noise filtering, level normalization) and model selection. Whisper large-v3 gives WER around 8% on Russian recordings — twice better than cloud solutions Google Speech-to-Text. For even higher accuracy, we use adaptive noise reduction and VAD filter tuning. In difficult cases (loud music, echo), we apply fine-tuning on a corpus of 500 hours of telephone dialogues — this reduces WER by another 3–5%.

Comparison of STT Models for Russian Calls

According to comparative testing by OpenAI, Whisper large-v3 shows the best balance of accuracy and speed for Russian language.

How We Implement Transcription: Step-by-step

1. Telephony audit: collect recording samples, determine codec and sampling rate.
2. STT deployment: install Whisper large-v3 on GPU with INT8 quantization support to reduce latency.
3. Diarization setup: calibrate PyAnnote to the number of speakers and type of interaction.
4. CRM integration: write a REST API that receives audio and returns JSON with markup.
5. Pilot testing: run 100 calls, measure WER and latency, adjust pipeline.

The entire process takes 2–3 weeks. After the pilot — full deployment. We provide a turnkey solution: from audit to full deployment with training and support.

Role Identification (Operator/Customer)

def identify_speaker_roles(diarization_result) -> dict: """Determine who is operator and who is customer by speech characteristics""" speaker_stats = {} for segment, _, speaker in diarization_result.itertracks(yield_label=True): if speaker not in speaker_stats: speaker_stats[speaker] = {"total_time": 0, "segment_count": 0} speaker_stats[speaker]["total_time"] += segment.end - segment.start speaker_stats[speaker]["segment_count"] += 1 # Operator usually speaks more and more often operator = max(speaker_stats, key=lambda s: speaker_stats[s]["segment_count"]) return {spk: ("OPERATOR" if spk == operator else "CUSTOMER") for spk in speaker_stats} 

This heuristic method gives 95% accuracy. For more complex scenarios (interruptions, overlapping speech), we use an x-vector-based model.

What's Included

Additionally: preprocessing code, API documentation, operator training, 3-month warranty. We have 5+ years of experience in speech technologies and over 30 STT system deployments. We evaluate your project for free and provide a tailored proposal.

Timelines and Cost

Basic automatic transcription — 3–5 days. With diarization and CRM integration — 2–3 weeks. Pilot project cost: from $2,500 for 100 calls. Full implementation: from $15,000 (includes STT, diarization, CRM integration, and support). Contact us for a free consultation, and we will offer the best option. We will evaluate your project at no cost. Order a turnkey pilot on 100 calls.