Conversation Quality Analyst (Internship)

About the role

Bolna builds voice AI agents that call, converse and convert customers in any language, and is building automated systems to grade calls at scale using LLMs as judges. Before trusting a model's judgment, it verifies it against a human. This is an internship role for someone early in their career, based in Bengaluru, where you listen to real calls, annotate what happened and check whether the automated systems got it right.

Key responsibilities

• Annotate real customer calls: transcription review, issue tagging and labeling with tools such as Label Studio, and help sharpen annotation guidelines for Hindi, English and Hinglish
• Verify LLM-as-judge evaluations: confirm whether flagged events like barge-ins really happened, and mark agreements and disagreements with reasoning
• Check system-flagged quantitative signals (e.g. agent interruptions) against the actual call and flag false positives and negatives
• Identify edge cases that current rubrics or detection logic miss, and bring patterns back to the team so rubrics, prompts and detection logic improve
• Regularly inspect calls beyond the flagged ones to spot new issues

Required profile

• Strong attention to detail and patience for focused, high-precision work across many calls
• Multilingual comfort preferred in Telugu, Tamil, Kannada, Marathi, Gujarati or Bengali, in addition to English and Hindi
• Comfortable learning new tools quickly and curious about AI and voice AI
• Good to have: prior data annotation, labeling or QA experience, familiarity with spreadsheets or basic SQL, background in linguistics, call center operations or content moderation

Required skills

• Label Studio and dashboards (tools provided)
• Spreadsheets and basic SQL, or comfort reading dashboards such as Metabase (good to have)

The role is designed as an entry point: consistent annotation leads to QA or Annotation Lead, interest in agent logic to a Forward Deployed Engineer track, and pattern recognition to a Data/Eval Engineer or Voice AI Analyst track. The team is small (~25 people) and meets in person in Bengaluru.