AI Product & Evals Analyst (Voice Agents)
Company: Bluemachines.ai
Location: Bangalore / 5 days WFO
Experience: 3–4 years
Function: Analytics
Role Summary
We’re hiring an AI AI Product & Evals Analyst Analyst to own evaluation, quality measurement, dashboarding, and enterprise analytics for Bluemachines’ AI voice agents.
You will work at the intersection of AI Engineering, Product, and Customer Delivery to define how voice agent quality is measured, monitored, and improved at scale. This includes building evaluation frameworks, detecting regressions, analyzing business impact, and enabling enterprise teams with clear, actionable insights.
Beyond evaluation, we’re looking for someone with an AI-first mindset who can leverage modern AI tools and automation workflows to improve analytics, reporting, and operational efficiency. The role will also contribute toward building reusable analytics frameworks, internal intelligence tooling, and self-serve analytics capabilities for enterprise clients.
Your work will directly influence launch quality, customer trust, operational transparency, and how enterprise AI systems are measured and scaled.
What will you do?
Business & Enterprise Analytics
- Track and analyze business KPIs tied to voice agent deployments (automation, containment, conversion, CSAT, cost savings, etc.)
- Support enterprise billing, audit, and reconciliation workflows through reliable usage and conversation analytics
- Translate AI behavior into clear operational and business impact insights for enterprise clients
AI-First Analytics & Automation
- Leverage AI tools and agentic workflows to improve speed, quality, and scalability of analytics, reporting, and RCA workflows
- Automate repetitive analytics and reporting tasks using LLMs, copilots, and workflow tooling
- Drive AI-native ways of working across SQL, Python, dashboards, and insight generation
Analytics Platform & Internal Tooling
- Build reusable analytics frameworks, standardized dashboards, and internal tooling for enterprise deployments
- Contribute to self-serve analytics and intelligence capabilities such as text-to-data, AI-powered insights, and conversational analytics interfaces
- Help evolve Bluemachines’ analytics and observability layer for enterprise AI voice agents
Voice Agent Evaluation
- Design and own AI eval frameworks specifically for voice agents
- Define conversation- and turn-level metrics
- Build and maintain golden voice datasets across accents, languages, and noise conditions
What We’re Looking For
Must-Have
- 2–5 years of experience in Product Analytics, Data Analytics, AI Quality, or related analytical roles
- Strong analytical thinking, metric design, and problem-solving skills
- Hands-on proficiency in SQL and Python for analysis, evaluation, and data investigation
- Ability to reason deeply about conversational, voice, and workflow-driven systems
- Experience building dashboards and enterprise-grade visualizations using tools like Tableau, Power BI, Looker, or equivalent
- Strong communication and storytelling skills with the ability to translate AI/system behavior into business insights
- Comfort working cross-functionally with AI Engineering, Product, Operations, and customer-facing teams
- AI-first mindset with strong adoption of modern AI tools, copilots, and automation workflows to improve productivity and scale analytics
Good-to-Have
- Experience with voice systems such as STT, TTS, call analytics, or conversational AI platforms
- Exposure to LLM-based products, prompt engineering, AI evals, or agentic workflows
- Experience with enterprise analytics, client-facing reporting, billing/reconciliation, or operational analytics
- Interest in building internal analytics tooling, reusable frameworks, or self-serve intelligence systems
- Experience working in fast-moving AI or startup environments
Similar roles
Broking Compliance Manager
Green Lane Talent Management·Mumbai, India
Job Title: Manager – Broking Compliance Reporting To: Vice President – Compliance Location: Lower Parel, Mumbai Position Overview The Manager – Broking Compliance will oversee end-to-end regulatory compliance, secretarial governance, and regulatory reporting for Institutional Broking, Depository Participant (DP) services, Research Analysis, and Offshore operations. This role acts as a primary liaison with key market regulators and ensures robust adherence…
- Full-time
Senior Back Office Supervisor
Datamark, Inc.·Mumbai, India
Position: Senior Back Office Supervisor at DATAMARK, Inc. DATAMARK, Inc. is seeking a dynamic and experienced Senior Back Office Supervisor to join our team in Datamark Mumbai. In this pivotal role, you will oversee and enhance the effectiveness of our back office operations, ensuring that our team meets organizational and client needs. As a Senior Back Office Supervisor, you will…
- Full-time
Senior Software Engineer - Backend (Rust)
Fortanix·Bengaluru, India
In today's world, where data spreads across various clouds and devices, traditional security measures aren't enough. Businesses need a dynamic approach to defend against constant cyber threats and ensure agile data security. Fortanix leads the way in data-centric cybersecurity for hybrid multicloud environments, using advanced cryptography, encryption, and confidential AI solutions. As data breaches become more frequent and traditional defenses…
- Hybrid
- Full-time
Data Operations Manager
Origin·Bengaluru, India
About Origin Origin (previously 10xConstruction) is building general-purpose autonomous robots for US construction to tackle rising costs, safety risks, and labour shortages. Our modular, multi-trade platform combines purpose-built hardware with real-time site intelligence to navigate complex environments and execute tasks with precision. Trained in high-fidelity simulation and already deployed on live sites, our robots deliver faster execution, stronger project economics,…
- Full-time
Hardware QA Engineer - Mechanical
Origin·Bengaluru, India
About Origin Origin (previously 10xConstruction) is building general-purpose autonomous robots for US construction to tackle rising costs, safety risks, and labour shortages. Our modular, multi-trade platform combines purpose-built hardware with real-time site intelligence to navigate complex environments and execute tasks with precision. Trained in high-fidelity simulation and already deployed on live sites, our robots deliver faster execution, stronger project economics,…
- Full-time
Junior Data Analyst
Immediate Media Co·London, United Kingdom
Junior Data Analyst (up to £35,000 per annum) Immediate is home to some of the biggest and most loved consumer brands in the UK, including Radio Times, Good Food and BBC Gardeners World magazine. Our trusted, quality content reaches millions of people a month across digital, print, video, podcasts, apps and live events. We’re the destination for people looking to…
- Hybrid
- Full-time
Senior Analytics Engineer
Perk·London, Canada
About Us Perk (formerly TravelPerk) is the intelligent platform for travel and spend management. Built to tackle the time-consuming, manual work that gets in the way of real work, our tools automate everything from travel bookings to expenses, invoice processing, and more. By eliminating this shadow work that wastes hours, erodes morale, and saps innovation, we're on a mission to…
- On-site
- Full-time
- Express Entry — PR day one, no employer
Lead- Business Development
SWIGGY·Kolkata, India
Lead – Business Development Key Responsibilities: * Identify and onboard high-potential restaurants, bars, and venues with strong event and experiential offerings. * Drive city-level supply expansion by curating a pipeline of upcoming restaurant events and nightlife experiences. * Ensure partner readiness with complete listing hygiene (visuals, pricing, logistics, terms). * Develop and scale owned IPs for seasonal campaigns, experience nights,…
- On-site
- Full-time
