caferemote
Innodata

AI Voice Evaluation Specialist - 3 month contract

Remote Canada Contract CAD 20 – CAD 25 / hour Posted 3 days ago
Apply now
Work from
Canada
Contract
Contract
Salary
CAD 20 – CAD 25 / hour
Posted
17 November 2026
Apply by
16 January 2027

This role is a 3-month, W2, full-time contract (40 hours per week.)

Responsibilities

  • Review an assigned scenario and roleplay a natural conversation with two different AI models.
  • Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible.
  • Maintain approximately the same number of conversational turns with each model to support a fair comparison.
  • Record your voice during each interaction and pay close attention to the quality and behavior of the model's responses.
  • Evaluate each model across five defined evaluation dimensions.
  • Identify and categorize relevant error clusters that may occur in the audio or conversation.
  • Compare the performance of Model A and Model B based on the evaluation criteria and your observations.
  • Select the model that demonstrated the stronger overall performance according to the defined evaluation framework.
  • Write a detailed rationale explaining your final preference, referencing specific examples and observations from both conversations.
  • Apply evaluation guidelines consistently across different scenarios and model interactions

You’ll Thrive in This Role If You Have

  • Bachelors Degree
  • Strong attention to detail and ability to notice subtle differences in conversational quality.
  • Excellent listening and comprehension skills.
  • Strong written communication skills, particularly the ability to explain observations clearly and objectively.
  • Ability to consistently follow detailed evaluation guidelines.
  • Comfort speaking naturally and role-playing different conversational scenarios.
  • Ability to compare two interactions fairly without allowing personal preferences to influence the evaluation.
  • Strong critical-thinking and analytical skills.
  • Reliability and consistency when completing structured evaluation tasks.
  • Familiarity with AI assistants, voice-based AI, or conversational systems

Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide at

If you believe you’ve been targeted by a recruitment scam, please report it to Innodata at and consider reporting it to the FTC at .

Salary

The expected hourly salary range for this position is $20-$25 CAD p/hour, based on experience, skills, and qualifications.

About Innodata

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

Scope of the Role

We are looking for detail-oriented Voice Specialists to evaluate the performance of AI models through real-time, voice-based conversations. In this role, you will interact with two different AI models using the same assigned scenario, carefully compare their responses, and provide structured evaluations based on defined quality criteria.

The goal of the evaluation is to create a fair, consistent comparison between models and identify which model delivers the stronger overall conversational experience.

Discussion

No comments yet. Ask a question or share a tip about this role.

Apply now

More jobs from Innodata

View company →

Marketing Pitch Deck Reviewer

Innodata
Remote United States Part-time $25 / hour

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to…

yesterday Sales and Marketing

Business Development Representative, Healthcare & Life Sciences

Innodata
Remote United States Full-time $70K – $85K / year

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to…

4 days ago Sales and Marketing

Similar remote jobs

Browse all →

QA Engineer

SupplyHouse.com
Remote India Full-time $18K – $27K / year

Real people. Real service. At SupplyHouse.com , we value every individual team member and cultivate a community where people come first. Led by our core values of G enerosity, R espect, I…

4h ago Quality Assurance

Administrative Coordinator

Lynker Corporation
Remote United States Full-time

Overview Lynker Corporation is a leading provider of innovative solutions in weather and climate science. With a commitment to excellence and a passion for innovation, Lynker leverages…

5h ago Quality Assurance

Senior Automated QA Engineer (LATAM, Remote)

iDelsoft
Remote Anguilla, Antigua and Barbuda, Argentina, Aruba, Bahamas, Barbados, Belize, Bermuda, Bolivia, Bonaire, Sint Eustatius and Saba, Brazil, Cayman Islands, Chile, Colombia, Costa Rica, Cuba, Dominica, Dominican Republic, Ecuador, El Salvador, Falkland Is Full-time

We're hiring a Senior Automated QA Engineer for a US B2B SaaS company in fintech. You'll own automated test coverage across the product: design and build the test framework, run it on every…

6h ago Quality Assurance

Profee ED Auditor/QA

GeBBS
Remote United States Full-time $26 / hour

GeBBS Healthcare Solutions is a leading provider of Revenue Cycle Management (RCM) and Risk Adjustment services, helping healthcare organizations optimize operations, improve financial…

6h ago Quality Assurance

Testing Specialist

Cardinal Technology Systems
Remote United States Full-time

Responsibilities Cardinal Technology Systems Corp. is a government IT solutions provider servicing commercial and government initiative in various parts of the United States. We are…

7h ago Quality Assurance

Quality Engineering Manager

World Travel Holdings
Remote United States Full-time $135K – $150K / year

Overview World Travel Holdings is seeking an experienced Quality Engineering & Test Automation Manager to lead a blended Quality Engineering team and build an AI-enabled, automation-first…

8h ago Quality Assurance

Get remote jobs the moment they are posted

Choose your categories and skills. We push matching jobs to your device. No email, ever.