caferemote
Welocalize

Expert Rater

Remote India Full-time Posted yesterday
Apply now
Work from
India
Contract
Full-time
Posted
20 November 2026
Apply by
19 January 2027

If you have a Candidate Login already, but have forgotten your password please use the steps to reset your password. If you have forgotten your email login, please contact subject Workday Candidate Login

When creating your Workday account and entering personal information like name, address, please do not use ALL CAPS.

Thank you!

NOTICE: For Privacy Policy please review here

Responsibilities

Job Responsibilities

  • Welo Data is an award-winning localization and AI data company.
  • We run some of the world’s largest AI model relevance rating programs.
  • This will play a crucial role in analyzing and improving AI-driven online advertising platforms.

This role will help establish benchmarks and guidelines for data used to train AI models, particularly in areas lacking defined standards. That means you will be shaping the future of web-based advertising and AI models.

MAIN DUTIES

The following is a non-exhaustive list of responsibilities and areas of ownership of an Expert Rater

  • Rate Client-provided ads following pre-defined guidelines
  • Evaluate advertisements based on specific quality, relevance, and compliance criteria provided by the client.
  • Ensure ratings are objective, consistent, and aligned with the predefined rating standards.
  • Identify any guideline ambiguities and apply critical thinking when assessing borderline cases.
  • Maintain a high level of accuracy and efficiency, meeting daily/weekly rating quotas.
  • Review and refine rating guidelines, identifying areas for improvement and recommending updates to enhance accuracy, consistency, and overall rating quality.
  • Curate high-quality datasets ("golden sets") by selecting and validating top-tier rated ads, which can serve as benchmarks for AI model training and human rater calibration.
  • Collaborate with cross-functional teams (e.g., quality assurance, data analysts, and AI training teams) to ensure rating processes align with evolving industry and client needs.
  • Analyse trends and patterns in ad ratings, providing insights that contribute to guideline optimizations and overall rating effectiveness.
  • Assist in training and mentoring new raters, ensuring they understand guidelines and best practices for accurate evaluations.

Requirements

Experience

  • Have experience in AI model relevance, such as data labeling, linguistic annotation, relevance rating, etc.
  • Are analytical and data-driven with a proven track-record of data-based decision making
  • Have a strong interest and understanding of digital marketing and advertising, especially as it relates to artificial intelligence
  • Can effectively summarize complex information, questions, and solutions for various audiences Have experience researching and gathering insights, and understanding complex problems across a variety of topics
  • Can defining, explain, and defend their points of view in group discussions
  • Are comfortable reviewing and discussing a variety of topics – you do not need to have expertise on the topics, but willingness to research and understand to support consensus decisions is vital
  • Are comfortable with ambiguity in guidelines, workflows, and processes, and are motivated to help establish those systems
  • Native level fluency in the target language
  • Excellent understanding of both written & spoken English
  • A 4 year university degree and/or or at least 2 years work experience in one or more of the following: AI-based relevance rating, marketing/advertising, linguistics, content creation, writing, tech-related quality assurance, data labeling, or linguistic annotation.

Additional Job Details

Discussion

No comments yet. Ask a question or share a tip about this role.

Apply now

More jobs from Welocalize

View company →

Hindi & Punjabi Music Content Specialist

Welocalize
Remote India Contract INR 650K / year

Welo Data is looking for someone with a genuine passion for music and the artists behind it, across genres. This is an entry level role suited to a candidate…

8h ago Content Creator

Bengali Audio Transcription Raters

Welocalize
Remote India Part-time $4 – $5 / hour

About the Role   Review pre-segmented audio recordings, verify word-level segment accuracy, and correct timestamp or transcription errors to support…

19h ago Data

Bengali Audio Transcription QA

Welocalize
Remote India Part-time $6 – $7 / hour

About the Role   We are looking for experienced Adjudicators/QA to review and resolve discrepancies in word-level transcription and timestamp annotations…

19h ago Quality Assurance

Project Beid - Korean E-commerce Product Matching Annotation Reviewer

Welocalize
Remote Australia, Canada, China, United States Part-time $27 – $28 / hour

Responsibilities Review annotators’ ACCEPT/REJECT decisions and reject reason codes, with a focus on preventing false matches and false rejects. Sample-check…

20h ago Data

Similar remote jobs

Browse all →

AI Safety Expert - Bilingual

mercor
Remote United States Contract $17 – $25 / hour

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter…

2h ago Data

Senior AI & Transformation Enablement Specialist

Datasite
Remote United States Full-time $83K – $141K / year

About us Datasite is the essential infrastructure for M&A. Also comprising Grata, Blueflame AI, and Sherpany, we help dealbreakers find opportunities, manage transactions, and make key…

3h ago Data

Business Analyst III Claims, DRG

Centene Corporation
Remote United States Full-time $70K – $126K / year

Position Purpose: Perform various analysis and interpretation to link business needs and objectives for assigned function. Key Details: Must be authorized to work in the U.S. without the…

3h ago Data

Functional Consultant – Epicor Kinetic (Operations & Supply Chain) – 100% Remote

TeccWeb
Remote United States Full-time

About Teccweb Teccweb is an Epicor Service Partner specializing in the implementation and optimization of Epicor Kinetic for mid-market manufacturers and distributors across North America.…

3h ago Data

Product Owner

Dev Partners
Remote Philippines Full-time

💻 We Are Hiring- Product Owner | 100% Remote We are seeking an experienced Product Owner to oversee and coordinate the ongoing maintenance, enhancement, and continuous improvement of an…

3h ago Data

Mechanical Engineer III

DLB Associates
Remote United States Full-time $105K – $135K / year

DLB Associates is a premier engineering and consulting firm specializing in innovative, mission-critical solutions across hyperscale data centers, advanced infrastructure, and…

3h ago Data

Get remote jobs the moment they are posted

Choose your categories and skills. We push matching jobs to your device. No email, ever.