caferemote
V4C.ai

Lead Data Engineer, Databricks (Remote, US)

Remote United States Full-time Lead Posted 6h ago
Apply now
Work from
United States
Contract
Full-time
Experience
Lead
Posted
16 November 2026
Apply by
15 January 2027

You will work across batch and streaming, on messy enterprise source systems, under real deadlines, and in front of the customer explaining what you built and why.

About the role

Lead Data Engineers are the technical backbone of V4C delivery. You lead the engineering workstream on an engagement — translating an architecture into working pipelines, setting the patterns the team follows, and owning the quality of what ships. The difference from a Data Engineer is ownership: you are accountable for a workstream and the engineers on it.

Responsibilities

  • Lead engineering on customer engagements, from design through production handover
  • Build and optimize batch and streaming pipelines using PySpark, SQL, and Delta Lake on Databricks
  • Implement Medallion architecture across cloud ecosystems and set the layer conventions the team follows
  • Own performance and cost: benchmark Spark jobs, tune SQL, right-size compute, and explain the trade-offs to the customer
  • Implement data quality and validation as code, including Lakeflow expectations and automated testing in CI
  • Mentor mid-level and junior engineers through code review, pairing, and design feedback
  • Partner with the Data Architect on design decisions, and escalate when implementation reveals the design will not hold

Requirements

  • 7+ years of data engineering plus experience leading projects or teams
  • Strong proficiency in Python, SQL, Apache Spark, and Delta Lake
  • Production experience building both batch and streaming pipelines at enterprise scale
  • Hands-on experience with AWS, Azure, or GCP and its data services
  • Experience with data modeling, Git, and CI/CD for data workloads
  • Demonstrated experience optimizing Spark performance and managing platform cost
  • Strong client communication skills, including the ability to explain technical trade-offs to a non-technical stakeholder

Preferences

  • Healthcare or life sciences experience preferred but not required
  • Existing Databricks Certified Data Engineer Associate or Professional certification
  • Hands-on experience with Unity Catalog, Lakeflow, or Databricks Asset Bundles
  • Experience with dbt, Airflow, or equivalent orchestration and transformation tooling
  • Experience migrating a legacy ETL estate to a modern cloud platform
  • Consulting or client-facing engineering background

Benefits

  • Unlimited PTO
  • Health Care Plan (Medical, Dental & Vision)
  • Retirement Plan (401k, IRA)
  • Life Insurance (Basic, Voluntary & AD&D)
  • Family Leave (Maternity, Paternity)
  • Short Term & Long Term Disability
  • Training & Development

About V4C.ai

V4C is a pure-play Databricks partner, and in under two years we have earned 850+ Databricks certifications across 325+ customers and been named Databricks' 2026 Partner of the Year (C&SI Emerging).

As a Lead Data Engineer, you will lead the build of production Lakehouse pipelines on customer engagements, working directly with customers and often Databricks' own field teams. Every technical hire here gets a certification roadmap built for them and taught in-house by Databricks-approved trainers, plus a dedicated Employee Success Manager whose only responsibility is your experience at V4C.

Discussion

No comments yet. Ask a question or share a tip about this role.

Apply now

More jobs from V4C.ai

View company →

Databricks Architect (Remote, US)

V4C.ai
Remote United States Full-time

About V4C.ai V4C is a pure-play Databricks partner, and in under two years we have earned 850+ Databricks certifications across 325+ customers and been named…

2h ago Data

Databricks Engineer (Remote, US)

V4C.ai
Remote United States Full-time

About V4C V4C is a pure-play Databricks partner, and in under two years we have earned 850+ Databricks certifications across 325+ customers and been named…

10h ago DevOps and Sysadmin

Databricks QA Engineer

V4C.ai
Remote United States Contract $70 – $80 / hour

About V4C V4C is a pure-play Databricks partner, and in under two years we have earned 850+ Databricks certifications across 325+ customers and been named…

12h ago Quality Assurance

Similar remote jobs

Browse all →

Marketing Manager - US Retail

WSAudiology
Remote United States Full-time

WSA is a global leader in the hearing aid industry. Together with our 12,000 colleagues in 130 countries, we invite you to help unlock human potential by bringing back hearing for millions…

2h ago Data

Machine Learning Engineer, Personalization

Bask Health
Remote United States Full-time

About Bask Health Bask Health is a software platform that enables entrepreneurs, doctors, and healthcare businesses to launch, manage, and grow direct-to-consumer telehealth businesses. Our…

2h ago Data

Music Producer - Fully Remote | Upto $13/hr

mercor
Remote United States Contract $13 / hour

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter…

2h ago Data

AI Development Engineer

Proximity Works
Remote India Contract

We are looking for an AI Development Engineer who is passionate about solving complex engineering problems and using AI to improve the speed, quality, and cost efficiency of software…

2h ago Data

Director Technical Product Management - Data Platform

Alignment Health
Remote United States Full-time $172K – $258K / year

Alignment Health is breaking the mold in conventional health care, committed to serving seniors and those who need it most: the chronically ill and frail. It takes an entire team of…

2h ago Data

Sr. BI & Data Analyst | Remote | Canada

Sutherland
Remote Canada Full-time

As a Senior BI & Data Analyst , you’ll be a key driver of insight and innovation within our Enterprise Data and Analytics team. We’re looking for a curious, detail-oriented professional…

2h ago Data

Get remote jobs the moment they are posted

Choose your categories and skills. We push matching jobs to your device. No email, ever.