caferemote
SRM Technologies

Senior Data Engineer

Remote India Full-time Senior Posted 3 days ago
Apply now
Work from
India
Contract
Full-time
Experience
Senior
Posted
18 November 2026
Apply by
17 January 2027

Role:Senior Data Engineer

This is a remote position.

About the role

We are looking for aSenior Data Engineerwith strong hands-on expertise inPython, PySpark, Snowflake, dbt, Apache Iceberg, and AWSto design, develop, and maintain scalable enterprise data solutions.

The candidate should have strong experience working with high-volume data processing, cloud-based data platforms, modern lakehouse architectures, ETL/ELT pipelines, data modeling, performance optimization, and production-grade engineering practices.

The ideal candidate should be capable of independently owning complex data-engineering components, contributing to technical design and architecture decisions, troubleshooting production issues, and providing technical guidance to other engineers.

Responsibilities

1. Data Pipeline Engineering

  • Design, develop, test, and maintain scalableETL/ELT data pipelines.
  • Develop production-quality data-processing solutions usingPython and PySpark.
  • Build reusable frameworks and components for ingestion, transformation, validation, and publishing of data.
  • Process large structured, semi-structured, and distributed datasets.
  • Implement incremental and batch-processing patterns where appropriate.

2. Snowflake Development

  • Design and develop scalable data solutions usingSnowflake.
  • Develop complex SQL transformations, data models, views, and reusable data structures.
  • Optimize Snowflake workloads for performance, scalability, and cost.
  • Implement appropriate data-loading and transformation patterns between AWS data platforms and Snowflake.
  • Troubleshoot performance and data-quality issues across Snowflake workloads.

3. dbt Development

  • Build and maintain transformation pipelines usingdbt.
  • Develop modular, reusable, maintainable dbt models.
  • Implement dbt tests and documentation.
  • Follow appropriate development practices for source, staging, intermediate, and business-layer transformations.
  • Support automated deployment and CI/CD practices for dbt projects.

4. Apache Iceberg / Lakehouse

  • Design and implement data-lake and lakehouse solutions usingApache Iceberg.
  • Build scalable table structures for large analytical datasets.
  • Work with partitioning, schema evolution, incremental processing, and table-maintenance strategies.
  • Integrate Iceberg-based datasets with Spark and AWS-based data-processing services.
  • Ensure efficient storage and query patterns for high-volume datasets.

5. AWS Data Engineering

  • Design and implement cloud-native data solutions onAWS.
  • Build data-processing workloads leveraging services such asS3, Glue, EMR and Lambdawhere appropriate.
  • Implement secure access patterns using AWS IAM.
  • Monitor data workloads and troubleshoot operational issues.
  • Participate in designing scalable, reliable, secure, and cost-efficient cloud data architectures.

6. Performance & Scalability

  • Diagnose and optimizeSpark/PySpark jobs, SQL queries, Snowflake workloads, and data pipelines.
  • Identify bottlenecks involving compute, storage, partitioning, data skew, transformations, and queries.
  • Design solutions capable of supporting increasing data volumes without unnecessary infrastructure cost.

7. Data Quality & Governance

  • Implement automated data-quality checks across ingestion and transformation layers.
  • Establish proper logging, monitoring, exception handling, and reconciliation mechanisms.
  • Follow organizational standards for data security, governance, lineage, and access controls.
  • Ensure production pipelines are reliable, auditable, and maintainable.

8. Engineering Best Practices

  • Write clean, modular, reusable, testable, and maintainable code.
  • Perform code reviews and enforce engineering standards.
  • Implement unit, integration, and data-validation testing.
  • Use Git-based version control and CI/CD practices.
  • Create and maintain appropriate technical documentation.

9. Senior-Level Responsibilities

  • Independently drive technically complex data-engineering requirements from design through production deployment.
  • Participate in solution design and architecture discussions.
  • Evaluate alternative implementation approaches and recommend appropriate solutions.
  • Troubleshoot complex production and performance issues.
  • Mentor junior and mid-level data engineers.
  • Collaborate with Architects, Product Owners, Business Analysts, Data Scientists, QA, DevOps, and application teams.
  • Translate business/data requirements into scalable technical solutions.
  • Identify technical risks and proactively recommend improvements.

Requirements

8+ Years

Mandatory/Core:Python, PySpark, Snowflake, dbt, Apache Iceberg, AWS, SQL

Nice to have

Preferred

AWS Glue, S3, EMR, Lambda, Airflow, Snowpipe/Snowpark, CI/CD, Terraform, Data Modeling

Role Type:Senior hands-on Data Engineer

Focus:Cloud Data Engineering, Lakehouse, Data Transformation, Performance Optimization and Production Engineering

Preferred Qualifications

  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related discipline.
  • Strong experience deliveringenterprise-scale cloud data platforms.
  • Experience migrating legacy data workloads to modern AWS/Snowflake architectures.
  • Experience working with very large datasets and distributed processing.
  • Knowledge of data security and governance practices.
  • Experience working in Agile delivery environments.
  • AWS and/or Snowflake certification is an added advantage.

Detailed information

Experience: 8+ years of overall Data Engineering experience, with strong hands-on experience building enterprise-scale cloud data platforms and pipelines.

Primary Skills

  • Python
  • PySpark / Apache Spark
  • Snowflake
  • dbt (Data Build Tool)
  • Apache Iceberg
  • AWS Data Services
  • AdvancedSQL
  • Data Engineering / ETL / ELT
  • Data Lake / Lakehouse architecture

Secondary / Preferred Skills

  • AWS services such as:
    • S3
    • AWS Glue
    • EMR
    • Lambda
    • Step Functions
    • CloudWatch
    • IAM
  • Apache Airflow or other workflow orchestration tools
  • Snowflake performance optimization and cost optimization
  • Snowpipe / Snowpark
  • Spark performance tuning
  • Data modeling and dimensional modeling
  • Parquet and other columnar data formats
  • Data quality frameworks and automated validation
  • CI/CD for data pipelines
  • Git / GitHub / GitLab
  • Infrastructure as Code such as Terraform or AWS CDK
  • Docker / containerization
  • Data governance, lineage, security, and access control
  • Agile/Scrum delivery experience

Core Skills Expected

A strong candidate should demonstratedeep hands-on capability, not merely theoretical exposure, in the following areas:

Area

Expected Capability

Python

Advanced, production-quality data engineering development

PySpark

Large-scale distributed processing, optimization and troubleshooting

Snowflake

Development, modeling, optimization and performance tuning

dbt

Models, tests, macros, documentation and deployment practices

Apache Iceberg

Lakehouse/table design, partitioning, schema evolution and optimization

AWS

Hands-on cloud data platform development

SQL

Advanced SQL, query optimization and analytical processing

Data Engineering

ETL/ELT, batch/incremental pipelines, data quality and orchestration

Data Architecture

Data Lake, Data Warehouse and Lakehouse concepts

Engineering Practices

Git, testing, code reviews, CI/CD and production support

Discussion

No comments yet. Ask a question or share a tip about this role.

Apply now

Similar remote jobs

Browse all →

NEN SMIT Senior Solution Architect

Leidos
Remote United States Full-time $107K – $195K / year

Leidos is currently seeking a Senior Solution Architect to join our SMIT Growth Team. The Senior Solution Architect will be responsible for providing technical leadership and delivering…

1h ago DevOps and Sysadmin

Cloud Architecture Manager

Smart Tech Contracting
Remote United States Full-time $140K – $170K / year

Smart Tech Contracting delivers turnkey BAS & EPMS solutions for mission-critical colocation and hyperscale data centers operating at national program scale. As a Master Systems Integrator…

1h ago DevOps and Sysadmin

Cloud System Administrator (Shift B - Day)

NexusTek
Remote United States Full-time $28 – $31 / hour

Shift Schedule: This is a 12-hour day shift (5:45 AM to 6:15 PM Mountain Time), operating on a 12/7 rotational schedule . The rotation is as follows: Week 1: Work Thursday through Saturday…

3h ago DevOps and Sysadmin

Principal, Software Development

Common Securitization Solutions
Remote United States Full-time $181K – $208K / year

OVERVIEW The Company U.S. Financial Technology (U.S. FinTech) is seeking an experienced Principal, Software Development to join our team of talented professionals. This is a full-time…

4h ago DevOps and Sysadmin

Lead Devops Engineer

Way2B1
Remote United States Full-time

Our Company and the Role This is a hands-on Lead DevOps Engineer role responsible for designing, operating, and evolving a highly available, multi-tenant platform on AWS. You will work…

4h ago DevOps and Sysadmin

Senior Platform Engineer — Kubernetes, DevOps, & Cloud Infra (US)

Genios AI
Remote United States Full-time

We are on a mission to build the AI-native platform for finance —a multi-agent system that reimagines how investment teams make decisions and empowers them to move faster, think deeper,…

5h ago DevOps and Sysadmin

Get remote jobs the moment they are posted

Choose your categories and skills. We push matching jobs to your device. No email, ever.