Senior Data Engineer

Capital Technology Group is hiring a Senior Data Engineer to design, build, and maintain scalable data pipelines and systems using modern data engineering practices. You will work alongside other data engineers to evaluate and prototype new tools and technologies, weighing their risks and benefits for federal clients. This role is for someone who likes designing complex systems, is strong in cloud-based solution design, and can balance business needs against technical constraints and long-term strategy.

What you will do

• Design, build, and maintain data pipelines, ETL/ELT workflows, and data models using Python, Apache Spark (PySpark), Databricks, dbt, SQL (PostgreSQL), and AWS Glue.
• Develop and optimize AWS-native data platforms with AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), Lambda, Step Functions, Amazon S3, Redshift, RDS, DMS, and CloudWatch.
• Build ingestion, transformation, and orchestration workflows for structured and semi-structured data using Apache Iceberg, Parquet, ORC, and Avro.
• Design and optimize analytical platforms using Amazon Athena, Trino, Hive, OpenSearch, and enterprise data catalog technologies.
• Integrate enterprise and external data sources across relational and NoSQL platforms including PostgreSQL, Oracle, Redshift, GraphDB, and other NoSQL databases.
• Build AI-enabled data solutions using Amazon Bedrock, RAG pipelines, and vector search technologies including Amazon S3 Vector and OpenSearch vector indexes.
• Develop cloud infrastructure using CloudFormation (Infrastructure as Code), GitHub, Harness, and enterprise CI/CD pipelines, and use SNS, SQS, and EventBridge for event-driven architectures.
• Improve reliability, scalability, performance, and maintainability through monitoring, troubleshooting, automation, and continuous optimization.
• Support mission-critical analytics and reporting in large-scale AWS-based federal data environments, implementing solutions that comply with FedRAMP and NIST 800-53 security controls.
• Lead modernization work migrating legacy platforms including IBM DataStage, Hadoop, RunDeck, and shell-based workflows to cloud-native AWS services.
• Mentor junior engineers through technical guidance, architecture discussions, and code reviews.
• Collaborate with cross-functional teams in an Agile environment, defining requirements and explaining technical concepts to technical and non-technical stakeholders.

What they are looking for

• Bachelor's degree in Computer Science, Engineering, or a related technical field.
• 9+ years of professional experience in data engineering or related domains.
• Hands-on experience with Databricks, Apache Spark (PySpark), Python, SQL (PostgreSQL), and dbt for large-scale data engineering, ETL/ELT development, data transformation, and data modeling.
• Experience designing, building, and maintaining AWS-native data platforms using AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), AWS Lambda, AWS Step Functions, Amazon S3, Amazon Redshift, Amazon RDS, AWS DMS, and Amazon CloudWatch.
• Experience with data pipelines, workflow orchestration, and data integration across enterprise environments.
• Experience with Apache Iceberg and data formats such as Parquet, ORC, and Avro.
• Experience with relational and NoSQL databases including PostgreSQL, Redshift, Oracle, GraphDB, and other NoSQL platforms.
• Experience with performance tuning, system optimization, and enterprise-scale ETL/ELT architectures.
• Java development and modern CI/CD practices using Harness.
• Strong analytical and problem-solving skills, and experience in agile, iterative software development environments.
• Ability to quickly learn and apply new technologies and domain knowledge.
• Excellent written and verbal communication skills.

Nice to have

• Experience supporting analytics, data engineering, or modernization initiatives for financial regulators, capital markets, or other highly regulated environments.
• Kafka for streaming and data pipelines.
• Docker and Kubernetes for containerization and orchestration.
• Splunk for log aggregation and system monitoring.
• Terraform for infrastructure automation and management.
• Strong SQL skills, including performance tuning and complex query design.

Requirements and location

Applicants must be US citizens and able to obtain Public Trust clearance. The role is open in Australia, Canada, Germany, India, the United Kingdom, and the United States. Remote work is standard; hybrid roles will be specified in the job post.

Pay and benefits

The estimated salary range is USD 130,000 to 165,000 per year. The final offer may vary based on experience, skills, and other factors, and the range is not a guarantee and is subject to change.

Full-time employee benefits include remote work, a competitive compensation package, medical, dental, and vision, life insurance, short and long term disability, an employee assistance program, 401(k) with 4% matching, a liberal PTO vacation policy, an annual continuing education budget, an annual wellness budget, and bonus incentive programs for employee referrals and performance-based rewards.

About the company

Capital Technology Group provides consulting services in software development, digital transformation, human-centered design, data analytics and visualization, and cybersecurity. Its teams use agile methodologies to deliver value in close collaboration with clients. For over a decade the company has worked with federal and commercial clients on complex, mission-critical business challenges, and it is part of the Digital Services Coalition. It was recognized by The Washington Post as a Top Workplace in 2025 and 2026. Capital Technology Group is an equal opportunity employer.