Data Engineering Lead

About the role

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineering Lead based in the United States.

We are seeking an experienced Data Engineering Lead to drive the design, development, and delivery of scalable cloud-based data solutions. In this role, you will leverage AWS native services, Python, and PySpark to build robust data pipelines and modernize enterprise data platforms. You will play a key role in shaping data architecture, improving data integration, and developing high-performance data products. Working across complex enterprise environments, you will establish engineering standards and ensure the reliability, quality, and scalability of data solutions. You will also provide technical leadership, mentor engineering teams, and collaborate with architecture, technology, and business stakeholders. This is an opportunity to make a meaningful impact in a flexible, remote-first environment while contributing to large-scale data transformation initiatives.

Accountabilities As a Data Engineering Lead, you will oversee technical delivery and guide the development of enterprise-grade data engineering solutions. Your responsibilities will include:

-

Lead the architecture, design, and development of scalable data pipelines using AWS Glue, EMR, S3, Python, and PySpark.

-

Drive enterprise data integration initiatives and the implementation of ETL/ELT workflows.

-

Lead the development of reliable, scalable, and reusable data products.

-

Define and maintain engineering standards, best practices, and development guidelines.

-

Conduct technical design reviews and code reviews to ensure solution quality and consistency.

-

Optimize data pipelines for performance, data quality, processing efficiency, and scalability.

-

Mentor and guide Data Engineers, supporting their technical development and delivery.

-

Collaborate with architecture, technology, and business teams to align data engineering solutions with enterprise objectives.

Requirements

The ideal candidate combines strong hands-on AWS data engineering expertise with proven technical leadership experience. You should demonstrate:

-

Strong experience in AWS data engineering, particularly Amazon S3, AWS Glue, and Amazon EMR.

-

Proficiency in Python, PySpark, SQL, and ETL/ELT development.

-

Expertise in designing, developing, and optimizing large-scale data pipelines.

-

Solid experience with enterprise data integration and complex data engineering environments.

-

Proven ability to provide technical leadership, establish engineering standards, and conduct code and design reviews.

-