Lead Data Engineer

About the role

HackerRank helps companies like NVIDIA, Amazon, and Microsoft hire and upskill the next generation of developers based on skills, not pedigree. Our platform is trusted by over 2,500 of the world’s most innovative companies to build strong engineering teams ready for what’s next.

Software has entered an era where humans and AI build side by side. As this shift accelerates, the definition of strong technical talent is changing. We give companies better ways to identify and invest in next-generation skills.

People at HackerRank care deeply about the impact of their work and sweat the small details so our customers can be wildly successful with products they genuinely love to use. We move with urgency and believe great outcomes come from high standards.

About the role

We are seeking a Lead Data Engineer with deep expertise in distributed systems, real-time data processing, and modern data platforms to join our data team. This is a senior technical leadership role where you’ll help shape and stabilize our data architecture, drive business-critical initiatives, and mentor a high-performing team. You’ll be a senior individual contributor at the heart of the data organisation — owning complex platform decisions, collaborating cross-functionally with AI, product, and go-to-market teams, and shipping data-driven features that directly drive revenue. This is a greenfield opportunity to shape the next phase of data at HackerRank.

What you will do

- Architect and implement streaming-first data pipelines using Apache Kafka and real-time processing frameworks.

- Design and operate distributed data systems using Apache Spark and Flink to support high-volume data processing.

- Build and evolve LakeHouse architectures using Apache Iceberg, Apache Hudi, and Delta Lake with columnar file formats like Parquet and ORC.

- Design and implement distributed query and federation capabilities using Trino, enabling unified access across data lakes, warehouses, and heterogeneous data sources.

- Define and manage metadata and catalog architectures using Hive Metastore and other table catalog solutions.

- Build scalable data-serving and analytical layers using StarRocks and cloud data warehouses such as BigQuery, Redshift, and Snowflake.

- Design and maintain robust data models, data marts, and data cubes that power business intelligence, reporting, and self-serve analytics.

- Build the next-gen AI-optimised data layer: clean, structured datasets that power natural language querying and AI add-on features for HackerRank for Work customers.

- Implement agile development best practices including Scrum, sprint planning, and documentation using JIRA, Linear, Notion and Confluence.

- Collaborate with cross-functional teams including engineering, product, and data science to align technical solutions with business goals.

- Act as a mentor and technical leader to other engineers, fostering a culture of excellence, knowledge sharing, and continuous improvement.