Data Engineer

About the role

Plenty of data engineering roles are about keeping the pipeline alive. This one is about building the thing the whole business runs on.

At Resonate, we help enterprise brands understand not just who their customers are, but why they behave the way they do. That intelligence is only as good as the pipelines underneath it, and those pipelines move terabytes at a time. You will be on the big data team designing, building, and maintaining them - hands-on Spark and Scala work on AWS, at a scale where the usual answers stop working and you have to think properly about the problem.

It is also a team that has genuinely committed to Gen AI as part of how the work gets done, not as a pilot someone runs on the side. If you want to build at real scale and have room to change how the building happens, this is the role.

What you’ll be doing

- Designing, developing, and maintaining the ETL/ELT pipelines that power the Resonate business - Spark and Scala on AWS EMR, working with terabyte-scale datasets across S3 and Snowflake

- Tuning multi-terabyte and petabyte scale Spark applications for performance and cost, and debugging the problems that only ever show up at that size

- Partnering with senior engineers and product management to take product requests from design and planning through to pipelines running in production

- Monitoring pipeline health in Grafana, tracking down data quality issues, and fixing the cause rather than the symptom

- Writing clean, testable code with thorough unit and integration tests, using Gen AI to move faster through the parts that used to slow you down

- Using Gen AI tools across pipeline development, monitoring, and day-to-day engineering work - and helping the team figure out where they genuinely help and where they do not

- Taking part in code reviews, technical design discussions, and sprint planning, with your point of view on architecture actually counting

- Supporting production operations, including an on-call rotation and incident response

What we’re looking for

You’ll need

- Around five years or more of professional experience in software engineering, data engineering, or a closely related field

- At least three years hands-on with Spark and Scala, specifically the DataFrame and Dataset APIs

- Proven experience tuning Spark applications at multi-terabyte or petabyte scale - you have done it, not just read about it

- Real debugging and problem-solving experience in the big data ecosystem, including the jobs that only fail in production

- Solid relational database experience

- Working knowledge of modern cloud stacks for processing big data - AWS (EMR, S3, Lambda), Kafka, Snowflake, Grafana, Hadoop, Elastic Stack, and Docker

- A strong grasp of the full software development lifecycle, from exploration and design through to delivery in production, plus good instincts on solution architecture, data structures, and data modeling