Data Engineer

About the role

Are you looking to thrive in a stimulating work environment ? Join the leader in digital transformation and boost your career at Levio. We offer you a unique opportunity to develop your expertise within a high-caliber team. The variety of challenges presented by our clients will give you the opportunity to excel and keep your expertise at its peak. Experience the daily life of our clients and be a significant player in the most ambitious projects in technology and software solutions

POSITION OVERVIEW

We are seeking a Data Engineer to design, build, and support data feeds and integration pipelines from the ground up. The role will involve integrating data from APIs, SFTP files, databases, and other sources; processing and transforming data using Python and SQL; developing Snowflake data pipelines; and deploying containerized workloads in AWS. The ideal candidate is a hands-on data engineer with strong Python, Microsoft SQL Server, and Snowflake experience, along with the ability to troubleshoot production data feeds and work across cloud-based data and integration technologies.

YOUR RESPONSIBILITIES

- Design, develop, and maintain data feeds and integration pipelines from source systems through processing and downstream delivery.

- Build integrations using REST APIs, SFTP, SQL Server, Snowflake, and other data sources.

- Develop Python applications and scripts for data ingestion, transformation, validation, reconciliation, and processing.

- Develop and support Snowflake data pipelines, including incremental data-loading patterns and Snowflake-native capabilities such as stages, Snowpipe, Streams, Tasks, and stored procedures.

- Apply Data Vault modeling principles, including Hubs, Links, Satellites, hash keys, hash differences, and incremental loading.

- Write and troubleshoot T-SQL for data extraction, transformation, validation, reconciliation, and data processing.

- Containerize Python-based data-processing applications using Docker and run them as AWS ECS tasks.

- Monitor and troubleshoot data feeds and pipelines, resolving issues involving APIs, SFTP transfers, data processing, SQL Server, Snowflake, and downstream systems.

- Implement appropriate error handling, logging, monitoring, and retry mechanisms to support reliable production data feeds.

- Collaborate with technical and business stakeholders to understand requirements, resolve data issues, and support end-to-end data integration workflows

- Maintain technical documentation for data pipelines, integrations, data models, and operational procedures..

REQUIRED QUALIFICATIONS

- Production experience developing data feeds, data integrations, or data pipelines.

- Strong Python development experience, including data processing and transformation.

- Strong hands-on experience with Microsoft SQL Server and T-SQL, including complex queries, joins, CTEs, aggregations, data manipulation, views, and stored procedures.