Data Engineer Lead - Databricks

About the role

🚀 Join Our Data Products and Machine Learning Development Remote Startup! 🚀

Muttdata is a dynamic startup committed to crafting innovative systems using cutting-edge Big Data and Machine Learning technologies.

We’re looking for a Data Engineer Lead to help take our expertise to the next level. If you consider yourself a data nerd like us, we’d love to connect! 🐶🚀

🚀 What We Do - Leveraging our expertise, we build modern Machine Learning systems for demand planning and budget forecasting.

- Developing scalable data infrastructures, we enhance high-level decision-making, tailored to each client.

- Offering comprehensive Data Engineering and custom AI solutions, we optimize cloud-based systems.

- Using Generative AI, we help e-commerce platforms and retailers create higher-quality ads, faster.

- Building deep learning models, we enhance visual recognition and automation for various industries, improving product categorization, quality control, and information retrieval.

- Developing recommendation models, we personalize user experiences in e-commerce, streaming, and digital platforms, driving engagement and conversions.

🌟 Our Partnerships - Amazon Web Services

- Astronomer

- Databricks

🌟 Our Values - 📊 We are Data Nerds

- 🤗 We are Open Team Players

- 🚀 We Take Ownership

- 🌟 We Have a Positive Mindset

🔍 Curious about what we’re up to? Check out our case studies and dive into our blog post to learn more about our culture and the exciting projects we’re working on! 🚀

Responsibilities 🤓 - Lead end-to-end data migration initiatives, ensuring a smooth, secure, and business-aligned transition.

- Define and communicate data architecture decisions, evaluating impacts, risks, and performance considerations across cloud data platforms such as AWS Redshift, Databricks, and related technologies.

- Coordinate the discovery and documentation of dependencies, schema inventories, data flows, and critical processes involved in migration efforts.

- Own migration and cutover planning, ensuring all activities, validations, and contingency measures are properly defined and executed.

- Serve as the primary liaison between engineering, analytics, product, and business stakeholders, facilitating clear communication and alignment throughout the project lifecycle.

- Translate technical changes in data pipelines, data warehouses, and platform architecture into clear business impact assessments.

- Define and document assumptions, requirements, and acceptance criteria to ensure successful migration outcomes.

- Oversee data validation processes to guarantee the accuracy, consistency, and integrity of migrated datasets against source-of-truth systems.

- Proactively identify, assess, and mitigate migration-related risks, escalating blockers and dependencies when necessary to maintain project timelines and quality standards.