Senior Software Engineer- Data Engineering

About the role

The Data Engineering & AI/ML capability serves as the backbone of our organization's data-driven operations. Comprising skilled professionals with expertise in data engineering, database management, and infrastructure architecture, our team is dedicated to designing and maintaining robust systems that ensure the seamless flow and accessibility of data. We specialize in constructing and optimizing data pipelines, implementing efficient storage solutions, and orchestrating the infrastructure necessary to support the company's data-driven initiatives. Our focus extends beyond data management, encompassing the development of scalable and reliable systems that empower organizations to extract valuable insights, make informed decisions, and drive innovation.

We are seeking a talented and motivated Senior Data Engineer to join our dynamic Data Engineering team. As a key member of our engineering team, you will play a crucial role in constructing and optimizing data pipelines, implementing efficient storage solutions, and orchestrating the infrastructure necessary to support our customer’s data-driven initiatives. In addition, as a Senior Engineer you will be expected to lead engineering engagements by collaborating with cross-functional teams including customers, partners, and internal engineers.

- Location: Primarily Remote. Candidates must be based in CA or DC Metro Area for proximity to project and client teams.

- Security Clearance Requirement: Applicants must be US citizens and eligible to obtain and maintain an active Secret security clearance or above.

Key Responsibilities

Data Collection and Processing

- Acquire, clean, and preprocess diverse datasets from various sources, establishing best practices and ensuring data quality standards.

- Design and build required infrastructure for optimal extraction, transformation and loading of data from various data sources using CSP managed services and SQL technologies, focusing on scalability and cost efficiency.

- Develop, maintain, and optimize mission-critical data pipelines to ensure a continuous flow of high-quality data, implementing robust monitoring and alerting.

Data Migrations & Optimization

- Architect and develop data migration strategies and schemas to lead complex customer migrations from on-prem to cloud technologies

- Oversee and execute high-volume data migration activities, ensuring data integrity and minimal downtime.

- Optimize databases and data warehouses for efficient querying and data storage, implementing advanced partitioning, indexing, and tuning techniques.

Data Analysis and Visualization

- Perform exploratory data analysis to uncover patterns, trends, and insights, providing actionable recommendations to technical and business stakeholders.

- Create visualizations and reports to communicate findings effectively to stakeholders both internally and externally, driving data-driven decision making.