Data Engineer

About the role

M9 Solutions is dedicated to providing IT services and solutions to the Federal Government by mobilizing the right people, skills, clearance levels, and technologies to help organizations that desire improved performance and modern, sustainable change. M9 has provided quality IT services and support to more than 30 Federal Agencies and multiple commercial customers nationwide. Our capabilities include IT Talent Solutions, Data Delivery & Analytics, Cyber Security, Cloud Migration, Applications and Infrastructure, Software Development, and Finance & Accounting.

M9 Solutions is seeking a Data Engineer to support a government contract for a client in Washington, DC. An active Top Secret SCI clearance is required.

Responsibilities

- Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.

- Develop and design data pipelines to support an end-to-end solution.

- Develop and maintain artifacts (e.g., schemas, data dictionaries, and transforms related to ETL processes).

- Integrate data pipelines with AWS cloud services to extract meaningful insights.

- Work with stakeholders to support their data infrastructure needs and assist with data-related technical issues.

- Design and develop robust and functional data flows to support raw data and expected data.

- Provide Tier 3 technical support for deployed applications and dataflows.

- Define and communicate a clear product vision for the client’s software products, aligning with user needs and business objectives.

- Create and manage product roadmaps that reflect both innovation and growth strategies.

- Partner with a government product owner and a product team of 7 - 8 full-time employees.

- Collaborate with the rest of the data engineering team to design and launch new features.

- Coordinate and document dataflows, capabilities, etc.

- Occasionally (as needed) support for off-hours deployment, such as evenings or weekends.

Required Skills and Qualifications

- Top Secret SCI security clearance.

- Bachelor's degree or equivalent practical experience.

- Expertise in distributed computing frameworks to handle large-scale data processing.

- Familiarity with working with Amazon Web Services (AWS) or any other cloud environments.

- Working experience with datastores like PostgreSQL, S3, Redshift, MongoDB/DynamoDB, Redis, Elasticsearch/OpenSearch, and SQL.

- Proficient in utilizing Python with key libraries like pandas and PySpark, NiFi, Airflow, AWS Lambda, or similar technologies.

- Working knowledge of software platforms and services, such as Docker, Kubernetes, JMS/SQS, SNS, and Kafka.

- Familiar with Linux/Unix server environments.

- Experience with Agile development methodology.