Backend/Data Engineer

About the role

As a Backend Engineer at Sweetspot, you will join a team dedicated to pushing the boundaries of LLM usage over disparate and unstructured data sources across United States federal and state governments. You will play a critical role in building and integrating data pipelines that feed directly into a framework built for the aforementioned. More broadly, you will join a team that is highly focused on building a product that bolsters the American economy and provides every business, regardless of size, the tools they need to secure government opportunities.

Please note that this is role is only open to US citizens.

Things you should know about us

- We work extensively with industry experts to build software that is useful for our clients. This means frequent customer calls and a constant level of iteration.

- Talent density is far more important than team size. We want to hire the absolute best, and are willing to wait to find the right person.

- If you’re early in your career and/or don’t come from a “prestigious” background, don’t be afraid to apply - we all went to a state school and don’t place a large emphasis on prestige.

- Our office is in Manhattan.

What you’ll do

- Design, develop, and maintain scalable and efficient data pipelines to process and transform large volumes of data from disparate sources across the internet

- Implement and optimize data pipelines using Python/SQLAlchemy and Dagster, ensuring high performance, reliability, and scalability

- Work with PostgreSQL databases, including design, optimization, and troubleshooting

- Collaborate with teammates to build and maintain software that can query hundreds of millions of vectors in a single request

- Participate in code reviews, testing, and deployment processes to ensure code quality and maintainability

- Monitor and troubleshoot data infrastructure to identify and resolve issues proactively

- Stay up-to-date with industry trends and best practices in data engineering and incorporate them into your work

Requirements

- Bachelor's degree in Computer Science, Engineering, or a related field

- Strong proficiency in Python programming language

- Solid experience with PostgreSQL databases, including writing efficient queries, performance tuning, and database administration

- Familiarity with data pipeline frameworks such as Dagster, Airflow, Luigi, or similar

- Knowledge of data processing and transformation techniques, including ETL processes

- Experience with Git

- Experience with Docker

- Excellent problem-solving skills and ability to debug complex systems

- Strong communication and collaboration skills, with the ability to work effectively in a team environment

- US citizen

Preferred qualifications

- Experience with cloud platforms such as AWS, GCP, or Azure