Senior Backend API Data Developer

View full description

Requirements:

• 5+ years experience as a Software Engineer or similar role, with a focus on building data pipelines and infrastructure.
• Hands on experience in Python programming and experience with relevant libraries and frameworks for data processing is required (e.g. Pydantic, MongoDB, FastAPI, Redis, Pandas, NumPy, Spark). - Required
• 3+ years experience with MongoDB and larger data sets is required i.e. Terabyte's - Required
• Strong understanding of database systems, with experience in designing and optimizing queries.
• Hands-on experience with cloud platforms, particularly AWS (Amazon Web Services), and familiarity with services such as S3, ECS, SQS.
• Experience working with large-scale distributed systems and parallel processing frameworks.
• Solid understanding of data modeling concepts and techniques.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills, with the ability to work effectively in a team environment.

Responsibilities:

• Design, build, and maintain scalable data pipelines and infrastructure to support the collection, processing, and analysis of large volumes of data.
• Develop robust ETL processes to extract, transform, and load data from various sources into our data warehouse.
• Optimize data processing workflows for performance, reliability, and scalability.
• Implement data quality monitoring and validation processes to ensure accuracy and consistency of data.
• Work closely with frontend software engineers to integrate data-driven features and functionalities into our products and services.
• Stay abreast of emerging technologies and best practices in data engineering, and propose innovative solutions to enhance our data infrastructure.