Sr. Data Engineer (Poland)

View full description

Requirements:

• 2+ years of experience in Data Engineering.
• 2+ years of experience with Python.
• Experience in developing, maintaining, and ensuring the reliability, scalability, fault tolerance and observability of data pipelines in a production environment.
• Strong knowledge of SDLC and solid software engineering practices.
• Knowledge of and experience with Amazon Web Services (AWS) and Databricks.
• Demonstrated curiosity through asking questions, digging into new technologies, and always trying to grow.
• Strong problem solving and the ability to communicate ideas effectively.
• Familiar with infrastructure-as-code approach.
• Self-starter, independent, likes to take initiative.
• Have fundamental knowledge of data engineering techniques: ETL/ELT, batch and streaming, DWH, Data Lakes, distributed processing.
• Familiarity with at least some technologies in our current tech stack: Python, PySpark, Pandas, SQL (PostgreSQL), Airflow, Docker, Databricks & AWS (S3, Batch, Athena, RDS, DynamoDB, Glue, ECS), CircleCI, GitHub, Terraform

Responsibilities:

• Building and optimizing data pipelines (batch and streaming).
• Extracting, analyzing and modeling of rich and diverse datasets.
• Designing software that is easily testable and maintainable.
• Support in setting data strategies and our vision.
• Keep track of emerging technologies and trends in the Data Engineering world, incorporating modern tooling and best practices at Craft.
• Work on extendable data processing systems that allows to add and scale pipelines.
• Applying machine learning techniques such as anomaly detection, clustering, regression classification, summarization to extract value from our data sets.