Data Platform Engineer
New
B
Block LabsData & AI
Location: Portugal. Secondary Locations: Albania, Armenia, Bosnia and Herzegovina, Bulgaria, Croatia, Georgia, Spain, Malta, Greece, Ireland, Italy, Kosovo, Montenegro, Romania, Serbia., EU time zone overlap is preferred.Full-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 3+ years
- Required Skills
- SQLApache KafkaClickhouseCI/CDTerraformData modelingPySpark
Requirements
- 3+ years building and operating production data pipelines at scale.
- Expertise in Apache Kafka or Amazon MSK.
- Strong SQL and warehouse engineering skills (ClickHouse, Druid, BigQuery, or Redshift).
- PySpark / Spark Streaming proficiency.
- Data modelling discipline with experience in data contracts and schema governance.
- CI/CD and infrastructure-as-code experience (CloudFormation, Terraform, or CDK).
- Experience with containerised workloads (ECS Fargate or Kubernetes).
- Data quality and observability mindset with tools like Great Expectations.
Responsibilities
- Design, build, and maintain scalable data pipelines using AWS Glue (PySpark), or equivalent orchestration and transformation tools.
- Engineer and optimise the ClickHouse warehouse for sub-second query performance.
- Implement data contracts between back-office and the platform.
- Build the feature-serving layer providing pre-computed features to AI agents at millisecond latency.
- Integrate with third-party databases, back-office APIs, and external systems (CRM, affiliates, acquisition platforms).
- Establish monitoring, alerting, and maintenance procedures including pipeline health checks and anomaly detection.
- Own CI/CD and infrastructure-as-code for data workloads.
View Full Description & ApplyYou'll be redirected to the employer's site
