Senior Data Engineer
New
C
CleraData platforms
Remote, based in Latin America.Full-TimeSenior
Salary80,000 - 120,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 5 years building and operating production data platforms
- Required Skills
- PostgreSQLPythonSQLApache AirflowGCPdbt
Requirements
- Have at least 5 years of experience building and operating production data platforms.
- Have experience owning data platforms from architecture through deployment and ongoing operations.
- Have production experience with Google Cloud Platform and dbt.
- Have production experience with workflow orchestration tools such as Airflow, Dagster, Prefect, or Temporal.
- Have strong Python skills, including asynchronous HTTP and concurrent processing with asyncio, aiohttp, or HTTPX.
- Have strong SQL and PostgreSQL skills, including data modeling, query optimization, and incremental loading.
- Have experience with event-driven systems, messaging or task queues, asynchronous processing, and failure recovery.
- Have experience with data contracts, raw and curated storage, retention policies, automated testing, and data quality monitoring.
- Startup or small-team experience is valuable.
- Web scraping, entity resolution, data enrichment, and LLM-based extraction are useful.
Responsibilities
- Design and build a multi-source data platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.
- Ingest data from APIs, webhooks, files, and web scraping, with rate limits and failure handling.
- Orchestrate scheduled workflows and historical backfills using tools such as Airflow, Dagster, or Prefect.
- Build messaging and task queue workflows with retries, dead-letter queues, and controlled replay.
- Store original payloads in cloud object storage and transform data with dbt into raw, standardized, and curated layers.
- Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures.
- Design and optimize PostgreSQL schemas and loading patterns.
- Integrate enrichment and LLM-based structured extraction.
View Full Description & ApplyYou'll be redirected to the employer's site
