Data Engineer, Red Tape Index
L
LabrynthRegulatory Tech
LATAMFull-Time
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PostgreSQLPythonSQLData engineering
Requirements
- Strong Python and SQL skills with production experience designing and managing Postgres schemas and migrations.
- Professional experience with data pipelines using a lakehouse/medallion architecture.
- Expertise in web scraping beyond basic requests, including anti-bot evasion and browser automation.
- Statistical literacy for index methodology (winsorization, normalization, weighting, and sensitivity testing).
- Comfort with modern Python tooling and CI/CD discipline, including typing, linting, and coverage gates.
- Product discovery instincts with the ability to communicate data feasibility and constraints clearly.
- Ability to take an index project end-to-end, from source acquisition to publication.
- Pragmatic generalist approach to navigating data, backend, and infrastructure decisions.
Responsibilities
- Ship scrapers and ingestion flows resilient to hostile sources using HTTP/2 clients, TLS-fingerprint evasion, and browser automation.
- Own end-to-end Postgres schema design and migrations across per-country and per-domain schemas.
- Build and maintain idempotent, content-hashed, and lineage-tracked medallion (bronze to silver to gold) transforms.
- Implement and defend index methodology including normalization, weighting, and composite construction with 100% test coverage.
- Assess data feasibility and convert ambiguous index requirements into executable plans.
- Operate and monitor data pipelines using orchestration tools and ECS Fargate tasks.
View Full Description & ApplyYou'll be redirected to the employer's site
