Senior Data Engineer
New
T
TradewebData engineering
Locations: Warszawa, Żelazna 51, Warszawa, Country code: PL, NYC Time Zone (Minimum 5h overlap required); At least 5-6 hours overlap starting from 9am US Eastern.ContractSenior
Salary40 - 60 USD per hour
Apply NowOpens the employer's application page
Job Details
- Required Skills
- AWSPythonSQLETLGCPKubernetesApache KafkaSparkLinux
Requirements
- Demonstrate expert-level SQL and experience with databases or persistence technologies such as MySQL, PostgreSQL, SQL Server, Snowflake, Redis, or Presto.
- Have strong Python skills and familiarity with related ecosystems such as conda or pip.
- Have experience building ETL and stream-processing pipelines using Kafka, Spark, Flink, Airflow, or Prefect.
- Have production experience with AWS or GCP, Kubernetes, and Linux.
- Have experience with parallel and distributed computing.
- Have production operations and support experience.
- Have experience maintaining system stability and responsibly managing releases.
- Demonstrate attention to detail and a focus on data precision and quality.
- Be able to overlap with US Eastern working hours for at least 5–6 hours starting from 9am US Eastern.
Responsibilities
- Build and run Tradeweb’s data platform using AWS, GCP, Kafka, Spark, databases, and containers.
- Develop the data platform using open-source software and cloud services.
- Build and run ETL pipelines to onboard data into the platform.
- Define schemas and build DAG processing pipelines.
- Monitor data quality.
- Help develop machine-learning development frameworks and pipelines.
- Manage and run mission-critical production services.
View Full Description & ApplyYou'll be redirected to the employer's site
