Data Engineer, AWS Glue
New
N
NTT DATAData engineering
LATAM (including Mexico, Brazil, Chile, Peru)ContractSenior
Salary22 - 32 USD per hour
Apply NowOpens the employer's application page
Job Details
- Languages
- Must have working proficiency in English and Spanish, both written and spoken.
- Experience
- 5+ years of data engineering experience.
- Required Skills
- PythonSQLSparkPySpark
Requirements
- Have 5+ years of data engineering experience.
- Have working proficiency in written and spoken English and Spanish.
- Have hands-on experience building ETL pipelines with AWS Glue and PySpark.
- Have experience moving data from databases and files into Amazon S3.
- Have strong knowledge of Python, SQL, Spark, and data transformation.
- Have experience designing reliable, scalable, and restartable data pipelines.
- Understand S3 data lakes and layered data architectures.
- Have experience with AWS workflow, monitoring, and data-quality tools.
- Have experience supporting production pipelines and troubleshooting failures.
Responsibilities
- Design, build, and maintain ingestion pipelines from relational, file, API, and streaming sources into Amazon S3.
- Define and maintain landing and raw-zone layouts, including partitioning, file formats, naming, compression, retention, and immutability.
- Onboard source systems, covering connectivity, credentials, extraction strategy, volume analysis, and coordination with source owners.
- Reconcile loads against source data and manage control tables, reprocessing, and failed or partial runs.
- Implement cleansing, standardization, deduplication, conformance, and business rules across medallion architecture layers.
- Maintain datasets in the Glue Data Catalog and align schemas, partitions, and metadata with Athena, Redshift, and Lake Formation.
- Tune job performance and cost by monitoring DPU consumption, execution times, and file layouts.
- Embed data quality controls, manage rejected records, and publish quality metrics to stakeholders.
- Orchestrate pipelines with Glue Workflows, Step Functions, and EventBridge, including dependencies, alerting, and recovery.
- Provide L2 and L3 production support, investigate incidents, perform backfills, and improve development standards and promotion procedures.
View Full Description & ApplyYou'll be redirected to the employer's site
