AWS Data Engineer
New
S
ScalepexUtilities
US OnlyFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- Minimum of 5 years of experience in data engineering
- Required Skills
- AWSPythonETLData engineeringPandasAWS LambdaDistributed SystemsPySpark
Requirements
- Minimum of 5 years of experience in data engineering.
- Proficiency in AWS services: Step Functions, Lambda, Glue, S3, DynamoDB, and Redshift.
- Strong programming skills in Python with experience using PySpark and Pandas for large-scale data processing.
- Hands-on experience with distributed systems and scalable architectures.
- Knowledge of ETL/ELT processes for integrating diverse datasets into centralized systems.
- Knowledge of data governance practices to ensure accuracy, consistency, and security of data.
- Ability to work independently.
- Ability to work with cross-functional teams and business stakeholders.
- Strong problem solving and troubleshooting skills.
- Excellent teamwork and interpersonal skills.
- Ability to obtain and maintain the required clearance for this role.
Responsibilities
- Design and build scalable, reliable data pipelines using AWS services (e.g., Glue, S3, Redshift) to process large datasets from utility systems.
- Use AWS Step Functions to orchestrate workflows across data pipelines.
- Implement ETL/ELT processes using PySpark, Python, and Pandas to clean, transform, and integrate data from multiple sources.
- Leverage distributed systems to ensure reliability, scalability, and performance in handling large-scale utility data.
- Use AWS Lambda functions to build serverless solutions for automating data processing tasks.
- Design data models tailored for utilities use cases to enable advanced analytics.
- Monitor and optimize pipeline performance to reduce latency and enhance throughput.
- Implement security measures to protect utility data and ensure compliance.
View Full Description & ApplyYou'll be redirected to the employer's site
