Data Engineer
A
AssetWatch, Inc.Software Development
United States, Ontario, Canada, EST, PSTFull-TimeJunior
Salary not disclosed
Job Details
- Languages
- English
- Experience
- 2–4 years
- Required Skills
- AWSPythonSQLApache AirflowDynamoDBETLData engineeringTerraformData modeling
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Data Science, or related field.
- 2–4 years of experience in a data engineering role or similar environment.
- Hands-on experience building ETL/ELT jobs with AWS services (Glue, Lambda, Step Functions).
- Experience with orchestration tools like Airflow.
- Strong proficiency in Python and SQL.
- Experience working with Redshift Serverless, Aurora MySQL, Timestream, DynamoDB, or similar databases.
- Comfortable designing and managing S3-based data lakes with structured zone patterns.
- Experience implementing infrastructure-as-code using Terraform.
- Solid understanding of data modeling principles and performance optimization.
- Detail-oriented with a focus on quality, reliability, and maintainability.
- Excellent problem-solving abilities, organizational skills, and ability to manage multiple priorities.
- Strong communication skills, both written and verbal.
- Proven experience designing and implementing backend solutions in complex, scalable cloud environments.
- A proactive learner, eager to explore new technologies and methodologies.
- Comfortable in dynamic, collaborative environments, able to work independently and in teams.
Responsibilities
- Translate data models and product requirements into scalable data solutions.
- Develop and maintain ETL/ELT pipelines for data ingestion, processing, validation, and loading across AWS.
- Build and manage data workflows using AWS Glue, Lambda, and Step Functions.
- Work within structured S3 Raw, Curated, and Consumption zones.
- Participate in establishing data standards, naming conventions, and metadata practices.
- Incorporate AI-assisted development tools for data scrubbing, anomaly detection, and transformation.
- Use Terraform to define and manage data infrastructure.
- Troubleshoot data issues, performance bottlenecks, and pipeline failures.
- Implement data validation, monitoring, and quality checks.
- Load, maintain, and optimize datasets in Redshift Serverless, Aurora MySQL, DynamoDB, and Timestream.
- Write and optimize performant SQL queries, stored procedures, and database schemas.
- Monitor, manage, and optimize alerting systems (Sentry, Slack integrations).
- Create, manage, and improve infrastructure-as-code scripts and Terraform templates.
- Collaborate with Data Team to mature data practices and integration pipeline strategy.
- Collaborate with Product and Engineering Teams to implement system improvements.
- Participate in managing and coordinating production deployments and production support.
- Conduct code reviews, support Engineering Teams with backend best practices, and maintain documentation.
