Senior Software Engineer I (Data Pipelines)
New
W
WpromoteData engineering
This position may be performed remotely in most states within the US, with some exclusionsFull-TimeSenior
Salary135,000 - 155,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- Minimum 7+ years of experience in software development
- Required Skills
- PostgreSQLPythonSQLApache AirflowGCPKubernetesTerraformBigQuerydbt
Requirements
- Minimum 7+ years of experience in software development.
- Extensive experience with Python, SQL, and data pipelines.
- Expertise building and automating ETL/ELT pipelines using Airflow and DAG-based workflow management software.
- Deep experience managing dbt models using macros and dbt tests.
- Experience with API integrations and software architecture.
- Deep experience with PostgreSQL and BigQuery, including query optimization for performance and cost.
- Experience with Google Cloud Platform, Kubernetes, and infrastructure as code using Terraform.
- Proficiency with Parquet, Avro, Hive, JSONL, and data integration techniques.
- Familiarity with version control systems and CI/CD tools such as GitHub Actions.
- Strong command of agile methodologies, continuous integration, and test-driven development.
- Understanding of business intelligence tools such as Looker or comparable alternatives.
- Ability to influence and guide cross-functional teams and projects.
Responsibilities
- Act as a technical leader and subject matter expert in data pipeline automation and workflow orchestration.
- Design, implement, and maintain reliable data solutions using Airflow, dbt, and Google Cloud data products.
- Manage dbt models using macros and DRY patterns.
- Support QA in developing reliable testing and continuous integration and delivery processes.
- Monitor, troubleshoot, and optimize data pipeline and workflow performance.
- Architect scalable solutions and reusable patterns.
- Provide implementation, configuration, and deployment documentation.
- Participate in agile ceremonies, including daily standups and sprint planning.
- Mentor engineers and guide cross-functional teams and projects.
- Ensure data security, governance, and compliance with relevant standards and privacy restrictions.
View Full Description & ApplyYou'll be redirected to the employer's site
