Senior Software Engineer I (Data Pipelines)
New
W
WpromoteData engineering
This position may be performed remotely in most states within the US, with some exclusionsFull-TimeSenior
Salary135,000 - 155,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- Minimum 7+ years of experience in software development
- Required Skills
- PostgreSQLPythonSQLGCPKubernetesAirflowTerraformBigQuerydbt
Requirements
- Have a minimum of 7+ years of software development experience.
- Bring extensive experience with Python, SQL, and data pipelines.
- Have expertise building and automating ETL/ELT pipelines with Airflow and DAG-based workflow management software.
- Have deep experience managing dbt models, macros, and dbt tests.
- Have mastery of Python or a comparable scripting language, API integrations, and software architecture.
- Have deep experience with PostgreSQL and BigQuery, including query optimization for performance and cost.
- Have experience with Google Cloud Platform, Kubernetes, and infrastructure as code using Terraform.
- Be proficient with Parquet, Avro, Hive, JSONL, and data integration techniques.
- Have familiarity with version control systems and CI/CD tools such as GitHub Actions.
- Have a strong command of agile methodologies, continuous integration, and test-driven development.
- Bring technical leadership, problem-solving skills, and the ability to guide cross-functional teams and projects.
- Understanding of Looker or comparable business intelligence tools is required; Airbyte and monitoring/logging tools such as Prometheus and Grafana are pluses.
Responsibilities
- Act as a technical leader and subject matter expert in data pipeline automation and workflow orchestration.
- Design, implement, and maintain reliable data solutions focused on automation using Airflow, dbt, and Google Cloud data products.
- Manage dbt models using macros and DRY patterns.
- Advocate for test-driven development and assist QA with continuous integration and delivery processes.
- Monitor, troubleshoot, and optimize data pipeline and workflow performance.
- Architect scalable solutions and reusable patterns.
- Provide implementation, configuration, and deployment documentation.
- Participate in agile ceremonies, including daily standups and sprint planning.
- Mentor engineers and foster technical excellence.
- Ensure data security, governance, and compliance with relevant standards and privacy restrictions.
View Full Description & ApplyYou'll be redirected to the employer's site
