Senior Software Engineer II (Data Pipelines)

New

WpromoteData Engineering

Remote, United StatesFull-TimeSenior

Salary155,000 - 180,000 USD per year

Apply NowOpens the employer's application page

Job Details

Experience: 7+ years
Required Skills: PostgreSQLPythonSQLGCPKubernetesAirflowCI/CDTerraformBigQuerydbt

Minimum 7+ years of experience in software development, with extensive experience in Python, SQL, and data pipelines.
Expertise in building and automating ETL/ELT pipelines using Airflow and DAG-based workflow management software.
Deep experience managing dbt models using macros and dbt tests.
Mastery of Python or comparable scripting language, API integrations, and software architecture.
Deep experience with databases like PostgreSQL and BigQuery, including query optimization.
Experience with Google Cloud Platform, Kubernetes, and infrastructure as code using Terraform.
Proficiency with advanced data formats (Parquet, Avro, Hive, JSONL) and data integration techniques.
Familiarity with version control systems and CI/CD tools like GitHub Actions.
Strong command of agile methodologies, continuous integration, and test-driven development.
Proven experience in technical leadership and influencing cross-functional teams.

Act as a technical leader and Subject Matter Expert (SME) in data pipeline automation and workflow orchestration.
Design, implement, and maintain complex, reliable data solutions using Airflow, dbt, and Google Cloud data products.
Manage a large portfolio of dbt models, leveraging macros and DRY patterns.
Advocate for test-driven development and assist in developing a robust CI/CD process.
Monitor, troubleshoot, and optimize the performance of data pipelines and workflows.
Architect solutions and reusable patterns that scale with business needs.
Mentor other engineers and foster a culture of technical excellence.
Ensure data security, governance, and compliance.

View Full Description & ApplyYou'll be redirected to the employer's site