Principal Data Engineer
New
C
Constant ContactData platforms
Remote, US or Remote, Ontario, CanadaFull-TimePrincipal
Salary137,800 - 172,250 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 12+ years of experience
- Required Skills
- AWSPythonSQLSnowflakeSparkData modelingdbt
Requirements
- Bring 12+ years of experience working with data, including SQL development, shell scripting, Python, Snowflake, AWS, software engineering, or database development.
- Have experience developing end-to-end data pipelines for transforming and processing data for reporting.
- Have experience working with large-scale structured and unstructured data.
- Have production experience with at least two of Trino/Presto, Iceberg/Delta/Hudi, Dagster/Airflow, dbt, and Spark.
- Understand continuous integration and delivery, including branching strategies, automated deployment, and functional automation in CI/CD pipelines.
- Have strong data modeling experience, including dimensional/Kimball modeling, conformed dimensions, slowly changing dimensions, and modeling for multiple brands or tenants.
- Have experience with Kubernetes (EKS), infrastructure as code (Terraform/CDK), and containerized data services.
- Hold a bachelor's or master's degree in computer science, information technology, or equivalent.
- Be able to work in an agile, self-directed team environment.
- Experience leading platform architecture for a multi-team data organization is ideal.
Responsibilities
- Architect large-scale data intelligence solutions and assess options for Snowflake Data Warehouse and/or Google BigQuery.
- Design and evolve an AWS lakehouse using S3, Apache Iceberg or Delta formats, medallion architecture, and Trino.
- Work with IT and business stakeholders to finalize ingestion solutions using tools such as AWS EMR, Snowpipe, and AWS DMS.
- Develop ETL pipelines using tools and languages including Python, Scala, and SnowSQL.
- Own Dagster orchestration patterns, including asset-based pipelines, code locations, lineage, sensors, schedules, partitioning, and backfills.
- Set enterprise dbt standards for model layering, testing, documentation, and CI.
- Tune Trino for performance and cost, including query planning, catalog configuration, resource groups, and Kubernetes scaling.
- Set technical direction and write architecture decision records for peer and leadership review.
- Mentor engineers across distributed teams in the US and India, and conduct code and design reviews.
View Full Description & ApplyYou'll be redirected to the employer's site
