Sr. Data Engineer

Posted 8 months agoInactiveViewed
140000 - 175000 USD per year
United StatesFull-TimeSoftware Development
Company:InMarket
Location:United States
Languages:English
Seniority level:Senior, 4+ years
Experience:4+ years
Skills:
DockerPythonSQLApache AirflowETLGitHadoopKubernetesData engineeringSparkCI/CD
Requirements:
Strong SQL experience Expert in at least one data pipelining orchestration framework (Airflow, Luigi, etc.) Experience building ETL pipelines Strong software engineering skills in an Object Oriented language such as Java or Python Experience building and optimizing a data warehouse on a major cloud platform (Big Query preferred but not required) Experience and deep understanding of Big Data Technologies such as Hadoop and Spark Experience with building and optimizing large scale and high-performance systems Extensive knowledge of data related tools and architecture concepts in a major cloud platform Knowledge of Kubernetes, Docker, Airflow, Git, and CD/CI best practices Strong collaboration and communication skills within and across teams B.S. or M.S. in Computer Science or a related technical field At least 4 years of software engineering experience, ideally in data engineering
Responsibilities:
Design and implement ETL pipelines in Apache Airflow, Big Query, Python, and Spark Stay up to date with and advocate internally for Data Engineering best practices, technologies, and testing frameworks Architect and plan projects involving highly distributed and high performance data platform systems Provide technical guidance to other engineers Communicate complex concepts and the results of the analyses in a clear and effective manner Identify areas for improvement in existing pipelines and processes Understand business requirements and convert them to technical solutions
Similar Jobs:
Posted 2 minutes ago
United StatesFull-TimeMental Health Tech
Staff AI Product Engineer
Posted 14 minutes ago
United StatesFull-TimeSoftware Development
Senior MuleSoft Engineer
Company:Green Irony
Posted 25 minutes ago
United States, CanadaFull-TimeHealthcare Technology
Principal DevOps Engineer