Senior Data Engineer (Remote - US)

View full description

Requirements:

• A bachelor's degree in computer science or information technology plus 8 years minimum of relevant experience.
• High proficiency in programming languages commonly used in ETL development, such as PLSQL, SQL, Python.
• Expertise in utilizing AWS services, including but not limited to Amazon S3, Glue, Data Catalog, Amazon Redshift, Redshift Spectrum, and Amazon Athena.
• Proficiency in working with relational databases such as Postgres, Oracle, MySQL, or SQL Server.
• Experience in performance tuning and optimizing database operations.
• Familiarity with data governance frameworks and data security best practices.
• Passion for learning new technologies, staying up to date with industry trends.

Responsibilities:

• Build, automate, and manage near-real-time scalable data ingestion pipelines for master data management, deep-learning, and predictive analytics.
• Build and maintain cloud native big data environments on AWS that are highly secure, scalable, flexible, and high-performing using SQL, NoSQL, and NewSQL technologies.
• Lead data governance and data profiling efforts to ensure data quality and proper metadata documentation for data lineage.
• Provide technical input into build/buy/partner decisions for all components of the data infrastructure.
• Partner closely with Data Scientists, BI developers, and Product Managers to design and implement data models, database schemas, data structures, and processing logic.
• Design and develop ETL processes to validate and transform data, calculate metrics, and model features using Spark, Python, SQL, and AWS technologies.
• Lead by example, demonstrating best practices for code development and optimization.
• Define SLAs for data availability and correctness and automate monitoring.