Mid Level Data Engineer

View full description

Requirements:

• 3+ years in the Data field
• Experience in the construction and optimization of data pipelines, architectures, and 'big data' datasets
• Proficiency with Apache Spark with a general-purpose programming language such as Python or Scala
• Ability to create processes supporting data transformation, data structures, metadata, dependency, and workload management
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL databases
• Experience with data pipeline and workflow management tools

Responsibilities:

• Be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams;
• Create and maintain optimal data pipeline architecture;
• Assemble large, complex data sets that meet business requirements;
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc;
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
• Work with data and analytics experts to strive for greater functionality in our data systems.