Data Engineer (AI/NLP)

View full description

Requirements:

• Bachelor or Master’s degree in data science, data engineering, or Computer Science with a focus on math and statistics (Master’s preferred).
• 3+ years experience as a data engineer building data pipelines with Python, Spark, or similar.
• Strong foundation in computer science principles and statistical methods.
• Strong experience with cloud technology (AWS or Azure).
• Experience creating data ingestion pipelines and ET processes.
• Knowledge of big data tools such as Spark, Databricks, and Python.
• Basic understanding of common machine learning techniques.
• Familiarity with Natural Language Processing (NLP) concepts.
• Knowledge of scrum practices and agile mindset.

Responsibilities:

• Cleanse and preprocess raw text data for analysis and modelling purposes.
• Analyze text corpora, including customer support history and the company knowledge base.
• Extract actionable insights to inform project development and decision-making.
• Build and maintain data ingestion pipelines from various sources.
• Ensure data quality, consistency, efficiency, and integrity during processes.
• Assist in indexing and vectorizing text data.
• Collaborate with data scientists on embedding methods.
• Align data engineering efforts with machine learning requirements.