Senior Data Engineer, AI applications

View full description

Requirements:

• Significant experience in software development using Python in a high-performance environment.
• Hands-on experience with Spark and PySpark.
• Experience with data modeling and handling both structured and unstructured data.
• Experience with Apache Kafka.
• Experience with cloud environments and distributed software system design.
• Excellent problem-solving and communication skills.

Responsibilities:

• Design, develop, and maintain scalable data processing pipelines.
• Develop and maintain microservices in Python for production.
• Create internal tools for CI/CD pipelines, experiment tracking, and data versioning.
• Ensure data integrity through data quality processes.