Data Platform Engineer

View full description

Requirements:

• 5+ years of experience in software/data engineering, data architecture, or a related field.
• Strong programming skills in at least one language: Java, Scala, Python, or Go.
• Strong SQL and data modeling skills.
• Hands-on experience with Apache Big Data frameworks such as Hadoop, Hive, Spark, Airflow, Iceberg, etc.
• Proficiency in AWS cloud services.
• Strong understanding of distributed systems, large-scale data processing, and data storage/retrieval.
• Experience with data governance, security, and compliance will be a plus.
• Familiarity with CI/CD and DevOps practices will be a plus.
• Excellent problem-solving skills.
• Ability to work independently or as part of a team.
• Strong communication and collaboration skills.

Responsibilities:

• Design, build, and maintain scalable and reliable data storage solutions to support diverse data processing needs.
• Optimize and scale the platform to accommodate increasing data volumes and user requests.
• Improve data storage, retrieval, query performance, and overall system performance.
• Collaborate with data scientists, analysts, and other stakeholders to understand requirements and deliver tailored solutions.
• Work with engineering teams to ensure data pipelines, analytics tools, ETL processes, and other systems are properly integrated with the Lyft Data Platform.
• Troubleshoot and resolve data platform issues in a timely manner.
• Participate in on-call rotations.
• Develop and maintain monitoring and alerting systems to ensure platform availability and reliability.
• Participate in code reviews, design discussions, and other collaborative team activities to maintain high-quality standards.
• Continuously evaluate new technologies and tools to enhance the data platform.
• Contribute to platform documentation, knowledge sharing, and best practice development.