Lead Data Engineer
New
J
JobgetherAI Platform
IndiaContractLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 7+ years
- Required Skills
- PostgreSQLPythonApache AirflowElasticSearchKafkaData engineeringNosqlPandasSpark
Requirements
- 7+ years of professional experience, with significant experience in dedicated data engineering roles and ownership of complex data infrastructure.
- Strong experience designing and building scalable data pipelines and distributed data systems.
- Solid experience with relational databases, preferably PostgreSQL; experience with MySQL or comparable technologies is also relevant.
- Experience working with NoSQL databases and modern vector databases used in AI applications.
- Strong Python programming skills, including experience with data-processing libraries such as Pandas or Polars.
- Demonstrated ability to make, communicate, and justify architectural decisions.
- Experience building scalable backend systems and designing robust data models and storage architectures.
- Strong understanding of data processing performance, scalability, and optimisation.
- Experience with technologies such as Apache Spark, Apache Airflow, Kafka, Elasticsearch, or OpenSearch is highly desirable.
- Experience with PostgreSQL, MongoDB, and vector technologies such as Qdrant, Milvus, or pgvector is highly desirable.
- Strong collaboration and communication skills.
- Experience in AI/ML platforms, event-driven architectures, cloud infrastructure, or high-growth startup environments is a plus.
Responsibilities
- Architect, build, and scale robust data pipelines and infrastructure supporting both AI products and operational applications.
- Design and maintain data ingestion, transformation, processing, and storage architectures for batch and real-time workloads.
- Develop scalable systems for vector search, retrieval, and machine-learning data workflows.
- Build and optimise data models, distributed processing systems, and large-scale query infrastructure.
- Establish frameworks for data reliability, quality, security, governance, monitoring, and observability.
- Collaborate with AI and backend engineering teams to support model training, inference, and data-driven product capabilities.
- Contribute to technical architecture decisions, engineering standards, and the long-term data strategy.
- Partner with founders and product leadership to translate data capabilities into meaningful product and business decisions.
- Take ownership as the founding data specialist, helping define the team’s technical culture, standards, processes, and future hiring requirements.
View Full Description & ApplyYou'll be redirected to the employer's site
