Lead Data Engineer
New
S
Smart WorkingProperty Tech AI
India / Hyderabad / Bangalore / Mumbai / Pune / Mohali / Delhi NCRPart-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 7+ years
- Required Skills
- PostgreSQLPythonApache AirflowElasticSearchKafkaData engineeringNosqlPandasSpark
Requirements
- 7+ years of professional experience, with the majority of that experience in dedicated data engineering roles.
- Strong experience designing and building data pipelines and distributed data systems.
- Experience working with relational databases, with PostgreSQL preferred, although MySQL or similar is acceptable.
- Experience working with NoSQL databases.
- Experience with vector databases used in modern AI systems.
- Strong programming experience in Python.
- Demonstrated ability to make and justify architectural decisions, rather than only implementing them.
- Experience building scalable backend systems.
- Experience designing data models and storage architectures.
- Strong understanding of data processing performance and optimisation.
- Experience with data frameworks such as Apache Spark, Apache Airflow, Kafka, and Elasticsearch or OpenSearch.
- Experience with database technologies including MongoDB and vector databases such as Qdrant, Milvus or pgvector.
- Experience with Python data-processing libraries such as Pandas or Polars.
Responsibilities
- Architect and build scalable data pipelines and infrastructure to support AI and product systems.
- Design and maintain data ingestion, transformation and storage architectures for operational and AI workloads.
- Develop and manage batch and real-time data pipelines.
- Build and optimise systems for vector search, retrieval and machine learning data pipelines.
- Ensure data reliability, security and governance across the platform.
- Collaborate with AI and backend engineering teams to support training, inference and product features.
- Implement monitoring, observability and data quality frameworks.
- Optimise the performance of large-scale datasets and query systems.
- Contribute to technical architecture decisions and long-term data strategy.
- Act as the founding data hire, defining culture, standards and the hiring bar for the data function as it scales.
- Partner directly with founders and product leadership to translate data capabilities into product decisions.
View Full Description & ApplyYou'll be redirected to the employer's site
