Senior Data Engineer (AI/ML)
New
I
IDT CorporationData engineering
Listing location: Pune / Bangalore; Workplace type: Remote; Structured job location: Bangalore; PuneFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Excellent English communication skills.
- Experience
- 6-7+ years of experience in building scalable data infrastructure and 1-2 years of hands-on exposure to AI/ML workflows.
- Required Skills
- PythonSQLHadoopKafkaSnowflakeSpark
Requirements
- Bring 6-7+ years of experience building scalable data infrastructure and 1-2 years of hands-on exposure to AI/ML workflows.
- Have hands-on experience with Apache Spark, Hadoop, and Kafka.
- Have experience designing data pipelines that extract from RDBMS, JSON, API, and flat-file sources.
- Demonstrate SQL and PL/SQL programming skills and knowledge of Business Intelligence and data warehouse methodologies.
- Have hands-on experience with relational databases and cloud data warehouses such as Snowflake or Redshift.
- Use Python for data engineering, transformation, advanced data manipulation, and large-scale processing.
- Understand vector databases, RAG architectures, and semantic retrieval workflows.
- Have experience integrating open-source LLM frameworks into model-training, customization, and inference workflows.
- Have experience with cloud platforms such as AWS or Azure Machine Learning for managed LLM deployments.
- Understand software engineering principles, Unix/Linux/Windows operating systems, Agile methodologies, and version control.
- Demonstrate excellent English communication skills.
Responsibilities
- Design, develop, and maintain scalable data pipelines for feature stores, model-training workflows, and real-time inference services.
- Design and optimize ETL/ELT pipelines and data structures in Snowflake or Redshift for Business Intelligence.
- Build workflows to extract, store, and retrieve semantic representations of unstructured data.
- Architect analytics and dashboarding solutions with natural-language queries and AI-backed insights.
- Manage prompt engineering, orchestration flows, and model fine-tuning routines for conversational interfaces.
- Oversee vector data stores and develop indexing methods for retrieval-augmented generation workflows.
- Partner with data stakeholders to gather language-model requirements and develop scalable solutions.
- Document data processes, workflows, and model deployment routines.
View Full Description & ApplyYou'll be redirected to the employer's site
