Senior Data Engineer
New
3
3PillarAI Transformation
IndiaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- PythonSQLApache AirflowETLSnowflakePandasSpark
Requirements
- 5+ years of experience as a data engineer or in a similar role.
- Advanced SQL skills and expertise in relational database design.
- Experience with cloud data warehouse solutions (e.g., Snowflake, Redshift, BigQuery, Azure Synapse).
- Strong Python programming skills with experience in Pandas, NumPy, and data libraries.
- Experience with Big Data technologies including Spark, MapReduce, Hadoop, or Hive.
- Proficiency in pipeline and workflow management tools like Airflow.
- Experience with data visualization tools such as PowerBI, Tableau, or AWS QuickSight.
- Demonstrated experience with AWS, Azure, or GCP data engineering services.
- Deep knowledge of Azure services (ADF, Azure Synapse Analytics, ADLS Gen2, Azure SQL DB) or AWS equivalents.
- Proven experience working with data lake and data warehouse solutions.
- Ability to work independently and collaboratively in a team environment.
- Strong problem-solving and organizational skills.
Responsibilities
- Understand business requirements to implement effective data solutions.
- Write, test, and maintain code for complex data systems.
- Create and manage versioned data pipelines with robust change management.
- Develop ETL/ELT processes to extract and manipulate data from various sources.
- Design, build, and maintain production-grade batch or real-time data pipelines.
- Automate data workflows including ingestion, aggregation, and processing.
- Implement logging and instrumentation for pipelines and services.
- Ensure data accuracy, integrity, privacy, security, and compliance.
- Train and mentor junior engineers on the team.
View Full Description & ApplyYou'll be redirected to the employer's site
