Senior Data Engineer

Posted 4 days agoViewed

💎 Seniority level: Senior, 5+ years

📍 Location: India

🏢 Company: BlackStone eIT👥 251-500 Augmented Reality Robotics Analytics Project Management

⏳ Experience: 5+ years

🪄 Skills: SQLApache HadoopCloud ComputingData AnalysisETLData engineeringRDBMSSparkData visualizationData modelingData management

Requirements:

5+ years of experience in data engineering or a similar role.
Proficiency in SQL and experience with relational databases.
Hands-on experience with data pipeline tools and ETL processes.
Familiarity with big data technologies (e.g., Hadoop, Spark) is a plus.
Experience with cloud-based data solutions is an advantage.

Responsibilities:

Design, implement, and maintain scalable data architectures and pipelines.
Develop ETL processes to facilitate smooth data transfer and transformation from various sources.
Collaborate with data scientists and analysts to fulfill data needs for analytics and reporting.
Optimize database performance and maintain data integrity across systems.
Conduct data quality checks and resolve discrepancies.
Mentor junior data engineers and provide technical guidance.
Stay current with emerging technologies and best practices in data engineering.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer

Posted about 2 months ago

📍 Worldwide

🔍 Event technology

🔧 Requirements

Experience in data engineering and building data pipelines.
Proficiency in programming languages like Python, Java, or Scala.
Familiarity with cloud platforms and data architecture design.

💡 Responsibilities

Design and develop data solutions to enhance the functionality of the platform.
Implement efficient data pipelines and ETL processes.
Collaborate with cross-functional teams to define data requirements.

AWSDockerPostgreSQLPythonSQLApache AirflowCloud ComputingData AnalysisETLKubernetesAlgorithmsApache KafkaData engineeringData StructuresCI/CDRESTful APIsMicroservicesData visualizationData modeling

Posted about 2 months ago

Apply

🔥 Senior Data Engineer

Posted about 2 months ago

📍 South Africa, Mauritius, Kenya, Nigeria

🔍 Technology, Marketplaces

🔧 Requirements

BSc degree in Computer Science, Information Systems, Engineering, or related technical field or equivalent work experience.
3+ years related work experience.
Minimum of 2 years experience building and optimizing ‘big data’ data pipelines, architectures and maintaining data sets.
Experienced in Python.
Experienced in SQL (PostgreSQL, MS SQL).
Experienced in using cloud services: AWS, Azure or GCP.
Proficiency in version control, CI/CD and GitHub.
Understanding/experience in Glue and PySpark highly desirable.
Experience in managing data life cycle.
Proficiency in manipulating, processing and architecting large disconnected data sets for analytical requirements.
Ability to maintain and optimise processes supporting data transformation, data structures, metadata, dependency and workload management.
Good understanding of data management principles - data quality assurance and governance.
Strong analytical skills related to working with unstructured datasets.
Understanding of message queuing, stream processing, and highly scalable ‘big data’ datastores.
Strong attention to detail.
Good communication and interpersonal skills.

💡 Responsibilities

Suggest efficiencies and execute on implementation of internal process improvements in automating manual processes.
Implement enhancements and new features across data systems.
Improve streamline processes within data systems with support from Senior Data Engineer.
Test CI/CD process for optimal data pipelines.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Highly efficient in ETL processes.
Develop and conduct unit tests on data pipelines as well as ensuring data consistency.
Develop and maintain automated monitoring solutions.
Support reporting and analytics infrastructure.
Maintain data quality and data governance as well as upkeep of overall maintenance of data infrastructure systems.
Maintain data warehouse and data lake metadata, data catalogue, and user documentation for internal business users.
Ensure best practice is implemented and maintained on database.

AWSPostgreSQLPythonSQLETLGitCI/CD

Posted about 2 months ago

Apply