Data Engineer, Data Insights

Posted 4 days agoViewed

💎 Seniority level: Senior, 5+ years

📍 Location: Japan

🔍 Industry: FinTech

⏳ Experience: 5+ years

🪄 Skills: AWSPythonSQLApache AirflowMachine LearningSparkTerraformScala

5+ years experience as a Data Engineer or in a similar role.
Hands-on experience with Apache Hudi, Delta Lake, Spark, and Scala.
Experience designing, building, and operating a DataLake or Data Warehouse.
Strong expertise in AWS services, including Glue, Step Functions, Lambda, and EMR.
Proficiency in Terraform for infrastructure as code (IaC).
Experience with data warehousing tools like AWS Athena, BigQuery, and Databricks.

Design, develop, and maintain scalable data ingestion pipelines using AWS Glue, Step Functions, Lambda, and Terraform.
Optimize and manage large scale data pipelines to ensure high performance, reliability, and efficiency.
Implement data processing workflows using Hudi, Delta Lake, Spark, and Scala.
Maintain and enhance Lakeformation and Glue Data Catalog for effective data management and discovery.
Collaborate with cross-functional teams to ensure seamless data flow and integration across the organization.