Experience with GIS, POI/Location data ingestion pipeline.
Experience with AWS Stack used for Data engineering EC2, S3, EMR, ECS, Lambda, and Step functions
Hands on experience with Python/Java for orchestration of data pipelines
Experience in writing analytical queries using SQL
Experience in Airflow
Experience in Docker
Responsibilities:
Create and maintain various ingestion pipelines for the GroundTruth platform.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, GIS and AWS ‘big data’ technologies.
Work with stakeholders including the Product, Analytics and Client Services teams to assist with data-related technical issues and support their data infrastructure needs.
Prepare detailed specifications and low-level design.
Participate in code reviews.
Test the product in controlled, real situations before going live.
Maintain the application once it is live.
Contribute ideas to improve the location platform.