GCP Data Engineer- Lead Consultant
New
Listing location: IndiaFull-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 7+ years of experience as a Data Engineer working with GCP cloud-based infrastructure & systems.
- Required Skills
- PythonSQLETLGCPGitJiraAirflowData engineeringNosqlData modelingConfluenceBigQuerydbt
Requirements
- At least 7+ years of experience as a Data Engineer working with GCP cloud-based infrastructure & systems.
- Deep knowledge of Google Cloud Platform and cloud computing services.
- Extensive experience in design, build, and deploy data pipelines in the cloud, to ingest data from various sources like databases, APIs or streaming platforms.
- Proficient in database management systems such as SQL (Big Query is a must), NoSQL.
- Programming skills (SQL, Python, other scripting).
- Proficient in data modeling techniques and database optimization. Knowledge of query optimization, indexing, and performance tuning.
- Knowledge of at least one orchestration and scheduling tool (Airflow is a must).
- Experience with data integration tools and techniques, such as ETL and ELT.
- Knowledge of modern data transformation tools (such as DBT, Dataform).
- Excellent communication skills to effectively collaborate with cross-functional teams and convey technical concepts to non-technical stakeholders.
- Ability to actively participate/lead discussions with clients to identify and assess concrete and ambitious avenues for improvement.
- Tools knowledge: Git, Jira, Confluence, etc.
- Open to learn new technologies and solutions.
- Experience in multinational environment and distributed teams.
Responsibilities
- Design, model and develop the whole GCP data ecosystem for one of our Client’s (Cloud Storage, Cloud Functions, BigQuery).
- Gather, analyze, model, and document business/technical requirements, with direct client contact.
- Model data from various sources and technologies.
- Troubleshoot and support complex and high impact problems to deliver new features and functionalities.
- Design and optimize data storage architectures, including data lakes, data warehouses, or distributed file systems.
- Implement techniques like partitioning, compression, or indexing to optimize data storage and retrieval.
- Identify and resolve bottlenecks, tune queries, and implement caching strategies to enhance data retrieval speed and overall system efficiency.
- Identify and resolve issues related to data processing, storage, or infrastructure.
- Monitor system performance, identify anomalies, and conduct root cause analysis.
- Train and mentor less experienced data engineers, providing guidance and knowledge transfer.
View Full Description & ApplyYou'll be redirected to the employer's site
