Data Engineer
New
C
CyberMedia TechnologiesInformation Technology
Remote, United StatesFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 7+ years
- Required Skills
- PythonSQLAgileETLMicrosoft Power BICI/CDPySpark
Requirements
- 7+ years of experience in data engineering, ETL development, or large-scale data integration.
- Strong proficiency in SQL and Python with hands-on PySpark experience.
- Strong experience with Azure Databricks environments.
- Experience with cloud-based data platforms and lakehouse architectures (preferably Microsoft Azure).
- Experience implementing layered lakehouse data architectures.
- Experience supporting data migration from legacy databases.
- Experience integrating Databricks with BI tools like Power BI.
- Familiarity with data governance, metadata management, and data security.
- Experience with source control and CI/CD pipelines.
- Working knowledge of SDLC and Agile methodologies.
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or equivalent experience.
Responsibilities
- Design, develop, and maintain scalable ETL pipelines and data workflows.
- Build, optimize, and maintain data processing solutions using Azure Databricks and lakehouse architectures.
- Support phased data migration from legacy databases and ETL tools to Azure Databricks.
- Implement layered lakehouse data architectures (bronze, silver, gold) in Databricks.
- Develop data processing workflows using Python and PySpark.
- Develop data validation, reconciliation, and testing processes.
- Integrate Databricks data platforms with analytics and reporting tools.
- Support data governance initiatives including metadata management and security.
- Maintain source control and CI/CD pipelines for data engineering workflows.
- Provide ongoing operational support, troubleshooting, and mentorship.
View Full Description & ApplyYou'll be redirected to the employer's site
