QA Automation Engineer – Enterprise Data & AI
New
Fully remote opportunity within the United States.Full-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- PythonSQLTableauCI/CDDatabricksAzure DevOpsPySpark
Requirements
- 5+ years of experience in QA automation, SDET, or data validation engineering roles.
- Strong hands-on experience with Databricks, including notebook development and data pipeline validation.
- Advanced proficiency in Python, PySpark, SQL, and data processing workflows.
- Proven experience in data reconciliation and large-scale data validation across enterprise systems.
- Experience building, extending, or maintaining data quality frameworks in complex environments.
- Familiarity with CI/CD pipelines such as Azure DevOps for test integration and execution.
- Strong analytical, debugging, and problem-solving skills with attention to detail.
- Ability to collaborate effectively with data engineers, QA teams, and cross-functional stakeholders.
- Experience with tools such as Azure Purview or Profisee MDM is a plus.
Responsibilities
- Execute and extend automated data validation tests within Databricks using Python, PySpark, SQL, and notebook-based frameworks.
- Validate end-to-end data pipelines, including ingestion, batch and incremental loads, transformations, joins, and business rule accuracy.
- Perform data reconciliation between source systems and target datasets to ensure completeness and consistency.
- Enhance and maintain existing data quality frameworks, including rule sets for accuracy, completeness, and reliability.
- Implement and monitor validation checks, thresholds, alerts, and exception handling mechanisms.
- Develop reusable and scalable automated test scripts aligned with enterprise data testing standards.
- Integrate automated tests into CI/CD pipelines (e.g., Azure DevOps) and ensure reliable execution across environments.
- Support testing activities across QA and staging environments, including defect triage and root cause analysis.
- Collaborate with data engineering and analytics teams to ensure data integrity for reporting and visualization tools such as Tableau.
View Full Description & ApplyYou'll be redirected to the employer's site
