Staff Data Scientist - Experimentation & Causal Inference
New
H
HighLevelB2B SaaS, CRM
United StatesFull-TimeStaff
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 9+ years
- Required Skills
- PythonSQLA/B testingR
Requirements
- 9+ years in data science, product analytics, or applied statistics.
- Deep hands-on experience designing and analyzing online controlled experiments at scale.
- Strong applied statistics knowledge, including frequentist foundations, Bayesian methods, power analysis, and variance reduction.
- Deep understanding of A/B testing failure modes, such as peeking, multiple testing, and network/cluster effects.
- Practical experience applying causal inference with sound judgment in distinguishing signal from selection bias and seasonality.
- Proven ability to operate in small-sample, fast-paced, multi-product environments.
- Strong SQL proficiency.
- Working proficiency in Python or R.
- Ability to influence senior leadership and cross-functional partners without direct authority.
- Experience elevating experimentation quality across teams.
Responsibilities
- Define the end-to-end methodology every team follows (hypothesis, metrics, design, power, readout, decision) and establish it as the company default.
- Own statistical approaches like significance testing, sequential testing, and variance reduction (e.g., CUPED) for fast-paced, small-sample environments.
- Develop methods for analyzing clustered, hierarchical data where randomization and analysis units differ.
- Apply rigorous causal inference techniques (matching, diff-in-diff, instrumental variables, synthetic control) to navigate non-experimental scenarios like churn and onboarding.
- Design discipline for managing concurrent experiments, including layering, orthogonal experiments, and holdouts to prevent test contamination.
- Partner with AI/ML teams to design and evaluate experiments for non-deterministic AI features.
- Facilitate an experiment review forum to ensure statistical rigor in product decisions.
- Develop curricula and templates to train PMs and analysts on experiment design principles.
- Collaborate with Analytics Engineering to ensure governed, experiment-ready data and consistent metric definitions.
View Full Description & ApplyYou'll be redirected to the employer's site
