Technical Lead, Data Acquisition

View full description

Requirements:

• Experience in data engineering, including designing and maintaining data pipelines and transformation processes at scale.
• Proficiency in Python and .NET (C#), with a strong foundation in SQL for validation and analysis.
• Experience with data scraping tools like BeautifulSoup, Selenium, or Zyte.
• Experience with data storage technologies like MongoDB.
• Familiarity with scheduling tools such as Hangfire.
• Experience in cloud environments (AWS, GCP, or Azure).
• Leadership experience in mentoring teams alongside technical work.
• Strong project management skills for task prioritization and stakeholder communication.
• Ability to communicate technical concepts to varied audiences.

Responsibilities:

• Design, build, and maintain robust data pipelines that run proprietary ML models and power internal data dashboarding.
• Ensure data integrity and accuracy across systems using SQL, Python, and .NET.
• Manage architecture and performance of data scrapers using Python, Zyte, and MongoDB.
• Oversee backend services built with C#.
• Administer scheduling systems like Hangfire for job orchestration and task automation.
• Coordinate projects and tasks across a distributed team.
• Advocate for team goals and progress with leadership and other departments.