Software Developer (Content Integration & Data Pipelines)
New
J
JobgetherMedia intelligence
Based in CanadaFull-TimeMiddle
Salary75,000 - 110,000 CAD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- At least three years of professional software development experience
- Required Skills
- SQLGitJavaJavascriptKafkaTypeScriptLinux
Requirements
- Hold a degree in Computer Science or a related field, or have equivalent practical experience.
- Have at least three years of professional software development experience.
- Demonstrate strong TypeScript or JavaScript skills, particularly in content integration, parsing, and data processing.
- Have experience building backend services with Java or a comparable programming language.
- Have hands-on experience developing and operating production web scrapers and crawlers.
- Understand robots.txt, rate limiting, responsible crawling practices, and applicable terms of use.
- Have practical experience with event-driven systems and message-streaming technologies, particularly Kafka.
- Know HTML parsing, content extraction, data transformation, and normalization techniques.
- Have experience designing or maintaining high-volume, fault-tolerant data pipelines.
- Be familiar with SQL and data modeling principles.
- Have experience in Linux environments and with Git and modern software development workflows; familiarity with Maven is expected.
- Understand distributed systems, scalability trade-offs, reliability, and performance optimization.
- Be able to work independently, take responsibility for projects, and investigate complex technical issues.
- Solr experience, including indexing, schema design, and query tuning, is an asset.
- Experience building content ingestion pipelines for news, media, or similar data-intensive applications is strongly preferred.
Responsibilities
- Build and maintain production-ready web scrapers and crawlers that follow responsible crawling practices.
- Implement parsing, extraction, and normalization logic using TypeScript or JavaScript.
- Build Kafka-based services to process, enrich, and route content.
- Integrate third-party content providers, RSS and Atom feeds, and external APIs.
- Design and implement Java-based RESTful services for content processing, integration, and delivery.
- Improve data quality and reliability through error handling, validation, normalization, and fault-tolerant operations.
- Monitor pipeline performance, investigate processing issues, and improve reliability and operating costs.
- Contribute to system design discussions and technical direction for content acquisition and data processing.
- Support content indexing and retrieval workflows using search technologies such as Solr.
- Collaborate with technical colleagues and stakeholders, contribute to code reviews, and support junior developers.
View Full Description & ApplyYou'll be redirected to the employer's site
