Data Engineer

Posted 2024-11-14

💎 Seniority level: Entry, minimum of 2 years of related experience, or an advanced degree without experience, or equivalent work experience

📍 Location: Canada

🔍 Industry: Automotive industry

🏢 Company: J.D. Power

🗣️ Languages: English

⏳ Experience: Minimum of 2 years of related experience, or an advanced degree without experience, or equivalent work experience

🪄 Skills: PythonSQLData AnalysisGCPData analysisData engineeringCommunication SkillsAnalytical SkillsCollaborationMicrosoft ExcelProblem SolvingAttention to detailWritten communication

Requirements:

Basic understanding of data cleaning and quality assurance techniques
Familiarity with SAS, SQL, or basic data manipulation in Python, PySpark, and R
Exposure to basic data joining and pipeline building concepts, including tools like GCP Dataflow and Palantir
Knowledge of basic Big Data technologies
Basic knowledge of Excel for data processing tasks
Strong communication, critical thinking, and problem-solving skills
Knowledge of statistical analysis is desired
Ability and willingness to learn in a fast-paced environment
Resilient work ethic
Bachelor's degree and a minimum of 2 years of related experience, or an advanced degree without experience, or equivalent work experience

Responsibilities:

Daily immersion in automotive industry data
Work on problems of moderate scope requiring analysis of various factors
Exercise judgment within defined procedures to determine appropriate action
Involve in basic data processing and cleaning tasks
Focus on foundational skills development in data operations
Build productive internal and external working relationships

Apply

Related Jobs

Apply

🔥 Cloud Data Engineer (AWS) (Remote, Canada)

Posted 2024-11-21

📍 Canada

🔍 Artificial Intelligence

Strong background in AWS DevOps and data engineering.
Expertise with AWS and SageMaker is essential.
Experience with Snowflake for analytics and data warehousing is highly desirable.

Manage and optimize the data infrastructure.
Focus on both data engineering and DevOps responsibilities.
Deploy machine learning models to AWS using SageMaker.

AWSMachine LearningSnowflakeData engineeringDevOps

Posted 2024-11-21

Apply

🔥 Junior Data Engineer (V)

Posted 2024-11-15

📍 Latin America, United States, Canada

🔍 Life insurance

The ideal candidate will be independent and a great communicator.
Attention to detail is critical.
Must possess problem-solving skills.

Develop and maintain enterprise data and analytics systems for a US client.
Optimize performance by building and supporting decision-making tools.
Collaborate closely with software engineering, AI/ML, Cybersecurity, and DevOps/SysOps teams.
Support end-to-end data pipelines using Python or Scala.
Participate in Agile framework-related tasks.

PythonSoftware DevelopmentAgileData engineeringDevOpsAttention to detail

Posted 2024-11-15

Apply

🔥 Data Engineer, Blockchain

Posted 2024-11-15

📍 North America, South America, Europe

💸 100000 - 500000 USD per year

🔍 Web3, blockchain

🏢 Company: Edge & Node

A self-motivated, team member with keen attention to detail.
Proactive collaboration with team members and a willingness to adapt to a growing environment.
Familiarity and experience with Rust, particularly focusing on data transformation and ingestion.
A strong understanding of blockchain data structures and ingestion interfaces.
Experience in real-time data handling, including knowledge of reorg handling.
Familiarity with blockchain clients like Geth and Reth is a plus.
Adaptability to a dynamic and fully-remote work environment.
Rigorous approach to software development that reflects a commitment to excellence.

Develop and maintain data ingestion adapters for various blockchain networks and web3 protocols.
Implement data ingestion strategies for both historical and recent data.
Apply strategies for handling block reorgs.
Optimize the latency of block ingestion at the chain head.
Write interfaces with file storage protocols such as IPFS and Arweave.
Collaborate with upstream data sources, such as chain clients and tracing frameworks, and monitor the latest upstream developments.
Perform data quality checks, cross-checking data across multiple sources and investigating any discrepancies that arise.

Software DevelopmentBlockchainData StructuresRustCollaborationAttention to detail

Posted 2024-11-15

Apply

🔥 Data Engineer, Cyber

Posted 2024-11-12

📍 Quebec

🔍 Cybersecurity

🏢 Company: Qohash

4+ years of experience in data warehousing and ETL development.
3+ years of experience with Apache Spark or Databricks.
2+ years of experience building, deploying, and optimizing data-ingestion solutions.
Experience working directly with external and internal stakeholders with good communication skills.
A bachelor's or master's degree in computer science or engineering or equivalent professional experience.

Analyze, design, and implement features.
Write well-designed and efficient code, including unit and integrated tests.
Correct anomalies and problems as they arise.
Test features to ensure acceptance criteria are met.
Participate in projects from initial idea to launch.
Advise the product owner on technological choices and development decisions.
Contribute to continuous improvement in development activities such as agility and automated testing.
Collaborate with the security team to integrate security best practices into software processes.
Communicate risks related to activities, technologies, or processes as they are identified.

Software DevelopmentETLSparkCommunication SkillsTime ManagementMicroservices

Posted 2024-11-12

Apply

🔥 Lead Data Engineer

Posted 2024-11-07

📍 North America, Latin America, Europe

🔍 Data consulting

Bachelor’s degree in engineering, computer science or equivalent area.
5+ years in related technical roles such as data management, database development, and ETL.
Expertise in evaluating and integrating data ingestion technologies.
Experience in designing and developing data warehouses with various platforms.
Proficiency in building ETL/ELT ingestion pipelines with tools like DataStage or Informatica.
Cloud experience on AWS; Azure and GCP experience is a plus.
Proficiency in Python scripting; Scala is required.

Designing and developing Snowflake Data Cloud solutions.
Creating data ingestion pipelines and working on data architecture.
Ensuring data governance and security throughout customer projects.
Leading technical teams and collaborating with clients on data initiatives.

AWSLeadershipPythonSQLAgileETLOracleSnowflakeData engineeringSparkCollaboration

Posted 2024-11-07

Apply

🔥 Staff Data Engineer

Posted 2024-11-07

📍 Canada, United States, United Kingdom

🔍 Smart home technology

🏢 Company: ecobee

10+ years of experience in data/software engineering with proven track record.
Extensive experience in building and maintaining scalable data pipelines with tools like Apache Spark, Kafka, and Airflow.
Expertise in cloud data platforms (AWS, GCP, or Azure), focusing on distributed systems.
Solid understanding of end-to-end data systems and machine learning deployment.
Knowledge in data security, governance, and compliance.
Experience in data architecture and engineering methodologies across industries.
Experience with real-time data processing and analytics platforms.
Proven ability to mentor and guide engineers across teams.

Lead the design and implementation of scalable data pipelines and systems for complex problems.
Contribute to ecobee’s system architecture with impactful designs.
Take end-to-end ownership of components within your domain.
Define and track SLAs for components to ensure reliability.
Mentor engineers and drive knowledge-sharing initiatives.
Collaborate across squads to align technical discussions with strategic goals.
Anticipate future data challenges and propose strategies.
Evaluate and recommend new technologies for data engineering.

AWSLeadershipGCPIoTKafkaMachine LearningStrategyAirflowAzureData engineeringSparkCollaboration

Posted 2024-11-07

Apply

🔥 Senior Data Engineer

Posted 2024-11-07

📍 Canada, UK, US

🔍 Smart home technology

🏢 Company: ecobee

Proficiency in building data pipelines using Python and SQL.
Experience with Apache Spark, Apache Kafka, and Apache Airflow.
Experience with cloud-based data platforms, preferably GCP.
Familiarity with SQL-based operational databases.
Good understanding of machine learning lifecycle.
Strong experience in data modeling and schema design.
Experience with both batch and real-time data processing.
Excellent communication skills for collaborative work.

Design, build, and maintain scalable and efficient ETL/ELT pipelines.
Implement data extraction and processing solutions for analytics and machine learning.
Integrate diverse data sources into centralized data repositories.
Develop and maintain data warehousing solutions.
Monitor and optimize data workflows for performance and reliability.
Implement monitoring and logging for data pipelines.
Collaborate with cross-functional teams to understand data requirements.
Translate business requirements into technical specifications.
Implement data quality checks and cleansing procedures.
Create and maintain documentation for data pipelines.
Share knowledge and best practices within the team.
Architect data pipelines for massive IoT data streams.

LeadershipPythonSQLApache AirflowETLGCPIoTKafkaMachine LearningAirflowApache KafkaData engineeringSparkCommunication SkillsCollaboration

Posted 2024-11-07

Apply

🔥 Staff Data Engineer, Ads

Posted 2024-11-07

📍 ANY COUNTRY WITH A PHYSICAL PRESENCE

🧭 Full-Time

💸 206700 - 289400 USD per year

🔍 Internet

🏢 Company: Reddit

MS or PhD in a quantitative discipline: engineering, statistics, operations research, computer science, informatics, applied mathematics, economics, etc.
7+ years of experience with large-scale ETL systems, building clean and maintainable code (Python preferred).
Strong programming proficiency in Python, SQL, Spark, Scala, etc.
Experience with data modeling, ETL concepts, and handling structured and unstructured data.
Experience with data workflows (Airflow), front-end or back-end engineering, and data visualization tools.
Understanding of relational and MPP databases and proven track record of cross-functional collaboration.
Experience mentoring junior data scientists and analytics engineers; self-starter able to work autonomously and in teams.

Act as the analytics engineering lead within Ads DS team and a key contributor to data quality and automation initiatives.
Work on ETLs, reporting dashboards, and data aggregations for business tracking and ML model development.
Develop and maintain robust data pipelines and workflows for data ingestion and transformation.
Create user-friendly tools and applications for internal use, streamlining data analysis processes.
Lead efforts to enable data self-service and build a data-driven culture at Reddit.
Provide technical guidance and mentorship to data analysts and serve as a thought partner for various teams.

LeadershipPythonSQLData AnalysisETLTableauStrategyAirflowData analysisData engineeringData scienceSparkCommunication SkillsCollaborationMentoringCoaching

Posted 2024-11-07

Apply

🔥 Data Engineer (Databricks)

Posted 2024-10-25

📍 Canada, United States

🧭 Full-Time

💸 130000 - 160000 CAD per year

🔍 Performance marketing

🏢 Company: Fluent, LLC

Bachelor's or Master's degree in computer science.
3+ years of industry experience in Data Engineering.
Expertise in Spark and SQL.
1+ years of experience with Databricks environment.
Familiarity with real-time ML systems within Databricks is beneficial.

Execute on building data products to power Fluent’s business lines.
Develop, deploy, and manage scalable pipelines on Databricks.
Ensure robust integration with a Feature Store leveraging online tables for ML models.
Investigate and leverage Databricks’ capabilities for real-time data processing.
Maintain high quality of code with comprehensive data observability.
Partner with data science, UI, and reporting teams to understand data requirements.
Share knowledge by giving tech talks and evangelizing best practices.

SQLData engineeringSparkCollaboration

Posted 2024-10-25

Apply

🔥 Sr. Data Engineer (Databricks)

Posted 2024-10-25

📍 Canada, United States

🧭 Full-Time

💸 160000 - 180000 CAD per year

🔍 Performance marketing

🏢 Company: Fluent, LLC

Bachelor's or Master's degree in computer science.
5+ years of industry experience in Data Engineering.
Expertise in Spark and SQL.
2+ years of experience with the Databricks environment.
Familiarity with real-time ML systems within Databricks is beneficial.

Majority of the role will be software engineering – creating tables, views, Spark jobs, orchestrating within the Databricks environment.
Develop, deploy, and manage scalable data pipelines on Databricks with robust integration for machine learning models.
Implement real-time data processing and streaming using Databricks capabilities.
Maintain high code quality with observability, metadata standards, and best practices.
Work with data science, UI, and reporting teams to translate data requirements into models.
Stay updated on emerging tech trends within the Databricks ecosystem.
Share knowledge through events and empower internal teams with architectural insights.

SQLData engineeringSparkCollaboration

Posted 2024-10-25

Apply

Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities