Data Engineer

Posted 2024-08-26

View full description

💎 Seniority level: Middle, 4+ years

📍 Location: Americas, EMEA, APAC

🔍 Industry: Crypto and blockchain technology

🗣️ Languages: Over 50 languages

⏳ Experience: 4+ years

🪄 Skills: PythonSQLETLAirflowData engineeringSpark

Requirements:

4+ years of work experience in relevant fields such as Data Engineer, DWH Engineer, or Software Engineer.
Experience with data warehouse technologies and relevant data modeling best practices like Presto, Athena, Glue, etc.
Experience building data pipelines/ETL and familiarity with design principles; knowledge of Apache Airflow is a plus.
Excellent SQL and data manipulation skills using frameworks like Spark/PySpark or similar.
Proficiency in a major programming language such as Scala, Python, or Golang.
Experience with business requirements gathering for data sourcing.

Responsibilities:

Build scalable and reliable data pipeline that collects, transforms, loads, and curates data from internal systems.
Augment data platform with data pipelines from select external systems.
Ensure high data quality for pipelines built and maintain auditability.
Drive data systems to approach real-time processing.
Support the design and deployment of a distributed data store as the central source of truth.
Build data connections to internal IT systems.
Develop and customize self-service tools for data consumers.
Evaluate new technologies and create prototypes for continuous improvements in data engineering.

Apply

Related Jobs

Apply

🔥 Lead Data Engineer (SAP)

Posted 2024-11-23

📍 North America, Latin America, Europe

🔍 Data Consultancy

Bachelor’s degree in engineering, computer science or equivalent area.
5+ years in related technical roles, including data management, database development, ETL, and data warehouses.
Experience with data ingestion technologies and architectural decisions for high-throughput frameworks.
Familiarity with Snowflake, SAP, AWS, Azure, GCP, and relevant ETL tools.

Develop database architectures and data warehouses.
Ensure optimal data delivery architecture throughout ongoing customer projects.
Lead technical teams in data system optimization and development.

AWSLeadershipPythonSQLAgileETLOracleSAPSnowflakeData engineeringSparkCollaborationProblem Solving

Posted 2024-11-23

Apply

🔥 Tech Lead Data Engineer & Analyst

Posted 2024-11-23

📍 Brazil

🧭 Full-Time

🔍 Digital Engineering and Modernization

🏢 Company: Encora

Experience creating data warehouses.
Experience in data modeling.
Experience developing and maintaining data pipelines.
Proficiency in Google Cloud Platform (BigQuery).
Experience with Fivetran.
Knowledge of dbt (data build tool).
Experience creating dashboards with BI tools.

Develop and maintain efficient and scalable data pipelines.
Model and transform data to meet analysis and reporting needs.
Collaborate with BI teams to ensure data integration and quality.
Create and maintain detailed technical documentation.
Develop dashboards in Looker as needed.

LeadershipSQLBusiness IntelligenceGCPCross-functional Team LeadershipData engineeringRDBMSAnalytical SkillsCollaborationAttention to detailDocumentation

Posted 2024-11-23

Apply

🔥 Senior Data Engineer

Posted 2024-11-23

📍 Spain

💸 80000 - 110000 EUR per year

🔍 Financial services

🏢 Company: Affirm

5+ years of professional experience in Data Engineering or similar roles.
Proficiency with SQL and DBT for data transformations.
Experience with Python or other modern programming languages.
Knowledge of infrastructure as code languages, such as Terraform.
Experience with data pipelines, data modeling, data warehouse technologies, and cloud infrastructures.
Familiarity with AWS or other cloud providers like Azure, GCP.
Strong cross-team communication and collaboration skills.
Ability to thrive in ambiguity.

Work with engineering managers and tech leads to identify and plan projects based on team goals.
Collaborate with cross-functional teams within Affirm to deliver technology for analytical use cases.
Write high-quality, understandable code.
Review others' work and provide constructive feedback.
Serve as a technical resource and mentor for other engineers.
Foster an inclusive and supportive team environment.
Participate in on-call rotation.

AWSPythonSQLGCPAzureData engineeringCollaborationTerraform

Posted 2024-11-23

Apply

🔥 Databricks Data Engineer - Data & Analytics team (remote / Costa Rica- or LATAM-based)

Posted 2024-11-22

📍 Costa Rica, LATAM

🧭 Full-Time

🔍 IT solutions and consulting

5+ years of Data Engineering experience including 2+ years designing and building Databricks data pipelines is REQUIRED.
2+ years of hands-on Python/Pyspark/SparkSQL and/or Scala experience is REQUIRED.
2+ years of experience with Big Data pipelines or DAG tools is REQUIRED.
2+ years of Spark experience, especially Databricks Spark and Delta Lake, is REQUIRED.
2+ years of hands-on experience implementing Big Data solutions in a cloud ecosystem is REQUIRED.
2+ years of SQL experience, specifically writing complex queries, is HIGHLY DESIRED.
Experience with source control (git) on the command line is REQUIRED.

Scope and execute together with team leadership.
Work with the team to understand platform capabilities and how to improve them.
Design, develop, enhance, and maintain complex data pipeline products.
Support analytics, data science, and engineering teams and address their challenges.
Commit to continuous learning and developing technical maturity across the company.

LeadershipPythonSQLGitKafkaAirflowAzureData engineeringSpark

Posted 2024-11-22

Apply

🔥 Senior Data Engineer

Posted 2024-11-22

📍 Ontario

🔍 Customer engagement platform

🏢 Company: Braze

5+ years of hands-on experience in data engineering, cloud data warehouses, and ETL development.
Proven expertise in designing and optimizing data pipelines and architectures.
Strong proficiency in advanced SQL and data modeling techniques.
A track record of leading impactful data projects from conception to deployment.
Effective collaboration skills with cross-functional teams and stakeholders.
In-depth understanding of technical architecture and data flow in a cloud-based environment.
Ability to mentor and guide junior team members on best practices for data engineering and development.
Passion for building scalable data solutions that enhance customer experiences and drive business growth.
Strong analytical and problem-solving skills, with a keen eye for detail and accuracy.
Extensive experience working with and aggregating large event-level data.
Familiarity with data governance principles and ensuring compliance with industry regulations.
Preferable experience with Kubernetes for container orchestration and Airflow for workflow management.

Lead the design, implementation, and monitoring of scalable data pipelines and architectures using tools like Snowflake and dbt.
Develop and maintain robust ETL processes to ensure high-quality data ingestion, transformation, and storage.
Collaborate closely with data scientists, analysts, and other engineers to design and implement data solutions that drive customer engagement and retention.
Optimize and manage data flows and integrations across various platforms and applications.
Ensure data quality, consistency, and governance by implementing best practices and monitoring systems.
Work extensively with large-scale event-level data, aggregating and processing it to support business intelligence and analytics.
Implement and maintain data products using advanced techniques and tools.
Collaborate with cross-functional teams including engineering, product management, sales, marketing, and customer success to deliver valuable data solutions.
Continuously evaluate and integrate new data technologies and tools to enhance our data infrastructure and capabilities.

SQLBusiness IntelligenceETLSnowflakeData engineeringCollaborationCompliance

Posted 2024-11-22

Apply

🔥 Cloud Data Engineer (AWS) (Remote, Canada)

Posted 2024-11-21

📍 Canada

🔍 Artificial Intelligence

Strong background in AWS DevOps and data engineering.
Expertise with AWS and SageMaker is essential.
Experience with Snowflake for analytics and data warehousing is highly desirable.

Manage and optimize the data infrastructure.
Focus on both data engineering and DevOps responsibilities.
Deploy machine learning models to AWS using SageMaker.

AWSMachine LearningSnowflakeData engineeringDevOps

Posted 2024-11-21

Apply

🔥 Senior Data Engineer

Posted 2024-11-21

📍 Poland

🧭 Full-Time

🔍 Software development

🏢 Company: Sunscrapers sp. z o.o.

At least 5 years of professional experience as a data engineer.
Undergraduate or graduate degree in Computer Science, Engineering, Mathematics, or similar.
Excellent command in spoken and written English, at least C1.
Strong professional experience with Python and SQL.
Hands-on experience with DBT and Snowflake.
Experience in building data pipelines with Airflow or alternative solutions.
Strong understanding of various data modeling techniques like Kimball Star Schema.
Great analytical skills and attention to detail.
Creative problem-solving skills.
Great customer service and troubleshooting skills.

Modeling datasets and schemes for consistency and easy access.
Design and implement data transformations and data marts.
Integrating third-party systems and external data sources into data warehouse.
Building data flows for fetching, aggregation and data modeling using batch pipelines.

PythonSQLSnowflakeAirflowAnalytical SkillsCustomer serviceDevOpsAttention to detail

Posted 2024-11-21

Apply

🔥 Data Engineer

Posted 2024-11-21

📍 Poland

🔍 Healthcare

🏢 Company: Sunscrapers sp. z o.o.

At least 3 years of professional experience as a data engineer.
Undergraduate or graduate degree in Computer Science, Engineering, Mathematics, or similar.
Excellent command in spoken and written English, at least C1.
Strong professional experience with Apache Spark.
Hands-on experience managing production spark clusters in Databricks.
Experience in CI/CD of data jobs in Spark.
Great analytical skills, attention to detail, and creative problem-solving skills.
Great customer service and troubleshooting skills.

Design and manage batch data pipelines, including file ingestion, transformation, and Delta Lake/table management.
Implement scalable architectures for batch and streaming workflows.
Leverage Microsoft equivalents of BigQuery for efficient querying and data storage.

SparkAnalytical SkillsCI/CDCustomer serviceAttention to detail

Posted 2024-11-21

Apply

🔥 Senior Data Engineer

Posted 2024-11-21

📍 Belgium, Spain

🔍 Hospitality industry

🏢 Company: Lighthouse

5+ years of professional experience using Python, Java, or Scala for data processing (Python preferred)
Experience with writing data processing pipelines and with cloud platforms like AWS, GCP, or Azure
Experience with data pipeline orchestration tools like Apache Airflow (preferred), Dagster or Prefect
Deep understanding of data warehousing strategies
Experience with transformation tools like dbt to manage data transformation in your data pipelines
Some experience in managing infrastructure with IaC tools like Terraform
Stay updated with industry trends, emerging technologies, and best practices in data engineering
Improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed
Ship large features independently, generate architecture recommendations with the ability to implement them
Strong communicator that can describe complex topics in a simple way to a variety of technical and non-technical stakeholders.

Design and develop scalable, reliable data pipelines using the Google Cloud stack.
Ingest, process, and store structured and unstructured data from various sources into our data-lakes and data warehouses.
Optimise data pipelines for cost, performance and scalability.
Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
Mentor and provide technical guidance to other engineers working with data.
Partner with Product, Engineering & Data Science teams to operationalise new solutions.

PythonApache AirflowGCPJavaKafkaKubernetesAirflowData engineeringGrafanaPrometheusSparkCI/CDTerraformDocumentationCompliance

Posted 2024-11-21

Apply

🔥 Data Engineer

Posted 2024-11-20

📍 Argentina, Spain, England, United Kingdom, Lisbon, Portugal

🧭 Full-Time

🔍 Web3

🏢 Company: Reown

5+ years working in the analytics stack within a fast-paced environment.
3+ years production experience with SQL templating engines like DBT.
Experience with distributed query engines (Bigquery, Athena, Spark), data warehouses, and BI tools.
Strong understanding of software engineering principles, coding standards, design patterns, version control (e.g., Git), testing methodologies, and CI/CD processes.
Experience with AWS/GCP/Azure services for deployment and management.
Familiarity with GitHub, CI/CD pipelines, GitHub Actions, and Terraform.
Ability to write Python scripts for ETL processes and data manipulation.
Proficient with libraries like pandas for analysis and transformation.
Experience handling various data formats (e.g., CSV, JSON, Parquet).
Strong problem-solving skills and communication abilities to discuss technical concepts.

Write complex SQL queries that extract and combine data from on-chain and off-chain logs for analytics.
Create dashboards and tools for team data discoverability and KR tracking.
Perform deep-dive analyses into specific topics for internal stakeholders.
Help design, implement, and evolve Reown's on-chain data infrastructure.
Build, maintain, and monitor end-to-end data pipelines for new datasets and features.
Write health-checks and alerts to ensure data correctness, consistency, and freshness.
Meet with product managers and stakeholders to understand data needs and detect new product opportunities.

AWSPythonSQLData AnalysisDesign PatternsETLGCPGitTableauAzureClickhouseData analysisPandasSparkCommunication SkillsCI/CDTerraformWritten communication

Posted 2024-11-20

Apply

Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities