Apply

Data Engineer

Posted about 1 month agoViewed

View full description

πŸ“ Location: Worldwide

πŸ” Industry: Decentralized Computing

🏒 Company: io.netπŸ‘₯ 11-50πŸ’° $30,000,000 Series A about 1 year agoCloud ComputingInformation TechnologyCloud InfrastructureGPU

πŸ—£οΈ Languages: English

πŸͺ„ Skills: AWSPostgreSQLPythonSQLApache AirflowCloud ComputingETLKafkaData engineeringData modeling

Requirements:
  • Strong programming skills in Python or Java.
  • Experience with SQL and relational databases (e.g., PostgreSQL, MySQL).
  • Knowledge of data pipeline tools like Apache Airflow, Spark, or similar.
  • Familiarity with cloud-based data warehouses (e.g., Redshift, Snowflake).
Responsibilities:
  • Design and build scalable ETL pipelines to handle large volumes of data.
  • Develop and maintain data models and optimize database schemas.
  • Work with real-time data processing frameworks like Kafka.
  • Ensure data quality, consistency, and reliability across systems.
  • Collaborate with backend engineers and data scientists to deliver insights.
  • Monitor and troubleshoot data workflows to ensure high availability.
Apply

Related Jobs

Apply
πŸ”₯ Associate Data Engineer
Posted about 9 hours ago

πŸ“ USA

🧭 Full-Time

πŸ” Digital Media Agency

🏒 Company: Spring & BondπŸ‘₯ 11-50Digital MarketingAdvertisingDigital MediaConsulting

  • 1+ years of experience in a data engineering or related role.
  • Strong SQL skills, including the ability to write complex queries using JOINs and aggregate functions.
  • Proficiency in Python and data manipulation libraries such as Pandas.
  • Experience with data validation techniques and tools.
  • Familiarity with AWS cloud services, particularly S3 and Lambda.
  • Experience with Git for code version control.
  • Detail-oriented with a focus on data accuracy and quality.
  • Organized with a systematic approach to managing data workflows.
  • Comfortable working in an ambiguous environment and able to independently drive projects forward.
  • Nimble and able to adapt to changing priorities.
  • Design, develop, and maintain ETL pipelines to ingest and transform data from various sources.
  • Implement data validation processes to ensure data accuracy and consistency throughout the data lifecycle, using tools like Regex.
  • Write and optimize SQL queries for data extraction, aggregation, and analysis.
  • Develop and maintain Python scripts and Pandas dataframes for data manipulation and analysis.
  • Utilize AWS Lambda functions to automate data processing tasks.
  • Manage code using Git for version control and collaborative development.
  • Collaborate with data analysts, media strategists, and other stakeholders to understand their data requirements and provide solutions.
  • Communicate technical concepts to non-technical stakeholders and translate business needs into technical specifications.
  • Troubleshoot and resolve data-related issues, identifying areas for improvement and efficiency gains.
  • Document data processes, pipelines, and transformations for knowledge sharing and maintainability.
  • Work with vendors to ensure seamless data integration and resolve any data delivery issues.
  • Apply critical thinking skills to analyze complex data problems and develop innovative solutions.

AWSPythonSQLData AnalysisETLGitData engineeringPandas

Posted about 9 hours ago
Apply
Apply

πŸ“ United States

🧭 Contract

πŸ’Έ 91520.0 - 120000.0 USD per year

🏒 Company: Third Eye SoftwareπŸ‘₯ 11-50ConsultingInformation TechnologyRecruitingSoftware

  • 3+ years of experience in data engineering
  • Advanced expertise in SQL
  • Experience with Google Cloud Platform (GCP)
  • Hands-on experience with ETL/ELT processes and data pipeline development
  • Proficiency in Python
NOT STATED

PythonSQLETLGCPJenkinsNumpyData engineeringPandas

Posted about 10 hours ago
Apply
Apply
πŸ”₯ Data Engineer
Posted 1 day ago

πŸ“ Italy

🧭 Full-Time

πŸ’Έ 40000.0 - 60000.0 EUR per year

πŸ” Fintech

🏒 Company: Qomodo

  • Experience in the design and development of scalable data pipelines
  • Excellent knowledge of SQL and relational databases (we use PostgreSQL)
  • You like Python and you wink at PySpark!
  • Familiarity with workflow orchestration tools (we use Airflow and Glue Workflow)
  • Knowledge of cloud services for data management (we mainly use AWS Glue and Athena)
  • Experience with ETL/ELT tools and data modeling practices
  • Understanding of best practices for data governance, quality and data security
  • Model data in a way that makes it easy for the business to extract insights
  • Create robust and scalable pipelines to support analysis, reporting, and decision-making

PostgreSQLPythonSQLApache AirflowCloud ComputingETLData engineeringData modeling

Posted 1 day ago
Apply
Apply

πŸ“ United States

πŸ’Έ 64000.0 - 120000.0 USD per year

  • Strong PL/SQL, SQL development skills
  • Proficient in multiple languages used in data engineering such as Python, Java
  • Minimum 3-5 years of experience in Data engineering working with Oracle and MS SQL
  • Experience with data warehousing concepts and technologies including cloud-based services (e.g. Snowflake)
  • Experience with cloud platforms like Azure and knowledge of infrastructure
  • Experience with data orchestration tools (e.g. Azure Data Factory, DataBricks workflows)
  • Understanding of data privacy regulations and best practices
  • Experience working with remote teams
  • Experience working on a team with a CI/CD process
  • Familiarity using tools like Git, Jira
  • Bachelor's degree in Computer Science or Computer Engineering
  • Design, implement and maintain scalable pipelines and architecture to collect, process, and store data from various sources.
  • Unit test and document solutions that meet product quality standards prior to release to QA.
  • Identify and resolve performance bottlenecks in pipelines due to data, queries and processing workflows to ensure efficient and timely data delivery.
  • Implement data quality checks and validations processes to ensure accuracy, completeness and consistency of data delivery.
  • Work with Data Architect and implement best practices for data governance, quality and security.
  • Collaborate with cross-functional teams to identify and address data needs.
  • Ensure technology solutions support the needs of the customer and/or organization.
  • Define and document technical requirements.

PythonSQLETLGitJavaOracleSnowflakeAzureData engineeringCI/CDRESTful APIs

Posted 2 days ago
Apply
Apply

πŸ“ Poland

🧭 Full-Time

πŸ” Software Development

🏒 Company: Craft Machine Inc

  • 2+ years of experience in Data Engineering.
  • 2+ years of experience with Python.
  • Experience in developing, maintaining, and ensuring the reliability, scalability, fault tolerance and observability of data pipelines in a production environment.
  • Strong knowledge of SDLC and solid software engineering practices.
  • Knowledge of and experience with Amazon Web Services (AWS) and Databricks.
  • Demonstrated curiosity through asking questions, digging into new technologies, and always trying to grow.
  • Strong problem solving and the ability to communicate ideas effectively.
  • Familiar with infrastructure-as-code approach.
  • Self-starter, independent, likes to take initiative.
  • Have fundamental knowledge of data engineering techniques: ETL/ELT, batch and streaming, DWH, Data Lakes, distributed processing.
  • Familiarity with at least some technologies in our current tech stack: Python, PySpark, Pandas, SQL (PostgreSQL), Airflow, Docker, Databricks & AWS (S3, Batch, Athena, RDS, DynamoDB, Glue, ECS), CircleCI, GitHub, Terraform
  • Building and optimizing data pipelines (batch and streaming).
  • Extracting, analyzing and modeling of rich and diverse datasets.
  • Designing software that is easily testable and maintainable.
  • Support in setting data strategies and our vision.
  • Keep track of emerging technologies and trends in the Data Engineering world, incorporating modern tooling and best practices at Craft.
  • Work on extendable data processing systems that allows to add and scale pipelines.
  • Applying machine learning techniques such as anomaly detection, clustering, regression classification, summarization to extract value from our data sets.

AWSDockerPostgreSQLPythonSQLETLMachine LearningAirflowAmazon Web ServicesData engineeringPandasCI/CDTerraformData modelingSoftware Engineering

Posted 2 days ago
Apply
Apply

πŸ“ AZ, CA, CO, CT, FL, GA, IL, MA, NV, NJ, NM, NY, OH, OR, PA, TX, VA, WA

🧭 Full-Time

πŸ’Έ 105000.0 - 120000.0 USD per year

πŸ” Software Development

🏒 Company: Committee for Children

  • 5+ years’ experience working with relational database systems
  • 5+ years’ experience performing business and financial analysis
  • 3+ years’ experience working with Power BI to develop reports and dashboards
  • Advanced proficiency in Power BI (Power Query M, DAX), Power Automate, Excel, SQL, and Microsoft Fabric Analytics
  • Experience with ETL processes, data warehousing, subscription business models, and SaaS KPIs
  • Experience with ERP systems (NetSuite preferred)
  • Experience with different data warehouse designs
  • Ability to prioritize tasks and manage work efficiently maintaining a high level of productivity
  • Demonstrated ability to navigate and articulate the workflow between data warehousing, data transformation, and reporting tools to ensure accuracy and relevance of insights generated
  • Experience working independently and in a team-oriented, collaborative environment
  • Strong critical thinking, analytical, and problem-solving skills
  • Sound decision making, discretion, and confidentiality
  • Develop and maintain datasets, data models, and data visualizations to support business decisions
  • Develop different data warehouse designs like star schema, snowflake schema, and dimensional modeling
  • Identify data sources, definitions, and timelines appropriate for analysis
  • Write, optimize and maintain complex SQL queries to support data analysis and reporting needs
  • Develop and generate ad hoc reports based on stakeholder requirements to support decision making processes in various departments
  • Work with Fabric to connect to various data sources such as databases, cloud storage, or APIs
  • Integrate data warehouse with business intelligence tools to create reports and dashboards
  • Design and build interactive reports using Power BI to present findings and identify trends to stakeholders
  • Ensure data quality and integrity by identifying and resolving data issues
  • Perform root cause analysis and uncover core issues using data, then assist the organization to improve
  • Analyze and interpret various sources of internal data and external data sources to support business decision-making
  • Design, build and maintain automated workflows using Power Automate to streamline business processes
  • Identify opportunities for process improvement and develop solutions to reduce manual effort

SQLBusiness IntelligenceData AnalysisETLMicrosoft SQL ServerData engineeringCommunication SkillsAnalytical SkillsCollaborationMicrosoft ExcelProblem SolvingRESTful APIsCritical thinkingReportingTroubleshootingJSONData visualizationFinancial analysisData modelingScriptingData analyticsData managementSaaS

Posted 2 days ago
Apply
Apply
πŸ”₯ Data Engineer
Posted 2 days ago

πŸ“ United States

🧭 Full-Time

πŸ” Sustainable Agriculture

🏒 Company: Agrovision

  • Experience with RDBMS (e.g., Teradata, MS SQL Server, Oracle) in production environments is preferred
  • Hands-on experience in data engineering and databases/data warehouses
  • Familiarity with Big Data platforms (e.g., Hadoop, Spark, Hive, HBase, Map/Reduce)
  • Expert level understanding of Python (e.g., Pandas)
  • Proficient in shell scripting (e.g., Bash) and Python data application development (or similar)
  • Excellent collaboration and communication skills with teams
  • Strong analytical and problem-solving skills, essential for tackling complex challenges
  • Experience working with BI teams and tooling (e.g. PowerBI), supporting analytics work and interfacing with Data Scientists
  • Collaborate with data scientists to ensure high-quality, accessible data for analytical and predictive modeling
  • Design and implement data pipelines (ETL’s) tailored to meet business needs and digital/analytics solutions
  • Enhance data integrity, security, quality, and automation, addressing system gaps proactively
  • Support pipeline maintenance, troubleshoot issues, and optimize performance
  • Lead and contribute to defining detailed scalable data models for our global operations
  • Ensure data security standards are met and upheld by contributors, partners and regional teams through programmatic solutions and tooling

PythonSQLApache HadoopBashETLData engineeringData scienceRDBMSPandasSparkCommunication SkillsAnalytical SkillsCollaborationProblem SolvingData modeling

Posted 2 days ago
Apply
Apply

πŸ“ LATAM

🧭 Full-Time

πŸ” Financial Services

🏒 Company: South GeeksπŸ‘₯ 101-250Web DevelopmentSoftware EngineeringEnterprise SoftwareSoftware

  • Proficiency in administering one or more platforms: Snowflake (required), DBT (required), GitHub, Workato (and/or Made), or other tools (preferred).
  • Data engineering experience with a focus on DBT and Snowflake.
  • Strong desire to expand skills across multiple platforms and contribute to both administrative and engineering functions.
  • Comfort with dynamic, multi-role environments that blend administration with engineering work.
  • Collaborative mindset, able to thrive in a team-first, cross-functional setting.
  • Administer one or more platforms, focusing on: Snowflake, DBT, Fivetran, and other tools: GitHub, Workato (and/or Make), and High Touch.
  • Participate in cross-training to administer multiple platforms, ensuring seamless coverage across the team.
  • Collaborate on data engineering projects, using DBT and Snowflake as part of the stack.
  • Take part in development opportunities and engineering work to broaden your expertise and career path.
  • Focus on team-based service delivery, rather than individual responsibilities, ensuring all systems are effectively managed by the group.

SQLSnowflakeData engineering

Posted 2 days ago
Apply
Apply

πŸ“ Lithuania

πŸ’Έ 4000.0 - 6000.0 EUR per month

πŸ” Software Development

🏒 Company: Softeta

  • 4+ years of experience as a Data Engineer
  • Experience with Azure (Certifications are a Plus)
  • Experience with Databricks, Azure Data Lake, Data Factory and Apache Airflow
  • CI/CD or infrastructure as code
  • Knowledge of Medallion Architecture or Multihop architecture
  • Experience developing and administering ETL processes in the Cloud (Azure, AWS or GCP) environment
  • Strong programming skills in Python and SQL
  • Strong problem-solving and analytical skills
  • Design, develop, and maintain data pipelines and ETL processes
  • Data modeling, data cleansing
  • Automating data processing workflows using tools such as Airflow or other workflow management tools
  • Optimizing the performance of databases, including designing and implementing data structures and using indexes appropriately
  • Implement data quality and data governance processes
  • Being a data advocate and helping unlock business value by using data

PythonSQLApache AirflowETLAzureData engineeringCI/CDData modeling

Posted 3 days ago
Apply
Apply

πŸ“ United States, Latin America, India

πŸ” Software Development

🏒 Company: phDataπŸ‘₯ 501-1000πŸ’° $2,499,997 Seed about 7 years agoInformation ServicesAnalyticsInformation Technology

  • 4+ years as a hands-on Data Engineer and/or Software Engineer
  • Experience with software development life cycle, including unit and integration testing
  • Programming expertise in Java, Python and/or Scala
  • Experience with core cloud data platforms including Snowflake, AWS, Azure, Databricks and GCP
  • Experience using SQL and the ability to write, debug, and optimize SQL queries
  • Client-facing written and verbal communication skills
  • Design and implement data solutions
  • Help ensure performance, security, scalability, and robust data integration
  • Develop end-to-end technical solutions into production
  • Multitask, prioritize, and work across multiple projects at once
  • Create and deliver detailed presentations
  • Detailed solution documentation (e.g. including POCS and roadmaps, sequence diagrams, class hierarchies, logical system views, etc.)

AWSPythonSoftware DevelopmentSQLCloud ComputingData AnalysisETLGCPJavaKafkaSnowflakeAzureData engineeringSparkCommunication SkillsCI/CDProblem SolvingAgile methodologiesRESTful APIsDocumentationScalaData modeling

Posted 3 days ago
Apply