Apply

Senior Data Engineer

Posted 4 months agoViewed

View full description

πŸ’Ž Seniority level: Senior, Minimum of 3 years

πŸ“ Location: Poland

πŸ” Industry: Enterprise security products

🏒 Company: Intuition Machines, Inc.πŸ‘₯ 51-100InternetEducationInternet of ThingsMachine LearningSoftware

⏳ Experience: Minimum of 3 years

πŸͺ„ Skills: AWSPythonSoftware DevelopmentSQLKafkaKubernetesStrategyAzureClickhouseData engineeringNosqlCI/CD

Requirements:
  • Thoughtful, conscientious, and self-directed.
  • Experience with data engineering services on major cloud providers.
  • Minimum of 3 years in a data role involving data store design, feature engineering, and building reliable data pipelines.
  • Proven ability to independently decide on data processing strategies.
  • At least 2 years of professional software development experience outside data engineering.
  • Significant coding experience in Python.
  • Experience in building/maintaining distributed data pipelines.
  • Experience with Kafka infrastructure and applications.
  • Deep understanding of SQL and NoSQL databases (preferably Clickhouse).
  • Familiarity with public cloud providers (AWS or Azure).
  • Experience with CI/CD and orchestration platforms: Kubernetes, containerization.
Responsibilities:
  • Maintain, extend, and improve existing data/ML workflows, and implement new workflows for high-velocity data.
  • Provide systems for ML engineers and researchers to build datasets on demand.
  • Influence data storage and processing strategies.
  • Collaborate with ML, frontend, and backend teams to enhance the data platform.
  • Reduce deployment time for dashboards and ML models.
  • Establish best practices and develop pipelines for efficient dataset usage.
  • Handle large datasets under performance constraints.
  • Iterate quickly to ensure deployment of products or features to millions of users.
Apply

Related Jobs

Apply

πŸ“ Europe, APAC, Americas

🧭 Full-Time

πŸ” Software Development

🏒 Company: DockerπŸ‘₯ 251-500πŸ’° $105,000,000 Series C almost 3 years agoDeveloper ToolsDeveloper PlatformInformation TechnologySoftware

  • 4+ years of relevant industry experience
  • Experience with data modeling and building scalable pipelines
  • Proficiency with Snowflake or BigQuery
  • Experience with data governance and security controls
  • Experience creating ETL scripts using Python and SQL
  • Familiarity with a cloud ecosystem: AWS/Azure/Google Cloud
  • Experience with Tableau or Looker
  • Manage and develop ETL jobs, warehouse, and event collection tools
  • Build and manage the Central Data Model for reporting
  • Integrate emerging methodologies and technologies
  • Build data pipelines for ML and AI projects
  • Contribute to SOC2 compliance across the data platform
  • Document technical architecture

PythonSQLETLSnowflakeAirflowData engineeringData visualizationData modeling

Posted 3 days ago
Apply
Apply

πŸ“ Poland

🧭 Full-Time

πŸ” Data Engineering

🏒 Company: Softeta

  • Advanced degree in computer science or related fields
  • 5+ years of experience as a data engineer
  • Proficiency with AirFlow, DBT, DataFlow or similar products
  • Strong knowledge of data structures and data modeling
  • CI/CD pipeline and MLOPs experience
  • Experience with large data sets
  • Experience with GCP / BigQuery
  • Create and maintain pipeline architectures in AirFlow and DBT
  • Assemble large and/or complex datasets for business requirements
  • Improve processes and infrastructure for scale, delivery and automation
  • Maintain and improve data warehouse structure
  • Adjust methods and techniques for large data environments
  • Adopt best-practice coding and review processes
  • Communicate technical details to stakeholders
  • Investigate and resolve anomalies in data
  • Develop and maintain documentation for data products

SQLGCPAirflowData engineeringCI/CDData modeling

Posted 12 days ago
Apply
Apply

πŸ“ Poland, Spain, United Kingdom

πŸ” Beauty marketplace

🏒 Company: BooksyπŸ‘₯ 501-1000πŸ’° Debt Financing 5 months agoMobile PaymentsMarketplaceSaaSPaymentsMobile AppsWellnessSoftware

  • 5+ years of experience in backend and data engineering, with strong system design skills.
  • Practical proficiency in cloud technologies (ideally GCP), with expertise in tools like BigQuery, Dataflow, Pub/Sub, or similar.
  • Hands-on experience with CI/CD tools (e.g., GitLab CI) and infrastructure as code.
  • Strong focus on data quality, governance, and building scalable, automated workflows.
  • Experience designing self-service data platforms and infrastructure.
  • Proven ability to mentor and support others, fostering data literacy across teams.
  • Design and implement robust data solutions.
  • Enable teams to make informed, data-driven decisions.
  • Ensure data is accessible, reliable, and well-governed.
  • Play a key role in driving growth, innovation, and operational excellence.

GCPData engineeringCI/CDData modeling

Posted 22 days ago
Apply
Apply
πŸ”₯ Senior Data Engineer
Posted about 1 month ago

πŸ“ South Africa, Mauritius, Kenya, Nigeria

πŸ” Technology, Marketplaces

  • BSc degree in Computer Science, Information Systems, Engineering, or related technical field or equivalent work experience.
  • 3+ years related work experience.
  • Minimum of 2 years experience building and optimizing β€˜big data’ data pipelines, architectures and maintaining data sets.
  • Experienced in Python.
  • Experienced in SQL (PostgreSQL, MS SQL).
  • Experienced in using cloud services: AWS, Azure or GCP.
  • Proficiency in version control, CI/CD and GitHub.
  • Understanding/experience in Glue and PySpark highly desirable.
  • Experience in managing data life cycle.
  • Proficiency in manipulating, processing and architecting large disconnected data sets for analytical requirements.
  • Ability to maintain and optimise processes supporting data transformation, data structures, metadata, dependency and workload management.
  • Good understanding of data management principles - data quality assurance and governance.
  • Strong analytical skills related to working with unstructured datasets.
  • Understanding of message queuing, stream processing, and highly scalable β€˜big data’ datastores.
  • Strong attention to detail.
  • Good communication and interpersonal skills.
  • Suggest efficiencies and execute on implementation of internal process improvements in automating manual processes.
  • Implement enhancements and new features across data systems.
  • Improve streamline processes within data systems with support from Senior Data Engineer.
  • Test CI/CD process for optimal data pipelines.
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Highly efficient in ETL processes.
  • Develop and conduct unit tests on data pipelines as well as ensuring data consistency.
  • Develop and maintain automated monitoring solutions.
  • Support reporting and analytics infrastructure.
  • Maintain data quality and data governance as well as upkeep of overall maintenance of data infrastructure systems.
  • Maintain data warehouse and data lake metadata, data catalogue, and user documentation for internal business users.
  • Ensure best practice is implemented and maintained on database.

AWSPostgreSQLPythonSQLETLGitCI/CD

Posted about 1 month ago
Apply
Apply

πŸ“ Poland

πŸ” Financial services

🏒 Company: CapcoπŸ‘₯ 101-250Electric VehicleProduct DesignMechanical EngineeringManufacturing

  • Strong cloud provider’s experience on GCP
  • Hands-on experience using Python; Scala and Java are nice to have
  • Experience in data and cloud technologies such as Hadoop, HIVE, Spark, PySpark, DataProc
  • Hands-on experience with schema design using semi-structured and structured data structures
  • Experience using messaging technologies – Kafka, Spark Streaming
  • Strong experience in SQL
  • Understanding of containerisation (Docker, Kubernetes)
  • Experience in design, build and maintain CI/CD Pipelines
  • Enthusiasm to pick up new technologies as needed
  • Work alongside clients to interpret requirements and define industry-leading solutions
  • Design and develop robust, well tested data pipelines
  • Demonstrate and help clients adhere to best practices in engineering and SDLC
  • Lead and mentor the team of junior and mid-level engineers
  • Contribute to security designs and have advanced knowledge of key security technologies
  • Support internal Capco capabilities by sharing insight, experience and credentials

DockerPythonSQLETLGCPGitHadoopKafkaKubernetesSnowflakeAirflowSparkCI/CD

Posted 2 months ago
Apply