Remoote.app

Work from anywhere. Start here.

JobsFree resume builderCompaniesLog in
Apply

Senior Data Engineer

Posted 23 days agoViewed

View full description

💎 Seniority level: Senior, 5+ years

📍 Location: Brazil

🔍 Industry: Data analysis and AI technology for government affairs

🏢 Company: Quorum👥 251-500💰 over 4 years agoCRMGovernmentPoliticsSaaSData VisualizationSoftware

⏳ Experience: 5+ years

🪄 Skills: AWSPythonSQLApache AirflowHadoopKafkaPyTorchNosqlSparkTensorflow

Requirements:
  • Bachelor's or Master's degree in Computer Science, Engineering, or related field.
  • 5+ years in data engineering with experience in scaling data-driven products.
  • Expertise in building data pipelines and AWS services (EC2, EMR, RDS, Redshift).
  • Proficient in big data tools (Hadoop, Spark, Kafka) and machine learning frameworks (TensorFlow, PyTorch).
  • 3+ years experience with Python.
  • Deep knowledge of SQL and NoSQL databases, and workflow management tools.
Responsibilities:
  • Architect and implement highly scalable advanced Retrieval-Augmented Generation (RAG) data pipelines.
  • Design robust data pipelines for real-time processing and analysis.
  • Lead cloud-based deployments in AWS ensuring performance and security.
  • Innovate on data architecture to meet Quorum Copilot's needs.
Apply

Related Jobs

Apply
🔥 Senior Data Engineer
Posted 9 days ago

📍 South Africa, Mauritius, Kenya, Nigeria

🔍 Technology, Marketplaces

  • BSc degree in Computer Science, Information Systems, Engineering, or related technical field or equivalent work experience.
  • 3+ years related work experience.
  • Minimum of 2 years experience building and optimizing ‘big data’ data pipelines, architectures and maintaining data sets.
  • Experienced in Python.
  • Experienced in SQL (PostgreSQL, MS SQL).
  • Experienced in using cloud services: AWS, Azure or GCP.
  • Proficiency in version control, CI/CD and GitHub.
  • Understanding/experience in Glue and PySpark highly desirable.
  • Experience in managing data life cycle.
  • Proficiency in manipulating, processing and architecting large disconnected data sets for analytical requirements.
  • Ability to maintain and optimise processes supporting data transformation, data structures, metadata, dependency and workload management.
  • Good understanding of data management principles - data quality assurance and governance.
  • Strong analytical skills related to working with unstructured datasets.
  • Understanding of message queuing, stream processing, and highly scalable ‘big data’ datastores.
  • Strong attention to detail.
  • Good communication and interpersonal skills.

  • Suggest efficiencies and execute on implementation of internal process improvements in automating manual processes.
  • Implement enhancements and new features across data systems.
  • Improve streamline processes within data systems with support from Senior Data Engineer.
  • Test CI/CD process for optimal data pipelines.
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Highly efficient in ETL processes.
  • Develop and conduct unit tests on data pipelines as well as ensuring data consistency.
  • Develop and maintain automated monitoring solutions.
  • Support reporting and analytics infrastructure.
  • Maintain data quality and data governance as well as upkeep of overall maintenance of data infrastructure systems.
  • Maintain data warehouse and data lake metadata, data catalogue, and user documentation for internal business users.
  • Ensure best practice is implemented and maintained on database.

AWSPostgreSQLPythonSQLETLGitCI/CD

Posted 9 days ago
Apply
Apply
🔥 Senior Data Engineer
Posted 23 days ago

📍 Brazil

🧭 Full-Time

🔍 Government affairs technology

  • Bachelor's or Master's degree in Computer Science, Engineering, or related field.
  • 5+ years in data engineering with a proven track record.
  • Expertise in building data pipelines and architectures.
  • Experience in AWS cloud services (EC2, EMR, RDS, Redshift).
  • Proficient in big data tools (Hadoop, Spark, Kafka) and machine learning frameworks (TensorFlow, PyTorch).
  • 3+ years experience with Python.
  • Deep knowledge of SQL and NoSQL databases, workflow management tools (Azkaban, Luigi, Airflow).
  • Understanding of the machine learning model deployment cycle.
  • Experience with vector databases and RAG systems (Langchain, Pinecone, OpenAI/ChatGPT) is a plus.

  • Architect and implement highly scalable advanced Retrieval-Augmented Generation (RAG) data pipelines.
  • Design robust data pipelines for real-time processing and analysis of vast datasets.
  • Design and implement data cleansing and transformation pipelines.
  • Lead cloud-based deployments in AWS ensuring performance and security.
  • Innovate on data architecture for Quorum Copilot's evolving needs.
  • Drive build vs buy, tool selection, and analysis using engineering principles.

AWSPythonSQLApache AirflowETLHadoopKafkaMachine LearningPyTorchNosqlSparkTensorflow

Posted 23 days ago
Apply
Apply
🔥 Senior Data Engineer
Posted about 2 months ago

📍 Brazil, Argentina

🧭 Full-Time

🔍 Manufacturing services

🏢 Company: Xometry👥 501-1000💰 $75,000,000 Series E over 4 years agoArtificial Intelligence (AI)3D PrintingIndustrial EngineeringSoftware

  • Bachelor's degree required, or relevant experience.
  • 3-5+ years of prior experience as a software engineer or data engineer in a fast-paced, technical, problem-solving environment.
  • Cloud Data Warehouse experience - Snowflake.
  • Expert in SQL.
  • Expert in ETL, data modeling, and version control, dbt and GitHub preferred.
  • Data modeling best practices for transactional and analytical processing.
  • Experience with data extraction tools (Fivetran, Airbyte, etc.).
  • Experience with event tracking software (Segment, Tealium, etc).
  • Experience with programming language like Python, JavaScript.
  • Experience with Business Intelligence tools, Looker preferred.
  • Ability to communicate effectively and influence others.
  • Ability to work in a fast-paced environment and shift gears quickly.
  • Must be able to work core aligned hours to US Eastern Time / GMT-5.

  • Close collaboration with other engineers and product managers to become a valued member of an autonomous, cross-functional team.
  • Build analytics models that utilize the data pipeline to provide actionable insights into key business performance metrics.
  • Develop data models for analytics and data scientist team members that assist them in building and optimizing data.
  • Maintain data pipelines and perform any changes or alterations as requested.
  • Develop deployment and release of functionality through software integration to support devops and CI/CD pipelines.

PythonSQLBusiness IntelligenceETLJavascriptSnowflakeCollaborationCI/CDDevOps

Posted about 2 months ago
Apply
Apply
🔥 (830) Senior Data Engineer
Posted about 2 months ago

📍 LATAM

🔍 Staff augmentation

🏢 Company: Nearsure👥 501-1000Staffing AgencyOutsourcingSoftware

  • Bachelor's Degree in Computer Science, Engineering, or related field.
  • 5+ Years of experience working in Data Engineering.
  • 3+ Years of experience working with SQL.
  • 3+ Years of experience working with Python.
  • 2+ Years of experience working with Cloud.
  • 2+ Years of experience working with Microservices.
  • 2+ Years of experience working with Databases.
  • 2+ Years of experience working with CI/CD tools.
  • Advanced English Level is required.

  • Write code to test and validate business logic in SQL and code repositories.
  • Test back-end API interfaces for the financial platform.
  • Build developer productivity tools.
  • Provide risk assessments on identified defects.
  • Define quality engineering scope and maintain automated scripts.
  • Participate in Scrum meetings for status updates.
  • Identify and clarify information needs.
  • Organize and resolve current backlog issues.

PythonSQLSCRUMData engineeringCI/CDMicroservices

Posted about 2 months ago
Apply
Apply
🔥 Senior Data Engineer
Posted 2 months ago

📍 Mexico, Gibraltar, Colombia, USA, Brazil, Argentina

🧭 Full-Time

🔍 FinTech

🏢 Company: Bitso

  • Proven English fluency.
  • 3+ years professional working experience with analytics, ETLs, and data systems.
  • 3+ years with SQL databases, data lake, big data, and cloud infrastructure.
  • 3+ years experience with Spark.
  • BS or Master's in Computer Science or similar.
  • Strong proficiency in SQL, Python, and AWS.
  • Strong data modeling skills.

  • Build processes required for optimal extraction, transformation, and loading of data from various sources using SQL, Python, Spark.
  • Identify, design, and implement internal process improvements while optimizing data delivery and redesigning infrastructure for scalability.
  • Ensure data integrity, quality, and security.
  • Work with stakeholders to assist with data-related technical issues and support their data needs.
  • Manage data separation and security across multiple data sources.

AWSPythonSQLBusiness IntelligenceMachine LearningData engineeringData StructuresSparkCommunication SkillsData modeling

Posted 2 months ago
Apply
Apply
🔥 Senior Data Engineer
Posted 4 months ago

📍 Brazil

🧭 Contract

🔍 Digital Learning

  • 4+ years of professional experience in data engineering.
  • Proficiency in processes used in modern data engineering.
  • Solid understanding of cloud infrastructure.
  • Hands-on experience with relevant data-related AWS services.
  • Advanced understanding of the relational data model.
  • Experience interacting with RESTful APIs and webhooks.
  • Understanding of DevOps disciplines and Git version control.
  • A commitment to writing clean, efficient, and maintainable code.
  • Familiarity with monitoring tools like CloudWatch or New Relic.
  • Familiarity with incident management tools like PagerDuty is desirable.
  • Understanding Software Engineering paradigms and lifecycle is desirable.
  • Strong SQL skills.
  • Expertise utilizing Python for data wrangling.
  • Experience using Airflow for data pipeline orchestration.
  • Experience with Spark creating big data workloads.
  • Experience working with large-scale data warehouses like Redshift.
  • Experience with Terraform composing AWS infrastructure is desirable.

  • Partner with Engineering and Product teams on high-impact initiatives.
  • Design and implement robust data pipelines using AWS services.
  • Enhance event collection, queueing, and processing mechanisms.
  • Build and refine meaningful alerts with automated corrective actions.
  • Troubleshoot and resolve issues, ensuring system reliability and stability.
  • Architect scalable data products with long-term vision and efficiency in mind.
  • Identify and resolve complex technical challenges involving system integration, infrastructure, and software bugs.

AWSPythonSQLETLGitJavaKafkaStrategyData engineeringCollaborationRESTful APIsTerraform

Posted 4 months ago
Apply
Apply
🔥 Senior Data Engineer, Data Services
Posted 5 months ago

📍 Central EU or Americas

🧭 Full-Time

🔍 Real estate investment

🏢 Company: Roofstock👥 501-1000💰 $240,000,000 Series E almost 3 years ago🫂 Last layoff almost 2 years agoRental PropertyPropTechMarketplaceReal EstateFinTech

  • BS or MS in a technical field: computer science, engineering or similar.
  • 8+ years technical experience working with data.
  • 5+ years strong experience building scalable data services and applications using SQL, Python, Java/Kotlin.
  • Deep understanding of microservices architecture and RESTful API development.
  • Experience with AWS services including messaging and familiarity with real-time data processing frameworks.
  • Significant experience building and deploying data-related infrastructure and robust data pipelines.
  • Strong understanding of data architecture and related challenges.
  • Experience with complex problems and distributed systems focusing on scalability and performance.
  • Strong communication and interpersonal skills.
  • Independent worker able to collaborate with cross-functional teams.

  • Improve and maintain the data services platform.
  • Deliver high-quality data services promptly, ensuring data governance and integrity while meeting objectives and maintaining SLAs.
  • Develop effective architectures and produce key code components contributing to technical solutions.
  • Integrate a diverse network of third-party tools into a cohesive, scalable platform.
  • Continuously enhance system performance and reliability by diagnosing and resolving operational issues.
  • Ensure rigorous testing of the team's work through automated methods.
  • Support data infrastructure and collaborate with the data team on scalable data pipelines.
  • Work within an Agile/Scrum framework with cross-functional teams to deliver value.
  • Influence the enterprise data platform architecture and standards.

AWSDockerPythonSQLAgileETLSCRUMSnowflakeAirflowData engineeringgRPCRESTful APIsMicroservices

Posted 5 months ago
Apply