Remoote.app

Work from anywhere. Start here.

JobsFree resume builderCompaniesLog in
Apply

Data Engineer

Posted about 1 month agoViewed

View full description

💎 Seniority level: Senior, 4+ years

📍 Location: India

🔍 Industry: Data and cloud engineering services

🏢 Company: Enable Data Incorporated

⏳ Experience: 4+ years

🪄 Skills: AWSPythonSQLETLMicrosoft AzureSnowflakeData engineering

Requirements:
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • 4+ years of experience with Big Data technologies like Azure/AWS.
  • Experience with Databricks and Snowflake, with Python scripting and SQL knowledge.
  • Strong knowledge and experience with Microsoft Azure cloud platform.
  • Proficiency in SQL and experience with SQL-based database systems.
  • Experience with batch and data streaming.
  • Experience using Azure data services such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
  • Strong analytical and problem-solving skills.
  • Good understanding of data engineering principles and best practices.
  • Experience with programming languages such as Python or Scala.
  • Relevant certifications in Azure data services or data engineering are a plus.
Responsibilities:
  • Design, develop, and implement scalable and reliable data solutions on the Azure/AWS platform.
  • Collaborate with cross-functional teams to gather and analyze data requirements.
  • Design and implement data ingestion pipelines to collect data from various sources.
  • Perform data integration and transformation activities, ensuring data quality and consistency.
  • Implement data storage and retrieval mechanisms using Azure services.
  • Monitor data pipelines and troubleshoot issues.
  • Implement data quality measures and governance practices.
  • Collaborate with data scientists and analysts to support their data needs.
Apply

Related Jobs

Apply
🔥 Sr Data Engineer
Posted 11 days ago

📍 US, Europe and India

🔍 Software Development

  • Extensive experience in developing data and analytics applications in geographically distributed teams
  • Hands-on experience in using modern architectures and frameworks, structured, semi-structured and unstructured data, and programming with Python
  • Hands-on SQL knowledge and experience with relational databases such as MySQL, PostgreSQL, and others
  • Hands-on ETL knowledge and experience
  • Knowledge of commercial data platforms (Databricks, Snowflake) or cloud data warehouses (Redshift, BigQuery)
  • Knowledge of data catalog and MDM tooling (Atlan, Alation, Informatica, Collibra)
  • CICD pipeline for continuous deployment (CloudFormation template)
  • Knowledge of how machine learning / A.I. workloads are implemented in batch and streaming, including the preparing of datasets, training models, and using pre-trained models
  • Exposure to software engineering processes that can be applied to Data Ecosystems
  • Design and develop our best-in-class cloud platform, working on all parts of the code stack from front-end, REST and asynchronous APIs, back-end application logic, SQL/NoSQL databases and integrations with external systems
  • Develop solutions across the data and analytics stack from ETL and Streaming data
  • Design and develop reusable libraries
  • Enhance strong processes in Data Ecosystem
  • Write unit and integration tests

PythonSQLApache AirflowCloud ComputingETLMachine LearningSnowflakeAlgorithmsApache KafkaData engineeringData StructuresCommunication SkillsAnalytical SkillsCI/CDRESTful APIsDevOpsMicroservicesExcellent communication skillsData visualizationData modelingData analyticsData management

Posted 11 days ago
Apply
Apply
🔥 Data Engineer - Tableau
Posted 21 days ago

📍 India

🧭 Full-Time

🔍 Technology Services

🏢 Company: computer_aid

  • 1+ years of working knowledge in Tableau.
  • 3+ years of experience with AWS Data services like S3, Glue, Lake Formation, EMR, Kinesis, RDS, DMS, and Redshift.
  • 3+ years of experience building Data Warehouses on Snowflake, Redshift, HANA, Teradata, Exasol, etc.
  • 3+ years of experience in building Delta Lakes using technologies like Apache or Databricks.
  • 3+ years of experience working with ETL tools and technologies.
  • 3+ years of experience in any programming language (Python, R, Scala, Java).
  • Bachelor's degree in computer science, information technology, data science, data analytics, or related field.
  • Experience working on Agile projects and familiarity with Agile methodology.
  • 4+ years of experience operating on AWS Cloud building Data Lake architectures.
  • Strong RDBMS and data modeling skills.
  • AWS cloud certification is a plus.
  • Strong communication skills, both written and spoken.
  • Design and develop data lakes and manage data flows integrating various information sources.
  • Code and manage delta lake implementations on S3 using Databricks or Apache Hoodie.
  • Triage, debug, and fix technical issues related to Data Lakes.
  • Design and develop data warehouses for scalable solutions.
  • Evaluate various data models like Star, Snowflake, and Flattened.
  • Design data access patterns for OLTP and OLAP transactions.
  • Coordinate with business and technical teams throughout the software development life cycle.
  • Participate in major technical and architectural decision-making.

AWSPythonSQLETLSnowflakeTableau

Posted 21 days ago
Apply
Apply
🔥 Senior Data Engineer
Posted about 1 month ago

📍 South Africa, Mauritius, Kenya, Nigeria

🔍 Technology, Marketplaces

  • BSc degree in Computer Science, Information Systems, Engineering, or related technical field or equivalent work experience.
  • 3+ years related work experience.
  • Minimum of 2 years experience building and optimizing ‘big data’ data pipelines, architectures and maintaining data sets.
  • Experienced in Python.
  • Experienced in SQL (PostgreSQL, MS SQL).
  • Experienced in using cloud services: AWS, Azure or GCP.
  • Proficiency in version control, CI/CD and GitHub.
  • Understanding/experience in Glue and PySpark highly desirable.
  • Experience in managing data life cycle.
  • Proficiency in manipulating, processing and architecting large disconnected data sets for analytical requirements.
  • Ability to maintain and optimise processes supporting data transformation, data structures, metadata, dependency and workload management.
  • Good understanding of data management principles - data quality assurance and governance.
  • Strong analytical skills related to working with unstructured datasets.
  • Understanding of message queuing, stream processing, and highly scalable ‘big data’ datastores.
  • Strong attention to detail.
  • Good communication and interpersonal skills.
  • Suggest efficiencies and execute on implementation of internal process improvements in automating manual processes.
  • Implement enhancements and new features across data systems.
  • Improve streamline processes within data systems with support from Senior Data Engineer.
  • Test CI/CD process for optimal data pipelines.
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Highly efficient in ETL processes.
  • Develop and conduct unit tests on data pipelines as well as ensuring data consistency.
  • Develop and maintain automated monitoring solutions.
  • Support reporting and analytics infrastructure.
  • Maintain data quality and data governance as well as upkeep of overall maintenance of data infrastructure systems.
  • Maintain data warehouse and data lake metadata, data catalogue, and user documentation for internal business users.
  • Ensure best practice is implemented and maintained on database.

AWSPostgreSQLPythonSQLETLGitCI/CD

Posted about 1 month ago
Apply
Apply
🔥 Data Engineer
Posted 3 months ago

📍 India

🔍 Artificial intelligence and aquaculture

🏢 Company: Nacre Capital👥 10-100Venture Capital

  • Proficiency in English (spoken and written).
  • Strong experience in Python and SQL.
  • Hands-on experience with data processing in Apache Airflow.
  • Experience working with databases, including MongoDB (NoSQL) and relational databases.
  • Understanding of data modeling, ETL processes, and data warehousing concepts.
  • Experience with cloud platforms like AWS, GCP, or Azure.
  • Develop, maintain, and optimize data pipelines and workflows to support ML research and model development.
  • Design and implement scalable data architectures for handling large datasets used in ML models.
  • Collaborate closely with ML researchers and data scientists to understand data requirements and ensure data availability and quality.
  • Prepare and transform data for ML experiments using databases and data integration processes.
  • Utilize MongoDB and other NoSQL databases to manage unstructured and semi-structured data.
  • Write efficient, reliable, and maintainable code in Python and SQL for data processing tasks.
  • Implement data validation and monitoring systems to ensure data integrity and performance.
  • Support the deployment of ML models by integrating data solutions into production environments.
  • Ensure the scalability and performance of data systems through rigorous testing and optimization.

AWSPythonSQLApache AirflowETLGCPMongoDBAzure

Posted 3 months ago
Apply
Apply
🔥 Staff Data Engineer - R12645
Posted 3 months ago

📍 India

🔍 Fintech

🏢 Company: Oportun👥 1001-5000💰 $235,000,000 Post-IPO Debt 4 months ago🫂 Last layoff over 1 year agoDebit CardsConsumer LendingFinancial ServicesFinTech

  • Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
  • 10+ years of experience in data engineering focused on architecture, ETL, and database management.
  • Proficiency in programming languages like Python/PySpark, Java/Scala.
  • Expertise in big data technologies such as Hadoop, Spark, Kafka.
  • In-depth knowledge of SQL and experience with various database technologies.
  • Familiarity with cloud platforms and their data services.
  • Strong leadership, problem-solving, and decision-making skills.
  • Provide technical leadership and mentorship to the data engineering team.
  • Lead the design and implementation of complex data architectures.
  • Design and develop efficient and scalable data pipelines.
  • Oversee design and management of databases and ETL processes.
  • Manage complex data engineering projects ensuring timely delivery.
  • Establish data quality standards and monitoring processes.
  • Collaborate with stakeholders for data solutions.

AWSPostgreSQLPythonSQLApache AirflowCloud ComputingETLGCPHadoopJavaJenkinsKafkaMySQLAzureData engineeringNosqlSparkScala

Posted 3 months ago
Apply
Apply
🔥 Data Engineer
Posted 4 months ago

📍 India

🏢 Company: Unison Consulting Pte Ltd

  • Minimum 6+ years of Data Ingestion, Integration, ETL, or security engineering experience.
  • Extensive knowledge of AWS, Azure, GCP.
  • Strong understanding of Data Management or Data Engineering.
  • Experienced in Agile methods and Atlassian stack (e.g., JIRA).
  • Ability to develop roadmaps for data-centric products.
  • Experience with monitoring frameworks and observability products.
  • Expertise in SIEM solutions and cloud-based data sources.
  • Familiarity with security monitoring solutions like Splunk and Datadog.
  • Experience in DevSecOps/IRE and agile environments.
  • Expertise in scripting languages (PowerShell, Python, Bash).
  • Experience with Docker, Kubernetes, Ansible, or Terraform.
  • Related security certifications (e.g., CISSP, CCSP).
  • Experience with Linux/Ubuntu/Mac systems.
  • Experience in creating dashboards and troubleshooting connectivity issues.
  • Define and manage data models, schemas, metadata, and security rules.
  • Design, create, deploy, and manage databases and data structures on-premise and in the cloud.
  • Identify and mitigate potential security risks.
  • Ensure compliance with data privacy laws and regulations.
  • Conduct risk assessments and take appropriate actions to mitigate data security risks.
  • Train and educate stakeholders about data management.
  • Collaborate with IT team members and stakeholders to secure data architectures.

AWSDockerPythonAgileBashData AnalysisETLGCPKubernetesJiraAzureData engineeringData StructuresCollaborationLinuxTerraformCompliance

Posted 4 months ago
Apply
Apply
🔥 Sr Data Engineer
Posted 4 months ago

📍 India

🔍 SaaS-based global employment platform

🏢 Company: G-P

  • Extensive experience in developing data and analytics applications in geographically distributed teams.
  • Hands-on experience in using modern architectures and frameworks, structured, semi-structured and unstructured data, and programming with Python.
  • Hands-on SQL knowledge and experience with relational databases such as MySQL, PostgreSQL, and others.
  • Hands-on ETL knowledge and experience.
  • Knowledge of commercial data platforms (Databricks, Snowflake) or cloud data warehouses (Redshift, BigQuery).
  • Knowledge of data catalog and MDM tooling (Atlan, Alation, Informatica, Collibra).
  • CICD pipeline for continuous deployment (CloudFormation template).
  • Knowledge of how machine learning / A.I. workloads are implemented in batch and streaming, including the preparing of datasets, training models, and using pre-trained models.
  • Exposure to software engineering processes that can be applied to Data Ecosystems.
  • Excellent analytical and troubleshooting skills.
  • Excellent communication skills.
  • Excellent English (both verbal and written).
  • BS. in Computer Science or equivalent.
  • Design and develop our best-in-class cloud platform, working on all parts of the code stack from front-end, REST and asynchronous APIs, back-end application logic, SQL/NoSQL databases and integrations with external systems.
  • Develop solutions across the data and analytics stack from ETL and Streaming data.
  • Design and develop reusable libraries.
  • Enhance strong processes in Data Ecosystem.
  • Write unit and integration tests.

PostgreSQLPythonSQLAgileETLMachine LearningMySQLSnowflakeNosqlCommunication Skills

Posted 4 months ago
Apply
Apply
🔥 Data Engineer (Bengaluru)
Posted 5 months ago

📍 India

🔍 Data Technology

🏢 Company: Demyst👥 51-100💰 about 2 years agoBig DataFinancial ServicesBroadcastingData IntegrationAnalyticsInformation TechnologyFinTechSoftware

  • Bachelor's in Computer Science, Data Science, Engineering or similar technical discipline (or commensurate work experience); Master's degree preferred.
  • 1-3 years of Python programming (with Pandas experience).
  • Experience with CSV, JSON, parquet, Avro, and other common formats.
  • Data cleaning and structuring (ETL experience).
  • Knowledge of API (REST and SOAP), HTTP protocols, API Security and best practices.
  • Experience with SQL, Git, and Airflow.
  • Strong written and oral communication skills.
  • Excellent attention to detail.
  • Ability to learn and adapt quickly.
  • Willingness to align to US working hours.
  • Collaborate with internal project managers, sales directors, account managers, and clients’ stakeholders to identify requirements and build external data-driven solutions.
  • Perform data appends, extracts, and analyses to deliver curated datasets and insights to clients to help achieve their business objectives.
  • Understand and keep current with external data landscapes such as consumer, business, and property data.
  • Engage in projects involving entity detection, record linking, and data modelling projects.
  • Design scalable code blocks using Demyst’s APIs/SDKs that can be leveraged across production projects.
  • Govern releases, change management and maintenance of production solutions in close coordination with clients' IT teams.

AWSPythonSQLETLGitSnowflakeSoftware ArchitectureAirflowData sciencePandasCommunication SkillsJSON

Posted 5 months ago
Apply