Data Engineer - Open Application
T
TruelogicStaff Augmentation
Latin AmericaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English
- Experience
- 5+ years
- Required Skills
- AWSPythonETLGCPHadoopKafkaMicrosoft SQL ServerMongoDBRabbitmqSnowflakeTableauAirflowAzureCassandraREST APIPandasSpark
Requirements
- 5+ years of professional experience.
- Proficiency in Python, Pandas, and PySpark.
- Experience consuming data from REST APIs.
- Experience with relational and non-relational data stores (HBASE, Cassandra, MongoDB, S3, blobs).
- Experience with data streaming (Kafka, Kinesis) and message queuing (SQS, SNS, RabbitMQ).
- Ensure data model scalability and high performance.
- Experience with distributed data stores.
- Experience with data/stream processing (Spark, Flink, Hadoop).
- Experience building data pipelines, data ingestion pipelines, and scalable streaming data pipelines.
- Experience with ETL tools: Talend, Informatica, or SQL Server Integration Services (SSIS).
- Experience with data warehouses (Snowflake, Redshift, Hive).
- Experience designing and implementing data applications on AWS, GCP, or Azure using PaaS.
- Familiarity with data privacy regulations and best practices.
Responsibilities
- Code Python for data processing solutions using Pandas and PySpark.
- Consume data from various sources like REST APIs.
- Work with relational and non-relational data stores (HBASE, Cassandra, MongoDB, S3, blobs).
- Design and implement data and stream processing solutions (Spark, Flink, Hadoop).
- Build scalable data ingestion and streaming data pipelines.
- Implement ETL processes using Talend, Informatica, or SSIS.
- Implement data warehouse solutions and provide near real-time data.
- Design and implement data applications on public cloud (AWS, GCP, Azure) using PaaS.
- Understand data privacy regulations and best practices.
View Full Description & ApplyYou'll be redirected to the employer's site
