ApplyData Engineer - Open Application
Posted about 2 months agoViewed
View full description
💎 Seniority level: Senior, 5+ years
📍 Location: United States, Canada
🔍 Industry: Software Development
🏢 Company: Truelogic👥 101-250ConsultingWeb DevelopmentWeb DesignSoftware
🗣️ Languages: English
⏳ Experience: 5+ years
🪄 Skills: AWSPythonETLGCPHadoopKafkaMicrosoft SQL ServerMongoDBRabbitmqSnowflakeTableauAirflowAzureCassandraREST APIPandasSpark
Requirements:
- Coding Python and use in data processing solutions and related data technologies like Pandas, and PySpark.
- Consume data from different sources like REST APIs.
- Work with relational and non-relational data stores (like: HBASE, Cassandra or MongoDB; S3, blobs).
- Data Streams (Kafka, Kinesis, Flume,) and message queuing (SQS, SNS, RabbitMQ, etc).
- Ensure that the data model scales and enables high performance.
- Data/Stream processing (Spark, Flink, Hadoop).
- Data pipelines, data ingestion pipelines, scalable streaming data pipelines processing.
- ETL using solutions: Talend; Informatica; SQL Server Integration Services (SSIS).
- Data warehouse (Snowflake, Redshift, Hive).
- Implementation of data warehouse solutions, providing near real-time data to a variety of client systems;
- Using SQL databases to construct data storage.
- Reporting / BI, design, implementation, and enhancement of BI tool is a plus.
- Experience designing and implementing data applications and services on the public cloud, AWS, GCP, or Azure using PaaS platforms.
Responsibilities:
- Coding Python and use in data processing solutions and related data technologies like Pandas, and PySpark.
- Consume data from different sources like REST APIs.
- Work with relational and non-relational data stores (like: HBASE, Cassandra or MongoDB; S3, blobs).
- Design.
- Data Streams (Kafka, Kinesis, Flume,) and message queuing (SQS, SNS, RabbitMQ, etc).
- Ensure that the data model scales and enables high performance.
- Data/Stream processing (Spark, Flink, Hadoop).
- Data pipelines, data ingestion pipelines, scalable streaming data pipelines processing.
- ETL using solutions:
- Implementation of data warehouse solutions, providing near real-time data to a variety of client systems;
- Using SQL databases to construct data storage.
- Reporting / BI, design, implementation, and enhancement of BI tool is a plus.
- Experience designing and implementing data applications and services on the public cloud, AWS, GCP, or Azure using PaaS platforms.
Apply