Enformion

👥 101-250AnalyticsInformation Technology💼 Private Company
Website LinkedIn Email

Enformion empowers businesses and governments with advanced data and research solutions. Our platform offers flexible inputs, leverages best-of-breed public and proprietary data, and delivers rapid results from our extensive databases, containing billions of records. We specialize in helping clients uncover valuable insights on target individuals and companies, supporting data-driven business decisions and strategic targeting. Our engineering team builds and maintains a cutting-edge data processing platform. We work with technologies including Spark, EMR, MySQL, and NoSQL databases to ensure scalability and efficiency. We embrace a fast-paced, agile environment where innovation thrives. You'll collaborate with experienced engineers to tackle complex, large-scale data challenges, contributing to the continuous improvement of our data infrastructure. Our remote-friendly culture allows engineers to contribute from anywhere. As a growing company, Enformion fosters an inclusive environment that values individual contributions. We are committed to providing opportunities for career growth, professional development, and a supportive culture. This position offers a great opportunity to contribute to a company focused on innovation. We are seeking a Senior Data Engineer to join our team.

Related companies:

Jobs at this company:

Apply
🔥 Senior Data Engineer
Posted 6 days ago

💸 110000.0 - 125000.0 USD per year

🔍 Software Development

  • 5+ years minimum experience in language such as Java, Scala, PySpark, Perl, Shell Scripting and Python
  • Working knowledge of the Hadoop ecosystem applications (MapReduce, YARN, Pig, Hbase, Hive, Spark and more!)
  • Strong Experience working with data pipelines in multi-terabyte data warehouses. Experience in dealing with performance and scalability issues
  • Strong SQL (MySQL, Hive, etc.) and No-SQL (MongoDB, Hbase, etc.) skills, including writing complex queries and performance tuning
  • Knowledge of data modeling, partitioning, indexing, and architectural database design.
  • Experience using Source Code and Version Control systems like GIT etc.
  • Experience on continuous build and test process using tools such as GitLab, SBT, Postman, etc.
  • Experience with Search Engines, Name/Address Matching, or Linux text processing
  • Implement and maintain big data platform and infrastructure
  • Develop, optimize and tune MySQL stored procedures, scripts, and indexes
  • Develop Hive schemas and scripts, Spark Jobs using pyspark and Scala and UDFs in Java
  • Design, develop and maintain automated, complex, and efficient ETL processes to do batch records-matching of multiple large-scale datasets, including supporting documentation
  • Develop and maintains pipelines using Airflow or any other tools to monitor, debug, and analyze data pipelines
  • Troubleshoot Hadoop cluster and query issues, evaluate query plans, and optimize schemas and queries
  • Strong interpersonal skills to resolve problems in a professional manner, lead working groups, and negotiate consensus
Posted 6 days ago
Apply