Hadoop Solutions Developer
New
B
Bright Vision TechnologiesBig data
100% Remote (Continental United States)Full-TimeSenior
Salary100,000 - 150,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- Five or more years of professional experience designing and operating big-data pipelines on Hadoop.
- Required Skills
- PythonSQLHadoopKafkaAirflowSpark
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
- Five or more years of professional experience designing and operating big-data pipelines on Hadoop.
- Strong hands-on Apache Spark experience in production, using Scala, Python, or Java.
- Experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem.
- Hands-on experience with Kafka, Spark Streaming, or Flink.
- Strong SQL skills and experience with relational and NoSQL data stores.
- Experience with workflow orchestration tools such as Airflow or Oozie.
- Understanding of distributed systems concepts, including partitioning, replication, and fault tolerance.
- Strong Python or Shell scripting skills.
- Troubleshooting, debugging, and documentation skills.
- Candidates must work directly as a full-time W2 employee; C2C, 1099, and third-party arrangements are not accepted.
Responsibilities
- Design, develop, and operate end-to-end big-data pipelines on Hadoop.
- Build ETL/ELT workflows using Apache Spark, Hive, Pig, and Sqoop.
- Develop streaming pipelines using Kafka, Spark Streaming, or Flink.
- Tune Spark and MapReduce jobs to meet performance and SLA requirements.
- Design and maintain data models and storage layouts across HDFS, Hive, HBase, and lakehouse formats.
- Implement data governance, lineage, and quality controls with governance and security teams.
- Build monitoring, alerting, and logging strategies for pipelines.
- Partner with data scientists and analysts to deliver curated, documented datasets.
- Automate pipeline orchestration with Airflow, Oozie, or similar tools.
- Document architectures, schemas, pipeline behavior, and operational runbooks.
View Full Description & ApplyYou'll be redirected to the employer's site
