Staff Software Engineer - Data Ingestion
New
J
JobgetherIT Security
Based in United StatesFull-TimeStaff
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 8+ years of production-level software engineering experience
- Required Skills
- DockerPythonSQLCloud ComputingETLJavaKubernetesCI/CDDistributed Systems
Requirements
- Bachelor’s or higher degree in Computer Science, Engineering, or a related technical field.
- 8+ years of production-level software engineering experience building highly scalable and reliable systems.
- 4+ years serving as a trusted technical decision-maker within engineering teams.
- 4+ years of experience working with SQL or other database query languages across large, multi-table datasets.
- Demonstrated experience architecting, developing, and deploying large-scale distributed systems.
- Experience working with cloud platforms such as AWS, Azure, or GCP.
- Strong experience building and maintaining continuous integration and continuous delivery pipelines.
- Strong familiarity with server-side technologies and programming languages such as Java, Python, Scala, C#, C++, or Go.
- Extensive experience designing, optimizing, and orchestrating robust data pipelines and ingestion systems for large-scale real-time and batch processing.
- Experience with data warehouses such as Snowflake or Redshift and analytics technologies such as Spark, SQL, Python, or Databricks.
- Hands-on experience with containerization technologies including Docker and Kubernetes.
- Strong understanding of distributed architectures, including event-driven systems and in-memory computing.
Responsibilities
- Identify, architect, and develop scalable, highly performant solutions for large-scale data ingestion, storage, and processing.
- Lead the refactoring and optimization of existing data pipelines to improve reliability, scalability, maintainability, and query performance.
- Design and develop next-generation distributed data storage and processing systems capable of supporting large-scale, real-time, and batch workloads.
- Build robust ETL/ELT and data ingestion architectures that efficiently process complex and high-volume datasets.
- Develop clean, expressive interfaces and abstractions that simplify access to complex data infrastructure for web applications, analytics, and AI use cases.
- Work cross-functionally with Product, Engineering, Analytics, and other disciplines to shape product strategy, technical direction, and execution.
- Establish strong foundations for code architecture, engineering quality, reliability, scalability, and maintainability.
- Set and uphold engineering processes, standards, and best practices that support consistent delivery of high-quality software.
- Evaluate database architecture and performance strategies, including partitioning, indexing, replication, sharding, caching, and query optimization.
- Design and maintain CI/CD pipelines and engineering workflows that support reliable, automated software delivery.
View Full Description & ApplyYou'll be redirected to the employer's site
