Data Platform Engineer

Posted 1 day agoViewed

View full description

💎 Seniority level: Senior, 5+ years

📍 Location: United States

💸 Salary: 180000.0 - 220000.0 USD per year

🔍 Industry: Software Development

🏢 Company: Prepared👥 51-100💰 $27,000,000 Series B 8 months agoEnterprise Software Public Safety

🗣️ Languages: English

⏳ Experience: 5+ years

🪄 Skills: AWSDockerPostgreSQLPythonSQLApache AirflowETLKubernetesSnowflakeApache KafkaData engineeringSparkScalaData modeling

Requirements:

5+ years of experience in data engineering, software engineering with a data focus, data science, or a related role
Knowledge of designing data pipelines from a variety of source (e.g. streaming, flat files, APIs)
Proficiency in SQL and experience with relational databases (e.g., PostgreSQL)
Experience with real-time data processing frameworks (e.g., Apache Kafka, Spark Streaming, Flink, Pulsar, Redpanda)
Strong programming skills in common data-focused languages (e.g., Python, Scala)
Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Prefect, Temporal)
Familiarity with AWS-based data solutions
Strong understanding of data warehousing concepts and technologies (Snowflake)
Experience documenting data dependency maps and data lineage
Strong communication and collaboration skills
Ability to work independently and take initiative
Proficiency in containerization and orchestration tools (e.g., Docker, Kubernetes)

Responsibilities:

Design, implement, and maintain scalable data pipelines and infrastructure
Collaborate with software engineers, product managers, customer success managers, and others across the business to understand data requirements
Optimize and manage our data storage solutions
Ensure data quality, reliability, and security across the data lifecycle
Develop and maintain ETL processes and frameworks
Work with stakeholders to define data availability SLAs
Create and manage data models to support business intelligence and analytics

Apply

Related Jobs

Apply

🔥 Senior Data Platform Engineer

Posted about 1 month ago

📍 California, Colorado, Illinois, Florida, Georgia, Massachusetts, Nevada, New Jersey, New York, North Carolina, Oregon, Pennsylvania, South Carolina, Texas, Virginia, Washington, and Maine

🧭 Full-Time

💸 175000.0 - 195000.0 USD per year

🔍 Software Development

🏢 Company: Crunchbase👥 101-250💰 $50,000,000 Series D almost 3 years ago🫂 Last layoff almost 2 years agoDatabase Information Services Business Intelligence Lead Generation Marketing Automation Software

🔧 Requirements

4+ years of experience in data engineering with a focus on designing and building scalable data solutions.
Strong proficiency in Python.
Strong experience with data modeling, ETL processes, and data warehousing.
Experience with cloud platforms (e.g., AWS, Azure, GCP)
Excellent problem-solving skills and a proactive approach to addressing challenges.
Strong communication and interpersonal skills.

💡 Responsibilities

Design and implement scalable and efficient data solutions.
Design, build, and optimize data pipelines for both batch and real-time processing.
Develop and maintain data models that align with business needs and support analytical requirements.
Work closely with data scientists, analysts, and other stakeholders to understand their needs and provide data engineering support.
Implement monitoring solutions to proactively identify and address performance issues.
Stay abreast of industry trends and evaluate new tools and technologies to enhance our data capabilities.

AWSPythonSQLCloud ComputingData AnalysisETLData engineeringCommunication SkillsProblem SolvingData modeling

Posted about 1 month ago

Apply

🔥 Sr. Data Platform Engineer - GenAI Apps

Posted 3 months ago

📍 United States

🧭 Full-Time

💸 140000.0 - 190000.0 USD per year

🔍 Healthcare

🏢 Company: Qventus👥 101-250💰 $85,000,000 Series D 5 months agoArtificial Intelligence (AI)Machine Learning Hospital Analytics Health Care

🔧 Requirements

Proven experience as a Data Engineer; Excellence in quality data pipeline design, development, and optimization to create reliable, modular, secure data foundations for the organization's data delivery system from applications to analytics & ML
Expertise in Python, SQL, and familiarity with AWS tools, Databricks, and DBT.
Strong understanding of database structure systems and data mining.
Innate aptitude for interpreting complex datasets with demonstrated ability to discern underlying patterns, identify anomalies, and extract meaningful insights, demonstrating advanced data intuition and analytical skills.
Proven ability to independently handle ambiguous project requirements and lead data initiatives from start to finish, while collaborating extensively with cross-functional, non-technical teams to inform and shape product development.

💡 Responsibilities

Lead the design, development, and management across investments to platform, model infrastructure, and data pipelines in support of data intensive AI solutions.
Work closely with a close knit team of solution experts & innovators to design, iterate, and develop key pipelines to unlock new solution functionality, analytical insights, and machine learning features.
Identify, monitor, and lead initiatives to ensure our data platform remains scalable, reliable, and efficient in light of evolving data requirements of our products and services - particularly in the rapidly changing Gen AI / LLM ecosystem
Partner with cross-functional stakeholders & peers to translate data needs into technical solutions with rigorous technical scope to enable more junior engineers
Spearhead the discovery, evaluation, and integration of new datasets, collaborating (incl. pipeline development and data modeling/documentation), working closely with key data stakeholders to understand their impact and relevance to our core products and the healthcare domain.
Translate product / analytical vision into highly functional data pipelines supporting high quality & highly trusted data products (incl. designing data structures, building and scheduling data transformation pipelines, improving transparency etc.).
Engage in the rapid new AI product development from concept to deployment, ensuring robust platform supporting easy iteration and monitoring.
Set the standard for data engineering practices within the company, guiding the architectural approaches, data pipeline designs, and the integration of cutting-edge technologies to foster a culture of innovation and continuous improvement.

AWSPythonSQLAgileETLKubernetesMachine LearningData engineeringREST APICommunication SkillsAnalytical SkillsCI/CDProblem SolvingData visualizationData modelingData analytics

Posted 3 months ago

Apply