Data Engineer

Posted 23 days agoViewed

View full description

💎 Seniority level: Senior, 5+ years

📍 Location: United States

💸 Salary: 165000.0 - 190000.0 USD per year

🔍 Industry: Software Development

🏢 Company: ThalamusGME

🗣️ Languages: English

⏳ Experience: 5+ years

🪄 Skills: AWSPostgreSQLPythonSQLHadoopSnowflakeTableauData engineeringSparkJSONData visualization

Requirements:

5+ years experience in data engineering
5+ years demonstrated experience with Spark & PySpark
5+ demonstrated experience with a variety of JSON, data parsing, XML, YAML, unstructured and structured data
5+ years with python best practices for coding and documentation
5+ years experience with BI (tableau reporting) visualization & database experience (postgres, sqlserver, and snowflake) and distributed databases (hadoop, databricks)
Strong SQL knowledge
Working in Azure or AWS

Responsibilities:

Collaborate with application engineers, data scientists, product managers, and technical support (CX team)
Implement data pipelines from RDBMS, application logs and unstructured data sources
Implement data aggregation, data cubes, verification, and cleansing solutions
Work with devops engineers to design scalable cloud solutions
Write readable, maintainable code for agile development in a highly collaborative workspace
Plan new data acquisition, storage and maintenance solutions
Work with application engineers to create efficient, reliable data products for ingestion into other applications
Develop data engineering model and best practices for a growing data-centric organization

Apply

Related Jobs

Apply

🔥 Data Engineer

Posted 1 day ago

📍 United States

💸 112800.0 - 126900.0 USD per year

🔍 Software Development

🏢 Company: Titan Cloud

🔧 Requirements

4+ years of work experience with ETL, Data Modeling, Data Analysis, and Data Architecture.
Experience operating very large data warehouses or data lakes.
Experience with building data pipelines and applications to stream and process datasets at low latencies.
MySQL, MSSQL Database, Postgres, Python

💡 Responsibilities

Design, implement, and maintain standardized data models that align with business needs and analytical use cases.
Optimize data structures and schemas for efficient querying, scalability, and performance across various storage and compute platforms.
Provide guidance and best practices for data storage, partitioning, indexing, and query optimization.
Developing and maintaining a data pipeline design.
Build robust and scalable ETL/ELT data pipelines to transform raw data into structured datasets optimized for analysis.
Collaborate with data scientists to streamline feature engineering and improve the accessibility of high-value data assets.
Designing, building, and maintaining the data architecture needed to support business decisions and data-driven applications. This includes collecting, storing, processing, and analyzing large amounts of data using AWS, Azure, and local tools and services.
Develop and enforce data governance standards to ensure consistency, accuracy, and reliability of data across the organization.
Ensure data quality, integrity, and completeness in all pipelines by implementing automated validation and monitoring mechanisms.
Implement data cataloging, metadata management, and lineage tracking to enhance data discoverability and usability.
Work with Engineering to manage and optimize data warehouse and data lake architectures, ensuring efficient storage and retrieval of structured and semi-structured data.
Evaluate and integrate emerging cloud-based data technologies to improve performance, scalability, and cost efficiency.
Assist with designing and implementing automated tools for collecting and transferring data from multiple source systems to the AWS and Azure cloud platform.
Work with DevOps Engineers to integrate any new code into existing pipelines
Collaborate with teams in trouble shooting functional and performance issues.
Must be a team player to be able to work in an agile environment

AWSPostgreSQLPythonSQLAgileApache AirflowCloud ComputingData AnalysisETLHadoopMySQLData engineeringData scienceREST APISparkCommunication SkillsAnalytical SkillsCI/CDProblem SolvingTerraformAttention to detailOrganizational skillsMicroservicesTeamworkData visualizationData modelingScripting

Posted 1 day ago

Apply

🔥 Data Engineer II (Remote, US)

Posted 1 day ago

📍 United States

🧭 Full-Time

💸 108000.0 - 162000.0 USD per year

🔍 Insurance

🏢 Company: Openly👥 251-500💰 $100,000,000 Series D over 1 year agoLife Insurance Property Insurance Insurance Commercial Insurance Auto Insurance

🔧 Requirements

1 to 2 years of data engineering and data management experience.
Scripting skills in one or more of the following: Python.
Basic understanding and usage of a development and deployment lifecycle, automated code deployments (CI/CD), code repositories, and code management.
Experience with Google Cloud data store and data orchestration technologies and concepts.
Hands-on experience and understanding of the entire data pipeline architecture: Data replication tools, staging data, data transformation, data movement, and cloud based data platforms.
Understanding of a modern next generation data warehouse platform, such as the Lakehouse and multi-data layered warehouse.
Proficiency with SQL optimization and development.
Ability to understand data architecture and modeling as it relates to business goals and objectives.
Ability to gain an understanding of data requirements, translate them into source to target data mappings, and build a working solution.
Experience with terraform preferred but not required.

💡 Responsibilities

Design, create, and maintain data solutions. This includes data pipelines and data structures.
Work with data users, data science, and business intelligence personnel, to create data solutions to be used in various projects.
Translating concepts to code to enhance our data management frameworks and services to strive towards providing a high quality data product to our data users.
Collaborate with our product, operations, and technology teams to develop and deploy new solutions related to data architecture and data pipelines to enable a best-in-class product for our data users.
Collaborating with teammates to derive design and solution decisions related to architecture, operations, deployment techniques, technologies, policies, processes, etc.
Participate in domain, stand ups, weekly 1:1's, team collaborations, and biweekly retros
Assist in educating others on different aspects of data (e.g. data management best practices, data pipelining best practices, SQL tuning)
Build and share your knowledge within the data engineer team and with others in the company (e.g. tech all-hands, tech learning hour, domain meetings, code sync meetings, etc.)

DockerPostgreSQLPythonSQLApache AirflowCloud ComputingETLGCPKafkaKubernetesData engineeringGoREST APICI/CDTerraformData modelingScriptingData management

Posted 1 day ago

Apply

🔥 Staff Data Engineer

Posted 1 day ago

📍 United States

🧭 Full-Time

🔍 Software Development

🏢 Company: Life360👥 251-500💰 $33,038,258 Post-IPO Equity over 2 years ago🫂 Last layoff about 2 years agoAndroid Family Apps Mobile Apps Mobile

🔧 Requirements

Minimum 7 years of experience working with high volume data infrastructure.
Experience with Databricks and AWS.
Experience with dbt.
Experience with job orchestration tooling like Airflow.
Proficient programming in Python.
Proficient with SQL and the ability to optimize complex queries.
Proficient with large-scale data processing using Spark and/or Presto/Trino.
Proficient in data modeling and database design.
Experience with streaming data with a tool like Kinesis or Kafka.
Experience working with high volume event based data architecture like Amplitude and Braze.
Experience in modern development lifecycle including Agile methodology, CI/CD, automated deployments using Terraform, GitHub Actions, etc.
Knowledge and proficiency in the latest open source and data frameworks, modern data platform tech stacks and tools.
Always learning and staying up to speed with the fast moving data world.
You have good communication and collaboration skills and can work independently.
BS in Computer Science, Software Engineering, Mathematics, or equivalent experience.

💡 Responsibilities

Design, implement, and manage scalable data processing platforms used for real-time analytics and exploratory data analysis.
Manage our financial data from ingestion through ETL to storage and batch processing.
Automate, test and harden all data workflows.
Architect logical and physical data models to ensure the needs of the business are met.
Collaborate across the data teams, engineering, data science, and analytics, to understand their needs, while applying engineering best practices.
Architect and develop systems and algorithms for distributed real-time analytics and data processing.
Implement strategies for acquiring data to develop new insights.
Mentor junior engineers, imparting best practices and institutionalizing efficient processes to foster growth and innovation within the team.
Champion data engineering best practices and institutionalizing efficient processes to foster growth and innovation within the team.

AWSProject ManagementPythonSQLApache AirflowETLKafkaAlgorithmsData engineeringData StructuresREST APISparkCommunication SkillsAnalytical SkillsCollaborationCI/CDProblem SolvingAgile methodologiesMentoringTerraformData visualizationTechnical supportData modelingData analyticsData managementDebugging

Posted 1 day ago

Apply

🔥 Data Engineer (Remote/Hybrid)

Posted 1 day ago

📍 United States

💸 70000.0 - 105000.0 USD per year

🔍 Software Development

🏢 Company: VUHL

🔧 Requirements

Relevant experience in data engineering or a related discipline.
Demonstrated ability to code effectively and a solid understanding of software engineering principles.
Experience using SQL or other query language to manage and process data.
Experience using Python to build ETL pipelines
Experience working with data from various sources and in various formats, including flat files, REST APIs, Excel files, JSON, XML, etc.
Experience with Snowflake, SQL Server, or related database technologies.
Experience using orchestration tools like Dagster (preferred), Apache Airflow, or similar.
Preference for Agile product delivery.
Familiarity with GIT, Change Management, and application lifecycle management tools.
Ability to influence others without positional control.

💡 Responsibilities

Create and deliver functional ETL pipelines and other data solutions using core technologies like SQL, Python, Snowflake, Dagster, and SSIS in an agile development environment. Apply sound database design principles and adhere to Clean Code practices.
Engage in whole team planning, retrospectives, and communication. Interact with Architects and Product Owners to translate requirements into actionable business logic.
Participate in proposing and adopting Engineering standards related to architectural considerations and non-functional requirements such as security, reliability, and stability. Ensure proper management and visibility of borrower data and the life of a loan. Contribute to data governance initiatives.
Actively contribute to strengthening the team and culture by taking on various duties as needed, excluding licensed activities.

PythonSQLAgileApache AirflowETLGitSnowflakeData engineeringREST APIJSONData modelingSoftware EngineeringData management

Posted 1 day ago

Apply

🔥 Principal Data Engineer

Posted 2 days ago

📍 United States, Canada

🧭 Full-Time

💸 150000.0 - 180000.0 USD per year

🔍 SaaS

🔧 Requirements

8+ years of experience in data engineering, product analytics, or SaaS.
Strong experience with SQL, AWS Redshift, and ETL pipelines.
Proficiency with BI tools (Looker, Metabase, Mixpanel).
Experience with AI-driven analytics and NLP techniques.
Strong communication and stakeholder management skills.
Highly organized and capable of managing multiple priorities.

💡 Responsibilities

Own and optimize data architecture, ensuring scalability and efficiency.
Partner with Ops Engineering to improve data pipelines and integrations.
Manage and optimize AWS Redshift and other key platforms.
Define and enforce data governance best practices.
Extract insights from customer interactions using AI-driven analytics.
Identify trends in feature requests, pain points, and product issues.
Develop dashboards to provide actionable insights for stakeholders.
Ensure data is structured and available for sales tracking, forecasting, and segmentation.
Support revenue modeling and churn risk analysis.
Maintain CRM data integrity and enable data-driven sales strategies.
Support A/B testing and controlled experiments to optimize product and sales decisions.
Develop models to measure customer engagement and sales effectiveness.
Build predictive models for customer retention and revenue growth.
Work with stakeholders to enable data-driven decision-making.
Develop self-service tools and training materials.
Promote best practices and data accessibility across teams.

AWSPythonSQLETLData engineeringData visualizationData modelingScriptingData analyticsSaaS

Posted 2 days ago

Apply

🔥 Senior Data Engineer (Remote)

Posted 2 days ago

📍 United States

🔍 Software Development

AWSPostgreSQLPythonSQLApache AirflowCloud ComputingData AnalysisETLApache KafkaData engineeringCI/CDRESTful APIsMicroservicesData visualizationData modelingData analyticsData management

Posted 2 days ago

Apply

🔥 Senior Data Engineer/Scientist

Posted 3 days ago

📍 United States, Canada

🧭 Full-Time

💸 105825.0 - 136950.0 CAD per year

🔍 Data Engineering

🏢 Company: Samsara👥 1001-5000💰 Secondary Market over 4 years ago🫂 Last layoff almost 5 years agoCloud Data Services Business Intelligence Internet of Things SaaS Software

🔧 Requirements

BS degree in Computer Science, Statistics, Engineering, or a related quantitative discipline
6+ years experience in a data engineering and data science-focused role
Proficiency in data manipulation and processing in SQL and Python
Expertise building data pipelines with new API endpoints from their documentation
Proficiency in building ETL pipelines to handle large volumes of data
Demonstrated experience in designing data models at scale

💡 Responsibilities

Build and maintain highly reliable computed tables, incorporating data from various sources, including unstructured and highly sensitive data
Access, manipulate, and integrate external datasets with internal data
Building analytical and statistical models to identify patterns, anomalies, and root causes
Leverage SQL and Python to shape and aggregate data
Incorporate generative AI tools (ChatGPT Enterprise) into production data pipelines and automated workflows
Collaborate closely with data scientists, data analysts, and Tableau developers to ship top quality analytic products
Champion, role model, and embed Samsara’s cultural principles (Focus on Customer Success, Build for the Long Term, Adopt a Growth Mindset, Be Inclusive, Win as a Team) as we scale globally and across new offices

PythonSQLETLTableauAPI testingData engineeringData scienceSparkCommunication SkillsAnalytical SkillsData visualizationData modeling

Posted 3 days ago

Apply

🔥 Sr Data Engineer

Posted 3 days ago

📍 United States

💸 117400.0 - 190570.0 USD per year

🏢 Company: healthfirst

🔧 Requirements

8+ Years overall IT experience
Enterprise experience in scripting languages primarily Python and Pyspark building enterprise frameworks
Enterprise experience in data ingestion methodologies using different etl tools(Glue,DBT or any other)
Enterprise experience in data warehousing concepts and big data technologies like EMR, Hadoop
Enterprise experience in any cloud infrastructure like AWS,GCP,Azure
Strong SQL expertise across different relational and NoSQL Databases.

💡 Responsibilities

Designs and implements standardized data management procedures around data staging, data ingestion, data preparation, data provisioning, and data destruction (e.g., scripts, programs, automation, etc.)
Ensures quality of technical solutions as data moves across multiple zones and environments
Provides insight into the changing data environment, data processing, data storage and utilization requirements for the company, and offer suggestions for solutions
Ensures managed analytic assets to support the company’s strategic goals by creating and verifying data acquisition requirements and strategy
Develops, constructs, tests, and maintains architectures
Aligns architecture with business requirements and uses programming language and tools
Identifies ways to improve data reliability, efficiency, and quality
Conducts research for industry and business questions
Deploys sophisticated analytics programs, machine learning, and statistical methods to efficiently implement solutions
Prepares data for predictive and prescriptive modeling and find hidden patterns using data
Uses data to discover tasks that can be automated
Creates data monitoring capabilities for each business process and works with data consumers on updates
Aligns data architecture to the solution architecture; contributes to overall solution architecture
Develops patterns for standardizing the environment technology stack
Helps maintain the integrity and security of company data
Additional duties as assigned or required

AWSPythonSQLETLHadoopData engineeringCI/CDDevOpsData modelingScripting

Posted 3 days ago

Apply

🔥 Associate Data Engineer (Consultant)

Posted 4 days ago

📍 United States

💸 65000.0 - 100000.0 USD per year

🔍 Data Analytics

🏢 Company: DAS42👥 51-100💰 almost 4 years agoCloud Data Services Business Intelligence Big Data Data Integration Analytics

🔧 Requirements

1-3 years of work experience in a data analytics or data engineering role.
Advanced proficiency with Python and SQL.
Previously worked within a consulting/professional services environment (preferred)
Experience with business visualization tools (e.g. Looker, Tableau, Microsoft PowerBI).
Ability to explain technical concepts to non-technical people.
Experience with data warehousing and using Snowflake.
Experience with the AWS or Google Cloud ecosystem (e.g. BigQuery, Redshift).
Worked with or for Media & Entertainment or Telecom companies (preferred).
Bachelor's or Master’s degree in a quantitative field (i.e. Statistics or Operations Research) or technical field (i.e. Computer Science or Engineering) (preferred).

💡 Responsibilities

Implement data warehouse solutions, ETL pipelines, and business intelligence reporting tools.
Develop data models around stated use cases to capture clients' KPIs, data transformations, and data requirements; validation and testing of data models.
Support, maintain, and document clients’ data environments.
Contribute to and collaborate on project deliverables, either independently or as part of a larger consulting team.
Work within a project management framework (i.e., Agile) to meet objectives, understand scope, and impact of your work across an organization.
Guide clients on optimizing their data environment to operate most effectively and efficiently for them. Clarify management objectives through data solutions.
Work with the Sales and Marketing teams to develop Thought Leadership and workable POCs and support our sellers by talking to clients about DAS42’s go-to-market capabilities.

AWSPythonSQLBusiness IntelligenceCloud ComputingETLGCPSnowflakeData engineeringREST APIData visualizationData modeling

Posted 4 days ago

Apply

🔥 Lead Data Engineer

Posted 5 days ago

📍 United States

🧭 Full-Time

💸 152500.0 - 178000.0 USD per year

🔍 Software Development

🔧 Requirements

10+ years of professional software development or data engineering experience (10+ with a STEM B.S. or 8+ with a relevant Master's degree)
Strong proficiency in Python and familiarity with Java and Bash scripting
Hands-on experience implementing database technologies, messaging systems, and stream computing software (e.g., PostgreSQL, PostGIS, MongoDB, DuckDB, KsqlDB, RabbitMQ)
Experience with data fabric development using publish-subscribe models (e.g., Apache NiFi, Apache Pulsar, Apache Kafka and Kafka-based data service architecture)
Proficiency with containerization technologies (e.g., Docker, Docker-Compose, RKE2, Kubernetes, and Microk8s)
Experience with version control systems (e.g., Git), CI/CD tools (e.g., Jenkins), and collaborative development workflows
Strong knowledge of data modeling and database optimization techniques
Familiarity with data serialization languages (e.g., JSON, GeoJSON, YAML, XML)
Excellent problem-solving and analytical skills that have been applied to high visibility, important data engineering projects
Strong communication skills and ability to lead the work of other engineers in a collaborative environment
Demonstrated experience in coordinating team activities, setting priorities, and managing tasks to ensure balanced workloads and effective team performance
Experience managing and mentoring development teams in an Agile environment
Ability to make effective architecture decisions and document them clearly
Must be a US Citizen and eligible to obtain and maintain a US Security Clearance

💡 Responsibilities

Develop and continuously improve a data service that underpins cloud-based applications
Support data and database modeling efforts
Contribute to the development and maintenance of reusable component libraries and shared codebase
Participate in the entire software development lifecycle, including requirement gathering, design, development, testing, and deployment, using an agile, iterative process
Collaborate with developers, designers, testers, project managers, product owners, and project sponsors to integrate the data service to end user applications
Communicate tasking estimation and progress regularly to a development lead and product owner through appropriate tools
Ensure seamless integration between database and messaging systems and the frontend / UI they support
Ensure data quality, reliability, and performance through code reviews and effective testing strategies
Write high-quality code, applying best practices, coding standards, and design patterns
Team with other developers, fostering a culture of continuous learning and professional growth

AWSDockerLeadershipPostgreSQLPythonSoftware DevelopmentSQLAgileBashCloud ComputingGitJavaJenkinsKubernetesMongoDBRabbitmqApache KafkaData engineeringCommunication SkillsCI/CDProblem SolvingRESTful APIsMentoringTerraformMicroservicesJSONData visualizationTeam managementAnsibleData modelingSoftware EngineeringData analyticsData management

Posted 5 days ago

Apply