Senior Data Engineer
New
J
Jit TeamData engineering
Locations: Gdańsk, Gdańsk, Gdańsk, Country code: pl; Model pracy: ZdalnieContractSenior
Salary1,150 - 1,300 PLN per day
Apply NowOpens the employer's application page
Job Details
- Required Skills
- SQLApache AirflowJavaTerraformBigQuery
Requirements
- Strong Java skills for data processing, including batch and streaming pipelines, Apache Beam, and GCP client libraries.
- Strong SQL skills, particularly MySQL and BigQuery (GoogleSQL).
- Knowledge of Avro, Parquet, Protobuf, and JSON, including schema design and schema evolution.
- Experience with BigQuery data modeling, partitioning, clustering, query performance, and cost optimization.
- Knowledge of data ingestion tools including Pub/Sub, Datastream (CDC), BigQuery Data Transfer Service, and Cloud Storage.
- Experience with data processing tools including Dataflow, Dataform or dbt, and Dataproc or Serverless Spark.
- Apache Airflow skills, including DAG design, scheduling, dependencies, retries, monitoring, troubleshooting, and Cloud Composer.
- Knowledge of GCP orchestration tools such as Workflows and Cloud Scheduler.
- Knowledge of Looker or Looker Studio and BigQuery ML; basic knowledge of Vertex AI.
- Knowledge of GCP data services, governance, privacy, IAM, security, monitoring, and logging.
- Experience with cloud cost management, including billing exports, budgets, alerts, and cost optimization.
- Knowledge of Terraform, Git, and Cloud Build or GitHub Actions.
- Apply data quality testing, documentation, code review, and reusable pipeline patterns.
- Collaborate with marketing business stakeholders and translate their needs into data solutions.
Responsibilities
- Support the Direct Marketing team and its platforms in data processing, reporting, and analysis.
- Build batch and streaming data pipelines using Java, Apache Beam, and GCP client libraries.
- Model data in BigQuery and optimize query performance and costs.
- Ingest data using Pub/Sub, Datastream, BigQuery Data Transfer Service, and Cloud Storage.
- Orchestrate data workflows using Apache Airflow and Cloud Composer.
- Support marketing analytics and machine learning use cases, including segmentation, propensity, and churn.
- Share data with internal services and support integrations with GCP services.
- Apply data governance and privacy practices, including handling PII and consent data.
- Monitor and optimize cloud costs and pipeline performance.
- Collaborate with marketing stakeholders to translate business needs into data-driven solutions.
View Full Description & ApplyYou'll be redirected to the employer's site
