Senior GCP Engineer (вакансія неактивна)

UST Ukraine is looking for a Senior GCP Engineer to join the team of our client who is gearing up to revolutionize the data landscape by building a cutting-edge Enterprise Data Lakehouse Platform. They are forming pivotal platform teams that will spearhead the creation of the platform’s foundational components. These teams go beyond traditional data ingestion; they are architects of a microservices-driven platform, providing abstractions that empower other teams to seamlessly extend the platform.

Location: Ukraine, Poland.

Work schedule: overlap with Eastern US needed.

Role overview: we are seeking a dynamic and highly skilled Engineer to lead these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but is also at the forefront of their game. The ideal candidate will contribute significantly to platform development, leading a team of contractors while actively shaping the future of our data ecosystem.

Responsibilities:

• Design and develop core data platform components with a microservices architecture, abstracting platform, and infrastructure intricacies.
• Create and maintain essential data platform SDKs and libraries, adhering to industry best practices.
• Design and develop connector frameworks and modern connectors to source data from disparate application both on-prem and cloud.
• Design and optimize data storage, processing, and querying performance for large-scale datasets using industry best practices.
• Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data.
• Collaborate with data scientists, analysts, and cross-functional teams to design data models, database schemas and data storage solutions.
• Design and develop advanced analytics and machine learning capabilities on the data platform.
• Design and develop observability and data governance frameworks and practices.
• Stay up to date with the latest data engineering trends, technologies, and best practices.
• Drive the deployment and release cycles, ensuring a robust and scalable platform.

Requirements:

• 12+ of proven experience in modern cloud data engineering, data architectures, data warehousing, and software engineering.
• Expertise in designing, and building end to end data platforms in the GCP environment using BigQuery and other services while adhering to best practices guidelines such as open standards, cost, performance, time to market and minimize vendor lock
• Solid experience building data platforms in GCP environment.
• Solid experience designing and developing modular, distributed data platform components with a microservices architecture. Strong experience with Docker, Kubernetes, APIs is needed.
• Proficiency in data engineering tools and technologies — SQL, Python, Spark, DBT, Airflow, Kafka.
• Solid experience implementing data lineage, data quality and data observability for big data workflows.
• Strong experience with modern data modeling, data architecture, and data governance principles.
• Excellent experience with DataOps principles and test automation.
• Excellent experience with observability tools — Grafana and Datadog.

Nice to have:

• Experience with Data Mesh architecture.
• Experience building Semantic layers for data platforms.
• Experience building scalable IoT architectures.

We offer:

• Flexible schedule and opportunities to work remotely (8 hours workday).
• Friendly professional staff and warm atmosphere.
• The environment where you can implement your ideas.
• Paid vacations and sick leaves.
• Medical insurance.
• Opportunity to work remotely.
• Participation in educational activities and thematic conferences.
• Team parties and corporate events.