We are looking for a highly passionate and results-oriented Data Engineer specializing in Azure Synapse, Spark, and SQL who will focus on leading the design, development, optimization, and governance of enterprise-scale data platforms and pipelines on the Microsoft Azure cloud. Our customer is one of the international companies providing audit, tax, consulting, and financial advisory services.
This role requires a deep blend of technical expertise, architectural insight, and leadership skills.
Responsibilities:
Data Platform&Architecture
- Architecture Architect and implement scalable, secure, and cost-efficient cloud-native data solutions, primarily utilizing Azure Synapse Analytics (SQL Pools, Spark Pools, Serverless SQL).
- Define and enforce data modeling best practices optimized for Synapse.
- Lead the design and implementation of Lakehouse/Medallion Architecture patterns (Bronze, Silver, Gold layers) using Azure Data Lake Storage (ADLS) Gen2 and Delta Lake/Parquet formats.
Data Pipeline Development (ETL/ELT)
- Design, build, and orchestrate complex ETL/ELT data pipelines using Azure Synapse Pipelines (formerly Azure Data Factory) and other Azure services.
- Develop highly optimized data transformation, cleansing, and aggregation jobs leveraging Apache Spark (PySpark/Scala) in Azure Synapse Spark Pools (or Azure Databricks).
- Implement advanced SQL logic (T-SQL, Spark SQL) for data manipulation, stored procedures, functions, and query performance tuning in Synapse Dedicated/Serverless SQL Pools.
Performance&Optimization
- Tune and optimize Synapse SQL and Spark queries, jobs, and data models to ensure peak performance and minimize cloud compute costs.
- Manage data partitioning, indexing, and resource allocation within Azure Synapse workspaces.
- Implement comprehensive data quality checks and validation frameworks withs.in the data pipeline
Collaboration&Leadership
- Collaborate with business stakeholders to translate business needs into robust technical data requirements.
- Mentor junior/mid-level data engineers and drive technical excellence and best practices across the team.
Qualifications and Experience
- Experience: 5+ years in a Data Engineering role, with significant experience operating at a Senior or Expert level on cloud platforms.
- Upper-Intermediate level of spoken and written English
- Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related quantitative field.
- Certifications (Highly Desired): Microsoft Certified: Azure Data Engineer Associate (DP-203) or equivalent advanced cloud/big data certifications.