Data /Databricks Engineer (contract/remote) (вакансія неактивна)

The Role

As a Core Data Engineer, you will play a pivotal role in designing, building, and optimising data pipelines, architectures, and workflows for a diverse client base. You’ll work within dynamic, multi-disciplinary project teams to implement scalable data solutions while adhering to best practices and emerging trends in data engineering.

Key Responsibilities:

• Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources.
• Data Architecture: Develop and optimize data architectures, including data warehouses, data lakes, and other storage solutions.
• ETL/ELT Processes: Create and maintain reliable ETL/ELT workflows that ensure data quality and accessibility.
• Collaboration: Work closely with data scientists, analysts, and client stakeholders to understand requirements and deliver impactful solutions.
• Performance Optimisation: Monitor and enhance the performance of data systems to ensure minimal downtime and fast query responses.
• Automation and Tools: Identify opportunities for automation and recommend tools to improve data engineering workflows.
• Documentation: Maintain detailed technical documentation for all solutions and processes.

Technical Skills:

• Programming: Proficiency in Python, Java, Scala, or similar languages for data processing.
• Big Data Technologies: Hands-on experience with big data tools e.g., Databricks, Apache Spark, Hadoop.
• Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks.
• Database Management: Expertise in relational databases (e.g. postgres, sql server)
• Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend.
• Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design.
• Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment.

Experience

• 2+ years of experience in data engineering or a similar role.
• Proven track record of delivering data solutions in a consulting or client-facing environment is a plus.
• Experience with Agile or Scrum methodologies is beneficial.

Personal Qualities

• Strong problem-solving and analytical thinking abilities.
• Excellent communication skills to explain complex technical concepts to non-technical stakeholders.
• A team-oriented mindset with a focus on collaboration and adaptability