Above 3 years of experience in designing, implementing, and deploying software in a production environment.
Solid programming skills in Python
Understanding of distributed systems, software architecture & agile development
Experience with CI/CD, data processing frameworks like Apache Spark or similar and experience doing data analysis with SQL.
Very good knowledge of relational databases (e.g. PostgreSQL)
It is essential that you pay high interest and attention to data quality assurance and testing methods.
Knowledge of workflow management tools such as Airflow and hands-on experience in cloud technologies (AWS ideally).
Knowledge in technologies like Kubernetes and Docker but not limited to them
Pragmatic and curious, find the right tool for the right job. We solve business problems in the best possible way, but not better than that.
Great communication skills in English.
Gathering, transforming and providing high-quality data on time to stakeholders (both external — like the brand partners and also internal like the Data Lake or Data Processing teams)
You and your team will own your code, and decide on the technologies and tools to deliver in a certain frame. Also, you will be designing, developing, and operating robust data integration pipelines to provide high-quality datasets for analytical and machine learning use-cases.
You will be involved in building new ETL processes that might include combining data from multiple sources and maintaining the existing ones.
Consulting with Analysts and Product Managers to build and continuously improve a variety of data products that create value for our business.
Be a sparring partner to other team members, and provide support and guidance to other engineers to help develop their technical capabilities.
As a Data Engineer within the Merchant Operations Data team, you will take end to end ownership of the Data Platform for the partner-facing portal. You are responsible to ingest new Data Sources, often in collaboration with our stakeholders. You will increase the processing speed and optimize the costs of our infrastructure.
You will choose emerging technologies and approaches to create high-performance data processing solutions that scale. You will challenge our status quo and help us define best practices for how we work by designing data-intensive applications.