Data Engineer (Hadoop, Scala, Spark, Nifi), Investment Management Solution (вакансія неактивна)

Our client is one of the world’s top 10 investment companies with headquarters in Great Britain with branch offices in the US and Hong Kong. The IT environment of this company is constantly growing: there are around 30 programs and more than 60 projects actively developing at present time. During the next 5 years, they are going to replace 150 out of 400 systems and integrate US, HK and UK environments.

Our client is forming a team that will be working on different projects that will integrate US and UK environments and adopt investment management software to US market requirements and regulations.

The project is a Single Source of Truth for investment management data and investment analysis solution that will aggregate and process data from multiple data feeds and systems like stock exchanges, news feeds, stock exchange brokers, internal quantitative systems, and others. The development is just started and it’s a great chance to join the team from the very beginning.

As a Data Engineer, you will be developing a platform that models and predicts trends in the financial markets.

TECHNOLOGY STACK

Hadoop (HDFS, MapReduce, HBase), NiFi, SQL, Scala, Spark 2.0 or greater.

Requirements

• Experience working with Hadoop (HDFS, MapReduce, HBase)
• Hands-on experience with Shell scripting
• Hands-on experience with Spark 2.0 or greater (+ Scala)
• Strong experience with SQL
• Spoken English

Nice to have

• Experience working with Hive And Impala
• Experience working with Apache NiFi
• Experience working in an agile environment and understanding of various phases of the software development life cycle
• Experience with Data modelling
• Skills working with Airflow (preferred) or Oozie
• Experience with CDP (Cloudera Data Platform)
• Knowledge of Hadoop architecture components

We offer

• Professional Development:
— Experienced colleagues who are ready to share knowledge;
— The ability to switch projects, technology stacks, try yourself in different roles;
— More than 150 workplaces for advanced training;
— Study and practice of English: courses and communication with colleagues and clients from different countries;
— Support of speakers who make presentations at conferences and meetings of technology communities.
• The ability to focus on your work: a lack of bureaucracy and micromanagement, and convenient corporate services;
• Friendly atmosphere, concern for the comfort of specialists;
• Flexible schedule (there are core mandatory hours), the ability to work remotely upon agreement with colleagues.