BigData/Python ETL Team Lead #6413 (вакансія неактивна)

PURPOSE OF THE JOB
This Big Data Team Lead will work with the team to deliver automated consumer behavior prediction platform to companies. Our customer is automating and commoditizing cutting edge AI results directly from a client data lake, and your job will be to help platform handling data at the massive scale needed. The data sizes can reach PetaByte ranges, and it is still expected to process rapidly.

MAIN TASKS AND RESPONSIBILITIES
Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
Implementing ETL process.
Part of ETL is analyzing and understanding customer’s client data well enough to integrate into the API.
Propose, design and implement Big Data Architecture including infrastructure.
Monitoring performance and advising any necessary infrastructure changes.
Defining data retention policies.

EDUCATION, SKILLS AND EXPERIENCE
1+ year of management experience
3 years of experience with Hadoop v2 and MapReduce.
Proficiency with the management of our Hadoop cluster and accompanying services including Hive, Spark, Kafka, Scoop and Oozie.
Proficiency with Presto.
3 years of experience in NoSql databases. Cassandra preferred.
Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming.
Experience with integration of data from multiple data sources.
Knowledge of various ETL techniques.
3 years experience in creating Lambda Architecture, along with knowledge of its advantages and drawbacks.
Experience with Cloudera/MapR/Hortonworks.
Strong written English skills are required, any internal or external document uses the SCQSR (Situation-Complication-Question-Solution-Resolution) narrative method and must
Outstanding academic results. Only the smartest people can deliver the results we expect.

LOHIKA BENEFITS
Friendly and highly professional teams
Modern office facilities (kitchens, gym, yoga, playroom, coffee and tea points etc.)
Regular (twice a year) performance reviews
Fully paid English classes with certified teachers and native speakers
Internal and external training
Paid sick-leaves
Premium health insurance (medication, massage, and doctor in the office etc.)
20 working days of annual paid vacation
Christmas and other state holidays
Corporate events (corporate parties, sports competitions, What? Where? When? Team etc.)
Incentives (marriage, childbirth gifts)

And much more!

Please send you CV or contact us with more questions!