— At least 2 years of a relevant experience;
— Good knowledge and Hands-on Scala Programming;
-Proficiency in Spark Scala for technical development and implementation;
— Data pipeline using Spark Scala;
— Load disparate data sets by leveraging Kafka consumers;
— Good Understanding of Big Data technologies like Hadoop, Spark, Scala, Hive, HBase, Pig, Cascading;
— Ability to utilize Hive, Spark, Cassandra, Mesos and Kafka;
— Good knowledge on Hive and HiveQL;
-Good Hands on in Hadoop stack (MapReduce, Hive, Sqoop, Oozie);
— Experience with AWS components and services, particularly EMR, S3, and Lambda;
— Good understanding of file formats including JSON, Parquet, Avro, and others;
— Experience with open source technologies such as Cassandra;
— Experience with messaging and complex event-processing systems such as Kafka and Storm;
— Extensive hands on Data frame and Dataset operations of Spark;
— Excellent in writing shell scripts;
— Experience with application architecture in a big data environment;
— Experience with AWS technologies or equivalent cloud stack As a hands-on engineer, influence all architecture decisions
— Build reusable code, with the ability to scale with very large data volumes;
— English — Upper-Intermediate.
— Knowledge on Cassandra Architecture and CQL;
— Knowledge on Java Programming;
— Data Migration/ETL knowledge.
— Compensation of medical service;
— Sport compensation: gym-membership reimbursement, small gym on-site;
— Language school;
— Compensation of training: our employees can participate in the training given by external
and internal experts;
— PE accounting and support.
— Programs, tests, and documents systems in accordance with programming standards and validation procedures;
— For more senior candidate — to serve as technical lead on a project;
— Plans and coordinates the complex design, development, implementation, maintenance, and level 4 support of associated systems;
— Provides software, system and/or programming solutions across projects and/or technologies to meet internal and external needs;
— Engineer systems and/or software, and/or transfer data for internal and external clients;
— Plans and coordinates peer review of work products such as code, designs, and test plans produced by other team members;
— Interact with corporate and project team members to negotiate timelines, responsibilities, and deliverables;
— Provides expert technical advice and solutions for internal and external clients;
— Provides expert review of database development work with regard to programming standards and validation procedures;
— Provides long term evaluations of systems, company wide;
— Meets personal and professional milestones as assigned, and interacts with the project team to organize timelines, responsibilities and deliverables;
— Performs other duties as directed.
Avenga is a global IT and digital transformation champion. Together, we are more than 2500 professionals with over 20 years of experience. We deliver strategy, customer experience, solution engineering, managed services and software products. Our mission is to shake up the conventional IT market.
Now, as we grow fast and smart, Avenga is looking for a Strong Middle/Senior BigData Engineer to join our team on a full-time basis. Our client is one among the largest Healthcare companies in the USA. We expect the successful candidate to demonstrate high motivation and ability to solve complex problems and achieve set goals.