ABOUT THE PROJECT:
As a Senior Big Data engineer you will work on one of the world largest marketplace, dealing with one billion events per day and contribute as part of R&D self-organized team working in a challenging, innovative environment for our Customer.
• Work proactively on the system architecture, choosing optimal solutions to build scalable and reliable systems to process key financial data.
• Data processing system optimization and tuning
• Establish monitoring system
• Ensure high quality development standards (unit/integration tests, etc.)
• Collaboration with the product management team to incorporate the needs of our customers.
• Develop new generation big data pipelines, by migrating from batch processing into near-real time architecture
• 1+ years of commercial Scala development (at least)
• 5+ years of commercial JVM-based languages development experience
• Several years of experience in big data technologies and tools.
• ETL or Pipeline design/implementation with Large Distributed Databases (Hive/Spark/Scala preferred)
• Hands-on experience with: Spark, Kafka, HBase, NoSQL-Databases.
• Pipeline stability and performance tuning
• Working knowledge of relational databases and query authoring (SQL)
• Familiarity with the AWS-Stack (AWS EMR)
• Rigor in high code quality, automated testing, and other engineering best practices
• Experience in data validation, sampling & regression tests
• Good English (oral & written) and communication skills in general
WOULD BE A PLUS:
• Commercial experience with Real-time processing systems (Spark Streaming, Storm)
• Design and development of robust distributed systems
• Having experience working with AirFlow, Hive, Presto
• Payment data processing or system design knowledge
• Friendly and highly professional teams
• Flexible working hours with no overtimes
• Regular performance reviews
• Comfortable office facilities (kitchens, gym, sports activities, yoga, lounge rooms, coffee machines etc.)
• Fully paid English classes (twice per week) with own English teachers
• 20 working days of annual paid vacation
• Christmas holidays
• Paid sick-leaves
• Premium Medical & Life insurance
• Incentives (marriage, childbirth)
• Bonuses for internal training delivery
• Corporate events