Senior Data Engineer (LLM, vector DBs)

Are you passionate about safeguarding countries, societies, and businesses from online manipulation and disinformation? Look no further! Our rapidly scaling company is passionately committed to this crucial mission, offering you a privileged opportunity to collaborate with some of the world’s most influential organizations, including NATO and the EU. As a Ukrainian team, we are determined to deliver substantial and meaningful changes!

If you’re ready to join a dynamic team working towards a common vision of a safer digital world, we invite you to be a part of our journey. Shape the future with us and help defend against online threats with purpose and innovation.

Role Overview:

We are seeking a seasoned Senior Data Engineer with a profound mastery of data warehousing and processing to enhance our data infrastructure. You will be responsible for the design and implementation of a data architecture of a high-loaded system, working with textual and media content, its vectorised representation (embeddings), real-time statistical data and graph data (connections between different actors). Your data solutions will feed our customer-facing AI platform, enhancing our search, AI data enrichment, RAG and more.

If you are passionate about data engineering, thrive in optimizing data workflows, and excel in architecting scalable solutions for intricate systems, this opportunity is tailor-made for you.

Role Responsibilities:

• Implement and support data lake, data warehouse and relevant data storages for various types of data: texts, vectors, statistics, graphs.
• Implement ETL pipelines to populate different storages with enriched data and guarantee data consistency.
• Implement tools for standardized internal and external querying of the data in various formats and places, including full-text search, vector similarity search, analytical aggregations and graph search.
• Collaborate with the data fetching team on the data format and ingestion process
• Break down complex problems into executable tasks.
• Monitor the performance of the system, make sure the data consistency and update latency meet the product requirements.

Required Skills and Abilities:

• Expertise in Python programming language and data-processing libraries, like pandas, numpy.
• Experience with Elasticsearch or similar technology for full text search
• Proficiency in SQL and analytical DBs Postgresql.

Minimum Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 3+ years experience as a Data Engineer or similar role, particularly working on tasks related to data processing and storage with a strong understanding of complex data architectures.

Preferred Qualifications:

• Experience in building RAG pipelines.
• Production experience with embeddings and high-loaded Vector Storages (Milvus, Quadrant, Pinecone, or similar).
• Experience in deployments with Docker containers into K8s.
• Familiarity with containerization (Docker) and orchestration systems (Kubernetes), Linux and Shell Scripting.

We Offer:

• Opportunity to present your product and assist prestigious clients, such as world’s governments and leading NGOs, in combating information threats and security challenges.
• Autonomy and freedom to drive experiments and bring your own ideas to life.
• Becoming a key contributor, you’ll be also rewarded with our stock compensation program.
• Flexibility of fully remote work while having the option to leverage our vibrant co-working space in Kyiv, Ukraine.

Apply now and contribute to a mission that truly matters!