Lead Data Software Engineer

We are seeking a Lead Data Software Engineer to lead the evolution and maturity of data solutions in a dynamic GCP environment.

The role involves implementing orchestration with Airflow, optimizing data ingestion pipelines, and driving the delivery of scalable, high-performance systems.

Responsibilities

• Improve architecture to enhance the scalability and maturity of data solutions
• Implement orchestration for workflows using Airflow for efficient processes
• Build secure REST services in Python 3.x with technologies like FastAPI and asynchronous patterns
• Optimize data pipelines and workflows by leveraging BigQuery's advanced features like partitioning, clustering, and query tuning
• Design high-load systems with a focus on throughput, backpressure, and scalability
• Utilize GCP services such as Cloud Functions, Pub/Sub, and Cloud Storage to build robust data solutions
• Develop end-to-end ETL pipelines with job orchestration frameworks like Airflow
• Drive the design process from RFCs to full operational runbooks with clear documentation and ownership

Requirements

• 7+ years professional software experience; 4+ years of expertise in Python 3.x within production environments
• Knowledge of FastAPI and Pydantic, with proficiency in async patterns and secure REST APIs
• Competency in SQL, data modeling, and hands-on experience with BigQuery (partitioning, clustering, query optimizations)
• Production expertise in GCP services including Cloud Functions, Pub/Sub, Cloud Storage, IAM/Secret Manager, Cloud Build, or equivalent AWS/Azure stacks
• Background in designing high-load systems with solutions for backpressure, idempotency, and scalability
• Experience working on ETL pipelines using tools like Airflow for job orchestration
• Showcase of excellent communication, ownership, and problem-solving skills

Nice to have

• Familiarity with parsing file formats at scale like CSV, Excel, or XML along with schema evolution/versioning techniques
• Understanding of data quality frameworks such as assertions or expectations and tools like dbt or Dataform
• Background in event-driven architectures or streaming technologies; experience with Beam/Dataflow is a plus
• Expertise in concurrency and performance profiling in Python using asyncio or multiprocessing
• Knowledge of IaC practices using Terraform or SRE methodologies
• Awareness of security concepts such as authentication/authorization, service accounts, and principles of least privilege

Benefits

With us you can:

• Work on a flexible schedule remotely or from any of our comfortable offices or coworking spaces in Ukraine
• Receive the necessary equipment to perform your work tasks
• Change projects and technology stacks within EPAM
• Gain experience in various business domains (Insurance, E-commerce, Healthcare, Finance, Travelling, Media, Artificial Intelligence, and more)
• Relocation opportunities may be available for eligible candidates, depending on the role and openings at other EPAM locations
• Participate in volunteer, charity programs and communities (both technical and interest-based)

We focus on your professional growth:

• You can plan your individual career path together with your manager
• Receive regular feedback from colleagues
• Improve your English for free with certified teachers (Speaking Clubs, client interview preparation courses, etc.)
• Get the opportunity to undergo free training and certification in AWS, GCP, or Azure Clouds
• Use the internal E-learn training program (18,200+ specialized training and mentoring programs)
• Access corporate accounts on LinkedIn Learning, Get Abstract and other partner resources
• Study at EPAM Solution Architecture School with the instructors who are practicing architects
• Develop as a leader, join Delivery Management, Resource Management, Leadership Essentials school and more
• Participate in internal communities (500+ meetups, technical discussions, brainstorming sessions, online events and conferences annually)

What we offer:

• Vacation and sick leave (including a sick leave without a medical certificate)
• A wide range of Voluntary Medical Insurance programs providing both medical treatment and various preventive options (including sports activities)
• Medical insurance for family members at corporate rates
• Company support during significant life events (childbirth or adoption, marriage, etc.)
• Support for psychological comfort: discounts on services from mental health specialists or coaches, thematic training
• E-kids program - a free programming language training program for EPAMers' children