Senior Data Platform Engineer

You will build and operate foundational tooling for large-scale on-premise big data platforms. You will develop distributed systems, improve deployment environments, enhance data discoverability and quality, build streaming applications, and maintain data and compute ecosystems used by analysts, data scientists, and engineers.

Responsibilities

• Develop and maintain scalable high-performance big data platforms
• Work with distributed databases, filesystems, and compute systems
• Improve continuous release and deployment environments
• Build tools for data discoverability and metadata presentation
• Extend platform-wide data quality tooling
• Introduce technologies that reduce time to insight
• Develop streaming processing applications and frameworks
• Develop and operate large-scale private cloud infrastructure

Requirements

• 6+ years of Python object-oriented programming experience
• Experience with distributed data and compute systems such as Spark, Trino, and Druid
• Experience with data modelling tooling
• Experience with DevOps pipelines and development ecosystems
• Experience with real-time and batch data pipelines
• Experience with Kafka and Spark Streaming
• Experience with Kubernetes, Docker, and/or Hadoop ecosystems
• Strong communication skills
• Ability to work with diverse technical stakeholders