Senior Software Engineer, Data Platform
Design, implement, and operate data infrastructure that processes petabytes of blockchain and transactional data in real time. Build reliable data services integrating multiple blockchains, optimize ETL and streaming workflows, create fast-querying data models, deploy and monitor database clusters, automate scaling and maintenance, and deliver pragmatic first-version solutions that can be iterated quickly.
Responsibilities
- Build highly reliable data services to integrate with multiple blockchains
- Develop complex ETL pipelines to transform and process petabytes of data in real time
- Design and architect data models for optimal storage and sub-second query latency
- Deploy, scale, and monitor large database clusters for performance and high availability
- Implement data pipeline orchestration and streaming workflows
- Automate scaling and maintenance tasks such as pgbouncer provisioning, disk scaling, and cluster updates
- Deliver pragmatic first-version solutions quickly and iterate based on stakeholder feedback
Requirements
- Bachelor's degree or equivalent in Computer Science or a related field
- 8+ years of hands-on experience architecting distributed systems and shipping production deployments
- Exceptional programming skills in Python
- Proficiency in SQL or SparkSQL
- Experience with ClickHouse, Elasticsearch, Postgres, Redis, and Neo4j
- Familiarity with Airflow, DBT, Luigi, Azkaban, or Storm
- Expertise in Spark, Kafka, and Flink
- Experience deploying and monitoring public-cloud infrastructure using Docker, Terraform, Kubernetes, and Datadog
- Ability to load, query, and transform very large datasets