Senior or Staff AI Infrastructure Engineer

Design, build, and operate infrastructure supporting large-scale AI and agent systems, including CI/CD workflows, model versioning, observability, evaluation pipelines, and production LLM and agent deployments.

Responsibilities

• Build reusable CI/CD workflows for model training, evaluation, and deployment
• Automate model versioning, approval workflows, and compliance checks
• Build modular scalable AI infrastructure including vector databases, feature stores, model registries, and observability tooling
• Partner with engineering and data science to embed AI models and agents into real-time applications and workflows
• Continuously evaluate and integrate state-of-the-art AI tools and frameworks
• Drive AI reliability and governance to ensure compliance, security, and uptime
• Ensure data accuracy, consistency, and reliability for model training and inference
• Deploy infrastructure for offline and online evaluation, regression testing, cost monitoring, and human-in-the-loop workflows
• Enable researchers with sandboxes, dashboards, and reproducible environments
• Improve AI and ML model performance

Requirements

• Write high-quality maintainable software primarily in Python
• Strong background in scalable infrastructure, including Docker and Kubernetes
• Experience with infrastructure as code and deployment tools such as Terraform and CI/CD pipelines
• Familiarity with Datadog, Prometheus, and OpenTelemetry
• Knowledge of MLOps practices including model versioning, rollback strategies, automated evaluation, and drift detection
• Experience with scalable model and agent serving infrastructure such as vLLM, Triton, and BentoML
• Experience deploying and maintaining LLM and agentic workflows in production
• Experience monitoring cost, latency, and performance and capturing traces for debugging
• Experience optimizing prompt and response flows with real-time data access
• Strong ownership, pragmatism, and ability to balance infrastructure design with iterative delivery

Benefits

• Remote work
• Eligibility to participate in TRM's equity plan