Senior Data Engineer (Databricks)
Summary
Senior Data Engineer designing and maintaining scalable data pipelines on Databricks and Apache Spark, defining batch and streaming architectures, and integrating data from multiple sources on Azure for analytics and business teams.
- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
- Define data architectures that support batch and streaming processing.
- Integrate data from multiple sources, including databases, APIs, and cloud storage.
- Optimize data solutions for performance, reliability, and cost efficiency.
- Collaborate closely with data science, analytics, and business teams.
- Provide technical leadership and guidance on data engineering best practices.
- Translate complex Databricks architectures and data solutions into clear, business-friendly explanations for technical and non-technical audiences.
- Document data architectures, pipelines, and technical decisions.
- Support production systems and troubleshoot data-related issues as needed.
- Minimum of 5 years of experience as a Data Engineer or in a similar role.
- Strong hands-on experience with Databricks and Apache Spark.
- Advanced SQL and Python skills.
- Experience working in cloud environments (Azure preferred).
- Solid understanding of data modeling, ETL/ELT processes, and data architecture concepts.
- Ability to work independently while collaborating with distributed teams.
• Competitive salary
• Flexible work environment
• Opportunities for growth and certifications
• Inclusive and collaborative team culture
• Access to cutting-edge tools and technologies