The job
Senior Azure Data Engineer (Databricks) Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent
Build scalable Azure and Databricks data platforms that transform how financial services organisations use data.
The Role As a Senior Azure Data Engineer (Databricks) at Capco, you'll play a hands-on role in designing, building and deploying scalable, secure data engineering pipelines using the Databricks platform on Azure.
You'll partner with clients to understand their data needs and develop solutions that drive business transformation. Taking ownership of complex engineering deliverables, you'll contribute to technical design decisions, solve challenging data problems and build reliable, maintainable solutions that deliver practical value for clients.
Working with cross-functional teams, you'll contribute to engineering best practices and continuous improvement. You'll combine strong technical expertise with sound engineering judgement, constructive collaboration and a commitment to delivering production-ready data platforms that meet both technical and business objectives.
What You'll Do
- Design and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog and Azure-native services, taking ownership of implementation quality, reliability and maintainability.
- Build real-time, event-driven solutions using technologies such as Kafka and Azure Event Hubs, applying sound engineering judgement to performance, scalability and operational resilience.
- Develop and maintain CI/CD pipelines using Azure DevOps, Jenkins, GitHub Actions and modern DevOps practices to support reliable deployments. Use approved AI-assisted development tools and automation, where appropriate, to support activities such as test generation, code review preparation and technical documentation, improving delivery efficiency while maintaining human validation, security controls and escalation of potential risks.
- Collaborate with clients and multidisciplinary teams to translate business requirements into scalable, production-ready data solutions, constructively challenging assumptions, evaluating technical trade-offs and recommending practical engineering approaches.
- Champion clean code, data lifecycle optimisation and engineering best practices, supporting colleagues through knowledge sharing, constructive code reviews and continuous improvement to deliver high-quality client outcomes.
What We're Looking For
- Strong hands-on engineering capability, with experience taking ownership of complex technical deliverables and building reliable, production-ready data solutions.
- Sound technical judgement and problem-solving skills, with the ability to assess engineering trade-offs, identify delivery risks and balance performance, security, scalability and maintainability.
- A collaborative engineering mindset, with the confidence to contribute to technical discussions, challenge assumptions constructively and work effectively with multidisciplinary teams.
- A commitment to engineering quality, responsible technology adoption and continuous improvement, ensuring that delivery efficiency does not compromise security, governance or accountability.
- Strong communication and stakeholder engagement skills, with the ability to translate business requirements into practical technical solutions, explain engineering decisions clearly and build effective client relationships.
Technical Skills & Experience
- Strong hands-on experience with at least one OOP language such as Python, Scala or Java.
- Technical skills in spark, or experience working with other distributed data processing frameworks.
- Proven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions.
- Expertise delivering end-to-end data pipelines across ingestion, transformation and serving layers within modern Data Lakehouse architectures.
- Experience with schema design, GDPR-compliant data solutions, DevOps tooling and CI/CD processes.
Bonus Points For
- Familiarity with Cloudera, Hadoop, Hive, Spark, and related distributed data ecosystems.
- Experience working with sensitive data and applying data privacy regulations, including GDPR.
- A collaborative approach to engineering with experience using AI-enabled tools responsibly to improve productivity while maintaining appropriate human oversight, governance and solution quality.
- Curiosity to explore new technologies, including responsible AI and automation, to continuously improve engineering practices.
- Passion for collaboration, knowledge sharing, and continuous learning within high-performing engineering teams.
Why Join Capco
- Deliver high-impact technology solutions for Tier 1 financial institutions.
- Work in a collaborative, flat, and entrepreneurial consulting culture.
- Access continuous learning, training, and industry certifications.
- Be part of a team shaping the future of digital financial services.
- Help shape the future of digital transformation across FS & Energy.
Benefits We offer a competitive, people-first benefits package designed to support every aspect of your life:
- Core Benefits: Discretionary bonus, competitive pension, health insurance, life insurance and critical illness cover.
- Mental Health: Easy access to CareFirst, Unmind, Aviva consultations, and in-house first aiders.
- Family-Friendly: Maternity, adoption, shared parental leave, plus paid leave for sickness, pregnancy loss, fertility treatment, menopause, and bereavement.
- Family Care: 8 complimentary backup care sessions for emergency childcare or elder care.
- Holiday Flexibility: 5 weeks of annual leave with the option to buy or sell holiday days based on your needs.
- Continuous Learning: Minimum 40 Hours of Training Annually: Take your pick—workshops, certifications, e-learning—your growth, your way. Also, Business Coach assigned from Day One: Get one-on-one guidance to fast-track your goals and accelerate your development.
- Healthcare Access: Convenient online GP services.
- Extra Perks: Gympass (Wellhub), travel insurance, Tastecard, season ticket loans, Cycle to Work, and dental insurance.
Inclusion at Capco We’re committed to making our recruitment process accessible and straightforward for everyone. If you need any adjustments at any stage, just let us know – we’ll be happy to help. We value each person’s unique perspective and contribution. At Capco, we believe that being yourself is your greatest strength. Our #BeYourselfAtWork culture encourages individuality and collaboration – a mindset that shapes how we work with clients and each other every day.
