Spark Expert

Senior Data Solutions Consultant — Apache Spark & Databricks

About the Role

We're seeking a Senior Data Solutions Consultant with deep expertise in Apache Spark and Databricks, and a decisive, agentic-forward approach to problem solving. This individual will independently lead complex customer engagements—bridging hands-on data engineering excellence with strategic technical leadership.

In this role, you'll serve as a trusted advisor who can not only identify performance and design opportunities, but proactively build and implement the solutions that unlock value for our customers.

Key Responsibilities

• Lead customer engagements from discovery to delivery, operating with ownership and accountability to drive successful outcomes.
• Architect, implement, and optimize large-scale Spark and Databricks workloads, focusing on performance, scalability, and cost-efficiency.
• Apply expert-level Spark fundamentals to troubleshoot, tune, and streamline distributed data jobs and SQL workloads.
• Design and execute PoCs and workshops that inspire confidence and demonstrate solution vision.
• Act as the customer's technical advocate, anticipating issues and proactively applying creative, data-driven solutions.
• Partner cross-functionally with product, engineering, and success teams to continuously improve customer experience.
• Mentor junior engineers, championing best practices in Databricks, PySpark, and data lakehouse design.

Required Qualifications

• 5–10 years of experience in data engineering or analytics solution delivery.
• 3+ years of proven, hands-on Spark and Databricks expertise, including job optimization, debugging, and workload governance.
• Mastery of Spark internals, PySpark, and Databricks SQL (DBSQL) performance tuning techniques.
• Strong understanding of modern data lakehouse design: Delta Lake, structured streaming, and cloud infrastructure.
• Demonstrated ability to work autonomously—identifying, prioritizing, and solving complex technical problems with minimal oversight.
• Exceptional communication skills, with the ability to simplify technical complexity for business leaders.
• A self-starter mindset: proactive, curious, and comfortable taking initiative to deliver measurable impact.

Preferred Qualifications

• Databricks Certified Data Engineer Professional or equivalent advanced Databricks certifications.
• Experience with structured streaming, Delta Live Tables, and Unity Catalog.
• Experience with large-scale data migration, streaming pipeline design, or AI-assisted analytics.
• Background working across enterprise and technical consulting environments.

130/140$ an hour (270-291k)