Principal Commercial Data Engineer

J  o  b    D  e  s  c  r  i  p  t  i  o  n

Job Profile Title:

Principal Commercial Data Engineer

Job Code:

12778

Business_Title

Principal Commercial Data Engineer

Profile Title:

12778 Principal Commercial Data Engineer

Grade / Band:

IC5

FLSA Status:

Exempt

The Principal Commercial Data Engineer is a top-level technical expert and strategic architect responsible for shaping the enterprise data ecosystem. This role defines the long-term architecture for the Lakehouse platform, sets standards for governance and observability, and leads cross-functional engineering initiatives to modernize and scale data capabilities enterprise-wide. The position partners with senior leadership, architects, data science, analytics, IT, and domain leaders to design resilient, performant, and trusted data systems that support business-critical decisioning, advanced analytics, and AI/ML applications.

Define multi-year architectural vision for Lakehouse, including compute, storage, orchestration, semantic layers, governance, and reliability.

Architect scalable solutions for high-volume Gaming, Hospitality, Loyalty, and Marketing datasets across real-time, batch, and federated environments.

Establish enterprise data modeling standards for fact/event layers, KPIs, dimensional schemas, and unified semantic definitions.

Lead adoption of modern data patterns (Delta Live Tables, streaming pipelines, feature stores, ML observability).

Own enterprise-wide semantic model and certified curated datasets, ensuring metric consistency and governance.

Define organizational standards for data quality, observability, SLAs/SLOs, and incident lifecycle management.

Implement frameworks like Great Expectations, Monte Carlo, Soda, and Databricks-native Quality Gates.

Architect automated systems for schema drift mitigation, anomaly detection, and pipeline self-healing.

Optimize Databricks workloads for performance and cost efficiency, including ZORDER, OPTIMIZE, and workload isolation.

Mentor Senior and Staff Data Engineers, lead engineering councils, and influence enterprise modernization initiatives.

Performs other job-related duties as requested

Proof of eligibility to work in the United States

Bachelor's Degree

Computer Science, Information Technology, or related field

Required

Master's Degree

Computer Science, Information Technology, or related field

Preferred

5+ Years of Prior Relevant Experience

Architecting large-scale workloads in Databricks or equivalent Spark platforms

Required

10+ Years of Prior Relevant Experience

Data engineering, data platforms, distributed systems, or related fields

Required

Expert-level Databricks & Lakehouse engineering.

Required

Advanced coding skills in Python, PySpark, and SQL.

Required

Deep expertise in Unity Catalog governance and lineage.

Required

Domain Experience (at least one of the following): Gaming & Hospitality Operations, Loyalty & Customer Engagement, Marketing & Commercial Analytics.

Preferred

Ability to define multi-year Lakehouse architecture and governance standards.

Deep knowledge of Databricks, Delta Lake internals, and Unity Catalog governance.

Expertise in designing enterprise semantic layers and KPI logic.

Experience implementing frameworks for SLAs, anomaly detection, and automated monitoring.

Ability to design self-healing pipelines and manage incident lifecycle at scale.

Skills in workload isolation, autoscaling, and Delta Lake optimization (ZORDER, compaction).

Architect-level Python, PySpark, and SQL for pipelines and automation frameworks.

Familiarity with Terraform, GitHub Actions, and orchestration tooling.

Strong understanding of shuffle, skew, caching, and broadcast strategies.

Ability to evaluate and implement modern patterns like streaming and ML observability.

Proven ability to mentor engineers and lead cross-functional technical councils.

Ability to translate complex technical decisions into business-aligned outcomes.

Balancing

X

Bending

X

Carrying 10 pounds

X

Clear speech - simple

X

Clear speech - complex

X

Climbing

X

Distant vision

X

Driving

X

Flexibility - upper body

X

Flexibility - lower body

X

Hearing/Listening

X

Kneeling

X

Lifting 10 pounds

X

Near vision

X

Normal vision

X

Pushing/Pulling

X

Reaching

X

Sitting

X

Standing

X

Typing

X

Walking

X

Communication - verbal

X

Communication - written

X

Confined area

X

Contacts - works alone

X

Contacts - works around others

X

Contacts - works with others

X

Exposure to dust / dirt

X

Exposure to fumes / odors

X

Extreme cold

X

Extreme heat

X

Fast pace

X

Hazardous conditions - chemicals

X

Hazardous conditions - high structures

X

Hazardous conditions - high voltage

X

Indoors

X

Noise levels - low to moderate

X

Noise levels - high

X

Office conditions

X

Outdoors

X

Restricted area

X

Shifts

X

Smoke

X

Travel

X

Wet/Humid

X

Analytical

X

Clerical

X

Comprehension

X

Crisis incidents

X

Customer service

X

Decision making

X

High pressure

X

Judgment

X

Long hours

X

Math skills - advance

X

Math skills - basic

X

Organization

X

Reading - simple

X

Reading - complex

X

Repetition

X

Tight deadlines

X

Writing - simple

X

Writing - complex

X