Data Quality in Databricks with Great Expectations (GX Library)

This data sheet aims to provide a formal foundation and essential prerequisites for mastering data quality on the Databricks platform using the Great Expectations library.

resources

Read our Case Studies

Case Studies, Healthcare

LumenData Leads Healthcare Data Management: Helped WeightWatchers use Oracle Customer Hub to consolidate 30MM client records

Discover how LumenData helped WeightWatchers consolidate 30M records, improving campaigns and driving a $7MM ROI with Oracle Customer Hub.

Learn more

Case Studies, Financial Services

Renowned Global Bank uses Oracle Customer Hub to enable automated inbound feeds from sales credit and CRM systems

Learn more

Case Studies, Healthcare

A Leading Pharmaceutical Manufacturing Company achieves data modernization with Reltio MDM deployment

Learn more

top

:root{--vamtam-default-bg-color:#fff;--vamtam-default-line-color:rgba(var(--vamtam-accent-color-7-rgb),1);--vamtam-small-padding:20px;--vamtam-horizontal-padding:50px;--vamtam-vertical-padding:30px;--vamtam-horizontal-padding-large:60px;--vamtam-vertical-padding-large:60px;--vamtam-no-border-link:none;--vamtam-border-radius:0px;--vamtam-border-radius-oval:0px;--vamtam-border-radius-small:0px;--vamtam-overlay-color:#000;--vamtam-overlay-color-hc:#fff;--vamtam-box-outer-padding:60px;--vamtam-body-link-regular:#242627;--vamtam-body-link-visited:#242627;--vamtam-body-link-hover:rgba(0,0,0,.6);--vamtam-body-link-active:rgba(0,0,0,.6);--vamtam-input-border-radius:4px 4px 4px 4px;--vamtam-input-border-color:rgba(0,0,0,.15);--vamtam-btn-text-color:#fff;--vamtam-btn-hover-text-color:#fff;--vamtam-btn-bg-color:#010ed0;--vamtam-btn-hover-bg-color:#242627;--vamtam-site-max-width:1280px;--vamtam-primary-font-color:rgba(0,0,0,.6);--vamtam-h1-color:#0a0d31;--vamtam-h2-color:#0a0d31;--vamtam-h3-color:#0a0d31;--vamtam-h4-color:#0a0d31;--vamtam-h5-color:#0a0d31;--vamtam-h6-color:#0a0d31;--vamtam-accent-color-1:#010ed0;--vamtam-accent-color-1-hc:#fff;--vamtam-accent-color-1-rgb:1,14,208;--vamtam-accent-color-2:#f5f5f5;--vamtam-accent-color-2-hc:#000;--vamtam-accent-color-2-rgb:245,245,245;--vamtam-accent-color-3:#5f6567;--vamtam-accent-color-3-hc:#fff;--vamtam-accent-color-3-rgb:95,101,103;--vamtam-accent-color-4:#dee0ff;--vamtam-accent-color-4-hc:#000;--vamtam-accent-color-4-rgb:222,224,255;--vamtam-accent-color-5:#fff;--vamtam-accent-color-5-hc:#000;--vamtam-accent-color-5-rgb:255,255,255;--vamtam-accent-color-6:#242627;--vamtam-accent-color-6-hc:#fff;--vamtam-accent-color-6-rgb:36,38,39;--vamtam-accent-color-7:rgba(0,0,0,.15);--vamtam-accent-color-7-hc:#fff;--vamtam-accent-color-7-rgb:0,0,0;--vamtam-accent-color-8:rgba(0,0,0,.6);--vamtam-accent-color-8-hc:#fff;--vamtam-accent-color-8-rgb:0,0,0;--vamtam-sticky-header-bg-color:#010ed0;--vamtam-primary-font-font-family:"Raleway";--vamtam-primary-font-font-weight:500;--vamtam-primary-font-font-style:normal;--vamtam-primary-font-transform:none;--vamtam-primary-font-font-size-desktop:16px;--vamtam-primary-font-font-size-tablet:16px;--vamtam-primary-font-font-size-phone:16px;--vamtam-primary-font-line-height-desktop:1.6em;--vamtam-primary-font-line-height-tablet:1.6em;--vamtam-primary-font-line-height-phone:1.5em;--vamtam-primary-font-letter-spacing-tablet:0px;--vamtam-primary-font-letter-spacing-phone:0px;--vamtam-h1-font-family:"Raleway";--vamtam-h1-font-weight:600;--vamtam-h1-font-style:normal;--vamtam-h1-transform:none;--vamtam-h1-font-size-desktop:50px;--vamtam-h1-font-size-tablet:35px;--vamtam-h1-font-size-phone:30px;--vamtam-h1-line-height-desktop:1em;--vamtam-h1-line-height-tablet:1em;--vamtam-h1-line-height-phone:1.3em;--vamtam-h1-letter-spacing-tablet:0px;--vamtam-h1-letter-spacing-phone:0px;--vamtam-h2-font-family:"Raleway";--vamtam-h2-font-weight:700;--vamtam-h2-font-style:normal;--vamtam-h2-transform:none;--vamtam-h2-font-size-desktop:48px;--vamtam-h2-font-size-tablet:25px;--vamtam-h2-font-size-phone:25px;--vamtam-h2-line-height-desktop:1.2em;--vamtam-h2-line-height-tablet:1.1em;--vamtam-h2-line-height-phone:1.3em;--vamtam-h2-letter-spacing-tablet:0px;--vamtam-h2-letter-spacing-phone:0px;--vamtam-h3-font-family:"Raleway";--vamtam-h3-font-weight:600;--vamtam-h3-font-style:normal;--vamtam-h3-transform:none;--vamtam-h3-font-size-desktop:30px;--vamtam-h3-font-size-tablet:24px;--vamtam-h3-font-size-phone:20px;--vamtam-h3-line-height-desktop:1.4em;--vamtam-h3-line-height-tablet:1.1em;--vamtam-h3-line-height-phone:1.1em;--vamtam-h3-letter-spacing-tablet:0px;--vamtam-h3-letter-spacing-phone:0px;--vamtam-h4-font-family:"Raleway";--vamtam-h4-font-weight:600;--vamtam-h4-font-style:normal;--vamtam-h4-transform:none;--vamtam-h4-font-size-desktop:22px;--vamtam-h4-font-size-tablet:22px;--vamtam-h4-font-size-phone:18px;--vamtam-h4-line-height-desktop:1.3em;--vamtam-h4-line-height-tablet:1.3em;--vamtam-h4-line-height-phone:1.4em;--vamtam-h4-letter-spacing-tablet:0px;--vamtam-h4-letter-spacing-phone:0px;--vamtam-h5-font-family:"Raleway";--vamtam-h5-font-weight:600;--vamtam-h5-font-style:normal;--vamtam-h5-transform:none;--vamtam-h5-font-size-desktop:20px;--vamtam-h5-font-size-tablet:18px;--vamtam-h5-font-size-phone:16px;--vamtam-h5-line-height-desktop:1.3em;--vamtam-h5-line-height-tablet:1.3em;--vamtam-h5-line-height-phone:1.3em;--vamtam-h5-letter-spacing-tablet:0px;--vamtam-h5-letter-spacing-phone:0px;--vamtam-h6-font-family:"Raleway";--vamtam-h6-font-weight:600;--vamtam-h6-font-style:normal;--vamtam-h6-transform:none;--vamtam-h6-font-size-desktop:16px;--vamtam-h6-font-size-tablet:15px;--vamtam-h6-font-size-phone:14px;--vamtam-h6-line-height-desktop:1.6em;--vamtam-h6-line-height-tablet:1em;--vamtam-h6-line-height-phone:1em;--vamtam-h6-letter-spacing-tablet:0px;--vamtam-h6-letter-spacing-phone:0px;--vamtam-loading-animation:url("https://cdn-ikpkfih.nitrocdn.com/jRlBJxtaqsXYKiIATouCdoPKpAFNvkPx/assets/images/optimized/rev-97ed04f/lumendata.com/wp-content/themes/tecnologia/vamtam/assets/images/loader-ring.gif");}img.wp-smiley,img.emoji{display:inline !important;border:none !important;box-shadow:none !important;height:1em !important;width:1em !important;margin:0 .07em !important;vertical-align:-.1em !important;background:none !important;padding:0 !important;}.wp-block-button__link{color:#fff;background-color:#32373c;border-radius:9999px;box-shadow:none;text-decoration:none;padding:calc(.667em + 2px) calc(1.333em + 2px);font-size:1.125em;}.wp-block-file__button{background:#32373c;color:#fff;text-decoration:none;}:root{--wp--preset--aspect-ratio--square:1;--wp--preset--aspect-ratio--4-3:4/3;--wp--preset--aspect-ratio--3-4:3/4;--wp--preset--aspect-ratio--3-2:3/2;--wp--preset--aspect-ratio--2-3:2/3;--wp--preset--aspect-ratio--16-9:16/9;--wp--preset--aspect-ratio--9-16:9/16;--wp--preset--color--black:#000;--wp--preset--color--cyan-bluish-gray:#abb8c3;--wp--preset--color--white:#fff;--wp--preset--color--pale-pink:#f78da7;--wp--preset--color--vivid-red:#cf2e2e;--wp--preset--color--luminous-vivid-orange:#ff6900;--wp--preset--color--luminous-vivid-amber:#fcb900;--wp--preset--color--light-green-cyan:#7bdcb5;--wp--preset--color--vivid-green-cyan:#00d084;--wp--preset--color--pale-cyan-blue:#8ed1fc;--wp--preset--color--vivid-cyan-blue:#0693e3;--wp--preset--color--vivid-purple:#9b51e0;--wp--preset--gradient--vivid-cyan-blue-to-vivid-purple:linear-gradient(135deg,rgba(6,147,227,1) 0%,#9b51e0 100%);--wp--preset--gradient--light-green-cyan-to-vivid-green-cyan:linear-gradient(135deg,#7adcb4 0%,#00d082 100%);--wp--preset--gradient--luminous-vivid-amber-to-luminous-vivid-orange:linear-gradient(135deg,rgba(252,185,0,1) 0%,rgba(255,105,0,1) 100%);--wp--preset--gradient--luminous-vivid-orange-to-vivid-red:linear-gradient(135deg,rgba(255,105,0,1) 0%,#cf2e2e 100%);--wp--preset--gradient--very-light-gray-to-cyan-bluish-gray:linear-gradient(135deg,#eee 0%,#a9b8c3 100%);--wp--preset--gradient--cool-to-warm-spectrum:linear-gradient(135deg,#4aeadc 0%,#9778d1 20%,#cf2aba 40%,#ee2c82 60%,#fb6962 80%,#fef84c 100%);--wp--preset--gradient--blush-light-purple:linear-gradient(135deg,#ffceec 0%,#9896f0 100%);--wp--preset--gradient--blush-bordeaux:linear-gradient(135deg,#fecda5 0%,#fe2d2d 50%,#6b003e 100%);--wp--preset--gradient--luminous-dusk:linear-gradient(135deg,#ffcb70 0%,#c751c0 50%,#4158d0 100%);--wp--preset--gradient--pale-ocean:linear-gradient(135deg,#fff5cb 0%,#b6e3d4 50%,#33a7b5 100%);--wp--preset--gradient--electric-grass:linear-gradient(135deg,#caf880 0%,#71ce7e 100%);--wp--preset--gradient--midnight:linear-gradient(135deg,#020381 0%,#2874fc 100%);--wp--preset--font-size--small:13px;--wp--preset--font-size--medium:20px;--wp--preset--font-size--large:36px;--wp--preset--font-size--x-large:42px;--wp--preset--spacing--20:.44rem;--wp--preset--spacing--30:.67rem;--wp--preset--spacing--40:1rem;--wp--preset--spacing--50:1.5rem;--wp--preset--spacing--60:2.25rem;--wp--preset--spacing--70:3.38rem;--wp--preset--spacing--80:5.06rem;--wp--preset--shadow--natural:6px 6px 9px rgba(0,0,0,.2);--wp--preset--shadow--deep:12px 12px 50px rgba(0,0,0,.4);--wp--preset--shadow--sharp:6px 6px 0px rgba(0,0,0,.2);--wp--preset--shadow--outlined:6px 6px 0px -3px rgba(255,255,255,1),6px 6px rgba(0,0,0,1);--wp--preset--shadow--crisp:6px 6px 0px rgba(0,0,0,1);}:where(.is-layout-flex){gap:.5em;}:where(.is-layout-grid){gap:.5em;}body .is-layout-flex{display:flex;}.is-layout-flex{flex-wrap:wrap;align-items:center;}.is-layout-flex > :is(*,div){margin:0;}body .is-layout-grid{display:grid;}.is-layout-grid > :is(*,div){margin:0;}:where(.wp-block-columns.is-layout-flex){gap:2em;}:where(.wp-block-columns.is-layout-grid){gap:2em;}:where(.wp-block-post-template.is-layout-flex){gap:1.25em;}:where(.wp-block-post-template.is-layout-grid){gap:1.25em;}.has-black-color{color:var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-color{color:var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-color{color:var(--wp--preset--color--white) !important;}.has-pale-pink-color{color:var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-color{color:var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-color{color:var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-color{color:var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-color{color:var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-color{color:var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-color{color:var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-color{color:var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-color{color:var(--wp--preset--color--vivid-purple) !important;}.has-black-background-color{background-color:var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-background-color{background-color:var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-background-color{background-color:var(--wp--preset--color--white) !important;}.has-pale-pink-background-color{background-color:var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-background-color{background-color:var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-background-color{background-color:var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-background-color{background-color:var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-background-color{background-color:var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-background-color{background-color:var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-background-color{background-color:var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-background-color{background-color:var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-background-color{background-color:var(--wp--preset--color--vivid-purple) !important;}.has-black-border-color{border-color:var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-border-color{border-color:var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-border-color{border-color:var(--wp--preset--color--white) !important;}.has-pale-pink-border-color{border-color:var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-border-color{border-color:var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-border-color{border-color:var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-border-color{border-color:var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-border-color{border-color:var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-border-color{border-color:var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-border-color{border-color:var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-border-color{border-color:var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-border-color{border-color:var(--wp--preset--color--vivid-purple) !important;}.has-vivid-cyan-blue-to-vivid-purple-gradient-background{background:var(--wp--preset--gradient--vivid-cyan-blue-to-vivid-purple) !important;}.has-light-green-cyan-to-vivid-green-cyan-gradient-background{background:var(--wp--preset--gradient--light-green-cyan-to-vivid-green-cyan) !important;}.has-luminous-vivid-amber-to-luminous-vivid-orange-gradient-background{background:var(--wp--preset--gradient--luminous-vivid-amber-to-luminous-vivid-orange) !important;}.has-luminous-vivid-orange-to-vivid-red-gradient-background{background:var(--wp--preset--gradient--luminous-vivid-orange-to-vivid-red) !important;}.has-very-light-gray-to-cyan-bluish-gray-gradient-background{background:var(--wp--preset--gradient--very-light-gray-to-cyan-bluish-gray) !important;}.has-cool-to-warm-spectrum-gradient-background{background:var(--wp--preset--gradient--cool-to-warm-spectrum) !important;}.has-blush-light-purple-gradient-background{background:var(--wp--preset--gradient--blush-light-purple) !important;}.has-blush-bordeaux-gradient-background{background:var(--wp--preset--gradient--blush-bordeaux) !important;}.has-luminous-dusk-gradient-background{background:var(--wp--preset--gradient--luminous-dusk) !important;}.has-pale-ocean-gradient-background{background:var(--wp--preset--gradient--pale-ocean) !important;}.has-electric-grass-gradient-background{background:var(--wp--preset--gradient--electric-grass) !important;}.has-midnight-gradient-background{background:var(--wp--preset--gradient--midnight) !important;}.has-small-font-size{font-size:var(--wp--preset--font-size--small) !important;}.has-medium-font-size{font-size:var(--wp--preset--font-size--medium) !important;}.has-large-font-size{font-size:var(--wp--preset--font-size--large) !important;}.has-x-large-font-size{font-size:var(--wp--preset--font-size--x-large) !important;}:where(.wp-block-post-template.is-layout-flex){gap:1.25em;}:where(.wp-block-post-template.is-layout-grid){gap:1.25em;}:where(.wp-block-columns.is-layout-flex){gap:2em;}:where(.wp-block-columns.is-layout-grid){gap:2em;}:root :where(.wp-block-pullquote){font-size:1.5em;line-height:1.6;}.e-con.e-parent:nth-of-type(n+4):not(.e-lazyloaded):not(.e-no-lazyload),.e-con.e-parent:nth-of-type(n+4):not(.e-lazyloaded):not(.e-no-lazyload) *{background-image:none !important;}@media screen and (max-height: 1024px){.e-con.e-parent:nth-of-type(n+3):not(.e-lazyloaded):not(.e-no-lazyload),.e-con.e-parent:nth-of-type(n+3):not(.e-lazyloaded):not(.e-no-lazyload) *{background-image:none !important;}.e-con.e-parent:nth-of-type(n+3):not(.e-lazyloaded):not(.e-no-lazyload).nitro-lazy,.e-con.e-parent:nth-of-type(n+3):not(.e-lazyloaded):not(.e-no-lazyload) *.nitro-lazy{background-image:none !important;}}@media screen and (max-height: 640px){.e-con.e-parent:nth-of-type(n+2):not(.e-lazyloaded):not(.e-no-lazyload),.e-con.e-parent:nth-of-type(n+2):not(.e-lazyloaded):not(.e-no-lazyload) *{background-image:none !important;}.e-con.e-parent:nth-of-type(n+2):not(.e-lazyloaded):not(.e-no-lazyload).nitro-lazy,.e-con.e-parent:nth-of-type(n+2):not(.e-lazyloaded):not(.e-no-lazyload) *.nitro-lazy{background-image:none !important;}}.e-con.e-parent:nth-of-type(n+4):not(.e-lazyloaded):not(.e-no-lazyload).nitro-lazy,.e-con.e-parent:nth-of-type(n+4):not(.e-lazyloaded):not(.e-no-lazyload) *.nitro-lazy{background-image:none !important;}#sub-header{display:none;}#page.main-container #main-content .sjb-container{max-width:1280px;margin:0 auto;padding:0 30px;}#sjb-application-form{display:none;}.sjb-page{font-family:"Raleway" !important;}.sjb-page header .header-margin-top .row .col-md-2:first-child{display:none;}.sjb-page .sjb-detail .list-data .v1 ul li::before,.sjb-page .sjb-detail .list-data .v2 ul li::before{color:#f15a29 !important;}.sjb-page .list-data .v1 .job-type,.sjb-page .list-data .v1 .job-location,.sjb-page .list-data .v1 .job-date,.sjb-page .list-data .v2 .job-type,.sjb-page .list-data .v2 .job-location,.sjb-page .list-data .v2 .job-date,.sjb-page .sjb-detail .list-data .v1 h3{color:#f15a29 !important;}.sjb-page .sjb-detail .list-data .v1 ul li{line-height:1.9em;}.sjb-page .list-data .v1 .job-description p{line-height:1.5em;}.sjb-page .list-data .v1 .job-description p strong{font-size:22px;margin:40px 0 0;display:block;}.sjb-page .btn-primary{background-color:#fff6ee !important;color:#f15a29 !important;font-family:"Raleway" !important;font-size:18px;}.sjb-page .btn-primary:hover{background-color:#f15a29 !important;color:#fff !important;}.sjb-page .sjb-detail .list-data .v1 .jobpost-form .file div{background:#fff6ee !important;color:#f15a29 !important;}

Data Quality in Databricks with Great Expectations (GX Library)

Read our Case Studies

Our Partners

Solutions

LumenData Accelerator for MDM Modernization

LumenData 360++ Extension for Supplier 360

LumenData Accelerator for Higher Ed 360

LumenData Axon to CDGC Modernization

Life Science Accelerator for Customer360 SaaS

Salesforce Accelerator for Customer360 SaaS

SAP Accelerator for Customer360 SaaS

Salesforce Connector for Informatica MDM

Salesforce Connector for Oracle MDM Product

Reltio Integration for Salesforce

Industry Focus

Public Sector

Financial Services

Higher Education

Retail

Healthcare

Manufacturing

High Tech

Featured Case Studies

Services Focus

Cloud Modernization

Data Engineering & Analytics

Data Strategy & Business Value Assessment

Solutions

Industry Focus

Telemedicine

Dating Apps

Fintech

Consulting Providers

Featured Case Studies

Simplifying IT for a complex world.

Platform partnerships

Services

Business Challenges

Digital Transformation

Security

Automation

Gaining Efficiency

Industry Focus

Simplifying IT
for a complex world.