DataOps

Data Quality Gates

Every change is tested before it ships, and every run is tested before the business sees it.

Quality is checked on two paths. On the change path, a pull request runs code tests and a contract check, deploys to Dev and QA, and reaches Prod only through a promotion gate. On the data path, every run checks the contract on landing and tests the data before it is published; a failure stops the run, quarantines the rows and alerts the owner. Lineage, dashboards and test evidence come out of every run.

promotesame contractdeployspassfailresults · lineageChange path · every pull requestData path · every scheduled runObserve · every runData engineerPull requestcode + contractContract checkbreaking change failsCI testsunit · SQL · lintDevdeploy · smoke testQAtests on real shapePromotion gateapproved releaseProdsame definitionDrift checkenvs stay alikeSourceDB · app · filesIngestbatch · streamSchema gatecontract on landingBronzeraw · versionedTransformSpark · dbtData testsfresh · volume · rulesQuality gatepass or stopSilver · goldpublished productReports · ML ·agentsQuarantinefailed rows heldOwner alertedrunbook · fixCatalogowner · meaningLineagesource to reportDashboardssuccess · freshnessTest evidencekept per runRunbooksSRE on-call

Point at a component to light its links; select it for what it does here and the practice that makes it right.

When to use it

  • Errors are found by the business, not by the pipeline
  • Data changes ship by hand, or differ between environments
  • Audit asks where a number came from and whether it was checked

How NGT delivers it

Built from code, reviewed at a gate by an NGT architect, and handed over with runbooks your team can run.

DataOps