Unity Catalog and governance

Unity Catalog is Databricks' governance layer: one metastore per region, shared across workspaces, with the objects organized in three levels.

SELECT * FROM lakehouse.gold.daily_sales;   -- catalog.schema.table

Permissions: the whole path has to be open

To read a table, the user (or group) needs three privileges: USE CATALOG on the catalog, USE SCHEMA on the schema and SELECT on the table. A privilege granted at one level is inherited by the objects below it: SELECT on the schema applies to all of its tables, including those created later.

GRANT USE CATALOG ON CATALOG lakehouse TO `analysts`;
GRANT USE SCHEMA ON SCHEMA lakehouse.gold TO `analysts`;
GRANT SELECT ON SCHEMA lakehouse.gold TO `analysts`;

GRANT USE CATALOG ON CATALOG lakehouse TO `finance`;
GRANT USE SCHEMA ON SCHEMA lakehouse.gold TO `finance`;
GRANT SELECT ON TABLE lakehouse.gold.daily_sales TO `finance`;

SHOW GRANTS ON TABLE lakehouse.gold.daily_sales;

The analysts read any table in gold; finance reads only daily_sales. Neither group sees bronze or silver. The lineage demo on this site's home page applies exactly this rule.

Best practices

• Grant permissions to groups, never to people: the person joins and leaves the group, and the rule stays.
• Separate environments by catalog (dev, prod) and layers by schema, or the other way around, but consistently.
• Use *views* or row filters and column masks for sensitive data, instead of copying tables.

Lineage

Unity Catalog records on its own which tables and columns each table was derived from, based on the queries that run on Databricks. Before changing a silver column, lineage shows which gold tables and which dashboards depend on it.

• Managed tables (Unity Catalog takes care of the files) get features such as predictive optimization; external ones point to a path you control.
• Volumes store files (CSV, images, models) with the same governance as tables.