Model Data

Data quality

The Data Quality suite provides continuous monitoring and automated testing across your data models and sources. Define rule-based assertions, schedule continuous runs, profile your tables, enforce publication gates, and review suggested fixes, so dashboards and reports are built on trustworthy data.

Source compatibility

Column profiling and source quality tests support MySQL, MariaDB, PostgreSQL, Redshift, CockroachDB, SQL Server and Azure SQL. Uploaded and pulled sources that use DataSquares staging run their checks against staged data. Other source dialects return a support error before running a query.

For MySQL and MariaDB, the schema field selects a database. Leave it blank to use the database saved in the source connection. When you specify a database, column discovery and profiling use that same database; the connection user must have permission to read it.

MariaDB uses the MySQL family of profiling and rule queries, including text lengths, value distributions, regular expressions and current-date checks. Regex behavior still depends on your database engine and version. SQL Server uses approximate email/phone checks and refuses complex custom regex patterns.

Data quality test rules

Click Create Test to define assertion rules against model tables or specific columns. DataSquares supports 14 built-in rule types:

Test Rule	Scope	What it asserts
Not Null	Column	Verifies a column contains no missing (NULL) values: one NULL fails it.
Unique	Column	Asserts distinct values across all rows — ideal for primary key validation.
Duplicate Detection	Table / Multi-column	Checks for duplicate rows across a specified set of columns.
Foreign Key	Column	Validates referential integrity against a target reference table and column.
Value Range	Column	Asserts numeric values stay within configured `min` and `max` boundaries.
No Negative	Column	Verifies numeric values are greater than or equal to zero.
Row Count	Table	Asserts that the table size falls within expected `minRows` and `maxRows` thresholds.
Date Range	Column	Asserts date values fall within a specific chronological window (`minDate` to `maxDate`).
No Future Date	Column	Flags dates after today — detecting faulty ETL or data entry errors. A date, or a date and time from any hour today, passes. For a date or date-and-time column.
Email Format	Column	Checks a basic email-address pattern; this is not full RFC validation or a delivery check.
Phone Format	Column	Checks for at least seven digits on MySQL, MariaDB and PostgreSQL-family sources; SQL Server uses an approximation.
Regex Pattern	Column	Validates strings against custom regex patterns you supply.
Allowed Values	Column	Asserts column entries belong to an explicit whitelist of allowed values.
Custom SQL	Model / Source	Runs one read-only `SELECT` (or `WITH … SELECT`) you write, such as a multi-table consistency check or a custom business rule. It must pass the same safety check the SQL editor applies for the source, when you save the test and again when it runs: a query that fails it is not saved, and the error says why. Moving a test to another source checks its query in that source’s syntax. An older test whose query fails it ends each run in error with the reason. A run keeps sample rows only when the query reads no labelled column.

Severity levels

Assign severity when configuring each test:

Critical — Enforces publication gating on downstream models and dashboards. Gating is part of Business: on another plan the dialog doesn’t let you choose it (A critical check holds publishing while it fails — quality gating is part of the Business plan.).
High — A failing test records FAIL, as a critical one does, but doesn’t block publishing.
Medium (unless you choose another) / Low / Info — A failing test records WARN and doesn’t block publishing.

Scheduling & execution

On-demand execution: Run individual tests or click Run All in the Rules tab. Results report PASS, WARN, FAIL, or ERROR along with runtime telemetry and failure messages.
Scheduled runs (Enable scheduled runs, then a Cron Expression): Enable CRON-based schedules (e.g. 0 0 * * * for nightly execution).
Run on sync (Run on data sync): Trigger test execution automatically whenever data source synchronization occurs.

Try a check before you save it

In the Create Test dialog, Try it runs the check once against its source without saving it, and says whether it passes, would warn or fail (as its severity decides), or could not run, with the run’s message. Nothing is stored: no test, no run and no issue. It is rule-based and uses no AI credits. A Custom SQL check meets the same safety check as saving it, and its sample rows come back only when its query reads no labelled column. Trying a check needs the data quality create permission. Through the API: POST /api/data-quality/tests/dry-run, which takes a test as POST /api/data-quality/tests does, its name optional.

Describe the check

When you create a test on a model’s table, Describe the check turns a sentence such as “amounts are never negative” or “every order’s customer exists” into one check. Choose the model and the table, write the sentence, then choose Draft check. The dialog fills in the test type, the column, its settings and, if you haven’t typed your own, a name and a description, and marks them AI-written. Read them, Try it if you like, and save: nothing is saved until you do.

It uses AI credits, charged to you, as the dialog says. It needs the data quality create permission, and the workspace’s AI controls apply: the Data quality switch, your role, and the budgets.
What the AI is sent: your sentence; the table’s field names and types, with the counts from its latest profiling scan (rows, empty values, distinct values, duplicates and text lengths); and the fields of the tables the model joins it to, by name and type, for a foreign key check. Never a value stored in your tables. A labelled field goes by its name and type only. The counts are from a scan of that table in that model, never another model’s table of the same name. A table left out of the model’s AI scope can’t be described: set its check up yourself.
Only where checks run. On a model whose source data quality tests don’t run on (see the list at the top of this page), the dialog says so and drafts nothing, so no AI credits are used for a check that could never run.
One typed check, never SQL. The AI answers with one of the rule types above (never Custom SQL), and the draft must pass the checks a saved test passes: its column is the table’s and fits the check (a date check on a date, a range on a number), and its settings are complete. A draft that doesn’t is sent back to the AI once to be fixed. When no single check fits the sentence, for example one that compares two tables’ totals, the dialog says why.
API keys can’t use it: POST /api/data-quality/tests/draft is for the app.

Suggested checks

The Suggested checks tab lists checks worked out from your data models and how they are used. They are found by rules and use no AI credits:

Not null and Unique on a table’s declared key, its primary key in the source as a Sync read it; for a key of several columns, Not null on each and Duplicate detection on all of them together;
Foreign key for each one-to-many or one-to-one relationship: the field on the “many” side must match the field it joins to;
No future date on each metric’s time dimension that is a date or a date and time (not a number, such as a date key, or text);
Row count (at least one row) on each table a card on a live dashboard reads.

Only fields with a column of their own are suggested (not calculated fields), and only on models whose source data quality tests run on. The checks read most come first: by how many cards on live dashboards, metrics and measures read the field, then those of certified models or that certified content reads. A check one of your tests already makes is not suggested. Each says why it is suggested and what reads its field; pick a model to see its checks alone.

A check is on a source’s table and column, whichever model reads them. When two models read the same table of the same source, its checks are suggested once, counting what reads the column in both models, and listed under either model; the line names the other models. A test in either model makes the check, so it isn’t suggested again for the other.

Accept creates the test, named as the model names the table and field, with medium severity. Select several to accept up to 200 at once: they are created all together or not at all. Dismiss hides a check for 90 days, after which it comes back, marked as suggested again. When someone else accepted or dismissed a check first, or a test now makes it, the action is refused, nothing changes, and the list refreshes. Deleting a test that came from a suggestion brings the suggestion back. Accepting needs the data quality create permission and dismissing the update permission; the Audit log records who did either. Through the API: GET /api/data-quality/check-suggestions and POST /api/data-quality/check-suggestions/accept or /dismiss with the suggestions’ ids.

Publication quality gates (Critical checks)

When publishing a dashboard, report, or data model, DataSquares evaluates active critical data quality tests:

Automatic hold: If any critical test bound to the underlying models or sources is currently failing or in an error state, publication is held. A card or report block that runs a saved query counts the query’s data source, and the model it was built on with that model’s source.
Continuous availability: The previously published version continues serving end users without interruption.
Fail-safe rules:
- Only tests marked with Critical severity gate publication.
- Tests that encounter execution errors are treated as failures to prevent publishing unverified data.
- Newly added tests that have not yet run do not retroactively freeze publication.

Audit-logged overrides

Anyone who may publish the dashboard, report or model can override a held publication with Publish anyway in Publishing held by data quality:

Overriding requires a mandatory reason explaining why the release is proceeding despite failing checks.
Overrides are permanently recorded in the Audit log, documenting the author, reason, and list of failing tests.

Walkthrough: create and run tests

Creating and running rules needs the data quality create permission, which company admins have.

Open Data Quality and choose the Rules tab. Each rule checks one table or column; the list shows its table, column, type, last status and last run. Creating and running rules needs the data quality create permission, which company admins have.
Choose Create Test. Name the rule, pick a test type (Not Null, Unique, Value Range, Row Count and more), then the data model, table and column it checks. Severity (Critical to Info), an optional schedule and type-specific settings follow further down.
The new rules appear in the list, not yet run.
Choose Run All, or the play button on one row to run just that rule. Each row then shows PASS, WARN, FAIL or ERROR and when it last ran. Here the id column passes, and the unique check on customer_id fails because customers place more than one order; its severity is Critical, so it reads FAIL (Critical and High fail a run; Medium, Low and Info only warn).
Use Filter by status to show only Failing or only Passing rules.
The Lineage tab shows the relationships between the tables of a model and how healthy they are. Choose a Data Model to see them. It is the place to look when a rule on a joined table fails.

Deleting a rule closes its open issues, marking them as ignored with a reason, so a removed test leaves nothing behind on the Issues tab.

Monitoring views & tabs

The Data Quality workspace (/data-quality) provides dedicated monitoring tabs:

Overview: The Quality Score and its Quality Breakdown (Completeness, Validity, Uniqueness, Freshness, weighted 30%, 30%, 20% and 20%), with Total Rows, Valid Rows, Missing Values, Duplicates, Stale Tables, Total Tables, Critical/High Issues, Relationship Issues and Open Issues, Open Issues by Severity and Tables Needing Attention. Above the tabs, Tests Passed, Tests Failed and Row-count changes count the whole workspace. A table that has never been scanned shows Not scanned rather than a score, and the workspace score appears once the first scan completes — a score is never shown for data nobody has checked.
Column Profile: Pick a model and a table and choose Run Profiling Scan. Each column shows its total, valid, missing and distinct counts, its minimum, maximum and average, its text lengths and its top values. A labelled column shows its values only to company admins. The scan runs on rules and uses no AI. Tick Add AI findings (uses AI credits) before you scan to have the AI review the findings too (see Suggestions); it is off each time you open the tab.
Rules: Full registry of configured quality tests (/api/data-quality/tests) with inline run, edit, delete, and bulk execution controls.
Suggested checks: checks worked out from your data models and their usage, to accept or dismiss (see Suggested checks).
Issues: One open issue per failing test (/api/data-quality/issues), with its type (such as Missing Values, Duplicates, Range Violation, Pattern Violation or Stale Data), status (Open, In Progress, Fixed or Ignored) and a resolution note. A test that passes again closes its issue.
Suggestions: Fixes found by profiling scans, such as trimming whitespace, standardizing case, reviewing duplicate values or adding an email or phone rule, found by rules. When a scan was run with Add AI findings ticked and AI is set up, the AI also ranks them by confidence, sharpens their wording and may add up to five more; that uses AI credits. A scan without it keeps the rule-based ones and spends no AI credits. Through the API (POST /api/data-quality/scans), the AI findings come only with aiFindings: true, which an API key can’t ask for. Wording the AI wrote is marked AI-written. Apply marks a suggestion done and Dismiss hides it; neither changes your data or creates a rule.
Freshness: each table’s freshness (/api/data-quality/freshness): Fresh when its data was updated in the last 24 hours, Stale when longer ago, Unknown when DataSquares can’t tell. A Warehouse dataset is judged against its own refresh schedule instead. There is no freshness target to set.
Scan History: Past profiling scans, each with its table, status, start time, duration and quality score.
Row-count changes: Tables whose exact row count grew or shrank by 25% or more between two runs of the Row-count check, each with its model, both counts, the change, and how and when the counts were taken. Acknowledge marks a change as seen and Dismiss as not worth following up; either takes it off the list, and needs the data quality update permission. A change stays listed for up to 90 days.
Lineage: Relationship Health for a model: the relationships between its tables and its join path problems.

Walkthrough: profile a table and read the health score

Open Data Quality, then the Column Profile tab. Choose a data model and a table, then Run Profiling Scan.
The scan reads the table and reports when it last ran, its status, an overall quality score and how many rows it covered. The column profiles below refresh by themselves when the scan completes.
Each column of the table gets a card with its type, how complete and valid its values are, and its distinct and missing values. Open a column to see its minimum, maximum, average and its most common values.
The Overview tab rolls the scans up into a Quality Score with a breakdown into Completeness, Validity, Uniqueness and Freshness, plus row and issue counts.
Issues lists what the tests and scans found, with severity and the number of rows affected. Use the buttons on an issue to mark it fixed or ignore it, and filter by status, severity, type or table.

Columns labelled sensitive

Data quality reads your tables directly, not through a data model, so it applies sensitivity labels itself. A column counts as labelled when a field over it in one of your models carries a label. Columns are matched by name, so a column that shares its name with a labelled one is treated as labelled too (the Column Profile tab matches across all of the workspace’s sources), and when the labels can’t be read, every column is.

The AI never sees a labelled column. When a profiling scan asks the AI to rank its findings (Add AI findings), a labelled column is left out entirely: no values, no statistics, no findings and no failed checks, whoever runs the scan, company admins included. Its rule-based findings keep the rules’ wording.
Column Profile shows its values only to company admins. Anyone else, an API key included, sees the column’s counts and text lengths, and Values hidden: this column is labelled sensitive where its minimum, maximum, average and top values would be.
A suggestion keeps no wording that could quote a labelled column’s values. A profiling scan records whether the rules or the AI worded each of its suggestions. When a column is labelled after a scan whose AI findings worded a suggestion about it, anyone but a company admin (an API key included) sees that suggestion in the rules’ words, without its preview, and with a note saying why. The rules’ own wording names the column and counts, never a value.
Describe the check sends a labelled field by its name and type only: no counts from its profile, and never a value. See Describe the check.
A Custom SQL test keeps sample rows only when its query reads no labelled column. A run keeps up to ten of the rows its query returned only when the query provably reads no labelled column of the test’s source: SELECT *, and SQL the safety check can’t read, don’t qualify, as in raw SQL. The run’s message and counts are kept either way. Company admins get the kept rows; anyone else gets them only while the query still reads no labelled column by today’s labels. No screen shows sample rows: they come back from GET /api/data-quality/tests/{id}/runs.

Who can see data quality results

The Data Quality pages need the data quality view permission, and they are refused to anyone a row-level security policy in the workspace applies to: profiles, scores and test results describe whole tables. The same goes for suggested checks, Describe the check and Try it. A scan and its column profiles can be read only in the workspace they belong to; another workspace’s scan answers 404, like one that doesn’t exist.

Row-count check

Row-count check in the page header, like Run check on the Row-count changes tab, runs a rule-based check: it calls no AI and uses no AI credits, so the workspace’s AI controls don’t affect it. The same check also runs once a day for every workspace. Running it yourself needs the data quality create permission.

The check doesn’t query your sources, and it compares exact row counts only. For each table in your data models it takes the latest count DataSquares recorded:

for a model on a Warehouse dataset, the dataset’s row count at its last refresh;
otherwise, the count the last read of the source’s tables took: a Sync, or the read after a scheduled refresh or a file replacement. DataSquares counts the rows of uploaded files and connector data exactly when it reads them, and Snowflake, BigQuery, Databricks, Elasticsearch and OpenSearch report an exact count, and so do ClickHouse’s plain MergeTree tables (not those whose count a merge changes: the other MergeTree engines, such as ReplacingMergeTree, a table TTL, or rows removed by a lightweight delete).

Other databases (PostgreSQL, MySQL, SQL Server and Oracle among them) report an estimate from their statistics. An estimate can be a quarter or more off and can lag behind a load, so the check doesn’t compare it, and it tells you how many tables it left out for that reason. To watch those tables, add a Row count test: it counts the rows at the source each time it runs.

The first exact count of a table is saved. When the table has a new count, the next check compares it with the saved one, and lists the table on the Row-count changes tab when its row count grew or shrank by 25% or more, from a table that had at least 100 rows. It then saves the new count, so a change is listed once and the table’s next count is compared with this one. A table with no new count since the previous check has nothing new to compare, and the check says so rather than calling it unchanged. Only two counts taken the same way are compared: when a table’s count comes from somewhere else than last time, the check saves it and compares from the table’s next count on.

When it finishes, it says what it did: how many new counts it compared, in how many models, and how many changes it found; how many tables have no new count since the previous check; how many it saved a starting count for; how many have only an estimated row count; and how many have no count from a refresh or a Sync, so it couldn’t check them.

Data models & relationships — defining tables and semantic relationships.
Joins & fan-out — ensuring metric accuracy across table joins.
Audit log — tracking publication quality gate overrides and system events.