Warehouse

Warehouse (managed datasets)

The Warehouse holds managed datasets — copies of your data, imported into storage DataSquares owns and manages. You point at a source table or a saved query; DataSquares reads the rows through the same driver your charts already use, writes them into the Warehouse, and catalogues the result as a dataset.

Once imported, a dataset registers itself as a data source. Models, dashboards, the SQL editor and SquareX measures reach it through the normal path — there is no separate Warehouse API to learn, and nothing downstream needs to know the data is a copy.

Why import anything

Both reasons are about the system the data came from:

Speed. The Warehouse is columnar storage built for analytical scans. A dashboard over a managed dataset generally beats the same dashboard pointed at a transactional database.
Load relief. You read the copy instead of hammering production — your operational database keeps serving your application while analysts keep refreshing their dashboards.

What it is not

Not a data model. A dataset is data, not semantics. You build a model on top of a dataset — relationships, calculated fields, SquareX measures — exactly as you would on a database table.
Not a pipeline. An import is a single copy of one table or one query; a pipeline is a DAG that moves several things in order. The two do meet: a pipeline’s Load step also lands data in a managed dataset, and a dataset’s scheduled refresh is executed as a run of a hidden system pipeline.

Who can import, refresh and delete

There is no separate Warehouse role. An import or a refresh reads its table or saved query as the person doing it, so their role must allow, in Administration → Roles:

Create on the Data Sources menu (an import registers a data source), and View on the Data Sources menu;
for a saved query, View on the SQL menu as well.

The table or saved query then meets the same checks as everywhere else it is read (see below). Open as dashboard also needs Create on the Data Models and Dashboards menus, since it makes a model and a dashboard. Any member can delete a dataset.

An API key needs the dataset:read scope to list or read datasets, and dataset:write to import, refresh, schedule or delete one. Like every key it never does more than its owner: it imports and refreshes only when its owner’s role has the permissions above, and Open as dashboard also needs dashboard:write. A refusal says which is missing, the key’s scope or its owner’s permission.

Isolation between workspaces

The Warehouse is one managed service with many tenants in it, so the question worth asking is what stops one workspace reading another’s data.

Each company (account) gets its own warehouse credential, granted read access to only its own database. Workspaces within one account share that database, and DataSquares keeps their datasets apart. A query that reaches for another account’s data is refused by the database engine — not by a check in application code that could be bypassed by a query the check didn’t anticipate. Writes and administrative statements are refused for the same reason.

There is nothing to configure and nothing to see: the platform generates, encrypts and rotates the credential, and it is selected from the database being queried, so the wrong one cannot be presented by mistake. You never handle it.

Import a table or a saved query

Two entry points, both labelled Import to DataSquares:

Data sources — the button on a source’s table row.
Saved queries — the row’s overflow menu.

Choose what to copy. From a data source: pick a Schema (shown only when the source exposes more than one) and a Table — the table options carry row estimates (~N rows), so you know the size of what you’re about to copy. From a saved query there’s nothing to pick: the query is the source.
Name it. Dataset name defaults to the table or query name, takes up to 160 characters, and must be unique in the workspace. A name that’s already taken is refused with a clear message — never silently renamed.
Optionally set a refresh schedule. The first option, No schedule (refresh on demand), is the default. See Refreshing datasets for the options.
Import. The copy runs in one request; the dialog shows “Copying the data into your Warehouse. Larger imports can take a few minutes.”

While the import runs

The dataset’s status goes importing → ready (or error).

If it finishes in the dialog, you’re told the dataset is ready and offered View in Warehouse.
If it takes longer than the dialog waits (about five minutes), you’re told the dataset is still importing and offered Open the Warehouse. The import keeps running on the server — don’t start it again. The product never claims a slow import failed.
If the origin can’t be read, no dataset is left behind at all. There’s no half-imported husk to clean up. If the copy fails after the origin was read, the dataset stays, marked with an error; delete it or import again.
If the saved query fails the safety check (anything but a single read-only SELECT or WITH query that DataSquares can parse in its source’s SQL), the import is refused with the check’s reason, naming the query, before the source is contacted, and no dataset is left behind.

What refuses an import

A dataset is read by everyone who can read the Warehouse, and it carries neither its source’s row-level security policies nor its sensitivity labels. So an import or refresh of a table or a saved query is refused, naming it and saying why, before the source is contacted:

when your role lacks a permission above;
when row-level security applies to you on the source: neither a table copy nor a saved query can be filtered to your rows;
when the source has any active row-level security policy, whoever imports it: everyone who can read the dataset would see every row;
for a table, when one of its columns is labelled, whatever your role. Import a saved query that leaves the column out;
for a saved query, when it reads a labelled column, or uses SELECT * on a source with labelled columns, whatever your role. Leave the column out of the query.

For an uploaded file (CSV, Excel, SQLite, Parquet or JSON), tick Move this file’s dashboards to the dataset and choose Import and move: it moves the source’s models, with their row-level security and labels, onto the dataset, so only the first two apply to it.

A table’s copy and the table it was made from protect each other’s labelled columns for as long as both are live, whether the copy was imported or moved: a label in a model of the dataset protects that column in the source’s models and in raw SQL on the source (a moved file’s saved queries keep running on the file), and a label in a model of the source protects it in the dataset’s models and in raw SQL on the dataset (Field sensitivity). A label on the dataset’s own models never refuses the dataset’s own refresh. A label on the source’s own models still refuses an import or a refresh of its table, as above.

A scheduled refresh reads as the dataset’s owner, as they are when it runs; see when a scheduled refresh fails.

The Warehouse page

Warehouse lists every managed dataset with Name, Status, Last refreshed, Rows, Size and Schedule. Open one for its detail page:

Open as dashboard: Generates a live dashboard wired to the dataset’s columns on first click, or jumps directly to the existing dashboard.
Column snapshot: Column names and inferred data types (Column / Type).
Lineage panel: Traces upstream pipeline feeds (with exact column-level origin maps and catalog impact links) and downstream consumers (models, fields, dashboard cards, and metrics).
Refresh history: Paged log of refreshes, manual and scheduled, with status, duration and rows.
Action buttons: Open as dashboard, Schedule, Delete, and Refresh now.

Status	Means
`importing`	the first copy — or a refresh — is running
`ready`	the last import or refresh landed; the data is queryable
`error`	the last refresh didn’t land. It does not mean “no data” — see what `error` really means

Using a dataset

A managed dataset appears in the source pickers as a Managed dataset. From there it’s an ordinary source:

build a data model on it — relationships, calculated fields, SquareX measures;
chart it on a dashboard;
query it in the SQL editor and save the result as a saved query.

Two rules worth knowing:

A dataset only ever comes from an import. The connector wizard can’t create one, and you can’t import a dataset from another dataset.
On the Data sources page, a managed row offers “Open in Warehouse” instead of edit/delete. Deleting the raw source underneath would orphan the imported data — delete the dataset from the Warehouse instead.

Deleting a dataset

Delete drops the imported data and removes the dataset along with its auto-registered data source.

If anything still references the dataset, the delete is refused — and the dialog lists exactly what’s in the way: dashboards, models, saved queries, metrics, reports and shares, each linked, under “These assets still reference the dataset. Remove or repoint them first, then delete again.”

Honest limits

100,000 rows by default. Import and refresh load the whole result in memory. There is no streaming import yet.
No incremental refresh. Every refresh re-imports everything — there’s no delta or partition mode.
Column types follow the source; what doesn’t fit stays text. Each column keeps the type its source declares: integers, decimals, floating point, booleans, dates and timestamps land as those types. A column holding a value the Warehouse type can’t store exactly — a date before 1900 or after 2299, a decimal with more than 10 decimal places or 28 integer digits — is stored as text instead, never rounded or clamped. IBM Db2, SAP HANA and Exasol don’t report their column types, so their columns are typed from the values: numbers stay numeric, booleans stay boolean, and everything else is text. To get a different type, cast the column at the origin (import a saved query instead of a raw table) or handle it in the model.
Both “Refresh now” and scheduled refreshes appear in the refresh history, with duration and rows moved. See Refreshing datasets.
No Warehouse role of its own. Importing and refreshing take the Data Sources (and, for a saved query, the SQL) permissions above; anyone who can see the Warehouse can delete a dataset.
Storage is a server-side prerequisite. If the Warehouse’s storage tier isn’t configured on your instance, every Warehouse call fails cleanly with “Warehouse storage is not configured on this server” — nothing half-works.

Refreshing datasets — refresh now, schedules, history.
Connector catalog — where imports come from.
Saved queries — the other import entry point.
Data models & relationships — model a dataset once it’s imported.
Pipelines — multi-step loads into managed datasets.
Datasets API — import, refresh, schedule and history over REST.