Skip to content

Google Cloud Storage

Google Cloud Storage is generally available in the DataSquares connection wizard. Bucket + service-account JSON — browse and import Parquet/JSON/CSV objects.

  1. In DataSquares, open Data Sources and choose Add data source.

  2. Pick Google Cloud Storage in the connector picker (search or browse by category).

  3. Fill in the connection details below, then test the connection and save.

Field What to enter
Bucket The bucket to read from.
Service-account key (JSON) Paste the JSON key of a service account with Storage Object Viewer on the bucket.

An object store is not a SQL database — there are no schemas or tables to SELECT from. Testing the connection performs a bounded listing, which proves the credentials and the container, nothing more. From there:

  • Browse the bucket or container from the source’s detail page.
  • Import an object (Parquet, JSON/NDJSON or CSV) as a staged dataset — it is decoded and landed in your workspace, and from that point behaves like any other table: models, dashboards, SQL, SquareX.
  • Import a whole prefix as one dataset. A partitioned export (exports/2026-08-09/part-00000 … part-00031) is many objects that are logically one table, so every importable object under the prefix is read and stacked into a single staged dataset.
  • Read and write in bulk from a pipeline — extract objects under a prefix or glob, load results back (optionally hive-partitioned), and trigger runs on file arrival. See Building a pipeline and Destinations.

Format is read from the file, not the filename

Section titled “Format is read from the file, not the filename”

An object’s format comes from its bytes, so an extensionless key — what most exporters write — imports fine, and a .csv that actually holds Parquet is read as Parquet rather than failing with a parse error that blames your data. Only self-describing formats are detected this way (Parquet, JSON/NDJSON); CSV has no signature, so a CSV object still needs a .csv/.tsv name.

Everything answerable from the listing is checked before anything is downloaded, so you find out immediately rather than after a partial fetch:

Refusal Why
The objects total more than the import limit This path reads objects into memory. A dataset that large belongs in a pipeline.
More objects than a single import reads It refuses rather than quietly importing the first N — a table that looks complete and is not is the worse outcome.
The prefix holds more than one format Two formats under one prefix is two datasets. Narrow the prefix, or import them separately.
An object shares no columns with the ones before it Same reason — that is a different dataset, and null-filling it would produce a mostly-empty table that looks successful.

Parts whose columns differ are unioned by name, not by position: a partition that gained a column mid-backfill is null-filled for the earlier parts, one that dropped a column keeps it, and a writer that reorders columns is realigned rather than silently transposed.

Importing stages a copy: re-import (or re-run the pipeline) to pick up changes to the object. Objects are fetched over the platform’s validated outbound path and read locally — DataSquares never queries your bucket in place.

credentials is a service-account key JSON — the same key shape BigQuery takes. It must contain client_email and private_key; the connection is refused with a clear message if it doesn’t.

Grant that service account Storage Object Viewer on the bucket to read, and Storage Object Admin (or Creator plus a delete permission) if pipelines write files back.

Pass the key as a JSON string:

Terminal window
curl -X POST https://your-instance/api/data-sources \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Analytics bucket",
"type": "gcs",
"config": {
"bucket": "my-analytics-bucket",
"credentials": "{\"type\":\"service_account\",\"client_email\":\"…\",\"private_key\":\"…\"}"
}
}'