Google Cloud Storage

You connect Google Cloud Storage from the DataSquares connection wizard. Bucket + service-account JSON — browse and import Parquet/JSON/CSV objects.

1. In DataSquares, open Data Sources and choose Connect Source. The Connect a data source page opens.

2. Pick Google Cloud Storage on that page (search or browse by category).

3. Fill in the connection details below, then test the connection and save.

Connection details

Field	What to enter
Bucket	The bucket to read from.
Service-account key (JSON)	Paste the JSON key of a service account with Storage Object Viewer on the bucket.

What you can do with it

An object store is not a SQL database — there are no schemas or tables to SELECT from. Testing the connection performs a bounded listing, which proves the credentials and the container, nothing more. From there:

• Browse the bucket or container from the source’s detail page.
• Import an object (Parquet, JSON/NDJSON or CSV) as a staged dataset — it is decoded and landed in your workspace, and from that point behaves like any other table: models, dashboards, SQL, SquareX.
• Import a whole prefix as one dataset. A partitioned export (exports/2026-08-09/part-00000 … part-00031) is many objects that are logically one table, so every importable object under the prefix is read and stacked into a single staged dataset — with key=value folders (year=2026/month=10/) read back as columns, see Folder names as columns.
• Read and write in bulk from a pipeline — extract objects under a prefix or glob (hive-partitioned folders such as year=2026/month=10/ read back as columns), load results back (optionally hive-partitioned), and trigger runs on file arrival. See Building a pipeline and Destinations.

Format is read from the file, not the filename

An object’s format comes from its bytes, so an extensionless key — what most exporters write — imports fine, and a .csv that actually holds Parquet is read as Parquet rather than failing with a parse error that blames your data. Only self-describing formats are detected this way (Parquet, JSON/NDJSON); CSV has no signature, so a CSV object still needs a .csv/.tsv name.

Folder names as columns

When every object under the listed prefix sits in the same key=value folders — the layout Spark, Hive, Glue, DuckDB and the pipeline Files load target write — the browser shows Folder names are columns, ticked: each folder becomes a column of every row of its objects (sales/year=2026/month=10/part-0.parquet gives year and month). The import reads them exactly as a pipeline’s Files extract does with folder partitions on:

• Values are text, kept exactly as the folder spells them: month=01 keeps its zero, percent-encoded characters are decoded (city=New%20York is New York), spaces and an empty value are kept. NULL and Hive’s __HIVE_DEFAULT_PARTITION__ are NULL.
• The partition columns come after the files’ own columns; a column of the same name inside the files takes the folder’s value.
• Only folders count, never the file name, and every object must sit in the same key=value folders.
• What is not data is refused, never read as rows or skipped: a non-empty marker or checksum (_SUCCESS, _committed_*, .crc), anything under a hidden folder such as an unfinished write’s _temporary/, and a Delta, Hudi or Iceberg table or a Spark streaming sink, whose folders keep files the table no longer counts — a prefix that starts inside one (tbl/region=emea/) included. The refusal names the object; since an import reads everything under its prefix, narrow the prefix, or read the folder with a pipeline, whose Files extract can take a glob such as sales/**/*.parquet.

Through the REST API, send "hivePartitioning": true with the prefix import, and hiveTypes to type a partition ({ "year": "integer" } — string, integer, double, boolean, date or timestamp); a value its type cannot read is refused.

When a prefix import refuses

Everything answerable from the listing is checked before anything is downloaded, so you find out immediately rather than after a partial fetch:

Refusal	Why
The objects total more than the import limit	This path reads objects into memory. A dataset that large belongs in a pipeline — see Too large to import.
More objects than a single import reads	It refuses rather than quietly importing the first N — a table that looks complete and is not is the worse outcome. The whole prefix is listed before anything is decided.
The prefix holds more than one format	Two formats under one prefix is two datasets. Narrow the prefix, or import them separately.
An object shares no columns with the ones before it	Same reason — that is a different dataset, and null-filling it would produce a mostly-empty table that looks successful.

Parts whose columns differ are unioned by name, not by position: a partition that gained a column mid-backfill is null-filled for the earlier parts, one that dropped a column keeps it, and a writer that reorders columns is realigned rather than silently transposed.

Too large to import

An import reads into memory, so it has limits: the objects under a prefix (500, and 256 MB together), one object’s size, and the rows one import holds. Refused for any of them, a prefix or an object offers Create a pipeline dataset: the pipeline editor opens on a new pipeline that reads the same objects with a Files extract — folder names as columns when they were on for the import, and enough Max files per run for the folder — and loads them into a dataset of the name you gave. A pipeline reads in the background with no such limit. Nothing runs until you save and run it.

Importing stages a copy: re-import (or re-run the pipeline) to pick up changes to the object. Objects are fetched over the platform’s validated outbound path and read locally — DataSquares never queries your bucket in place.

The service-account key

credentials is a service-account key JSON — the same key shape BigQuery takes. It must contain client_email and private_key; the connection is refused with a clear message if it doesn’t.

Grant that service account Storage Object Viewer on the bucket to read, and Storage Object Admin (or Creator plus a delete permission) if pipelines write files back.

Creating the connection through the API

Pass the key as a JSON string:

curl -X POST https://your-instance/api/sources \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Analytics bucket",
    "type": "gcs",
    "config": {
      "bucket": "my-analytics-bucket",
      "credentials": "{\"type\":\"service_account\",\"client_email\":\"…\",\"private_key\":\"…\"}"
    }
  }'

Iceberg REST catalog (Preview)

A pipeline can write an Apache Iceberg table into this bucket and publish it through your Iceberg REST catalog (Apache Polaris / Snowflake Open Catalog, Lakekeeper, Databricks Unity’s Iceberg endpoint …). Add the catalog in the connection form’s optional Iceberg REST catalog section, or with these config fields:

Field	Notes
icebergCatalogUri	the catalog’s REST address — the part before /v1/config; https:// only. Blank removes the catalog
icebergWarehouse	optional — the warehouse (or catalog name) the catalog’s /v1/config expects
icebergAuth	oauth2 (client credentials), bearer or none
icebergClientId, icebergClientSecret	OAuth2 only
icebergTokenUri	OAuth2, optional — defaults to <catalog URI>/v1/oauth/tokens; set it for an external issuer (Keycloak, Databricks /oidc/v1/token)
icebergScope	OAuth2, optional — defaults to catalog (Polaris wants PRINCIPAL_ROLE:ALL, Databricks all-apis)
icebergToken	bearer only

The client ID, client secret and token are encrypted with the rest of the connection and never shown again; an edit that leaves them blank keeps them. Test Connection also signs in to the catalog and reads its configuration. The catalog principal needs to load, create and update tables in the namespace. The table’s files are written with this connection’s own credentials, so the table’s location must be inside this bucket. SigV4-signed catalogs (AWS Glue, S3 Tables) and vended credentials are not supported yet.

Related

• Connector catalog — everything DataSquares connects to.
• Quickstart — from connection to shared dashboard.