Amazon S3

Amazon S3 is generally available in the DataSquares connection wizard. Any S3-compatible store — AWS, MinIO, Cloudflare R2 (set Endpoint for non-AWS). Browse the bucket and import Parquet/JSON/CSV objects as datasets.

1. In DataSquares, open Data Sources and choose Add data source.

2. Pick Amazon S3 in the connector picker (search or browse by category).

3. Fill in the connection details below, then test the connection and save.

Connection details

Field	What to enter
Bucket	The bucket to read from.
Region	Optional — defaults to us-east-1.
Access key ID / Secret access key	IAM credentials with s3:ListBucket on the bucket and s3:GetObject on the keys you read.
Endpoint	Leave blank for AWS. Set it for any S3-compatible store — MinIO, Cloudflare R2 (e.g. https://minio.example.com).

What you can do with it

An object store is not a SQL database — there are no schemas or tables to SELECT from. Testing the connection performs a bounded listing, which proves the credentials and the container, nothing more. From there:

• Browse the bucket or container from the source’s detail page.
• Import an object (Parquet, JSON/NDJSON or CSV) as a staged dataset — it is decoded and landed in your workspace, and from that point behaves like any other table: models, dashboards, SQL, SquareX.
• Import a whole prefix as one dataset. A partitioned export (exports/2026-08-09/part-00000 … part-00031) is many objects that are logically one table, so every importable object under the prefix is read and stacked into a single staged dataset.
• Read and write in bulk from a pipeline — extract objects under a prefix or glob, load results back (optionally hive-partitioned), and trigger runs on file arrival. See Building a pipeline and Destinations.

Format is read from the file, not the filename

An object’s format comes from its bytes, so an extensionless key — what most exporters write — imports fine, and a .csv that actually holds Parquet is read as Parquet rather than failing with a parse error that blames your data. Only self-describing formats are detected this way (Parquet, JSON/NDJSON); CSV has no signature, so a CSV object still needs a .csv/.tsv name.

When a prefix import refuses

Everything answerable from the listing is checked before anything is downloaded, so you find out immediately rather than after a partial fetch:

Refusal	Why
The objects total more than the import limit	This path reads objects into memory. A dataset that large belongs in a pipeline.
More objects than a single import reads	It refuses rather than quietly importing the first N — a table that looks complete and is not is the worse outcome.
The prefix holds more than one format	Two formats under one prefix is two datasets. Narrow the prefix, or import them separately.
An object shares no columns with the ones before it	Same reason — that is a different dataset, and null-filling it would produce a mostly-empty table that looks successful.

Parts whose columns differ are unioned by name, not by position: a partition that gained a column mid-backfill is null-filled for the earlier parts, one that dropped a column keeps it, and a writer that reorders columns is realigned rather than silently transposed.

Importing stages a copy: re-import (or re-run the pipeline) to pick up changes to the object. Objects are fetched over the platform’s validated outbound path and read locally — DataSquares never queries your bucket in place.

Any S3-compatible store

The same connector reaches AWS S3, MinIO and Cloudflare R2 — set Endpoint to your server’s URL for anything that isn’t AWS, and leave it blank for AWS itself. There is no separate MinIO tile: the backend kind is s3 either way.

Permissions

The credentials need s3:ListBucket on the bucket and s3:GetObject on the keys you read. To let a pipeline write files back, add s3:PutObject — and s3:DeleteObject if a replace load should clear its own previous objects.

Creating the connection through the API

The wizard is the shortest path, but the connection is an ordinary data source and can be created through the REST API:

curl -X POST https://your-instance/api/data-sources \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Analytics bucket",
    "type": "s3",
    "config": {
      "bucket": "my-analytics-bucket",
      "region": "us-east-1",
      "accessKeyId": "AKIA…",
      "secretAccessKey": "…"
    }
  }'

region defaults to us-east-1 if you omit it. Add "endpoint": "https://…" for MinIO or R2.

Related

• Connector catalog — everything DataSquares connects to.
• Quickstart — from connection to shared dashboard.