Skip to content
DataSquaresDocs
datasquares.aiSign inStart free

Connect DataCloud Storage & Lakes

SFTP

You connect SFTP from the DataSquares connection wizard. Host, login and a password or private key — a file source and sink for pipelines, watchable for arrivals.

  1. In DataSquares, open Data Sources and choose Connect Source. The Connect a data source page opens.

  2. Pick SFTP on that page (search or browse by category).

  3. Fill in the connection details below, then test the connection and save.

Connection details

Section titled “Connection details”
Field What to enter
Host e.g. sftp.partner.example
Port Optional. Defaults to 22.
Username Your Username.
Password Or authenticate with a private key below. Optional.
Private key PEM/OpenSSH private key for key-based login. Optional.
Key passphrase Only for an encrypted private key. Leave it blank to keep the stored one. Optional.
Host key fingerprint Optional but recommended — pins the server (ssh-keygen -lf on its host key). Without it the server’s identity is not checked. e.g. SHA256:… Optional.

An SFTP server is a store in the pipeline hub: a source (a partner’s drop folder), a sink (CSV/Excel deliveries) and an arrival trigger. Test Connection logs in and lists the login directory.

An encrypted private key needs its passphrase in Key passphrase; it is sent exactly as typed, spaces included, and an edit that leaves it blank keeps the stored one.

Pin the server’s host key

Section titled “Pin the server’s host key”

Paste the server’s fingerprint into Host key fingerprint, in the SHA256:… form ssh-keygen -lf prints. With it set, Test Connection, the arrival trigger and every pipeline run check the key the server presents before any login is sent, and refuse any other key. Without it, the server’s identity is not checked.

Pin the key the server presents to DataSquares: its Ed25519 key when it has one, otherwise its ECDSA key, otherwise its RSA key. On the server, ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub prints the fingerprint; from a machine you trust, ssh-keyscan -t ed25519 sftp.partner.example | ssh-keygen -lf - does the same.

Common errors

Section titled “Common errors”
Error Cause / fix
The server’s host key does not match the pinned fingerprint The server presented another key: its key changed (a reinstall or a rotation), the pin is the fingerprint of another of its keys, or something else answered the address. Confirm the new fingerprint with the server’s owner before you update Host key fingerprint. A pipeline run names both fingerprints and is not retried.
A run fails with SFTP listing of … failed: … Permission denied The login may not list a folder the step reads, or a folder inside it that the step’s path reaches. A folder DataSquares cannot list fails the run instead of being read as empty, so no file is skipped without a word. Folders the path cannot reach are not listed at all: drop/*.csv reads only the files directly in drop, while drop/ (or drop) and a pattern with ** reach every folder beneath it, as does a read with folder partitions on (it looks for a table’s log). Give the login read access to the folders the path reaches. A folder that does not exist is read as empty.
A run fails with SFTP listing of … failed after the connection dropped The server ended the connection in the middle of a listing. The step fails, and its retry policy applies, rather than read part of the folder.
Section titled “Related”