connect-data

Connect your data (S3 + rclone)

Create an EU S3 bucket, push data with the AWS CLI, and mount it on the machine with rclone — all with egress-free data residency.

Your S3 bucket is created automatically when you provision a workspace. This guide shows you how to push data to it and mount it on the machine.

Step 1 — Confirm your bucket credentials

In the instance dashboard, open the Data card. You'll find your bucket name, the S3 endpoint, and your access key and secret. Keep the secret safe — it's shown once.

Step 2 — Push data with the AWS CLI

Install the AWS CLI locally if you don't have it, then configure a named profile:

aws configure --profile saig
# AWS Access Key ID: <your-key>
# AWS Secret Access Key: <your-secret>
# Default region name: nl-ams-1
# Default output format: json

aws s3 cp ./my-dataset/ s3://<your-bucket>/dataset/ \
    --recursive \
    --endpoint-url https://s3.nl.sovereignaigrid.nl \
    --profile saig

Egress (download from the bucket to your machine) is free. Push bandwidth from your local machine to the bucket is charged at standard rate.

Step 3 — Mount the bucket with rclone

SSH into the machine, then configure rclone to use the SAIG S3 endpoint:

rclone config create saig s3 \
    provider Other \
    access_key_id <your-key> \
    secret_access_key <your-secret> \
    endpoint https://s3.nl.sovereignaigrid.nl \
    acl private

mkdir -p /mnt/data
rclone mount saig:<your-bucket> /mnt/data \
    --daemon \
    --allow-other \
    --vfs-cache-mode full

Your bucket is now accessible at /mnt/data as a regular filesystem path. Read and write files as you would any directory.

Step 4 — Read from your notebook

In Jupyter, you can now read directly:

import pandas as pd
df = pd.read_parquet("/mnt/data/dataset/train.parquet")

All docs How it works