GoPie
Get started
Menu
On this page

Database & cloud sources

Connect to Amazon S3

Import a file sitting in Amazon S3 — or any S3-compatible store such as Cloudflare R2, MinIO, or Wasabi — as a dataset. Unlike the database sources, the S3 source imports a whole file rather than a query result.

Before you start

You need:

  • The object's path: s3://my-bucket/path/to/data.parquet
  • An access key ID and secret access key with read permission on the object — keys are required even if the bucket is public.
  • For S3-compatible stores, the service's endpoint URL (for example https://<account>.r2.cloudflarestorage.com).

Connect

  1. From your project, select Create Dataset, then the Amazon S3 tile under Connect to Database.
  2. Name the dataset and enter the S3 Path to the object.
  3. Fill in the Access key ID and Secret access key, and optionally the Region and Endpoint (only needed for S3-compatible stores).
  4. Pick the Format — Parquet (default), CSV, JSON, or TSV. If you skip it, the format is inferred from the file extension.
  5. Optionally add an AI context hint, then select Import dataset.

Refresh S3 datasets

S3-sourced datasets support full refresh only — GoPie re-reads the object at the stored path and replaces the dataset's rows. The timestamp-column field has no effect for S3 sources, and incremental refresh isn't available. See Refresh & scheduled refreshes.

Tip

Parquet is the best format for repeated refreshes of large files — schema and row counts read straight from the file's footer, so imports start faster.

Give feedback