# GoPie documentation — full corpus Every page from https://gopie.ai/docs/ as raw markdown. Index: https://gopie.ai/llms.txt # Getting started > What is GoPie > GoPie turns files and databases into queryable, chat-ready datasets — every AI answer grounded in real SQL against your data. URL: https://gopie.ai/docs/getting-started/what-is-gopie/ GoPie is a data platform that turns files and databases into queryable, chat-ready datasets. Upload a CSV or connect a database, and you get a live table you can query with SQL, ask questions about in plain language, and build dashboards on. Every AI answer is grounded in real queries against your data. When you ask a question in chat, the agent writes SQL, runs it, and shows you the answer together with the query that produced it — so you can verify every claim. ## The building blocks | Concept | What it is | | --- | --- | | **Organization** | The people you work with and the workspace you share | | **Project** | Groups related datasets, documents, dashboards and reports | | **Dataset** | One table — an uploaded file or a database source | | **View** | Saved SQL that behaves like a virtual dataset | | **Chat** | A conversation with your data, with citations | | **Dashboard** | A canvas of charts — AI-generated or hand-built | | **Report** | A frozen, shareable snapshot of an analysis | A typical setup looks like this: a **Sales Analytics** project holding an **Orders** dataset, a view that filters it to the current quarter, a dashboard tracking revenue by category, and a chat where you ask follow-up questions. ## Three ways to work with your data - **Write SQL.** Every dataset opens into a SQL editor with autocomplete and instant results. Start with [Query with SQL](/docs/getting-started/query-with-sql/). - **Ask in plain language.** Chat finds the right datasets, writes and runs the SQL, and cites its sources. Start with [Chat with your data](/docs/getting-started/chat-with-your-data/). - **Generate dashboards.** Describe what you want to track and GoPie builds a chart canvas you can refine. Start with [Generate your first dashboard](/docs/getting-started/generate-your-first-dashboard/). You move between the three freely — a question answered in chat can become a saved view, and a saved view can feed a dashboard. ## Beyond the app Every dataset also gets an instant REST API with filtering, pagination and ready-made code snippets, so the data you upload is immediately usable from your own applications. The whole platform can be self-hosted; deployment docs are covered in a later section. ## Where to go next The [Quickstart](/docs/getting-started/quickstart/) walks the full loop — create a project, [upload your first dataset](/docs/getting-started/upload-your-first-dataset/), and run your first query. Then read [AI in GoPie](/docs/getting-started/ai-in-gopie/) to see how the agent keeps its answers grounded. --- # Getting started > Quickstart > Sign in, get your bearings, and create your first project in a few minutes. URL: https://gopie.ai/docs/getting-started/quickstart/ ## Sign in Sign in with your email address and password. When the deployment has them configured, OAuth providers such as Google and magic links appear on the same screen. After signing in, you land on the home launchpad. ## Get oriented on the launchpad ![The GoPie home launchpad with a greeting, a chat composer, and lists of recent conversations, projects, and dashboards](/docs/getting-started/quickstart-launchpad.png "The launchpad — ask a question or jump back into recent work") The launchpad is the front door to everything you build. At the top, a composer — "Ask anything about your data…" — starts a [chat with your data](/docs/getting-started/chat-with-your-data/); use the **Select context** button to point it at a dataset or project first (send stays disabled until you do). Below it, Recent conversations, Your projects, and Your dashboards take you back to work in progress. You work inside one organization at a time. To switch, open the account menu at the bottom of the left rail. If your account has no active organization yet, GoPie asks you to pick one before continuing. ## Create a project A project is where datasets live, along with everything you build on top of them — views, dashboards, and reports. :::steps 1. **Open Projects** — select Projects in the left rail. 2. **Select Create Project** — the create dialog opens. 3. **Name and describe the project** — both fields are required. This guide uses "Sales Analytics" with the description "Orders and revenue across regions, channels and product categories." The description helps GoPie's AI understand your data, so write a real one. Leave Custom Prompt empty and the Enable Data Cleaning toggle off for now. 4. **Create the project** — GoPie takes you straight into it. ::: ![The Create Project dialog with Name, Description, and Custom Prompt fields and an Enable Data Cleaning toggle](/docs/getting-started/quickstart-create-project.png "Creating the Sales Analytics project") > [!TIP] Name projects after the domain ("Sales Analytics"), not the file you're > about to upload — a project holds many datasets. ## Explore the new project ![An empty project page showing the message "No datasets yet" and a Create Dataset button](/docs/getting-started/quickstart-new-project.png "A fresh project, ready for its first dataset") A fresh project shows "No datasets yet" with a **Create Dataset** button. The tabs across the top — Datasets, Views, Dashboards, Reports, Context, and Dictionaries — are the project's working surfaces, and the **Chat with project** and **Generate dashboard** actions in the header stay disabled until the project has at least one dataset. That is the next step: [upload your first dataset](/docs/getting-started/upload-your-first-dataset/). --- # Getting started > Upload your first dataset > Walk a CSV through the four-step upload wizard and land on a queryable dataset with AI-ready columns. URL: https://gopie.ai/docs/getting-started/upload-your-first-dataset/ ## Start the wizard You need a project first — create one in the [quickstart](/docs/getting-started/quickstart/) if you haven't. This guide uploads `orders.csv`, a 1,200-row file of order data, into the Sales Analytics project. From your project's Datasets tab, select **Create Dataset**. The wizard has four steps — Upload Dataset, Validate Dataset, AI Readiness, and Confirmation — and two modes at the top. **Manual** gives you step-by-step control: you review column names, types, and AI-generated descriptions at each stage. **Auto-pilot** does all of it in one click — AI validates the file, fixes column names, generates descriptions, and creates the dataset instantly. This guide follows Manual. ![The Upload Dataset step with a file dropzone and database connection tiles](/docs/getting-started/upload-source.png "Step 1 — drop a file, or connect a database source") ## Walk through the four steps :::steps 1. **Upload the file** — drop `orders.csv` on the dropzone, or browse for it. CSV, JSON, Parquet, and Excel files up to 100 GB work here; you can also connect a [database source](/docs/datasets/sources/) such as PostgreSQL, MySQL, Snowflake, Amazon S3, or MotherDuck. Validation runs in your browser before anything uploads, and the staged file shows a live manifest line: "orders.csv — CSV — 11 cols · 105 KB — Ready". 2. **Review the validation results** — step 2 reports "CSV Validation Successful" and lists the 11 detected columns with their types. 3. **Make the columns AI-ready** — step 3 opens a column editor. Each column gets a normalized name and an AI-generated description; edit either until every column reads Valid. 4. **Confirm and create** — GoPie suggests the dataset name "Orders" from the file and drafts a dataset description. Adjust both if you want, add an optional custom prompt, then select **Create Dataset**. ::: ![The AI Readiness step showing a column editor with names, types, and AI-generated descriptions](/docs/getting-started/upload-ai-readiness.png "Step 3 — every column name and description is editable") > [!NOTE] Column names are normalized to snake_case — lowercase, no leading digit, > no special characters — so SQL and the AI work reliably. You can edit any > AI-generated description before the dataset is created. ## See what you get ![The confirmation screen reading "Dataset Created Successfully" with a timing breakdown](/docs/getting-started/upload-success.png "The Orders dataset, created and ready to query") The success screen shows a timing breakdown and offers **View Dataset** or **Create Another Dataset**. Open the dataset: you get a preview of the first 100 rows from an auto-run SELECT, column profiles in the right rail, row and column counts, and the generated table name under Dataset Details. The dataset is queryable the moment it exists — by you, and by the AI. Next, run your first query in the [SQL playground](/docs/getting-started/query-with-sql/) — or, for the full upload reference (formats, Auto-pilot, database sources), see [Uploading files](/docs/datasets/uploading-files/). --- # Getting started > Query with SQL > Run SQL against a dataset in the built-in editor, and let AI draft queries from plain English. URL: https://gopie.ai/docs/getting-started/query-with-sql/ ## Open the editor Every dataset page embeds the SQL editor. Open a dataset — here, Orders in the Sales Analytics project — and it runs `SELECT * FROM LIMIT 100` on its own, so rows appear before you type anything. The editor autocompletes keywords, your table name and columns. GoPie speaks a rich [analytical SQL dialect](/docs/query/sql-dialects/). ## Write and run a query Replace the starter query with an aggregate: ```sql {2} SELECT category, SUM(revenue) AS total_revenue, COUNT(*) AS orders FROM gp_DzuutVJef9rSG GROUP BY category ORDER BY total_revenue DESC; ``` The highlighted table name comes from **Dataset Details** in the right rail — every dataset gets its own generated name, so yours differs from `gp_DzuutVJef9rSG`. Autocomplete suggests it after `FROM`. Press [[⌘ Enter]] ([[Ctrl Enter]] on Windows/Linux) to run, or switch on **Auto-run** to re-run as you type. > [!NOTE] The in-browser validator occasionally flags valid SQL — date functions > are a common case. Click **Run anyway**; the server executes the query and is > authoritative. ![The SQL editor on the Orders dataset with an aggregate query and its four-row result](/docs/getting-started/sql-playground.png "Revenue by category, straight from the dataset page") ## Read the results Results render in a virtualized table below the editor; a **Filters** button narrows them without SQL. On Orders, the query returns four rows: Electronics leads at $148,156.86, ahead of Furniture ($126,499.15), Apparel ($16,975.90) and Office Supplies ($9,215.07). In the right rail, **Query Results** reports rows, columns and execution time (23.9 ms here), and the per-column **Summary** lists distinct counts and null percentages. ## Ask AI to draft the query Above the editor sits **Ask AI to help write your query…**. Type plain English — `top 5 products by total revenue` — and press **Run**. GoPie writes dialect-correct SQL into the editor and runs it. The generated SQL is editable — extend and re-run it like anything you wrote. ![The Ask-AI bar with a plain-English prompt and the SQL it generated in the editor](/docs/getting-started/sql-ask-ai.png "AI turns a plain-English request into a runnable query") When one query turns into a conversation, move to [chat with your data](/docs/getting-started/chat-with-your-data/) — or, for the full playground reference (validation, cross-dataset queries, exports), see [SQL playground](/docs/query/sql-playground/). --- # Getting started > Chat with your data > Ask questions in plain language and get grounded answers with citations, runnable SQL and charts. URL: https://gopie.ai/docs/getting-started/chat-with-your-data/ ## Ask in plain language Chat answers questions you would otherwise write SQL for. The agent finds the right data, runs SQL against it, and answers in prose that cites the datasets it used. Generated queries land in the transcript with Run buttons, so you can inspect the SQL behind any answer. ## Start a chat Chat opens from the home launchpad composer, from **Chat** in the left rail, or pre-scoped via **Chat with dataset** and **Chat with project** on their pages. :::steps 1. **Open a new chat** from any entry point. 2. **Select a context** — click the @ button and pick datasets, a project or views; the composer requires one. A chip (here, Orders) marks it. 3. **Ask your question** — for example: "Which product category brings in the most revenue? Show the monthly trend as a chart." ::: ![A new chat with the Orders dataset attached as a context chip](/docs/getting-started/chat-composer.png "A fresh chat with the Orders context selected") ## Read the answer The agent streams its steps as it works — loading context, executing SQL, reasoning — then collapses them behind **Show steps**. Press [[Esc]] to stop a run. The answer is prose with the key figures bolded: Electronics leads at **$148,156.86**, strongest in January 2026 (**$21,023.37**). Beneath it, "Agent utilized the following dataset(s)" cites the sources. A results dock opens beside the chat with **SQL Results** and **Visualizations** tabs — the line chart renders because the question asked for one. ![The finished answer beside the results dock showing a monthly trend line chart](/docs/getting-started/chat-answer.png "The answer cites its dataset; the chart lives in the Visualizations tab") > [!TIP] Be specific: name the measure and the grouping ("revenue by month for > Electronics"), and ask for a chart explicitly when you want one. ## Follow up and keep the results The composer keeps your context for follow-ups. In the results dock, **Download results** exports the rows; **Save as view** turns the query into a reusable dataset. Chats are private by default with a **Share chat** button, and each conversation is auto-titled under Previous Chats. Ready for a full canvas? [Generate your first dashboard](/docs/getting-started/generate-your-first-dashboard/). And when you want the full picture — how the agent finds your data, sharing, saving results — the [Chat section](/docs/chat/how-the-agent-works/) goes deep. --- # Getting started > Generate your first dashboard > One click turns your dataset into a live dashboard of validated KPIs and charts. URL: https://gopie.ai/docs/getting-started/generate-your-first-dashboard/ ## Generate the dashboard Open the Orders dataset from [Upload your first dataset](/docs/getting-started/upload-your-first-dataset/) and click **Generate dashboard** in the right rail. GoPie opens the new dashboard and streams progress in a "Designing your dashboard" card: **Analyzing your data**, **Selecting key metrics**, **Validating queries**, then **Designing the layout**, each with its timing. Expect seconds to a minute, depending on your data. The same button on a project page opens a multi-dataset wizard instead. ![The Designing your dashboard progress card streaming four phases over a skeleton canvas](/docs/getting-started/dashboard-generating.png "GoPie designs the dashboard in four visible phases") ## Tour what you get For Orders, the result is a full canvas titled "Orders Dashboard": an AI-written summary, six KPI tiles with sparklines and a vs-previous-period comparison — Total Revenue $300.85K, Avg Discount 4.57% — and charts such as Top Products by Total Revenue, Total Revenue Over Time and Revenue by Customer Segment. Every number is backed by a real SQL query, validated against your dataset during generation: the dashboard is live, not a mockup. ![The finished Orders dashboard with KPI tiles, a bar list and a revenue trend chart](/docs/getting-started/dashboard-canvas.png "wide: The generated Orders dashboard — KPI tiles, charts and an AI-written summary") ## Make it yours The toolbar filters the whole canvas: set the time range (**All time** by default) and a comparison such as **Previous period**. On any tile, **Refine with AI** applies a change you describe in plain language, **Edit tile** adjusts it by hand, or download the tile as CSV or PNG. Press [[E]] to rearrange the layout, and open **Chat** to ask questions about the dashboard — see [Editing with dashboard chat](/docs/dashboards/dashboard-chat/). [Version history](/docs/dashboards/version-history/) keeps every change restorable. > [!TIP] "Refine with AI" takes requests like "show this as a monthly trend" — > describe the outcome and GoPie rewrites the query and chart. ## Find it later The dashboard saves automatically and appears on the project's **Dashboards** tab and your home launchpad. ## Keep going This is the last tutorial in the journey. Continue to [AI in GoPie](/docs/getting-started/ai-in-gopie/) to understand what the AI can do across the platform, or go deep with the [Dashboards section](/docs/dashboards/introduction/) — the project wizard, dashboard chat, and version history. --- # Getting started > AI in GoPie > How the SQL agent answers questions, where AI assists you across GoPie, and what keeps answers grounded in your data. URL: https://gopie.ai/docs/getting-started/ai-in-gopie/ When you ask GoPie a question, an agent answers it by querying your data — not by recalling facts from training. This page maps what the AI can do and how it stays trustworthy. ## How the agent answers a question The agent works in a loop you can follow: 1. **Find the data.** A schema search across your datasets — names, column descriptions, sample values — picks the tables that can answer the question. 2. **Write the SQL.** The agent drafts a query against the schemas it found. 3. **Run and check.** It executes the query and inspects the result before answering. If the query fails or the result looks wrong, it revises and tries again. When a question names a value loosely — "revenue for electronis", say — the agent matches the term against real values in the column and filters on **Electronics**, the value that actually exists. Ask the **Orders** dataset "Which product category brings in the most revenue?" and the answer comes back as prose plus the query that grouped revenue by category — with the executed results attached. The full mental model is in [How the SQL agent works](/docs/chat/how-the-agent-works/). ## Where AI shows up | Surface | What the AI does | | --- | --- | | [Chat with your data](/docs/getting-started/chat-with-your-data/) | Answers questions with executed SQL, charts and citations | | [SQL editor](/docs/getting-started/query-with-sql/) | Turns a plain-language prompt into a query you can edit and run | | [Dashboard generation](/docs/dashboards/ai-generation/) | Plans metrics and builds a chart canvas from a goal you describe | | [Dataset upload](/docs/getting-started/upload-your-first-dataset/) | Drafts dataset and column descriptions you can review and edit | | [Reports](/docs/reports/introduction/) | Suggests analysis questions grounded in your schema and generates a frozen, citable report | ## Grounding and transparency Every chat answer ships with its evidence: - **Citations** — the datasets the agent used. - **The SQL** — every query the agent ran, open for inspection in the results panel. - **The results** — the actual rows the answer is based on. If a question can't be answered from the data, the agent says so instead of inventing an answer. > [!NOTE] Your data stays in your GoPie deployment. The model sees dataset schemas and the small result samples needed to answer a question — [AI data privacy](/docs/trust/ai-data-privacy/) is the full account of what leaves your deployment and what never does. ## Limits - **Long-running questions time out.** If a query runs too long, the agent stops and tells you. Narrow the question or filter to a smaller slice of the data. - **Very large results are truncated for the model.** The agent reasons over a sample; the full results stay available in the results panel. - **Conversation memory is bounded.** Long chats eventually drop their oldest turns. Start a new chat when you change topics. --- # Datasets & sources > Datasets in GoPie > What a dataset is, the two ways to create one, and the lifecycle from upload to version history. URL: https://gopie.ai/docs/datasets/introduction/ A **dataset** is a queryable table that lives in a project. Everything GoPie does — SQL, chat, dashboards, reports, the per-dataset REST API — runs against datasets, so this section is where most journeys start. ## Know the two ways in - **[Upload a file](/docs/datasets/uploading-files/)** — drop a CSV, JSON, Parquet, or Excel file on the upload wizard. Validation runs in your browser before anything uploads, and AI prepares column names and descriptions along the way. - **[Connect a database or cloud source](/docs/datasets/sources/)** — run a SQL query against PostgreSQL, MySQL, Snowflake, or MotherDuck, or point at a file in Amazon S3, and import the result as a dataset. Source-backed datasets can be [refreshed on demand or on a schedule](/docs/datasets/refresh/). Either way you end up with the same thing: a table with an AI-ready schema, a preview grid, column profiles, and its own settings. ## Understand what a dataset carries Beyond the rows themselves, every dataset has: - **A name and description** — the display name you chose, plus an editable description and an optional custom AI prompt that steer how the AI queries it. - **An internal table name** — a generated identifier (shown under **Dataset Details** in the right rail, e.g. `gp_x7k2m9qwerty`) that you use in SQL. You never have to pick or manage it. - **Column descriptions and profiles** — per-column documentation the AI leans on, plus computed stats like null percentages and unique counts. See [Schema & column descriptions](/docs/datasets/column-descriptions/). - **Settings** — description, columns, custom metadata, access, and maintenance, in a dedicated settings area. See the [settings tour](/docs/datasets/settings/). - **A version history** — every ingest, append, refresh, and restore records a snapshot you can preview, compare, and restore. See [Version history & time travel](/docs/datasets/version-history/). ## Follow the lifecycle 1. **Create** — [upload a file](/docs/datasets/uploading-files/) or [import from a source](/docs/datasets/sources/). If your project has data cleaning enabled, a [Standardisation step](/docs/datasets/data-cleaning/) runs before creation. 2. **Explore** — the dataset page opens in **Preview** mode: the first rows in a virtualized grid with sorting and filters, column profiles in the right rail, and row/column counts. Run SQL or ask the AI in the query section and the grid switches to your results. 3. **Use it everywhere** — [chat with it](/docs/chat/starting-chats/), [generate a dashboard](/docs/dashboards/ai-generation/) from it, query it over [its REST API](/docs/query/dataset-api/), or join it with other datasets in [saved views](/docs/query/saved-views/) — see the [Query & analyze](/docs/query/sql-playground/) section. 4. **Keep it current** — [refresh from the file or the source](/docs/datasets/refresh/), on demand or on a schedule. 5. **Maintain** — edit [descriptions](/docs/datasets/column-descriptions/), manage [access](/docs/datasets/settings/), prune [version history](/docs/datasets/version-history/), and eventually [delete](/docs/datasets/deleting-datasets/). > [!NOTE] Creating, editing, and deleting datasets follows your organisation role: > owners and admins can create and edit datasets, members are read-only, and the > person who created a dataset keeps full control of it regardless of role. ## Find the right page | I want to… | Read | | --- | --- | | Upload CSV/JSON/Parquet/Excel files | [Uploading files](/docs/datasets/uploading-files/) | | Import from PostgreSQL, MySQL, Snowflake, MotherDuck, or S3 | [Database & cloud sources](/docs/datasets/sources/) | | Re-run an import or schedule it | [Refresh & scheduled refreshes](/docs/datasets/refresh/) | | Document columns for better AI answers | [Schema & column descriptions](/docs/datasets/column-descriptions/) | | Tour every dataset setting | [Dataset settings](/docs/datasets/settings/) | | Preview, compare, and restore past versions | [Version history & time travel](/docs/datasets/version-history/) | | Standardise messy data on the way in | [Data cleaning](/docs/datasets/data-cleaning/) | | Export, rename, or delete a dataset | [Deleting & managing datasets](/docs/datasets/deleting-datasets/) | | Fix an upload that won't go through | [Troubleshooting uploads](/docs/datasets/troubleshooting-uploads/) | --- # Datasets & sources > Uploading files > Formats, size limits, the upload wizard in Manual and Auto-pilot modes, and what happens after you select Create. URL: https://gopie.ai/docs/datasets/uploading-files/ The upload wizard turns a file into a dataset. If you haven't uploaded anything yet, the [getting-started walkthrough](/docs/getting-started/upload-your-first-dataset/) does it end to end with a sample file; this page is the full reference. ## Choose a supported format | Format | Extension | Notes | | --- | --- | --- | | CSV | `.csv` | Delimiter, quoting, and header row are detected automatically | | JSON | `.json` | Arrays of objects and newline-delimited JSON are both detected automatically | | Parquet | `.parquet` | Fastest to validate — schema and row count come straight from the file footer | | Excel | `.xlsx` | The **first sheet only** is imported; other sheets are ignored | Files with other extensions are rejected at the picker with "Unsupported file type. Upload one of: .csv, .json, .parquet, .xlsx". Convert tab-separated files to CSV, save `.xls` workbooks as `.xlsx`, and rename newline-delimited `.jsonl`/`.ndjson` files to `.json` — the format inside is detected automatically. The per-file size cap is shown in the dropzone hint (100 GB by default; your deployment may configure a different limit). Projects with [data cleaning](/docs/datasets/data-cleaning/) enabled cap uploads at 800 MB instead. ## Pick a mode From your project, select **Create Dataset**. Before you add a file, the top of the wizard offers two modes: - **Manual** — you review each stage: validation, column names, AI-generated descriptions, and the dataset name, editing anything along the way. One file per run. - **Auto-pilot** — one click does all of it: AI validates the data, fixes column names, generates descriptions, and creates the dataset instantly. Auto-pilot also accepts **multiple files — up to 100 at once** — and turns each file into its own dataset. ## Walk the manual wizard ![The file manifest showing a staged CSV with its format badge and Ready status](/docs/datasets/upload-manifest.png "The manifest tracks every staged file from Reading… through Ready") :::steps 1. **Upload Dataset** — drop a file on the dropzone or browse for it. The file appears in a live manifest with a format badge and a status that moves through *Reading…*, *Checking…*, and *Describing…* to **Ready**. Validation runs in your browser using an embedded query engine — it reads only the schema and a small preview, so even multi-gigabyte files check in seconds and nothing uploads yet. Files that can't be parsed show **Can't read this** with *Retry* and *Remove*. 2. **Validate Dataset** — a summary of the detected columns and types. When the file is valid the wizard shows the result briefly and moves on by itself. 3. **AI Readiness** — a column editor with AI-drafted descriptions. Column names are normalized to snake_case; fix any flagged names (the wizard can auto-fix them) and edit descriptions until every column reads Valid. You can't continue while a column name is invalid. 4. **Confirmation** — GoPie suggests a dataset name from the filename and drafts a description (editable, minimum 10 characters). Optionally add a **Custom Prompt** — extra context the AI will use whenever it queries this dataset. Select **Create Dataset**. ::: ## Watch the creation run After **Create Dataset**, the file uploads to storage (with a live progress bar), then the server takes over with a streamed step-by-step timeline: validating the project, ingesting the file, fetching dataset metrics, creating the record, and finalizing. The success screen — "Dataset Created Successfully!" — offers **View Dataset**, **Download Metadata** (a PDF or JSON report of the schema), and **Create Another Dataset**. Two things continue in the background: - **Column statistics** — the profile panel may briefly read *computing…* while row counts and unique values are calculated. It refreshes on its own. - **Search indexing** — the schema is indexed so the AI can find and chat with the dataset. If indexing fails, the dataset is still created; you'll see a warning with a **Re-index** button, and the same action is always available under **Settings → Maintenance**. ## Run Auto-pilot on many files In Auto-pilot mode, drop up to 100 files and select the create button (it counts what it will make — "Create 12 datasets"). Files process five at a time, each stepping through its own pipeline: validating, generating column descriptions, fixing column names, naming the dataset, generating its description, uploading, and creating. - Each file becomes one dataset, named from its filename. - A file that fails doesn't stop the rest — it gets a per-file **Retry** (plus a global **Retry All**), and unreadable files are skipped with a note: "N file(s) can't be read and will be skipped." > [!TIP] Auto-pilot is the fast path for bulk imports; you can still edit any > generated name or description afterwards in the dataset's > [settings](/docs/datasets/settings/). ## Upload into a cleaning-enabled project If the project has **Enable Data Cleaning** switched on, the wizard swaps its second step for **Standardisation** — schema inference, assisted fixes, and a reviewable cleaning plan run before the dataset is created, and uploads cap at 800 MB. That flow has its own page: [Data cleaning](/docs/datasets/data-cleaning/). Something not working? See [Troubleshooting uploads](/docs/datasets/troubleshooting-uploads/). --- # Datasets & sources > Database & cloud sources > Connect to PostgreSQL > Import the result of a SQL query from a PostgreSQL database as a GoPie dataset, with optional incremental refresh. URL: https://gopie.ai/docs/datasets/sources/postgresql/ Import any query result from a PostgreSQL database (or a compatible service such as Supabase, Neon, or RDS Postgres) as a dataset. ## Before you start You need a connection string for a user that can read the tables in your query: ```text /// copy: false postgres://user:password@host:5432/database ``` Encode any TLS options (`?sslmode=require`) and non-default ports directly in the string. Make sure the database is reachable from your GoPie deployment — private hosts may need network allow-listing. ![The PostgreSQL import form with connection string, SQL query, and timestamp column fields](/docs/datasets/sources-postgresql-form.png "The PostgreSQL Import form") ## Connect :::steps 1. From your project, select **Create Dataset**, then the **PostgreSQL** tile under **Connect to Database**. 2. Name the dataset and paste the **Connection string**. 3. Write the **SQL query** whose result becomes the dataset — for example `SELECT * FROM orders WHERE created_at > now() - interval '2 years'`. 4. Optionally set a **Timestamp column** (see below) and an **AI context hint**, then select **Import dataset**. ::: ## Enable incremental refresh Set **Timestamp column** to a column in your query's result that only grows — `updated_at` or `created_at` are typical. On each incremental refresh, GoPie appends only the rows where that column is later than the previous run's high-water mark, instead of re-importing everything. Full refresh remains available either way. See [Refresh & scheduled refreshes](/docs/datasets/refresh/). > [!TIP] Incremental refresh appends — it won't pick up updates or deletions to > rows it already imported. If your table mutates in place, use full refresh. --- # Datasets & sources > Database & cloud sources > Connect to MySQL > Import the result of a SQL query from a MySQL database as a GoPie dataset, with optional incremental refresh. URL: https://gopie.ai/docs/datasets/sources/mysql/ Import any query result from a MySQL (or MariaDB-compatible) database as a dataset. ## Before you start You need a connection string for a user with read access to the tables in your query: ```text /// copy: false mysql://user:password@host:3306/database ``` Non-default ports and connection options go directly in the string. The database must be reachable from your GoPie deployment. ## Connect :::steps 1. From your project, select **Create Dataset**, then the **MySQL** tile under **Connect to Database**. 2. Name the dataset and paste the **Connection string**. 3. Write the **SQL query** whose result becomes the dataset — for example `SELECT * FROM orders LIMIT 1000` while you experiment, then the real query for the import. 4. Optionally set a **Timestamp column** and an **AI context hint**, then select **Import dataset**. ::: ## Enable incremental refresh Set **Timestamp column** to a monotonically growing column in your result (such as `updated_at`). Incremental refreshes then append only rows newer than the last run, rather than re-importing the full result. Full refresh remains available either way. See [Refresh & scheduled refreshes](/docs/datasets/refresh/). > [!TIP] Incremental refresh appends — it won't pick up updates or deletions to > rows it already imported. If your table mutates in place, use full refresh. --- # Datasets & sources > Database & cloud sources > Connect to Snowflake > Import the result of a SQL query from Snowflake as a GoPie dataset, with optional incremental refresh. URL: https://gopie.ai/docs/datasets/sources/snowflake/ Import any query result from a Snowflake warehouse as a dataset. ## Before you start Snowflake credentials are packed into a single connection string: ```text /// copy: false user:password@account/database?warehouse=WH&role=ROLE ``` - **account** is your Snowflake account identifier (the part before `.snowflakecomputing.com`). - **warehouse** and **role** are optional — omit them to use the user's defaults. The user needs read access to the objects in your query, and the warehouse must be one it can use. ## Connect :::steps 1. From your project, select **Create Dataset**, then the **Snowflake** tile under **Connect to Database**. 2. Name the dataset and paste the credentials string. 3. Write the **SQL query** whose result becomes the dataset — for example `SELECT * FROM MY_DB.PUBLIC.ORDERS`. Qualify table names with database and schema as you would in Snowflake. 4. Optionally set a **Timestamp column** and an **AI context hint**, then select **Import dataset**. ::: ## Enable incremental refresh Set **Timestamp column** to a growing column in your result (such as `UPDATED_AT`). Incremental refreshes append only rows newer than the last run; full refresh remains available either way. See [Refresh & scheduled refreshes](/docs/datasets/refresh/). > [!NOTE] The import runs your query on your Snowflake warehouse, so normal > Snowflake compute charges apply — scope the query to what you need. --- # Datasets & sources > Database & cloud sources > Connect to MotherDuck > Import the result of a SQL query from MotherDuck as a GoPie dataset, with optional incremental refresh. URL: https://gopie.ai/docs/datasets/sources/motherduck/ Import any query result from a [MotherDuck](https://motherduck.com) database as a dataset. ## Before you start You need a MotherDuck **service token** (create one in MotherDuck under Settings → Access tokens). The connection field accepts the token in any of these shapes: ```text /// copy: false motherduck_token= ``` — or an `md:` connection string, or just the bare token pasted on its own. ## Connect :::steps 1. From your project, select **Create Dataset**, then the **MotherDuck** tile under **Connect to Database**. 2. Name the dataset and paste your token. 3. Write the **SQL query** whose result becomes the dataset — qualify tables with the database name, for example `SELECT * FROM my_db.main.orders`. 4. Optionally set a **Timestamp column** and an **AI context hint**, then select **Import dataset**. ::: ## Enable incremental refresh Set **Timestamp column** to a growing column in your result. Incremental refreshes append only rows newer than the last run; full refresh remains available either way. See [Refresh & scheduled refreshes](/docs/datasets/refresh/). > [!NOTE] The import executes on MotherDuck and transfers the result > cloud-to-cloud through a temporary export. Because of how that export works, > your query can't itself contain `COPY … TO` statements — plain `SELECT` > queries are the supported shape. --- # Datasets & sources > Database & cloud sources > Connect to Amazon S3 > Import a Parquet, CSV, JSON, or TSV file from Amazon S3 or any S3-compatible store as a GoPie dataset. URL: https://gopie.ai/docs/datasets/sources/amazon-s3/ Import a file sitting in Amazon S3 — or any S3-compatible store such as Cloudflare R2, MinIO, or Wasabi — as a dataset. Unlike the database sources, the S3 source imports a whole file rather than a query result. ## Before you start You need: - The object's path: `s3://my-bucket/path/to/data.parquet` - An **access key ID** and **secret access key** with read permission on the object — keys are required even if the bucket is public. - For S3-compatible stores, the service's **endpoint URL** (for example `https://.r2.cloudflarestorage.com`). ## Connect :::steps 1. From your project, select **Create Dataset**, then the **Amazon S3** tile under **Connect to Database**. 2. Name the dataset and enter the **S3 Path** to the object. 3. Fill in the **Access key ID** and **Secret access key**, and optionally the **Region** and **Endpoint** (only needed for S3-compatible stores). 4. Pick the **Format** — Parquet (default), CSV, JSON, or TSV. If you skip it, the format is inferred from the file extension. 5. Optionally add an **AI context hint**, then select **Import dataset**. ::: ## Refresh S3 datasets S3-sourced datasets support **full refresh only** — GoPie re-reads the object at the stored path and replaces the dataset's rows. The timestamp-column field has no effect for S3 sources, and incremental refresh isn't available. See [Refresh & scheduled refreshes](/docs/datasets/refresh/). > [!TIP] Parquet is the best format for repeated refreshes of large files — schema > and row counts read straight from the file's footer, so imports start faster. --- # Datasets & sources > Database & cloud sources > Import query results from PostgreSQL, MySQL, Snowflake, or MotherDuck — or files from Amazon S3 — as refreshable datasets. URL: https://gopie.ai/docs/datasets/sources/ Instead of uploading a file, you can create a dataset straight from a system you already run: write a SQL query against your database, and GoPie imports the result. Source-backed datasets remember where they came from, so you can [refresh them on demand or on a schedule](/docs/datasets/refresh/). ## See what you can connect ![The Connect to Database grid showing driver tiles for PostgreSQL, MySQL, Snowflake, S3, MotherDuck and more](/docs/datasets/sources-picker.png "The source picker on the upload page — dimmed tiles are coming soon") | Source | You provide | Custom SQL | Incremental refresh | | --- | --- | --- | --- | | [PostgreSQL](/docs/datasets/sources/postgresql/) | Connection string | Yes | Yes | | [MySQL](/docs/datasets/sources/mysql/) | Connection string | Yes | Yes | | [Snowflake](/docs/datasets/sources/snowflake/) | Account credentials string | Yes | Yes | | [MotherDuck](/docs/datasets/sources/motherduck/) | Service token | Yes | Yes | | [Amazon S3](/docs/datasets/sources/amazon-s3/) | Object path + access keys | — (imports a file) | No — full refresh only | BigQuery, Amazon Redshift, Azure Blob Storage, and Google Cloud Storage appear in the picker as dimmed "soon" tiles — direct imports for those are planned but not available yet. ## Import from a source :::steps 1. **Open the picker** — from your project, select **Create Dataset**. On the first step, below the file dropzone, you'll find **Connect to Database** with a grid of source tiles. (The grid shows in Manual mode before any file is staged.) 2. **Pick a source** and fill the form — every source shares the same shape: a **Dataset name**, a **connection string** (or S3 path), and for databases a **SQL query** whose entire result becomes the dataset. Two optional fields round it out: a **Timestamp column** that enables incremental refresh, and an **AI context hint** that helps the AI understand the data. 3. **Select Import dataset** — GoPie connects, runs your query (or reads the file), and creates the dataset. You land on the dataset page when it finishes. ::: There is no separate "test connection" step — a bad connection string, failed authentication, or a SQL error surfaces when the import runs, right in the form. > [!NOTE] There's no table browser: you write the query yourself, so anything you > can express in SQL — joins, filters, aggregations — can define the dataset. Add > a `LIMIT` while you experiment, then remove it for the real import. ## Understand how credentials are handled - Credentials are stored server-side with the dataset's source configuration and are **never displayed again** after creation — API responses omit the connection string and secret keys. - Each dataset carries its own connection; credentials aren't shared or reusable across datasets. - You can review and delete stored source credentials under **Settings → Secrets** — a read-only registry listing each source's driver, query, and creation date. Deleting an entry permanently removes the stored connection details. > [!IMPORTANT] A source connection can't be edited after creation. To change the > query or the credentials, create a new dataset from the source and delete the > old one. ## Keep source datasets fresh A source-backed dataset shows a refresh action on its page: re-run the stored query in full, or — if you configured a timestamp column — append just the rows that are new since the last run. Owners and admins can also put refreshes on an hourly, daily, weekly, or monthly schedule. Details on both: [Refresh & scheduled refreshes](/docs/datasets/refresh/). --- # Datasets & sources > Refresh & scheduled refreshes > Update a dataset from a new file or its source — replace or append, full or incremental — and put database imports on a schedule. URL: https://gopie.ai/docs/datasets/refresh/ Datasets don't have to be static. File-backed datasets accept a new file at any time; [source-backed datasets](/docs/datasets/sources/) re-run their stored query on demand — or on a schedule. The refresh action lives on the dataset page's right rail (the circular-arrows button, "Refresh dataset"). ## Refresh a file-upload dataset ![The refresh dialog for a file dataset offering Replace and Append modes](/docs/datasets/refresh-file-dialog.png "Refreshing a file dataset — swap every row, or append the new ones") Upload a new file in one of two modes: - **Replace** — "Swap all rows for the new file". Every existing row is dropped and the new file is ingested in full. - **Append** — "Add the new rows to existing data". The new file's rows are inserted alongside what's already there — right for periodic drops like monthly exports. Either way, **the new file must have the same columns as the current dataset** — the dialog validates the schema and names any missing or unexpected columns before anything uploads. To change the schema, create a new dataset instead. ## Refresh a database or cloud source dataset For datasets imported from PostgreSQL, MySQL, Snowflake, or MotherDuck, the refresh dialog offers: - **Full refresh** — "Reload every row from the source. Slower, but always consistent." Re-runs the stored SQL query and replaces the dataset's contents. - **Incremental refresh** — "Append only rows newer than the last refresh, based on the timestamp column." Available when the source was created with a **Timestamp column**; GoPie remembers the last imported value and appends only rows beyond it. If no timestamp column is configured, the option is disabled: "This source has no timestamp column configured — only full refresh is available." S3-sourced datasets support full refresh only. > [!TIP] Incremental refresh only appends. Rows that were updated or deleted at > the source after import won't be corrected — run a full refresh when you need > exact consistency. ## Know what a refresh preserves A refresh streams its progress live (validating, ingesting, updating metrics, re-syncing the AI schema) and is safe to lean on: - **Column descriptions, the dataset description, and custom metadata survive** — refreshes recompute row counts, statistics, and profiles, but your documentation is merged back in, never wiped. - A failed refresh doesn't leave the dataset half-updated — the write either completes or rolls back. - The schema index the AI uses is refreshed automatically, so chat answers reflect the new data. - Each refresh records a version you can preview or restore from the History panel — see [Version history & time travel](/docs/datasets/version-history/). ## Schedule automatic refreshes Database-sourced datasets can refresh themselves. From the dataset page's **⋯** menu, select **Schedule refresh…**: - Pick a cadence — **Hourly** (top of every hour), **Daily**, **Weekly**, or **Monthly**. - The **Active** toggle pauses and resumes the schedule without deleting it. - The dialog shows **Last run** and **Next run** once the schedule has fired. One schedule per dataset. File-upload datasets can't be scheduled — they have no source to pull from. > [!IMPORTANT] Schedules run in **UTC**. A daily schedule fires at 00:00 UTC — > plan around the offset if your team works in another timezone. ## Manage schedules in one place **Settings → Scheduled Refreshes** lists every schedule in your organisation — dataset, interval, last run, next run, and an Active/Inactive badge — with a delete action per row. To pause or change a schedule's cadence, use the **Schedule refresh…** dialog on its dataset. ## Check who can do what - **Manual refresh** — organisation owners and admins can refresh any dataset; members can refresh datasets they created. - **Schedules** — creating, editing, pausing, and deleting schedules is limited to organisation owners and admins. Two refreshes of the same dataset never overlap — if one is already running, the next waits its turn. > [!DETAILS] Self-hosting note > Manual refreshes work on every GoPie deployment out of the box. Scheduled > refreshes are executed by the platform's background worker — if schedules > can't be created on your deployment, ask your administrator to enable it > (covered in the self-hosting section of these docs, coming soon). --- # Datasets & sources > Schema & column descriptions > Document your dataset and its columns — by hand or with AI — and keep the search index that grounds AI answers fresh. URL: https://gopie.ai/docs/datasets/column-descriptions/ The AI is only as good as what it knows about your data. Dataset and column descriptions are that knowledge: they're embedded into GoPie's schema index, searched when you chat, and quoted when the AI writes SQL. Two minutes of documentation buys visibly better answers. ## Describe the dataset Every dataset has a **description** and an optional **custom AI prompt**, editable in two equivalent places: the collapsible description banner on the dataset page, and **Settings → General**. Both autosave when you click away — there is no Save button. - **Description** — what the data is, where it came from, what a row represents. Also drafted automatically at upload; edit it any time. - **Custom AI prompt** — instructions included whenever the AI generates queries for this dataset: "revenue figures are in cents", "always exclude rows where status = 'test'". Projects have their own prompt too; both apply. A **Regenerate** button (the sparkles icon) redrafts the description with AI. ## Describe the columns ![The Columns settings tab listing every column with its type and an editable description](/docs/datasets/columns-tab.png "Settings → Columns — click any description to edit it in place") **Settings → Columns** lists every column with its type, row/unique counts, and description. Click a description to edit it in place — [[⌘ Enter]] saves, [[Esc]] cancels, and clicking away saves too. Descriptions are first drafted by AI during upload (the AI Readiness step). To redraft the whole set later, use the tab's **Regenerate** button — GoPie generates descriptions for every column in batches, using your project's reference documents as context when they're indexed. > [!TIP] Prioritize columns with cryptic names or coded values. "cust_seg — > customer segment: E=enterprise, M=mid-market, S=small business" turns a column > the AI would guess about into one it uses correctly. ## Read the column profiles The **Summary** panel on the dataset page profiles every column: type, null percentage, and expandable stats, sortable by type, null %, or name. Right after creation the panel may read *computing…* while statistics are calculated — it fills in on its own within a minute or so. ## Keep the search index fresh GoPie indexes each dataset's name, description, and per-column names, types, descriptions, and sample values into a semantic search index — that's how chat finds the right datasets and grounds its SQL. The index updates itself when you edit descriptions or refresh data. If a dataset isn't showing up in chat (for example, indexing failed during creation — you'd have seen a warning with a **Re-index** button), rebuild it under **Settings → Maintenance → Re-index**. The action is safe to run any time. ## Trust that documentation survives Descriptions live with the dataset, not the data: [refreshes](/docs/datasets/refresh/) recompute statistics but merge your dataset and column descriptions back in. You won't lose documentation by keeping data current. --- # Datasets & sources > Dataset settings > A tour of the five settings tabs — General, Columns, Metadata, Access, and Maintenance — and the conventions they share. URL: https://gopie.ai/docs/datasets/settings/ Open a dataset and select **Settings** in the header. Settings is a sub-area with five tabs; the back link in the left rail returns you to the dataset. Two conventions apply throughout: - **Everything autosaves.** Edits commit when you click away (or press [[⌘ Enter]] in a text field) — there are no Save buttons. A brief "Saving…" spinner confirms each write. - **Editing follows your role.** Organisation owners and admins can edit any dataset; the dataset's creator can always edit their own. Everyone else sees read-only values. ![The General settings tab with the dataset description, custom AI prompt, and danger zone](/docs/datasets/settings-general.png "Settings → General") ## General The dataset's **description** and **custom AI prompt** (both feed the AI — see [Schema & column descriptions](/docs/datasets/column-descriptions/)), a **Regenerate** button that redrafts the description with AI, and — for those who can delete — the **Danger zone** with the delete action (see [Deleting & managing datasets](/docs/datasets/deleting-datasets/)). Renaming the dataset happens on the dataset page itself: the title in the header is editable in place. ## Columns Per-column descriptions with inline editing and a bulk AI **Regenerate**. Covered in detail in [Schema & column descriptions](/docs/datasets/column-descriptions/). ## Metadata Free-form key/value fields that travel with the dataset — things like *Time Period*, *Units*, *Source*, or *License*. Add rows with **Add field**; empty and duplicate keys are flagged. The tab's **Export** section offers **Download metadata** as a **PDF** or **JSON** report — the dataset's schema with each column's name, type, range, and description. Useful as a data-dictionary handoff. ## Access Control who in your organisation can access this dataset. The panel lists users with access and lets you add or remove them from your organisation's members — what a grant unlocks (and how it combines with project-level grants) is covered in [Sharing & access](/docs/collaboration/sharing-and-access/). - Only organisation **owners and admins** can add or remove users — everyone else sees a "View only" notice. - You can't remove yourself. ## Maintenance Two maintenance surfaces: - **Search index** — a **Re-index** button that rebuilds the semantic index behind chat. Reach for it if the dataset isn't findable in chat; it's safe to run any time. - **Storage & retention** — where version-history retention and storage cleanup live: how long to keep versions, a snapshot cap, storage statistics, and a **Clean up now** action. Full detail in [Version history & time travel](/docs/datasets/version-history/). --- # Datasets & sources > Version history & time travel > Every ingest, append, refresh, and restore records a version — preview any of them, compare two, restore one, and control how long history is kept. URL: https://gopie.ai/docs/datasets/version-history/ GoPie datasets keep their history. Every write — the first ingest, an append, a refresh, a restore — records an immutable version. You can look at the data exactly as it was, diff two versions, and roll back, all without ever losing the newer state. ## Open the history Select the **History** button (clock icon) in the dataset page's right rail. The panel lists every version newest-first, grouped by day. ![The version history panel showing a timeline of versions with row deltas and actions](/docs/datasets/version-history-sheet.png "Every write is a version — the newest is marked Current") Each row shows: - **What happened** — *Ingested*, *Appended rows*, *Refreshed from source*, *Restored from earlier version*, or *Named version*, with a timestamp and who did it. - **The row delta** against the previous version — `+200`, `−15`, or *no change*. - A star when the version is [pinned](#pin-or-name-a-version), and **Current** on the latest. Every version has an action menu: **Preview this version**, **Restore this version**, **Compare with previous**, **Name this version…**, **Pin**, and **Copy version id**. > [!NOTE] Version numbers grow platform-wide, not per dataset — it's normal for a > dataset's history to read v41, v45, v52. The order is what matters. ## Preview an old version **Preview this version** reloads the dataset page as of that version — an amber bar pins the context: > Viewing version 7 · 3 minutes ago · 1,234 rows · read-only, changes are not saved While previewing, the grid, profiles, and queries all read from that version; editing is disabled. The page URL carries the version (`?asof=…`), so you can share the link — anyone with access opens the same view. Select **Return to latest** to exit. You can even chat with a version: **Chat with dataset** from a preview starts the conversation against that snapshot, and the composer shows an "Exploring version N" pill. Query results produced this way carry a provenance caption — "Results as of version N" — so historical numbers never masquerade as current ones. ## Compare two versions **Compare** (in the panel header) or **Compare with previous** (on a row) opens the change feed. Pick the **From** and **To** versions and inspect three tabs: - **Inserted** — sample of rows added in the range, with a count. - **Deleted** — sample of rows removed, with a count. - **Schema** — columns added, columns removed, and type changes between the two versions. > [!NOTE] If retention cleanup has compacted the underlying files, row-level > samples for old ranges may read "Change feed unavailable" — the schema diff > still works. ## Restore a version **Restore this version** (from the row menu, or the button on the preview bar) rolls the dataset back: > This will replace the current data with version 7 (1,000 rows; −200 vs now). Restoring **adds a new version** that reproduces the old state — nothing is deleted, and history keeps flowing forward. Restored the wrong thing? Restore again, including back to what was current before. If the target version had a different schema, the dialog warns you: restoring reverts the schema too. Restore, rename, and pin are available to dataset editors (owners, admins, or the dataset's creator); previewing and comparing are open to anyone who can view the dataset. ## Pin or name a version - **Pin** protects a version from retention cleanup — pinned versions are kept forever, no matter the retention settings. Pin the version you baselined a report on, or the state before a risky refresh. - **Name this version…** labels it ("Q3 close", "before dedupe") so the timeline reads like a changelog. > [!IMPORTANT] Naming a version does **not** protect it — only pinning exempts a > version from cleanup. Name for meaning, pin for keeps. ## Control retention & storage History costs storage. **Settings → Maintenance → Storage & retention** is the dial: ![The storage and retention settings with keep-days presets, snapshot cap, and storage statistics](/docs/datasets/retention.png "Retention settings and live storage statistics per dataset") - **Keep version history for** — 7, 30, or 90 days, no limit, or a custom number. Versions older than this are expired during cleanup. - **Maximum snapshots to keep** — an optional cap; the oldest unpinned versions are pruned past it. - **Storage & compaction** — live statistics: total size, file count, version count, pinned count, and rows. - **Clean up now** — runs cleanup immediately, with a dry-run preview first so you see exactly what would be expired and compacted before confirming. Pinned versions are never removed, and the current version is always safe. Cleanup also runs as regular platform maintenance, so retention settings apply without you pressing the button. ## Take time travel further The same version pinning works across GoPie — these get their own pages in later sections: - **SQL & API** — [queries accept a `snapshot_version`](/docs/api/querying-data/#run-sql-post-sql), so scripts can read a dataset as of a version. - **Chat** — conversations can explore a version (the "Exploring version N" pill). - **Downloads** — exports can be pinned to a version, for reproducible extracts. --- # Datasets & sources > Data cleaning > Standardise messy files on the way in — AI schema inference, assisted value fixes, dedupe, and enrichment, all behind a reviewable plan. URL: https://gopie.ai/docs/datasets/data-cleaning/ Real-world files arrive messy — inconsistent labels, duplicate rows, cryptic columns. Data cleaning adds a **Standardisation** step to the upload wizard that fixes those problems *before* the dataset exists, with AI doing the drafting and you approving every change. ## Turn it on Cleaning is a per-project choice: switch on **Enable Data Cleaning** when creating a project, or later in the project's settings. The toggle notes the trade-off — "Enables data cleaning and standardisation during dataset upload. Maximum upload size: 800 MB." With cleaning on, the wizard's steps become **Upload Dataset → Standardisation → AI Readiness → Confirmation**. ## Review the inferred schema When your file lands, GoPie infers a schema and streams its progress: classifying columns, searching dictionaries, inferring enum values, assigning semantic groups. The result is an editable **cleaning schema** table — each column with its database type, a semantic type and role, and any matched dictionary — that you review and can edit inline before continuing. Two ways to shape it: - **Edit in place** — adjust what the inference got wrong. - **Refine with an instruction** — describe changes in plain language: *"treat state_name as geographical"*, *"mark population as a measure"*, *"remove the id column"*. Select **Apply edit** and the AI produces a new schema version. When it looks right, select **Approve & clean** — that starts the interactive cleaning waves. > [!NOTE] [Dictionaries](/docs/context/dictionaries/) — reference tables of > coded values and their canonical labels, managed on the project's > Dictionaries tab — are what the inference matches columns against. A matched > dictionary standardises that column's values and can add enrichment columns. ## Answer the quick questionnaire Before planning, two yes/no questions: - **Remove duplicate columns?** — columns with identical names are deduplicated, keeping the first. - **Remove rows where all values are null?** — entirely-empty rows are dropped. ## Work through the cleaning waves GoPie then works the dataset in stages, surfacing one **wave** of decisions at a time — grouped by column when there are many: - **Confirm** — yes/no calls, like applying a suggested fix. - **Choose** — value standardisations the AI is less sure about ("Karntaka" → "Karnataka"), picked from suggested options or entered as free input. - **Order** — a suggested column ordering you can drag to adjust. Answer a wave's decisions and select **Apply & continue**; GoPie validates them and advances to the next wave. Repeat until cleaning completes, at which point a report shows what changed. The cleaned data then carries forward automatically. From there the wizard continues as normal — AI Readiness, then Confirmation. The dataset is created from the cleaned data; your original file isn't modified. ## Get the cleaning report On the success screen, **Download Metadata** produces a report (PDF or JSON) of the finished dataset — overview (rows, columns, inferred time period, units, granularity), per-column details, and the dictionaries used during cleaning. It doubles as a data-dictionary you can hand to consumers of the dataset. ## Mind the limits - **800 MB per file** in cleaning-enabled projects (the standard cap applies elsewhere). - A cleaning session stays live for **72 hours** — leave the wizard for longer and you'll need to start the upload again. - Your accepted schema and answered decisions are saved in the session, so a recoverable wave failure can be retried from the wizard without re-uploading. --- # Datasets & sources > Deleting & managing datasets > Rename, export, and delete datasets — including what a delete cascades to and who can do it. URL: https://gopie.ai/docs/datasets/deleting-datasets/ Day-two dataset management: the small actions that keep a project tidy, and the one big destructive action. ## Rename a dataset Click the dataset's title in its page header and type — the name saves in place. Renaming changes only the display name; the internal table name your SQL uses stays stable, so nothing referencing the dataset breaks. ## Export data The **Download** button in the dataset's right rail exports data as **CSV**, **JSON**, or **Parquet**. The export is SQL-shaped — it defaults to `SELECT * FROM
`, and you can edit the query to export a filtered or aggregated slice instead. GoPie prepares the file and opens a download link when it's ready. The full export reference (query-result exports, link reuse, versioned exports) is at [Downloads & exports](/docs/query/downloads/). > [!NOTE] Download links are valid for 24 hours — save the file when you generate > it. Requesting the same export twice within that window reuses the prepared > file instead of exporting again. For a schema report rather than the data itself, use **Download metadata** (PDF/JSON) — see [Dataset settings → Metadata](/docs/datasets/settings/). ## Delete a dataset Delete lives in **Settings → General → Danger zone**. ![The delete confirmation dialog listing dependent views and dashboards with a type-to-confirm field](/docs/datasets/delete-dialog.png "The dialog shows everything that will go down with the dataset") Deleting a dataset **permanently removes it along with everything built on it**. The confirmation dialog enumerates the dependents before you commit: - **Views** whose SQL references the dataset. - **Dashboards** on the dataset or on those views. - The dataset's refresh schedule and stored source credentials, if any. Type the dataset's exact name to enable **Delete dataset**. There is no undo and no trash — the data, its version history, and the dependents are gone. > [!WARNING] Deletion cascades — it never blocks. If a view or dashboard someone > else relies on is built on this dataset, deleting the dataset deletes those > too. Read the dependents list carefully. Chat conversations are not deleted — past chats keep their transcripts, but queries against the deleted dataset will no longer run. ### Who can delete Organisation **owners** can delete any dataset; everyone else can delete only datasets **they created**. If you don't see the Danger zone, you don't have delete rights on this dataset. ### Deleting a project Deleting a project cascades the same way — every dataset, view, dashboard, and document inside it is removed. The confirmation dialog shows the counts and requires typing the project name when the project isn't empty. ## Clean up source credentials Datasets imported from [databases or cloud storage](/docs/datasets/sources/) store their connection details server-side. Deleting the dataset removes them; you can also review and delete stored credentials directly under **Settings → Secrets**. ## Know what doesn't exist (yet) - **Moving a dataset between projects** — not supported; re-upload or re-import into the target project. - **Duplicating a dataset** — not supported; create a new dataset from the same file or source. --- # Datasets & sources > Troubleshooting uploads > The upload errors you're most likely to meet, what causes each, and the fastest fix. URL: https://gopie.ai/docs/datasets/troubleshooting-uploads/ Upload problems fall into three buckets: the file was rejected before upload, the transfer itself failed, or the dataset was created but something after it hiccuped. Find your symptom below. ## "Unsupported file type. Upload one of: .csv, .json, .parquet, .xlsx" Only those four extensions are accepted. The content usually converts losslessly: - **`.tsv` / `.txt`** — save as CSV. - **`.jsonl` / `.ndjson`** — rename to `.json`; newline-delimited JSON is detected automatically. - **`.xls`** — save as `.xlsx` in Excel. ## The Excel file uploads but looks empty or wrong Only the workbook's **first sheet** is imported. Move the data to the first sheet, or export the sheet you want as its own file. A related message — "Excel file appears to be empty or contains no data." — usually means the first sheet is blank. Very large workbooks hit a browser limit: "Excel files over 250 MB can't be validated in the browser. Convert this to CSV or Parquet for large datasets." ## The manifest says "Can't read this" Browser-side validation couldn't parse the file (you'll see a specific reason like "No columns found in this CSV file."). Check that the file opens cleanly in another tool, that the extension matches the content, and that CSVs have a header row. **Retry** re-validates in place after you fix the file. If instead you see "File validation is unavailable", the browser validator itself couldn't start — you can still create the dataset, and the server validates during ingestion (its verdict is authoritative). ## "Upload stalled — no data transferred" The transfer to storage stopped (the default watchdog trips after 3 minutes without bytes). Usually connectivity: check your network, pause a VPN if you're on one, and retry. If it reproduces reliably on a self-hosted deployment, the storage endpoint or its CORS configuration is the usual suspect — loop in your administrator. ## The file is rejected as too large The per-file cap is shown in the dropzone hint. Two levers: export a smaller slice, or convert to **Parquet** (columnar compression routinely shrinks a CSV several-fold *and* validates faster). In cleaning-enabled projects the cap is a fixed 800 MB — upload into a non-cleaning project if the file is bigger. ## A column name is flagged in AI Readiness Column names must be snake_case — lowercase, no leading digit, no special characters — so SQL and the AI work reliably. Use the wizard's auto-fix, or edit the name in place. You can't continue while a name is invalid. ## "Dataset Creation Failed" after the upload finished The server couldn't ingest the file. The screen shows the server's reason — a parse error deep in the file is the common one (browser validation reads only the head; the server reads everything). **Try Again** retries the same file; **Start Over** resets the wizard. If the message names a format problem, re-export the file and check for corruption. ## "…search indexing failed. 'Chat with dataset' won't work until it's re-indexed." The dataset was created fine — only the semantic index behind chat didn't build. Select **Re-index** right on the warning, or later under **Settings → Maintenance**. Queries, previews, and dashboards work regardless. ## Rows were skipped during cleaning validation In cleaning-enabled projects, the deep scan may report "N row(s) with type mismatches will be excluded" — rows whose values can't be coerced to the detected column types. Continue to create the dataset from the valid rows, or go back and fix the source data if those rows matter. See [Data cleaning](/docs/datasets/data-cleaning/). ## Still stuck? Re-run the upload in **Manual** mode (it surfaces each stage's result), try a small sample of the same file to isolate whether size or content is the trigger, and note the exact error text before contacting your administrator or [support](/contact/). --- # Query & analyze > SQL playground > Write and run read-only SQL against any dataset or view — with column autocomplete, live validation, and instant results. URL: https://gopie.ai/docs/query/sql-playground/ Every dataset and view page has a built-in SQL workspace. Open a dataset and it's already there: an editor pre-loaded with `SELECT * FROM
LIMIT 100`, executed for you, with the results in a grid below. There's nothing to launch — the playground *is* the dataset page. ![The SQL playground on a dataset page — editor, auto-run toggle, and results grid](/docs/query/playground.png "The playground opens pre-loaded with a first query, already executed") ## Write and run a query The editor speaks SQL with help: - **Autocomplete** suggests the open dataset's column names as you type — each with its type and description — alongside SQL keywords. - Press [[⌘ Enter]] (or [[Ctrl Enter]]) to run, or use the **Run** button. - **Auto-run**, on by default, re-runs the query about a second after you stop typing — as soon as it validates. Flip the toggle off if you'd rather run explicitly. Queries are read-only: `SELECT`, `WITH`, `DESCRIBE`, and `SUMMARIZE` statements are allowed, one statement per run. Nothing you type in the playground can modify data. ## Let validation catch typos As you type, the query is validated locally in your browser against the dataset's schema — before anything reaches the server. Syntax errors get a wavy underline and a banner: ![The validation banner showing a syntax error with a Run anyway escape hatch](/docs/query/playground-validation.png "Local validation flags the mistake before the query runs") If you're sure the validator is wrong (it only knows the open dataset — references to other tables are deferred to the server), **Run anyway** sends the query regardless. Errors from the server itself appear in a separate banner with its real message, so you can see exactly what it objected to. > [!NOTE] Local validation and auto-run both run as you type — see > [The SQL GoPie speaks](/docs/query/sql-dialects/). ## Read the results Results render in a virtualized grid built for wide data — resizable columns, monospace values, `null` shown in muted italics. The right rail's **Query Results** panel shows the row count, column count, and execution time of the last run. Playground results are a working preview, not a bulk channel: each run returns a bounded number of rows (200 by default). For complete results, [export the query as a file](/docs/query/downloads/) — exports have no row cap — or page through [your dataset's REST API](/docs/query/dataset-api/). ## Query across datasets The `FROM` clause isn't limited to the open dataset. Reference any other dataset's table name and the server resolves it — joins across datasets work exactly as you'd expect: ```sql SELECT p.category, SUM(o.amount) AS revenue FROM gp_orders o JOIN gp_products p ON o.product_id = p.product_id GROUP BY p.category ORDER BY revenue DESC ``` Find a dataset's table name under **Dataset Details** in its right rail. Two things to know: autocomplete only offers the open dataset's columns, and access is checked per dataset — you can only query tables you're allowed to see. To keep a cross-dataset query around permanently, save it as a [view](/docs/query/saved-views/). ## Query a past version On deployments with [version history](/docs/datasets/version-history/), the playground is version-aware: while you're previewing an older version of the dataset, every query you run executes against **that version**, not the current data — and the results are captioned with their provenance. Exit the preview and queries return to head. ## Export what you found The right rail's download button opens the export dialog with the full-table query pre-filled — replace it with your playground SQL to export exactly the rows you were looking at, as CSV, JSON, or Parquet. See [Downloads & exports](/docs/query/downloads/). ## Know what the playground doesn't keep The editor resets to the default query whenever you leave the page — there's no query history. If a query is worth keeping, make it a [saved view](/docs/query/saved-views/): it becomes a first-class object you can preview, chat with, and serve over the API. --- # Query & analyze > Natural language to SQL > Describe the query you want in plain English and get runnable SQL in the playground — grounded in your schema, descriptions, and documents. URL: https://gopie.ai/docs/query/natural-language-sql/ You don't have to write the SQL yourself. Above the playground editor sits an AI bar — "Ask AI to help write your query..." — that turns a plain-English request into a runnable query. ## Ask for a query Type what you want and press **Run**: ![The AI bar with a plain-English request and the generated SQL in the editor below](/docs/query/nl2sql.png "One ask — formatted SQL in the editor, results already loaded") The generated SQL is formatted, dropped into the editor, and executed immediately — you see the query *and* its results in one step. From there it's ordinary SQL: edit it by hand, re-run it, or use it as the starting point for something bigger. ## Know what the AI sees The generation isn't guessing at your data — each ask is grounded in: - **The table schema** — column names, types, per-column descriptions, sample values, and value ranges. - **The dataset's description and its custom AI instructions**, if you've written any. - **Project documents** — uploaded reference material, used to map your business terms to actual column values. - **The SQL dialect** — the AI writes SQL in GoPie's dialect automatically. > [!TIP] The single best lever for better generated SQL is better column > descriptions. A column documented as "Order total in USD, after discounts" gets > used correctly; an undocumented `amt_2` gets guessed at. See > [Schema & column descriptions](/docs/datasets/column-descriptions/). ## Read the assistant's notes Sometimes the honest answer is that no SQL can answer your question — the data lacks the column, or the request isn't a query at all. Instead of failing, the bar shows an **Assistant** note explaining why (for example, that the dataset has no region column to group by). That's information, not an error: check the schema in the right rail, rephrase, and ask again. A genuine failure — the service being unreachable, say — shows as a distinct **Generation failed** message. ## Iterate on the result Each ask is independent — the bar doesn't remember your previous request, so "now filter that to 2024" won't work. To refine, either fold the refinement into a fuller ask ("total sales by category **in 2024**") or just edit the generated SQL directly in the editor. Running your own SQL dismisses any assistant note. ## Write asks that work Specific asks produce dramatically better SQL than vague ones: | Instead of | Ask | |---|---| | "Show sales" | "Total sales by product category for the last 90 days" | | "Best customers" | "Top 10 customers by lifetime order value" | | "Recent stuff" | "Orders placed in June 2026, newest first" | Three habits pay off: name a **time frame**, name the **aggregation** you want (total, average, count, top N), and use the **terms your columns are documented with**. If your team's vocabulary differs from the column names, document the mapping in column descriptions or project documents — the AI reads both. ## Choose the right tool The playground bar is a SQL authoring tool: one ask, one query, results in the grid. For a conversation — follow-up questions, charts, explanations, answers across many datasets — use [chat](/docs/chat/how-the-agent-works/) instead. Same intelligence, different shape. --- # Query & analyze > The SQL GoPie speaks > GoPie runs a single, rich analytical SQL dialect everywhere — the playground, view definitions, and the API. Here's what's guaranteed and the conveniences worth knowing. URL: https://gopie.ai/docs/query/sql-dialects/ Every query you write — in the playground, in a view definition, over the API — runs against the same analytical SQL dialect. It's a modern, analyst-friendly dialect: the same SQL works identically everywhere in GoPie, so nothing you learn on one screen surprises you on another. ## Rely on what never changes - **Read-only** — `SELECT`, `WITH`, `DESCRIBE`, and `SUMMARIZE`. Writes are rejected. - **One statement per request.** - **Double-quoted identifiers** — `"column name"` — for names with spaces or mixed case. - API query requests return at most 1,000 rows each (the playground shows 200); [exports](/docs/query/downloads/) have no cap. ## Conveniences worth knowing The dialect is generous with analyst shortcuts: ```sql /// copy: false -- Instant per-column statistics for a whole table SUMMARIZE gp_orders; -- FROM-first syntax reads the way you think FROM gp_orders SELECT category, SUM(amount) GROUP BY category; -- Select columns by regex, or everything except a few SELECT COLUMNS('sales_.*') FROM gp_orders; SELECT * EXCLUDE (internal_id, raw_json) FROM gp_orders; -- Work on a sample while exploring something huge SELECT * FROM gp_orders USING SAMPLE 10%; ``` Also in the everyday toolkit: `date_trunc` and `strftime` for time work, `approx_count_distinct` for fast cardinality, and full window-function support. ## Feature availability | Capability | Available | |---|---| | `SELECT` / `WITH` / `DESCRIBE` | ✓ | | `SUMMARIZE` | ✓ | | Playground local validation & auto-run | ✓ | | [Version history & time travel](/docs/datasets/version-history/) | ✓ | | AI SQL generation | ✓ | > [!NOTE] The dialect only matters for SQL you write by hand. The > [AI bar](/docs/query/natural-language-sql/) and > [chat](/docs/getting-started/chat-with-your-data/) generate matching SQL for > you automatically. --- # Query & analyze > Saved views > Save a SQL query as a view — a virtual dataset you can preview, chat with, dashboard, and serve over the API. URL: https://gopie.ai/docs/query/saved-views/ A **view** is a SQL query saved as a first-class object. Once saved, it behaves like a dataset: it has a name and description, a preview, its own [REST API](/docs/query/dataset-api/), a **Chat with view** button, and dashboards can be generated from it. Views are how a one-off query becomes part of your project. Views live in each project's **Views** tab, searchable and sortable, in table or card layout. ## Create a view from scratch The view builder is a full-page playground with every dataset in the project at hand: ![The view builder with the SQL editor and the Available Datasets panel](/docs/query/views-new.png "The view builder — every table name in the project one copy-click away") :::steps 1. **Open the builder** — from the project's Views tab, select **Create View**. 2. **Write the query** — the right rail lists **Available Datasets** ("Copy table names to use in your query") with each table's columns. The [AI bar](/docs/query/natural-language-sql/) works here too, aware of every table in the project. 3. **Run it** — previewing the results first is optional but wise. 4. **Save View** — give it a name (at least 3 characters) and, optionally, a description. Leave the description blank and GoPie writes one from the query. ::: ## Save a chat answer as a view When [chat](/docs/chat/save-and-export/) produces a result worth keeping, the results panel's **Save as view** button turns the answer's SQL into a view directly — pick the project, name it, done. The query is shown read-only in the dialog so you can see exactly what you're saving. ## Join datasets in one view A view's query can join any datasets in the project: ```sql SELECT o.order_date, p.product_name, p.category, o.amount FROM gp_orders o JOIN gp_products p ON o.product_id = p.product_id ``` That's the quiet superpower: the join happens once, in the view definition, and everything downstream — chat, dashboards, the API — treats the result as a single flat dataset. Anyone can then ask questions of "Orders with product details" without knowing a join was ever involved. ## Work with a view A view's page mirrors a dataset's page: ![A view's detail page with the View Definition panel expanded](/docs/query/view-detail.png "The View Definition panel shows the dependent datasets and the defining query") - **View Definition** — a collapsible panel showing the datasets the view depends on and the formatted defining query, with a copy button. - **A full SQL playground and preview** below it, for querying the view like any table. - **Settings** — edit the description, a custom AI prompt, and per-column descriptions (they autosave, like everywhere else). - **API** — the same [API explorer](/docs/query/dataset-api/) datasets get. - **Dashboard** — generate a dashboard from the view. ## Trust that views stay live A view stores the *query*, not a copy of the results. Every time anything reads the view — you, chat, a dashboard, an API caller — the SQL re-executes against the current data. When an underlying dataset [refreshes](/docs/datasets/refresh/), the view reflects it immediately; there is no separate "refresh the view" step and nothing to go stale. > [!NOTE] Views don't have their own version history, and time-travel queries > can't target a view — to query the past, use > [version history](/docs/datasets/version-history/) on the underlying dataset > itself. ## Know that the query is fixed A view's defining SQL is set at creation and can't be edited afterward — the name, description, AI prompt, and column descriptions can. To change what a view computes, create a new view with the revised query and delete the old one. You may also notice views badged **Auto-generated**: dashboards sometimes create supporting views automatically to feed their tiles. They're ordinary views — the badge just tells you where they came from. ## Delete a view From the Views list, each row's menu has **Delete** (multi-select works for bulk deletion). The confirmation is plain about the stakes: deletion is permanent. Deleting a view never touches the datasets it queried. Organisation owners can delete any view; admins and members can delete the views they created. --- # Query & analyze > Your dataset's REST API > Every dataset and view is an instant read-only REST API — explore it in the app, copy a snippet, and call it with an API key. URL: https://gopie.ai/docs/query/dataset-api/ The moment a dataset exists in GoPie, it's queryable over HTTP. No configuration, no publishing step — every dataset and every [view](/docs/query/saved-views/) gets a read-only REST endpoint with filtering, pagination, and column selection, plus a generated OpenAPI spec. ## Explore it in the app The dataset page's right rail has an **API Playground** button — an interactive explorer for that dataset's endpoint: ![The API Playground with request parameters on the left and code snippets on the right](/docs/query/api-playground.png "Build the request visually; the code snippets update live") - **Left:** the request builder — page, limit (up to 1,000 rows per request), which columns to retrieve, and filters (`=`, `>`, `<` per column, combined with AND). - **Right:** the exact `POST` URL, live code snippets in **cURL, JavaScript, Python, Go, and Java** that update as you tweak parameters, and a **Send Request** button that runs the call and shows the JSON response with its record count and timing. ## Authenticate your own calls Inside the explorer, requests run as you. From your own code, you authenticate with an **API key**: 1. Go to **Settings → Applications** and create an [application](/docs/account/applications-and-api-keys/) (an application groups keys and defines which datasets they can reach). 2. Open it, switch to the **API Keys** tab, and generate a key. **Copy it immediately — it's shown only once.** 3. Send it as a bearer token on every request. ```bash curl -X POST "https:///v1/api/tables/gp_orders" \ -H "Authorization: Bearer gp_pub_..." \ -H "Content-Type: application/json" \ -d '{"page": 1, "limit": 20}' ``` The explorer shows your deployment's real API base URL in the snippets. A key only reaches the datasets its application is scoped to. ## Query rows The endpoint is `POST /v1/api/tables/{table_name}` with a JSON body: ```json /// copy: false { "columns": ["order_date", "category", "amount"], "sort": "-order_date,category", "limit": 50, "page": 2, "filter": { "filter[category]": "Electronics", "filter[amount]gt": "100" } } ``` | Field | Meaning | |---|---| | `columns` | Columns to return; omit for all columns | | `sort` | Comma-separated columns; `-` prefix for descending | | `limit` | Rows per page, up to 1,000 | | `page` | 1-based page number | | `filter` | Map of conditions, AND-ed together | Filter keys follow `filter[column]op`. The operators: | Key form | Condition | |---|---| | `filter[col]` | equals | | `filter[col]neq` | not equal | | `filter[col]gt` / `filter[col]gte` | greater than / or equal | | `filter[col]lt` / `filter[col]lte` | less than / or equal | Comparison operators apply to numeric values; text values filter by equality. The response is a stable envelope: ```json /// copy: false { "data": [ { "order_date": "2026-06-01", "category": "Electronics", "amount": 129.5 } ], "count": 4283, "executionTime": 12 } ``` `count` is the total matching rows — use it with `limit`/`page` to paginate through everything. ## Run full SQL over HTTP When the filter grammar isn't enough, `POST /v1/api/sql` takes arbitrary read-only SQL — the same `SELECT`/`WITH`/`DESCRIBE`/`SUMMARIZE` rules as the [playground](/docs/query/sql-playground/), same 1,000-row cap per request, same per-dataset access checks: ```bash curl -X POST "https:///v1/api/sql" \ -H "Authorization: Bearer gp_pub_..." \ -H "Content-Type: application/json" \ -d '{"query": "SELECT category, SUM(amount) FROM gp_orders GROUP BY 1"}' ``` The body also accepts `"snapshot_version"` to query a dataset [as of a past version](/docs/datasets/version-history/). The full endpoint catalogue — schemas, summaries, downloads, chat and more — is the [API & MCP reference](/docs/api/overview/); [Querying data](/docs/api/querying-data/) covers this endpoint and its siblings in depth. ## Grab the OpenAPI spec `GET /v1/api/openapi/{table_name}` returns an **OpenAPI 3.0 spec generated live from the dataset's actual schema** — its columns become typed response properties. Feed it to Postman, client generators, or anything else that speaks OpenAPI. ## Remember views are APIs too Everything above works identically for views — a view's page has the same API Playground, and its table name works in the same endpoints. Define a join once as a view, and consumers get one clean API instead of orchestrating joins client-side. --- # Query & analyze > Downloads & exports > Export datasets, views, and query results as CSV, JSON, or Parquet — generated server-side, with links that last 24 hours. URL: https://gopie.ai/docs/query/downloads/ One export dialog serves the whole product. Wherever you see a download button — on a dataset, a view, or a chat answer — it opens the same flow: pick a format, optionally shape the data with SQL, and GoPie generates the file server-side. ## Export a dataset or view The download button lives in the right rail of every dataset and view page: ![The download dialog with a format picker and an editable SQL query](/docs/query/download-dialog.png "Pick a format — and shape the export with SQL if you want less than everything") Choose **CSV**, **JSON**, or **Parquet** and select **Download**. The export runs on the server with live progress — no browser memory limits, no tab you can't close — and the finished file opens in a new tab when it's ready. > [!TIP] Exporting to hand off to another data tool? Use **Parquet** — it's > compressed, preserves column types exactly, and loads faster than CSV > everywhere it's supported. ## Shape the export with SQL The dialog's query box is pre-filled with the full table (`SELECT * FROM …`) — but it's editable. Filter, aggregate, or join before exporting; the file contains exactly what the query returns: ```sql SELECT order_date, category, amount FROM gp_orders WHERE order_date >= '2026-01-01' ``` Unlike API requests, **exports have no row cap** — this is the right channel for pulling a complete dataset out of GoPie. ## Export query and chat results - **From [chat](/docs/chat/save-and-export/):** the results panel's **Download results** button opens the dialog pre-filled with the SQL behind the answer. - **From the [playground](/docs/query/sql-playground/):** open the dataset's download dialog and paste your playground query into the SQL box. ## Export metadata, not just data On a dataset, the download button is a small menu: **Dataset** exports the rows; **Metadata** exports a **PDF or JSON** document describing the dataset — schema, column types, and descriptions. Useful as shareable documentation for people who don't have a GoPie login. ## Understand links and reuse - Export files land in GoPie's storage and the download link stays valid for **24 hours**. After that, just export again. - Repeat exports are deduplicated: request the exact same export (same dataset, query, format, and version) within the window and you get the **same file back instantly** instead of a re-run. - If an identical export is still being generated, GoPie says so — "A previous download for this query is still processing" — rather than starting a duplicate. Anyone who can see a dataset can export it — export permissions simply follow read access. ## Export a past version On deployments with [version history](/docs/datasets/version-history/), exports are version-aware: while previewing an older version, the download button exports **as of that version** (the tooltip says so — "Download Dataset (as of version N)"). That's the clean way to hand someone the data exactly as it stood last quarter. --- # Chat > How the SQL agent works > The tool loop behind every chat answer — schema search, SQL generation with self-repair, fuzzy value matching, and what keeps answers grounded. URL: https://gopie.ai/docs/chat/how-the-agent-works/ When you ask a question in chat, GoPie doesn't recall an answer — an agent **queries your data** and reports what it finds. This page is the mental model: how the agent finds the right tables, writes and repairs SQL, and why every answer comes with evidence. If you haven't chatted yet, start with the [chat tutorial](/docs/getting-started/chat-with-your-data/). ## Follow the loop The agent works in a loop of tool calls, and the chat streams each step as it happens — you can watch the phases in the transcript (**Context**, **Discovering datasets**, **Planning query**, **Running query**, **Validating results**) and expand them later with **Show steps**. 1. **Read the context.** The agent starts from what you attached: the schemas of your selected datasets and views, and an overview of the selected projects. 2. **Find the data.** If the attached context doesn't obviously answer the question, it searches your datasets semantically (see below) — and, when the project has [context documents](/docs/context/documents/), it can search those too. 3. **Write and run SQL.** It drafts a query against the schemas it found and runs it in the [same SQL dialect](/docs/query/sql-dialects/) as the SQL playground. 4. **Check and repair.** If the query errors, the error goes back into the loop and the agent rewrites the query. After a couple of failed repairs it stops and explains what went wrong instead of guessing. 5. **Answer.** The reply is prose grounded in the executed results, with the SQL and the rows attached as evidence. A chart is generated only when you ask for one — "show the monthly trend as a line chart" — never speculatively. See [Reading results](/docs/chat/reading-results/). ## Know how it finds your data Dataset discovery is a **semantic search**, not a keyword match. Every dataset and view is indexed with its name, description, column names, column descriptions, and sample values — as vectors that capture meaning, combined with a classic keyword ranking. That's what lets "sales" find a `revenue` column, or "customers" find a `user_accounts` table, even when the words don't match. Two practical consequences: - **Descriptions are leverage.** The richer your [dataset and column descriptions](/docs/datasets/column-descriptions/), the better the agent's aim — they are indexed and searched, not just displayed. - **Business terms work.** You don't need to know table names. Ask in the vocabulary your team uses, and keep using it consistently. ## Watch loose values get matched When a question names a value loosely — "revenue for electronis", "orders from califronia" — the agent doesn't guess the spelling. It searches the actual column values with pattern and fuzzy matching, finds **Electronics** or **California**, and filters on the value that really exists. You'll see this in the steps timeline as a lookup against the column before the main query runs. This works for spelling, casing, and partial names. It can't invent values that aren't there — if nothing in the column resembles the term, the agent says so. ## Know what the agent sees Every piece of context you invest in reaches the agent: | Context | Used in chat | | --- | --- | | Dataset schemas — columns, types, sample values | Always, for attached and discovered datasets | | [Dataset & column descriptions](/docs/datasets/column-descriptions/) | Always — indexed for search and injected into prompts | | [Custom AI prompts](/docs/context/custom-prompts/) on datasets and views | Always, for the datasets/views in play | | Project-level [custom AI prompt](/docs/context/custom-prompts/) | Always, once per attached project | | [Context documents](/docs/context/documents/) | Searched on demand when the question needs them | | Query results | A sample of the executed rows — large results are truncated for the model, while the full rows stay in the results panel | [Dictionaries](/docs/context/dictionaries/) are the exception: they inform [data cleaning](/docs/datasets/data-cleaning/) during ingestion, not chat. ## Understand the limits - **Each turn has a budget.** The agent gets a bounded number of tool calls (around ten) and a few minutes of wall-clock time per question. If it can't converge, it stops and tells you rather than looping forever. - **Conversation memory is bounded.** Long chats keep roughly the last twenty messages of context. When you change topics, start a new chat. - **The model reasons over a sample.** Very large query results are truncated before the model sees them (a few hundred rows); the full results stay available in the results panel and in [exports](/docs/chat/save-and-export/). ## Trust, but verify Every answer carries its own evidence: the datasets it used are cited under the reply, every executed query is inspectable in the steps and the results panel, and the actual rows are one click away. If a question can't be answered from the data, the agent says so in plain language instead of inventing a number — that's a normal response, not an error. For exactly what does and doesn't leave your deployment on the way to the model, see [AI data privacy](/docs/trust/ai-data-privacy/). --- # Chat > Starting chats & choosing context > Every way into a chat, and how the @ context picker scopes what the agent can see. URL: https://gopie.ai/docs/chat/starting-chats/ A chat always runs against an explicit **context** — the datasets, projects, or views you attach. This page covers the entry points and how to scope a conversation well. ## Start from anywhere - **The home launchpad** — the composer on your home page ("Ask anything about your data…"). Pick a context, type a question, and GoPie opens a new chat and answers it in one step. Your recent conversations, projects, and dashboards are listed right below. - **Chat in the left rail** — opens the chat workspace with an empty composer and a **Previous Chats** link. - **Pre-scoped buttons** — **Chat with dataset**, **Chat with view**, and **Chat with project** on their respective pages (and the chat icon on dataset and view cards) open a new chat with that context already attached. ## Pick the context The [[@]] button in the composer opens the context picker. ![The context picker open over the composer, listing projects and datasets with a search box](/docs/chat/context-picker.png "Search across projects and datasets; selected items become chips in the composer") :::steps 1. **Search or browse** — the picker lists your **Projects** and **Datasets** (with each dataset's parent project shown underneath), filtered as you type. 2. **Select one or more** — each selection becomes a chip in the composer. Mix freely: several datasets, a whole project, or both at once. 3. **Adjust anytime** — remove a chip with its **×**, or reopen the picker mid-conversation to add more context for your next question. ::: You can't send without a context — the composer asks you to *"Select a dataset or project first using the @ button"*. **What each context contributes:** | Attach | The agent gets | | --- | --- | | A dataset | That table's full schema, descriptions, and sample values | | A project | The project's description and its datasets' schemas, plus [semantic search](/docs/chat/how-the-agent-works/) across them and the project's [context documents](/docs/context/documents/) | | A view | The view's schema — queried like any dataset | > [!NOTE] Views join a chat through the **Chat with view** button on the view's > page — the picker itself lists projects and datasets. > [!TIP] Attach the narrowest context that can answer the question. A single > dataset gives the agent a smaller search space than a whole project — faster > answers, less ambiguity. ## Chat with a past version On deployments with [version history](/docs/datasets/version-history/), previewing an older snapshot of a dataset changes the button to **Chat (v*N*)** — the conversation then queries the dataset **as of that version**, and results in the panel carry a version pill so it's clear you're not looking at current data. ## Ask the question Write it the way you'd ask a colleague — name the measure, the grouping, and the time window when you can ("revenue by product category, monthly, for 2026"). The composer keeps your context for follow-ups, so refine step by step. For what makes questions land reliably, see [Tips for better questions](/docs/chat/better-questions/). --- # Chat > Reading results > The steps timeline, the grounded answer with citations, and the results panel with SQL results and visualizations. URL: https://gopie.ai/docs/chat/reading-results/ A chat answer is more than the reply text — it's a transcript of how the agent got there, plus a results panel holding the actual rows and charts. Knowing where everything lands makes answers easy to verify. ## Watch the steps While the agent works, its progress streams into the transcript grouped by phase — **Context**, **Discovering datasets**, **Planning query**, **Running query**, **Validating results** — with a step counter ("Processing · 4 steps"). When the answer finishes, the timeline collapses behind **Show steps**, with the total time alongside. ![An expanded steps timeline showing the agent's phases and an executed SQL query](/docs/chat/steps-timeline.png "Expand Show steps to audit exactly what the agent did") Inside the steps you'll find every SQL query the agent executed, each with a **Run** link that re-executes it into the results panel. Press [[Esc]] (or the stop button in the composer) to stop a response mid-generation. ## Read the answer The reply is prose grounded in the executed results, with key figures highlighted. Two provenance strips follow it: - **"Agent utilized the following dataset(s):"** — chips linking to each dataset the answer drew on. - **"Sources cited:"** — when the agent consulted [context documents](/docs/context/documents/), numbered references linking to them. Under each answer, two pills bind that turn's artifacts to the results panel: **View results** and **View visualization**. When a question can't be answered from the data — or doesn't need SQL at all — the agent replies in plain prose explaining why, with no results attached. That's a deliberate answer, not a failure. ## Work the results panel The results panel opens beside the chat whenever a turn produces rows or a chart: - **SQL Results** — the executed rows in a virtualized table (with the row count in the tab), the query's execution time, and infinite scroll for large results. If the query ran against a past dataset version, a **v*N*** pill marks it. - **Visualizations** — the chart, when you asked for one. The agent picks from GoPie's chart catalog (line, area, bar, pie, scatter, heatmap, and more) to match the shape of the data — and only draws when a chart was requested. The toolbar also holds [**Download results** and **Save as view**](/docs/chat/save-and-export/). Need more room? Collapse the chat pane with the panel toggle ([[⌘ \]]) and give the results the full width. On mobile, results open as a bottom sheet instead. ## Recognise the failure modes - **A failed query** shows a "Query execution failed" alert in the results panel with the error, the SQL that failed, and — when available — a suggestion for fixing it. The agent usually repairs failed queries itself mid-turn; this surface matters when you re-run a query by hand. - **A slow response** raises a banner — *"This is taking longer than usual."* — with **Reconnect** and **Stop**. If the stream dies, the banner turns into an error with **Retry**. - **A reloaded page resumes.** If you reload (or lose connection) mid-answer, the chat picks the stream back up where it left off — see [History, sharing & reliability](/docs/chat/history-and-sharing/). --- # Chat > History, sharing & reliability > Where conversations live, how visibility and share links work, and what happens when a stream drops mid-answer. URL: https://gopie.ai/docs/chat/history-and-sharing/ Chats persist automatically — every conversation is titled, listed, and re-openable. This page covers managing them, sharing them, and what GoPie does when a connection drops. ## Find past conversations The **History** button in the chat header (or **Previous Chats** on the empty state, or "Recent conversations" on the home launchpad) opens the full history: ![The chat history table with titles, scopes, visibility and dates](/docs/chat/chat-history.png "Every chat with its context scope, visibility, and timestamps") - Each row shows the **title**, the chat's **scope**, its **visibility**, and updated/created dates — sortable, searchable by title. - Titles are derived automatically from your first question; rename any chat by clicking its title in the chat header and typing — it saves in place. - Select rows to **delete** chats (singly or in bulk). Deletion is permanent and removes the transcript. > [!NOTE] Conversations attached to dashboards don't appear here — they live > with their dashboard, under its own Conversations menu. See > [Dashboard chat](/docs/dashboards/dashboard-chat/). Reopening a chat restores the transcript and its context chips, and re-runs the last query into the results panel. If a dataset the chat used has since been deleted, the transcript still reads fine — the queries just can't run anymore. ## Share a chat Chats are **private by default** — only you can open them. The **Share** button in the header sets the visibility: - **Private** — only you can view the chat. - **Organisation** — marks the chat as shared with your organisation. - **Public** — anyone with the link can view the chat, sign-in or not. Choosing Organisation or Public reveals the chat's link with a copy button. The link is the chat's address — there's no separate share token, so flipping visibility back to Private closes access immediately. > [!IMPORTANT] Today, a chat link opens for someone else only when visibility is > **Public** — organisation-wide viewing is still being wired up. To show a > teammate a conversation, use Public, or keep the analysis in a > [saved view](/docs/chat/save-and-export/) or [dashboard](/docs/dashboards/introduction/) > they can access through normal > [project permissions](/docs/collaboration/sharing-and-access/). A small icon next to the chat title always shows the current visibility. ## Rely on the stream Chat answers stream token by token, and GoPie assumes connections drop: - **Reload mid-answer and the stream resumes.** Your question is saved before generation starts, so refreshing the page (or briefly losing network) picks the answer back up rather than losing the turn. - **A watchdog watches the stream.** If tokens stop flowing, a banner offers **Reconnect**; if the stream truly fails, it shows the error with **Retry**. - **Stop anytime** with [[Esc]] or the stop button — the transcript keeps everything generated up to that point. --- # Chat > Save & export results > Turn a chat result into a reusable saved view, or download the rows as CSV, JSON, or Parquet. URL: https://gopie.ai/docs/chat/save-and-export/ A good chat answer is worth keeping. The results panel offers two ways: **Save as view** makes the query a permanent, queryable object; **Download results** exports the rows as a file. ## Save a result as a view The eye icon in the results panel toolbar turns the current query into a [saved view](/docs/query/saved-views/) — a live, reusable dataset that stays current as the underlying data changes. ![The Save as View dialog with project, name and description fields above the SQL preview](/docs/chat/save-as-view.png "The chat's SQL becomes the view's defining query") :::steps 1. **Open the dialog** — available once a turn has a successful query with rows. 2. **Pick the project** — the dialog offers the projects the query's datasets belong to, so the view lands next to its sources. 3. **Name it** — at least 3 characters. The description is optional (10–1000 characters): leave it blank and GoPie writes one from the SQL automatically. 4. **Create View** — GoPie saves it and opens the new view's page. ::: The chat's SQL becomes the view's defining query — [fixed at creation](/docs/query/saved-views/), like any view. From there it behaves like any dataset: chat with it, query it, [generate a dashboard from it](/docs/dashboards/ai-generation/), or expose it over [its REST API](/docs/query/dataset-api/). > [!TIP] Save a view when the *question* is durable — "monthly revenue by > category" — and you'll re-ask it. Download when you need the *rows* somewhere > else right now. ## Download the rows The download icon in the results panel opens the export dialog: - **Format** — CSV, JSON, or Parquet. - **SQL Query** — prefilled with the turn's query and editable, so you can widen a filter or drop columns before exporting (**Format SQL** tidies it up). Chat results shown in the panel are paged, but the export runs the query in full — no row cap. GoPie prepares the file server-side and opens it in a new tab when ready. Links stay valid for 24 hours, and repeating the same export within that window reuses the prepared file — the full export reference is at [Downloads & exports](/docs/query/downloads/). --- # Chat > Tips for better questions > How to phrase questions the SQL agent answers well — specificity, time windows, business terms, and productive follow-ups. URL: https://gopie.ai/docs/chat/better-questions/ The agent turns your sentence into SQL, so the more your sentence pins down, the less it has to guess. A useful habit: include a **measure**, a **grouping**, and a **time window**. ## Be specific | Too vague | Specific | | --- | --- | | "Show sales" | "Show total revenue by product category for 2026" | | "Customer data" | "List the top 10 customers by order count this quarter" | | "How are we doing?" | "Compare monthly revenue this year to last year" | What the agent extracts from a good question: an **action** (aggregate, list, compare), a **measure** (revenue), a **dimension** (product category), a **time window** (2026), and — if you ask — a **chart**. Vague questions force it to pick defaults you may not want. Templates that reliably work: - "Show me *[measure]* by *[dimension]* for *[time period]*" - "Compare *[measure]* between *[segment A]* and *[segment B]*" - "Top *[N]* *[things]* by *[measure]*, *[time period]*" ## Use your own vocabulary You don't need table or column names — [semantic search](/docs/chat/how-the-agent-works/) maps business terms to the schema, and fuzzy value matching fixes loose spellings ("electronis" finds **Electronics**). Two habits multiply this: - **Be consistent.** If your team says "bookings", keep saying "bookings" — don't alternate with "revenue" mid-chat. - **Invest in descriptions.** Terms resolve through your [dataset and column descriptions](/docs/datasets/column-descriptions/); the better they are, the more reliably your vocabulary lands. Broader background — metric definitions, business rules — belongs in [context documents](/docs/context/documents/). ## Say the ambiguous parts out loud Data has defaults you may not share. Spell them out when they matter: - **Nulls** — "treat missing discounts as zero" vs "exclude orders with no discount". - **Dates** — "group by week starting Monday", "fiscal year starting in July". - **Similar columns** — if the data has `revenue`, `net_revenue`, and `gross_revenue`, say which: "show gross revenue before discounts". - **Units and rounding** — "revenue in thousands", "percentages to one decimal". ## Ask for the chart you want Charts render only when requested. Name the type when you have one in mind — "as a line chart", "as a monthly trend" — or just say "chart" and the agent picks a fit for the data's shape. See [Reading results](/docs/chat/reading-results/). ## Follow up instead of restarting The conversation keeps your context — the agent knows what "that" refers to: 1. "Show me revenue by region" → 2. "Now break that down by product category" → 3. "Just the top 5, as a bar chart" Filters and refinements stack naturally: "only 2026", "exclude the West region", "same thing but for units sold". Two caveats: conversation memory is bounded (roughly the last twenty messages), and it's chat that remembers — not [the playground's Ask AI bar](/docs/query/natural-language-sql/), where every prompt stands alone. When you change topics, start a new chat. ## When the agent says it can't A plain-prose "I can't answer that from this data" is the agent being honest rather than inventing numbers. Useful responses: | Problem | Try | | --- | --- | | The data genuinely lacks the answer | Check the [dataset's columns](/docs/datasets/column-descriptions/) — the measure you want may live in another dataset; attach it | | The question used terms the schema doesn't reflect | Rephrase with more concrete terms, or fix the column descriptions so the terms resolve next time | | The scope was too broad | Attach the specific dataset instead of the whole project | | The question wasn't a data question | Chat answers from your data — general knowledge questions fall outside it | And when an answer looks *wrong* rather than missing: open **Show steps**, read the SQL, and check the assumption it made — then re-ask with that assumption corrected. Reading the generated SQL is also the fastest way to learn how your own data is structured. --- # Chat integrations > How chat integrations work > Connect a workspace once, then ask GoPie questions and get grounded answers inside Slack, Microsoft Teams and Google Chat. URL: https://gopie.ai/docs/integrations/overview/ Chat integrations bring GoPie into the messaging tools your team already uses. Instead of switching to the GoPie app, anyone can ask a question about your data right where they work — in a direct message, a channel, or with a slash command — and GoPie answers in the thread with a short summary, the table behind it, a chart where it helps, and a link back into GoPie for the full picture. GoPie connects to three platforms: - **Slack** — direct messages, channel `@mentions`, the assistant pane, and the `/gopie` command. - **Microsoft Teams** — personal chats and channel or group `@mentions`. - **Google Chat** — direct messages, spaces, and the `/ask` command. The same grounded Q&A you get in the GoPie app comes through in chat: every answer is produced by a real SQL query against your connected datasets, and it runs with the asker's own permissions. ## What you can do - **Ask a question** in plain language and get an answer in the conversation — the numbers, a table, and often a chart. - **Open the full result in GoPie** from a deep link in every answer, to keep exploring, save a view, or build a dashboard. - **Preview a GoPie link** — paste a dataset, dashboard, report or chat link into Slack or Teams and it expands into a compact card. - **Receive reports** — route scheduled and one-off [report deliveries](/docs/integrations/reports-to-channels/) to a channel or space, alongside email. ## How it works, end to end :::steps 1. **An admin connects the workspace** — an organisation owner or admin connects Slack, Teams or Google Chat from **Settings → Integrations**, once per platform. See the per-platform guides for the exact steps. 2. **People find GoPie in chat** — message the GoPie app directly, `@mention` it in a channel it belongs to, or use a slash command. 3. **Each person links their account** — the first question prompts the asker to link their GoPie account by signing in. From then on their questions run with their own data permissions. 4. **GoPie answers in the thread** — it picks the relevant datasets, writes and runs the query, and posts a self-contained answer with a link back to GoPie. 5. **Reports arrive where you work** — scheduled or manual report deliveries can be routed to a channel or space. ::: ## Who does what Connecting a workspace is an **admin action** — only an organisation owner or admin can do it, and it binds the whole workspace to your GoPie organisation. Asking questions is open to **anyone in the workspace** who links a GoPie account that belongs to your organisation. > [!IMPORTANT] Linking your account is mandatory before GoPie will answer with > data. Membership of the Slack, Teams or Google Chat workspace alone never grants > access to your data — the asker must have a linked GoPie account that is a member > of the organisation. This keeps chat access aligned with the permissions people > already have in GoPie. ## Your data and privacy GoPie only acts on the message you send it. It does **not** read your channel history, and it stores only the small amount of linking information needed to route replies and match a chat user to their GoPie account — not the content of your conversations. Answers are generated the same way as in the GoPie app: a real query against your data, cited back to the datasets it used. ## Next steps Pick your platform to connect it and see how your team asks: - [Slack](/docs/integrations/slack/) - [Microsoft Teams](/docs/integrations/microsoft-teams/) - [Google Chat](/docs/integrations/google-chat/) - [Deliver reports to channels](/docs/integrations/reports-to-channels/) --- # Chat integrations > Connect Slack > Connect your Slack workspace to GoPie, then ask questions and get answers in DMs, channels and the assistant pane. URL: https://gopie.ai/docs/integrations/slack/ Connecting Slack lets anyone in your workspace ask GoPie about your data — in a direct message, by `@mention`ing GoPie in a channel, from the assistant pane, or with the `/gopie` command — and get the answer back in the thread. ## Connect your workspace You connect Slack from inside GoPie. Always use GoPie's **Connect** button rather than Slack's own "Install to Workspace" button — connecting from GoPie is what binds the workspace to your organisation. :::steps 1. **Open integration settings** — in GoPie, go to **Settings → Integrations**. 2. **Start the Slack connection** — select **Connect** on the Slack row. 3. **Approve the permissions** — Slack shows what GoPie can do; review it and select **Allow**. 4. **Confirm it connected** — back in GoPie, the Slack row now reads *Connected*, with the workspace name and who connected it. ::: > [!NOTE] If your Slack workspace restricts which apps can be installed, a Slack > workspace admin may need to approve GoPie before the connection completes. This > is a Slack setting, separate from your GoPie role. ## Ask GoPie in Slack Once connected, anyone in the workspace can ask: - **Direct message** the GoPie app and type your question. - **`@mention` GoPie** in a channel it belongs to — add the app to the channel first so it can see the mention. - **Open the assistant pane** (Slack's Agents & AI apps view) and ask there. - **Use `/gopie`** followed by your question. GoPie replies in the thread with a short answer, the table behind it, a chart where it helps, and a link to open the full result in GoPie. ## Link your account The first time you ask, GoPie replies with a **Link account** prompt instead of an answer. This connects your Slack identity to your GoPie account so answers run with your own permissions. :::steps 1. **Select Link account** — from the prompt GoPie posts on your first question. 2. **Sign in with Slack** — confirm you're signed in to GoPie, then complete the Slack sign-in. 3. **Ask again** — your question now returns a real answer, running as your linked GoPie user. ::: > [!TIP] A fresh link can take up to a minute to take effect. If you still see the > prompt right after linking, wait a moment and ask again. You can unlink at any time from **Settings → Profile → Linked chat accounts**; your next question will ask you to link again. ## Preview GoPie links When someone pastes a GoPie link — a dataset, dashboard, report, view or chat — into a channel or DM the app can see, it expands into a compact card showing the item's title, type, project and when it was last updated. Only linked members of your organisation see the full card; others get an invitation to connect their account. ## Deliver reports to a channel Scheduled and one-off reports can be routed to a Slack channel. See [Deliver reports to channels](/docs/integrations/reports-to-channels/). ## Disconnect To disconnect, return to **Settings → Integrations** and disconnect the Slack row. This revokes GoPie's access to the workspace and removes the connection. --- # Chat integrations > Connect Microsoft Teams > Connect your Microsoft Teams tenant to GoPie so your team can ask questions in personal chats and channels. URL: https://gopie.ai/docs/integrations/microsoft-teams/ Connecting Teams lets your team ask GoPie about your data in a personal chat or by `@mention`ing it in a channel or group chat, and get the answer back in the conversation. Setting up Teams has two parts: an **organisation owner or admin** connects the tenant from GoPie, and a **Teams administrator** allows and uploads the GoPie app in the Teams admin center. The order matters. > [!IMPORTANT] Connect in GoPie **first**, then install the app in Teams. GoPie > recognises your tenant only after you connect it, so if the app is installed in > Teams before the GoPie connection exists, GoPie ignores the install and the bot > won't come online until you connect and re-add it. ## Step 1 — Connect the tenant in GoPie :::steps 1. **Open integration settings** — in GoPie, go to **Settings → Integrations**. 2. **Start the Teams connection** — select **Connect** on the Microsoft Teams row. 3. **Grant admin consent** — sign in with a Microsoft administrator account and consent to GoPie for your tenant. 4. **Confirm it connected** — back in GoPie, the Teams row now shows the tenant as connected. ::: ## Step 2 — Allow and upload the app in Teams A Teams administrator does this in the [Teams admin center](https://admin.teams.microsoft.com/). :::steps 1. **Allow custom app upload** — go to **Teams apps → Setup policies → Global** and turn **Upload custom apps** on. It is off by default in most tenants. 2. **Allow the app org-wide** — under **Teams apps → Manage apps**, make sure custom apps are permitted. 3. **Upload the GoPie app** — under **Manage apps → Upload new app**, upload the GoPie Teams app package for the whole organisation (or let individual users add it with **Upload a custom app**). ::: > [!NOTE] Your GoPie contact provides the Teams app package to upload. Changes to > "Upload custom apps" can take up to 24 hours to take effect across a tenant. > [!WARNING] Custom app upload is not available in some Microsoft environments > (GCC High, DoD and 21Vianet), and some organisations disable it by policy. If > your tenant blocks custom apps, talk to your GoPie contact about the Teams Store > distribution path. ## Ask GoPie in Teams Once the app is installed, anyone in the tenant can ask: - **Message the GoPie bot** in a personal (1:1) chat. - **`@mention` GoPie** in a channel or group chat it belongs to. GoPie replies in the conversation with a short answer, the table behind it, a chart where it helps, and a link to open the full result in GoPie. ## Link your account Just like Slack, the first question prompts each person to link their GoPie account so answers run with their own permissions. :::steps 1. **Select Link account** — from the prompt GoPie posts on your first message. 2. **Sign in** — confirm your GoPie account and complete the sign-in. 3. **Ask again** — your question now returns a real answer, running as your linked GoPie user. ::: You can unlink at any time from **Settings → Profile → Linked chat accounts**. ## Deliver reports to a channel Scheduled and one-off reports can be routed to a Teams channel. See [Deliver reports to channels](/docs/integrations/reports-to-channels/). ## Disconnect To disconnect, return to **Settings → Integrations** and disconnect the Microsoft Teams row. --- # Chat integrations > Connect Google Chat > Allowlist the GoPie app, connect your Google Workspace with a one-time code, and ask questions in Google Chat. URL: https://gopie.ai/docs/integrations/google-chat/ Connecting Google Chat lets your team ask GoPie about your data in a direct message or a space, and get the answer back in the conversation. Google Chat connects a little differently from Slack and Teams: there's no install button in GoPie. Instead you mint a one-time **connect code** in GoPie and redeem it in Chat with `/connect`. Before that works, a **Google Workspace administrator** has to allow the GoPie app for your organisation. > [!IMPORTANT] Google Chat integrations require a **Google Workspace** account. > Personal Google accounts (`@gmail.com`) can't connect. ## Step 1 — Allowlist the app (the biggest hurdle) The most common reason a Google Chat setup stalls is this step. Most organisations only let their users install apps that an administrator has explicitly allowed, so until GoPie is on that list, nobody in your organisation can find or message it. A Google Workspace administrator does this once, in the [Google Admin console](https://admin.google.com/): :::steps 1. **Open Marketplace apps** — go to **Apps → Google Workspace Marketplace apps**. 2. **Add and allowlist GoPie** — add the GoPie app and allow it for everyone, or for the specific organisational units or groups that should use it. 3. **Confirm availability** — once allowed, GoPie becomes findable in Google Chat for those users. ::: > [!NOTE] If GoPie doesn't appear when you search for it in Google Chat, it hasn't > been allowlisted yet — check with your Workspace administrator before going > further. ## Step 2 — Connect your workspace with a code With the app allowlisted, an organisation owner or admin links the workspace to your GoPie organisation. :::steps 1. **Generate a connect code** — in GoPie, go to **Settings → Integrations → Google Chat** and select **Connect** to mint a single-use code (it expires after 15 minutes). 2. **Redeem it in Chat** — in Google Chat, open a direct message with the GoPie app and send `/connect` followed by your code. 3. **Finish in GoPie** — the bot replies with a **Finish in GoPie** card; select it to return to settings, where the connection is completed and marked connected. ::: ## Ask GoPie in Google Chat Once connected, anyone in the workspace can ask: - **Direct message** the GoPie app and type your question. - **`@mention` GoPie** in a space it has been added to. - **Use `/ask`** followed by your question. GoPie replies in the conversation with a short answer and the table behind it, plus a link to open the full result in GoPie. ## Link your account The first question prompts each person to link their GoPie account by signing in with Google, so answers run with their own permissions. :::steps 1. **Select the link prompt** — GoPie posts it on your first question. 2. **Sign in with Google** — this proves you own a GoPie account. 3. **Ask again** — your question now returns a real answer, running as your linked GoPie user. ::: You can unlink at any time from **Settings → Profile → Linked chat accounts**. ## Deliver reports to a space Scheduled and one-off reports can be routed to a Google Chat space. See [Deliver reports to channels](/docs/integrations/reports-to-channels/). ## Disconnect To disconnect, return to **Settings → Integrations** and disconnect the Google Chat row. --- # Chat integrations > Deliver reports to channels > Route scheduled and one-off report deliveries to a Slack channel, Teams channel or Google Chat space, alongside email. URL: https://gopie.ai/docs/integrations/reports-to-channels/ Once a chat platform is connected, GoPie can deliver [reports](/docs/reports/introduction/) to a channel or space, so the people who need them see them where they already work — not only in their inbox. A short "report ready" message links straight to the full report in GoPie. > [!NOTE] Delivering to a channel is in addition to, not instead of, email > delivery. You can send a report to a channel, to email recipients, or both. ## Before you start - The chat platform must be **connected** to your organisation. See [Slack](/docs/integrations/slack/), [Microsoft Teams](/docs/integrations/microsoft-teams/) or [Google Chat](/docs/integrations/google-chat/). - GoPie must be **present in the destination** — add the GoPie app to the channel or space you want to deliver to. GoPie can only post where it has been added. ## Choose a chat destination You set a report's delivery when you schedule it, or when you send one on demand — the same place you'd add email recipients. Pick a connected workspace and the channel or space to post to, and GoPie delivers the report there each time it runs. Managing report schedules and recipients is covered in the [Reports](/docs/reports/introduction/) section; this page covers only the chat destination. ## Per-platform notes :::tabs == Slack Deliver to any channel the GoPie app has been added to. Add the app to a private channel first if you're routing there — GoPie can't post to a channel it isn't a member of. == Microsoft Teams Deliver to a channel in a team where the GoPie app is installed. If a channel isn't available as a destination, confirm the app is installed for that team. == Google Chat Deliver to a space the GoPie app has been added to. Add GoPie to the space before selecting it as a destination. ::: > [!TIP] Route each report to the channel whose members actually own it — a weekly > revenue report to the revenue channel, a pipeline report to the sales space — so > the "report ready" nudge lands with the right people. --- # Dashboards > Dashboards in GoPie > The three dashboard kinds, the live canvas and its tiles, and the toolbar that filters everything at once. URL: https://gopie.ai/docs/dashboards/introduction/ A GoPie dashboard is a **live canvas** — every tile is backed by a real SQL query that re-runs as you change filters, so what you see is always the current data, never a saved picture. (Need a frozen, citable document instead? That's a [report](/docs/reports/introduction/).) ## Know the three kinds Every dashboard belongs to one owner: - **Dataset dashboards** — one per dataset, [generated](/docs/dashboards/ai-generation/) from its page. Single-source. - **View dashboards** — one per [saved view](/docs/query/saved-views/), the same way. Single-source, and a neat trick: build a view that joins datasets, then dashboard the view. - **Project dashboards** — composed through a wizard, spanning **multiple datasets and views** at once. A project can have any number. The project's **Dashboards** tab lists all three in one table, with **All / Project / Datasets / Views** filter tabs and search. Your home launchpad also lists your recent dashboards. ## Tour the canvas ![A generated dashboard canvas with KPI tiles, charts and an AI-written summary](/docs/dashboards/canvas.png "wide: Every tile is a live query — KPIs, charts, leaderboards, tables and text") A canvas is rows of **tiles**, each one of five types: | Tile | What it shows | | --- | --- | | **KPI grid** | Headline numbers with sparklines and a vs-previous-period comparison | | **Chart** | One of GoPie's chart kinds — line, area, bar, pie, scatter, heatmap, and more | | **Leaderboard** | A ranked "top N by measure" list | | **Table** | Raw rows for the detail-hungry | | **Text** | Markdown — AI-written summaries land here, and your own notes can too | The **toolbar** governs the whole canvas at once: a time range (default **All time**), a comparison mode (such as **Previous period**), filter pills you can add and remove, and a refresh button that re-runs every query. Since tiles are live queries, a filter change updates every number on the page. A **results dock** slides up when [dashboard chat](/docs/dashboards/dashboard-chat/) produces query results — **Hide** collapses it to a strip, the **×** dismisses it. ## Edit by hand **Edit dashboard** (the pencil) switches the canvas into edit mode — press **Done** to leave it: - **Add component** creates a new Chart, Table, Text, KPI, or Leaderboard tile. - Every tile gets an **Edit tile** control, plus **Duplicate** and **Delete** (deleting offers an **Undo** in the toast). - Arrange with the move controls (or [[⌘ ⇧]] + arrow keys), and size tiles with the width stepper (a 12-column grid). For everything beyond arrangement — changing what a tile *shows* — describe it to [dashboard chat](/docs/dashboards/dashboard-chat/) instead: "make this a monthly trend", "add a tile for returns by region". Every change, manual or AI, lands in [version history](/docs/dashboards/version-history/). ## Know who can edit Organisation **owners and admins** can edit any dashboard; **members** can edit dashboards they created. Viewing follows access to the dashboard's source data. Editing controls simply don't appear when you lack the permission — the full model is in [Roles & permissions](/docs/collaboration/roles-and-permissions/). ## Know what doesn't exist (yet) - **No public sharing or share links** — dashboards are visible inside your organisation only. - **No image or PDF export** of the canvas. Individual tiles can be downloaded as CSV or PNG from their menus; for a portable document, [generate a report](/docs/reports/introduction/). - **No full-screen or presentation mode** for the canvas as a whole — individual chart tiles do have a **Fullscreen** expand. --- # Dashboards > AI dashboard generation > One-click generation from a dataset or view, and the multi-dataset project wizard — with every phase streamed as it happens. URL: https://gopie.ai/docs/dashboards/ai-generation/ GoPie designs dashboards for you: it profiles the data, proposes the metrics that matter, **validates every query against your actual schema**, and lays out the canvas. Nothing lands on the dashboard that didn't execute. ## Generate from a dataset or view Click **Generate dashboard** on a dataset or view page. GoPie opens the dashboard and streams its progress phase by phase, each with its timing: 1. **Analyzing your data** 2. **Selecting key metrics** 3. **Validating queries** 4. **Designing the layout** 5. **Finalizing your dashboard** Expect seconds to a minute. The result is the full canvas from the [introduction](/docs/dashboards/introduction/) — KPI tiles, charts, and an AI-written summary. The walkthrough with a real dataset is in [Generate your first dashboard](/docs/getting-started/generate-your-first-dashboard/). > [!NOTE] If the AI layout step can't produce a valid canvas, GoPie falls back > to a deterministic design built from the dataset's profile — you still get a > working dashboard, just a plainer one. Metrics that fail validation are > replaced the same way, so every tile always runs. ## Compose a project dashboard **Generate from project** (on the Dashboards tab) opens a four-step wizard for multi-dataset dashboards: ![The project dashboard wizard on the dataset-selection step with AI recommendations](/docs/dashboards/project-wizard.png "The wizard recommends datasets for your goal — adjust and continue") :::steps 1. **Intent** — describe the goal in a sentence or two ("Track revenue, margin by supplier, and monthly trends"). **Suggest from project context** drafts one from your project's descriptions. 2. **Datasets** — GoPie profiles the project's datasets and pre-selects the ones **Recommended for your goal**; adjust the selection freely (dashboards cap the number of sources for performance). 3. **Suggested Views** — where the chosen datasets share join keys, the wizard proposes joined [views](/docs/query/saved-views/) to power cross-dataset tiles. Inspect each **View SQL definition**, toggle any off, or **Skip all**. 4. **Generate** — the same streamed phases as above, then **Open Dashboard**. ::: The suggested views are created as real saved views alongside the dashboard, so you can query and reuse them independently. ## Regenerate Running **Generate dashboard** again on a dataset or view that already has one asks before overwriting — regeneration replaces the canvas (the old design stays recoverable in [version history](/docs/dashboards/version-history/)). Generating the first dashboard is open to anyone who can see the dataset; regenerating an existing one follows the [edit rules](/docs/dashboards/introduction/). ## When generation fails If generation fails outright, the progress card says so with **Try Again** and **Edit Selections**; cancelling mid-run saves nothing and keeps your wizard selections. A stall notice appears if a phase takes unusually long — generation continues in the background either way. --- # Dashboards > Editing with dashboard chat > Ask questions about a dashboard or edit it in plain language — with Ask previews you apply deliberately and Edit changes that land directly. URL: https://gopie.ai/docs/dashboards/dashboard-chat/ The **Chat** button on the dashboard toolbar opens a conversation that knows the whole dashboard — its data sources, its metrics, and the current canvas. It answers questions *and* makes changes, depending on the mode. ![The dashboard chat panel beside the canvas, with a proposed change awaiting Apply](/docs/dashboards/dashboard-chat.png "In Ask mode, changes arrive as previews — nothing lands until you Apply") ## Ask questions In **Ask** mode ("Chat with dashboard"), the conversation behaves like [chat with your data](/docs/chat/how-the-agent-works/), pre-scoped to the dashboard's sources: "why did March dip?", "what's the average order value in the West region?". Query results open in the dashboard's results dock, and answers cite their SQL the same way chat always does. If an Ask-mode question produces a canvas improvement, it arrives as a **preview card** — a dashed outline describing the change with **Apply** and **Dismiss**. Nothing touches the dashboard until you Apply. ## Edit in plain language In **Edit** mode ("Edit dashboard"), your instructions change the canvas directly: "add a KPI for average discount", "make the revenue chart monthly", "drop the segment table". The change is validated the same way generation is — the new tile's query must execute — and the canvas updates in place. Every applied change records a [version](/docs/dashboards/version-history/), so nothing is unrecoverable. ## Scope an edit to one tile Edits target the **whole dashboard** by default. To work on a single tile, focus it — click **Refine with AI** on the tile — and the chat's scope chip switches to that tile: "show this as a share of total", "sort descending". The agent then sees just that tile (and its row neighbours for layout), leaving the rest of the canvas untouched. ## Keep the conversations Each dashboard keeps its own chat threads, separate from your [global chat history](/docs/chat/history-and-sharing/): - The **Conversations** menu in the panel header lists past threads by title — reopen any, or start a **New chat**. - **Closing the panel ends the session** — reopening starts a fresh conversation (pick up an old one from Conversations). Reloading the page with the panel open resumes the stream, like chat everywhere. Who can do what follows the [dashboard edit rules](/docs/dashboards/introduction/): anyone who can view the dashboard can Ask; applying changes and Edit mode require edit permission. --- # Dashboards > Version history & restore > Every dashboard change is a restorable version — browse them, restore safely, and pin the ones that must never expire. URL: https://gopie.ai/docs/dashboards/version-history/ Every change to a dashboard — a manual tile edit, an AI generation, a chat edit, a restore — records a version. The canvas is never one bad edit away from lost work. ## Browse the history The **Version history** button on the toolbar opens the panel: ![The version history panel listing versions with their sources and actions](/docs/dashboards/version-history.png "Each version shows what changed it — manual edit, AI chat, generation, or restore") Each row shows the version's label (a custom name, or the AI's summary of the change), when and who, and its origin: - **Manual edit** — canvas edits by hand. Rapid consecutive edits by the same person fold into one version, so dragging tiles around for five minutes doesn't create fifty entries. - **AI generated** / **AI refined** / **Edited via AI chat** — each AI change is its own version, labelled with what the AI did. - **Restored from earlier version** — restores are versions too. The current canvas is tagged **Current**. ## Restore, name, pin The row menu holds three actions: - **Restore this version** — brings the canvas back to that state. **Your current dashboard is saved as a new version first**, so a restore never destroys anything — you can restore the restore. - **Name this version…** — give a milestone a label ("Q2 board review"). Named versions are kept forever. - **Pin** — same permanence without renaming. Restoring is available to anyone with [edit permission](/docs/dashboards/introduction/); naming and pinning are offered while the dashboard is in edit mode. ## Understand retention History is pruned in the background: each dashboard keeps its **most recent ~50 versions**, but **pinned and named versions never expire**. Pin anything you might want to return to months later; the unpinned middle history is working memory, not an archive. > [!TIP] Before a big AI restructuring — "redesign this around weekly cohorts" — > pin the current version. If the new direction disappoints, restore is one > click, and the experiment cost nothing. --- # Reports > Reports in GoPie > What a report is, how frozen snapshots differ from live dashboards, and who can create and run them. URL: https://gopie.ai/docs/reports/introduction/ A **report** is a recurring deep-dive document: you define the business questions once, and GoPie generates a written analysis — narrative, charts, tables, and an executive summary — answering each of them against your data. Think of it as the document a great analyst would write, on demand. ## Frozen, not live The defining property: a generated report is a **snapshot**. Its numbers, charts, and tables are captured at generation time and never change afterwards — unlike a [dashboard](/docs/dashboards/introduction/), whose every tile re-queries live data. | | Dashboard | Report | | --- | --- | --- | | Data | Live — re-queries on every filter change | Frozen at generation time | | Form | Interactive canvas of tiles | Written document with narrative | | Answers | What's happening *now* | What was true *then*, with the analysis attached | | Evidence | Each tile's SQL | Each section's SQL, plus citations into [context documents](/docs/context/documents/) | | Best for | Monitoring | Month-end reviews, stakeholder updates, anything you'd cite later | That permanence is the point: a report generated for the Q2 review still says exactly what it said in the Q2 review. ## Know the anatomy A report has two layers: - **The definition** — the title, an optional intent, the data sources (datasets and views), and the ordered list of **questions**, each bound to a source and optionally a chart. Editing the definition never touches existing documents. - **The snapshots** (runs) — every generation produces a new numbered snapshot from the current definition and the data as of that moment. The report page lists them all: your run history *is* the historical record. Each question becomes a **section** of the document: a heading, a written narrative, the frozen result table, an optional chart, and the SQL that produced it. An **executive summary** opens the document. Sections answered from context documents carry numbered **citations** back to the source files. ## Know who can do what - **Create** — organisation owners and admins (the Reports tab says so if you can't). - **Edit the definition, generate snapshots, edit snapshots** — owners, admins, and the report's creator. - **Delete a report or snapshot** — the organisation owner or the report's creator. Deleting a report removes all its snapshots (type the report's name to confirm). - **Read** — everyone in the organisation with access to the project. Continue with [Creating a report](/docs/reports/creating-a-report/), then [Snapshots, editing & export](/docs/reports/runs-and-exports/). --- # Reports > Creating a report > The four-step wizard — goal, sources, questions with AI suggestions and validation, and the pre-generation review. URL: https://gopie.ai/docs/reports/creating-a-report/ **New report** on the project's Reports tab opens a four-step wizard: **Goal → Datasets → Questions → Review**. The craft is almost entirely in step 3 — the questions *are* the report. ## Set the goal Give the report a **title** ("Quarterly Revenue Deep-Dive") and, optionally, describe what it should answer. The intent is worth a sentence or two: it drives the dataset recommendations and AI-drafted questions in the next steps. ## Pick the sources Choose the datasets and [views](/docs/query/saved-views/) the report draws from — each question is answered against exactly one of them. With an intent set, GoPie profiles the project and pre-selects sources **Recommended for your goal**; adjust freely, up to 12 sources. > [!TIP] A view that pre-joins your datasets often makes a better source than > the raw tables — the questions get simpler, and the join logic lives in one > inspectable place. ## Write the questions ![The questions step with AI-suggested questions, per-question sources and validation states](/docs/reports/wizard-questions.png "Each question becomes a section of the document, in this order") Add the business questions in the order the document should answer them. Per question: - **The question itself** — phrased like you'd [ask in chat](/docs/chat/better-questions/): a measure, a grouping, a time window. - **A source** — the dataset or view it runs against. - **Include chart** — optionally, with the kind (line, bar, donut, heatmap, and more). - **A section group** — optionally, to organise long reports. Two AI assists sit above the list: - **Suggest questions** drafts questions from your intent and the selected schemas — grounded in what the data can actually answer, so it won't propose metrics your columns can't support. Suggestions are appended with an **AI** tag (they never overwrite yours); edit or delete them like any question. - **Validate** probes every question *now*: GoPie generates SQL for each, executes it, and shows a pass/fail per question — with the SQL and a sample of rows to inspect. Passing questions get their SQL **frozen** into the definition, so future snapshots re-run the exact same query deterministically. If the project has indexed [context documents](/docs/context/documents/), questions can also be **From documents** (answered by researching the documents rather than SQL), and data questions can be **grounded with documents** for narrative context. ## Review and generate The review step shows the report's outline with each question's validation state, and flags the consequences plainly: unvalidated or failing questions fall back to fresh NL→SQL at snapshot time — they'll usually work, but the SQL is regenerated per run rather than frozen. **← Back to fix** returns to the editor; **Validate now** probes everything in place. **Preview report** renders the passing questions as a draft document. **Generate report** creates the report, starts the first snapshot in the background, and takes you to the report page — generation takes a couple of minutes for a typical question list, and the page updates when it's ready. > [!NOTE] Scheduling is coming soon — reports run on demand today. Generate a > new snapshot anytime with **Generate now**. The definition stays editable after creation — **Settings** on the report page reopens the same question editor. Changes apply to the *next* snapshot; existing snapshots stay frozen. Continue with [Snapshots, editing & export](/docs/reports/runs-and-exports/). --- # Reports > Snapshots, editing & export > The run history, reading a generated document, editing a snapshot without losing the original, and exporting to PDF. URL: https://gopie.ai/docs/reports/runs-and-exports/ Every generation produces a numbered **snapshot** — a frozen document. The report page is the run history; each snapshot is the deliverable. ## Track the runs The report page lists snapshots newest-first with a status each: **Queued → Generating → Complete** (or **Failed**, with the error). **Generate now** starts a fresh snapshot from the current definition; one runs at a time per report. A snapshot whose individual question fails still completes — the failed question becomes a section marked as such, never a lost document. Snapshots accumulate until you delete them (row menu → **Delete snapshot**) — there's no automatic expiry, so the history doubles as an archive of what the data said over time. ## Read the document A completed snapshot opens as a document: - **Executive summary** — the AI's synthesis across all sections, up top. - **Sections**, one per question, in your order: a heading, a written narrative of the finding, the **frozen result table**, and the chart if you asked for one. Charts and tables are baked at generation — they render instantly and never drift. - **Provenance everywhere** — each section records the SQL that produced it and the source it ran against; sections grounded in [context documents](/docs/context/documents/) carry numbered citations that link to the exact source files at the bottom of the document. ## Edit a snapshot Generated prose is a draft you're allowed to improve. With edit permission, the document itself is editable in place — the executive summary, section headings and narratives (markdown), chart titles, and which table columns show. You can also **Add question** between sections: GoPie answers it against the *current* data and splices the new section in. Two safety nets: - The first edit preserves the original — **Revert to generated** restores it anytime. - The snapshot records when it was last edited and by whom, so an edited document is never mistaken for raw AI output. ## Export to PDF **Export PDF** in the snapshot header produces the portable version. The dialog offers portrait or landscape, which figures to include, and a row cap per table (50–500) — the summary and written analysis are always included, with a page estimate before you commit. PDF is the export format; for machine-readable data, export the underlying rows from [the datasets themselves](/docs/query/downloads/). > [!TIP] Wide tables read far better in **Landscape** — the dialog warns you > when a table is likely to overflow portrait width. --- # Context & knowledge > What makes answers smarter > The four kinds of context GoPie's AI draws on — descriptions, documents, custom prompts, and dictionaries — and which features use which. URL: https://gopie.ai/docs/context/overview/ The AI is only as good as what it knows about your data. GoPie gives you four places to invest, each with a different job. This page is the map; the rest of the section covers each in depth. ## Know the four layers 1. **[Dataset & column descriptions](/docs/datasets/column-descriptions/)** — what each table and column *means*. The foundation: descriptions are indexed for the agent's semantic search and injected into every prompt. If you do one thing, do this. 2. **[Context documents](/docs/context/documents/)** — project-level reference material: data dictionaries, codebooks, metric definitions, methodology notes. The AI searches them when a question needs background, and [reports](/docs/reports/introduction/) cite them. 3. **[Custom AI prompts](/docs/context/custom-prompts/)** — standing instructions attached to a project, dataset, or view: "revenue figures are in EUR", "always exclude test accounts". Injected verbatim, every time. 4. **[Dictionaries](/docs/context/dictionaries/)** — CSV lookup tables mapping coded values to labels, used by [data cleaning](/docs/datasets/data-cleaning/) during ingestion. ## See what reaches which feature | Context | [Chat](/docs/chat/how-the-agent-works/) | [Playground Ask AI](/docs/query/natural-language-sql/) | [Dashboard generation](/docs/dashboards/ai-generation/) | [Reports](/docs/reports/introduction/) | | --- | --- | --- | --- | --- | | Dataset & column descriptions | ✓ | ✓ | ✓ | ✓ | | Dataset / view custom prompt | ✓ | ✓ | — | — | | Project custom prompt | ✓ | — | — | — | | Context documents | ✓ searched on demand | ✓ | — | ✓ suggestions, document questions & grounding | | Dictionaries | — | — | — | — (data cleaning only) | Two rows worth reading twice: the **project custom prompt reaches chat but not the playground's Ask AI bar** (per-dataset instructions belong on the dataset), and **dictionaries influence no query-time feature at all** — they act during data cleaning. ## Invest where it pays - **Answers keep missing the right column?** Fix the [column descriptions](/docs/datasets/column-descriptions/) — discovery is a search over them. - **The AI misuses a metric?** Put the definition in a dataset [custom prompt](/docs/context/custom-prompts/) ("gross_revenue is pre-discount; use net_revenue for 'revenue'"). - **Questions need business background?** Upload the methodology doc as a [context document](/docs/context/documents/) — chat will cite it, and report questions can be grounded in it. - **Coded values confuse ingestion?** That's a [dictionary](/docs/context/dictionaries/). --- # Context & knowledge > Context documents > Upload reference documents — data dictionaries, codebooks, methodology notes — that the AI searches and reports cite. URL: https://gopie.ai/docs/context/documents/ Context documents are project-level reference material the AI can actually read: once indexed, [chat](/docs/chat/how-the-agent-works/) searches them when a question needs background, the [playground's Ask AI](/docs/query/natural-language-sql/) consults them while planning SQL, and [reports](/docs/reports/creating-a-report/) can answer whole questions from them — with citations. Good candidates: metric definitions, column codebooks, methodology write-ups, survey questionnaires, business glossaries. The test: *would a new analyst need this document to interpret the data correctly?* ## Upload documents The project's **Context** tab lists the documents; **Upload documents** opens a two-step wizard. ![The Context documents list showing files with their indexing status](/docs/context/documents.png "Each document shows its type, size, and indexing status") :::steps 1. **Upload** — drop one or more files: PDF, DOCX, XLSX, CSV, JSON, MD, or TXT. Then **Add documents**. 2. **Confirm** — the success screen tracks each document's indexing live. Indexing runs in the background (documents are chunked and embedded for semantic search), so you can **Go to Context** immediately. ::: Each document in the list carries a status — **Indexed**, **Indexing…**, **Not indexed**, or **Failed**. Only indexed documents are searchable; if any fail or stall, a **Re-index** button appears above the list to retry them in bulk. GoPie also writes a short description of each document automatically once it's read. ## Use them Nothing to configure — once indexed, the documents are part of the project's context: - In **chat**, the agent searches them on demand and lists consulted files under the answer as **Sources cited**. - In **reports**, indexed documents unlock **From documents** questions and the **Ground with documents** option on data questions; generated sections cite the exact files. - In the **playground**, Ask AI consults them while planning a query. > [!TIP] Documents complement descriptions, not replace them. A column's > one-line meaning belongs in its [description](/docs/datasets/column-descriptions/) > (always injected); the ten-page methodology belongs in a document (searched > when needed). ## Manage the library - **Open a document** to read it in place (PDFs, markdown, and text render inline) or download the original. - **Delete** removes the document and its search index — permanently. Existing report snapshots that cited it keep their citations; the links just have nowhere to go. --- # Context & knowledge > Dictionaries > CSV lookup tables that map coded values to human-readable labels, used by data cleaning during ingestion. URL: https://gopie.ai/docs/context/dictionaries/ A **dictionary** is a lookup table — a CSV mapping coded values to their human-readable meanings: state codes to state names, survey response codes to answer text, product SKU prefixes to categories. During [data cleaning](/docs/datasets/data-cleaning/), GoPie searches your project's dictionaries to infer what coded columns mean and how to standardise them. ## Create a dictionary The project's **Dictionaries** tab lists them; **Create Dictionary** opens the editor. ![The dictionary editor with name, description and the CSV upload](/docs/context/dictionary-form.png "A dictionary is a named CSV lookup table") :::steps 1. **Name it** — for example "State codes". A description of what it covers is optional but helps. 2. **Upload the CSV** — a lookup table mapping coded values to their labels (up to 50 MB). 3. **Create dictionary** — GoPie reads the file and infers a column-by-column schema for it automatically, including per-column descriptions. ::: Editing later works the same way — update the name and description, or replace the CSV outright. Dictionaries are **project-scoped**: they apply to datasets cleaned in this project. ## Know what they do (and don't) Dictionaries feed exactly one flow: the **data-cleaning standardisation step** during dataset creation, where GoPie infers a target schema for messy data. A matching dictionary tells that inference what `"TX"` or `"resp_4"` actually means, producing better column mappings and standardised values. > [!NOTE] Dictionaries do not influence chat, SQL generation, dashboards, or > reports — at query time the AI works from > [descriptions](/docs/datasets/column-descriptions/), > [documents](/docs/context/documents/), and > [custom prompts](/docs/context/custom-prompts/). If you want the *meaning* of > coded values available when querying, put it in the column description or > upload the codebook as a context document. ## Delete Deleting a dictionary (row menu, or bulk-select) is permanent. Datasets already cleaned with its help are unaffected — the dictionary informed their creation; it isn't referenced afterwards. --- # Context & knowledge > Custom AI prompts > Standing instructions on a project, dataset, or view that the AI receives every time it works with that data. URL: https://gopie.ai/docs/context/custom-prompts/ A custom prompt is a standing instruction the AI receives **every time** it touches the data it's attached to — no need to repeat "amounts are in EUR" in every chat. GoPie has three levels; each has a distinct job and a distinct reach. ## Know the three levels | Level | Where to edit | Reaches | | --- | --- | --- | | **Dataset** | Dataset **Settings → General → Custom AI prompt** | [Chat](/docs/chat/how-the-agent-works/) and the [playground's Ask AI](/docs/query/natural-language-sql/), whenever this dataset is in play | | **View** | View **Settings → Custom AI Prompt** | Same — for queries against the view | | **Project** | The project overview's description banner | **Chat only**, when the project is attached — the playground's Ask AI doesn't receive it | All three save as you type them — edit, click away, done. There's no length limit, but shorter is better: these tokens spend context on every question. > [!IMPORTANT] Because the project prompt doesn't reach the playground, any > instruction that must hold for *SQL generation on a specific dataset* belongs > on the **dataset**, not the project. Reserve the project prompt for > conversational ground rules. ## Write instructions that work Good custom prompts state facts and rules the schema can't express: - **Semantics** — "`amount` is in cents; divide by 100 for euros." - **Business rules** — "Always exclude rows where `is_test = true`." - **Preferred defaults** — "When asked about revenue, use `net_revenue` unless gross is requested explicitly." - **Domain framing** (project level) — "This project tracks clinical trial data; 'subjects' means trial participants." Anti-patterns: restating what descriptions already say, listing column definitions one by one (that's what [column descriptions](/docs/datasets/column-descriptions/) are for), or pasting whole documents (upload them as [context documents](/docs/context/documents/) instead — searched when relevant, rather than spent on every prompt). ## Verify the effect Custom prompts are invisible in the transcript — the fastest way to confirm one is working is to ask a question it should change ("what's our revenue?" after the net-revenue rule) and check the generated SQL under **Show steps**. --- # Collaboration & admin > Organisations & members > Creating and switching organisations, inviting teammates, changing roles, transferring ownership, and leaving or deleting an organisation. URL: https://gopie.ai/docs/collaboration/organisations-and-members/ Everything in GoPie lives inside an **organisation** — projects, datasets, chats, dashboards, applications. Your team shares one organisation; every resource you create belongs to it, and only its members can ever see it. When you sign up, GoPie quietly creates a first organisation for you (named after you) and makes you its **owner** — you never start without one. ## Switch and create organisations You can belong to several organisations, but you work in exactly one at a time — the **active organisation**. The switcher lives in the account menu at the bottom of the sidebar: it lists your organisations with your role in each, and **Create organisation** sits at the bottom of the list. Switching lands you back on the home page with everything — projects, chats, dashboards — re-scoped to the new organisation. Creating an organisation takes a name (an URL slug is derived automatically); you become its owner. > [!NOTE] On self-hosted deployments where registration is disabled, creating > organisations is disabled too — an administrator provisions them from the > [admin console](/docs/collaboration/admin-console/). ## Manage members **Settings → Organisation** is the team page: the organisation's name (owners can rename it in place), the members table with each person's role badge, and pending invitations. ![The organisation settings page with members and their role badges](/docs/collaboration/org-members.png "wide: Members with owner/admin/member badges — the invite form is owner-only") ### Invite someone Owners invite by email: enter the address, pick **Member** or **Admin** as the starting role, and press **Invite**. The invitation: - lands in the recipient's inbox as a link (and, if they already use GoPie, appears under **Settings → Invitations** and as a badge in their sidebar), - **expires after 48 hours** — cancel a pending one anytime with the **×** next to it, - is accepted on a page showing the organisation, the role offered, and who invited them; new users create their account first, then accept. > [!NOTE] Invitations are how teams grow on gopie.ai. On self-hosted > deployments with registration disabled, the invite form is replaced by a > notice — users are added by a system administrator instead. ### Change roles and remove members The row menu on each member (owner-only) offers **Change role** — between Member, Admin, and Owner — and **Remove member**, which immediately revokes their access to everything in the organisation. What each role can do is the subject of [Roles & permissions](/docs/collaboration/roles-and-permissions/). You can't change your own role from the members table. To hand the organisation over, use ownership transfer. ## The danger zone At the bottom of the organisation page: - **Transfer ownership** *(owners)* — pick a member; they become the owner and **you are demoted to admin**, not member. This is also the only way to step down as owner: owners can't leave. - **Delete this organisation** *(owners)* — removes the organisation, its members, and all of its data. You must type the organisation's name to confirm. There is no undo. - **Leave this organisation** *(admins and members)* — removes you and your access; the organisation and its data stay. If deleting or leaving strips you of your last organisation, GoPie shows a non-dismissable prompt to pick or create one — you can't work without an active organisation. --- # Collaboration & admin > Roles & permissions > The two independent role axes, the owner/admin/member matrix, the creator override, and how members see only what they're granted. URL: https://gopie.ai/docs/collaboration/roles-and-permissions/ GoPie's permission model has **two independent axes**. Understanding them separately prevents nearly every "why can't I…" moment: | Axis | Roles | What it controls | | --- | --- | --- | | **Organisation role** | Owner · Admin · Member | Everything you do with data — creating, editing, deleting and sharing projects, datasets and the rest, inside one organisation | | **System role** | System Admin · User | Exactly one thing: access to the [admin console](/docs/collaboration/admin-console/) that manages users and organisations platform-wide | The two never mix. A System Admin whose organisation role is Member can run the whole platform's user management — and still can't create a project in their own organisation. Your organisation role can also differ per organisation: owner of one, member of another. ## The organisation-role matrix | Can they… | Owner | Admin | Member | | --- | --- | --- | --- | | See every project & dataset in the org | ✓ | ✓ | Only granted ones | | Create projects & datasets | ✓ | ✓ | — | | Edit projects & datasets | ✓ | ✓ | — | | Delete projects & datasets | Any | Only their own | — | | Refresh datasets from sources | ✓ | ✓ | — | | Delete database source connections | ✓ | ✓ | — | | Grant & revoke access ([sharing](/docs/collaboration/sharing-and-access/)) | ✓ | ✓ | — | | Manage [applications & API keys](/docs/account/applications-and-api-keys/) | ✓ | ✓ | — | | Invite members, change roles, rename/delete the org | ✓ | — | — | Two rules cut across the matrix: - **The creator override.** Whatever your role, you keep full edit and delete rights on things *you* created. That's what "Only their own" means for admins — and it extends to members for the analysis artifacts below. - **Fail closed.** Any unrecognised role is treated as Member — read-only, never more. ## What members actually see Members don't get a read-only view of *everything* — they see only: - projects and datasets **they created**, and - projects and datasets **explicitly granted to them** by an owner or admin (see [Sharing & access](/docs/collaboration/sharing-and-access/)). Everything else simply doesn't appear in their lists. A brand-new member's GoPie is empty until someone grants them a project. ## Analysis artifacts follow the data Chats, saved views, dashboards and reports aren't role-gated the way projects are — they follow your **access to the underlying data**: - **Anyone who can open a dataset can [chat](/docs/chat/how-the-agent-works/) with it**, query it, and generate from it — including members, on their granted datasets. - **[Chats](/docs/chat/history-and-sharing/) are personal**: only the creator sees a chat (unless it's shared with a public link) — even owners don't browse other people's chats. - **Dashboards and reports** are visible to whoever can reach their source data. *Editing* one is for owners, admins, and its creator; everyone else gets a read-only canvas — the edit controls just don't appear. ## The system-role axis The system role lives on the *user*, not on any organisation membership. **System Admins** see an **Admin** entry in their account menu and can manage every user and organisation on the deployment — create users, ban accounts, impersonate, reassign roles. Everyone else is a **User** and never sees that console. The full tour is in [The admin console](/docs/collaboration/admin-console/). > [!TIP] When someone reports "I can't edit this," check both axes in order: > what's their **organisation role** here, and did they **create** the thing? > System Admin status is never the answer to a data-access question. --- # Collaboration & admin > Sharing & access > Granting members access to projects and datasets, what a grant actually unlocks, and how chats and dashboards differ. URL: https://gopie.ai/docs/collaboration/sharing-and-access/ Owners and admins see everything in the organisation, so **sharing in GoPie is about members**: a grant is what makes a project or dataset appear in a member's GoPie at all. This page covers each surface and what it actually unlocks. ## Share a project **Manage access** (in the project's **⋯** actions menu) opens the access panel: pick any organisation member from the list and **Add** them. The table below it shows everyone with a grant, each removable with the trash icon. ![The project access panel with the member picker and the users-with-access table](/docs/collaboration/project-access.png "Grants are member-scoped — owners and admins never need one") A project grant gives the member the project and everything inside it: its datasets to read and query, its views, its dashboards, and the ability to [chat](/docs/chat/starting-chats/) over any of it. It does **not** let them edit or delete the project itself — that stays with the [role matrix](/docs/collaboration/roles-and-permissions/). Only owners and admins can grant or revoke access. Revoking removes the project from the member's world immediately. ## Share a single dataset Sometimes a whole project is too much. The **Access** tab in a [dataset's settings](/docs/datasets/settings/) is the same picker, scoped to one dataset — the member gets that dataset (query, chat, its dashboard) without seeing its siblings. Access to a dataset is the union of three paths: you created it, you hold a direct dataset grant, or you hold a grant on its parent project. Any one suffices. > [!NOTE] Grants shape what *members* see. They're unrelated to > [applications](/docs/account/applications-and-api-keys/), which carry their > own project/dataset grants for API keys — granting a member a dataset does > nothing for any API key, and vice versa. ## What about chats, dashboards, views and reports? - **Chats** have their own mechanism — a per-chat visibility switch with a public link, covered in [History, sharing & reliability](/docs/chat/history-and-sharing/). Chats are personal: project grants never expose your conversations. - **Dashboards** have no share switch. A dashboard is visible to whoever can reach its source dataset, view or project — share the data, and the dashboard comes with it. Editing follows the [owner/admin/creator rule](/docs/collaboration/roles-and-permissions/). - **Views and reports** inherit access from their project the same way. There are no per-view or per-report grants. ## What sharing never does Sharing in GoPie stays inside the organisation, by design: - **No cross-organisation sharing** — resources can't be shared into another organisation. - **No public links for projects, datasets, dashboards or reports** — the only public link in the product is a [chat's](/docs/chat/history-and-sharing/). - **No guest accounts** — everyone you share with must be a member of your organisation first ([invite them](/docs/collaboration/organisations-and-members/)). To hand data to someone outside the organisation, [export it](/docs/query/downloads/) — or give a system integration an [API key](/docs/account/applications-and-api-keys/) scoped to exactly the datasets it needs. --- # Collaboration & admin > The admin console > Platform-wide user and organisation management for System Admins — creating users, banning, impersonation, and org membership repair. URL: https://gopie.ai/docs/collaboration/admin-console/ The admin console at **/admin** is the platform operator's view: every user and every organisation on the deployment, in one place. It's gated by the [system role](/docs/collaboration/roles-and-permissions/) — only **System Admins** see it (it appears in their account menu); everyone else is turned away regardless of their organisation role. > [!NOTE] There's no self-service path to becoming a System Admin: the first > one is designated by the deployment operator, and from then on System Admins > promote others from this console. ## Manage users **Admin → Users** lists every account with its system role, status (Active or Banned), email-verification state, and creation date — searchable by name or email. ![The admin console users page with search and per-user actions](/docs/collaboration/admin-users.png "wide: Every account on the deployment — the row menu holds Impersonate, role change and ban") Each row's menu offers: - **Impersonate** — become that user (below). - **Change role** — flip between **User** and **System Admin**. The dialog is explicit that this controls admin-console access, *not* organisation permissions. - **Ban / Unban** — a banned user can't sign in until unbanned; you can record a reason. Banned users can't be impersonated. **Create User** provisions an account directly — name, email, password, system role. The email arrives pre-verified, and (when registration is enabled) the new user gets a starter organisation of their own, exactly as if they'd signed up. This is the standard way to add users on self-hosted deployments where public registration is switched off. There is no delete-user action, and no admin-side password reset — users recover passwords through the sign-in page's own flow. ## Impersonate a user Impersonation is the support tool: you see exactly what the user sees — their organisations, projects, and permissions, not yours. While active, a banner pins to the top of every page: ![The impersonation banner across the top of the app](/docs/collaboration/impersonation-banner.png "wide: “You are impersonating. Acting as …” — Stop impersonating returns you to your own session") Click **Stop impersonating** to return to your own session. Anything you do while impersonating happens *as that user* — treat it as read-mostly debugging, not a way to act on someone's behalf. ## Manage organisations **Admin → Organisations** lists every organisation with its slug and member count. From here you can: - **Create an organisation** for any user — pick the owner from a user search, name it, done. This pairs with **Create User** for fully admin-provisioned setups. - **View members** of any organisation — and add members, change their organisation roles, or remove them, without being a member yourself. The console enforces one invariant an organisation owner can't break either: **an organisation always keeps at least one owner** — the last owner can't be demoted or removed. > [!TIP] The console is also the escape hatch for locked-out teams: when an > organisation's only owner leaves the company, a System Admin can promote a > new owner from here — no database surgery required. --- # Account & settings > Profile & security > Your name and password, two-factor authentication with backup codes, linked sign-in providers, and active sessions. URL: https://gopie.ai/docs/account/profile-and-security/ **Settings → Profile** is everything about *you* — it follows you across organisations. This page walks it top to bottom. ![The profile page's security sections: two-factor authentication, linked accounts and active sessions](/docs/account/profile-security.png "wide: The security half of the page — 2FA, linked accounts, and every signed-in session with per-device revoke") ## Name & email Edit your display name and press **Save changes**. Your **email is permanent** — it's your identity across sign-in, invitations and sessions, and can't be changed after sign-up. ## Password Changing your password asks for the current one first, then the new one twice. Forgot it entirely? Sign out and use **Forgot password** on the sign-in page — the reset link arrives by email. > [!NOTE] If you only ever signed in with Google, GitHub or Microsoft, you may > not have a password at all — that's fine; the linked provider *is* your > sign-in method. ## Two-factor authentication 2FA adds a 6-digit code from an authenticator app to your sign-in: :::steps 1. **Enable Two-Factor** — confirm your password first. 2. **Scan the QR code** with any authenticator app (or copy the setup key into it by hand). 3. **Enter the 6-digit code** the app shows, to prove the pairing worked. 4. **Save your backup codes.** GoPie shows **10 one-time backup codes** exactly once — copy them somewhere safe. Each one signs you in a single time if you ever lose the authenticator. ::: From then on, sign-in asks for a code after your password. Disabling 2FA asks for your password again. > [!IMPORTANT] Backup codes are shown only at setup and each works once. > If you run out and lose your authenticator, you're locked out — store them > in a password manager, not a screenshot. ## Linked accounts You can attach Google, GitHub, or Microsoft to your account and use any of them to sign in — same account, same email. **Link** starts the provider's consent flow; **Unlink** detaches it. Which providers appear depends on what your deployment has configured. ## Active sessions The sessions list shows every device currently signed in — browser, IP address, and when it was created and expires, with your current one marked. **Revoke** any single session to sign that device out, or **revoke all others** after using a shared machine. > [!TIP] Changed your password because something felt off? Revoke the other > sessions too — a password change alone doesn't sign out devices that are > already in. --- # Account & settings > Applications & API keys > Grouping API keys into applications, granting them exactly the projects and datasets they need, and creating keys safely. URL: https://gopie.ai/docs/account/applications-and-api-keys/ Every dataset in GoPie is [an API](/docs/query/dataset-api/), and API keys are how systems outside GoPie call it. **Settings → Applications** is where those keys live — managed by organisation owners and admins, scoped to the active organisation. ## Think in applications, not keys An **application** represents one integration — "Reporting service", "Nightly ETL", "Partner portal" — and bundles two things: - **Grants** — the projects and datasets its keys are allowed to touch. - **API keys** — one or more credentials, each individually named, expiring and revocable. That structure is the security model: a key can only reach what its application has been granted, so a leaked reporting key exposes the reporting datasets — not the organisation. ![An application's detail page with its grants and API keys tabs](/docs/account/application-detail.png "wide: One application, three tabs — the keys under API Keys can reach exactly what the Projects and Datasets tabs grant") ## Create an application **Create application** asks for a name and a description of what it's for. On the application's page, use the **Projects** and **Datasets** tabs to add grants — a project grant covers all its datasets, a dataset grant covers just the one. Grants are editable anytime and take effect immediately; removing the application deletes it *and all of its keys*. ## Create a key Under the application's **API Keys** tab: :::steps 1. **Name it** for the place it will live — "prod cron", "Ana's notebook" — and add a description. 2. **Pick an expiry.** 1 day to 1 year presets (or a custom period). Prefer short and renewable over long and forgotten. 3. **Copy the key immediately.** It's shown **exactly once**, prefixed `gp_pub_`. GoPie stores only a hash — lose it and you create a new one. ::: Keys authenticate against GoPie's data API as a bearer token: ```bash curl -H "Authorization: Bearer gp_pub_…" \ https://your-gopie-host/v1/api/tables/your_table ``` Each key's card shows its status — **Active**, **Expired**, or **Disabled** — and when it was last used, which is how you find dead keys to delete. Deleting a key revokes it instantly. > [!IMPORTANT] Treat keys like passwords: environment variables and secret > managers, never source control or client-side code. Rotating is cheap — > create the new key, switch the consumer, delete the old one. GoPie stores > keys only as hashes and never returns them again — see the > [Security overview](/docs/trust/security-overview/) for how secrets are held. ## Scope with grants, not the checkbox matrix The key dialog also offers an optional per-key **permissions** matrix (resources × actions). Leave it alone: today the selection is recorded with the key but **not yet enforced** — every key operates with its application's full grants regardless. The application's project and dataset grants are the boundary that actually holds, so scope there. > [!NOTE] API keys made here are **application-scoped**, and that's what the > [REST data API](/docs/api/overview/) expects. GoPie's > [MCP server](/docs/api/mcp-overview/) (for connecting AI clients like Claude) > uses a separate organisation-wide key type instead and refuses application > keys — see [Connect an MCP client](/docs/api/mcp-setup/). --- # Account & settings > Global search & shortcuts > The ⌘K palette that jumps to any project, dataset or dashboard, and the keyboard shortcuts worth knowing. URL: https://gopie.ai/docs/account/search-and-shortcuts/ The fastest way around GoPie isn't the sidebar — it's [[⌘ K]] ([[Ctrl K]] on Windows/Linux), from anywhere, even mid-typing in another field. ## Search everything The palette searches **projects, datasets and dashboards** across your organisation as you type, grouped by kind, each result with its breadcrumb (which project a dataset lives in, which source a dashboard belongs to). [[↑]] [[↓]] to move, [[Enter]] to jump. ![The global search palette with grouped results](/docs/account/global-search.png "One box for projects, dashboards and datasets — recent items appear before you type") Two details worth knowing: - **Recent items** — open the palette with an empty query and your last five opened items are already there. For the "back to the thing from this morning" jump, [[⌘ K]] [[Enter]] usually does it. - **Filter by kind** — [[⌘ 1]] narrows to projects, [[⌘ 2]] to datasets, [[⌘ 3]] to dashboards; press the same one again to widen back out. Search matches names — for finding data by *meaning* ("which dataset has revenue by region?"), ask [chat](/docs/chat/how-the-agent-works/) instead, which searches schemas and descriptions semantically. > [!NOTE] Chats, views and reports aren't in the palette — find chats in > [chat history](/docs/chat/history-and-sharing/), views and reports on their > project's tabs. ## Keyboard shortcuts Press [[?]] anywhere for the built-in shortcut card. The full working set: | Keys | Where | What it does | | --- | --- | --- | | [[⌘ K]] | Everywhere | Open global search | | [[?]] | Everywhere | Show the shortcuts card | | [[↑]] [[↓]] · [[Enter]] · [[Esc]] | Search palette | Move · open · close | | [[⌘ 1]] / [[⌘ 2]] / [[⌘ 3]] | Search palette | Filter to projects / datasets / dashboards | | [[Enter]] | Chat composer | Send the message ([[⇧ Enter]] for a new line) | | [[Esc]] | Chat, mid-answer | Stop the stream, keep what's generated | | [[⌘ Enter]] | [SQL playground](/docs/query/sql-playground/) | Run the query | | [[⌘ ⇧]] + arrows | Dashboard edit mode | Move the selected tile | | [[Esc]] | Inline edits & renames | Cancel without saving | On Windows and Linux, read [[⌘]] as [[Ctrl]] throughout. --- # API & MCP > API overview & authentication > The base URL, the two kinds of API key, how requests authenticate, the identifiers you'll juggle, the error envelope, and rate limits. URL: https://gopie.ai/docs/api/overview/ GoPie exposes a read-only HTTP API for querying your data, plus an [MCP server](/docs/api/mcp-overview/) for connecting AI clients. Both live on the same surface, authenticated by an **API key** sent as a bearer token. This page is the shared foundation — read it once, then jump to the endpoint you need. Everything here is already reachable from inside the app: [every dataset is an instant REST API](/docs/query/dataset-api/), and you create keys under [applications](/docs/account/applications-and-api-keys/). The reference pages exist for when you're calling GoPie from your own code, a scheduled job, or another service. ## The base URL The public API and MCP are served on a dedicated port — **8002** — separate from the app itself. In a hosted deployment that port sits behind your GoPie host, so every path in these docs is relative to: ``` https:///v1/api ``` The one exception is MCP, which is served at `/mcp` (no `/v1/api` prefix). On a local self-hosted stack the surface is `http://localhost:8002`. The API is **read-only** — there are no write endpoints for data on this port; ingestion happens in the app or through [MCP tools](/docs/api/mcp-tools/). ## Authenticate every request Send your key as a bearer token: ```bash curl -H "Authorization: Bearer gp_pub_your_key_here" \ https:///v1/api/config ``` There is no session, no cookie, no OAuth handshake — just the header, on every request. A missing or malformed header is rejected immediately: ```json /// copy: false { "code": 401, "error": "Authorization header is missing", "message": "Unauthorized" } ``` ## Two kinds of key GoPie has two distinct key types, and using the wrong one is the most common first mistake: | | Application key | Organisation-wide key | |---|---|---| | Prefix | `gp_pub_` | `gp_mcp_` | | Created in | **Settings → Applications** ([guide](/docs/account/applications-and-api-keys/)) | no UI yet — see [MCP overview](/docs/api/mcp-overview/) | | Reaches | only the projects & datasets its [application grants](/docs/account/applications-and-api-keys/) | everything in the organisation | | Use it for | the REST data API | the REST data API **and** [MCP](/docs/api/mcp-overview/) | | MCP endpoint | rejected (`403`) | accepted | Both act with **organisation-owner privileges** for role checks — the only thing that narrows an application key is its grants. So an application key scoped to one dataset can read exactly that dataset; an organisation-wide key can read every dataset in the org. Treat organisation-wide keys with extra care. > [!IMPORTANT] Application keys and the datasets/projects you [share with a > teammate](/docs/collaboration/sharing-and-access/) are unrelated systems. > Granting a person a dataset does nothing for any API key, and an application's > grants have nothing to do with member access — they're scoped independently. ## The identifiers you'll juggle Four identifiers show up across the endpoints. Getting them straight saves a lot of `404`s: | Identifier | Looks like | What it is | |---|---|---| | Project id | `550e8400-…` (UUID) | a project | | Dataset id | `550e8400-…` (UUID) | a dataset, for metadata & sharing endpoints | | **Table name** | `gp_Dh790Asdf17kd` | the dataset's **physical name** — what you query with | | Alias | `Orders` | the dataset's human-friendly display name (not a path param) | The load-bearing subtlety: **query endpoints take the `gp_…` table name, metadata endpoints take the UUID.** They're two keys for the same dataset. The alias is only for display and search. ## Discover what you can reach There is no "list projects" endpoint on this port. Instead, list datasets — the response embeds each dataset's project, so one call gives you projects, datasets, and the table names you query with: ```bash curl -H "Authorization: Bearer gp_pub_…" \ "https:///v1/api/datasets?limit=20" ``` ```json /// copy: false { "results": [ { "id": "…", "name": "gp_DzuutVJef9rSG", "alias": "Orders", "row_count": 1200, "project_id": "…", "project_name": "Sales Analytics" } ], "offset": 0, "limit": 20, "total": 3 } ``` Take `name` (the `gp_…` value) and you're ready to [query](/docs/api/querying-data/). Full listing details are on the [Datasets API](/docs/api/datasets-api/) page. ## The error envelope Errors return a small JSON object. The numeric `code` mirrors the HTTP status; `message` is a human summary and `error` carries the specific reason: ```json /// copy: false { "code": 403, "error": "This API key is not scoped to the requested dataset", "message": "Forbidden" } ``` What you'll see: | Status | When | |---|---| | `400` | malformed body, an invalid filter operator, a non-read-only or multi-statement SQL query | | `401` | missing, malformed, expired, or disabled key | | `403` | an application key reaching a dataset or project it wasn't granted | | `404` | no dataset with that table name or id in your organisation | | `500` | an unexpected server-side error | > [!NOTE] The `error` field is usually a string, but validation failures on > `POST /sql` return an array of `{field, tag, value}` objects instead. Read > `code` to branch, then inspect `error` for detail. ## Rate limits Rate limiting is **off by default**. When a deployment enables it (`GOPIE_RATELIMIT_ENABLED=true`), requests are counted per API key — the default budget is **1,000 requests per minute**. Over the limit, you get: ```json /// copy: false { "code": 429, "error": "rate_limited", "message": "too many requests; slow down and retry" } ``` with a `Retry-After: 60` header. There are no `X-RateLimit-*` headers — honour `Retry-After` and back off. ## Where to next - [**Querying data**](/docs/api/querying-data/) — `/sql`, the `/tables` REST facade, `/nl2sql`, schemas, summaries and config. Start here. - [**Datasets API**](/docs/api/datasets-api/) — list and inspect datasets, edit column descriptions, share, reindex. - [**Downloads API**](/docs/api/downloads-api/) — export query results to a file over a streamed connection. - [**Chat API**](/docs/api/chat-api/) — drive the SQL agent and consume its streamed answer. - [**Dashboards API**](/docs/api/dashboards-api/) — generate and manage dashboards programmatically. - [**MCP**](/docs/api/mcp-overview/) — connect Claude, Cursor and other AI clients to your data. --- # API & MCP > Querying data > The endpoints that read your data over HTTP — the /tables REST facade, arbitrary read-only SQL, natural-language SQL, schemas, summaries and dialect discovery. URL: https://gopie.ai/docs/api/querying-data/ These are the endpoints you'll use most: they turn any dataset into queryable data over HTTP. Everything here is read-only, capped at **1,000 rows per request**, and authenticated with a [bearer key](/docs/api/overview/#authenticate-every-request). The `{table_name}` in every path is the dataset's physical `gp_…` [name](/docs/api/overview/#the-identifiers-you-ll-juggle), which you get from [listing datasets](/docs/api/datasets-api/). ## Query rows — `POST /tables/{table_name}` The REST facade is the simplest way to read a dataset: pick columns, filter, sort and paginate with a JSON body. No SQL required. :::tabs == cURL ```bash curl -X POST "https:///v1/api/tables/gp_DzuutVJef9rSG" \ -H "Authorization: Bearer gp_pub_…" \ -H "Content-Type: application/json" \ -d '{ "columns": ["product", "revenue"], "filter": { "filter[category]": "Electronics", "filter[revenue]gte": "500" }, "sort": "-revenue", "limit": 50, "page": 1 }' ``` == JavaScript ```js const res = await fetch( "https:///v1/api/tables/gp_DzuutVJef9rSG", { method: "POST", headers: { Authorization: "Bearer gp_pub_…", "Content-Type": "application/json" }, body: JSON.stringify({ columns: ["product", "revenue"], filter: { "filter[category]": "Electronics", "filter[revenue]gte": "500" }, sort: "-revenue", limit: 50, page: 1 }) } ); const { data, count } = await res.json(); ``` == Python ```python import requests res = requests.post( "https:///v1/api/tables/gp_DzuutVJef9rSG", headers={"Authorization": "Bearer gp_pub_…"}, json={ "columns": ["product", "revenue"], "filter": {"filter[category]": "Electronics", "filter[revenue]gte": "500"}, "sort": "-revenue", "limit": 50, "page": 1, }, ) rows = res.json()["data"] ``` ::: Every field is optional — an empty body `{}` returns the first page of all columns. | Field | Meaning | |---|---| | `columns` | Array of columns to return; omit for all | | `filter` | Map of conditions, AND-ed together — see below | | `sort` | Comma-separated columns; `-` prefix for descending (`-revenue,product`) | | `limit` | Rows per page, capped at 1,000 | | `page` | 1-based page number | | `snapshot_version` | Read the dataset as of a past [version](/docs/datasets/version-history/) | ### Filter syntax This is the one non-obvious part. Filter **keys** encode the column and operator as `filter[column]op` — a plain `{"category": "Electronics"}` key is silently ignored: | Key form | Condition | |---|---| | `filter[col]` | equals | | `filter[col]neq` | not equal | | `filter[col]gt` / `filter[col]gte` | greater than / or equal | | `filter[col]lt` / `filter[col]lte` | less than / or equal | Comparison operators (`gt`, `lte`, …) work on numeric values only; text values filter by equality. There is no `like`, `in`, or `between` — reach for [SQL](#run-sql-post-sql) when you need those. An unrecognised operator is a `400`: ```json /// copy: false { "code": 400, "error": "invalid filter operator", "message": "Invalid rest params" } ``` ### Response ```json /// copy: false { "data": [ { "product": "27-inch Monitor", "revenue": 2490 } ], "count": 42, "executionTime": 23 } ``` `count` is the **total** matching rows (not the page size) — use it with `limit` and `page` to walk the whole result. If the count couldn't be computed it comes back as `-1`; the `data` is still valid. ## Run SQL — `POST /sql` When the filter grammar isn't enough, send arbitrary **read-only** SQL. Authorisation is derived by parsing the table references out of your query, so a key can only run SQL over datasets it can already reach. ```bash curl -X POST "https:///v1/api/sql" \ -H "Authorization: Bearer gp_pub_…" \ -H "Content-Type: application/json" \ -d '{"query": "SELECT category, SUM(revenue) AS total FROM gp_DzuutVJef9rSG GROUP BY 1 ORDER BY 2 DESC"}' ``` ```json /// copy: false { "columns": ["category", "total"], "count": 3, "data": [ { "category": "Electronics", "total": 148156.86 } ], "executionTime": 75 } ``` | Body field | Meaning | |---|---| | `query` | The SQL (required) | | `limit` | Rows to return, capped at 1,000 | | `offset` | Rows to skip | | `snapshot_version` / `snapshot_timestamp` | [Time travel](/docs/datasets/version-history/) to a past version | Only `SELECT`, `WITH`, `DESCRIBE` and `SUMMARIZE` are allowed — the same read-only rules as the [SQL playground](/docs/query/sql-playground/). Anything that writes, and any attempt to send more than one statement, is rejected: ```json /// copy: false { "code": 400, "error": "multiple sql statements are not allowed", "message": "Invalid SQL query" } ``` > [!NOTE] Notice the response shapes differ: `/sql` includes a `columns` array, > `/tables` does not. Both include `count` and `executionTime`. ## Generate SQL from a question — `POST /nl2sql` Turn a plain-English question into SQL **without running it** — useful for building query UIs or letting users preview what a question resolves to. ```bash curl -X POST "https:///v1/api/nl2sql" \ -H "Authorization: Bearer gp_pub_…" \ -H "Content-Type: application/json" \ -d '{"query": "total revenue by category", "table": "gp_DzuutVJef9rSG"}' ``` ```json /// copy: false { "sql": "SELECT \"category\", SUM(\"revenue\") AS \"total_revenue\"\nFROM \"gp_DzuutVJef9rSG\"\nGROUP BY \"category\"\nORDER BY \"total_revenue\" DESC" } ``` Provide exactly one of `table` (a single table name) or `tables` (an array, for multi-table questions). If the question can't be answered from the schema, you get a `200` explaining why instead of SQL: ```json /// copy: false { "sql": "", "non_sql_response": true, "message": "The data doesn't contain information about customer support tickets." } ``` This is a single-shot translation with no conversation memory. For a full analytical conversation — schema search, self-correcting SQL, grounded prose — use the [Chat API](/docs/api/chat-api/). ## Inspect structure — schemas & summaries **`GET /schemas/{table_name}`** returns the columns and their types: ```json /// copy: false { "schema": [ { "column_name": "revenue", "column_type": "DOUBLE" } ] } ``` **`GET /summary/{table_name}`** returns per-column statistics — min, max, average, quartiles, distinct and null counts, plus any [column descriptions](/docs/datasets/column-descriptions/): ```json /// copy: false { "summary": [ { "column_name": "revenue", "column_type": "DOUBLE", "min": 0, "max": 9999, "avg": 123.4, "approx_unique": 812, "null_percentage": 0, "description": "Line-item revenue in USD" } ] } ``` By default the summary is served from a fast cached copy. Add `?source=olap` to force a live recompute over the whole table — accurate but expensive on large datasets, so use it deliberately. The full statistical summary uses `SUMMARIZE`. ## Get an OpenAPI spec — `GET /openapi/{table_name}` Returns an **OpenAPI 3.0 document generated live from the dataset's actual schema** — its columns become typed response properties. Feed it to Postman, a client generator, or any tool that speaks OpenAPI to scaffold a typed client for that specific dataset. --- # API & MCP > Datasets API > List and inspect datasets, read and edit column descriptions, share a dataset with a teammate, and reindex it for chat — all over HTTP. URL: https://gopie.ai/docs/api/datasets-api/ The datasets endpoints are how an integration discovers what's available and manages dataset metadata. To read a dataset's *rows*, use the [querying endpoints](/docs/api/querying-data/) instead. Note the identifier split: these routes take the dataset **UUID** (`:datasetID`), while the query routes take the `gp_…` [table name](/docs/api/overview/#the-identifiers-you-ll-juggle). There is no create, update, or delete for datasets on this API — ingestion happens [in the app](/docs/datasets/uploading-files/) or through the [`create_dataset` MCP tool](/docs/api/mcp-tools/). ## List datasets — `GET /datasets` The entry point for discovery. Returns every dataset your key can reach, each paired with its project, so one call maps out projects, datasets and table names together. ```bash curl -H "Authorization: Bearer gp_pub_…" \ "https:///v1/api/datasets?query=orders&limit=20&page=1" ``` | Query param | Default | Meaning | |---|---|---| | `query` | — | case-insensitive substring match on name, alias or description | | `limit` | 10 | page size | | `page` | 1 | 1-based page number | ```json /// copy: false { "results": [ { "id": "550e8400-…", "name": "gp_DzuutVJef9rSG", "alias": "Orders", "description": "Line-item order facts", "row_count": 1200, "project_id": "…", "project_name": "Sales Analytics", "created_at": "2026-07-04T…", "created_by": "…" } ], "offset": 0, "limit": 20, "total": 3 } ``` A dataset that lives in more than one project appears once per project. Use `name` (the `gp_…` value) with the [query endpoints](/docs/api/querying-data/). ## Get one dataset — `GET /datasets/{datasetID}` Full metadata for a dataset by its UUID, including its column list: ```json /// copy: false { "id": "550e8400-…", "name": "gp_DzuutVJef9rSG", "alias": "Orders", "description": "Line-item order facts", "row_count": 1200, "size": 1048576, "columns": [ { "name": "revenue", "type": "DOUBLE" } ], "source": "file", "custom_prompt": "", "has_dashboard": true, "created_at": "…", "updated_at": "…", "org_id": "…" } ``` A [view](/docs/query/saved-views/) id resolves here too, returning the same shape. **`GET /datasets/{datasetID}/project`** returns just `{ "project_id": "…" }` — the hop from a dataset back to its project. ## Column descriptions Descriptions power better [chat and search](/docs/chat/how-the-agent-works/) results, and you can manage them over the API. **`GET /datasets/{datasetID}/column-descriptions`** returns the stored summary with each column's description. **`PATCH`** the same path updates them: ```bash curl -X PATCH "https:///v1/api/datasets/550e8400-…/column-descriptions" \ -H "Authorization: Bearer gp_pub_…" \ -H "Content-Type: application/json" \ -d '{"column_descriptions": {"revenue": "Line-item revenue in USD", "region": "Sales region code"}}' ``` Only keys matching real columns are applied; if none match you get a `400`. A successful patch also triggers a background reindex so chat picks up the new descriptions. Descriptions [survive refreshes](/docs/datasets/refresh/) — you don't need to re-apply them after new data lands. ## Reindex for chat — `POST /datasets/{datasetID}/reindex` Re-runs semantic schema indexing so [chat](/docs/chat/starting-chats/) works — useful if a dataset's create-time indexing failed. No file re-upload, no data change. ```bash curl -X POST "https:///v1/api/datasets/550e8400-…/reindex" \ -H "Authorization: Bearer gp_pub_…" ``` The body is optional; pass `{"project_id": "…"}` to disambiguate a dataset that lives in several projects. A successful call returns `{ "message": "Dataset re-indexed successfully", "indexed": true }`. If the AI service is unreachable you get a `502` with `"indexed": false`. ## Share a dataset The `:datasetID/users` sub-routes mirror the app's [Manage access](/docs/collaboration/sharing-and-access/) panel — granting an organisation member direct access to one dataset. Only owners and admins can grant, and the target must already be an [organisation member](/docs/collaboration/organisations-and-members/) (a `403` otherwise). | Method + path | Body | Effect | |---|---|---| | `GET /datasets/{datasetID}/users` | — | list members with a grant (paginated) | | `PUT /datasets/{datasetID}/users` | `{"user_id": "…"}` | grant access | | `DELETE /datasets/{datasetID}/users` | `{"user_id": "…"}` | revoke access | > [!NOTE] These grants control what **people** see in the app. They are > separate from [application grants](/docs/account/applications-and-api-keys/), > which control what **API keys** reach — see [Sharing & > access](/docs/collaboration/sharing-and-access/). --- # API & MCP > Downloads API > Export a query result to a file — a streamed create endpoint that runs your SQL, writes it to storage, and hands back a time-limited download link. URL: https://gopie.ai/docs/api/downloads-api/ The downloads endpoints export a query result to a file (CSV, JSON, or Parquet) and give you a link to fetch it. Unlike the [1,000-row cap](/docs/api/querying-data/) on live queries, a download can be **any size** — it streams the full result straight to object storage. This is the API behind the app's [Downloads & exports](/docs/query/downloads/). ## Create a download — `POST /downloads` The one surprise: **create is a streamed connection, not a fire-and-forget id.** You `POST` the request and hold the connection open; the server reports progress as **Server-Sent Events** and delivers the finished download link in the final event. ```bash curl -N -X POST "https:///v1/api/downloads" \ -H "Authorization: Bearer gp_pub_…" \ -H "Content-Type: application/json" \ -d '{ "dataset_id": "gp_DzuutVJef9rSG", "sql": "SELECT * FROM gp_DzuutVJef9rSG WHERE region = '\''West'\''", "format": "csv" }' ``` | Body field | Required | Meaning | |---|---|---| | `dataset_id` | yes | the dataset UUID **or** its `gp_…` table name (either works) | | `sql` | yes | the read-only SQL to export | | `format` | yes | `csv`, `json`, or `parquet` | | `snapshot_version` | no | export a past [version](/docs/datasets/version-history/) | The stream is a sequence of `data:` events. Progress arrives as `status_update`, a `heartbeat` fires every 15 seconds so the connection stays alive, and the terminal `complete` event carries the **presigned download URL** in its `message`: ``` /// copy: false data: {"event":"status_update","message":"Processing query..."} data: {"event":"status_update","message":"Executing data export to S3..."} data: {"event":"complete","message":"https:///…/export.csv?signature=…"} ``` Fetch that URL to get the file. **It expires 24 hours after creation.** If a failure occurs mid-export it arrives as an `error` event rather than an HTTP status code — the connection had already upgraded to a stream. > [!NOTE] Identical requests are deduplicated. If you ask for the same > `(dataset, SQL, format, version)` you exported recently and the data hasn't > changed, the `complete` event returns the existing link immediately instead of > re-running the query. ## List your downloads — `GET /downloads` Returns your recent downloads (a bare JSON array, newest paged by `limit`/`offset`): ```json /// copy: false [ { "id": "550e8400-…", "sql": "SELECT * FROM gp_DzuutVJef9rSG WHERE region = 'West'", "dataset_id": "…", "status": "completed", "format": "csv", "pre_signed_url": "https:///…/export.csv?signature=…", "expires_at": "2026-07-05T…", "completed_at": "2026-07-04T…" } ] ``` `status` is one of `pending`, `processing`, `completed`, or `failed`. The `pre_signed_url` and `expires_at` are populated once a download completes. ## Get or delete one — `GET` / `DELETE /downloads/{id}` **`GET /downloads/{id}`** returns a single download record by its UUID — handy for re-fetching a still-valid link. **`DELETE /downloads/{id}`** removes the record and returns `204 No Content`. --- # API & MCP > Chat API & streaming > Drive the SQL agent over HTTP — create chats, stream a grounded answer as it's generated, resume an interrupted stream, and read a chat's history. URL: https://gopie.ai/docs/api/chat-api/ The Chat API drives the same [SQL agent](/docs/chat/how-the-agent-works/) the app uses: you send a question with some dataset or project context, and the agent searches the schema, writes and self-corrects SQL, runs it, and streams back a grounded answer. This page focuses on the **streaming** endpoint — the flagship — and the supporting chat-management routes. ## Choose context with headers Every turn needs to know which data it can query. Unlike the rest of the API, chat context is passed in **headers**, not the body: | Header | Meaning | |---|---| | `x-project-ids` | comma-separated project ids — the agent can reach every dataset inside | | `x-dataset-ids` | comma-separated dataset ids (or `gp_…` table names) | | `x-view-ids` | comma-separated [view](/docs/query/saved-views/) ids | | `x-chat-id` | an existing chat id, to continue the conversation | At least one of project / dataset / view ids is required. Omit `x-chat-id` to start a fresh conversation — the first stream event returns the new chat's id. ## Stream an answer — `POST /chat/stream` This is the endpoint to use. It streams the answer as it's produced, exactly like the app's chat panel. ```bash curl -N -X POST "https:///v1/api/chat/stream" \ -H "Authorization: Bearer gp_pub_…" \ -H "Content-Type: application/json" \ -H "x-dataset-ids: gp_DzuutVJef9rSG" \ -d '{"messages": [{"role": "user", "content": "What are the top products by revenue?"}]}' ``` | Body field | Meaning | |---|---| | `messages` | the conversation so far — an array of `{role, content}` (required) | | `snapshot_version` | query datasets as of a past [version](/docs/datasets/version-history/) | ### Understand the stream The response is a stream of **newline-delimited JSON** — one JSON object per line (the [Vercel AI SDK](https://ai-sdk.dev/) UI-message format, signalled by the `x-vercel-ai-ui-message-stream: v1` response header). Each line has a `type`; you reassemble the answer by handling the types you care about and ignoring the rest. The ones that matter: | `type` | Carries | |---|---| | `data-chat-created` | the new chat's `chatId` — the first line of a fresh conversation | | `text-start` / `text-delta` / `text-end` | the answer prose, streamed token by token in `delta` | | `data-sql-query` | the SQL the agent ran (emitted `pending` then `success`, with any `snapshotVersion`) | | `data-datasets-used` | which datasets the answer drew on | | `data-citations` | source documents backing the answer, when any | | `tool-input-available` / `tool-output-available` | the agent's tool calls, as they happen | | `data-heartbeat` | a keep-alive; ignore it | | `finish` | terminal — carries token counts, model, and latency in `messageMetadata` | A typical turn: ```json /// copy: false {"type":"data-chat-created","data":{"chatId":"f47ac10b-…"}} {"type":"data-sql-query","id":"sql-1","data":{"query":"SELECT product, SUM(revenue) AS total FROM gp_DzuutVJef9rSG GROUP BY 1 ORDER BY 2 DESC LIMIT 10","status":"pending"}} {"type":"text-start","id":"text-1"} {"type":"text-delta","id":"text-1","delta":"The top product is "} {"type":"text-delta","id":"text-1","delta":"the 27-inch Monitor at $2,490."} {"type":"text-end","id":"text-1"} {"type":"data-sql-query","id":"sql-1","data":{"query":"SELECT product, SUM(revenue) …","status":"success"}} {"type":"data-datasets-used","data":{"datasets":["Orders"]}} {"type":"finish","messageMetadata":{"totalTokens":1420,"inputTokens":380,"outputTokens":1040,"model":"claude-sonnet","latencyMs":2340}} ``` A turn runs to a wall-clock limit (three minutes by default). If it's exceeded, or if the LLM is unavailable, the stream ends with an error notification instead of a `finish`. **Closing the connection cancels the turn** server-side — there's no orphaned work. ## Resume an interrupted stream — `GET /chat/stream/resume` If a stream drops mid-turn (a flaky network, a reconnect), reattach with the chat id: ```bash curl -N "https:///v1/api/chat/stream/resume?chat_id=f47ac10b-…" \ -H "Authorization: Bearer gp_pub_…" ``` You'll receive the buffered part of the turn so far, then the live tail. Resume is **best-effort and in-memory** — if the turn already finished, the buffer expired, or you're talking to a different server replica, you get `204 No Content` (nothing to resume). Persistence happens on the original `POST`; a resume never re-saves. ## A note on `POST /chat/completions` There's also an OpenAI-compatible `POST /chat/completions` that streams `chat.completion.chunk` deltas ending in `data: [DONE]`. It exists for tools that speak the OpenAI wire format. For GoPie-native clients, prefer `/chat/stream` above — it carries the richer typed parts (SQL, citations, datasets used) that `/completions` flattens away. One caveat: `snapshot_version` is honoured only on `/stream`, not `/completions`. ## Manage chats | Method + path | Effect | |---|---| | `GET /chat` | list your chats (`limit`, `page`) | | `POST /chat/create` | create an empty chat with context attached, without running a turn | | `GET /chat/{chatID}` | chat details | | `GET /chat/{chatID}/messages` | the full message history | | `PUT /chat/{chatID}/title` | rename it | | `PUT /chat/{chatID}/visibility` | set `private`, `public`, or `organization` | | `DELETE /chat/{chatID}` | delete it (`204`) | Reading a chat's details or messages requires that you created it or that its visibility is `public` — otherwise `403`. The [visibility model and its caveats](/docs/chat/history-and-sharing/) are covered in the chat docs. --- # API & MCP > Dashboards API > Generate dashboards for a dataset, view or project, manage them with optimistic concurrency, and browse their version history — over HTTP. URL: https://gopie.ai/docs/api/dashboards-api/ The Dashboards API programmatically drives what the app does on the [dashboard canvas](/docs/dashboards/introduction/): generating a dashboard from your data, saving edits safely under concurrent access, and browsing [version history](/docs/dashboards/version-history/). Most people build dashboards in the app — reach for this API when you're automating (nightly regeneration, provisioning a dashboard per new dataset, an external editor). ## Generate a dashboard There are three generate targets, matching the three [kinds of dashboard](/docs/dashboards/introduction/): | Endpoint | For | |---|---| | `POST /datasets/{datasetId}/dashboard/generate` | a single dataset | | `POST /projects/{projectId}/views/{viewId}/dashboard/generate` | a [saved view](/docs/query/saved-views/) | | `POST /projects/{projectID}/dashboards/generate` | a whole project (multi-source) | The dataset and view forms take no body — control them with `?use_ai=true` (default) and `?force=false` (regenerate an existing one). They return the full dashboard object with a `spec`, plus an `ETag` header (see [concurrency](#save-edits-safely) below). The **project** form takes a JSON body naming the sources: ```bash curl -X POST "https:///v1/api/projects/…/dashboards/generate" \ -H "Authorization: Bearer gp_pub_…" \ -H "Content-Type: application/json" \ -d '{"name": "Revenue overview", "intent": "track revenue and top products", "dataset_ids": ["…", "…"], "use_ai": true}' ``` At most 12 datasets and 6 reused views (18 sources total). Validation failures (`name_required`, `no_sources`, `too_many_datasets`) come back as `400`. ### Stream generation progress Each generate endpoint has a `/generate/stream` twin that reports progress as **Server-Sent Events** while the dashboard is built — the same five phases the app shows: ``` /// copy: false data: {"type":"progress","message":"Analyzing your data","data":{"phase":"analyzing"}} data: {"type":"progress","message":"Selecting key metrics","data":{"phase":"metrics"}} data: {"type":"progress","message":"Validating queries","data":{"phase":"validating"}} data: {"type":"progress","message":"Designing the layout","data":{"phase":"layout"}} data: {"type":"progress","message":"Finalizing your dashboard","data":{"phase":"finalizing"}} data: {"type":"complete","message":"Dashboard generated","data":{"id":"…","spec":{…}}} ``` Phases are `analyzing` → `metrics` → `validating` → `layout` → `finalizing`, a `heartbeat` fires during long steps, and a terminal `complete` carries the finished dashboard. If the AI service is unavailable or `use_ai=false`, GoPie builds a deterministic dashboard in-process and the stream still completes — it never stalls. ## Save edits safely Dashboards use **optimistic concurrency** so two editors don't clobber each other. Every `GET` and `PUT` returns an `ETag` header — a fingerprint of the dashboard's current state: ``` ETag: "1720107000123456789" ``` To update, send that value back in `If-Match`: ```bash curl -X PUT "https:///v1/api/dashboards/…" \ -H "Authorization: Bearer gp_pub_…" \ -H 'If-Match: "1720107000123456789"' \ -H "Content-Type: application/json" \ -d '{"spec": {…}, "display_name": "Q3 revenue"}' ``` If someone else changed the dashboard since you read it, the tags won't match and you get a `412` — reload and retry: ```json /// copy: false { "code": 412, "error": "dashboard_modified", "message": "The dashboard was modified by another session. Please reload." } ``` `If-Match` is optional; omit it to write unconditionally (at the cost of the safety check). ## List, read & delete | Method + path | Effect | |---|---| | `GET /dashboards` | list dashboards (`query`, `limit`, `page`) | | `GET /dashboards/{dashboardId}` | one dashboard + `can_edit` + `ETag` | | `PUT /dashboards/{dashboardId}` | update `spec` / `display_name` (honours `If-Match`) | | `DELETE /dashboards/{dashboardId}` | delete it (`204`) | | `GET /projects/{projectId}/dashboards` | dashboards in a project | ## Refine with a prompt — `POST /dashboards/{dashboardId}/refine` Apply a natural-language edit ("add a discount KPI", "make the revenue chart a line") to a dashboard or a single tile — the API behind [dashboard chat](/docs/dashboards/dashboard-chat/): ```json /// copy: false { "user_prompt": "add an average-discount KPI", "scope": "dashboard" } ``` For a tile-scoped edit, set `"scope": "tile"` and a `"tile_path"` like `rows.0.items.2`. The refined dashboard comes back in the response. ## Browse version history Every generate, edit and refine records a [version](/docs/dashboards/version-history/): | Method + path | Effect | |---|---| | `GET /dashboards/{dashboardId}/versions` | list versions (spec omitted, for a light payload) | | `GET /dashboards/{dashboardId}/versions/{versionId}` | one version, with its full `spec` | | `POST /dashboards/{dashboardId}/versions/{versionId}/restore` | restore it (honours `If-Match`) | | `PATCH /dashboards/{dashboardId}/versions/{versionId}` | name it (`{"message": "…"}`) or clear the name | | `POST /dashboards/{dashboardId}/versions/{versionId}/pin` · `/unpin` | pin against [cleanup](/docs/dashboards/version-history/) | ## Plan a project dashboard (wizard) The app's [project dashboard wizard](/docs/dashboards/ai-generation/) is three endpoints you can call directly to build a guided flow: `POST /projects/{projectID}/dashboards/recommend-intent` (suggest what the dashboard should cover), `recommend-datasets` (rank which datasets to include), and `plan-views` (propose the joins as validated view definitions). Each returns structured suggestions you can present, edit, then feed into the project `generate` call above. When the AI service is down, `recommend-datasets` falls back to ranking by row count so the flow still works. --- # API & MCP > MCP server overview > What GoPie's MCP server is, the organisation-wide key it requires, and how to get one — so an AI client like Claude or Cursor can query your data directly. URL: https://gopie.ai/docs/api/mcp-overview/ GoPie ships a [Model Context Protocol](https://modelcontextprotocol.io) (MCP) server, so AI clients — Claude, Cursor, and anything else that speaks MCP — can browse your projects, inspect schemas, and run read-only SQL against your data through a standard interface. Instead of copying results into a chat, the assistant queries GoPie live. ## What you get Connect a client and it gains **9 [tools](/docs/api/mcp-tools/)** (list projects, inspect a dataset, run SQL, create a dataset from a file, and more) and **3 resources** (browsable projects, datasets and schemas). The server also injects live guidance — your organisation context and SQL tips — so the assistant writes queries that actually run the first time. ## The endpoint MCP is served over **streamable HTTP** at: ``` https:///mcp ``` On a local self-hosted stack that's `http://localhost:8002/mcp`. It's stateless — no session handshake to manage — and every request authenticates with a bearer [API key](/docs/api/overview/#two-kinds-of-key), exactly like the REST API. > [!IMPORTANT] MCP requires an **organisation-wide** key (prefix `gp_mcp_`). > Application-scoped `gp_pub_` keys are refused with a `403` — an assistant that > can write arbitrary SQL shouldn't be boxed into one application's grants, so > MCP only accepts keys that carry full organisation access. Getting one is the > next section. ## Get an organisation-wide key This is the one rough edge today. Application keys have a [tidy UI](/docs/account/applications-and-api-keys/); organisation-wide MCP keys **don't have a create button yet**. Until one ships, an organisation **owner or admin** can mint a key directly against the auth service while signed in to GoPie. > [!DETAILS] Mint an organisation-wide key (advanced) > While signed in to your GoPie web app as an owner or admin, call the auth > service's key-creation endpoint with your session cookie. The `configId: "mcp"` > is what produces an organisation-wide `gp_mcp_` key (rather than an > application-scoped one): > > ```bash > curl -X POST "https:///api/auth/api-key/create" \ > -H "Content-Type: application/json" \ > -H "Origin: https://" \ > -b "better-auth.session_token=" \ > -d '{"configId": "mcp", "name": "My MCP key", "organizationId": ""}' > ``` > > The response returns the plaintext key **once**, starting with `gp_mcp_` — > copy it immediately; only a hash is stored. Keep it secret: an organisation-wide > key can read every dataset in the org. Once you have the key, head to [MCP setup](/docs/api/mcp-setup/) to wire it into your client. ## A local alternative: the stdio binary There's also a standalone `gopie-mcp` binary that speaks MCP over stdio for local, single-user setups. It doesn't use API keys at all — it connects **directly to your datastores** with credentials in its environment. That makes it convenient for a self-hoster on their own machine and unsuitable for anything shared. It's covered alongside the HTTP setup on the [next page](/docs/api/mcp-setup/#local-stdio-binary). ## What MCP can and can't do - **Read-only SQL** — the same `SELECT` / `WITH` / `DESCRIBE` / `SUMMARIZE` allowlist as the [REST API](/docs/api/querying-data/#run-sql-post-sql), capped at 1,024 rows and a 30-second timeout per query. - **Create datasets** — the `create_dataset` and `get_upload_url` tools can ingest a file from a URL, a genuine write path (see the [tools reference](/docs/api/mcp-tools/#create-a-dataset)). - **Not** chat, dashboards, or reports — those richer flows are the [REST API's](/docs/api/chat-api/) job. MCP is the discovery-and-query surface. --- # API & MCP > Connect an MCP client > Wire GoPie's MCP server into Claude Code, Claude Desktop, Cursor, or any streamable-HTTP client — plus the local stdio binary. URL: https://gopie.ai/docs/api/mcp-setup/ Once you have an [organisation-wide `gp_mcp_` key](/docs/api/mcp-overview/#get-an-organisation-wide-key), connecting a client is a few lines of config. Every client points at the same endpoint — `https:///mcp` — and sends your key as a bearer token. Pick your client below. > [!NOTE] GoPie's MCP server uses **streamable HTTP**. Modern clients (Claude, > Cursor) speak it natively. Older SSE-only MCP clients that expect a > long-lived server-to-client event stream won't connect — use a streamable-HTTP > client, or the [stdio binary](#local-stdio-binary) locally. ## Claude Code Add the server from the terminal: ```bash claude mcp add --transport http gopie https:///mcp \ --header "Authorization: Bearer gp_mcp_your_org_wide_key" ``` Then ask Claude about your data — "what datasets are in my Sales project?" — and it'll use the GoPie tools. Manage it later with `claude mcp list` and `claude mcp remove gopie`. ## Cursor Add GoPie to your `mcp.json` (`.cursor/mcp.json` in a project, or the global one): ```json { "mcpServers": { "gopie": { "url": "https:///mcp", "headers": { "Authorization": "Bearer gp_mcp_your_org_wide_key" } } } } ``` Cursor picks it up on reload; the GoPie tools become available to the agent. ## Claude Desktop Claude Desktop connects to a hosted GoPie two ways: :::tabs == Remote connector Add GoPie as a custom connector pointing at `https:///mcp` with an `Authorization: Bearer gp_mcp_…` header. This needs a publicly reachable HTTPS endpoint (the streamable-HTTP connector, not the SSE one). == Local stdio binary If you're running GoPie locally, use the [stdio binary](#local-stdio-binary) below — no API key, but it needs datastore credentials in its config. ::: ## Any streamable-HTTP client For a client not listed here, the contract is: - **URL:** `https:///mcp` - **Method:** `POST` JSON-RPC (no `Mcp-Session-Id` — the server is stateless) - **Headers:** `Authorization: Bearer gp_mcp_…`, `Content-Type: application/json`, `Accept: application/json, text/event-stream` A missing key returns `401`; an [application-scoped key](/docs/api/overview/#two-kinds-of-key) returns `403` — MCP needs an organisation-wide key. ## Local stdio binary For a single developer running GoPie on their own machine, the standalone `gopie-mcp` binary is often simplest — it speaks MCP over stdio and needs no API key. Build it from the server source: ```bash go build -o gopie-mcp ./cmd/mcp ``` Then point your client's config at it. A Claude Desktop entry: ```json /// copy: false { "mcpServers": { "gopie": { "command": "/path/to/gopie-mcp", "args": ["stdio"], "env": { "GOPIE_MCP_ORG_ID": "", "GOPIE_POSTGRES_HOST": "localhost", "GOPIE_POSTGRES_PORT": "5432", "GOPIE_POSTGRES_DB": "gopie", "GOPIE_POSTGRES_USER": "postgres", "GOPIE_POSTGRES_PASSWORD": "postgres", "GOPIE_S3_ENDPOINT": "http://localhost:9000", "GOPIE_S3_ACCESS_KEY": "minioadmin", "GOPIE_S3_SECRET_KEY": "minioadmin" } } } } ``` > [!WARNING] The stdio binary connects **directly to your Postgres and > object storage** — there is no API-key auth layer, and organisation scoping > comes only from `GOPIE_MCP_ORG_ID`. It's for local, single-user use. For > anything shared or remote, use the HTTP endpoint with an organisation-wide key. The binary accepts flags to tune limits — `--query-timeout` (default 30s), `--max-rows` (default 10, capped at 1,024), and `--org-id`. The full [tool set](/docs/api/mcp-tools/) is identical to the HTTP server. ## Verify the connection Whichever client you used, ask it to list your projects. If it comes back with your real project names, the tools are wired up. If it reports an auth error, re-check that your key starts with `gp_mcp_` (not `gp_pub_`) — that's the most common cause. --- # API & MCP > MCP tools & resources > The 9 tools and 3 resources a connected MCP client can use — what each does, its limits, and the upload-then-create recipe for ingesting a file. URL: https://gopie.ai/docs/api/mcp-tools/ Once a client is [connected](/docs/api/mcp-setup/), it has these tools and resources. You rarely call them by hand — the AI client chooses them to answer your questions — but knowing what's available (and what's off-limits) helps you ask for the right things. ## Tools ### Discovery | Tool | Does | Key inputs | |---|---|---| | `list_projects` | List the organisation's projects | `search`, `limit` (≤500), `page` | | `get_project` | Project details by id | `project_id` | | `list_datasets` | Datasets in a project | `project_id`, `limit`, `page` | | `get_dataset_info` | The primary inspection tool — full metadata, columns, and optionally statistics and sample rows | `dataset_id`, `include_statistics`, `include_sample_rows` (≤100) | ### Query | Tool | Does | Key inputs | |---|---|---| | `execute_sql` | Run read-only SQL and return rows | `query`, `limit` (default 10, ≤1,024), `snapshot_version` | | `get_query_count` | Count matching rows without fetching them | `query`, `snapshot_version` | `execute_sql` enforces the same allowlist as the [REST API](/docs/api/querying-data/#run-sql-post-sql) — only `SELECT`, `WITH`, `DESCRIBE` and `SUMMARIZE`. It caps results at **1,024 rows**, truncates very large payloads at ~50,000 characters, and times out at **30 seconds**. `snapshot_version` reads a dataset as of a past [version](/docs/datasets/version-history/). ### Management | Tool | Does | Key inputs | |---|---|---| | `create_project` | Create a project | `name`, `description` | | `get_upload_url` | A presigned URL to upload a file | `filename`, `project_id` | | `create_dataset` | Ingest a file from a URL into a queryable dataset | `project_id`, `name`, `description`, `url`, `column_descriptions` | ## Create a dataset `create_dataset` is a genuine write path — it's how an assistant turns a file into a queryable dataset. It pairs with `get_upload_url` in a three-step recipe: :::steps 1. **Get an upload URL.** `get_upload_url(filename, project_id)` returns a presigned `upload_url` (valid one hour) and the `s3_url` the file will live at. 2. **Upload the file.** `PUT` the raw bytes to `upload_url`. 3. **Create the dataset.** `create_dataset(project_id, name, description, url = s3_url, column_descriptions = {…})` ingests the file into a new dataset, records it, and — on success — deletes the uploaded source file. Provide a description for every column; malformed column names are auto-fixed to snake_case. ::: For a file already reachable at a public URL, you can skip the first two steps and pass that URL to `create_dataset` directly. ## Resources Resources are browsable data an MCP client can read without a tool call: | Resource | Returns | |---|---| | `gopie://projects` | every project in the organisation | | `gopie://project/{project_id}/datasets` | the datasets in one project | | `gopie://dataset/{dataset_id}/schema` | a dataset's columns and types | All three return JSON. Clients that support resource browsing surface these as an explorable tree; others reach the same data through the discovery tools above. ## Built-in guidance Every session, the server sends the client an instructions block tailored to your deployment: your organisation id and **SQL tips** — conveniences like `SELECT * EXCLUDE (…)` and `SUMMARIZE`. It also spells out the read-only constraint and the recommended workflow (list projects → list datasets → inspect a dataset → query). That's why a connected assistant tends to write SQL that runs correctly the first time. --- # Trust & security > AI data privacy > What the AI model actually sees, where your data stays, how to bring your own LLM provider, and the things GoPie deliberately does not do with your data. URL: https://gopie.ai/docs/trust/ai-data-privacy/ GoPie answers questions by querying your data, not by memorising it. To do that, the AI agent needs *some* context — but far less than people assume. This page is a precise, honest account of what leaves your deployment, what never does, and the choices you control. ## What the model sees When you ask a question, GoPie does **not** hand your dataset to a language model. It sends the model just enough to write and check a query — then runs that query against your own data and shows the model only a small sample of the result. | Sent to the model | How much | | --- | --- | | Dataset and column names and types | the schema only | | Column descriptions | your text, as written | | Sample column values | up to ~5 representative values per column | | Query results | a small sample — large results are trimmed to roughly 20 rows before the model sees them | | Context documents | only the top matching passages the agent retrieves, never whole files | | Custom prompts | your instructions, as written | The model **never** receives your full tables, your complete query results, or your raw uploaded files. It reasons over the schema and small samples; the full result of every query stays inside GoPie and is what *you* see in the results panel. > [!NOTE] The row count is always honest. When a result is trimmed for the model, GoPie still reports the true total row count alongside the sample — so the agent knows it's looking at 20 rows out of 4 million, and its answer reflects that. ## Where your data lives Your data stays in the datastores your GoPie deployment runs on: - **Metadata and chats** — the platform's PostgreSQL database. - **Uploaded files** — your object storage. Any S3-compatible store works: AWS S3, Google Cloud Storage, Cloudflare R2, MinIO, and the like. - **Schema embeddings** — your Qdrant instance, used for semantic schema search. - **Analytical data & queries** — your datasets are stored as columnar files in that same S3-compatible object storage, and every query runs inside your deployment. The only content that leaves your deployment is what's sent to the **AI provider** you configure (and to the embedding provider, if you choose a hosted one). Both are your choice — see below. ## Choose your AI provider GoPie is model-agnostic. It talks to language models through a pluggable gateway, so an operator can point it at a hosted API, a private gateway, or a model running on their own hardware: - **Hosted gateways** — OpenRouter, Portkey Cloud, or Cloudflare AI Gateway. - **Self-hosted gateways** — a LiteLLM or Portkey proxy you run yourself. - **Any OpenAI-compatible endpoint** — including a local model — through the "custom" provider. Embeddings are configured separately and **can run entirely on-device** — an in-process model with no external call at all — or through a hosted embedding API. Because you bring your own provider and keys, *you* decide which company (if any) processes your prompts, and under which terms. > [!NOTE] GoPie doesn't ship with a built-in model. The operator supplies the model identifiers and provider credentials in the deployment's configuration, so there's no default third party in the loop until one is chosen. ## Training and retention This is the part people most need to get right, so here it is plainly: **Whether an AI provider retains or trains on the prompts GoPie sends is governed entirely by the provider and plan you choose, and their terms.** GoPie does not add a "do-not-train" instruction to its requests — it can't make a provider honour a policy the provider doesn't offer. If you need a strict data posture: - Pick a provider and plan with a **zero-retention or no-training** guarantee — many vendors offer this on enterprise tiers. - Or run a **self-hosted or local model**, so prompts never leave your infrastructure at all. Paired with local embeddings, the AI path stays fully inside your deployment. > [!WARNING] GoPie does not detect, mask, or redact sensitive values before sending context to the model. Sample column values and result-row samples are sent as-is — so if a column holds sensitive data, those values can appear in what the model sees. Choose your provider accordingly, and be mindful of what lives in the columns you upload. ## What GoPie does and doesn't do To summarise, in one place: - **It does** run every query against your own engine, send the model only schemas and small samples, let you bring your own provider and keys, and let embeddings run locally. - **It does not** send your full datasets or complete result sets to any model, add a no-training flag to provider requests, or perform any PII detection or redaction. For the mechanics of how the agent turns a question into a checked, grounded answer, see [How the SQL agent works](/docs/chat/how-the-agent-works/). For the platform's broader security posture — authentication, isolation, and secrets — see the [Security overview](/docs/trust/security-overview/). --- # Trust & security > Security overview > How GoPie authenticates people and programs, isolates organisations, protects secrets at rest and in transit — and what it deliberately leaves to your own infrastructure. URL: https://gopie.ai/docs/trust/security-overview/ GoPie protects your data across three services (the web app, the API, and the AI agent) and the datastores behind them. Some protections are built into the platform; others depend on how you deploy and operate it. This page describes both, honestly — treat it as an account of the platform's mechanisms, not a compliance attestation. ## How people and programs authenticate GoPie exposes three network surfaces, each with its own access model: | Surface | Who uses it | How it's authenticated | | --- | --- | --- | | Web app | People | A signed session cookie, validated on every request | | Public API and MCP | Programs and AI clients | A Bearer [API key](/docs/account/applications-and-api-keys/) | | Internal service API | GoPie's own services | Trusted network only — no authentication | Sign-in is email-and-password with **required email verification**, and accounts can turn on [two-factor authentication](/docs/account/profile-and-security/) — a code from any authenticator app, backed by ten one-time recovery codes. Passwords are stored only as **salted hashes** (scrypt), never in plaintext. > [!WARNING] The internal service API performs no authentication — it trusts request headers and exists solely for service-to-service calls inside the cluster. It must never be exposed to the public internet; its security depends entirely on network isolation. ## How access is controlled Authorisation runs on [two independent axes](/docs/collaboration/roles-and-permissions/): - A **system role** (admin or user) gates the system-wide admin console. - An **organisation role** (owner, admin, or member) gates who can create and manage resources in an organisation. On top of roles, individual projects and datasets can be **shared** with specific members, and the person who created a resource always keeps access to it. [API keys](/docs/account/applications-and-api-keys/) act with organisation-owner authority within their organisation; an *application-scoped* key is additionally narrowed to only the projects and datasets you grant it. > [!NOTE] MCP refuses application-scoped keys by design. Because an MCP request carries its target datasets and SQL inside the request body — where per-request grant checks can't reach — the [MCP endpoint](/docs/api/mcp-overview/) accepts only organisation-wide keys and rejects application-scoped ones outright, failing closed. ## How organisations are isolated Every read of a project or dataset is **scoped to the organisation making the request** — the organisation filter is part of the query itself, not applied afterwards, so one organisation cannot address another's data. GoPie can also go further with **schema-per-organisation** isolation: each organisation gets its own database schema, its own scoped database role with a distinct credential, and its own storage prefix — so the isolation is enforced at the storage layer, not just in application code. ## How secrets are protected at rest - **Source credentials** — the connection strings and cloud-storage keys for your [database and cloud sources](/docs/datasets/sources/) — are **encrypted with AES-256-GCM** before they're stored, and are **never returned by the API**. You can update a credential, but you can't read it back. - **Passwords** are salted-hashed (scrypt). - **API keys** are stored as hashes, never in plaintext — the full key is shown to you exactly once, at creation. > [!NOTE] Credential encryption uses a key from your deployment's configuration — application-level envelope encryption, not an external key-management service. Everything else GoPie stores (dataset contents, chats, files, embeddings) relies on the at-rest encryption of the datastore you run it on. Enable disk/bucket encryption on your PostgreSQL, object storage, and Qdrant to cover that data. ## How data is protected in transit - **TLS** is terminated at your ingress or load balancer — run GoPie behind HTTPS in production. - **Download links** are time-limited, signed URLs: [exports](/docs/query/downloads/) expire after 24 hours and document links after one hour, so a shared link doesn't grant indefinite access. - Keep the internal API and the database on a **private network** — GoPie assumes a trusted network between its own services. ## Rate limiting Request rate limiting is **optional and off by default**. When enabled, requests are limited per user and per API key; exceeding the limit returns `429 Too Many Requests` with a `Retry-After` header telling the client when to retry. Turn it on for any internet-facing deployment — see the [API overview](/docs/api/overview/). ## Administrative audit log GoPie keeps an **append-only audit log of access-control events** — membership changes, API-key creation and revocation, organisation lifecycle, and user bans or impersonation. It is an administrative trail for who-changed-access, not a record of every data query. ## What GoPie leaves to you Being clear about the boundary matters as much as the features: > [!IMPORTANT] GoPie ships with no compliance certifications (such as SOC 2 or ISO 27001) and no external key-management service, and it does not terminate TLS itself. Encryption in transit, the at-rest encryption of the underlying datastores, network isolation of the internal service API, and rate limiting on internet-facing endpoints are all operational responsibilities of whoever deploys GoPie. The platform gives you the mechanisms; a secure deployment is the two working together. For what the AI agent does and doesn't send to a model, see [AI data privacy](/docs/trust/ai-data-privacy/).