> ## Documentation Index
> Fetch the complete documentation index at: https://botpress-ak-docs-20-document-updating-variables-from-outsid.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge

Knowledge bases provide context to your agent's AI models. They allow your agent to answer questions based on documents, websites, or other data sources.

## Creating a knowledge base

Create a knowledge source in `src/knowledge/`:

```typescript theme={null}
import { Knowledge, DataSource } from "@botpress/runtime";

export default new Knowledge({
  name: "documentation",
  description: "Product documentation",
  sources: [
    // Add knowledge sources
  ],
});
```

## Data sources

You can add knowledge from several different data sources:

### Website source

There are multiple ways to index websites:

#### From a sitemap

Index a website using a sitemap XML file:

```typescript theme={null}
import { Knowledge, DataSource } from "@botpress/runtime";

const WebsiteSource = DataSource.Website.fromSitemap(
  "https://example.com/sitemap.xml",
  {
    filter: ({ url }) => !url.includes("/admin"),
    maxPages: 1000,
    maxDepth: 10,
  }
);

export default new Knowledge({
  name: "website-docs",
  sources: [WebsiteSource],
});
```

#### From a base URL

Crawl a website starting from a base URL (requires the Browser integration):

```typescript theme={null}
import { Knowledge, DataSource } from "@botpress/runtime";

const WebsiteSource = DataSource.Website.fromWebsite(
  "https://example.com",
  {
    filter: ({ url }) => !url.includes("/admin"),
    maxPages: 500,
    maxDepth: 5,
  }
);

export default new Knowledge({
  name: "website-docs",
  sources: [WebsiteSource],
});
```

#### From `llms.txt`

Index pages referenced in an `llms.txt` file:

```typescript theme={null}
import { Knowledge, DataSource } from "@botpress/runtime";

const WebsiteSource = DataSource.Website.fromLlmsTxt(
  "https://example.com/llms.txt"
);

export default new Knowledge({
  name: "website-docs",
  sources: [WebsiteSource],
});
```

#### From specific URLs

Index a specific list of URLs:

```typescript theme={null}
import { Knowledge, DataSource } from "@botpress/runtime";

const WebsiteSource = DataSource.Website.fromUrls([
  "https://example.com/page1",
  "https://example.com/page2",
  "https://example.com/page3",
]);

export default new Knowledge({
  name: "website-docs",
  sources: [WebsiteSource],
});
```

#### Website source options

| Option     | Type                 | Description                                                                           |
| ---------- | -------------------- | ------------------------------------------------------------------------------------- |
| `id`       | `string`             | Optional unique identifier for the source                                             |
| `filter`   | `(ctx) => boolean`   | Filter function receiving `{ url, lastmod?, changefreq?, priority? }`                 |
| `fetch`    | `string \| function` | Fetch strategy: `'node:fetch'` (default), `'integration:browser'`, or custom function |
| `maxPages` | `number`             | Maximum pages to index (1-50000, default: 50000)                                      |
| `maxDepth` | `number`             | Maximum sitemap depth (1-20, default: 20)                                             |

### File source

Index files from a local directory (development only):

```typescript theme={null}
const FileSource = DataSource.Directory.fromPath("./docs", {
  filter: (filePath) => filePath.endsWith(".md") || filePath.endsWith(".txt"),
});

export default new Knowledge({
  name: "local-docs",
  sources: [FileSource],
});
```

<Note>
  Directory sources only work during development (`adk dev`). For production, use website sources or upload files to Botpress Cloud.
</Note>

#### Directory source options

| Option   | Type                            | Description                               |
| -------- | ------------------------------- | ----------------------------------------- |
| `id`     | `string`                        | Optional unique identifier for the source |
| `filter` | `(filePath: string) => boolean` | Filter function to include/exclude files  |

### Table source

Index data from a table:

```typescript theme={null}
import { Knowledge, DataSource } from "@botpress/runtime";
import OrderTable from "../tables/OrderTable";

const TableSource = DataSource.Table.fromTable(OrderTable, {
  id: "orders",
  transform: ({ row }) => {
    return `Order #${row.id}\nCustomer: ${row.userId}\nTotal: $${row.total}\nStatus: ${row.status}`;
  },
});

export default new Knowledge({
  name: "orders-kb",
  sources: [TableSource],
});
```

## Using knowledge in conversations

Provide knowledge bases to your conversation handlers:

```typescript highlight={1, 8} theme={null}
import { WebsiteKB } from "../knowledge/docs";

export default new Conversation({
  channel: "*",
  handler: async ({ execute }) => {
    await execute({
      instructions: "You are a helpful assistant.",
      knowledge: [WebsiteKB],
    });
  },
});
```

## Refreshing knowledge

Refresh knowledge bases to update their content:

```typescript highlight={6, 8} theme={null}
import { Workflow } from "@botpress/runtime";
import WebsiteKB from "../knowledge/docs"

export default new Workflow({
  name: "refresh-knowledge",
  schedule: "0 0 * * *", // Daily
  handler: async () => {
    await WebsiteKB.refresh();
  },
});
```

### Force refresh

Force re-indexing of all content even if unchanged:

```typescript theme={null}
await WebsiteKB.refresh({ force: true });
```

### Refresh a specific source

Refresh a single data source within a knowledge base:

```typescript theme={null}
await WebsiteKB.refreshSource("my-source-id", { force: true });
```

## Reference

### Knowledge props

<Expandable title="Knowledge props">
  <ResponseField name="name" type="string" required>
    Unique name for the knowledge base.
  </ResponseField>

  <ResponseField name="description" type="string">
    Optional description of what the knowledge base contains.
  </ResponseField>

  <ResponseField name="sources" type="DataSource[]" required>
    Array of data sources to index. Can include `Website`, `Directory`, or `Table` sources.
  </ResponseField>
</Expandable>