> ## Documentation Index > Fetch the complete documentation index at: https://gigi-f9937525.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # LLM And Cognitive Layer Plan > Planned provider registry, credential ownership, memory, and reasoning layers. # LLM And Cognitive Layer Plan This page is planning material. It describes how Gigi should grow from the current foundation into a Discord-native agent runtime without weakening the existing permission, audit, memory, and plugin safety model. ## Design Principles * Keep Discord as an adapter. The cognitive layer should not depend on `discordgo`. * Keep `internal/llm` as the low-level provider port for text and embedding calls. * Add agent orchestration as a separate package, such as `internal/agent`, above provider calls, memory, plugins, permissions, and audit. * Keep deterministic plugin prefix matching before LLM planning. * Treat `@Gigi` guild mentions as conversational agent requests, not as a broad deterministic natural-language command parser. * Keep exact slash commands as the deterministic source for explicit behavior and syntax. * Treat LLM output and plugin descriptions as untrusted hints, never executable authority. * Build final tool plans from registered tool schemas, stored manifests, capability checks, confirmation policy, and audit rules. * Keep deterministic tools for exact work such as memory counts, message search, plugin dispatch, permission checks, and usage summaries. * Use the LLM for interpretation, planning, clarification, and answer composition, not as the source of truth for counts, permissions, or action execution. * Never store raw prompts, completions, provider responses, planned commands, or secrets in audit metadata. ## Agent Runtime Core The new core model is: ```text theme={null} Discord intake -> deterministic trigger handlers -> agent planner -> context broker -> tool executor -> answer composer -> confirmation executor for risky actions -> audit ``` This avoids writing one custom parser for every natural-language phrasing. Gigi still keeps small deterministic tools, but arbitrary user language maps onto those tools through a typed planning step. Guild mentions should enter that planner path after only narrow deterministic gates that are already part of the documented transition: enabled external app prefix matching and exact current-channel memory-count pre-handling. Those gates should stay small compatibility paths, not grow into a second command language. When behavior needs stable syntax, strict arguments, or admin-grade guarantees, it belongs in a slash command and should be documented through the command guide. Example: ```text theme={null} @Gigi how often do we talk about postgres here? ``` Planner proposal: ```json theme={null} { "intent": "answer_with_context", "tool_calls": [ {"name": "memory.count", "args": {"text": "postgres", "scope": "this-channel"}}, {"name": "memory.search", "args": {"query": "postgres", "scope": "this-channel", "limit": 5}} ] } ``` Gigi then validates the proposal, checks `memory.read.guild`, enforces channel visibility, runs SQL-backed memory tools, packs a small context bundle, asks the answer model to explain the result, and audits the request without storing raw prompt or raw snippets by default. Core packages should move toward these boundaries: | Package | Responsibility | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------- | | `internal/agent` | Surface-independent request orchestration and agent loop | | `internal/agent/planner` | Typed JSON planning against registered tools and available context | | `internal/contextbroker` | Chooses and retrieves scoped context such as recent channel messages, memory search results, and plugin catalog summaries | | `internal/tools` | Registry for native tools and enabled external app tools | | `internal/agent/policy` | Capability checks, channel visibility checks, confirmation rules, redaction, and audit classification | | `internal/agent/answer` | Final answer composition from prompt, tool results, citations, and policy notes | The important invariant: the LLM proposes; Gigi validates and executes deterministic code. ## Provider Registry Direction Gigi should support a provider registry with first-class entries for: * OpenAI * Anthropic * Gemini * future custom or compatible providers Provider support should be purpose-aware instead of hard-coded to one chat model. Planned purposes: | Purpose | Use | | ----------- | ------------------------------------------------- | | `chat` | Normal DM or mention conversation | | `reasoning` | Higher-effort planning and tool proposal | | `embedding` | Message history and retrieval vectors | | `routing` | Lightweight intent and plugin candidate selection | Model IDs should remain flexible strings validated by the provider service. Provider model catalogs change faster than Discord command definitions, so slash commands should not require a fixed model enum for every provider model. ## Credential Ownership Model The storage and resolver should support multiple credential owners from the start: | Owner type | Meaning | v0 behavior | | ---------- | -------------------------------------------------------- | ------------------------------------ | | `guild` | A Discord server owns the credential and model profiles. | Active UX | | `user` | A Discord user owns a personal BYOK credential. | Schema/resolver-ready only | | `tenant` | The deployment/operator owns a fallback credential. | Optional single-tenant fallback only | V0 should be multi-owner architecture with guild-only product behavior. That means the schema, usage events, and resolver should understand `guild`, `user`, and `tenant`, but the first shipped commands should only let authorized guild admins configure guild credentials. ## Revised V0 V0 should establish the Discord agent runtime core, not only isolated chat or one-off semantic routes. Current and planned V0 LLM/provider foundation includes: * `internal/llm/provider` registry, credential store, secret sealing, model profile selection, provider testing, and credential resolution. * Provider-backed text clients for OpenAI, Anthropic, and Gemini using resolved model credentials. * Runtime generation wrapper that resolves active models and records token usage. * Guild-scoped provider credentials configured by users with `llm.provider.write`. * Guild-scoped model profiles selected by users with `llm.provider.select`. * Provider test commands gated by `llm.provider.test`. * Encrypted write-only secrets using a swappable `SecretSealer`. * Usage events that record actor, guild, provider, model, purpose, billing owner type, billing owner ID, token counts, status, and classified error. * `/llm usage guild` aggregate reporting with no raw prompt, completion, provider response, or secret storage. * Guild-mention cognitive fallback chat after deterministic external app matching fails. * Metadata-only conversation turn storage with no raw prompt or completion columns. * Semantic routing that treats LLM output as a proposal, validates it against native tools and enabled manifests, applies capability checks, and returns dry-run or allowed read-only/public-safe results only. * `/ask` command routing through the agent runtime with explicit `context:none` or dynamic current-channel context fetching through `context:channel`. * Initial `internal/agent` runtime contracts, planner handler, native tool registry, memory count/search/recent tools, plugin plan/list tools, permission check tool, LLM usage summary tool, and chat fallback adapter. * Initial `internal/contextbroker` context fetcher/packer seam for current-channel retained memory and bounded snippet packing, now extended into a typed pack builder with stable source IDs, citation labels, stale/invalidation markers, pinned snippets, omitted-unchanged records, restore handles, and next-state fingerprints. * Initial durable agent run storage with SQL `agent_runs`, `agent_run_steps`, and `agent_run_confirmations` tables, runner lifecycle status, termination reasons, budget fields, cancellation checks, pending confirmation records, usage counters, and sanitized step observations. * Initial `internal/evalharness` package for repeatable agent golden cases that capture responses, audit events, durable steps, run lifecycle records, pending confirmations, and observability redaction failures without a database or network. * Guild policy storage that defaults personal keys to `off`. * No per-user BYOK for guild mentions, shared memory, embeddings, or semantic routing. * No silent fallback between billing owners. Agent-runtime V0 should add: * A single `AgentHandler` fallback behind deterministic external app matching. Started with agent-backed chat fallback and `/ask`. * A typed planner contract for intents, context requests, tool calls, clarification questions, and confirmation requirements. Guild mention planning now starts with the general LLM planner over registered tool schemas, with the older LLM-backed memory planner as a compatibility fallback. * A general `AgentRunner` with `MaxSteps`, `MaxToolCalls`, LLM-call budget, and redacted trace. Started with a compatibility runner that preserves current mention and `/ask` behavior while moving orchestration out of one-shot planning handlers, and now persists run lifecycle and step observations when a run store is available. Discord responses carry run IDs for confirmation-required runs, and `/agent` can trace, cancel, inspect, confirm, or reject records. * A split policy/executor/answerer core. Started with policy-owned routing mode and capability checks, executor-owned tool lookup/execution/masked errors, deterministic write-tool confirmation guards, LLM answer synthesis over tool results, bounded short-term per-user/channel follow-up context, safe in-process `/agent trace last` diagnostics, and audit trace metadata for `run_id`, `step_index`, tool kind, and capability. * A native tool registry with at least `memory.count`, `memory.search`, `memory.recent`, `plugins.enabled`, `plugins.plan`, `permissions.check`, `llm.usage.guild`, and `llm.chat`. Started with the read-only tools plus chat fallback; `llm.chat` remains an answerer/fallback concern instead of an LLM-calling tool. * A context broker that decides whether a run needs no context, current-channel context, permitted guild memory, or enabled plugin catalog context. Started with bounded context pack building, restore-safe status metadata, and a first permitted recent-channel memory provider; cross-channel guild context, plugin catalog context, and richer retrieval-provider wiring remain incremental. * A token packer that reserves budget for tool results, fetched context snippets, citations/source IDs, restore handles, and output, then trims context deterministically. Current implementation uses deterministic character budgets, pinned snippet priority, citation labels, and omitted-unchanged restore handles until token accounting lands. * Read-only tool execution for memory, plugin planning, permission checks, and usage-style queries after capability and channel visibility checks. Started with current-channel memory tools, cited memory evidence payloads with message IDs/source IDs/restore handles/retention proof, `plugins.enabled`, `plugins.plan`, `permissions.check`, and `llm.usage.guild`. * Public-safe external app dispatch only through existing manifest-grounded `send_message` rules; restricted actions remain dry-run or confirmation-required. * Audits, durable steps, and safe in-process traces for `agent.context`, `agent.plan`, `agent.tool`, `agent.answer`, and `agent.confirmation` using request IDs, run IDs, ordered step indexes, status, scopes, tool names, tool kind, capability, token counts, termination reasons, and redacted metadata only. Debug trace output is actor/channel-scoped and safe-metadata-only; it does not expose raw prompts, memory snippets, provider responses, planned commands, or secrets. * Compatibility migration from current wrappers: keep exact fast paths while agent-native tools replace `SemanticMemoryHandler`, `AssistantFallbackHandler`, and later semantic plugin handling. Dynamic context fetching must not widen storage or audit exposure. Trace metadata may record `run_id`, `step_index`, context scope, fetch status, source type, result count, truncation flag, tool name, tool kind, capability, and message/source IDs. It must not store raw user prompts, raw fetched snippets, provider payloads, planned commands, embeddings, or secrets. Deferred beyond V0: * Rich semantic retrieval over vectors beyond current count/search scaffolding. * `/llm key` personal BYOK behavior. * Tenant/operator fallback credentials. * Automatic resume after confirmed pending actions. * Restricted dispatch and multi-step external app actions. The important product rule: an admin can personally pay for a provider key, but if that key powers shared server behavior, Gigi treats it as a guild credential governed by guild policy and audit. ## Implementation Slice Order Each slice should be implemented as a small, reviewable project. For any non-trivial slice, prefer spawning caveman-style subagents so investigation, coding, review, testing, and documentation stay focused while keeping token use low. A typical slice should use: * `planning` subagent: inspect current files, identify exact write scope, list risks, and return a compact implementation outline. * `builder` subagent: own a narrow file set and make the bounded code change. * `reviewer` subagent: review the diff for bugs, regressions, permission gaps, privacy leaks, and missing tests. * `tester` subagent: run focused tests, add or propose missing harness coverage, and report exact failures. * `docs` subagent: update user/admin docs after behavior is implemented and verified. Recommended slice order: | Order | Slice | Goal | Caveman subagent split | | ----- | ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | Safety rails | Harden memory visibility, plugin import trust, audit redaction, provider-secret handling, and public/private response defaults before adding more agent power. | `planning`: map safety matrix. `builder`: implement one guard at a time. `reviewer`: fail-closed review. `tester`: denied-path tests. `docs`: admin-facing safety notes. | | 2 | Context broker | Promote `internal/contextbroker` from a snippet packer into a Grape-inspired Discord context pack builder with source IDs, citations, stale markers, pinned snippets, omitted unchanged context, and restore handles. | `planning`: define pack schema. `builder`: implement pack/provider interfaces. `reviewer`: check privacy and truncation. `tester`: golden pack fixtures. `docs`: explain context behavior. | | 3 | Durable agent runs | Move from one-shot plan execution toward persisted run state, step records, observations, termination reasons, budget accounting, cancellation, and resumable confirmations. | `planning`: schema and state-machine outline. `builder`: run store plus runner changes. `reviewer`: idempotency and resume review. `tester`: loop/confirmation tests. `docs`: operator/debug docs. | | 4 | Memory retrieval | Add exact, FTS/BM25, and later vector retrieval over opted-in memory with provenance, retention proof, delete handling, backfill checkpoints, and safe citation output. | `planning`: retrieval contracts. `builder`: store/retriever worker slices. `reviewer`: channel visibility and retention review. `tester`: ingest-to-answer harness. `docs`: memory commands and policy. | | 5 | Plugin manifest v1 | Evolve plugin manifests from plugin-level prefix matching into action-level typed tool contracts with arg schemas, safety class, per-action capability, config validation, dispatch adapter, and explicit public-dispatch consent. | `planning`: manifest compatibility plan. `builder`: parser/store/matcher slices. `reviewer`: trust and permission review. `tester`: manifest corpus tests. `docs`: plugin author/admin guide. | | 6 | Discord UX | Add component routing, buttons, modals, task status, cancel/retry, permission previews, plugin install review, and confirmation flows. | `planning`: interaction flow map. `builder`: component router and commands. `reviewer`: auth/ephemeral review. `tester`: fake Discord interaction tests. `docs`: command guide refresh. | | 7 | Eval and observability harness | Add repeatable golden evals and integration harnesses before broad semantic behavior expands. Started with `internal/evalharness` for agent runner response/status/trace/confirmation/redaction cases. | `planning`: eval matrix. `builder`: fake Discord/provider/audit harness. `reviewer`: signal quality review. `tester`: CI/race/integration jobs. `docs`: test and ops guide. | The rule for every slice: safety, tests, and docs travel with code. Do not ship restricted plugin dispatch, guild-wide memory retrieval, arbitrary plugin code, or multi-agent Discord personas until durable runs, confirmation records, channel visibility, retention purge, and audit/query views are in place. ## Revised V1 V1 can enable controlled personal BYOK: * `/llm key add`, `test`, `prefer`, and `remove` for personal credentials. * Personal keys allowed in DMs that do not use guild memory or guild actions. * Guild policy for personal keys: `off`, `dm-only`, or `guild-allowed`. * Explicit per-request billing choice where appropriate: `server` or `me`. * Clear disclosure before a personal key can process guild context. * No personal-key usage for shared guild embeddings by default. * No automatic fallback from a failed personal key to a guild key, or from a failed guild key to a personal key. * BYOK pays for reasoning but does not grant capabilities. The existing capability engine still decides whether actions can run. ## Guild Memory Retrieval Plan Guild memory should be explicit, bounded, and asynchronous. New guilds start with memory off. A guild admin must enable memory per channel before Gigi indexes messages from that channel. No DM, private thread, or group DM content should enter guild memory. The target behavior has two retrieval paths: * deterministic aggregate queries for questions such as `@Gigi how many times did @sam mention "postgres"?` * semantic recall for questions that need similar-message retrieval and citations Count questions should not depend on an LLM answer. Gigi should resolve the mentioned user when present, normalize the requested text, apply guild/channel/retention/deletion filters, and answer from SQL counts. The agent planner may map flexible language onto `memory.count`, but the count itself comes from deterministic SQL. Semantic recall can use embeddings after deterministic filters and capability checks. Efficient ingestion should never block Discord event handling: ```text theme={null} Discord MESSAGE_CREATE -> policy gate -> lightweight message upsert -> enqueue ingest job -> return Memory worker -> redact -> chunk -> batch embed -> write segments and vectors -> checkpoint ``` Backfill should use `/memory sync` to queue a resumable job. The worker fetches Discord history page by page with rate limits, saves checkpoints, batch-inserts messages, and processes embeddings later. If the worker lags or fails, the bot should keep serving HTTP and Discord interactions. Planned data model: ```sql theme={null} guild_memory_policies ( guild_id text primary key, default_retention_days integer not null, raw_storage_mode text not null, embeddings_enabled boolean not null, updated_by_user_id text, updated_at timestamptz not null default now() ); guild_memory_channels ( guild_id text not null, channel_id text not null, mode text not null, retention_days integer, updated_by_user_id text, updated_at timestamptz not null default now(), primary key (guild_id, channel_id) ); guild_memory_messages ( message_id text primary key, guild_id text not null, channel_id text not null, author_user_id text not null, normalized_text text, content_ciphertext bytea, content_hash text, created_at timestamptz not null, edited_at timestamptz, deleted_at timestamptz, retention_until timestamptz not null ); guild_memory_segments ( id text primary key, message_id text not null, guild_id text not null, channel_id text not null, token_count integer not null, search_text text, created_at timestamptz not null default now() ); guild_memory_embeddings ( segment_id text primary key, provider_id text not null, model_id text not null, embedding vector, embedded_at timestamptz not null default now() ); ``` The exact schema can change during implementation, but these boundaries should stay: * message ledger for exact count and audit-safe source metadata * segment table for search units * vector table for embeddings * policy table for guild and channel opt-in * queue/checkpoint tables for live ingest, backfill, retries, edits, deletes, and retention purge Memory policy: * default off * per-channel opt-in * ignore bot messages by default * no shared-memory embeddings from personal BYOK * raw content encrypted if retained * retention required before ingest * edit events supersede old segments * delete events tombstone messages and purge segments/vectors * audit events store metadata only, never raw message text, raw query text, snippets, embeddings, prompts, completions, provider responses, or secrets Planned capabilities: | Capability | Use | | --------------------- | -------------------------------------------------------- | | `memory.read.guild` | Count, search, and ask over permitted guild memory | | `memory.manage.guild` | Enable channels, set retention, queue sync, purge memory | | `memory.export.guild` | Future bulk export path, if allowed by policy | Channel visibility should be enforced in addition to Gigi capabilities. A user should not retrieve memory from channels they cannot normally see. ## Later Extensions Later slices can add: * tenant/operator hosted credentials for single-tenant deployments * per-channel and per-role model policies * per-user, per-guild, and per-provider budgets * provider failover policies * provider-specific model discovery * persistent confirmation records * Discord buttons and modals for confirmation and credential entry * memory export and user deletion workflows * webhook dispatch and multi-step plugin pipelines ## Planned Data Shape The exact SQL can evolve during implementation, but the first schema should preserve these ownership concepts: ```sql theme={null} llm_credentials ( id text primary key, owner_type text not null, guild_id text, user_id text, provider_id text not null, label text not null, credential_ciphertext bytea not null, credential_nonce bytea not null, credential_key_id text not null, credential_fingerprint text not null, status text not null, last_test_status text, last_tested_at timestamptz, last_error_code text, created_by_user_id text, updated_by_user_id text, revoked_at timestamptz, created_at timestamptz not null default now(), updated_at timestamptz not null default now(), unique (id, provider_id) ); Active credential labels should be unique per owner. A label should identify one active credential in a guild, user, or tenant scope, not one credential per provider, so command UX can safely address credentials by label. llm_model_profiles ( id text primary key, owner_type text not null, guild_id text, user_id text, purpose text not null, credential_id text not null, provider_id text not null, model_id text not null, params jsonb not null default '{}'::jsonb, enabled boolean not null default true, selected_by_user_id text, updated_at timestamptz not null default now(), foreign key (credential_id, provider_id) references llm_credentials(id, provider_id) ); llm_usage_events ( id text primary key, request_id text not null, guild_id text, channel_id text, actor_user_id text not null, billing_owner_type text not null, billing_owner_id text not null, provider_id text not null, model_id text not null, purpose text not null, input_tokens integer, output_tokens integer, status text not null, error_class text, created_at timestamptz not null default now() ); ``` ## Command Shape Live guild/admin commands: ```text theme={null} /llm provider list /llm provider add provider: label: