feat: add comprehensive API key manager (/gsd keys) (#1089)

* feat: add comprehensive API key manager (/gsd keys)

Add /gsd keys command with 6 subcommands for full API key lifecycle
management: list, add, remove, test, rotate, and doctor.

- list/status: Dashboard grouped by category (LLM, search, tool, remote)
  with masked key previews, OAuth expiry, env var source detection
- add: Interactive provider picker with OAuth vs API key choice,
  prefix validation, and env var activation
- remove: Multi-key support with individual or bulk removal
- test: Lightweight API validation per provider with latency reporting
  and error classification (401/429/5xx/timeout)
- rotate: Remove-and-replace flow with optional pre-save validation
- doctor: Health checks for expired OAuth, empty keys, duplicates,
  env var conflicts, file permissions, missing LLM provider

Includes unified provider registry (22 providers), tab completions,
and redirect from /gsd setup keys. 44 unit tests.

* fix: convert key-manager tests from vitest to node:test for CI typecheck

Extension tests use node:test + node:assert/strict (not vitest) since
tsconfig.extensions.json includes test files and vitest types are not
available in the CI typecheck step.
This commit is contained in:
Jeremy McSpadden 2026-03-17 23:32:26 -05:00 committed by GitHub
parent 5ef52b8a59
commit 76a834cdf6
4 changed files with 1738 additions and 2 deletions

302
.plans/api-key-manager.md Normal file
View file

@ -0,0 +1,302 @@
# API Key Manager — Implementation Plan
## Problem Statement
GSD has solid API key infrastructure (AuthStorage, OAuth flows, rate-limit backoff, multi-key rotation) but lacks a user-facing CLI for day-to-day key management. Users currently must either:
- Run the full onboarding wizard to add keys
- Manually edit `~/.gsd/agent/auth.json`
- Use the limited `/gsd setup keys` flow (only covers 5 tool keys, no LLM keys)
There's no way to list, test, remove, or inspect key health from the CLI.
## Scope
Build `/gsd keys` — a comprehensive API key management command with subcommands:
```
/gsd keys → Show key status dashboard
/gsd keys list → List all configured keys with status
/gsd keys add <provider> → Add/replace a key for a provider
/gsd keys remove <provider> → Remove a key for a provider
/gsd keys test [provider] → Validate key(s) by making a lightweight API call
/gsd keys rotate <provider> → Remove old key and prompt for new one
/gsd keys doctor → Health check all keys (expired OAuth, empty keys, backoff state)
```
## Architecture
### New Files
| File | Purpose |
|------|---------|
| `src/resources/extensions/gsd/key-manager.ts` | Core key manager logic (list, add, remove, test, rotate, doctor) |
| `src/resources/extensions/gsd/tests/key-manager.test.ts` | Unit tests |
### Modified Files
| File | Change |
|------|--------|
| `src/resources/extensions/gsd/commands.ts` | Add `/gsd keys` subcommand routing + completions |
### No changes to core packages
All work stays in the GSD extension layer. We use `AuthStorage` as-is — no modifications to `pi-coding-agent` or `pi-ai`.
---
## Phase 1: Key Status Dashboard (`/gsd keys` and `/gsd keys list`)
### What it shows
```
GSD API Key Manager
LLM Providers
✓ anthropic — OAuth (expires in 23h 41m)
✓ openai — API key (sk-...a4Bf)
✗ google — not configured
✗ groq — not configured (env: GROQ_API_KEY)
Tool Keys
✓ tavily — API key (tvly-...x92k)
✓ context7 — API key (c7-...m3np)
✗ brave — not configured (env: BRAVE_API_KEY)
✗ jina — not configured (env: JINA_API_KEY)
Remote Integrations
✓ discord_bot — API key (configured)
✗ slack_bot — not configured
✗ telegram_bot — not configured
Search Providers
✓ tavily — API key (tvly-...x92k)
Source: ~/.gsd/agent/auth.json
3 keys configured | 2 from env vars | 1 OAuth token
```
### Implementation
- Read all known provider IDs from `env-api-keys.ts` envMap + LLM_PROVIDER_IDS + TOOL_KEYS
- Check `authStorage.has()`, `authStorage.get()`, and `getEnvApiKey()` for each
- For API keys: show masked preview (first 4 + last 4 chars)
- For OAuth: show expiration time remaining
- For env vars: indicate source is environment
- Group by category (LLM, Tools, Remote, Search)
- Show backoff status if any keys are currently backed off
---
## Phase 2: Add Key (`/gsd keys add <provider>`)
### Flow
1. If `<provider>` not specified → show interactive provider picker (grouped by category)
2. If provider has OAuth available → offer "Browser login" or "API key" choice
3. For API key: masked password input → prefix validation → save to auth.json
4. For OAuth: delegate to existing `authStorage.login()` flow
5. Confirm save with masked preview
### Provider Registry
Build a unified provider registry that merges:
- `LLM_PROVIDER_IDS` from onboarding.ts
- `TOOL_KEYS` from commands.ts
- `envMap` from env-api-keys.ts
- Remote bot tokens (discord_bot, slack_bot, telegram_bot)
Each entry has:
```typescript
interface ProviderInfo {
id: string
label: string
category: 'llm' | 'tool' | 'search' | 'remote'
envVar?: string // Known env var name
prefixes?: string[] // Expected key prefixes for validation
hasOAuth?: boolean // Whether OAuth login is available
dashboardUrl?: string // Where to get the key
}
```
---
## Phase 3: Remove Key (`/gsd keys remove <provider>`)
### Flow
1. If `<provider>` not specified → show picker of configured keys only
2. Confirm removal (show what will be removed)
3. Call `authStorage.remove(provider)`
4. Clear corresponding env var from process.env
5. Notify success
### Multi-key handling
If a provider has multiple keys (round-robin), show:
```
anthropic has 3 API keys configured:
[1] sk-ant-...a4Bf
[2] sk-ant-...x92k
[3] sk-ant-...m3np
Remove: all | specific index?
```
---
## Phase 4: Test Key (`/gsd keys test [provider]`)
### Validation Strategy
For each provider, make the lightest possible API call:
| Provider | Test Method |
|----------|------------|
| anthropic | `POST /v1/messages` with `max_tokens: 1` and a trivial prompt |
| openai | `GET /v1/models` (list models endpoint) |
| google | `GET /v1beta/models` |
| groq | `GET /openai/v1/models` |
| brave | `GET /res/v1/web/search?q=test&count=1` |
| tavily | `POST /search` with minimal params |
| context7 | Lightweight search query |
| jina | `GET /` health check |
| discord_bot | `GET /api/v10/users/@me` |
| slack_bot | `POST auth.test` |
| telegram_bot | `GET /getMe` |
### Output
```
Testing API keys...
✓ anthropic — valid (claude-sonnet-4-20250514 available) 142ms
✓ openai — valid (gpt-4o available) 89ms
✗ groq — invalid (401 Unauthorized)
✓ tavily — valid 203ms
⚠ brave — rate limited (retry in 28s)
— jina — skipped (not configured)
3 valid | 1 invalid | 1 rate-limited | 1 skipped
```
### Error Classification
- 401/403 → "invalid key"
- 429 → "rate limited (retry in Xs)"
- 5xx → "server error"
- timeout → "unreachable"
- success → "valid" + model info if available
---
## Phase 5: Rotate Key (`/gsd keys rotate <provider>`)
### Flow
1. Show current key (masked)
2. Prompt for new key
3. Validate prefix format
4. Optionally test the new key before saving (`/gsd keys test` logic)
5. Replace in auth.json
6. Update process.env
7. Confirm
---
## Phase 6: Key Doctor (`/gsd keys doctor`)
### Checks
1. **Expired OAuth tokens** — OAuth credentials past their expiration
2. **Empty keys** — Providers with empty string keys (from skipped onboarding)
3. **Duplicate keys** — Same key stored under multiple providers
4. **Missing required keys** — LLM provider not configured at all
5. **Backoff state** — Keys currently in rate-limit backoff
6. **Env var conflicts** — Key in auth.json differs from env var
7. **File permissions** — auth.json not 0o600
### Output
```
API Key Health Check
⚠ anthropic: OAuth token expires in 4m — will auto-refresh
✗ groq: empty key stored (from skipped setup) — run /gsd keys add groq
⚠ openai: env var OPENAI_API_KEY differs from auth.json — env takes priority
✗ auth.json permissions: 0644 (should be 0600) — fixing...
✓ No duplicate keys found
✓ No keys in backoff state
2 warnings | 1 issue fixed | 1 action needed
```
---
## Integration Points
### Command Registration (commands.ts)
Add to the `/gsd` subcommand router:
```typescript
if (trimmed === "keys" || trimmed.startsWith("keys ")) {
const keysArgs = trimmed.replace(/^keys\s*/, "").trim();
await handleKeys(keysArgs, ctx);
return;
}
```
Add tab completions for `keys` subcommands:
```typescript
if (parts[0] === "keys" && parts.length <= 2) {
// list, add, remove, test, rotate, doctor
}
```
### Redirect `/gsd setup keys`
Update `handleSetup` to route `keys` to the new handler instead of `handleConfig`.
### Help Text
Add to the help output in the appropriate category.
---
## Testing Strategy
### Unit Tests (`key-manager.test.ts`)
1. **Provider registry** — All known providers have correct metadata
2. **Key masking** — Masks correctly for various key lengths
3. **Status formatting** — Dashboard output matches expected format
4. **Add key** — Stores via AuthStorage.inMemory()
5. **Remove key** — Removes correctly, handles multi-key providers
6. **Doctor checks** — Detects expired OAuth, empty keys, permission issues
7. **Test key result formatting** — Correct status symbols and messages
### Integration-level (manual)
- Full add → test → rotate → remove flow
- OAuth provider login flow
- Multi-key round-robin after adding multiple keys
---
## Implementation Order
1. **Phase 1**`key-manager.ts` with provider registry + list/status dashboard
2. **Phase 2** — Add key (interactive picker + validation)
3. **Phase 3** — Remove key (with multi-key handling)
4. **Phase 4** — Test key (lightweight API calls per provider)
5. **Phase 5** — Rotate key (remove + add in one flow)
6. **Phase 6** — Key doctor (health checks)
7. **Wire up** — Command registration, completions, help text, redirect setup keys
8. **Tests** — Unit tests for all phases
---
## Out of Scope
- Encrypted-at-rest storage (would require a master password / keyring integration — separate effort)
- Per-project key scoping (would require project-level auth.json — separate effort)
- Key usage tracking/audit log (would require persistent metrics — separate effort)
- Changes to `pi-coding-agent` or `pi-ai` packages

View file

@ -77,7 +77,7 @@ export function projectRoot(): string {
export function registerGSDCommand(pi: ExtensionAPI): void {
pi.registerCommand("gsd", {
description: "GSD — Get Shit Done: /gsd help|next|auto|stop|pause|status|visualize|queue|quick|capture|triage|dispatch|history|undo|skip|export|cleanup|mode|prefs|config|hooks|run-hook|skill-health|doctor|forensics|migrate|remote|steer|knowledge|new-milestone|parallel|update",
description: "GSD — Get Shit Done: /gsd help|next|auto|stop|pause|status|visualize|queue|quick|capture|triage|dispatch|history|undo|skip|export|cleanup|mode|prefs|config|keys|hooks|run-hook|skill-health|doctor|forensics|migrate|remote|steer|knowledge|new-milestone|parallel|update",
getArgumentCompletions: (prefix: string) => {
const subcommands = [
{ cmd: "help", desc: "Categorized command reference with descriptions" },
@ -101,6 +101,7 @@ export function registerGSDCommand(pi: ExtensionAPI): void {
{ cmd: "mode", desc: "Switch workflow mode (solo/team)" },
{ cmd: "prefs", desc: "Manage preferences (model selection, timeouts, etc.)" },
{ cmd: "config", desc: "Set API keys for external tools" },
{ cmd: "keys", desc: "API key manager — list, add, remove, test, rotate, doctor" },
{ cmd: "hooks", desc: "Show configured post-unit and pre-dispatch hooks" },
{ cmd: "run-hook", desc: "Manually trigger a specific hook" },
{ cmd: "skill-health", desc: "Skill lifecycle dashboard" },
@ -180,6 +181,21 @@ export function registerGSDCommand(pi: ExtensionAPI): void {
.map((s) => ({ value: `setup ${s.cmd}`, label: s.cmd, description: s.desc }));
}
if (parts[0] === "keys" && parts.length <= 2) {
const subPrefix = parts[1] ?? "";
const subs = [
{ cmd: "list", desc: "Show key status dashboard" },
{ cmd: "add", desc: "Add a key for a provider" },
{ cmd: "remove", desc: "Remove a key" },
{ cmd: "test", desc: "Validate key(s) with API call" },
{ cmd: "rotate", desc: "Replace an existing key" },
{ cmd: "doctor", desc: "Health check all keys" },
];
return subs
.filter((s) => s.cmd.startsWith(subPrefix))
.map((s) => ({ value: `keys ${s.cmd}`, label: s.cmd, description: s.desc }));
}
if (parts[0] === "prefs" && parts.length <= 2) {
const subPrefix = parts[1] ?? "";
const subs = [
@ -355,6 +371,13 @@ export function registerGSDCommand(pi: ExtensionAPI): void {
return;
}
if (trimmed === "keys" || trimmed.startsWith("keys ")) {
const { handleKeys } = await import("./key-manager.js");
const keysArgs = trimmed.replace(/^keys\s*/, "").trim();
await handleKeys(keysArgs, ctx);
return;
}
if (trimmed === "setup" || trimmed.startsWith("setup ")) {
const setupArgs = trimmed.replace(/^setup\s*/, "").trim();
await handleSetup(setupArgs, ctx);
@ -734,6 +757,7 @@ function showHelp(ctx: ExtensionCommandContext): void {
" /gsd mode Set workflow mode (solo/team) [global|project]",
" /gsd prefs Manage preferences [global|project|status|wizard|setup]",
" /gsd config Set API keys for external tools",
" /gsd keys API key manager [list|add|remove|test|rotate|doctor]",
" /gsd hooks Show post-unit hook configuration",
"",
"MAINTENANCE",
@ -831,7 +855,8 @@ async function handleSetup(args: string, ctx: ExtensionCommandContext): Promise<
}
if (args === "keys") {
await handleConfig(ctx);
const { handleKeys } = await import("./key-manager.js");
await handleKeys("", ctx);
return;
}

View file

@ -0,0 +1,995 @@
/**
* API Key Manager /gsd keys
*
* Comprehensive CLI for managing API keys: list, add, remove, test, rotate, doctor.
* Works with AuthStorage from pi-coding-agent no core package changes needed.
*/
import type { ExtensionCommandContext } from "@gsd/pi-coding-agent";
import {
AuthStorage,
type AuthCredential,
type ApiKeyCredential,
type OAuthCredential,
} from "@gsd/pi-coding-agent";
import { getEnvApiKey } from "@gsd/pi-ai";
import { existsSync, statSync, chmodSync } from "node:fs";
import { join, dirname } from "node:path";
import { mkdirSync } from "node:fs";
// ─── Provider Registry ─────────────────────────────────────────────────────────
export type ProviderCategory = "llm" | "tool" | "search" | "remote";
export interface ProviderInfo {
id: string;
label: string;
category: ProviderCategory;
envVar?: string;
prefixes?: string[];
hasOAuth?: boolean;
dashboardUrl?: string;
}
export const PROVIDER_REGISTRY: ProviderInfo[] = [
// LLM Providers
{ id: "anthropic", label: "Anthropic (Claude)", category: "llm", envVar: "ANTHROPIC_API_KEY", prefixes: ["sk-ant-"], hasOAuth: true, dashboardUrl: "console.anthropic.com" },
{ id: "openai", label: "OpenAI", category: "llm", envVar: "OPENAI_API_KEY", prefixes: ["sk-"], dashboardUrl: "platform.openai.com/api-keys" },
{ id: "github-copilot", label: "GitHub Copilot", category: "llm", envVar: "GITHUB_TOKEN", hasOAuth: true },
{ id: "openai-codex", label: "ChatGPT Plus/Pro (Codex)",category: "llm", hasOAuth: true },
{ id: "google-gemini-cli",label: "Google Gemini CLI", category: "llm", hasOAuth: true },
{ id: "google-antigravity",label: "Antigravity", category: "llm", hasOAuth: true },
{ id: "google", label: "Google (Gemini)", category: "llm", envVar: "GEMINI_API_KEY", dashboardUrl: "aistudio.google.com/apikey" },
{ id: "groq", label: "Groq", category: "llm", envVar: "GROQ_API_KEY", dashboardUrl: "console.groq.com" },
{ id: "xai", label: "xAI (Grok)", category: "llm", envVar: "XAI_API_KEY", dashboardUrl: "console.x.ai" },
{ id: "openrouter", label: "OpenRouter", category: "llm", envVar: "OPENROUTER_API_KEY", dashboardUrl: "openrouter.ai/keys" },
{ id: "mistral", label: "Mistral", category: "llm", envVar: "MISTRAL_API_KEY", dashboardUrl: "console.mistral.ai" },
{ id: "ollama-cloud", label: "Ollama Cloud", category: "llm", envVar: "OLLAMA_API_KEY" },
{ id: "custom-openai", label: "Custom (OpenAI-compat)", category: "llm", envVar: "CUSTOM_OPENAI_API_KEY" },
{ id: "cerebras", label: "Cerebras", category: "llm", envVar: "CEREBRAS_API_KEY" },
{ id: "azure-openai-responses", label: "Azure OpenAI", category: "llm", envVar: "AZURE_OPENAI_API_KEY" },
// Tool Keys
{ id: "context7", label: "Context7 Docs", category: "tool", envVar: "CONTEXT7_API_KEY", dashboardUrl: "context7.com/dashboard" },
{ id: "jina", label: "Jina Page Extract", category: "tool", envVar: "JINA_API_KEY", dashboardUrl: "jina.ai/api" },
// Search Providers
{ id: "tavily", label: "Tavily Search", category: "search", envVar: "TAVILY_API_KEY", dashboardUrl: "tavily.com/app/api-keys" },
{ id: "brave", label: "Brave Search", category: "search", envVar: "BRAVE_API_KEY", dashboardUrl: "brave.com/search/api" },
// Remote Integrations
{ id: "discord_bot", label: "Discord Bot", category: "remote", envVar: "DISCORD_BOT_TOKEN" },
{ id: "slack_bot", label: "Slack Bot", category: "remote", envVar: "SLACK_BOT_TOKEN", prefixes: ["xoxb-"] },
{ id: "telegram_bot", label: "Telegram Bot", category: "remote", envVar: "TELEGRAM_BOT_TOKEN" },
];
// ─── Utilities ──────────────────────────────────────────────────────────────────
/**
* Mask an API key for display: show first 4 + last 4 chars.
* Keys shorter than 12 chars show only first 2 + last 2.
*/
export function maskKey(key: string): string {
if (!key) return "(empty)";
if (key.length <= 8) return key.slice(0, 2) + "***" + key.slice(-2);
return key.slice(0, 4) + "***" + key.slice(-4);
}
/**
* Format a duration in milliseconds to human-readable.
*/
export function formatDuration(ms: number): string {
if (ms <= 0) return "expired";
const seconds = Math.floor(ms / 1000);
if (seconds < 60) return `${seconds}s`;
const minutes = Math.floor(seconds / 60);
if (minutes < 60) return `${minutes}m`;
const hours = Math.floor(minutes / 60);
const remainMinutes = minutes % 60;
return remainMinutes > 0 ? `${hours}h ${remainMinutes}m` : `${hours}h`;
}
/**
* Describe a credential's type and status.
*/
export function describeCredential(cred: AuthCredential): string {
if (cred.type === "api_key") {
const apiCred = cred as ApiKeyCredential;
if (!apiCred.key) return "empty key";
return `API key (${maskKey(apiCred.key)})`;
}
if (cred.type === "oauth") {
const oauthCred = cred as OAuthCredential;
const remaining = oauthCred.expires - Date.now();
if (remaining <= 0) return "OAuth (expired — will auto-refresh)";
return `OAuth (expires in ${formatDuration(remaining)})`;
}
return "unknown";
}
/**
* Get the auth.json path.
*/
export function getAuthPath(): string {
return join(process.env.HOME ?? "~", ".gsd", "agent", "auth.json");
}
/**
* Create an AuthStorage instance for key management.
*/
export function getKeyManagerAuthStorage(): AuthStorage {
const authPath = getAuthPath();
mkdirSync(dirname(authPath), { recursive: true });
return AuthStorage.create(authPath);
}
/**
* Look up a provider by ID (case-insensitive).
*/
export function findProvider(idOrLabel: string): ProviderInfo | undefined {
const lower = idOrLabel.toLowerCase();
return PROVIDER_REGISTRY.find(
(p) => p.id.toLowerCase() === lower || p.label.toLowerCase() === lower,
);
}
// ─── Key Status / List ──────────────────────────────────────────────────────────
export interface KeyStatus {
provider: ProviderInfo;
configured: boolean;
source: "auth.json" | "env" | "none";
credentialCount: number;
description: string;
backedOff: boolean;
}
/**
* Get the status of all known providers.
*/
export function getAllKeyStatuses(auth: AuthStorage): KeyStatus[] {
return PROVIDER_REGISTRY.map((provider) => {
const creds = auth.getCredentialsForProvider(provider.id);
const envKey = provider.envVar ? process.env[provider.envVar] : undefined;
if (creds.length > 0) {
const firstCred = creds[0];
// Skip empty keys (from skipped onboarding)
if (firstCred.type === "api_key" && !(firstCred as ApiKeyCredential).key) {
return {
provider,
configured: false,
source: "none" as const,
credentialCount: 0,
description: "empty key (skipped setup)",
backedOff: false,
};
}
const desc =
creds.length > 1
? `${creds.length} keys (round-robin)`
: describeCredential(firstCred);
return {
provider,
configured: true,
source: "auth.json" as const,
credentialCount: creds.length,
description: desc,
backedOff: auth.areAllCredentialsBackedOff(provider.id),
};
}
if (envKey) {
return {
provider,
configured: true,
source: "env" as const,
credentialCount: 1,
description: `env ${provider.envVar}`,
backedOff: false,
};
}
return {
provider,
configured: false,
source: "none" as const,
credentialCount: 0,
description: provider.dashboardUrl
? `not configured (${provider.dashboardUrl})`
: provider.envVar
? `not configured (env: ${provider.envVar})`
: "not configured",
backedOff: false,
};
});
}
/**
* Format statuses into a grouped dashboard string.
*/
export function formatKeyDashboard(statuses: KeyStatus[]): string {
const categories: { label: string; key: ProviderCategory }[] = [
{ label: "LLM Providers", key: "llm" },
{ label: "Search Providers", key: "search" },
{ label: "Tool Keys", key: "tool" },
{ label: "Remote Integrations", key: "remote" },
];
const lines: string[] = ["GSD API Key Manager\n"];
for (const cat of categories) {
const items = statuses.filter((s) => s.provider.category === cat.key);
if (items.length === 0) continue;
lines.push(` ${cat.label}`);
for (const item of items) {
const icon = item.configured ? "✓" : "✗";
const backoff = item.backedOff ? " [backed off]" : "";
const pad = item.provider.id.padEnd(20);
lines.push(` ${icon} ${pad}${item.description}${backoff}`);
}
lines.push("");
}
// Summary
const configured = statuses.filter((s) => s.configured);
const fromAuth = configured.filter((s) => s.source === "auth.json");
const fromEnv = configured.filter((s) => s.source === "env");
const oauthCount = statuses.filter((s) => {
if (!s.configured || s.source !== "auth.json") return false;
return s.description.startsWith("OAuth");
}).length;
const parts: string[] = [];
parts.push(`${configured.length} configured`);
if (fromAuth.length > 0) parts.push(`${fromAuth.length} in auth.json`);
if (fromEnv.length > 0) parts.push(`${fromEnv.length} from env`);
if (oauthCount > 0) parts.push(`${oauthCount} OAuth`);
lines.push(` Source: ${getAuthPath()}`);
lines.push(` ${parts.join(" | ")}`);
return lines.join("\n");
}
// ─── Add Key ────────────────────────────────────────────────────────────────────
/**
* Add a key interactively.
*/
export async function handleAddKey(
providerArg: string,
ctx: ExtensionCommandContext,
auth: AuthStorage,
): Promise<boolean> {
let provider: ProviderInfo | undefined;
if (providerArg) {
provider = findProvider(providerArg);
if (!provider) {
ctx.ui.notify(`Unknown provider: "${providerArg}". Use /gsd keys list to see available providers.`, "error");
return false;
}
} else {
// Interactive provider picker
const options = PROVIDER_REGISTRY.map((p) => {
const creds = auth.getCredentialsForProvider(p.id);
const existing = creds.length > 0 ? " (configured)" : "";
return `[${p.category}] ${p.label}${existing}`;
});
const choice = await ctx.ui.select("Add key for which provider?", options);
if (!choice || typeof choice !== "string") return false;
const idx = options.indexOf(choice);
if (idx === -1) return false;
provider = PROVIDER_REGISTRY[idx];
}
// If OAuth is available, offer choice
if (provider.hasOAuth) {
const methods = ["API key", "Browser login (OAuth)"];
const method = await ctx.ui.select(
`${provider.label} — how do you want to authenticate?`,
methods,
);
if (!method || typeof method !== "string") return false;
if (method.includes("OAuth")) {
ctx.ui.notify(
`Use /login to authenticate via OAuth with ${provider.label}.\n` +
`The /login command handles the full browser flow.`,
"info",
);
return false;
}
}
// API key input
const input = await ctx.ui.input(
`API key for ${provider.label}:`,
provider.envVar ? `or set ${provider.envVar} env var` : "paste your key here",
);
if (input === null || input === undefined) return false;
const key = input.trim();
if (!key) {
ctx.ui.notify("No key provided.", "warning");
return false;
}
// Prefix validation
if (provider.prefixes && provider.prefixes.length > 0) {
const valid = provider.prefixes.some((pfx) => key.startsWith(pfx));
if (!valid) {
ctx.ui.notify(
`Warning: key doesn't start with expected prefix (${provider.prefixes.join(" or ")}). Saving anyway.`,
"warning",
);
}
}
auth.set(provider.id, { type: "api_key", key });
if (provider.envVar) {
process.env[provider.envVar] = key;
}
ctx.ui.notify(`Key saved for ${provider.label}: ${maskKey(key)}`, "success");
return true;
}
// ─── Remove Key ─────────────────────────────────────────────────────────────────
/**
* Remove a key interactively.
*/
export async function handleRemoveKey(
providerArg: string,
ctx: ExtensionCommandContext,
auth: AuthStorage,
): Promise<boolean> {
let provider: ProviderInfo | undefined;
if (providerArg) {
provider = findProvider(providerArg);
if (!provider) {
ctx.ui.notify(`Unknown provider: "${providerArg}".`, "error");
return false;
}
} else {
// Show only configured providers
const configured = PROVIDER_REGISTRY.filter((p) => {
const creds = auth.getCredentialsForProvider(p.id);
return creds.length > 0;
});
if (configured.length === 0) {
ctx.ui.notify("No keys configured to remove.", "info");
return false;
}
const options = configured.map((p) => p.label);
const choice = await ctx.ui.select("Remove key for which provider?", options);
if (!choice || typeof choice !== "string") return false;
provider = configured.find((p) => p.label === choice);
if (!provider) return false;
}
const creds = auth.getCredentialsForProvider(provider.id);
if (creds.length === 0) {
ctx.ui.notify(`No keys found for ${provider.label}.`, "info");
return false;
}
// Multi-key handling
if (creds.length > 1) {
const options = creds.map((c, i) => `[${i + 1}] ${describeCredential(c)}`);
options.push("Remove all");
const choice = await ctx.ui.select(
`${provider.label} has ${creds.length} keys. Remove which?`,
options,
);
if (!choice || typeof choice !== "string") return false;
if (choice === "Remove all") {
auth.remove(provider.id);
} else {
// Remove specific index — need to rebuild the array without that entry
const idx = options.indexOf(choice);
if (idx === -1 || idx >= creds.length) return false;
const remaining = creds.filter((_, i) => i !== idx);
auth.remove(provider.id);
for (const c of remaining) {
auth.set(provider.id, c);
}
}
} else {
const confirmed = await ctx.ui.confirm(
"Remove key?",
`Remove ${describeCredential(creds[0])} for ${provider.label}?`,
);
if (!confirmed) return false;
auth.remove(provider.id);
}
// Clear env var
if (provider.envVar && process.env[provider.envVar]) {
delete process.env[provider.envVar];
}
ctx.ui.notify(`Key removed for ${provider.label}.`, "success");
return true;
}
// ─── Test Key ───────────────────────────────────────────────────────────────────
export interface TestResult {
provider: ProviderInfo;
status: "valid" | "invalid" | "rate_limited" | "error" | "skipped";
message: string;
latencyMs?: number;
}
/** Test endpoint configurations per provider */
const TEST_ENDPOINTS: Record<string, { url: string; method?: string; headers?: (key: string) => Record<string, string>; body?: string }> = {
anthropic: {
url: "https://api.anthropic.com/v1/messages",
method: "POST",
headers: (key) => ({
"x-api-key": key,
"anthropic-version": "2023-06-01",
"content-type": "application/json",
}),
body: JSON.stringify({ model: "claude-sonnet-4-20250514", max_tokens: 1, messages: [{ role: "user", content: "hi" }] }),
},
openai: {
url: "https://api.openai.com/v1/models",
headers: (key) => ({ Authorization: `Bearer ${key}` }),
},
google: {
url: "https://generativelanguage.googleapis.com/v1beta/models",
headers: (key) => ({ "x-goog-api-key": key }),
},
groq: {
url: "https://api.groq.com/openai/v1/models",
headers: (key) => ({ Authorization: `Bearer ${key}` }),
},
brave: {
url: "https://api.search.brave.com/res/v1/web/search?q=test&count=1",
headers: (key) => ({ "X-Subscription-Token": key }),
},
tavily: {
url: "https://api.tavily.com/search",
method: "POST",
headers: () => ({ "content-type": "application/json" }),
body: JSON.stringify({ query: "test", max_results: 1 }),
},
discord_bot: {
url: "https://discord.com/api/v10/users/@me",
headers: (key) => ({ Authorization: `Bot ${key}` }),
},
slack_bot: {
url: "https://slack.com/api/auth.test",
headers: (key) => ({ Authorization: `Bearer ${key}` }),
},
telegram_bot: {
url: "", // Constructed dynamically with token in URL
headers: () => ({}),
},
xai: {
url: "https://api.x.ai/v1/models",
headers: (key) => ({ Authorization: `Bearer ${key}` }),
},
mistral: {
url: "https://api.mistral.ai/v1/models",
headers: (key) => ({ Authorization: `Bearer ${key}` }),
},
openrouter: {
url: "https://openrouter.ai/api/v1/models",
headers: (key) => ({ Authorization: `Bearer ${key}` }),
},
};
/**
* Test a single provider's key.
*/
export async function testProviderKey(
provider: ProviderInfo,
auth: AuthStorage,
): Promise<TestResult> {
// Get the API key
const key = await auth.getApiKey(provider.id);
if (!key || key === "<authenticated>") {
if (!key) {
return { provider, status: "skipped", message: "not configured" };
}
return { provider, status: "skipped", message: "uses credential chain (not testable)" };
}
const endpoint = TEST_ENDPOINTS[provider.id];
if (!endpoint) {
return { provider, status: "skipped", message: "no test endpoint configured" };
}
// Special handling for Telegram (token in URL)
let url = endpoint.url;
if (provider.id === "telegram_bot") {
url = `https://api.telegram.org/bot${key}/getMe`;
}
// Special handling for Tavily (API key in body)
let body = endpoint.body;
if (provider.id === "tavily" && body) {
const parsed = JSON.parse(body);
parsed.api_key = key;
body = JSON.stringify(parsed);
}
const start = Date.now();
try {
const res = await fetch(url, {
method: endpoint.method ?? "GET",
headers: endpoint.headers?.(key) ?? {},
body: body ?? undefined,
signal: AbortSignal.timeout(15_000),
});
const latencyMs = Date.now() - start;
if (res.ok) {
return { provider, status: "valid", message: "valid", latencyMs };
}
if (res.status === 401 || res.status === 403) {
return { provider, status: "invalid", message: `invalid key (${res.status})`, latencyMs };
}
if (res.status === 429) {
return { provider, status: "rate_limited", message: "rate limited", latencyMs };
}
return { provider, status: "error", message: `HTTP ${res.status}`, latencyMs };
} catch (err) {
const latencyMs = Date.now() - start;
const msg = err instanceof Error ? err.message : String(err);
if (msg.includes("timeout") || msg.includes("AbortError")) {
return { provider, status: "error", message: "timeout (15s)", latencyMs };
}
return { provider, status: "error", message: msg, latencyMs };
}
}
/**
* Format test results for display.
*/
export function formatTestResults(results: TestResult[]): string {
const lines: string[] = ["API Key Test Results\n"];
for (const r of results) {
const icon =
r.status === "valid" ? "✓" :
r.status === "invalid" ? "✗" :
r.status === "rate_limited" ? "⚠" :
r.status === "error" ? "✗" :
"—";
const pad = r.provider.id.padEnd(20);
const latency = r.latencyMs !== undefined ? ` ${r.latencyMs}ms` : "";
lines.push(` ${icon} ${pad}${r.message}${latency}`);
}
lines.push("");
const valid = results.filter((r) => r.status === "valid").length;
const invalid = results.filter((r) => r.status === "invalid").length;
const rateLimited = results.filter((r) => r.status === "rate_limited").length;
const errors = results.filter((r) => r.status === "error").length;
const skipped = results.filter((r) => r.status === "skipped").length;
const parts: string[] = [];
if (valid > 0) parts.push(`${valid} valid`);
if (invalid > 0) parts.push(`${invalid} invalid`);
if (rateLimited > 0) parts.push(`${rateLimited} rate-limited`);
if (errors > 0) parts.push(`${errors} errors`);
if (skipped > 0) parts.push(`${skipped} skipped`);
lines.push(` ${parts.join(" | ")}`);
return lines.join("\n");
}
// ─── Rotate Key ─────────────────────────────────────────────────────────────────
/**
* Rotate a key: show current, prompt for new, optionally test, then save.
*/
export async function handleRotateKey(
providerArg: string,
ctx: ExtensionCommandContext,
auth: AuthStorage,
): Promise<boolean> {
let provider: ProviderInfo | undefined;
if (providerArg) {
provider = findProvider(providerArg);
if (!provider) {
ctx.ui.notify(`Unknown provider: "${providerArg}".`, "error");
return false;
}
} else {
// Show only configured API key providers
const configured = PROVIDER_REGISTRY.filter((p) => {
const creds = auth.getCredentialsForProvider(p.id);
return creds.some((c) => c.type === "api_key");
});
if (configured.length === 0) {
ctx.ui.notify("No API keys configured to rotate.", "info");
return false;
}
const options = configured.map((p) => p.label);
const choice = await ctx.ui.select("Rotate key for which provider?", options);
if (!choice || typeof choice !== "string") return false;
provider = configured.find((p) => p.label === choice);
if (!provider) return false;
}
const creds = auth.getCredentialsForProvider(provider.id);
const apiKeyCreds = creds.filter((c) => c.type === "api_key") as ApiKeyCredential[];
if (apiKeyCreds.length === 0) {
ctx.ui.notify(`No API keys for ${provider.label} (may use OAuth instead).`, "info");
return false;
}
// Show current key(s)
const currentDesc = apiKeyCreds.map((c) => maskKey(c.key)).join(", ");
ctx.ui.notify(`Current key${apiKeyCreds.length > 1 ? "s" : ""}: ${currentDesc}`, "info");
// Prompt for new key
const input = await ctx.ui.input(
`New API key for ${provider.label}:`,
"paste your new key here",
);
if (input === null || input === undefined) return false;
const newKey = input.trim();
if (!newKey) {
ctx.ui.notify("No key provided. Rotation cancelled.", "warning");
return false;
}
// Prefix validation
if (provider.prefixes && provider.prefixes.length > 0) {
const valid = provider.prefixes.some((pfx) => newKey.startsWith(pfx));
if (!valid) {
ctx.ui.notify(
`Warning: key doesn't start with expected prefix (${provider.prefixes.join(" or ")}).`,
"warning",
);
}
}
// Offer to test before saving
const shouldTest = await ctx.ui.confirm(
"Test key?",
"Validate the new key before saving?",
);
if (shouldTest) {
// Temporarily test the new key
const tempAuth = AuthStorage.inMemory({ [provider.id]: { type: "api_key", key: newKey } });
const result = await testProviderKey(provider, tempAuth);
if (result.status === "invalid") {
ctx.ui.notify(`Key validation failed: ${result.message}. Rotation cancelled.`, "error");
return false;
}
if (result.status === "valid") {
ctx.ui.notify(`Key validated successfully (${result.latencyMs}ms).`, "success");
} else {
ctx.ui.notify(`Key test result: ${result.message}. Proceeding anyway.`, "warning");
}
}
// Remove old keys and add new one
// Preserve any OAuth credentials
const oauthCreds = creds.filter((c) => c.type === "oauth");
auth.remove(provider.id);
for (const c of oauthCreds) {
auth.set(provider.id, c);
}
auth.set(provider.id, { type: "api_key", key: newKey });
if (provider.envVar) {
process.env[provider.envVar] = newKey;
}
ctx.ui.notify(`Key rotated for ${provider.label}: ${maskKey(newKey)}`, "success");
return true;
}
// ─── Key Doctor ─────────────────────────────────────────────────────────────────
export interface DoctorFinding {
severity: "error" | "warning" | "info" | "fixed";
provider?: string;
message: string;
}
/**
* Run health checks on all API keys.
*/
export function runKeyDoctor(auth: AuthStorage): DoctorFinding[] {
const findings: DoctorFinding[] = [];
// 1. Check auth.json permissions
const authPath = getAuthPath();
if (existsSync(authPath)) {
try {
const stats = statSync(authPath);
const mode = stats.mode & 0o777;
if (mode !== 0o600) {
chmodSync(authPath, 0o600);
findings.push({
severity: "fixed",
message: `auth.json permissions were ${mode.toString(8)} — fixed to 600`,
});
}
} catch {
// Can't check permissions — skip
}
}
// 2. Check for empty keys
for (const provider of PROVIDER_REGISTRY) {
const creds = auth.getCredentialsForProvider(provider.id);
for (const cred of creds) {
if (cred.type === "api_key" && !(cred as ApiKeyCredential).key) {
findings.push({
severity: "warning",
provider: provider.id,
message: `${provider.label}: empty key stored (from skipped setup) — run /gsd keys add ${provider.id}`,
});
}
}
}
// 3. Check expired OAuth
for (const provider of PROVIDER_REGISTRY) {
const creds = auth.getCredentialsForProvider(provider.id);
for (const cred of creds) {
if (cred.type === "oauth") {
const oauthCred = cred as OAuthCredential;
const remaining = oauthCred.expires - Date.now();
if (remaining <= 0) {
findings.push({
severity: "warning",
provider: provider.id,
message: `${provider.label}: OAuth token expired — will auto-refresh on next use`,
});
} else if (remaining < 5 * 60 * 1000) {
findings.push({
severity: "info",
provider: provider.id,
message: `${provider.label}: OAuth token expires in ${formatDuration(remaining)} — will auto-refresh`,
});
}
}
}
}
// 4. Check for env var conflicts
for (const provider of PROVIDER_REGISTRY) {
if (!provider.envVar) continue;
const envValue = process.env[provider.envVar];
if (!envValue) continue;
const creds = auth.getCredentialsForProvider(provider.id);
const apiKey = creds.find((c) => c.type === "api_key") as ApiKeyCredential | undefined;
if (apiKey?.key && apiKey.key !== envValue) {
findings.push({
severity: "warning",
provider: provider.id,
message: `${provider.label}: env ${provider.envVar} differs from auth.json — auth.json takes priority`,
});
}
}
// 5. Check for backed-off keys
for (const provider of PROVIDER_REGISTRY) {
if (auth.areAllCredentialsBackedOff(provider.id)) {
const remaining = auth.getProviderBackoffRemaining(provider.id);
findings.push({
severity: "warning",
provider: provider.id,
message: `${provider.label}: all keys in backoff${remaining > 0 ? ` (${formatDuration(remaining)} remaining)` : ""}`,
});
}
}
// 6. Check for missing LLM provider
const llmProviders = PROVIDER_REGISTRY.filter((p) => p.category === "llm");
const hasAnyLlm = llmProviders.some((p) => {
const creds = auth.getCredentialsForProvider(p.id);
const hasValidKey = creds.some((c) => c.type === "api_key" ? !!(c as ApiKeyCredential).key : true);
const hasEnv = p.envVar ? !!process.env[p.envVar] : false;
return hasValidKey || hasEnv;
});
if (!hasAnyLlm) {
findings.push({
severity: "error",
message: "No LLM provider configured — run /gsd keys add or /login",
});
}
// 7. Check for duplicate keys across providers
const keyToProviders = new Map<string, string[]>();
for (const provider of PROVIDER_REGISTRY) {
const creds = auth.getCredentialsForProvider(provider.id);
for (const cred of creds) {
if (cred.type === "api_key" && (cred as ApiKeyCredential).key) {
const key = (cred as ApiKeyCredential).key;
const existing = keyToProviders.get(key) ?? [];
existing.push(provider.id);
keyToProviders.set(key, existing);
}
}
}
for (const [, providers] of keyToProviders) {
if (providers.length > 1) {
findings.push({
severity: "warning",
message: `Same key used by multiple providers: ${providers.join(", ")}`,
});
}
}
return findings;
}
/**
* Format doctor findings for display.
*/
export function formatDoctorFindings(findings: DoctorFinding[]): string {
if (findings.length === 0) {
return "API Key Health Check\n\n All checks passed. No issues found.";
}
const lines: string[] = ["API Key Health Check\n"];
for (const f of findings) {
const icon =
f.severity === "error" ? "✗" :
f.severity === "warning" ? "⚠" :
f.severity === "fixed" ? "✓" :
"";
lines.push(` ${icon} ${f.message}`);
}
lines.push("");
const errors = findings.filter((f) => f.severity === "error").length;
const warnings = findings.filter((f) => f.severity === "warning").length;
const fixed = findings.filter((f) => f.severity === "fixed").length;
const info = findings.filter((f) => f.severity === "info").length;
const parts: string[] = [];
if (errors > 0) parts.push(`${errors} error${errors > 1 ? "s" : ""}`);
if (warnings > 0) parts.push(`${warnings} warning${warnings > 1 ? "s" : ""}`);
if (fixed > 0) parts.push(`${fixed} fixed`);
if (info > 0) parts.push(`${info} info`);
lines.push(` ${parts.join(" | ")}`);
return lines.join("\n");
}
// ─── Main Handler ───────────────────────────────────────────────────────────────
/**
* Main entry point for /gsd keys [subcommand].
*/
export async function handleKeys(
args: string,
ctx: ExtensionCommandContext,
): Promise<void> {
const auth = getKeyManagerAuthStorage();
const parts = args.trim().split(/\s+/);
const subcommand = parts[0] || "";
const subArgs = parts.slice(1).join(" ").trim();
switch (subcommand) {
case "":
case "list":
case "status": {
const statuses = getAllKeyStatuses(auth);
ctx.ui.notify(formatKeyDashboard(statuses), "info");
return;
}
case "add": {
const changed = await handleAddKey(subArgs, ctx, auth);
if (changed) {
await ctx.waitForIdle();
await ctx.reload();
}
return;
}
case "remove":
case "rm":
case "delete": {
const changed = await handleRemoveKey(subArgs, ctx, auth);
if (changed) {
await ctx.waitForIdle();
await ctx.reload();
}
return;
}
case "test":
case "validate": {
let providers: ProviderInfo[];
if (subArgs) {
const p = findProvider(subArgs);
if (!p) {
ctx.ui.notify(`Unknown provider: "${subArgs}".`, "error");
return;
}
providers = [p];
} else {
// Test all configured providers
const statuses = getAllKeyStatuses(auth);
providers = statuses
.filter((s) => s.configured)
.map((s) => s.provider);
}
if (providers.length === 0) {
ctx.ui.notify("No configured keys to test.", "info");
return;
}
ctx.ui.notify(`Testing ${providers.length} key${providers.length > 1 ? "s" : ""}...`, "info");
const results: TestResult[] = [];
for (const p of providers) {
const result = await testProviderKey(p, auth);
results.push(result);
}
ctx.ui.notify(formatTestResults(results), "info");
return;
}
case "rotate": {
const changed = await handleRotateKey(subArgs, ctx, auth);
if (changed) {
await ctx.waitForIdle();
await ctx.reload();
}
return;
}
case "doctor":
case "health": {
const findings = runKeyDoctor(auth);
ctx.ui.notify(formatDoctorFindings(findings), "info");
return;
}
default:
ctx.ui.notify(
"Usage: /gsd keys [list|add|remove|test|rotate|doctor]\n\n" +
" /gsd keys Show key status dashboard\n" +
" /gsd keys list List all configured keys\n" +
" /gsd keys add [id] Add a key for a provider\n" +
" /gsd keys remove [id] Remove a key\n" +
" /gsd keys test [id] Validate key(s) with API call\n" +
" /gsd keys rotate [id] Replace an existing key\n" +
" /gsd keys doctor Health check all keys",
"info",
);
return;
}
}

View file

@ -0,0 +1,414 @@
import test from "node:test";
import assert from "node:assert/strict";
import { AuthStorage } from "@gsd/pi-coding-agent";
import {
maskKey,
formatDuration,
describeCredential,
findProvider,
getAllKeyStatuses,
formatKeyDashboard,
formatTestResults,
runKeyDoctor,
formatDoctorFindings,
PROVIDER_REGISTRY,
} from "../key-manager.ts";
function makeAuth(data: Record<string, any> = {}): AuthStorage {
return AuthStorage.inMemory(data);
}
// ─── maskKey ────────────────────────────────────────────────────────────────────
test("maskKey masks a normal API key showing first 4 and last 4", () => {
assert.equal(maskKey("sk-ant-api03-abcdefghijklmnop"), "sk-a***mnop");
});
test("maskKey masks a short key showing first 2 and last 2", () => {
assert.equal(maskKey("abc12345"), "ab***45");
});
test("maskKey returns (empty) for empty string", () => {
assert.equal(maskKey(""), "(empty)");
});
test("maskKey handles very short keys gracefully", () => {
assert.equal(maskKey("ab"), "ab***ab");
});
test("maskKey handles 12-char boundary", () => {
assert.equal(maskKey("123456789012"), "1234***9012");
});
// ─── formatDuration ─────────────────────────────────────────────────────────────
test("formatDuration formats seconds", () => {
assert.equal(formatDuration(30_000), "30s");
});
test("formatDuration formats minutes", () => {
assert.equal(formatDuration(5 * 60_000), "5m");
});
test("formatDuration formats hours and minutes", () => {
assert.equal(formatDuration(90 * 60_000), "1h 30m");
});
test("formatDuration formats exact hours without minutes", () => {
assert.equal(formatDuration(2 * 60 * 60_000), "2h");
});
test("formatDuration returns expired for zero or negative", () => {
assert.equal(formatDuration(0), "expired");
assert.equal(formatDuration(-1000), "expired");
});
// ─── describeCredential ─────────────────────────────────────────────────────────
test("describeCredential describes an API key with masked value", () => {
const result = describeCredential({ type: "api_key", key: "sk-ant-test-key-12345" });
assert.ok(result.includes("API key"));
assert.ok(result.includes("sk-a"));
assert.ok(result.includes("2345"));
});
test("describeCredential describes an empty API key", () => {
assert.equal(describeCredential({ type: "api_key", key: "" }), "empty key");
});
test("describeCredential describes an OAuth token with expiry", () => {
const result = describeCredential({
type: "oauth",
access: "token",
refresh: "refresh",
expires: Date.now() + 60 * 60_000,
});
assert.ok(result.includes("OAuth"));
assert.ok(result.includes("expires in"));
});
test("describeCredential describes an expired OAuth token", () => {
const result = describeCredential({
type: "oauth",
access: "token",
refresh: "refresh",
expires: Date.now() - 1000,
});
assert.ok(result.includes("expired"));
});
// ─── findProvider ───────────────────────────────────────────────────────────────
test("findProvider finds by exact ID", () => {
assert.equal(findProvider("anthropic")?.id, "anthropic");
});
test("findProvider finds by ID case-insensitively", () => {
assert.equal(findProvider("OPENAI")?.id, "openai");
});
test("findProvider finds by label", () => {
assert.equal(findProvider("Brave Search")?.id, "brave");
});
test("findProvider returns undefined for unknown", () => {
assert.equal(findProvider("nonexistent"), undefined);
});
// ─── PROVIDER_REGISTRY ──────────────────────────────────────────────────────────
test("PROVIDER_REGISTRY has at least 15 providers", () => {
assert.ok(PROVIDER_REGISTRY.length >= 15);
});
test("PROVIDER_REGISTRY has unique IDs", () => {
const ids = PROVIDER_REGISTRY.map((p) => p.id);
assert.equal(new Set(ids).size, ids.length);
});
test("PROVIDER_REGISTRY every provider has id, label, and category", () => {
const validCategories = ["llm", "tool", "search", "remote"];
for (const p of PROVIDER_REGISTRY) {
assert.ok(p.id, `provider missing id`);
assert.ok(p.label, `provider ${p.id} missing label`);
assert.ok(validCategories.includes(p.category), `provider ${p.id} has invalid category: ${p.category}`);
}
});
test("PROVIDER_REGISTRY includes all major LLM providers", () => {
const ids = PROVIDER_REGISTRY.map((p) => p.id);
assert.ok(ids.includes("anthropic"));
assert.ok(ids.includes("openai"));
assert.ok(ids.includes("google"));
assert.ok(ids.includes("groq"));
});
test("PROVIDER_REGISTRY includes all tool/search providers", () => {
const ids = PROVIDER_REGISTRY.map((p) => p.id);
assert.ok(ids.includes("tavily"));
assert.ok(ids.includes("brave"));
assert.ok(ids.includes("context7"));
assert.ok(ids.includes("jina"));
});
// ─── getAllKeyStatuses ───────────────────────────────────────────────────────────
test("getAllKeyStatuses shows unconfigured providers as not configured", () => {
const auth = makeAuth();
const statuses = getAllKeyStatuses(auth);
const anthropic = statuses.find((s) => s.provider.id === "anthropic");
assert.equal(anthropic?.configured, false);
assert.equal(anthropic?.source, "none");
});
test("getAllKeyStatuses detects keys in auth.json", () => {
const auth = makeAuth({ anthropic: { type: "api_key", key: "sk-ant-test" } });
const statuses = getAllKeyStatuses(auth);
const anthropic = statuses.find((s) => s.provider.id === "anthropic");
assert.equal(anthropic?.configured, true);
assert.equal(anthropic?.source, "auth.json");
assert.equal(anthropic?.credentialCount, 1);
});
test("getAllKeyStatuses detects multiple keys", () => {
const auth = makeAuth({
openai: [
{ type: "api_key", key: "sk-key1" },
{ type: "api_key", key: "sk-key2" },
],
});
const statuses = getAllKeyStatuses(auth);
const openai = statuses.find((s) => s.provider.id === "openai");
assert.equal(openai?.configured, true);
assert.equal(openai?.credentialCount, 2);
assert.ok(openai?.description.includes("round-robin"));
});
test("getAllKeyStatuses detects empty keys as not configured", () => {
const auth = makeAuth({ groq: { type: "api_key", key: "" } });
const statuses = getAllKeyStatuses(auth);
const groq = statuses.find((s) => s.provider.id === "groq");
assert.equal(groq?.configured, false);
assert.ok(groq?.description.includes("empty"));
});
test("getAllKeyStatuses detects env var keys", () => {
const original = process.env.OPENAI_API_KEY;
process.env.OPENAI_API_KEY = "sk-env-test";
try {
const auth = makeAuth();
const statuses = getAllKeyStatuses(auth);
const openai = statuses.find((s) => s.provider.id === "openai");
assert.equal(openai?.configured, true);
assert.equal(openai?.source, "env");
} finally {
if (original === undefined) {
delete process.env.OPENAI_API_KEY;
} else {
process.env.OPENAI_API_KEY = original;
}
}
});
// ─── formatKeyDashboard ─────────────────────────────────────────────────────────
test("formatKeyDashboard includes header and category sections", () => {
const auth = makeAuth({ anthropic: { type: "api_key", key: "sk-ant-test-key" } });
const statuses = getAllKeyStatuses(auth);
const output = formatKeyDashboard(statuses);
assert.ok(output.includes("GSD API Key Manager"));
assert.ok(output.includes("LLM Providers"));
assert.ok(output.includes("Search Providers"));
assert.ok(output.includes("Tool Keys"));
assert.ok(output.includes("Remote Integrations"));
});
test("formatKeyDashboard shows configured counts", () => {
const auth = makeAuth({
anthropic: { type: "api_key", key: "sk-ant-test" },
tavily: { type: "api_key", key: "tvly-test" },
});
const statuses = getAllKeyStatuses(auth);
const output = formatKeyDashboard(statuses);
assert.ok(output.includes("configured"));
assert.ok(output.includes("auth.json"));
});
// ─── formatTestResults ──────────────────────────────────────────────────────────
test("formatTestResults formats valid results with checkmark", () => {
const results = [
{
provider: { id: "anthropic", label: "Anthropic", category: "llm" as const },
status: "valid" as const,
message: "valid",
latencyMs: 142,
},
];
const output = formatTestResults(results);
assert.ok(output.includes("✓"));
assert.ok(output.includes("anthropic"));
assert.ok(output.includes("142ms"));
assert.ok(output.includes("1 valid"));
});
test("formatTestResults formats invalid results with X", () => {
const results = [
{
provider: { id: "groq", label: "Groq", category: "llm" as const },
status: "invalid" as const,
message: "invalid key (401)",
latencyMs: 89,
},
];
const output = formatTestResults(results);
assert.ok(output.includes("✗"));
assert.ok(output.includes("invalid"));
});
test("formatTestResults formats skipped results with dash", () => {
const results = [
{
provider: { id: "jina", label: "Jina", category: "tool" as const },
status: "skipped" as const,
message: "not configured",
},
];
const output = formatTestResults(results);
assert.ok(output.includes("—"));
assert.ok(output.includes("1 skipped"));
});
test("formatTestResults shows summary counts for mixed results", () => {
const results = [
{ provider: { id: "a", label: "A", category: "llm" as const }, status: "valid" as const, message: "ok", latencyMs: 100 },
{ provider: { id: "b", label: "B", category: "llm" as const }, status: "invalid" as const, message: "401", latencyMs: 50 },
{ provider: { id: "c", label: "C", category: "tool" as const }, status: "skipped" as const, message: "n/a" },
];
const output = formatTestResults(results);
assert.ok(output.includes("1 valid"));
assert.ok(output.includes("1 invalid"));
assert.ok(output.includes("1 skipped"));
});
// ─── runKeyDoctor ───────────────────────────────────────────────────────────────
test("runKeyDoctor reports empty keys", () => {
const auth = makeAuth({ groq: { type: "api_key", key: "" } });
const findings = runKeyDoctor(auth);
const emptyFinding = findings.find((f) => f.message.includes("empty key"));
assert.ok(emptyFinding, "should find empty key warning");
assert.equal(emptyFinding?.severity, "warning");
});
test("runKeyDoctor reports expired OAuth", () => {
const auth = makeAuth({
anthropic: { type: "oauth", access: "t", refresh: "r", expires: Date.now() - 10_000 },
});
const findings = runKeyDoctor(auth);
const oauthFinding = findings.find((f) => f.message.includes("expired"));
assert.ok(oauthFinding, "should find expired OAuth warning");
assert.equal(oauthFinding?.severity, "warning");
});
test("runKeyDoctor reports soon-to-expire OAuth as info", () => {
const auth = makeAuth({
anthropic: { type: "oauth", access: "t", refresh: "r", expires: Date.now() + 2 * 60_000 },
});
const findings = runKeyDoctor(auth);
const oauthFinding = findings.find((f) => f.message.includes("expires in"));
assert.ok(oauthFinding, "should find expiring OAuth info");
assert.equal(oauthFinding?.severity, "info");
});
test("runKeyDoctor reports missing LLM provider", () => {
const llmEnvVars = [
"ANTHROPIC_API_KEY", "ANTHROPIC_OAUTH_TOKEN", "OPENAI_API_KEY",
"GEMINI_API_KEY", "GROQ_API_KEY", "XAI_API_KEY", "OPENROUTER_API_KEY",
"MISTRAL_API_KEY", "GITHUB_TOKEN", "GH_TOKEN", "COPILOT_GITHUB_TOKEN",
"OLLAMA_API_KEY", "CUSTOM_OPENAI_API_KEY", "CEREBRAS_API_KEY",
"AZURE_OPENAI_API_KEY",
];
const saved: Record<string, string | undefined> = {};
for (const v of llmEnvVars) {
saved[v] = process.env[v];
delete process.env[v];
}
try {
const auth = makeAuth();
const findings = runKeyDoctor(auth);
const missingLlm = findings.find((f) => f.message.includes("No LLM provider"));
assert.ok(missingLlm, "should find missing LLM error");
assert.equal(missingLlm?.severity, "error");
} finally {
for (const [k, v] of Object.entries(saved)) {
if (v !== undefined) process.env[k] = v;
else delete process.env[k];
}
}
});
test("runKeyDoctor does not report missing LLM when one is configured", () => {
const auth = makeAuth({ anthropic: { type: "api_key", key: "sk-ant-test" } });
const findings = runKeyDoctor(auth);
const missingLlm = findings.find((f) => f.message.includes("No LLM provider"));
assert.equal(missingLlm, undefined);
});
test("runKeyDoctor reports duplicate keys across providers", () => {
const auth = makeAuth({
openai: { type: "api_key", key: "shared-key-123" },
groq: { type: "api_key", key: "shared-key-123" },
});
const findings = runKeyDoctor(auth);
const dupFinding = findings.find((f) => f.message.includes("Same key used"));
assert.ok(dupFinding, "should find duplicate key warning");
assert.equal(dupFinding?.severity, "warning");
});
test("runKeyDoctor reports env var conflicts", () => {
const original = process.env.OPENAI_API_KEY;
process.env.OPENAI_API_KEY = "env-key";
try {
const auth = makeAuth({ openai: { type: "api_key", key: "different-key" } });
const findings = runKeyDoctor(auth);
const conflict = findings.find((f) => f.message.includes("differs from auth.json"));
assert.ok(conflict, "should find env var conflict");
assert.equal(conflict?.severity, "warning");
} finally {
if (original === undefined) {
delete process.env.OPENAI_API_KEY;
} else {
process.env.OPENAI_API_KEY = original;
}
}
});
test("runKeyDoctor returns no issues when everything is healthy", () => {
const auth = makeAuth({ anthropic: { type: "api_key", key: "sk-ant-healthy" } });
const findings = runKeyDoctor(auth);
const nonFileFindings = findings.filter((f) => !f.message.includes("auth.json permissions"));
assert.equal(nonFileFindings.length, 0);
});
// ─── formatDoctorFindings ───────────────────────────────────────────────────────
test("formatDoctorFindings shows all-clear for no findings", () => {
const output = formatDoctorFindings([]);
assert.ok(output.includes("All checks passed"));
});
test("formatDoctorFindings shows findings with appropriate icons", () => {
const output = formatDoctorFindings([
{ severity: "error", message: "No LLM provider configured" },
{ severity: "warning", provider: "groq", message: "Empty key" },
{ severity: "fixed", message: "Permissions fixed" },
]);
assert.ok(output.includes("✗"));
assert.ok(output.includes("⚠"));
assert.ok(output.includes("✓"));
assert.ok(output.includes("1 error"));
assert.ok(output.includes("1 warning"));
assert.ok(output.includes("1 fixed"));
});