Prerequisites
Runtime
Section titled “Runtime”| Tool | Version | Purpose |
|---|---|---|
| Node.js | >= 20 | Runtime (set via engines in package.json) |
| pnpm | 10.x | Package manager (pinned via packageManager in root package.json) |
| Turbo | 2.x | Monorepo task runner (dev dependency) |
LLM CLI (at least one)
Section titled “LLM CLI (at least one)”An LLM CLI is a command-line tool that lets you interact with a large language model from your terminal — you send it a prompt, it sends back a response. semtest spawns these CLIs as child processes using child_process.spawn(), writes the test prompt to their stdin (or passes it as an argument for Claude), and reads back the structured JSON response from stdout. You need at least one installed and authenticated.
semtest supports 14 tools. Run semtest list to see all available model keys. The most commonly used:
| CLI | Install | Auth |
|---|---|---|
claude (Claude Code) | claude.ai/docs | claude auth login |
gemini (Gemini CLI) | ai.google.dev | gcloud auth login |
codex (Codex CLI) | npm i -g @openai/codex | OPENAI_API_KEY env var |
aider | pip install aider-chat | Provider-specific API key |
opencode | npm i -g opencode | Provider-specific API key |
goose | block.github.io/goose | Provider-specific API key |
Other supported tools: Crush, Qwen, GitHub Copilot, Forge, Plandex, OpenHands, Cursor, Amp.
GitHub PAT (for RC builds only)
Section titled “GitHub PAT (for RC builds only)”Stable releases are published to public npm — no authentication needed to install. RC (release candidate) builds are published to GitHub Packages for internal testing. To install RC builds, you need a GitHub Personal Access Token with read:packages scope.
Create one at Settings > Developer Settings > Personal Access Tokens > Tokens (classic) with the read:packages permission.