Config

Source files: src/config/schema.ts, src/config/loader.ts

Configuration is how the tool knows what to do. It answers the fundamental questions: which test files to find? Which LLM model should evaluate them? Should validation issues be treated as errors?

The config system has two layers. At compile time, the defineConfig() helper gives users TypeScript autocompletion as they write their semtest.config.ts file. At runtime, Zod validates the loaded config object — because TypeScript types are erased when code runs, the config file could contain anything (wrong types, invalid model keys, missing fields), and Zod catches these problems immediately with clear error messages rather than letting them cause cryptic failures later in the pipeline.

Loading the config involves a second tool: jiti. Node.js can’t natively import .ts files, but users want to write config in TypeScript for the autocompletion. jiti is a runtime TypeScript loader that transpiles on-the-fly when the config is loaded — no separate build step needed.

Schema

The config schema is defined with Zod in schema.ts. All fields have defaults, so a minimal config works out of the box:

Field	Type	Default	Description
output	string	"semtest-results/"	Directory for generated reports
testMatch	string[]	["**/*.spec.md", "**/*.test.md"]	Glob patterns for test file discovery
testPathIgnorePatterns	string[]	["node_modules", "dist", ".git", "vendor"]	Patterns to exclude from discovery
llm	ModelKey	"claude-code-sonnet-4-6"	Model key identifying which LLM CLI and model to invoke
timeout	number	0	Timeout per test in milliseconds (0 = no timeout)
repeat	number	1	Number of times to repeat each test (stops on first failure)
bail	boolean | number	false	Stop after first failure (true) or after N failures (number)
verbose	boolean	false	Show detailed per-test output
strict	boolean	false	Exit code 2 if validation issues found
skipValidation	boolean	false	Skip post-run validation entirely
debug	boolean	false	Log raw LLM output to debug files
timestamp	boolean	false	Generate timestamped report copies
includePassing	boolean	false	Include passing tests in MD report
junit	boolean	false	Generate JUnit XML report
skipPermissionsIfPossible	boolean	false	Skip tool permission prompts where supported

Model keys

The llm field accepts a model key — a string that identifies both the LLM CLI tool and the specific model to use. For example, "claude-code-sonnet-4-6" invokes Claude Code with the Sonnet 4.6 model, while "gemini-2.5-pro" invokes the Gemini CLI with the 2.5 Pro model. Run semtest list to see all available model keys.

The schema validates this field using z.enum(MODEL_KEYS), where MODEL_KEYS is derived from the model registry at src/runner/registry.ts. Invalid model keys fail validation immediately with a clear error.

Test discovery patterns

The testMatch field accepts an array of glob patterns (processed by tinyglobby). The defaults (**/*.spec.md and **/*.test.md) discover Markdown spec and test files recursively. The testPathIgnorePatterns field lists directory names to exclude — these are expanded to **/{pattern}/** globs internally.

Types

Two types are exported:

• SemtestConfig — The fully resolved config (z.infer). All fields are present and typed.
• SemtestUserConfig — The input type (z.input). Fields are optional where defaults exist.

defineConfig()

An identity function that provides type safety when writing config files:

import { defineConfig } from "@westopp/semtest";

export default defineConfig({
  output: "semtest-results/",
  llm: "claude-code-sonnet-4-6",
});

The function simply returns its argument — it exists purely for TypeScript autocompletion and validation.

Config loader

loadConfig() in loader.ts handles finding and parsing the config file:

1. Search — walks up from cwd looking for one of:
   • semtest.config.ts
   • semtest.config.js
   • semtest.config.mjs
2. Load — uses jiti to import the file (handles TypeScript natively, no pre-compilation needed)
3. Resolve — reads the default export (or the module itself if no default)
4. Validate — runs semtestConfigSchema.safeParse() and throws a formatted error on failure

If no config file is found, loadConfig() throws with a clear message.