Skip to main content

Knowledge, Evals, and Releases

This area is where you manage what Hunch knows, how you test it, and how you roll changes out safely.

Knowledge sources​

Hunch supports managed knowledge sources per website, including:

  • manual text sources
  • URL sources
  • text-based file uploads

Each source can be indexed, refreshed, and removed from the dashboard.

Snapshots​

Snapshots let you freeze the current operating state for a website before major changes.

A snapshot captures the current combination of:

  • website knowledge state
  • important AI operating settings
  • related assumptions used for rollout and evaluation

Use snapshots before:

  • policy changes
  • pricing changes
  • prompt or instruction changes
  • larger website content refreshes

Eval suites​

Eval suites let you test retrieval and answer behavior against known cases.

You can:

  • add suggested eval cases
  • write manual cases
  • run the suite on demand
  • pin a baseline run
  • compare the latest run against that baseline

Typical things to watch:

  • pass rate
  • regressions
  • improvements
  • confidence shifts
  • missing citations or wrong intent classification

Automation and gates​

Release automation lets you:

  • schedule recurring eval runs
  • require approval before deployment
  • create regression alerts
  • stop rollout when a gate is blocked

This is the recommended workflow for high-impact websites where knowledge or prompt changes can affect revenue or compliance-sensitive answers.

Release candidates​

You can turn a state into a release candidate, evaluate it, and then promote it when it is ready.

Supported deployment types:

  • stable for full rollout
  • canary for percentage-based limited rollout
  • experiment for variant-based testing

Each deployment records:

  • traffic share
  • variant key
  • deployment type
  • start and optional end window

Active deployments​

The widget resolves live config from the current deployment records. That means rollout choices in the dashboard directly affect what visitors see.

  1. Update or add knowledge sources.
  2. Save a snapshot before larger changes.
  3. Run evals and review regressions.
  4. Create a release candidate.
  5. Approve the candidate if gates pass.
  6. Roll out to stable, canary, or experiment traffic.
  7. Watch sessions, handoffs, and analytics after release.

See also: