Skip to main content

Knowledge, Evals, and Releases

This area is where you manage what Hunch knows, how you test it, and how you roll changes out safely.

Knowledge sources​

Hunch supports managed knowledge sources per website, including:

  • Manual text sources - Paste trusted internal documentation, policies, FAQs, or product positioning directly into Hunch
  • URL sources - Add individual URLs to index specific pages
  • Sitemap import - Import entire documentation or help-center sitemaps in one pass
  • Crawled pages - Hunch automatically indexes pages from your website when you run discovery

Each source can be indexed, refreshed, and removed from the dashboard.

Adding Knowledge Sources​

Sitemap Import​

To import a whole documentation or help-center site at once:

  1. Go to your website's Knowledge dashboard
  2. Click Import Sitemap
  3. Enter your sitemap URL (e.g., https://docs.yoursite.com/sitemap.xml)
  4. Optionally add a label prefix (e.g., "docs")
  5. Click Import

The import is duplicate-safe - running it again won't create duplicates.

Manual Text Sources​

For authoritative Q&A that should always be used first:

  1. Go to Canonical Facts Console in your knowledge dashboard
  2. Add Q&A pairs:
    • Question: "What is Hunch?"
      Answer: "Hunch is an AI-powered chat widget..."
    • Question: "How much does it cost?" Answer: "We offer a Starter plan at $79/month..."
  3. Save

Canonical facts take priority over crawled content, giving you full control over high-value answers.

Troubleshooting Low Confidence Answers​

If visitors see responses like "We couldn't find any information..." or low confidence scores, there are several things to check:

1. Add Canonical Facts​

When Hunch has low confidence in its answers, it's usually because:

  • The crawled content doesn't match what visitors ask
  • The AI policy is too strict (e.g., "Require citations" + no citations found)

Solution: Add authoritative Q&A in the Canonical Facts Console:

Q: What is [Your Product]?
A: [Your authoritative answer]

Q: What are your pricing plans?
A: Our Starter plan is $79/month with 3000 credits...

Canonical facts override crawled content for those specific questions.

2. Review AI Policy​

Check your AI Policy settings in the knowledge console:

  • Safe topics only - Only answers questions matching your content
  • Allow legal answers - Enable if you want legal questions answered
  • Allow refund answers - Enable for refund policy questions
  • Require pricing citations - Only answers pricing if sources cite prices
  • Require support citations - Only answers support if sources cite contacts

If you have strict policies but no matching sources, confidence will be low. Ease the policy or add more facts.

3. Refresh Stale Knowledge​

If pages are marked as "stale", re-run the discovery or add them as a source to refresh the content.

4. Check Answer Traces​

The Answer Trace Review section shows exactly what Hunch used for each answer. Use this to debug why certain questions get low confidence.

Snapshots​

Snapshots let you freeze the current operating state for a website before major changes.

A snapshot captures the current combination of:

  • website knowledge state
  • important AI operating settings
  • related assumptions used for rollout and evaluation

Use snapshots before:

  • policy changes
  • pricing changes
  • prompt or instruction changes
  • larger website content refreshes

Eval suites​

Eval suites let you test retrieval and answer behavior against known cases.

You can:

  • add suggested eval cases
  • write manual cases
  • run the suite on demand
  • pin a baseline run
  • compare the latest run against that baseline

Typical things to watch:

  • pass rate
  • regressions
  • improvements
  • confidence shifts
  • missing citations or wrong intent classification

Automation and gates​

Release automation lets you:

  • schedule recurring eval runs
  • require approval before deployment
  • create regression alerts
  • stop rollout when a gate is blocked

This is the recommended workflow for high-impact websites where knowledge or prompt changes can affect revenue or compliance-sensitive answers.

Release candidates​

You can turn a state into a release candidate, evaluate it, and then promote it when it is ready.

Supported deployment types:

  • stable for full rollout
  • canary for percentage-based limited rollout
  • experiment for variant-based testing

Each deployment records:

  • traffic share
  • variant key
  • deployment type
  • start and optional end window

Active deployments​

The widget resolves live config from the current deployment records. That means rollout choices in the dashboard directly affect what visitors see.

  1. Update or add knowledge sources.
  2. Save a snapshot before larger changes.
  3. Run evals and review regressions.
  4. Create a release candidate.
  5. Approve the candidate if gates pass.
  6. Roll out to stable, canary, or experiment traffic.
  7. Watch sessions, handoffs, and analytics after release.

See also: