Skip to content
Guth

Products and agents

Labs

Test, compare and improve how AI behaves, without becoming a machine-learning engineer.

Private preview

Labs is where Guth tests, compares and improves how AI behaves. Tilly, the research agent, guides the work so you do not need to be a machine-learning engineer. Parts of Labs are live today; most is in private preview.

What is live, and what is not

Part of LabsStatusWhat it means
Instrument and field guide docsLiveAnyone can read the instruments and field guides catalogs.
Public experiment recordsLiveWrite-ups of past checks, such as the instrument verification pass, and the Labs experiments page.
Labs app (signed in)Private previewInvited preview members can view Labs. It is read-only or limited for now.
Experiment builderPrivate previewBuilding and running your own experiments in the app is part of the preview and still limited.
Personal trainingIn developmentTraining an agent on your own examples is being built. It is not available yet.

What Tilly helps with

  • Turning a question into an experiment.
  • Choosing what to compare and how to score it.
  • Collecting evidence and keeping a research journal.
  • Comparing models, prompts, tools and settings.
  • Reading results and finding where things fail.
  • Spotting good training examples.

Experiments

An experiment
i

A fair test with a written setup: what was tried, on which examples, how it was scored and what happened, so someone else can repeat it.

Read more

is a fair test with a written setup: what was tried, on which examples, how it was scored, and what happened. Writing the setup down means someone else can run it again and get the same answer.

Common kinds:

  • A/B tests: two versions side by side.
  • Model comparisons
    i

    The AI system doing the thinking for an agent at a given moment. Models can be swapped; the agent keeps its identity, history and knowledge.

    Read more

    : the same task on different models.
  • Prompt sweeps: many small wording changes, to see which matters.
  • Robustness tests: tricky or unusual inputs, to see what breaks.
  • Did the training help?: checking a trained version against the old one.

Comparisons are evidence, not votes

When several models agree, that is not proof. Labs looks for evidence: scores on examples the model did not train on, failures that can be repeated, and results someone else can check.

How personal training will work In development

Personal model training
i

Teaching a model a specific job or agent from your own examples. Guth collects, checks, trains and tests, and switches it on only if it is better. In development.

Read more

is in development. The idea is simple: teach a model a specific job, or one of your agents, from your own examples, and let Guth handle the steps. The technical name for the usual method is LoRA, a light way to adjust a model without rebuilding it. The plan has seven steps, and nothing is switched on until it earns it:
  1. Collect examples.
  2. Group them by purpose.
  3. Check quality, consent and where each example came from.
  4. Train.
  5. Test on separate examples that were not used for training.
  6. Compare with how the agent behaved before.
  7. Turn it on only when the comparison justifies it.
Training on your data is always your choice
i

Your choice to let your examples help train or improve a model. It is off unless you turn it on, and a bug report does not count as consent.

Read more

. Nothing you do in Guth is used for training unless you opt in.

Local models on every device In development

Also in development: training and running local models across your own devices, such as PCs with graphics cards, Macs and even iPhones, working together as one fleet that shares one memory. You will be able to run chosen agents locally, so they stay with you, stay private and can work offline. This is separate from sharing idle computer power with the community, which is a later idea.

Where to go next

  • Agents: meet Tilly and the other agents.
  • Instruments: checked calculators and lookups that experiments can rely on.
  • Release status: the current label for every product.