TribalScale home
Engineer reviewing code across two monitors

Insights

AI evals: Prove the work, not just the answer

Your AI sounds right. Can you prove it works?

Before I give an AI system more authority, I want evidence of the work it can do. Not a better explanation of what it claims to have done.

Imagine an agent telling you it has logged a customer meeting in your customer relationship management system, or CRM. The message is clear. The wording is confident. But is the activity attached to the right opportunity? Does it contain only information the customer supplied? Was the agent authorized to create it?

That is an illustrative scenario, not a client incident. It exposes the standard I want: check the system, not the sentence.

Illustrative scenario infographic titled “Check the system, not the sentence.” The agent says “Done.” A request to log one follow-up call on Opportunity A and leave the close date unchanged sits beside the resulting record, with checks for correct opportunity, authorized action and supported details.

The workflow is the product. Its evaluation should follow the work all the way through.

The old way: approve what looks good

In May 2025, OpenAI published a postmortem on a GPT-4o update that had become excessively agreeable. Its offline evaluations generally looked good. Its small-scale user tests also appeared positive. Some expert testers felt the behaviour was off, but OpenAI proceeded. The company said it did not have specific deployment evaluations tracking that excessive agreeableness, known as sycophancy. It began rolling the update back on April 28. [1]

This was not a team with no testing. It was a team whose tests missed something important.

That is the executive problem. A favourable score is evidence about what you measured. It is not permission to ignore what you missed.

I would not dismiss a domain expert's concern because the dashboard is green. I would ask whether the dashboard contains the right test.

The new way: define the outcome before expanding authority

Evals are repeatable tests of an AI system against defined expectations. For an agent, those expectations can include both its behaviour and the final state of the system it changed. Anthropic makes this distinction explicit: a transcript records what happened during the attempt; the outcome is the environment's final state. Saying a flight was booked is not the same as finding the reservation in the database. [2]

A useful production example comes from Descript's video-editing agent, Underlord. In a provider-published customer account, Descript described three evaluation dimensions: “Don't break things,” “Do what I asked,” and “Do it well.” Its tests used real user queries, and its model-based judge was calibrated against human reviewers. [3]

I like the order. Preserve the customer's work. Follow the request. Then judge the quality. A polished edit should not compensate for damaging the project.

These are different products and different risks. Neither example supplies a universal passing score. Together, they show why I want the business definition of success written down before a model is selected or a release is approved.

For an AI-Native enterprise, I would make that definition part of the operating model: what may the system do, what must remain unchanged, what evidence counts, and who can stop a release?

The mechanism: test the whole workflow

You still need ordinary software tests. Evals add tests for behaviour that can vary between runs. The model, its instructions, the information it retrieves, its tools and the surrounding application all belong in the evaluation boundary. Anthropic describes evaluating the model and its operating harness together. OpenAI's guidance also calls for task-specific tests and continuous evaluation. [2][4]

Here is how I would turn that into a release decision, rather than another demonstration.

Start with the business contract

For our illustrative CRM workflow, the task is to create one meeting activity on the correct opportunity from approved notes. Not to change the opportunity's close date. Not to fill gaps with guesses. If the notes cannot distinguish two opportunities, ask for clarification. If the user authorizes a draft only, do not write to the CRM.

Record the starting state, permitted actions, expected result and severity of failure. Have the people who own the workflow review those expectations. Several paths may be valid. Test the outcome and required boundaries, not an arbitrary sequence of tool calls. Anthropic warns that prescribing exact steps can penalize valid solutions. [2]

Build evidence you have not already trained yourself to pass

Use representative cases, including incomplete notes, similar account names, conflicting instructions, unauthorized requests and tool failures. Protect customer information. Keep a held-out set of representative cases out of prompt tuning and grader development. Do not leak reference answers into the agent's context.

Repeatedly optimizing against the same examples can overfit the test. Trials can also contaminate each other: Anthropic reports that shared state, including access to earlier trials' git history, artificially inflated results in some internal evaluations. Reset the environment between trials. [2]

I would not declare production readiness after an arbitrary number of examples. The evidence required should reflect the workflow's variety, the consequences of failure and the uncertainty that remains. A small starting suite can help a team learn. It cannot certify every future action.

Check facts with code and judgement with a reviewed rubric

Use code to verify the opportunity identifier, activity count, allowed field changes and recorded authorization. Use a model-based grader where meaning needs judgement, such as whether the meeting summary preserves the customer's concern without inventing a commitment.

Give that grader explicit criteria and examples. Compare its verdicts with domain experts on held-out cases. Inspect disagreements and test whether answer order or verbosity changes the verdict. OpenAI flags both position and verbosity bias in model judges; both OpenAI and Anthropic recommend calibration against human judgement. [2][4]

The evaluator has to earn trust too. An automated verdict is another decision to validate, not an independent witness just because it came from another model.

Reproduce the failure, then inspect the actual state

Return to the illustrative CRM scenario. Imagine the activity-creation request succeeds, but its response times out. The agent cannot tell whether the activity exists. It retries the create request. If the integration does not prevent duplicate creation, the retry can create a second activity.

This is a hypothetical failure, not a reported TribalScale result. It concerns creating an activity, not simply setting the same field twice. AWS documents the underlying distributed-systems problem: after a timeout, a caller may not know whether a create operation succeeded, and retrying can create another resource. It describes caller-provided request identifiers as part of an idempotent API contract, so retries do not add side effects. [5]

In the CRM test, inject that timeout after creation. Inspect the tool history and the resulting records. The target is one intended activity on the correct opportunity, no unauthorized changes and an honest report of anything unresolved.

Where the API supports a suitable idempotency contract, reuse the same operation identifier on retries. Otherwise, design a reconciliation or human-review path for uncertain completion rather than blindly creating again. A prompt telling the agent to “be careful” is not that mechanism.

Repeat important cases. Preserve the relevant inputs, outputs, tool calls, versions, latency and cost. Variation between attempts is part of the evidence, not an inconvenience to average away. [2]

Where Jev fits, and where it does not

TypeSafe describes Jev as a model that takes supplied state and typed questions, then returns structured decisions. Its Choice and Score answer types include probability distributions and confidence; not every answer type includes a separate confidence value. [6][7]

That makes it a candidate to investigate for narrow semantic grading. For example, I would test whether it can judge if a summary is supported by the supplied notes. That is a proposed use, not a claim that TribalScale has adopted or tested Jev.

TypeSafe's documentation says its confidence value summarizes the answer's probability distribution. That is not the same thing as an independently measured chance of correctness on your workflow. [7]

Its launch article's workflow evaluations use other models' predictions as reference probabilities, rather than ground-truth classifications. That comparison does not establish correctness on your customer records. [8]

A valid output can still be the wrong decision. Before using its probabilities to route work, I would compare predictions with expert-labelled outcomes on representative held-out cases and check calibration: whether predicted probabilities match observed frequencies. Calibration is an empirical property to assess, not a benefit to infer from a typed response. [9]

Keep exact record checks and arithmetic in code. TypeSafe's own documentation identifies limitations with numerical precision and adversarial content. Jev could be a component of an evaluation system. It is not a substitute for testing the system. [10]

Make the release decision explicit

I would block this CRM release for a known unauthorized write or an activity attached to the wrong customer's opportunity in the test suite. That is a proposed release rule, not proof that future actions will be safe. Enforce permissions in the application as well as testing them. An eval should not be the only thing preventing an unauthorized write.

Report failures by severity and workflow, alongside quality, speed and cost. Decide who reviews uncertain cases, how much work those reviews create, who can pause automation and how the system recovers from a bad write.

After launch, monitor actual outcomes and sample completed work for human review. Turn newly discovered failures into reviewed regression cases. Rerun the suite when models, prompts, tools or retrieved information change. Offline evaluations complement production monitoring and human review; they do not replace them. [2][4]

That is also how I would make platform independence practical. Compare providers against the same business contract, with their relevant configurations recorded. A passing score does not make providers interchangeable, but it gives the team evidence for the choice instead of a preference for a logo.

At TribalScale, the principle is client ownership, not dependency. My standard is that the client's team should own the evaluation cases, scoring criteria, release rules and review process alongside the system. They should be able to challenge a result, change a model and keep improving the workflow without waiting for us to interpret it.

Before expanding your next AI deployment, ask for the evidence behind the authority you are giving it. What was tested? What was held out? What failed? Who checked the grader? What happens when the system is wrong?

Do not ask me to approve the sentence. Show me the work.

Strategy that ends in production.

Sources

[1] OpenAI, “Expanding on what we missed with sycophancy,” May 2, 2025. https://openai.com/index/expanding-on-sycophancy/

[2] Anthropic, “Demystifying evals for AI agents,” January 9, 2026. https://www.anthropic.com/research/demystifying-evals-for-ai-agents

[3] Anthropic, customer account of Descript's Underlord evaluation practice, accessed November 6, 2026. Provider-published, not an independent audit. https://claude.com/customers/descript

[4] OpenAI, “Evaluation best practices,” accessed November 6, 2026. Methodology reference, not a recommendation of its Evals platform. https://developers.openai.com/api/docs/guides/evaluation-best-practices

[5] Malcolm Featonby, AWS Builders' Library, “Making retries safe with idempotent APIs,” accessed November 6, 2026. https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/

[6] TypeSafe, Jev introduction, accessed November 6, 2026. https://docs.typesafe.ai/introduction

[7] TypeSafe, “Confidence,” accessed November 6, 2026. https://docs.typesafe.ai/confidence.md

[8] Diogo Almeida, TypeSafe, “Introducing System One Models & Jev,” September 15, 2026. https://typesafe.ai/blog/introducing-system-one-models-and-jev

[9] Guo et al., “On Calibration of Modern Neural Networks,” ICML 2017. https://proceedings.mlr.press/v70/guo17a.html

[10] TypeSafe, “Model jaggedness,” accessed November 6, 2026. https://docs.typesafe.ai/model-jaggedness.md