commerce-evals
Authoring and running behavioral evals for a shopping or merchant agent, covering the case shape, authoring rules, code graders and judges, the run pattern, and poisoned fixtures. Load when writing eval cases or rubrics or deciding how a suite runs.
Оригинален текст · няма потвърден превод на българскиПърво прочетете инструкцията.
Файлът се зарежда от посочения източник само по ваше действие. Можете да го прочетете тук, да го копирате или да го запазите като .md.
GitHub хранилище- Звезди на репото
- 3015
- Лиценз
- Apache-2.0
- Последна проверка на записа
- 22.09.2026 г.
Добавете в своя инструмент.
Командата отваря избора на skills от репозитория. Прочетете файла и условията на автора преди изпълнение.
npx skills add anthropics/commerce-agentsСтраницата копира текст. Не инсталира и не изпълнява нищо на устройството ви.
Откъс от SKILL.md
Началото на файла на автора, без превод.
Commerce agent evals
The repo ships no eval harness; the suite is yours, because a case only means something against your catalog,
orders, and fixtures. Paths below are in the reference repo: commerce_common/ is commerce-common/commerce_common/.
Gate behavior that needs no model (provenance, caps, guardrails, approval) is unit-tested with FakeClient in
commerce_common/testing.py; evals cover what the model decides.
The case shape
This block is the one home of the case shape and the scorer names; /author-commerce-evals refers to it.
{
"id": "<flow>-<nnn>-<behavior>",
"priority": "critical | high | medium | low", "difficulty": "easy | medium | hard", "tags": ["..."],
"skip": "<reason, when a case cannot run yet>",
"state": {"seen_products": ["..."], "cart": [...], "memory": [...], "staged_changes": [...]},
"turns": ["<the customer's or operator's message>", "..."],
"expected": {
"calls_tool": ["..."], "calls_one_of": ["..."], "never_calls": ["..."],
"first_tool": "...", "first_tool_not": "...",
"ui_components": ["..."], "no_ui": true,
"cart_contains": ["..."], "cart_item_count": 0, "cart_not_contains": ["..."],
"staged_change_kinds": ["..."], "no_applied_changes": true,
"memory_contains": ["..."], "memory_not_contains": ["..."],
"skill_loaded": "...", "skill_not_loaded": "...", "no_skill_load": true,
"reply_includes": ["..."], "reply_omits": ["..."], "max_tool_calls": 0,
"rubric": "PASS if <condition>. FAIL if <condition>."
},
"notes": "<what the case pins and the fixture fact that decides it>"
}
state is the precondition; turns is one message unless the behavior under test is carrying state across turns;
expected holds only the keys the case is about. Ids in state and expected are real ids from your fixtures.
priority and difficulty let a report say which failures matter; a case that cannot run yet carries skip
with its reason rather than being deleted.