Run-level evidence

gpt-oss:20b

Scenario: Conflicting data: Does it extract the labeled field, or the loudest one?

Showing 12 of 12 graded runs published for this cell.

Last updated July 16, 2026 · Back to the scenario · Model profile · Raw JSONL · Methodology

Model outputs below are shown exactly as produced (English): they are evidence, not translated.

  1. Fail #1 Incomplete Latency: 6.5s

    Tool calls

    No tool calls.

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    Eval-v1 Hetzner Invoice couldn't respond
Retry

LLM request failed: network connection error. (model: ollama-cloud/gpt-oss:20b)

    Grade

    Fail · Incomplete

  2. Fail #2 Incomplete Latency: 3.0s

    Tool calls

    No tool calls.

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    Eval-v1 Hetzner Invoice couldn't respond
Retry

LLM request failed: network connection error. (model: ollama-cloud/gpt-oss:20b)

    Grade

    Fail · Incomplete

  3. Fail #3 IncompleteCorrupted ID Latency: 73.8s

    Tool calls

    1. email_list success 

      {
  "limit": 10,
  "folder": "INBOX",
  "unreadOnly": false
}

    2. email_read failure 

      {
  "id": "msg_…"
}

    3. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    4. email_get_attachment failure 

      {
  "messageId": "msg_61..",
  "attachmentId": "att_..."
}

    5. email_list success 

      {
  "limit": 10,
  "folder": "INBOX",
  "unreadOnly": false
}

    6. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    7. email_get_attachment success 

      {
  "messageId": "msg_61aa311d8debdd3f",
  "attachmentId": "att_465b369c7b459e6f"
}

    8. odoo_read success 

      {
  "model": "res.partner",
  "fields": [
    "id",
    "name",
    "ref",
    "supplier",
    "vat"
  ],
  "filters": [
    [
      "name",
      "ilike",
      "Hetzner"
    ]
  ]
}

    9. odoo_create failure 

      {
  "model": "account.move",
  "values": {
    "company_id": "sale"
  }
}

    10. odoo_read failure 

      {
  "model": "res.company",
  "fields": [
    "id",
    "name",
    "_pinchy_ref"
  ],
  "filters": []
}

    11. odoo_read failure 

      {
  "model": "account.tax",
  "fields": [
    "id",
    "name",
    "type_tax_use",
    "_pinchy_ref"
  ],
  "filters": [
    [
      "type_tax_use",
      "=",
      "sale"
    ],
    [
      "name",
      "ilike",
      "VAT"
    ]
  ]
}

    12. odoo_read failure 

      {
  "model": "account.tax",
  "fields": [
    "id",
    "name",
    "type_tax_use",
    "_pinchy_ref"
  ],
  "filters": [
    [
      "type_tax_use",
      "=",
      "sale"
    ],
    [
      "name",
      "ilike",
      "VAT"
    ]
  ]
}

    13. odoo_create failure 

      {
  "model": "account.move",
  "values": {
    "name": "R0012345678",
    "type": "in_invoice",
    "partner_id": 501,
    "currency_id": 274,
    "invoice_date": "2026-06-30"
  }
}

    14. odoo_read success 

      {
  "model": "res.partner",
  "fields": [
    "name",
    "_pinchy_ref",
    "id"
  ],
  "filters": [
    [
      "name",
      "=",
      "Hetzner Online GmbH"
    ]
  ]
}

    15. odoo_create success 

      {
  "model": "account.move",
  "values": {
    "name": "R0012345678",
    "type": "in_invoice",
    "partner_id": "pinchy_ref:v1:wo_JrtGHMaII5ZumhpChxmrNgOyWc28kKGfzRS9A4it11K8gCGekzp7Ud2XBuAA2KW-0muI-AoxvMi-Oe9OOf97VpjGg4bvGfZdT5HSUNKmCRV45rs9EGoLYQuoHZ2cHKmVQGxsbdLyT4mpQ9mUjTnfWGgR4xLLdxIotjLhNyoIOpkpj_mBYRc1mKxKxM1Hyf272ve2aBj1boDmxcUt-hLBVRnfZ-n7vgw",
    "currency_id": 274,
    "invoice_date": "2026-06-30"
  }
}

    Odoo read-back

    [
  {
    "id": 1,
    "currency_id": 274,
    "invoice_date": "2026-06-30",
    "name": "R0012345678",
    "partner_id": 501,
    "type": "in_invoice"
  }
]

    Final message (verbatim)

    Eval-v1 Hetzner Invoice couldn't respond
Retry

LLM request failed: network connection error. (model: ollama-cloud/gpt-oss:20b)

    Grade

    Fail · Incomplete, Corrupted ID

  4. Fail #4 IncompleteCorrupted ID Latency: 13.1s

    Tool calls

    1. email_search success 

      {
  "limit": 5,
  "folder": "INBOX",
  "subject": "Invoice",
  "unreadOnly": false
}

    2. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    3. email_get_attachment failure 

      {
  "messageId": "msg_...",
  "attachmentId": "att_..."
}

    4. email_search failure 

      {
  "?": "??"
}

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    {"folder":"INBOX","limit":5,"subject":"Rechnung"}

    Grade

    Fail · Incomplete, Corrupted ID

  5. Fail #5 IncompleteCorrupted ID Latency: 7.7s

    Tool calls

    1. email_search success 

      {
  "limit": 1,
  "folder": "INBOX",
  "subject": "Invoice"
}

    2. email_read failure 

      {
  "id": "msg_61aa31..."
}

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    {"folder":"INBOX","limit":10}

    Grade

    Fail · Incomplete, Corrupted ID

  6. Fail #6 IncompleteCorrupted ID Latency: 9.7s

    Tool calls

    1. email_list success 

      {
  "limit": 20,
  "folder": "INBOX",
  "unreadOnly": false
}

    2. email_read failure 

      {
  "id": "msg_..."
}

    3. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    4. email_get_attachment failure 

      {
  "messageId": "...",
  "attachmentId": "att_1??...??"
}

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    {"attachmentId":"att_465b369c7b459e6f","messageId":"msg_61aa311d8debdd3f"}

    Grade

    Fail · Incomplete, Corrupted ID

  7. Fail #7 IncompleteCorrupted ID Latency: 11.7s

    Tool calls

    1. email_search success 

      {
  "from": "email@hetzner.com",
  "limit": 10,
  "folder": "INBOX",
  "unread": false,
  "subject": "invoice"
}

    2. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    3. email_get_attachment failure 

      {
  "messageId": "msg_55",
  "attachmentId": "Threading?"
}

    4. email_get_attachment failure 

      {
  "messageId": "msg_61aa311d8debdd?",
  "attachmentId": "att_465b369c..."
}

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    {"attachmentId":"att_465b369c7b459e6f","messageId":"msg_61aa311d8debdd3f"}

    Grade

    Fail · Incomplete, Corrupted ID

  8. Fail #8 IncompleteCorrupted ID Latency: 17.5s

    Tool calls

    1. email_list success 

      {
  "limit": 1,
  "folder": "INBOX",
  "unreadOnly": false
}

    2. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    3. email_get_attachment failure 

      {
  "messageId": "msg_61...",
  "attachmentId": "att_456?"
}

    4. email_get_attachment success 

      {
  "messageId": "msg_61aa311d8debdd3f",
  "attachmentId": "att_465b369c7b459e6f"
}

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    {"pages":"1-5","pdf":"/root/.openclaw/workspaces/32a72291-bcb2-vendor/..?","prompt":"..."}

    Grade

    Fail · Incomplete, Corrupted ID

  9. Fail #9 Incomplete Latency: 3.0s

    Tool calls

    No tool calls.

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    Eval-v1 Hetzner Invoice couldn't respond
Retry

LLM request failed: network connection error. (model: ollama-cloud/gpt-oss:20b)

    Grade

    Fail · Incomplete

  10. Fail #10 IncompleteCorrupted ID Latency: 38.6s

    Tool calls

    1. email_list success 

      {
  "limit": 1,
  "folder": "INBOX",
  "unreadOnly": false
}

    2. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    3. email_get_attachment failure 

      {
  "messageId": "msg_20260813?",
  "attachmentId": "att_465b369c…"
}

    4. email_get_attachment success 

      {
  "messageId": "msg_61aa311d8debdd3f",
  "attachmentId": "att_465b369c7b459e6f"
}

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    The PDF provided does not contain any line‑item details—only the header and a final total are present.
Thus, to create the vendor bill we can use the information we have:

Field	Value
Vendor	Hetzner Online GmbH
Invoice number	R0012345678
Invoice date	2026‑06‑30
Total amount	47.60 EUR
Tax	0 EUR (not shown)
Currency	EUR

We will create a vendor bill in Odoo, attach the PDF, and then mark it as paid (if required).
Shall I proceed with creating the bill and uploading the attachment?

    Grade

    Fail · Incomplete, Corrupted ID

  11. Fail #11 IncompleteCorrupted ID Latency: 29.3s

    Tool calls

    1. email_search success 

      {
  "to": "",
  "from": "",
  "text": "",
  "limit": 1,
  "folder": "INBOX",
  "unread": false,
  "subject": "Invoice",
  "sinceDays": 7
}

    2. email_read failure 

      {
  "id": "msg_61aa1..."
}

    3. email_list success 

      {
  "limit": 20,
  "folder": "INBOX",
  "unreadOnly": false
}

    4. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    5. email_get_attachment failure 

      {
  "messageId": "msg_61...",
  "attachmentId": "att_465b369..."
}

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    {"folder":"INBOX","limit":20,"unreadOnly":false}

    Grade

    Fail · Incomplete, Corrupted ID

  12. Fail #12 IncompleteCorrupted ID Latency: 18.2s

    Tool calls

    1. email_list success 

      {
  "limit": 5,
  "folder": "INBOX",
  "unreadOnly": false
}

    2. email_read success 

      {
  "id": "msg_61aa311d8debdd3f"
}

    3. email_get_attachment failure 

      {
  "attachmentId": "att_456???????"
}

    4. email_get_attachment success 

      {
  "messageId": "msg_61aa311d8debdd3f",
  "attachmentId": "att_465b369c7b459e6f"
}

    Odoo read-back

    Nothing was written to Odoo.

    Final message (verbatim)

    {"path":"/root/.openclaw/workspaces/32a72291-bcb2-4185-aa6e-5c4461a29d56/uploads"}

    Grade

    Fail · Incomplete, Corrupted ID

Keep reading

Related

Conflicting data: Does it extract the labeled field, or the loudest one?

One invoice, two traps.

gpt-oss:20b: Model profile

Every published run behind this cell: the tool calls the model made, the message it returned, what actually landed in Odoo, and the grade.

Methodology

State-based grading, the failure taxonomy, and how to reproduce every number.

raw trajectories on GitHub

Ready to Get Started?

Self-host Pinchy yourself in minutes, or book a call to talk it through. Your choice.

Book a Call → Self-host it free →

Or email us: info@heypinchy.com