fix(openai): never run a tool call cut off mid-stream #18

Open

thklein wants to merge 1 commit from fix/incomplete-tool-calls into main

pull from: fix/incomplete-tool-calls

merge into: basepeak-ai:main

basepeak-ai:main

basepeak-ai:dependabot/pip/examples/treasure-hunt/werkzeug-3.1.4

basepeak-ai:temperature

basepeak-ai:renovate/configure

Conversation 0 Commits 1 Files changed 2 +345 -3

thklein commented 2026-10-16 20:28:23 +00:00

Owner

Copy link

Why

Production, tenant allard, thread t16bmc4 (2026-10-16 08:25Z): the model router ended an HTTP 200 stream with an in-stream error (server_error: The model stopped sending output mid-stream) while the model was partway through a workspace_write call. The stream reader takes in-stream errors only from a line that begins with data: {"error":. A chunk that has choices and a top-level error with finish_reason: "error" is parsed as an ordinary chunk. After it comes a clean [DONE], so call returned the partial message as a finished completion. The engine then ran workspace_write with the unterminated arguments {"filename": "Bestandsregister_Dokumentensammlung.md" (0 bytes written) and saved the call into the chat state. From then on the provider rejected every request on that thread (invalid_request_error: The request was rejected as malformed), because the history held arguments it could not parse.

Change

pkg/openai/client.go:

• checkComplete: a completion is incomplete when its stream ends with finish_reason: "error", or when any tool call has non-empty arguments that are not valid JSON (this also catches a dropped connection). Empty arguments are still accepted, since models send them for tools with no parameters. A user cancel skips the check.
• call retries an incomplete completion up to 3 times; this is safe because nothing from it has run. If every attempt is incomplete it returns ErrIncompleteCompletion. The tool call is never handed to the engine, never executed and never cached. The non-streaming path runs the same check, and a response with no choices now returns an error instead of panicking on Choices[0].
• dropInvalidToolCalls (called from toMessages) is a second line of defence. Before history is sent, it removes every tool call whose arguments are invalid JSON, the tool result that answers it, and any assistant message left empty. It works on copies and leaves the engine's state alone. Threads that already carry such a call, like t16bmc4, recover on their next message.

Verification

• New pkg/openai/incomplete_test.go. Its SSE fixtures replay the incident: an errored stream, a dropped stream, and a truncated stream that succeeds on retry. Further fixtures cover a complete call and an empty-arguments call, plus a poisoned-history fixture for the sanitizer.
• All tests failed before the change (the partial arguments came back as a successful completion and the bad call was sent in history). They pass after it: go test -race ./pkg/openai/ passes.
• go test ./pkg/... passes except pkg/tests TestMCPLoad, which hangs to the 10-minute timeout in the MCP loader on this machine. That test has nothing to do with this change.
• golangci-lint run ./pkg/openai/... reports 0 issues (with the Go 1.26 toolchain CI uses). Run over the whole repo, the installed linter (v2.13.2, newer than the v2.9.0 pinned here) reports 32 issues. The same 32 are reported on main, and none of them are in this change.
• For bpai, the dependency bump is basepeak-ai/bpai#1393.

## Why Production, tenant allard, thread `t16bmc4` (2026-10-16 08:25Z): the model router ended an HTTP 200 stream with an in-stream error (`server_error: The model stopped sending output mid-stream`) while the model was partway through a `workspace_write` call. The stream reader takes in-stream errors only from a line that begins with `data: {"error":`. A chunk that has `choices` and a top-level `error` with `finish_reason: "error"` is parsed as an ordinary chunk. After it comes a clean `[DONE]`, so `call` returned the partial message as a finished completion. The engine then ran `workspace_write` with the unterminated arguments `{"filename": "Bestandsregister_Dokumentensammlung.md"` (0 bytes written) and saved the call into the chat state. From then on the provider rejected every request on that thread (`invalid_request_error: The request was rejected as malformed`), because the history held arguments it could not parse. ## Change `pkg/openai/client.go`: - **`checkComplete`**: a completion is incomplete when its stream ends with `finish_reason: "error"`, or when any tool call has non-empty arguments that are not valid JSON (this also catches a dropped connection). Empty arguments are still accepted, since models send them for tools with no parameters. A user cancel skips the check. - **`call`** retries an incomplete completion up to 3 times; this is safe because nothing from it has run. If every attempt is incomplete it returns `ErrIncompleteCompletion`. The tool call is never handed to the engine, never executed and never cached. The non-streaming path runs the same check, and a response with no choices now returns an error instead of panicking on `Choices[0]`. - **`dropInvalidToolCalls`** (called from `toMessages`) is a second line of defence. Before history is sent, it removes every tool call whose arguments are invalid JSON, the tool result that answers it, and any assistant message left empty. It works on copies and leaves the engine's state alone. Threads that already carry such a call, like `t16bmc4`, recover on their next message. ## Verification - New `pkg/openai/incomplete_test.go`. Its SSE fixtures replay the incident: an errored stream, a dropped stream, and a truncated stream that succeeds on retry. Further fixtures cover a complete call and an empty-arguments call, plus a poisoned-history fixture for the sanitizer. - All tests failed before the change (the partial arguments came back as a successful completion and the bad call was sent in history). They pass after it: `go test -race ./pkg/openai/` passes. - `go test ./pkg/...` passes except `pkg/tests` `TestMCPLoad`, which hangs to the 10-minute timeout in the MCP loader on this machine. That test has nothing to do with this change. - `golangci-lint run ./pkg/openai/...` reports 0 issues (with the Go 1.26 toolchain CI uses). Run over the whole repo, the installed linter (v2.13.2, newer than the v2.9.0 pinned here) reports 32 issues. The same 32 are reported on `main`, and none of them are in this change. - For bpai, the dependency bump is basepeak-ai/bpai#1393.

thklein added 1 commit 2026-10-16 20:28:23 +00:00

fix(openai): never run a tool call cut off mid-stream ... a191d5514a

A provider can end an HTTP 200 stream with an error frame
(finish_reason "error") or drop the connection, and the stream reader
handed whatever had arrived to the engine as a finished completion. A
workspace_write whose arguments stopped after the filename was then
executed and recorded, and every later request on that thread was
rejected as malformed because its history held unparseable arguments.

- Treat a completion ending with finish_reason "error", or holding a
  tool call whose arguments are not valid JSON, as incomplete: it is
  re-requested up to three times (nothing from it has run), then
  returned as ErrIncompleteCompletion instead of executed.
- Drop tool calls with invalid JSON arguments, and the results
  answering them, from the history sent to the model, so threads
  already carrying one recover on their next message.

thklein force-pushed fix/incomplete-tool-calls from a191d5514a to 4f45a84f97 2026-10-16 20:48:45 +00:00 Compare

thklein referenced this pull request from a commit 2026-10-16 21:01:50 +00:00

chore(deps): bump gptscript for cut-off tool calls

Some checks failed

Hide checks

smoke / check-label (pull_request_target) Failing after 1m42s

Details

smoke / gpt-4o-2024-08-06 (pull_request_target) Has been skipped

Details

smoke / gpt-4o-mini-2024-07-18 (pull_request_target) Has been skipped

Details

smoke / claude-3-7-sonnet-20250219 (pull_request_target) Has been skipped

Details

test / test (ubuntu-22.04) (pull_request) Failing after 59s

Details

integration / integration (ubuntu-22.04) (pull_request) Failing after 2m56s

Details

Validate docs build / validate-docs (pull_request) Failing after 1m37s

Details

integration / integration (windows-latest) (pull_request) Has been cancelled

Details

test / test (windows-latest) (pull_request) Has been cancelled

Details

This pull request can be merged automatically.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin fix/incomplete-tool-calls:fix/incomplete-tool-calls

git switch fix/incomplete-tool-calls

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main

git merge --no-ff fix/incomplete-tool-calls

git switch fix/incomplete-tool-calls

git rebase main

git switch main

git merge --ff-only fix/incomplete-tool-calls

git switch fix/incomplete-tool-calls

git rebase main

git switch main

git merge --no-ff fix/incomplete-tool-calls

git switch main

git merge --squash fix/incomplete-tool-calls

git switch main

git merge --ff-only fix/incomplete-tool-calls

git switch main

git merge fix/incomplete-tool-calls

git push origin main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/engine

  

clarity/clear

  

enhancement

  

security

  

size/M

No labels

area/engine

clarity/clear

enhancement

security

size/M

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

basepeak-ai/gptscript!18

No description provided.