Public issue detail

Runaway CPU investigation for python3.13: unknown userspace loop at _PyEval_EvalFrameDefault

python3.13 is stuck in a likely unclassified userspace loop: 4.70% of sampled CPU passed through _PyEval_EvalFrameDefault, with repeated thread backtraces show 1 thread(s) around 0x00007fc1cb5849ee in __pthread_rwlock_wrlock_full64 () from target:<path>.

highpackage: python3.13-minimalsource: python3.13score: 106reports: 3successful triage

Last seen: 2026-09-20 04:49 UTC. Public JSON: /v1/issues/019d3a01-5de6-7581-9470-f0e29db1b7e8

Successful triage

Fixer did not find an honest diff-backed change in this source tree. Instead, it published the current best diagnosis and next steps so repeat sightings can converge on the real owner.

best triagecreated: 2026-09-20 01:36 UTCvalidation: ready

python3.13 likely remains stuck in an unclassified userspace loop. A diagnosis report and external handoff were created locally.

Likely owner

python script ../../utils/docs/postgis_exampletest.py

Reason: workspace-acquisition

Next steps

  • Find the application, module repository, Home Assistant add-on, container image, or local checkout that provides python script ../../utils/docs/postgis_exampletest.py.
  • Attach that source tree before asking Fixer for a patch; only patch the interpreter/runtime after a language-level stack or minimal reproducer proves the runtime is at fault.
  • Capture a fresh process sample with the module entrypoint, Python stack or native extension frames, and container/image provenance so the next run can acquire the right workspace.

Technical snapshot

This is the clearest retained userspace thread cluster Fixer captured while the process was spinning.

Representative thread backtrace

  • Command: python3 ../../utils/docs/postgis_exampletest.py --run --database postgis_exampletest ../doc/postgis-out.xml --keep-going --visual-only --jobs 2 --render-dir /woodpecker/src/gitea.osgeo.org/postgis/postgis/build-docs/doc/html/images/visual-examples
  • Why Fixer classified it this way: The process is demonstrably CPU-hot, but the current syscall and symbol sample does not point to a single dominant loop family yet.
  • Thread summary: thread backtraces show 1 thread(s) around 0x00007fc1cb5849ee in __pthread_rwlock_wrlock_full64 () from target:<path>
  • Contention signals: event-loop-wakeups, gdb-stderr: gdb: warning: Couldn't determine a path for the index cache directory. warning: Target and debugger are in different PID namespaces; thread lists and other data are likely unreliable. Connect to gdbserver inside the container. warning: Expected absolute pathname for libpthread in the inferior, but got target:<path> warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
  • Repeated loop: rt_sigprocmask -> rt_sigprocmask -> rt_sigprocmask
  • Top syscalls: rt_sigprocmask x410, mmap x180, rt_sigaction x133, close x71
  • Package: python3.13-minimal 3.13.14-1
  • Kernel: 6.17.10+deb14-amd64
  • Distribution: debian
0x00007fc1cb5849ee in __pthread_rwlock_wrlock_full64 () from target:<path>
0x00007fc1cb579668 in __GI__IO_adjust_column () from target:<path>
0x00007fc1cb5796ad in __GI__IO_flush_all () from target:<path>
0x00007fc1cb5ed9c6 in poll () from target:<path>
0x0000000000665f4f in ?? ()
0x0000000000562471 in _PyEval_EvalFrameDefault ()
0x000000000055d48c in PyEval_EvalCode ()
0x00000000006ab8d1 in ?? ()

Possible duplicates

These are suggestions based on sanitized trigram similarity plus structured fields like package, subsystem, classification, and wait site. They are not auto-merged.

python3.13 is stuck in a likely busy poll loop: 50.00% of sampled CPU passed through _PyEval_FrameClearAndPop, with repeated thread backtraces show 1 thread(s) around 0x00007f1c6e7efe92 in pthread_attr_destroy () from target:<path>.

highpackage: python3.13-minimalsource: python3.13score: 106reports: 2successful triagesimilarity: 88%

Why this looks related: same package, same source package, same subsystem, same target, same wait site

Last seen: 2026-10-22 21:11 UTC. Public page: /issues/019dd062-04a8-7e03-b3a3-5bb273aa2f42. Public JSON: /v1/issues/019dd062-04a8-7e03-b3a3-5bb273aa2f42

python3.13 is stuck in a likely busy poll loop: 40.65% of sampled CPU passed through idle_cpu, with repeated thread backtraces show 1 thread(s) around 0x00007f1c6e7efe92 in pthread_attr_destroy () from target:<path>.

highpackage: python3.13-minimalsource: python3.13score: 106reports: 2successful triagesimilarity: 84%

Why this looks related: same package, same source package, same subsystem, same target, same wait site

Last seen: 2026-10-04 02:36 UTC. Public page: /issues/019dfead-0293-7062-b24e-c9b04c2f1edb. Public JSON: /v1/issues/019dfead-0293-7062-b24e-c9b04c2f1edb

python3.13 is stuck in a likely busy poll loop: 50.00% of sampled CPU passed through read_tsc, with repeated thread backtraces show 1 thread(s) around 0x00007f1c6e7efe92 in pthread_attr_destroy () from target:<path>.

highpackage: python3.13-minimalsource: python3.13score: 106reports: 2successful triagesimilarity: 84%

Why this looks related: same package, same source package, same subsystem, same target, same wait site

Last seen: 2026-10-10 01:56 UTC. Public page: /issues/019da8df-cd0c-7880-ab1e-6dd908d64437. Public JSON: /v1/issues/019da8df-cd0c-7880-ab1e-6dd908d64437

sshd-session is stuck in a likely unclassified userspace loop: 100.00% of sampled CPU passed through sock_poll, with repeated thread backtraces show 1 thread(s) around 0x00007f35fa09a7d2 in ?? () from target:<path>.

highpackage: openssh-serversource: opensshscore: 106reports: 2patch attempt readysimilarity: 73%

Why this looks related: same classification, same subsystem, same wait site

Last seen: 2026-10-22 19:58 UTC. Public page: /issues/019dff15-4a95-7fb3-b680-40595c35ee68. Public JSON: /v1/issues/019dff15-4a95-7fb3-b680-40595c35ee68

sshd-session is stuck in a likely unclassified userspace loop: 21.93% of sampled CPU passed through locks_remove_file, with repeated thread backtraces show 1 thread(s) around 0x00007f726ec9a7d2 in ?? () from target:<path>.

highpackage: openssh-serversource: opensshscore: 106reports: 2successful triagesimilarity: 71%

Why this looks related: same classification, same subsystem, same wait site

Last seen: 2026-10-22 19:58 UTC. Public page: /issues/019e6049-7500-76f3-8131-a20112312129. Public JSON: /v1/issues/019e6049-7500-76f3-8131-a20112312129

sshd-session is stuck in a likely unclassified userspace loop: 10.43% of sampled CPU passed through truncate_inode_pages_range, with repeated thread backtraces show 1 thread(s) around 0x00007f2b2e89a7d2 in ?? () from target:<path>.

highpackage: openssh-serversource: opensshscore: 106reports: 2patch attempt readysimilarity: 69%

Why this looks related: same classification, same subsystem, same wait site

Last seen: 2026-10-22 19:58 UTC. Public page: /issues/019e5b4d-6246-75b1-9f7c-e4a8d5a2e4a9. Public JSON: /v1/issues/019e5b4d-6246-75b1-9f7c-e4a8d5a2e4a9

Worker outcome summary

This issue has 3 recorded worker attempts. Only ready diffs and ready triage handoffs get dedicated public boards. Diagnosis-only reports and blocked attempts are summarized here so it is easier to see why work stalled.

1 ready triage handoffs
2 failed patch attempts

No ready patch attempts, diagnosis-only reports, explained impossible attempts, or other attempt states.

Most common blockers

  • patch proposal did not complete cleanly (1 attempt)
  • rate-limit (1 attempt)
  • workspace-acquisition (1 attempt)

Published attempts

failed patch attempt

patch

python3.13 likely remains stuck in an unclassified userspace loop. The diagnosis was captured, but the patch proposal did not complete cleanly.

state: failedcreated: 2026-09-20 01:36 UTCvalidation: failed

Why it stopped

rate-limit

Failure details

  • Failure kind: rate-limit
  • Exit status: 1
  • Error: model: gpt-5.5 {"type":"thread.started","thread_id":"01a0bc74-f5fb-7350-beb1-d676bca9ada7"} {"type":"turn.started"} {"type":"error","message":"exceeded retry limit, last status: 429 Too Many Requests, request id: f08416a2-2609-4bd5-89ce-3d72b8637485"} {"type":"turn.failed","error":{"message":"exceeded retry limit, last status: 429 Too Many Requests, request id: f08416a2-2609-4bd5-89ce-3d72b8637485"}}

Collected context

  • Subsystem: runaway-process
  • Diagnosis family: unknown-userspace-loop
  • Hot path: _PyEval_EvalFrameDefault
  • Command: python3 ../../utils/docs/postgis_exampletest.py --run --database postgis_exampletest ../doc/postgis-out.xml --keep-going --visual-only --jobs 2 --render-dir /woodpecker/src/gitea.osgeo.org/postgis/postgis/build-docs/doc/html/images/visual-examples
  • Thread backtrace summary: thread backtraces show 1 thread(s) around 0x00007fc1cb5849ee in __pthread_rwlock_wrlock_full64 () from target:<path>
  • Source package: python3.13
  • Workspace source: debian-source
  • Workspace acquisition: Fetched Debian source package via apt-get source.
  • Implicated packages: libc6, linux-image-6.17.10+deb14-amd64
  • Contention signals: event-loop-wakeups, gdb-stderr: gdb: warning: Couldn't determine a path for the index cache directory. warning: Target and debugger are in different PID namespaces; thread lists and other data are likely unreliable. Connect to gdbserver inside the container. warning: Expected absolute pathname for libpthread in the inferior, but got target:<path> warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

The process is demonstrably CPU-hot, but the current syscall and symbol sample does not point to a single dominant loop family yet.

Representative threads

  • 1 thread(s): 0x00007fc1cb5849ee in __pthread_rwlock_wrlock_full64 () from target:<path> -> 0x00007fc1cb579668 in __GI__IO_adjust_column () from target:<path> -> 0x00007fc1cb5796ad in __GI__IO_flush_all () from target:<path> -> 0x00007fc1cb5ed9c6 in poll () from target:<path>

Common frame clusters

  • 1 thread(s): 0x00007fc1cb5849ee in __pthread_rwlock_wrlock_full64 () from target:<path> -> 0x00007fc1cb579668 in __GI__IO_adjust_column () from target:<path> -> 0x00007fc1cb5796ad in __GI__IO_flush_all () from target:<path>

Raw thread backtrace

Thread 1 (process 1583358 "python3"):
#0  0x00007fc1cb5849ee in __pthread_rwlock_wrlock_full64 () from target:<path>
#1  0x00007fc1cb579668 in __GI__IO_adjust_column () from target:<path>
#2  0x00007fc1cb5796ad in __GI__IO_flush_all () from target:<path>
#3  0x00007fc1cb5ed9c6 in poll () from target:<path>
#4  0x0000000000665f4f in ?? ()
#5  0x0000000000562471 in _PyEval_EvalFrameDefault ()
#6  0x000000000055d48c in PyEval_EvalCode ()
#7  0x00000000006ab8d1 in ?? ()
#8  0x00000000006a899c in ?? ()
#9  0x00000000006b9943 in ?? ()
#10 0x00000000006b93e3 in ?? ()
#11 0x00000000006b921e in ?? ()
#12 0x00000000006b86f1 in Py_RunMain ()
#13 0x00000000006838eb in Py_BytesMain ()
#14 0x00007fc1cb513ca8 in cancel_handler () from target:<path>
#15 0x00007fc1cb513d65 in __libc_start_main () from target:<path>
#16 0x0000000000682c81 in _start ()
Published session

Prompt

## Plan Pass

You are planning a fixer patch before any edits happen.

Read the evidence bundle at `./evidence.json`. The prepared workspace is `./workspace` and it was acquired via `debian-source`. The original pre-edit snapshot is available at `./source` if you need to inspect it. For interpreter processes, plan from the script/application entrypoint evidence first and include the runtime only as a second investigation target unless the evidence proves a runtime bug.

Validation expectation: try the project-level build/test entrypoint from the workspace root before reporting only a focused leaf compile. Detected candidate(s): `./configure && make`. If the project-level command fails because dependencies or generated files are missing, include the exact command and failure reason in `## Validation`, then run the narrowest relevant compile/test that is still reproducible from a clean checkout.

Upstream-style expectation: before planning or editing, check for contribution/style docs (`CONTRIBUTING`, `HACKING`, `README-hacking`, `README.md`, `docs/`, `dev-docs/`) and scan the touched subsystem for local helpers. If the project has wrappers for file IO, path-relative IO, process spawning, memory allocation, logging, locking, or platform compatibility, prefer those wrappers over generic libc/std APIs. For public upstream review, prepare the final patch against the upstream default integration branch, usually `master` or `main`; use installed or stable source trees for reproduction/backport validation, not as the default merge-request base unless the contribution docs explicitly say to target a stable branch. Do not invent a reproducer or user-visible failure that is not in the evidence bundle; if the evidence is profiler-only or indirect, describe it as a targeted mitigation or stop with a diagnosis instead of presenting a speculative patch as a confirmed bug fix. In the plan and final validation, name any such helper, convention, branch-base choice, or evidence limit you found, or say that no relevant local helper was found. Treat this as a `python3.13` upstream patch, not just a Debian-local workaround.

Codex subagent/fan-out expectation: for any non-trivial investigation, do not keep all checking in the lead context. Proactively spawn bounded leaf subagents for independent source inspection, duplicate/upstream search, implementation sanity checks, and verification while the lead owns integration. Use the newest available mini model for read-only scouts, and use `gpt-5.3-codex-spark` aggressively for tiny coding, test, and verification leaves even when the lead model is `gpt-5.5`. Pass model names explicitly when spawning. Keep subagents as leaves: tell them not to spawn their own agents, not to run recursive `codex review`, not to perform public/external/destructive actions, and not to duplicate the same live-state operation. Merge their evidence into the final patch/report instead of merely saying a subagent looked. Inspect the relevant code, nearby callers, project contribution docs, and local helper/compat APIs, but do not edit files in this pass.

Return a short markdown plan with these exact sections:

## Problem
## Evidence Confidence
## Proposed Subject
## Patch Plan
## Risks
## Validation

Classify `## Evidence Confidence` as exactly one of `reproduced`, `observed`, or `inferred`. Use `inferred` only for a no-patch diagnosis/report plan unless you can name the extra evidence you will collect before editing; inferred source patches are blocked by Fixer because they are not pull-request-ready. For `observed` source-patch plans, plan to say in the final `## Issue Connection` that the failure was observed by Fixer and not independently reproduced. If the plan touches authentication, authorization, credentials, cryptography, sandboxing, permissions, timing/throttling behavior, or PostgreSQL database-core behavior such as dynamic library resolution, extension loading, transaction semantics, locking, planner/executor behavior, catalog changes, or storage changes, treat it as high-risk: do not plan a source patch unless validation can reproduce the behavior and analyze the semantic impact; otherwise plan a no-patch diagnosis/report. The plan must explain how the proposed code change addresses the observed issue evidence, call out any prior Fixer patch that should be improved or replaced, reject awkward control flow such as avoidable `goto` if there is a cleaner bounded alternative, name any local helper APIs or maintainer conventions the patch should follow, and keep the intended maintainer-facing explanation clear enough that someone unfamiliar with the local complaint wording can still follow the fix. In `## Validation`, name the reproducible configure/build/test entrypoint you will try from the workspace root before any focused leaf compile or smoke check, and include one bounded independent reproduction attempt for the collected failure signal when it is safe and cheap. Do not plan to claim `reproduced` unless that reproduction command or test can actually show the failure.

Response

## Plan Pass



## Workflow Note

model: gpt-5.5

{"type":"thread.started","thread_id":"01a0bc74-f5fb-7350-beb1-d676bca9ada7"}
{"type":"turn.started"}
{"type":"error","message":"exceeded retry limit, last status: 429 Too Many Requests, request id: f08416a2-2609-4bd5-89ce-3d72b8637485"}
{"type":"turn.failed","error":{"message":"exceeded retry limit, last status: 429 Too Many Requests, request id: f08416a2-2609-4bd5-89ce-3d72b8637485"}}

ready triage handoff

triage

python3.13 likely remains stuck in an unclassified userspace loop. A diagnosis report and external handoff were created locally.

state: readycreated: 2026-09-20 01:36 UTCvalidation: ready

Why it stopped

workspace-acquisition

Handoff

Likely owner: python script ../../utils/docs/postgis_exampletest.py

Reason: workspace-acquisition

  • Find the application, module repository, Home Assistant add-on, container image, or local checkout that provides python script ../../utils/docs/postgis_exampletest.py.
  • Attach that source tree before asking Fixer for a patch; only patch the interpreter/runtime after a language-level stack or minimal reproducer proves the runtime is at fault.
  • Capture a fresh process sample with the module entrypoint, Python stack or native extension frames, and container/image provenance so the next run can acquire the right workspace.

failed patch attempt

patch

python3.13 likely remains stuck in an unclassified userspace loop. The diagnosis was captured, but the patch proposal did not complete cleanly.

state: failedcreated: 2026-03-29 14:39 UTCvalidation: failed

Why it stopped

patch proposal did not complete cleanly

Collected context

  • This older failed attempt predates structured patch failure capture, so Fixer only retained the diagnosis and workspace context.
  • Subsystem: runaway-process
  • Diagnosis family: unknown-userspace-loop
  • Hot path: __strcmp_avx2
  • Source package: glibc
  • Workspace source: debian-source
  • Workspace acquisition: Fetched Debian source package via apt-get source.
  • Homepage: https://www.gnu.org/software/libc/libc.html

The process is demonstrably CPU-hot, but the current syscall and symbol sample does not point to a single dominant loop family yet.