python3.13 likely remains stuck in a file-not-found retry loop. A diagnosis report and patch proposal were created locally.
Published session
Prompt
## Plan Pass
You are planning a fixer patch before any edits happen.
Read the evidence bundle at `./evidence.json`. The prepared workspace is `./workspace` and it was acquired via `debian-source`. The original pre-edit snapshot is available at `./source` if you need to inspect it. For interpreter processes, plan from the script/application entrypoint evidence first and include the runtime only as a second investigation target unless the evidence proves a runtime bug.
Upstream-style expectation: before planning or editing, check for contribution/style docs (`CONTRIBUTING`, `HACKING`, `README-hacking`, `README.md`, `docs/`, `dev-docs/`) and scan the touched subsystem for local helpers. If the project has wrappers for file IO, path-relative IO, process spawning, memory allocation, logging, locking, or platform compatibility, prefer those wrappers over generic libc/std APIs. Do not invent a reproducer or user-visible failure that is not in the evidence bundle; if the evidence is profiler-only or indirect, describe it as a targeted mitigation or stop with a diagnosis instead of presenting a speculative patch as a confirmed bug fix. In the plan and final validation, name any such helper, convention, or evidence limit you found, or say that no relevant local helper was found. Treat this as a `unattended-upgrades` upstream patch, not just a Debian-local workaround. Inspect the relevant code, nearby callers, project contribution docs, and local helper/compat APIs, but do not edit files in this pass.
Return a short markdown plan with these exact sections:
## Problem
## Evidence Confidence
## Proposed Subject
## Patch Plan
## Risks
## Validation
Classify `## Evidence Confidence` as exactly one of `reproduced`, `observed`, or `inferred`. Use `inferred` only for a no-patch diagnosis/report plan unless you can name the extra evidence you will collect before editing; inferred source patches are blocked by Fixer because they are not pull-request-ready. For `observed` source-patch plans, plan to say in the final `## Issue Connection` that the failure was observed by Fixer and not independently reproduced. The plan must explain how the proposed code change addresses the observed issue evidence, call out any prior Fixer patch that should be improved or replaced, reject awkward control flow such as avoidable `goto` if there is a cleaner bounded alternative, name any local helper APIs or maintainer conventions the patch should follow, and keep the intended maintainer-facing explanation clear enough that someone unfamiliar with the local complaint wording can still follow the fix. In `## Validation`, name the reproducible configure/build/test entrypoint you will try from the workspace root before any focused leaf compile or smoke check, and include one bounded independent reproduction attempt for the collected failure signal when it is safe and cheap. Do not plan to claim `reproduced` unless that reproduction command or test can actually show the failure.
## Patch Pass
You are working on a bounded fixer proposal.
Read the evidence bundle at `./evidence.json`. The prepared workspace is `./workspace` and it was acquired via `debian-source`. Produce the smallest reasonable patch for the target repository, keep the change upstreamable, prefer the clearest control flow available, and do not keep avoidable `goto` when a simpler structure would read better. Before introducing new file, process, allocation, locking, networking, or platform APIs, inspect nearby code and project contribution docs for existing helpers or compatibility wrappers and use those local patterns unless you can explain why they do not fit. Validate from a reproducible workspace-root entrypoint before falling back to focused leaf commands; if a build or test cannot run, report the exact command, the exact blocker, and any narrower check you ran instead. During validation, also try one bounded independent reproduction of the collected failure signal when it is safe and cheap, such as a failing test, smoke command, perf/strace comparison, or before/after runtime check. Only use `reproduced` if that command or test actually reproduced the failure; otherwise keep `observed` and report the reproduction blocker. The final explanation must connect the observed issue evidence to the actual code change, not just paraphrase the diff. Write like a maintainer is going to read the patch mail cold: explain the bug in plain language, define subsystem-specific jargon the first time you need it, and make the causal story obvious. Explicitly classify evidence confidence as `reproduced`, `observed`, or `inferred`: `reproduced` means you reproduced the failure locally; `observed` means Fixer has direct crash/log/trace evidence but you did not independently reproduce it; `inferred` means the source patch is not pull-request-ready, so do not leave a source diff unless you first gather stronger observed/reproduced evidence; otherwise return a no-patch diagnosis/report. For any source-changing `observed` patch, say explicitly in `## Issue Connection` that the failure was observed by Fixer and not independently reproduced. If you introduce non-obvious state translation, index remapping, or backend split logic, add a short source comment that explains the invariant being preserved.
Start by explaining the likely root cause from the collected perf, strace, and /proc evidence. If you cannot land a safe patch, leave a diagnosis that is strong enough for an upstream bug report.
Interpreter process expectation: when the hot process is Python, Perl, shell, Node, Ruby, PHP, Lua, or another interpreter, inspect both the runtime and the script/application entrypoint from `details.interpreter_process`. Prefer the script or application package when evidence points there; do not patch the interpreter runtime merely because the hot binary is `python`, `perl`, `bash`, or similar. Runtime fixes are welcome only after checking the entrypoint and explaining why the runtime mishandles the workload or why running bad code faster/safer is itself the right upstream fix.
Upstream-style expectation: before planning or editing, check for contribution/style docs (`CONTRIBUTING`, `HACKING`, `README-hacking`, `README.md`, `docs/`, `dev-docs/`) and scan the touched subsystem for local helpers. If the project has wrappers for file IO, path-relative IO, process spawning, memory allocation, logging, locking, or platform compatibility, prefer those wrappers over generic libc/std APIs. Do not invent a reproducer or user-visible failure that is not in the evidence bundle; if the evidence is profiler-only or indirect, describe it as a targeted mitigation or stop with a diagnosis instead of presenting a speculative patch as a confirmed bug fix. In the plan and final validation, name any such helper, convention, or evidence limit you found, or say that no relevant local helper was found. Treat this as a `unattended-upgrades` upstream patch, not just a Debian-local workaround.
Keep the change narrowly scoped and summarize validation clearly.
In every authoring pass, your final response must start with `Subject: <single-line git commit subject>` and then include these markdown sections exactly:
## Commit Message
A short upstream-friendly explanation of what changed and why. Write it in plain language that a maintainer can follow without local complaint context. If you use subsystem jargon, define it immediately.
## Evidence Confidence
Exactly one word: `reproduced`, `observed`, or `inferred`. Use `reproduced` only when you reproduced the failure locally with a command or test, and include that command/test in `## Validation`. Use `observed` when Fixer has direct crash/log/trace evidence but you did not independently reproduce it. If `## Git Add Paths` lists source files for an `observed` patch, `## Issue Connection` must explicitly say the failure was observed by Fixer and not independently reproduced. Use `inferred` for profiler/strace/indirect evidence; inferred responses may be no-patch diagnoses or reports, but inferred source patches are not pull-request-ready until stronger evidence is gathered.
## Issue Connection
Write this as maintainer-facing patch mail, not as local Fixer notes. Cover four things explicitly in readable sentences: the user-visible symptom or the exact collected signal, the code-level cause or the cautious inference from evidence, the specific change you made, and the expected effect. Do not invent a reproducer, command line, crash, or user-visible failure that is not present in the evidence bundle. If the evidence is direct-but-not-reproduced, say it was observed by Fixer and not independently reproduced. If the evidence is indirect and you did not gather stronger evidence, do not leave a source diff; write a no-patch diagnosis/report instead. Include an explicit effect sentence such as `The expected effect is ...`, `This should reduce ...`, or `This prevents ...` for source patches. If the logic is non-obvious in code, mention that you added a short explanatory comment.
## Git Add Paths
List the repo-relative paths that belong in the final patch, one per line. Use `None` only when you intentionally made no source changes. Include intentionally new files, and do not list generated build artifacts.
## Validation
List the checks you ran, or say clearly that you could not run them. Include the independent reproduction command/test and result when `## Evidence Confidence` is `reproduced`; if reproduction was attempted but blocked, name the exact blocker and keep confidence at `observed` or `inferred`.
Before editing, read the plan at `./plan-output.txt` and follow it unless the code proves part of it wrong. If you change course, say so explicitly in the final write-up instead of silently drifting from the plan.
## Review Pass 1
You are reviewing a freshly generated fixer patch.
Read the evidence bundle at `./evidence.json`. The prepared workspace is `./workspace` and it was acquired via `debian-source`. Review the first patch pass. The original pre-edit snapshot is available at `./source` for diffing. The workspace currently changes these repo-relative paths: test/test_dependencies.py, unattended-upgrade. Verify that `## Git Add Paths` matches this exact set and that `## Issue Connection` explains every shipped functional file.
Upstream-style expectation: before planning or editing, check for contribution/style docs (`CONTRIBUTING`, `HACKING`, `README-hacking`, `README.md`, `docs/`, `dev-docs/`) and scan the touched subsystem for local helpers. If the project has wrappers for file IO, path-relative IO, process spawning, memory allocation, logging, locking, or platform compatibility, prefer those wrappers over generic libc/std APIs. Do not invent a reproducer or user-visible failure that is not in the evidence bundle; if the evidence is profiler-only or indirect, describe it as a targeted mitigation or stop with a diagnosis instead of presenting a speculative patch as a confirmed bug fix. In the plan and final validation, name any such helper, convention, or evidence limit you found, or say that no relevant local helper was found. Treat this as a `unattended-upgrades` upstream patch, not just a Debian-local workaround. The latest author response is at `./patch-output.txt`. Inspect the current code and changed paths like a strict code reviewer. Focus on correctness, regressions, maintainability, awkward control flow such as avoidable `goto`, missing validation, weak or non-gittable commit message text, and explanations that fail to connect the observed issue evidence to the code change. Also review the maintainer experience: the patch mail should be easy to accept upstream, the user-visible bug should be explained in plain language, subsystem-specific jargon should be defined when first used, and any non-obvious state translation or index remapping in code should have a short explanatory comment. Reject patches that omit `## Evidence Confidence`, use anything other than `reproduced`, `observed`, or `inferred`, or sound more certain than that evidence level allows. Reject patches that claim `reproduced` unless `## Validation` names the concrete reproduction command or test and its result. Reject patches that invent a reproducer, command line, crash, or user-visible failure not present in the evidence bundle; indirect profiler or strace evidence may justify a cautious mitigation, but the author must say it is indirect instead of presenting it as a confirmed upstream bug. Reject patches that introduce generic libc/std APIs when nearby code or project docs provide a local compat/helper API for the same job, unless the author explicitly justifies the exception. Reject validation that only reports a leaf object/syntax build when a project-level configure/build/test entrypoint exists and was not attempted; if a full build cannot run, the author must show the exact failed command and blocker.
Do not apply code changes in this pass.
Return a short markdown review report. The first non-empty line must be exactly one of:
RESULT: ok
RESULT: fix-needed
If you choose `RESULT: fix-needed`, add a `## Findings` section with concrete, actionable items.
Response
## Plan Pass
## Problem
Fixer observed `/usr/bin/unattended-upgrade` running hot under Python, with native samples in `apt_pkg`/`libapt-pkg` around `pkgSourceList::FindIndex()` and repeated `newfstatat()` calls for APT list files, including missing i386 package indexes. The application entrypoint is implicated before the Python runtime.
A likely upstream-side contributor is `transitive_dependencies(pkg, cache, acc=set(), ...)` in `unattended-upgrade`: the mutable default set is shared across calls. During package marking, that can leak dependency names from earlier packages into later calls, causing unnecessary `cache[dep]`, candidate adjustment, and `get_changes()` work in `UnattendedUpgradesCache.call_adjusted()`.
## Evidence Confidence
observed
## Proposed Subject
Avoid leaking dependency state between unattended-upgrades package-marking passes
## Patch Plan
- Follow project conventions from `HACKING.md`: keep the Python change small and add/adjust unit coverage; primary test entrypoint is `PYTHONPATH=. python3 -m unittest`.
- Change `transitive_dependencies()` so `acc` defaults to `None`, initializes a fresh `set()` per top-level call, and preserves the existing recursive accumulator behavior when an explicit set is passed.
- Add a focused unit test showing that two independent `transitive_dependencies()` calls do not share dependency names from a prior call.
- Keep the existing local helper flow intact: `UnattendedUpgradesCache.adjust_candidate()`, `call_adjusted()`, `check_changes_for_sanity()`, `rewind_cache()`, `is_in_allowed_origin()`, and `ver_in_allowed_origin()` remain the package policy/marking helpers to use.
- Do not replace the recursive dependency walk with awkward control flow. If any further bounding is needed, use the existing `level` parameter and a clean local accumulator rather than introducing avoidable jumps or broad restructuring.
- No prior Fixer patch is present in the workspace (`git diff` is empty), so there is no generated patch to replace.
## Risks
- The patch mitigates excess repeated APT work but does not prove that every observed `newfstatat()` retry comes from the mutable-default leak; final notes should state that the issue was observed by Fixer and not independently reproduced unless validation captures the signal.
- Some tests may rely unintentionally on the leaked accumulator state, but that would be a bug in the test expectation rather than desired behavior.
- This does not patch `apt_pkg`, `libapt-pkg`, or Python runtime behavior, because the evidence identifies a script entrypoint and does not prove a runtime bug.
## Validation
- Run the project’s reproducible test entrypoint from the workspace root: `PYTHONPATH=. python3 -m unittest`.
- Run focused tests around the changed behavior, including the new dependency accumulator test and existing package selection tests such as `PYTHONPATH=. python3 -m unittest test.test_rewind test.test_unavailable_candidate`.
- Bounded independent reproduction attempt: before/after the edit, run a small `PYTHONPATH=.` Python snippet or unit test that calls `transitive_dependencies()` twice with different mocked packages and confirms the second result does not include dependencies from the first. This can show the accumulator leak directly, but it will not by itself reproduce the full 99% CPU profiler signal.
## Patch Pass
Subject: Reset dependency walks between package marking passes
## Commit Message
`transitive_dependencies()` used a mutable default `set`, so independent dependency walks could share package names from previous calls. Use a fresh set for each top-level walk and add a unit test to keep package-marking dependency state isolated.
## Evidence Confidence
observed
## Issue Connection
Fixer observed `/usr/bin/unattended-upgrade` consuming CPU, with perf/backtrace samples in `apt_pkg`/`libapt-pkg` resolving APT index files and `strace` showing repeated `newfstatat()` probes, including missing APT list files. This failure was observed by Fixer and not independently reproduced as the full high-CPU/newfstatat loop.
The relevant script-side code repeatedly asks APT about package candidates while marking upgrades. In that path, `transitive_dependencies()` used `acc=set()` as a default argument; in Python that set is created once and reused, so dependency names from one package can leak into later dependency walks. That can make later package-marking passes inspect and adjust unrelated stale dependencies, increasing calls into APT metadata lookup.
The patch changes `transitive_dependencies()` to create a fresh accumulator when no caller supplies one, while preserving the explicit accumulator used by recursion. I followed the project’s `HACKING.md` test convention and the existing package-marking helpers; no relevant local file/process/platform wrapper was needed for this pure-Python state fix.
The expected effect is to prevent unrelated dependency walks from accumulating stale package names and reduce unnecessary APT candidate/index lookups during unattended-upgrades package marking.
## Git Add Paths
unattended-upgrade
test/test_dependencies.py
## Validation
Ran `PYTHONPATH=. python3 -m unittest` from the workspace root. It did not complete cleanly because the environment lacks `flake8` (`FileNotFoundError: [Errno 2] No such file or directory: 'flake8'`) and sandboxed APT integration tests hit privilege errors (`setgroups 65534 failed`, `setegid 65534 failed`, `seteuid 104 failed`).
Ran focused checks successfully:
`PYTHONPATH=. python3 -m unittest test.test_dependencies test.test_rewind test.test_unavailable_candidate`
Result: `Ran 3 tests ... OK`.
Bounded reproduction check: running a small `transitive_dependencies()` mock script against the original pre-edit snapshot showed the second call returning both `first-dependency` and `second-dependency`, demonstrating the shared accumulator bug. Running the same check in the patched workspace returned separate sets. I did not reproduce the full Fixer CPU/perf signal locally, so confidence remains `observed`.
## Review Pass 1
RESULT: ok
No blocking findings. The changed paths match `## Git Add Paths`, the evidence confidence is valid and appropriately cautious, and the issue connection ties the observed APT index lookup hot path to the dependency accumulator mitigation without overstating reproduction. Focused reviewer check passed: `PYTHONPATH=. python3 -m unittest test.test_dependencies`.
Diff
diff --git a/test/test_dependencies.py b/test/test_dependencies.py
new file mode 100644
index 0000000..9eae5c5
--- /dev/null
+++ b/test/test_dependencies.py
@@ -0,0 +1,33 @@
+#!/usr/bin/python3
+# -*- coding: utf-8 -*-
+
+import unittest
+from unittest.mock import Mock
+
+from unattended_upgrade import transitive_dependencies
+
+
+class TestDependencies(unittest.TestCase):
+
+ def _get_pkg_with_deps(self, *dep_names):
+ pkg = Mock()
+ pkg.candidate = Mock()
+ pkg.candidate.dependencies = [
+ [Mock(name=name, rawtype="Depends")] for name in dep_names]
+ for dep, dep_name in zip(pkg.candidate.dependencies, dep_names):
+ dep[0].name = dep_name
+ return pkg
+
+ def test_transitive_dependencies_keeps_independent_calls_separate(self):
+ cache = {}
+ first_pkg = self._get_pkg_with_deps("first-dependency")
+ second_pkg = self._get_pkg_with_deps("second-dependency")
+
+ self.assertEqual(
+ {"first-dependency"}, transitive_dependencies(first_pkg, cache))
+ self.assertEqual(
+ {"second-dependency"}, transitive_dependencies(second_pkg, cache))
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/unattended-upgrade b/unattended-upgrade
index 053b5f5..ce3a533 100755
--- a/unattended-upgrade
+++ b/unattended-upgrade
@@ -1265,7 +1265,7 @@ def is_pkg_change_allowed(pkg, blacklist, whitelist, strict_whitelist):
def transitive_dependencies(pkg, # type: apt.Package
cache, # type: apt.Cache
- acc=set(), # type AbstractSet[str]
+ acc=None, # type: Optional[Set[str]]
valid_types=None, # type: Optional[AbstractSet[str]]
level=None # type: Optional[int]
):
@@ -1274,6 +1274,9 @@ def transitive_dependencies(pkg, # type: apt.Package
Note that alternative (|) dependencies are collected, too
"""
+ if acc is None:
+ acc = set()
+
if not pkg.candidate or level is not None and level < 1:
return acc