Deduplicate all files in the wheel cache by charliermarsh · Pull Request #21327 · astral-sh/uv · GitHub

## Summary

When content hashing is enabled, we currently allocate and zero a new 64
KiB buffer for every file we copy and hash during streaming extraction.
This PR reuses one buffer across the wheel instead. For the PyTorch
wheel used in the benchmarks, that reduces buffer allocations for
hashing from 11,120 to one, while keeping the buffer size at 64 KiB per
active wheel.

The following measurements compare #21327 at
`a188b8e833aef3c3b4b60a32ed9fafe6ac74186a` with this optimization
applied on top, before moving the change onto `main`. They are not
measurements against `main`. The Linux benchmarks alternate base and
candidate, using pinned wheels served over local HTTP with
content-addressed caching enabled:

| Cold install | #21327 | #21327 + buffer reuse | Change |
| --- | ---: | ---: | ---: |
| AnyIO | 110 ms | 107 ms | -2.6% |
| SymPy | 845 ms | 775 ms | -8.3% |
| NumPy | 627 ms | 567 ms | -9.5% |
| PyTorch CPU | 6.50 s | 5.99 s | -7.8% |
| 14-package environment, concurrency 4 | 6.95 s | 6.47 s | -7.0% |

The individual results above use 16 paired rounds; the full environment
uses 12. AnyIO, SymPy, and NumPy were repeated after an initial 20-pair
run: the initial AnyIO timings were noisy, while the initial SymPy and
NumPy improvements were 7.8% and 6.9%. All original samples were
retained. Cached installs and local-wheel controls showed no consistent
change. Across the initial runs, repeats, and controls, we measured 672
installs, excluding warmups and cache priming.

Co-authored-by: Charlie Marsh