pioduino/uv
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Fix link-mode=clone on linux (#7620)

Browse Source 
- **Do not attempt to reflink directories on linux**
- **Refactor clone_recursive**

## Summary

On linux, reflink does not work on a directory. Currently, we first
attempt to reflink directory, and only if it fails with `AlreadyExists`
we attempt to reflink recursively.

This has the effect that, on linux, `uv pip install --link-mode=clone`
would always fall back to `copy`.

We resolve this by only attempting to reflink directories on macos. In
the process, we refactored `clone_recursive` in an attempt to make it
easier to reason about its logic.


## Test Plan

I tested that after this change, `uv pip install --link-mode=clone
numpy` would behave as expected in the following cases:

* linux, btrfs filesystem, venv on the same filesystem as cache
(correctly reflinked)
* linux, btrfs filesystem, venv on a different filesystem than cache
(fallback to copy)

I have not tested it on macos or windows, as I currently don't have
access to any macos or windows machines, unfortunately.
This commit is contained in:
Abdó Roig-Maranges
2024-09-22 19:40:52 +02:00
committed by GitHub
parent 5d328a4550
commit 542afe7474
1 changed files with 9 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/install-wheel-rs/src/linker.rs
+9 -2
View File
@@ -296,10 +296,17 @@ fn clone_wheel_files(
let mut count = 0usize;
let mut attempt = Attempt::default();
// On macOS, directly can be recursively copied with a single `clonefile` call.
// On macOS, directories can be recursively copied with a single `clonefile` call.
// So we only need to iterate over the top-level of the directory, and copy each file or
// subdirectory unless the subdirectory exists already in which case we'll need to recursively
// merge its contents with the existing directory.
//
// On linux, we need to always reflink recursively, as `FICLONE` ioctl does not support directories.
// Also note, that reflink is only supported on certain filesystems (btrfs, xfs, ...), and only when
// it does not cross filesystem boundaries.
//
// On windows, we also always need to reflink recursively, as `FSCTL_DUPLICATE_EXTENTS_TO_FILE` ioctl
// is not supported on directories. Also, it is only supported on certain filesystems (ReFS, SMB, ...).
for entry in fs::read_dir(wheel.as_ref())? {
clone_recursive(
site_packages.as_ref(),
@@ -365,7 +372,7 @@ fn clone_recursive(
debug!("Cloning {} to {}", from.display(), to.display());
if cfg!(windows) && from.is_dir() {
if (cfg!(windows) || cfg!(target_os = "linux")) && from.is_dir() {
// On Windows, reflinking directories is not supported, so we copy each file instead.
fs::create_dir_all(&to)?;
for entry in fs::read_dir(from)? {