Commit Graph

3136 Commits

Author SHA1 Message Date
renovate[bot] 3b488d8635 Update Rust crate cargo-util to v0.2.13 (#5536) 2024-07-28 20:30:28 -04:00
renovate[bot] f8f82b354d Update Rust crate assert_fs to v1.1.2 (#5535) 2024-07-28 20:30:21 -04:00
renovate[bot] 9164d5cf97 Update Rust crate anstream to v0.6.15 (#5533) 2024-07-28 20:30:14 -04:00
Pavel Dikov cb47aed9de feat(venv): add relocatable flag (#5515)
## Summary

Adds a `--relocatable` CLI arg to `uv venv`. This flag does two things:

* ensures that the associated activation scripts do not rely on a
hardcoded
absolute path to the virtual environment (to the extent possible; `.csh`
and
  `.nu` left as-is)
* persists a `relocatable` flag in `pyvenv.cfg`.

The flag in `pyvenv.cfg` in turn instructs the wheel `Installer` to
create script
entrypoints in a relocatable way (use `exec` trick + `dirname $0` on
POSIX;
use relative path to `python[w].exe` on Windows).

Fixes: #3863

## Test Plan

* Relocatable console scripts covered as additional scenarios in
existing test cases.
* Integration testing of boilerplate generation in `venv`.
* Manual testing of `uv venv` with and without `--relocatable`
2024-07-29 00:10:11 +00:00
Charlie Marsh 3626d08cca Enable benchmarking of uv tool and pipx (#5531)
## Summary

Closes https://github.com/astral-sh/uv/issues/5263.
2024-07-28 23:27:14 +00:00
Charlie Marsh 44a77a04d0 Move bench directory to benchmark (#5529)
## Summary

Removes the legacy `benchmark` directory (we'll always have it in Git)
and renames `bench` to `benchmark` for clarity. Fixes a variety of
commands and references.
2024-07-28 22:03:52 +00:00
Charlie Marsh d7c79182ea Extend Ruff configuration to sort imports (#5528) 2024-07-28 21:49:28 +00:00
Charlie Marsh 9732922929 Migrate benchmark setup to uv (#5526) 2024-07-28 21:37:57 +00:00
Charlie Marsh 2d5c166642 Compare simplified paths in Windows exclusion tests (#5525)
## Summary

Closes https://github.com/astral-sh/uv/issues/5521.
2024-07-28 17:26:46 -04:00
Charlie Marsh efbc9fb78d Add support for benchmarking uv sync and uv lock (#5524)
## Summary

This PR adds support for `uv lock` and `uv sync` in the standardized
benchmarks script.

Part of: https://github.com/astral-sh/uv/issues/5263.

## Test Plan

For example:

```sh
python scripts/bench/__main__.py --uv-project --benchmark resolve-cold ./scripts/requirements/trio.in --verbose
```
2024-07-28 21:09:08 +00:00
Mark Ture 12e92b7718 Fix typo in projects.md (#5523) 2024-07-28 21:05:26 +01:00
Charlie Marsh 83412837e5 Warn, but don't error, when encountering tilde .dist-info directories (#5520)
## Summary

Closes https://github.com/astral-sh/uv/issues/3668.
2024-07-28 19:13:06 +00:00
Charlie Marsh b0c841ee3b Ban --no-cache with --link-mode=symlink (#5519)
## Summary

Closes https://github.com/astral-sh/uv/issues/5360.
2024-07-28 15:01:17 -04:00
Krishnan Chandra 4b4128446d Support xz compressed packages (#5513)
## Summary

Closes #2187.

The [xz
backdoor](https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27)
is still fairly recent, but luckily the [Rust `xz2` crate bundles
version 5.2.5 of the C `xz`
package](https://github.com/alexcrichton/xz2-rs/tree/main/lzma-sys),
which is before the backdoor was introduced.

It's worth noting that a security risk still exists if you have a
compromised version of `xz` installed on your system, but that risk is
not introduced by `uv` or the Rust packages in general.

## Test Plan

Tried installing the package mentioned in the linked issue: `python-apt
@
https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.7.6/python-apt_2.7.6.tar.xz`

(Note that this will only work on Ubuntu - I tried on a Mac and while
the archive was extracted properly, the package did not install because
of some missing files)

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-07-28 18:37:48 +00:00
Charlie Marsh caf01735fa Avoid warning users for missing self-extra lower bounds (#5518)
## Summary

Closes https://github.com/astral-sh/uv/issues/5227.
2024-07-28 18:35:18 +00:00
Charlie Marsh 88340fbd0d Remove some unused methods (#5512) 2024-07-28 17:20:12 +00:00
Charlie Marsh 1734c7ed50 Use existing METADATA parser in wheel installer (#5508) 2024-07-27 14:38:16 -04:00
Jo ae11317cc0 Make pip list --editable conflicts with --exlcude-editable (#5506)
## Summary

I think it makes no sense to allow `--editable` and `--exclude-editable`
at the same time.

## Test Plan
```console
$ cargo run -- pip list --editable --exclude-editable
error: the argument '--editable' cannot be used with '--exclude-editable'

Usage: uv.exe pip list --editable

For more information, try '--help'.

```
2024-07-27 08:26:46 -04:00
Charlie Marsh 866d844977 Allow uv pip install for unmanaged projects (#5504)
## Summary

Just an oversight.

Closes: https://github.com/astral-sh/uv/issues/5500.
2024-07-27 02:10:18 +00:00
Charlie Marsh 24859bd3ee Upgrade to Rust 1.80.0 (#5472) 2024-07-27 01:49:47 +00:00
Charlie Marsh 3ea5e16e96 Respect reinstalls in cached environments (#5499)
## Summary

Closes #5493.
2024-07-27 01:36:58 +00:00
Charlie Marsh 8f16f1b746 Remove serialize traits from verbatim URL (#5501) 2024-07-27 00:51:24 +00:00
Charlie Marsh 4f3dde34dc Use 666 rather than 644 for default permissions (#5498)
## Summary

I don't know why I used 644 here. 666 is the actual default:
https://github.com/rust-lang/rust/blob/7c2012d0ec3aae89fefc40e5d6b317a0949cda36/library/std/src/sys/pal/unix/fs.rs#L1069

Closes https://github.com/astral-sh/uv/issues/5496.
2024-07-27 00:08:52 +00:00
Charlie Marsh 623ba3885f Add some missing reinstall-refresh calls (#5497)
## Summary

Turns out I missed a few of these.
2024-07-27 00:01:40 +00:00
Charlie Marsh 561625ed8c Use hasher to compute resolution hash (#5495)
## Summary

Addressing one TODO. This should be more efficient.
2024-07-26 23:24:09 +00:00
Ahmed Ilyas e8d7c0cb58 Editable installs for uv tool (#5454)
## Summary

Resolves #5436. 

## Test Plan

`cargo test` 

```console
❯ ./target/debug/uv tool install -e ~/black
warning: `uv tool install` is experimental and may change without warning
Resolved 6 packages in 894ms
   Built black @ file:///Users/ahmedilyas/black
Prepared 1 package in 468ms
Installed 6 packages in 6ms
 + black==24.4.3.dev23+g7e2afc9 (from file:///Users/ahmedilyas/black)
 + click==8.1.7
 + mypy-extensions==1.0.0
 + packaging==24.1
 + pathspec==0.12.1
 + platformdirs==4.2.2
Installed 2 executables: black, blackd
```

venv has the `.pth` files.
```console
❯ eza /Users/ahmedilyas/Library/Application\ Support/uv/tools/black/lib/python3.12/site-packages/
_black.pth       _virtualenv.py                         click                  mypy_extensions-1.0.0.dist-info  packaging                 pathspec                   platformdirs
_virtualenv.pth  black-24.4.3.dev23+g7e2afc9.dist-info  click-8.1.7.dist-info  mypy_extensions.py               packaging-24.1.dist-info  pathspec-0.12.1.dist-info  platformdirs-4.2.2.dist-info
```

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-07-26 16:30:15 -04:00
Andrew Gallant 2186e967f6 uv-resolver: fix basic case of overlapping markers
Consider the following packse scenario:

```toml
[root]
requires = [
  "a>=1.0.0 ; python_version < '3.10'",
  "a>=1.1.0 ; python_version >= '3.10'",
  "a>=1.2.0 ; python_version >= '3.11'",
]

[packages.a.versions."1.0.0"]
[packages.a.versions."1.1.0"]
[packages.a.versions."1.2.0"]
```

On current `main`, this produces a dependency on `a` that looks like
this:

```toml
dependencies = [
    { name = "fork-overlapping-markers-basic-a", marker = "python_version < '3.10' or python_version >= '3.11'" },
]
```

But the marker expression is clearly wrong here, since it implies that
`a` isn't installed at all for Python 3.10. With this PR, the above
dependency becomes:

```toml
dependencies = [
    { name = "fork-overlapping-markers-basic-a" },
]
```

That is, it's unconditional. Which is I believe correct here since there
aren't any other constraints on which version to select.

The specific bug here is that when we found overlapping dependency
specifications for the same package *within* a pre-existing fork, we
intersected all of their marker expressions instead of unioning them.
That in turn resulted in incorrect marker expressions.

While this doesn't fix any known bug on the issue tracker (like #4640),
it does appear to fix a couple of our snapshot tests. And fixes a basic
test case I came up with while working on #4732.

For the packse scenario test: https://github.com/astral-sh/packse/pull/206
2024-07-26 12:06:37 -07:00
Charlie Marsh 6901a14aa0 Bump version to v0.2.30 (#5486) 2024-07-26 13:36:05 -04:00
konsti f1eda3590b Update to packse 0.3.31 (#5441)
Update packse to 0.3.31, adding the instability scenarios.
2024-07-26 15:39:29 +00:00
konsti cb505d24f8 Unify resolutions only during graph building (#5479)
With our previous eager union, we were losing the fork markers. We now
carry this information into the resolution graph construction and, in
the next step, can read the markers there.

Part of
https://github.com/astral-sh/uv/issues/5180#issuecomment-2247696198
2024-07-26 16:29:48 +02:00
Andrew Gallant 77b005244d uv-resolver: propagate markers to sibling dependencies in forks
When a fork occurs, we divide not just the dependencies that
provoked a fork into distinct groups, but we also add the
corresponding sibling dependencies to each fork. Previously,
while we track markers on the fork itself, the individual
dependencies that had markers only corresponded to markers
written from the dependency specification.

This meant that the sibling dependencies that got added to
each fork would not themselves have markers attached to them.
This in turn meant they would not have markers associated with
them in the lock file.

In many cases, this is actually okay, because the resolver will
pick a version that is "universal" across all forks in most
cases. But in some cases, this just simply isn't possible as
the marker expressions in the fork can and do influence resolution.
In which case, it is possible for the same package with different
versions to show up in the lock file unconditionally. Which is a
big no-no.

So in this commit, after we determine the forks, we intersect the
markers on each fork with each of its dependencies.

This does seem to balloon the marker expressions in some cases.
I plucked one low hanging fruit to avoid doing `x and x` in
trivial cases. (And this eliminated a portion of the snapshot
diffs.) But some pretty gnarly diffs remain.

This commit also fixes another bug: previously, when we created a fork
to capture the "remaining" universe of an incomplete set of markers, we
left out dependencies that should be included in that fork. We rectify
that here.

Fixes #5086

Partially addresses #4732
2024-07-26 07:28:20 -07:00
Andrew Gallant 412780fd99 uv/tests: add regression test from #5086
The snapshot saved here is wrong, but we'll update it in a subsequent
commit.
2024-07-26 07:28:20 -07:00
Andrew Gallant cd1fc7c9a3 uv-normalize: make "name" types implement Default
Interestingly, the empty string appears to be valid for these
types. I'm not sure if that's intended, but having a Default
impl is useful for use with `std::mem::take`.
2024-07-26 07:28:20 -07:00
Andrew Gallant 7fce59e4bc pep508: some simplification in 'and' and 'or'
Basically, and'ing or or'ing the same expression can be entire
skipped. And we try harder to avoid singleton conjunctions or
disjunctions, as these are considered unequal otherwise. (Thus
defeating our attempts to avoid and'ing or or'ing a superfluous
marker.)
2024-07-26 07:28:20 -07:00
Charlie Marsh 8b8f34ac21 Avoid canonicalizing executables on Windows (#5446)
## Summary

If you have an executable path on a network share path (like
`\\some-host\some-share\...\python.exe`), canonicalizing it adds the
`\\?` prefix, but dunce cannot safely strip it.

This PR changes the Windows logic to avoid canonicalizing altogether. We
don't really expect symlinks on Windows, so it seems unimportant to
resolve them.

Closes: https://github.com/astral-sh/uv/issues/5440.
2024-07-26 08:57:33 -04:00
Charlie Marsh 967fcdbb83 Update bundled Pythons to include stripped variants (#5469)
## Summary

The previous update used a slightly botched release.
2024-07-26 00:34:45 +00:00
Charlie Marsh d51b429837 Add --no-config to replace --isolated (#5463)
## Summary

I'll deprecate `--isolated` separately, since it _is_ still used for
some other behaviors.

Closes #5428.
2024-07-25 19:58:36 -04:00
Charlie Marsh 8cb22f5500 Gate FromStr to pep508 feature (#5464)
## Summary

This is triggering when you run without `--all-features` (but we always
use `--all-features` in CI, so it went unnoticed).
2024-07-25 23:54:07 +00:00
Charlie Marsh e4d1039f49 Use sitecustomize.py to implement environment layering (#5462)
## Summary

After consultation with @carljm, we learned that modifying `PYTHONPATH`
is insufficient, because Python won't resolve `.pth` files (editables)
in the base environment. We also saw in
https://github.com/astral-sh/uv/issues/5459 that continuously appending
to `PYTHONPATH` can have some unintended effects.

This PR instead uses a `sitecustomize.py` in the ephemeral environment
to add the base environment's `site-packages`.

Closes https://github.com/astral-sh/uv/issues/5459.
2024-07-25 19:32:44 -04:00
Charlie Marsh 6f45403d31 Infer missing .exe in Windows Python discovery (#5456)
## Summary

Closes https://github.com/astral-sh/uv/issues/5445.
2024-07-25 16:57:38 -04:00
Charlie Marsh 1e965b47c3 Set standard permissions for temporary files (#5457)
## Summary

Closes https://github.com/astral-sh/uv/issues/5435.

## Test Plan

Before:

```
❯ ls -l .venv/lib/python3.12/site-packages/httpx-0.27.0.dist-info
total 48
-rw-------  1 crmarsh  staff     2 Jul 25 14:21 INSTALLER
-rw-r--r--  1 crmarsh  staff  7184 Jul 23 23:20 METADATA
-rw-r--r--  1 crmarsh  staff  2541 Jul 25 14:21 RECORD
-rw-------  1 crmarsh  staff     0 Jul 25 14:21 REQUESTED
-rw-r--r--  1 crmarsh  staff    87 Jul 23 23:20 WHEEL
-rw-r--r--  1 crmarsh  staff    37 Jul 23 23:20 entry_points.txt
drwxr-xr-x  3 crmarsh  staff    96 Jul 25 14:21 licenses
```

After:

```
❯ ls -l .venv/lib/python3.12/site-packages/flask-3.0.3.dist-info/
total 48
-rw-r--r--  1 crmarsh  staff     2 Jul 25 14:21 INSTALLER
-rw-r--r--  1 crmarsh  staff  1475 Jul 25 14:21 LICENSE.txt
-rw-r--r--  1 crmarsh  staff  3177 Jul 25 14:21 METADATA
-rw-r--r--  1 crmarsh  staff  2565 Jul 25 14:21 RECORD
-rw-r--r--  1 crmarsh  staff     0 Jul 25 14:21 REQUESTED
-rw-r--r--  1 crmarsh  staff    81 Jul 25 14:21 WHEEL
-rw-r--r--  1 crmarsh  staff    40 Jul 25 14:21 entry_points.txt
```
2024-07-25 20:50:30 +00:00
Charlie Marsh 75a042d5ff Allow distributions to be absent in deserialization (#5453)
## Summary

Closes https://github.com/astral-sh/uv/issues/5434.
2024-07-25 18:01:41 +00:00
Zanie Blue 6eb8f85668 Update documentation sections (#5452)
Reframes "the low-level interface" as "the pip interface"
Adds indexes to all sections
Renames "commercial indexes" to "alternative indexes"
2024-07-25 12:37:22 -05:00
Charlie Marsh a089e20a35 Use stripped variants by default in Python install (#5451)
## Summary

Will file a separate ticket to support `--debug`.

Closes https://github.com/astral-sh/uv/issues/5447.
2024-07-25 17:29:31 +00:00
Ibraheem Ahmed 4da34a6e2e Reorganize pep508 marker module (#5433)
## Summary

Just moves some code around into separate modules, the current file is a
bit too large to navigate.
2024-07-25 15:22:46 +00:00
Andrew Gallant 7dff3d7dfe uv/tests: updates snapshots for main
It looks like we had a bad merge where the result caused some test
failures. This commit just updates the snapshots to the new reality. I
haven't found the root cause of the bad merge yet.
2024-07-25 07:42:26 -07:00
InSync d17b18ee1e Minor consistency fixes for code blocks (#5437)
...as well as some typo fixes. I verified the changes manually.
2024-07-25 09:23:22 -05:00
Zanie Blue 42e76e2545 Prefer "lockfile" to "lock file" (#5427)
Closes https://github.com/astral-sh/uv/issues/5415
2024-07-25 09:22:36 -05:00
Charlie Marsh d0919329fd Make --reinstall imply --refresh (#5425)
## Summary

It's hard for me to imagine a scenario in which a user passed
`--reinstall`, but wanted us to keep respecting cached data for a
package. For example, to actually "rebuild and reinstall" an editable
today, you have to pass both `--reinstall` and `--refresh`.

This PR makes `--reinstall` imply `--refresh`, so we always validate
that the cached data is fresh.

Closes https://github.com/astral-sh/uv/issues/5424.
2024-07-25 09:45:58 -04:00
Charlie Marsh 4d9098a1d7 Cache metadata for source tree dependencies (#5423)
## Summary

This PR re-introduces caching for source trees. In short, we treat the
metadata as cached unless the `pyproject.toml`, `setup.py`, or
`setup.cfg` file changes. This is a heuristic and not a good one,
especially for extension modules, but without it, we have to rebuild
every project every time (unless you have static metadata, like a
`pyproject.toml` that we can read directly).

Now that we support persistent configuration, users should add:

```toml
[tool.uv]
reinstall = ["foo"]
```

If they want a package to always be refreshed (ignore cache) and
reinstalled (ignore environment).

Closes https://github.com/astral-sh/uv/issues/5420.
2024-07-25 09:45:52 -04:00