Commit Graph

3152 Commits

Author SHA1 Message Date
Charlie Marsh 7ea95239bd Use hatchling rather than implicit setuptools default (#5527)
## Summary

Closes https://github.com/astral-sh/uv/issues/5461.
2024-07-29 14:00:12 +00:00
konsti 0877f76aae Retry on incomplete body (#5555)
This is an attempt to add
https://github.com/astral-sh/uv/issues/3514#issuecomment-2253562096 to
retrying.

Relevant hyper code:
*
https://github.com/hyperium/hyper/blob/15cd6fa1fc52f88b58283a673c134a97bea275d3/src/proto/h1/decode.rs#L683
*
https://github.com/hyperium/hyper/blob/15cd6fa1fc52f88b58283a673c134a97bea275d3/src/proto/h1/decode.rs#L161-L164
2024-07-29 15:53:23 +02:00
renovate[bot] 24472108fe Update pre-commit dependencies (#5546) 2024-07-29 09:44:31 -04:00
renovate[bot] 77696c4ab4 Update Rust crate axoupdater to 0.7.0 (#5547) 2024-07-29 09:44:25 -04:00
renovate[bot] 5365794023 Update hynek/setup-cached-uv action to v2 (#5550) 2024-07-29 09:44:03 -04:00
konsti 2452314f20 Add exclude-newer to benches case priming (#5554)
Uses the same cutoff that the benchmark itself uses
2024-07-29 13:48:38 +02:00
konsti 78be9a6a6b Add trampoline tests to CI (#5331)
Add the tests added in #5204 to CI. The crate is not part of the
workspace (it requires nightly) and is windows only, so we have to test
it separately.

---------

Co-authored-by: samypr100 <3933065+samypr100@users.noreply.github.com>
2024-07-29 11:43:50 +02:00
Charlie Marsh 41c1fc0c4d Generate hashes for --find-links entries (#5544)
## Summary

Closes https://github.com/astral-sh/uv/issues/3874.
2024-07-29 08:49:38 +00:00
Charlie Marsh 51b7e9bff1 Retain editable designation for cached wheel installs (#5545)
## Summary

The package was being installed as editable, but it wasn't marked as
such in `uv pip list`, as the `direct-url.json` was wrong.

Closes https://github.com/astral-sh/uv/issues/5543.
2024-07-28 22:39:48 -04:00
Charlie Marsh 9af0ae2bef Add relocatable installs to support concurrency-safe cached environments (#5509)
## Summary

The idea here is similar to what we do for wheels: we create the
`CachedEnvironment` in the `archive-v0` bucket, then symlink it to its
content-addressed location. This ensures that we can always recreate
these environments without concern for whether anyone else is accessing
them.

Part of the challenge here is that we want the virtual environments to
be relocatable, because we're now building them in one location but
persisting them in another. This requires that we write relative (rather
than absolute) paths to scripts and entrypoints. The main risk with
relocatable virtual environments is that the scripts and entrypoints
_themselves_ are not relocatable, because they use a relative shebang.
But that's fine for cached environments, which are never intended to
leave the cache.

Closes https://github.com/astral-sh/uv/issues/5503.
2024-07-28 20:32:11 -04:00
renovate[bot] 600ef6f18c Update Rust crate assert_cmd to v2.0.15 (#5534) 2024-07-28 20:32:02 -04:00
renovate[bot] c9099b1ed3 Update Rust crate toml to v0.8.16 (#5541) 2024-07-28 20:31:46 -04:00
renovate[bot] 335fe82cfe Update Rust crate tokio to v1.39.2 (#5540) 2024-07-28 20:31:39 -04:00
renovate[bot] 834a1ed515 Update Rust crate serde_json to v1.0.121 (#5539) 2024-07-28 20:31:36 -04:00
renovate[bot] 96c65628b7 Update Rust crate predicates to v3.1.2 (#5538) 2024-07-28 20:31:29 -04:00
renovate[bot] f2860df91f Update Rust crate clap to v4.5.11 (#5537) 2024-07-28 20:30:53 -04:00
renovate[bot] 3b488d8635 Update Rust crate cargo-util to v0.2.13 (#5536) 2024-07-28 20:30:28 -04:00
renovate[bot] f8f82b354d Update Rust crate assert_fs to v1.1.2 (#5535) 2024-07-28 20:30:21 -04:00
renovate[bot] 9164d5cf97 Update Rust crate anstream to v0.6.15 (#5533) 2024-07-28 20:30:14 -04:00
Pavel Dikov cb47aed9de feat(venv): add relocatable flag (#5515)
## Summary

Adds a `--relocatable` CLI arg to `uv venv`. This flag does two things:

* ensures that the associated activation scripts do not rely on a
hardcoded
absolute path to the virtual environment (to the extent possible; `.csh`
and
  `.nu` left as-is)
* persists a `relocatable` flag in `pyvenv.cfg`.

The flag in `pyvenv.cfg` in turn instructs the wheel `Installer` to
create script
entrypoints in a relocatable way (use `exec` trick + `dirname $0` on
POSIX;
use relative path to `python[w].exe` on Windows).

Fixes: #3863

## Test Plan

* Relocatable console scripts covered as additional scenarios in
existing test cases.
* Integration testing of boilerplate generation in `venv`.
* Manual testing of `uv venv` with and without `--relocatable`
2024-07-29 00:10:11 +00:00
Charlie Marsh 3626d08cca Enable benchmarking of uv tool and pipx (#5531)
## Summary

Closes https://github.com/astral-sh/uv/issues/5263.
2024-07-28 23:27:14 +00:00
Charlie Marsh 44a77a04d0 Move bench directory to benchmark (#5529)
## Summary

Removes the legacy `benchmark` directory (we'll always have it in Git)
and renames `bench` to `benchmark` for clarity. Fixes a variety of
commands and references.
2024-07-28 22:03:52 +00:00
Charlie Marsh d7c79182ea Extend Ruff configuration to sort imports (#5528) 2024-07-28 21:49:28 +00:00
Charlie Marsh 9732922929 Migrate benchmark setup to uv (#5526) 2024-07-28 21:37:57 +00:00
Charlie Marsh 2d5c166642 Compare simplified paths in Windows exclusion tests (#5525)
## Summary

Closes https://github.com/astral-sh/uv/issues/5521.
2024-07-28 17:26:46 -04:00
Charlie Marsh efbc9fb78d Add support for benchmarking uv sync and uv lock (#5524)
## Summary

This PR adds support for `uv lock` and `uv sync` in the standardized
benchmarks script.

Part of: https://github.com/astral-sh/uv/issues/5263.

## Test Plan

For example:

```sh
python scripts/bench/__main__.py --uv-project --benchmark resolve-cold ./scripts/requirements/trio.in --verbose
```
2024-07-28 21:09:08 +00:00
Mark Ture 12e92b7718 Fix typo in projects.md (#5523) 2024-07-28 21:05:26 +01:00
Charlie Marsh 83412837e5 Warn, but don't error, when encountering tilde .dist-info directories (#5520)
## Summary

Closes https://github.com/astral-sh/uv/issues/3668.
2024-07-28 19:13:06 +00:00
Charlie Marsh b0c841ee3b Ban --no-cache with --link-mode=symlink (#5519)
## Summary

Closes https://github.com/astral-sh/uv/issues/5360.
2024-07-28 15:01:17 -04:00
Krishnan Chandra 4b4128446d Support xz compressed packages (#5513)
## Summary

Closes #2187.

The [xz
backdoor](https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27)
is still fairly recent, but luckily the [Rust `xz2` crate bundles
version 5.2.5 of the C `xz`
package](https://github.com/alexcrichton/xz2-rs/tree/main/lzma-sys),
which is before the backdoor was introduced.

It's worth noting that a security risk still exists if you have a
compromised version of `xz` installed on your system, but that risk is
not introduced by `uv` or the Rust packages in general.

## Test Plan

Tried installing the package mentioned in the linked issue: `python-apt
@
https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.7.6/python-apt_2.7.6.tar.xz`

(Note that this will only work on Ubuntu - I tried on a Mac and while
the archive was extracted properly, the package did not install because
of some missing files)

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-07-28 18:37:48 +00:00
Charlie Marsh caf01735fa Avoid warning users for missing self-extra lower bounds (#5518)
## Summary

Closes https://github.com/astral-sh/uv/issues/5227.
2024-07-28 18:35:18 +00:00
Charlie Marsh 88340fbd0d Remove some unused methods (#5512) 2024-07-28 17:20:12 +00:00
Charlie Marsh 1734c7ed50 Use existing METADATA parser in wheel installer (#5508) 2024-07-27 14:38:16 -04:00
Jo ae11317cc0 Make pip list --editable conflicts with --exlcude-editable (#5506)
## Summary

I think it makes no sense to allow `--editable` and `--exclude-editable`
at the same time.

## Test Plan
```console
$ cargo run -- pip list --editable --exclude-editable
error: the argument '--editable' cannot be used with '--exclude-editable'

Usage: uv.exe pip list --editable

For more information, try '--help'.

```
2024-07-27 08:26:46 -04:00
Charlie Marsh 866d844977 Allow uv pip install for unmanaged projects (#5504)
## Summary

Just an oversight.

Closes: https://github.com/astral-sh/uv/issues/5500.
2024-07-27 02:10:18 +00:00
Charlie Marsh 24859bd3ee Upgrade to Rust 1.80.0 (#5472) 2024-07-27 01:49:47 +00:00
Charlie Marsh 3ea5e16e96 Respect reinstalls in cached environments (#5499)
## Summary

Closes #5493.
2024-07-27 01:36:58 +00:00
Charlie Marsh 8f16f1b746 Remove serialize traits from verbatim URL (#5501) 2024-07-27 00:51:24 +00:00
Charlie Marsh 4f3dde34dc Use 666 rather than 644 for default permissions (#5498)
## Summary

I don't know why I used 644 here. 666 is the actual default:
https://github.com/rust-lang/rust/blob/7c2012d0ec3aae89fefc40e5d6b317a0949cda36/library/std/src/sys/pal/unix/fs.rs#L1069

Closes https://github.com/astral-sh/uv/issues/5496.
2024-07-27 00:08:52 +00:00
Charlie Marsh 623ba3885f Add some missing reinstall-refresh calls (#5497)
## Summary

Turns out I missed a few of these.
2024-07-27 00:01:40 +00:00
Charlie Marsh 561625ed8c Use hasher to compute resolution hash (#5495)
## Summary

Addressing one TODO. This should be more efficient.
2024-07-26 23:24:09 +00:00
Ahmed Ilyas e8d7c0cb58 Editable installs for uv tool (#5454)
## Summary

Resolves #5436. 

## Test Plan

`cargo test` 

```console
❯ ./target/debug/uv tool install -e ~/black
warning: `uv tool install` is experimental and may change without warning
Resolved 6 packages in 894ms
   Built black @ file:///Users/ahmedilyas/black
Prepared 1 package in 468ms
Installed 6 packages in 6ms
 + black==24.4.3.dev23+g7e2afc9 (from file:///Users/ahmedilyas/black)
 + click==8.1.7
 + mypy-extensions==1.0.0
 + packaging==24.1
 + pathspec==0.12.1
 + platformdirs==4.2.2
Installed 2 executables: black, blackd
```

venv has the `.pth` files.
```console
❯ eza /Users/ahmedilyas/Library/Application\ Support/uv/tools/black/lib/python3.12/site-packages/
_black.pth       _virtualenv.py                         click                  mypy_extensions-1.0.0.dist-info  packaging                 pathspec                   platformdirs
_virtualenv.pth  black-24.4.3.dev23+g7e2afc9.dist-info  click-8.1.7.dist-info  mypy_extensions.py               packaging-24.1.dist-info  pathspec-0.12.1.dist-info  platformdirs-4.2.2.dist-info
```

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-07-26 16:30:15 -04:00
Andrew Gallant 2186e967f6 uv-resolver: fix basic case of overlapping markers
Consider the following packse scenario:

```toml
[root]
requires = [
  "a>=1.0.0 ; python_version < '3.10'",
  "a>=1.1.0 ; python_version >= '3.10'",
  "a>=1.2.0 ; python_version >= '3.11'",
]

[packages.a.versions."1.0.0"]
[packages.a.versions."1.1.0"]
[packages.a.versions."1.2.0"]
```

On current `main`, this produces a dependency on `a` that looks like
this:

```toml
dependencies = [
    { name = "fork-overlapping-markers-basic-a", marker = "python_version < '3.10' or python_version >= '3.11'" },
]
```

But the marker expression is clearly wrong here, since it implies that
`a` isn't installed at all for Python 3.10. With this PR, the above
dependency becomes:

```toml
dependencies = [
    { name = "fork-overlapping-markers-basic-a" },
]
```

That is, it's unconditional. Which is I believe correct here since there
aren't any other constraints on which version to select.

The specific bug here is that when we found overlapping dependency
specifications for the same package *within* a pre-existing fork, we
intersected all of their marker expressions instead of unioning them.
That in turn resulted in incorrect marker expressions.

While this doesn't fix any known bug on the issue tracker (like #4640),
it does appear to fix a couple of our snapshot tests. And fixes a basic
test case I came up with while working on #4732.

For the packse scenario test: https://github.com/astral-sh/packse/pull/206
2024-07-26 12:06:37 -07:00
Charlie Marsh 6901a14aa0 Bump version to v0.2.30 (#5486) 2024-07-26 13:36:05 -04:00
konsti f1eda3590b Update to packse 0.3.31 (#5441)
Update packse to 0.3.31, adding the instability scenarios.
2024-07-26 15:39:29 +00:00
konsti cb505d24f8 Unify resolutions only during graph building (#5479)
With our previous eager union, we were losing the fork markers. We now
carry this information into the resolution graph construction and, in
the next step, can read the markers there.

Part of
https://github.com/astral-sh/uv/issues/5180#issuecomment-2247696198
2024-07-26 16:29:48 +02:00
Andrew Gallant 77b005244d uv-resolver: propagate markers to sibling dependencies in forks
When a fork occurs, we divide not just the dependencies that
provoked a fork into distinct groups, but we also add the
corresponding sibling dependencies to each fork. Previously,
while we track markers on the fork itself, the individual
dependencies that had markers only corresponded to markers
written from the dependency specification.

This meant that the sibling dependencies that got added to
each fork would not themselves have markers attached to them.
This in turn meant they would not have markers associated with
them in the lock file.

In many cases, this is actually okay, because the resolver will
pick a version that is "universal" across all forks in most
cases. But in some cases, this just simply isn't possible as
the marker expressions in the fork can and do influence resolution.
In which case, it is possible for the same package with different
versions to show up in the lock file unconditionally. Which is a
big no-no.

So in this commit, after we determine the forks, we intersect the
markers on each fork with each of its dependencies.

This does seem to balloon the marker expressions in some cases.
I plucked one low hanging fruit to avoid doing `x and x` in
trivial cases. (And this eliminated a portion of the snapshot
diffs.) But some pretty gnarly diffs remain.

This commit also fixes another bug: previously, when we created a fork
to capture the "remaining" universe of an incomplete set of markers, we
left out dependencies that should be included in that fork. We rectify
that here.

Fixes #5086

Partially addresses #4732
2024-07-26 07:28:20 -07:00
Andrew Gallant 412780fd99 uv/tests: add regression test from #5086
The snapshot saved here is wrong, but we'll update it in a subsequent
commit.
2024-07-26 07:28:20 -07:00
Andrew Gallant cd1fc7c9a3 uv-normalize: make "name" types implement Default
Interestingly, the empty string appears to be valid for these
types. I'm not sure if that's intended, but having a Default
impl is useful for use with `std::mem::take`.
2024-07-26 07:28:20 -07:00
Andrew Gallant 7fce59e4bc pep508: some simplification in 'and' and 'or'
Basically, and'ing or or'ing the same expression can be entire
skipped. And we try harder to avoid singleton conjunctions or
disjunctions, as these are considered unequal otherwise. (Thus
defeating our attempts to avoid and'ing or or'ing a superfluous
marker.)
2024-07-26 07:28:20 -07:00