Commit Graph

2559 Commits

Author SHA1 Message Date
Andrew Gallant a74c2c6792 pep508: add and and or mutators to MarkerTree
While this could be done by callers since the representation
of `MarkerTree` is public, they are just annoying enough to do
that I think it makes sense to provide them on `MarkerTree`
itself.

These could also be improved in the future to do even more
flattening of conjunctions/disjunctions (or perhaps even
more robust simplification). But for now, some basic flattening
is good enough.

These routines will be used to combine marker expressions when
merging forked resolutions.
2024-05-30 14:23:14 -04:00
Charlie Marsh 144566907e Use lockfile versions as resolution preferences (#3921)
## Summary

Ensures that we avoid upgrading packages unless `--upgrade` or similar
is passed.

For now, the resolver only respects these for registry distributions.

Closes https://github.com/astral-sh/uv/issues/3918.
2024-05-30 17:59:53 +00:00
Charlie Marsh 502e04200d Remove from_registry_built_dist2 (#3922)
## Summary

I think this was left over from a prior refactor. (The existing
`from_registry_built_dist` was unused.)
2024-05-30 01:57:19 +00:00
Charlie Marsh a1a5155d34 Move preference behavior out of resolve (#3919)
## Summary

Decoupling this behavior to give more control to clients. No change in
behavior.
2024-05-30 01:16:53 +00:00
Charlie Marsh 4859a27948 Add extra dependency annotations to lockfile and sync commands (#3913)
## Summary

This PR adds extras to the lockfile, and enables users to selectively
sync extras in `uv sync` and `uv run`. The end result here was fairly
simple, though it required a few refactors to get here. The basic idea
is that `DistributionId` now includes `extra: Option<ExtraName>`, so we
effectively treat extras as separate packages. Generating the lockfile,
and generating the resolution from the lockfile, fall out of this
naturally with no special-casing or additional changes.

The main downside here is that it bloats the lockfile significantly.
Specifically:

- We include _all_ distribution URLs and hashes for _every_ extra
variant.
- We include all dependencies for the extra variant, even though that
are dependencies of the base package.

We could normalize this representation by changing each distribution
have an `optional-dependencies` hash map that keys on extras, but we
actually don't have the information we need to create that right now
(specifically, we can't differentiate between dependencies that
_require_ the extra and dependencies on the base package).

Closes #3700.
2024-05-29 19:25:58 +00:00
Charlie Marsh 1bd5d8bc34 Include all extras when generating lockfile (#3912)
## Summary

This PR just ensures that when running `uv lock` (or `uv run`), we lock
with all extras. When we later install, we'll also _install_ with all
extras, but that will be changed in a future PR.
2024-05-29 15:08:20 -04:00
Charlie Marsh fb0dfef671 Migrate lockfile tests to uv lock and uv sync (#3914) 2024-05-29 18:00:37 +00:00
Charlie Marsh ed7f55606d Split requirements.txt-style resolution distribution into its own type (#3911) 2024-05-29 16:48:28 +00:00
konsti af06544347 Remove dead code from distribution-types (#3910)
Remove some dead code. Split out from #3904
2024-05-29 12:02:42 -04:00
Charlie Marsh 19c91e7dac Create distinct graph nodes for each package extra (#3908)
## Summary

Today, we represent each package as a single node in the graph, and
combine all the extras. This is helpful for the `requirements.txt`-style
resolution, in which we want to show each a single line for each package
with the extras combined into a single array.

This PR modifies the representation to instead use a separate node for
each (package, extra) pair. We then reduce into the previous format when
printing in the `requirements.txt`-style format, so there shouldn't be
any user-facing changes here.
2024-05-29 15:42:49 +00:00
konsti 0edb660205 Add missing i686 alias for x86 (#3899)
This alias is required for the 32-bit x86 manylinux builder.

I've also changed the reporting level for unusable python interpreter
errors to debug so it shows up with `-v`, otherwise we can't see what's
broken:

```console
# uv venv py312 --python python3.12 -vv
    0.000461s DEBUG uv_interpreter::discovery Searching for Python 3.12 in search path
  × No interpreter found for Python 3.12 in search path
```

Reproduction:

```console
$ docker run --rm -it quay.io/pypa/manylinux2014_i686
# curl -LsSf https://astral.sh/uv/install.sh | s
# source $HOME/.cargo/env
# RUST_LOG=trace uv venv py312 --python python3.12
DEBUG Searching for Python 3.12 in search path
TRACE Searching PATH for executables: python3.12, python3, python
TRACE Checking `PATH` directory for interpreters: /root/.cargo/bin
TRACE Checking `PATH` directory for interpreters: /opt/rh/devtoolset-10/root/usr/bin
TRACE Checking `PATH` directory for interpreters: /usr/local/sbin
TRACE Checking `PATH` directory for interpreters: /usr/local/bin
TRACE Found possible Python executable: /usr/local/bin/python3.12
TRACE Querying interpreter executable at /usr/local/bin/python3.12
TRACE Querying Python at `/usr/local/bin/python3.12` did not return the expected data
unknown variant `i686`, expected one of `aarch64`, `armv6l`, `armv7l`, `powerpc64le`, `powerpc64`, `x86`, `x86_64`, `s390x`
--- stdout:
{"result": "success", "markers": {"implementation_name": "cpython", "implementation_version": "3.12.3", "os_name": "posix", "platform_machine": "i686", "platform_python_implementation": "CPython", "platform_release": "6.1.0-10-amd64", "platform_system": "Linux", "platform_version": "#1 SMP PREEMPT_DYNAMIC Debian 6.1.37-1 (2023-07-03)", "python_full_version": "3.12.3", "python_version": "3.12", "sys_platform": "linux"}, "base_prefix": "/opt/_internal/cpython-3.12.3", "base_exec_prefix": "/opt/_internal/cpython-3.12.3", "prefix": "/opt/_internal/cpython-3.12.3", "base_executable": "/usr/local/bin/python3.12", "sys_executable": "/usr/local/bin/python3.12", "sys_path": ["/root/.cache/uv/.tmp7k64ZY", "/opt/_internal/cpython-3.12.3/lib/python312.zip", "/opt/_internal/cpython-3.12.3/lib/python3.12", "/opt/_internal/cpython-3.12.3/lib/python3.12/lib-dynload", "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages"], "stdlib": "/opt/_internal/cpython-3.12.3/lib/python3.12", "scheme": {"platlib": "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages", "purelib": "/opt/_internal/cpython-3.12.3/lib/python3.12/site-packages", "include": "/opt/_internal/cpython-3.12.3/include/python3.12", "scripts": "/opt/_internal/cpython-3.12.3/bin", "data": "/opt/_internal/cpython-3.12.3"}, "virtualenv": {"purelib": "lib/python3.12/site-packages", "platlib": "lib/python3.12/site-packages", "include": "include/site/python3.12", "scripts": "bin", "data": ""}, "platform": {"os": {"name": "manylinux", "major": 2, "minor": 17}, "arch": "i686"}, "gil_disabled": false, "pointer_size": "32"}
--- stderr:

---
TRACE Skipping bad interpreter at /usr/local/bin/python3.12
TRACE Checking `PATH` directory for interpreters: /usr/sbin
TRACE Checking `PATH` directory for interpreters: /usr/bin
TRACE Found possible Python executable: /usr/bin/python
TRACE Querying interpreter executable at /usr/bin/python
TRACE Can't use Python at `/usr/bin/python`
TRACE Skipping bad interpreter at /usr/bin/python
TRACE Checking `PATH` directory for interpreters: /sbin
TRACE Checking `PATH` directory for interpreters: /bin
TRACE Found possible Python executable: /bin/python
TRACE Querying interpreter executable at /bin/python
TRACE Can't use Python at `/bin/python`
TRACE Skipping bad interpreter at /bin/python
```
2024-05-29 11:31:39 -04:00
Ahmed Ilyas 1a9aa35eae Fix workspace settings - remove deny unknown fields (#3907)
## Summary

This PR addresses an issue where `tool.uv` settings are not read if
`tool.uv.sources` or `tool.uv.workspaces` are present in the TOML file.

## Test Plan

Tested locally.
2024-05-29 11:31:07 -04:00
Charlie Marsh 42b1ba04ec Remove unused PetGraph weight (#3906) 2024-05-29 15:16:25 +00:00
Charlie Marsh 2301e00863 Use lockfile in uv run (#3894)
## Summary

Modifies `uv run` to write and read from the lockfile, rather than
resolving the project requirements as-is on each invocation.

Closes https://github.com/astral-sh/uv/issues/3891.
2024-05-29 13:28:21 +00:00
Ibraheem Ahmed 9658485f06 Initialize multi-progress state before individual bars (#3901)
## Summary

Resolves https://github.com/astral-sh/uv/issues/3896. Adding progress
bars to the `MultiProgress` after configuring them seems to not
synchronize the required state fully.

## Test Plan

The repeated output is gone when testing locally.
2024-05-29 09:23:40 -04:00
konsti 1e1f49bfe5 Remove unnecessary pub in uv-build (#3900)
Expose only `SourceBuild` and the error type from `uv-build`, all other
types are internal.
2024-05-29 09:30:21 +00:00
Charlie Marsh 3461c8b585 Move extra specification into uv-configuration (#3897)
## Summary

I need to use this in the resolver (and it's at-home with other, similar
configuration options).
2024-05-29 04:49:57 +00:00
Charlie Marsh 2b6c24ed2d Rename project API error (#3893) 2024-05-28 21:59:24 -04:00
Ibraheem Ahmed 038af6e658 Disable concurrent progress bars in Jupyter Notebooks (#3890)
## Summary

Resolves https://github.com/astral-sh/uv/issues/3887 by disabling the
new progress output when the `JPY_SESSION_NAME` environment variable is
detected.
2024-05-28 17:05:11 -04:00
Charlie Marsh 47db418ba2 Bump version to v0.2.5 (#3885) 2024-05-28 18:05:32 +00:00
Charlie Marsh 78c43c6a7a Move editable discovery behind --preview for now (#3884)
## Summary

Just reduces the risk of shipping stable regressions for now.
2024-05-28 17:54:25 +00:00
konsti f92447772e Support x86 windows (#3873)
Add trampolines for x86 windows.

Closes https://github.com/astral-sh/uv/issues/3660.
2024-05-28 16:07:39 +00:00
Charlie Marsh cedd18e4c6 Remove some unused pub functions (#3872)
## Summary

I wrote a bad Python script to find these.
2024-05-28 15:58:13 +00:00
Charlie Marsh 1fc6a59707 Remove special-casing for editable requirements (#3869)
## Summary

There are a few behavior changes in here:

- We now enforce `--require-hashes` for editables, like pip. So if you
use `--require-hashes` with an editable requirement, we'll reject it. I
could change this if it seems off.
- We now treat source tree requirements, editable or not (e.g., both `-e
./black` and `./black`) as if `--refresh` is always enabled. This
doesn't mean that we _always_ rebuild them; but if you pass
`--reinstall`, then yes, we always rebuild them. I think this is an
improvement and is close to how editables work today.

Closes #3844.

Closes #2695.
2024-05-28 15:49:34 +00:00
konsti 063a0a4384 Add context to failed uv tool run (#3882)
These are not covered by `fs_err` and a missing binary otherwise only
give a generic "file not found" error.
2024-05-28 14:48:54 +00:00
Tim de Jager b12b25db10 fix: remove unused Result in uv-cache (#3875)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
It removes the unused result when creating the Cache with the
`from_path` constructor. I don't believe it does any io operations any
more at least.
2024-05-28 09:19:08 +00:00
konsti a89e146107 Initial workspace support (#3705)
Add workspace support when using `-r <path>/pyproject.toml` or `-e
<path>` in the pip interface. It is limited to all-editable
static-metadata workspaces, and tests only include a single main
workspace, ignoring path dependencies in another workspace. This can be
considered the MVP for workspace support: You can create a workspace,
you can install from it, but some options and conveniences are still
missing. I'll file follow-up tickets (support in lockfiles, support path
deps in other workspace, #3625)

There is also support in `uv run`, but we need
https://github.com/astral-sh/uv/issues/3700 first to properly support
using different current projects in the bluejay interface, currently the
resolution and therefore the lockfile depends on the current project.
I'd do this change first (it's big enough already), then #3700, and then
add workspace support properly to bluejay.

Fixes #3404
2024-05-28 07:41:53 +00:00
Charlie Marsh 89cfecea77 Add support for prepare_metadata_for_build_editable hook (#3870)
## Summary

See:
https://peps.python.org/pep-0660/#prepare-metadata-for-build-editable
2024-05-27 23:21:45 +00:00
Charlie Marsh 22dbc79994 Use consistent argument ordering for Dist conversions (#3868) 2024-05-27 19:48:58 +00:00
Charlie Marsh 3c61eabd9d Remove unnecessary ::try_from (#3867)
## Summary

I didn't realize this even worked? We only define a `From` for this
conversion.
2024-05-27 15:18:20 -04:00
Tim de Jager 65b17f6e81 feat: bump pep580 and pep440 types (#3860)
This bumps the versions of pep580 and pep440 to coincide with the
crates.io versions. While not strictly the same, the new types in uv us
an `Inner` struct. Practically I've found I'm still able to use the
patched versions, as can seen from the open PR here:
https://github.com/prefix-dev/pixi/pull/1436.

Would be great if this bump can be done so we can keep combining the
types :)
2024-05-27 09:38:39 +02:00
Zanie Blue 66eb642938 Add test context utility to uv-interpreter (#3832)
Follow-up to https://github.com/astral-sh/uv/pull/3797 to clean up the
test isolation in `uv-interpreter`.

I still want to expose a CLI at some point like `uv python <...>` for
discovery and test from there, hopefully this will make that transition
simpler.
2024-05-27 04:17:49 +00:00
Charlie Marsh 14fa49b7ba Move availability enums into their own module (#3858) 2024-05-27 00:12:53 -04:00
Zanie Blue 31b0ff6373 Gate discovery of managed toolchains with preview (#3835)
Prepares for merge of https://github.com/astral-sh/uv/pull/3797, gating
managed toolchain discovery with the preview flag to lower risk of
releasing.

e.g.

```
❯ cargo dev -q fetch-python         
❯ cargo run -q -- venv --python 3.11
Using Python 3.11.9 interpreter at: /opt/homebrew/opt/python@3.11/bin/python3.11
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
❯ cargo run -q -- venv --preview --python 3.11
Using Python 3.11.7 interpreter at: /Users/zb/Library/Application Support/uv/toolchains/cpython-3.11.7-macos-aarch64-none/install/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
```

We'll add automatic fetching of managed interpreters later.
2024-05-27 04:05:19 +00:00
Zanie Blue 30e780e1dd Add persistent storage of installed toolchains (#3797)
Extends #3726 

Moves toolchain storage out of `UV_BOOTSTRAP_DIR` (`./bin`) into the
proper user data directory as defined by #3726.

Replaces `UV_BOOTSTRAP_DIR` with `UV_TOOLCHAIN_DIR` for customization.
Installed toolchains will be discovered without opt-in, but the idea is
still that these are not yet user-facing.
2024-05-27 03:54:49 +00:00
Charlie Marsh 4191c331a7 Update bundled Python URLs and add "arm" architecture variant (#3855)
## Summary

This also adds filtering for the ARM Pythons, since that needs some libc
changes; and it closes https://github.com/astral-sh/uv/issues/3854 by
way of adding an "arm" branch.
2024-05-26 23:33:08 -04:00
Charlie Marsh 61ed0c82ed Remove unused Platform enum (#3856) 2024-05-26 23:21:56 -04:00
renovate[bot] 89daa58458 Update Rust crate hyper-util to v0.1.4 (#3847) 2024-05-27 01:25:23 +00:00
Ibraheem Ahmed 7dc322665c Concurrent progress bars (#3252)
## Summary

Implements concurrent progress bars. Resolves
https://github.com/astral-sh/uv/issues/1209.

## Test Plan

https://github.com/astral-sh/uv/assets/34988408/b21bdfbb-8817-4873-a65c-16c9e8c7c460
2024-05-27 01:21:07 +00:00
Charlie Marsh 70cbc32565 Remove tracing::log usage (#3853) 2024-05-27 01:03:28 +00:00
renovate[bot] 080f09586b Update pre-commit dependencies (#3852) 2024-05-27 00:58:51 +00:00
renovate[bot] e2971a7a4b Update Rust crate tracing-tree to v0.3.1 (#3851) 2024-05-27 00:58:06 +00:00
renovate[bot] 3416f4247f Update Rust crate serde to v1.0.203 (#3850) 2024-05-27 00:57:45 +00:00
renovate[bot] 2aba5867e7 Update Rust crate schemars to v0.8.21 (#3849) 2024-05-27 00:56:04 +00:00
renovate[bot] 57bb7a9507 Update Rust crate mimalloc to v0.1.42 (#3848) 2024-05-27 00:54:38 +00:00
Charlie Marsh 34e1cc8dd4 Use --exclude-newer in some pip sync tests (#3846)
## Summary

Saw a bunch of failures in
https://github.com/astral-sh/uv/actions/runs/9247239042/job/25435791135?pr=3252
due to `anyio` publishing v0.4.4.
2024-05-27 00:41:18 +00:00
Zanie Blue 98652e3954 Bump version to 0.2.4 (#3840) 2024-05-26 12:11:15 -05:00
konsti f4533c3a85 Ignore libc on other platforms (#3825)
Libc variants are only relevant on linux, we can ignore it on non
{linux, windows, mac}, e.g. BSDs.

Closes #3824
2024-05-25 10:05:10 +00:00
Charlie Marsh c2931e806d Remove discovery group from CLI (#3830)
## Summary

Allows, e.g., `UV_SYSTEM_PYTHON=false uv pip install --python
.venv/bin/python`.

This was intended to work after fixing
https://github.com/astral-sh/uv/issues/3000, but I think I misdiagnosed
the scope when closing that issue, and the linked PR there only fixed
some _other_ problems around index URLs.

The only thing we really lose here is we no longer error when
`--break-system-packages` is provided without `--system`, but we can
enforce that elsewhere if we want.

Closes https://github.com/astral-sh/uv/issues/3829.
2024-05-24 19:03:44 +00:00
Charlie Marsh 6682765630 Require pinned version in Preference type (#3828)
## Summary

This PR makes a variety of invalid states unrepresentable by changing
`Preference` to require a `PackageName` and `Version`, rather than
accepting a generic `Requirement`. There should be no meaningful
behavior changes.
2024-05-24 18:48:36 +00:00