Files
uv/CONTRIBUTING.md
T
samypr100 fee7f9d093 Support Git LFS with opt-in (#16143)
## Summary

Follow up to https://github.com/astral-sh/uv/pull/15563
Closes https://github.com/astral-sh/uv/issues/13485

This is a first-pass at adding support for conditional support for Git
LFS between git sources, initial feedback welcome.

e.g.
```
[tool.uv.sources]
test-lfs-repo = { git = "https://github.com/zanieb/test-lfs-repo.git", lfs = true }
```

For context previously a user had to set `UV_GIT_LFS` to have uv fetch
lfs objects on git sources. This env var was all or nothing, meaning you
must always have it set to get consistent behavior and it applied to all
git sources. If you fetched lfs objects at a revision and then turned
off lfs (or vice versa), the git db, corresponding checkout lfs
artifacts would not be updated properly. Similarly, when git source
distributions were built, there would be no distinction between sources
with lfs and without lfs. Hence, it could corrupt the git, sdist, and
archive caches.

In order to support some sources being LFS enabled and other not, this
PR adds a stateful layer roughly similar to how `subdirectory` works but
for `lfs` since the git database, the checkouts and the corresponding
caching layers needed to be LFS aware (requested vs installed). The
caches also had to isolated and treated entirely separate when handling
LFS sources.

Summary
* Adds `lfs = true` or `lfs = false` to git sources in pyproject.toml
* Added `lfs=true` query param / fragments to most relevant url structs
(not parsed as user input)
  * In the case of uv add / uv tool, `--lfs` is supported instead
* `UV_GIT_LFS` environment variable support is still functional for
non-project entrypoints (e.g. uv pip)
* `direct-url.json` now has an custom `git_lfs` entry under VcsInfo
(note, this is not in the spec currently -- see caveats).
* git database and checkouts have an different cache key as the sources
should be treated effectively different for the same rev.
* sdists cache also differ in the cache key of a built distribution if
it was built using LFS enabled revisions to distinguish between non-LFS
same revisions. This ensures the strong assumption for archive-v0 that
an unpacked revision "doesn't change sources" stays valid.

Caveats
* `pylock.toml` import support has not been added via git_lfs=true,
going through the spec it wasn't clear to me it's something we'd support
outside of the env var (for now).
* direct-url struct was modified by adding a non-standard `git_lfs`
field under VcsInfo which may be undersirable although the PEP 610 does
say `Additional fields that would be necessary to support such VCS
SHOULD be prefixed with the VCS command name` which could be interpret
this change as ok.
* There will be a slight lockfile and cache churn for users that use
`UV_GIT_LFS` as all git lockfile entries will get a `lfs=true` fragment.
The cache version does not need an update, but LFS sources will get
their own namespace under git-v0 and sdist-v9/git hence a cache-miss
will occur once but this can be sufficient to label this as breaking for
workflows always setting `UV_GIT_LFS`.

## Test Plan

Some initial tests were added. More tests likely to follow as we reach
consensus on a final approach.

For IT test, we may want to move to use a repo under astral namespace in
order to test lfs functionality.

Manual testing was done for common pathological cases like killing LFS
fetch mid-way, uninstalling LFS after installing an sdist with it and
reinstalling, fetching LFS artifacts in different commits, etc.

PSA: Please ignore the docker build failures as its related to depot
OIDC issues.

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
Co-authored-by: konstin <konstin@mailbox.org>
2025-12-02 12:23:51 +00:00

232 lines
8.2 KiB
Markdown

# Contributing
## Finding ways to help
We label issues that would be good for a first time contributor as
[`good first issue`](https://github.com/astral-sh/uv/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22).
These usually do not require significant experience with Rust or the uv code base.
We label issues that we think are a good opportunity for subsequent contributions as
[`help wanted`](https://github.com/astral-sh/uv/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22).
These require varying levels of experience with Rust and uv. Often, we want to accomplish these
tasks but do not have the resources to do so ourselves.
You don't need our permission to start on an issue we have labeled as appropriate for community
contribution as described above. However, it's a good idea to indicate that you are going to work on
an issue to avoid concurrent attempts to solve the same problem.
Please check in with us before starting work on an issue that has not been labeled as appropriate
for community contribution. We're happy to receive contributions for other issues, but it's
important to make sure we have consensus on the solution to the problem first.
Outside of issues with the labels above, issues labeled as
[`bug`](https://github.com/astral-sh/uv/issues?q=is%3Aopen+is%3Aissue+label%3A%22bug%22) are the
best candidates for contribution. In contrast, issues labeled with `needs-decision` or
`needs-design` are _not_ good candidates for contribution. Please do not open pull requests for
issues with these labels.
Please do not open pull requests for new features without prior discussion. While we appreciate
exploration of new features, we will almost always close these pull requests immediately. Adding a
new feature to uv creates a long-term maintenance burden and requires strong consensus from the uv
team before it is appropriate to begin work on an implementation.
## Setup
[Rust](https://rustup.rs/) (and a C compiler) are required to build uv.
On Ubuntu and other Debian-based distributions, you can install a C compiler with:
```shell
sudo apt install build-essential
```
On Fedora-based distributions, you can install a C compiler with:
```shell
sudo dnf install gcc
```
## Testing
For running tests, we recommend [nextest](https://nexte.st/).
If tests fail due to a mismatch in the JSON Schema, run: `cargo dev generate-json-schema`.
### Python
Testing uv requires multiple specific Python versions; they can be installed with:
```shell
cargo run python install
```
The storage directory can be configured with `UV_PYTHON_INSTALL_DIR`. (It must be an absolute path.)
### Snapshot testing
uv uses [insta](https://insta.rs/) for snapshot testing. It's recommended (but not necessary) to use
`cargo-insta` for a better snapshot review experience. See the
[installation guide](https://insta.rs/docs/cli/) for more information.
In tests, you can use `uv_snapshot!` macro to simplify creating snapshots for uv commands. For
example:
```rust
#[test]
fn test_add() {
let context = TestContext::new("3.12");
uv_snapshot!(context.filters(), context.add().arg("requests"), @"");
}
```
To run and review a specific snapshot test:
```shell
cargo test --package <package> --test <test> -- <test_name> -- --exact
cargo insta review
```
### Git and Git LFS
A subset of uv tests require both [Git](https://git-scm.com) and [Git LFS](https://git-lfs.com/) to
execute properly.
These tests can be disabled by turning off either `git` or `git-lfs` uv features.
### Local testing
You can invoke your development version of uv with `cargo run -- <args>`. For example:
```shell
cargo run -- venv
cargo run -- pip install requests
```
## Running inside a Docker container
Source distributions can run arbitrary code on build and can make unwanted modifications to your
system
(["Someone's Been Messing With My Subnormals!" on Blogspot](https://moyix.blogspot.com/2022/09/someones-been-messing-with-my-subnormals.html),
["nvidia-pyindex" on PyPI](https://pypi.org/project/nvidia-pyindex/)), which can even occur when
just resolving requirements. To prevent this, there's a Docker container you can run commands in:
```console
$ docker build -t uv-builder -f crates/uv-dev/builder.dockerfile --load .
# Build for musl to avoid glibc errors, might not be required with your OS version
cargo build --target x86_64-unknown-linux-musl --profile profiling
docker run --rm -it -v $(pwd):/app uv-builder /app/target/x86_64-unknown-linux-musl/profiling/uv-dev resolve-many --cache-dir /app/cache-docker /app/scripts/popular_packages/pypi_10k_most_dependents.txt
```
We recommend using this container if you don't trust the dependency tree of the package(s) you are
trying to resolve or install.
## Profiling and Benchmarking
Please refer to Ruff's
[Profiling Guide](https://github.com/astral-sh/ruff/blob/main/CONTRIBUTING.md#profiling-projects),
it applies to uv, too.
We provide diverse sets of requirements for testing and benchmarking the resolver in
`scripts/requirements` and for the installer in `scripts/requirements/compiled`.
You can use `scripts/benchmark` to benchmark predefined workloads between uv versions and with other
tools, e.g., from the `scripts/benchmark` directory:
```shell
uv run resolver \
--uv-pip \
--poetry \
--benchmark \
resolve-cold \
../scripts/requirements/trio.in
```
### Analyzing concurrency
You can use [tracing-durations-export](https://github.com/konstin/tracing-durations-export) to
visualize parallel requests and find any spots where uv is CPU-bound. Example usage, with `uv` and
`uv-dev` respectively:
```shell
RUST_LOG=uv=info TRACING_DURATIONS_FILE=target/traces/jupyter.ndjson cargo run --features tracing-durations-export --profile profiling -- pip compile scripts/requirements/jupyter.in
```
```shell
RUST_LOG=uv=info TRACING_DURATIONS_FILE=target/traces/jupyter.ndjson cargo run --features tracing-durations-export --bin uv-dev --profile profiling -- resolve jupyter
```
### Trace-level logging
You can enable `trace` level logging using the `RUST_LOG` environment variable, i.e.
```shell
RUST_LOG=trace uv
```
## Documentation
To preview any changes to the documentation locally:
1. Install the [Rust toolchain](https://www.rust-lang.org/tools/install).
2. Run `cargo dev generate-all`, to update any auto-generated documentation.
3. Run the development server with:
```shell
# For contributors.
uvx --with-requirements docs/requirements.txt -- mkdocs serve -f mkdocs.public.yml
# For members of the Astral org, which has access to MkDocs Insiders via sponsorship.
uvx --with-requirements docs/requirements-insiders.txt -- mkdocs serve -f mkdocs.insiders.yml
```
The documentation should then be available locally at
[http://127.0.0.1:8000/uv/](http://127.0.0.1:8000/uv/).
To update the documentation dependencies, edit `docs/requirements.in` and
`docs/requirements-insiders.in`, then run:
```shell
uv pip compile docs/requirements.in -o docs/requirements.txt --universal -p 3.12
uv pip compile docs/requirements-insiders.in -o docs/requirements-insiders.txt --universal -p 3.12
```
Documentation is deployed automatically on release by publishing to the
[Astral documentation](https://github.com/astral-sh/docs) repository, which itself deploys via
Cloudflare Pages.
After making changes to the documentation, format the markdown files with:
```shell
npx prettier --prose-wrap always --write "**/*.md"
```
Note that the command above requires Node.js and npm to be installed on your system. As an
alternative, you can run this command using Docker:
```console
$ docker run --rm -v .:/src/ -w /src/ node:alpine npx prettier --prose-wrap always --write "**/*.md"
```
## Releases
Releases can only be performed by Astral team members.
Changelog entries and version bumps are automated. First, run:
```shell
./scripts/release.sh
```
Then, editorialize the `CHANGELOG.md` file to ensure entries are consistently styled.
Then, open a pull request, e.g., `Bump version to ...`.
Binary builds will automatically be tested for the release.
After merging the pull request, run the
[release workflow](https://github.com/astral-sh/uv/actions/workflows/release.yml) with the version
tag. **Do not include a leading `v`**. The release will automatically be created on GitHub after
everything else publishes.