Files

T

konsti db371560bc Use prettier to format the documentation (#5708 )

To enforce the 100 character line limit in markdown files introduced in
https://github.com/astral-sh/uv/pull/5635, and to automate the
formatting of markdown files, i've added prettier and formatted our
markdown files with it.

I've excluded the changelog and the generated references documentation
from this for having too many changes, but we can also include them.

I'm not particular on which style we use. My main motivations are
(major) not having to reflow markdown files myself anymore and (minor)
consistence between all markdown files. I've chosen prettier for similar
reason as we chose black, it's a single good style that's automated and
shared in the community. I do prefer prettier's style of not breaking
inside of a link name though.

This PR is in two parts, the first adds prettier to CI and documents
using it, while the second actually formats the docs. When merge
conflicts arise, we can drop the last commit and regenerate it with `npx
prettier --prose-wrap always --write BENCHMARKS.md CONTRIBUTING.md
README.md STYLE.md docs/*.md docs/concepts/**/*.md docs/guides/**/*.md
docs/pip/**/*.md`.

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>

2024-08-02 08:58:31 -05:00

4.7 KiB

Raw Blame History

Benchmarks

All benchmarks were computed on macOS using Python 3.12.0 (for non-uv tools), and come with a few important caveats:

Benchmark performance may vary dramatically across different operating systems and filesystems. In particular, uv uses different installation strategies based on the underlying filesystem's capabilities. (For example, uv uses reflinking on macOS, and hardlinking on Linux.)
Benchmark performance may vary dramatically depending on the set of packages being installed. For example, a resolution that requires building a single intensive source distribution may appear very similar across tools, since the bottleneck is tool-agnostic.
Unlike Poetry, both uv and pip-tools do not generate multi-platform lockfiles. As such, Poetry is (by design) doing significantly more work than other tools in the resolution benchmarks. Poetry is included for completeness, as many projects may not need a multi-platform lockfile. However, it's critical to understand that benchmarking uv's resolution time against Poetry is an unfair comparison. (Benchmarking installation, however, is a fair comparison.)

This document benchmarks against Trio's docs-requirements.in, as a representative example of a real-world project.

In each case, a smaller bar (i.e., lower) is better.

Warm Installation

Benchmarking package installation (e.g., uv pip sync) with a warm cache. This is equivalent to removing and recreating a virtual environment, and then populating it with dependencies that you've installed previously on the same machine.

Cold Installation

Benchmarking package installation (e.g., uv pip sync) with a cold cache. This is equivalent to running uv pip sync on a new machine or in CI (assuming that the package manager cache is not shared across runs).

Warm Resolution

Benchmarking dependency resolution (e.g., uv pip compile) with a warm cache, but no existing lockfile. This is equivalent to blowing away an existing requirements.txt file to regenerate it from a requirements.in file.

Cold Resolution

Benchmarking dependency resolution (e.g., uv pip compile) with a cold cache. This is equivalent to running uv pip compile on a new machine or in CI (assuming that the package manager cache is not shared across runs).

Reproduction

All benchmarks were generated using the scripts/benchmark package, which wraps hyperfine to facilitate benchmarking uv against a variety of other tools.

The benchmark script itself has a several requirements:

A local uv release build (cargo build --release).
An installation of the production uv binary in your path.
The hyperfine command-line tool installed on your system.

To benchmark resolution against pip-compile, Poetry, and PDM:

uv run resolver \
    --uv-pip \
    --poetry \
    --pdm \
    --pip-compile \
    --benchmark resolve-warm --benchmark resolve-cold \
    --json \
    ../requirements/trio.in

To benchmark installation against pip-sync, Poetry, and PDM:

uv run resolver \
    --uv-pip \
    --poetry \
    --pdm \
    --pip-sync \
    --benchmark install-warm --benchmark install-cold \
    --json \
    ../requirements/compiled/trio.txt

Both commands should be run from the scripts/benchmark directory.

After running the benchmark script, you can generate the corresponding graph via:

cargo run -p uv-dev render-benchmarks resolve-warm.json --title "Warm Resolution"
cargo run -p uv-dev render-benchmarks resolve-cold.json --title "Cold Resolution"
cargo run -p uv-dev render-benchmarks install-warm.json --title "Warm Installation"
cargo run -p uv-dev render-benchmarks install-cold.json --title "Cold Installation"

You need to install the Roboto Font if the labels are missing in the generated graph.

Acknowledgements

The inclusion of this BENCHMARKS.md file was inspired by the excellent benchmarking documentation in Orogene.

Troubleshooting

Flaky benchmarks

If you're seeing high variance when running the cold benchmarks, then it's likely that you're running into throttling or DDoS prevention from your ISP. In that case, ISPs forcefully terminate TCP connections with a TCP reset. We believe this is due to the benchmarks making the exact same requests in a very short time (especially true for uv). A possible workaround is to connect to VPN to bypass your ISPs filtering mechanism.

4.7 KiB Raw Blame History