pioduino/uv
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Add DisplaySafeUrl newtype to prevent leaking of credentials by default (#13560)

Browse Source 
Prior to this PR, there were numerous places where uv would leak
credentials in logs. We had a way to mask credentials by calling methods
or a recently-added `redact_url` function, but this was not secure by
default. There were a number of other types (like `GitUrl`) that would
leak credentials on display.

This PR adds a `DisplaySafeUrl` newtype to prevent leaking credentials
when logging by default. It takes a maximalist approach, replacing the
use of `Url` almost everywhere. This includes when first parsing config
files, when storing URLs in types like `GitUrl`, and also when storing
URLs in types that in practice will never contain credentials (like
`DirectorySourceUrl`). The idea is to make it easy for developers to do
the right thing and for the compiler to support this (and to minimize
ever having to manually convert back and forth). Displaying credentials
now requires an active step. Note that despite this maximalist approach,
the use of the newtype should be zero cost.

One conspicuous place this PR does not use `DisplaySafeUrl` is in the
`uv-auth` crate. That would require new clones since there are calls to
`request.url()` that return a `&Url`. One option would have been to make
`DisplaySafeUrl` wrap a `Cow`, but this would lead to lifetime
annotations all over the codebase. I've created a separate PR based on
this one (#13576) that updates `uv-auth` to use `DisplaySafeUrl` with
one new clone. We can discuss the tradeoffs there.

Most of this PR just replaces `Url` with `DisplaySafeUrl`. The core is
`uv_redacted/lib.rs`, where the newtype is implemented. To make it
easier to review the rest, here are some points of note:

* `DisplaySafeUrl` has a `Display` implementation that masks
credentials. Currently, it will still display the username when there is
both a username and password. If we think is the wrong choice, it can
now be changed in one place.
* `DisplaySafeUrl` has a `remove_credentials()` method and also a
`.to_string_with_credentials()` method. This allows us to use it in a
variety of scenarios.
* `IndexUrl::redacted()` was renamed to
`IndexUrl::removed_credentials()` to make it clearer that we are not
masking.
* We convert from a `DisplaySafeUrl` to a `Url` when calling `reqwest`
methods like `.get()` and `.head()`.
* We convert from a `DisplaySafeUrl` to a `Url` when creating a
`uv_auth::Index`. That is because, as mentioned above, I will be
updating the `uv_auth` crate to use this newtype in a separate PR.
* A number of tests (e.g., in `pip_install.rs`) that formerly used
filters to mask tokens in the test output no longer need those filters
since tokens in URLs are now masked automatically.
* The one place we are still knowingly writing credentials to
`pyproject.toml` is when a URL with credentials is passed to `uv add`
with `--raw`. Since displaying credentials is no longer automatic, I
have added a `to_string_with_credentials()` method to the `Pep508Url`
trait. This is used when `--raw` is passed. Adding it to that trait is a
bit weird, but it's the simplest way to achieve the goal. I'm open to
suggestions on how to improve this, but note that because of the way
we're using generic bounds, it's not as simple as just creating a
separate trait for that method.
This commit is contained in:
John Mumm
2025-05-27 00:05:30 +02:00
committed by GitHub
parent b80cafd5e8
commit c19a294a48
100 changed files with 1266 additions and 2249 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/uv-redacted/src/lib.rs
+313 -39
View File
@@ -1,21 +1,241 @@
use std::borrow::Cow;
use serde::{Deserialize, Serialize};
use std::fmt::Write;
use std::ops::{Deref, DerefMut};
use std::str::FromStr;
use url::Url;
/// Return a version of the URL with redacted credentials, allowing the generic `git` username (without a password)
/// in SSH URLs, as in, `ssh://git@github.com/...`.
pub fn redacted_url(url: &Url) -> Cow<'_, Url> {
if url.username().is_empty() && url.password().is_none() {
return Cow::Borrowed(url);
}
if url.scheme() == "ssh" && url.username() == "git" && url.password().is_none() {
return Cow::Borrowed(url);
/// A [`Url`] wrapper that redacts credentials when displaying the URL.
///
/// `DisplaySafeUrl` wraps the standard [`url::Url`] type, providing functionality to mask
/// secrets by default when the URL is displayed or logged. This helps prevent accidental
/// exposure of sensitive information in logs and debug output.
///
/// # Examples
///
/// ```
/// use uv_redacted::DisplaySafeUrl;
/// use std::str::FromStr;
///
/// // Create a `DisplaySafeUrl` from a `&str`
/// let mut url = DisplaySafeUrl::parse("https://user:password@example.com").unwrap();
///
/// // Display will mask secrets
/// assert_eq!(url.to_string(), "https://user:****@example.com/");
///
/// // You can still access the username and password
/// assert_eq!(url.username(), "user");
/// assert_eq!(url.password(), Some("password"));
///
/// // And you can still update the username and password
/// let _ = url.set_username("new_user");
/// let _ = url.set_password(Some("new_password"));
/// assert_eq!(url.username(), "new_user");
/// assert_eq!(url.password(), Some("new_password"));
///
/// // It is also possible to remove the credentials entirely
/// url.remove_credentials();
/// assert_eq!(url.username(), "");
/// assert_eq!(url.password(), None);
/// ```
#[derive(Clone, Eq, PartialEq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
#[cfg_attr(feature = "schemars", schemars(transparent))]
pub struct DisplaySafeUrl(Url);
impl DisplaySafeUrl {
#[inline]
pub fn parse(input: &str) -> Result<Self, url::ParseError> {
Ok(Self(Url::parse(input)?))
}
let mut url = url.clone();
let _ = url.set_username("");
let _ = url.set_password(None);
Cow::Owned(url)
/// Parse a string as an URL, with this URL as the base URL.
#[inline]
pub fn join(&self, input: &str) -> Result<Self, url::ParseError> {
self.0.join(input).map(DisplaySafeUrl::from)
}
/// Serialize with Serde using the internal representation of the `Url` struct.
#[inline]
pub fn serialize_internal<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
self.0.serialize_internal(serializer)
}
/// Serialize with Serde using the internal representation of the `Url` struct.
#[inline]
pub fn deserialize_internal<'de, D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
Url::deserialize_internal(deserializer).map(DisplaySafeUrl::from)
}
#[allow(clippy::result_unit_err)]
pub fn from_file_path<P: AsRef<std::path::Path>>(path: P) -> Result<DisplaySafeUrl, ()> {
Url::from_file_path(path).map(DisplaySafeUrl::from)
}
/// Remove the credentials from a URL, allowing the generic `git` username (without a password)
/// in SSH URLs, as in, `ssh://git@github.com/...`.
#[inline]
pub fn remove_credentials(&mut self) {
// For URLs that use the `git` convention (i.e., `ssh://git@github.com/...`), avoid dropping the
// username.
if self.0.scheme() == "ssh" && self.0.username() == "git" && self.0.password().is_none() {
return;
}
let _ = self.0.set_username("");
let _ = self.0.set_password(None);
}
/// Returns string representation without masking credentials.
#[inline]
pub fn to_string_with_credentials(&self) -> String {
self.0.to_string()
}
}
impl Deref for DisplaySafeUrl {
type Target = Url;
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl DerefMut for DisplaySafeUrl {
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.0
}
}
impl std::fmt::Display for DisplaySafeUrl {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
fmt_with_obfuscated_credentials(&self.0, f)
}
}
impl std::fmt::Debug for DisplaySafeUrl {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{self}")
}
}
impl From<Url> for DisplaySafeUrl {
fn from(url: Url) -> Self {
DisplaySafeUrl(url)
}
}
impl From<DisplaySafeUrl> for Url {
fn from(url: DisplaySafeUrl) -> Self {
url.0
}
}
impl FromStr for DisplaySafeUrl {
type Err = url::ParseError;
fn from_str(input: &str) -> Result<Self, Self::Err> {
Ok(Self(Url::from_str(input)?))
}
}
fn fmt_with_obfuscated_credentials<W: Write>(url: &Url, mut f: W) -> std::fmt::Result {
if url.password().is_none() && url.username() == "" {
return write!(f, "{url}");
}
write!(f, "{}://", url.scheme())?;
if url.username() != "" && url.password().is_some() {
write!(f, "{}", url.username())?;
write!(f, ":****@")?;
} else if url.username() != "" {
write!(f, "****@")?;
} else if url.password().is_some() {
write!(f, ":****@")?;
}
write!(f, "{}", url.host_str().unwrap_or(""))?;
if let Some(port) = url.port() {
write!(f, ":{port}")?;
}
write!(f, "{}", url.path())?;
if let Some(query) = url.query() {
write!(f, "?{query}")?;
}
if let Some(fragment) = url.fragment() {
write!(f, "#{fragment}")?;
}
Ok(())
}
/// A wrapper around a [`url::Url`] ref that safely handles credentials for
/// logging purposes.
///
/// Uses the same underlying [`Display`] implementation as [`DisplaySafeUrl`].
///
/// # Examples
///
/// ```
/// use uv_redacted::DisplaySafeUrl;
/// use std::str::FromStr;
///
/// // Create from a `url::Url` ref
/// let url = Url::parse("https://user:password@example.com").unwrap();
/// let log_safe_url = DisplaySafeUrlRef::from(&url);
///
/// // Display will mask secrets
/// assert_eq!(url.to_string(), "https://user:****@example.com/");
///
/// // Since `DisplaySafeUrlRef` provides full access to the underlying `Url` through a
/// // `Deref` implementation, you can still access the username and password
/// assert_eq!(url.username(), "user");
/// assert_eq!(url.password(), Some("password"));
pub struct DisplaySafeUrlRef<'a>(&'a Url);
impl<'a> Deref for DisplaySafeUrlRef<'a> {
type Target = Url;
fn deref(&self) -> &'a Self::Target {
self.0
}
}
impl std::fmt::Display for DisplaySafeUrlRef<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
fmt_with_obfuscated_credentials(self.0, f)
}
}
impl std::fmt::Debug for DisplaySafeUrlRef<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{self}")
}
}
impl<'a> From<&'a Url> for DisplaySafeUrlRef<'a> {
fn from(url: &'a Url) -> Self {
DisplaySafeUrlRef(url)
}
}
impl<'a> From<&'a DisplaySafeUrl> for DisplaySafeUrlRef<'a> {
fn from(url: &'a DisplaySafeUrl) -> Self {
DisplaySafeUrlRef(url)
}
}
impl<'a> From<DisplaySafeUrlRef<'a>> for DisplaySafeUrl {
fn from(url: DisplaySafeUrlRef<'a>) -> Self {
DisplaySafeUrl(url.0.clone())
}
}
#[cfg(test)]
@@ -24,49 +244,103 @@ mod tests {
#[test]
fn from_url_no_credentials() {
let url = Url::parse("https://pypi-proxy.fly.dev/basic-auth/simple").unwrap();
let redacted = redacted_url(&url);
assert_eq!(redacted.username(), "");
assert!(redacted.password().is_none());
assert_eq!(
format!("{redacted}"),
"https://pypi-proxy.fly.dev/basic-auth/simple"
);
let url_str = "https://pypi-proxy.fly.dev/basic-auth/simple";
let url = Url::parse(url_str).unwrap();
let log_safe_url = DisplaySafeUrl::from(url);
assert_eq!(log_safe_url.username(), "");
assert!(log_safe_url.password().is_none());
assert_eq!(format!("{log_safe_url}"), url_str);
}
#[test]
fn from_url_username_and_password() {
let url = Url::parse("https://user:pass@pypi-proxy.fly.dev/basic-auth/simple").unwrap();
let redacted = redacted_url(&url);
assert_eq!(redacted.username(), "");
assert!(redacted.password().is_none());
let url_str = "https://user:pass@pypi-proxy.fly.dev/basic-auth/simple";
let url = Url::parse(url_str).unwrap();
let log_safe_url = DisplaySafeUrl::from(url);
assert_eq!(log_safe_url.username(), "user");
assert!(log_safe_url.password().is_some_and(|p| p == "pass"));
assert_eq!(
format!("{redacted}"),
"https://pypi-proxy.fly.dev/basic-auth/simple"
format!("{log_safe_url}"),
"https://user:****@pypi-proxy.fly.dev/basic-auth/simple"
);
}
#[test]
fn from_url_just_password() {
let url = Url::parse("https://:pass@pypi-proxy.fly.dev/basic-auth/simple").unwrap();
let redacted = redacted_url(&url);
assert_eq!(redacted.username(), "");
assert!(redacted.password().is_none());
let url_str = "https://:pass@pypi-proxy.fly.dev/basic-auth/simple";
let url = Url::parse(url_str).unwrap();
let log_safe_url = DisplaySafeUrl::from(url);
assert_eq!(log_safe_url.username(), "");
assert!(log_safe_url.password().is_some_and(|p| p == "pass"));
assert_eq!(
format!("{redacted}"),
"https://pypi-proxy.fly.dev/basic-auth/simple"
format!("{log_safe_url}"),
"https://:****@pypi-proxy.fly.dev/basic-auth/simple"
);
}
#[test]
fn from_url_just_username() {
let url = Url::parse("https://user@pypi-proxy.fly.dev/basic-auth/simple").unwrap();
let redacted = redacted_url(&url);
assert_eq!(redacted.username(), "");
assert!(redacted.password().is_none());
let url_str = "https://user@pypi-proxy.fly.dev/basic-auth/simple";
let url = Url::parse(url_str).unwrap();
let log_safe_url = DisplaySafeUrl::from(url);
assert_eq!(log_safe_url.username(), "user");
assert!(log_safe_url.password().is_none());
assert_eq!(
format!("{redacted}"),
format!("{log_safe_url}"),
"https://****@pypi-proxy.fly.dev/basic-auth/simple"
);
}
#[test]
fn parse_url_string() {
let url_str = "https://user:pass@pypi-proxy.fly.dev/basic-auth/simple";
let log_safe_url = DisplaySafeUrl::parse(url_str).unwrap();
assert_eq!(log_safe_url.username(), "user");
assert!(log_safe_url.password().is_some_and(|p| p == "pass"));
assert_eq!(
format!("{log_safe_url}"),
"https://user:****@pypi-proxy.fly.dev/basic-auth/simple"
);
}
#[test]
fn remove_credentials() {
let url_str = "https://user:pass@pypi-proxy.fly.dev/basic-auth/simple";
let mut log_safe_url = DisplaySafeUrl::parse(url_str).unwrap();
log_safe_url.remove_credentials();
assert_eq!(log_safe_url.username(), "");
assert!(log_safe_url.password().is_none());
assert_eq!(
format!("{log_safe_url}"),
"https://pypi-proxy.fly.dev/basic-auth/simple"
);
}
#[test]
fn to_string_with_credentials() {
let url_str = "https://user:pass@pypi-proxy.fly.dev/basic-auth/simple";
let log_safe_url = DisplaySafeUrl::parse(url_str).unwrap();
assert_eq!(&log_safe_url.to_string_with_credentials(), url_str);
}
#[test]
fn url_join() {
let url_str = "https://token@example.com/abc/";
let log_safe_url = DisplaySafeUrl::parse(url_str).unwrap();
let foo_url = log_safe_url.join("foo").unwrap();
assert_eq!(format!("{foo_url}"), "https://****@example.com/abc/foo");
}
#[test]
fn log_safe_url_ref() {
let url_str = "https://user:pass@pypi-proxy.fly.dev/basic-auth/simple";
let url = Url::parse(url_str).unwrap();
let log_safe_url = DisplaySafeUrlRef::from(&url);
assert_eq!(log_safe_url.username(), "user");
assert!(log_safe_url.password().is_some_and(|p| p == "pass"));
assert_eq!(
format!("{log_safe_url}"),
"https://user:****@pypi-proxy.fly.dev/basic-auth/simple"
);
}
}