Floating cheeses are collections of carefully handpicked cheeses from the cheese shop and other sources.
Analogies aside, the interplanetary wheels (IPWHL) are platform-unique, single-versioned Python binary distributions backed by IPFS. It aims to be a downstream wheel supplier in a similar fashion to GNU/Linux distributions, whilst take advantage of a content-addressing peer-to-peer network to provide a reproducible, easy-to-mirror source of packages.
On IPWHL, for each platform (architecture, operating system and Python implementation and version), there exists only one single binary distribution. The collection of these distribution packages are given as a single IPFS CID. An installer can use solely this content ID and packages names to reproduce the exactly same environment on every platform.
The official IPWHL repository will provide exclusively free software. However, deriving the repository should be trivial and is a supported use case.
The cheese shop is great, but choosing cheeses from it can often be confusing. Dependency resolution is expensive, and version requirements are not future-proof. In order to avoid breakage, people usually have to pin packages on the installer side, which is redundant and difficult to validate manually. Additionally, we believe it is not the packaging users' job to do this; they should be able to save their time doing what they do best: writing and using software.
Moreover, there are millions of ways for a piece of cheese to rot on the way home from the (almost) lawless cheese shop. Everyone can sell at the shop, and thus typosquatting is a common exploit. In addition, cheeses from the shop are not independently verifiable: the checksums are provided along with the files so the shop is the single point of failure for security attacks. There are ongoing efforts to integrate TUF into Python packaging toolchain, however it is unlikely that they can entirely mitigate this due to the centralized nature of the inherent architecture.
Centralization also makes it really difficult for mirrors to be useful for the users: the cheese shop is not aware if any of its mirrors, let alone redirecting to the closest one. Mirroring is hardly a collaborative effort, one either provide everything for an entire region, or give up. On the other hand, many organizations host their Python packages and their dependencies on dedicated machines running 24/7, but the resources are mostly gone to waste when unused by the companies themselves.
IPWHL makes use of IPFS and statically declared and carefully curated metadata to try to solve most the listed problems. In addition to providing only one wheel version at a time, source distributions are not supplied to avoiding executing untrusted code on the users' machine.
In order to add a wheel file to IPWHL, it must be statically declared and checked, then the declaration collection shall be published.
A IPWHL declaration is a flat TOML file consisting of the following fields:
source: Upstream URL to the wheel
content-id: CIDv0 of the wheel
requires-python(optional): Specifier set representing Python versions the wheel is compatible with, read from
Requires-Pythonin metadata 1.2
extras: Array of optional features, read from
Provides-Extrain metadata 2.1
dependencies: Array of requirements from
Requires-Distin metadata 1.2, converted to PEP 508 format
The file shall be placed in the following path:
project is the (pre-normalized) project name,
prefix is the smallest
8 bits in lowercase hexadecimal of the sum of characters in
wheelname is the file name of the wheel as specified
in PEP 427.
The following criteria are used to check the paths to declarations:
$prefix/$projectas described above.
.tomland were it
.whlthe hypothetical name must make up a valid wheel filename according to PEP 427.
The integrity of a declaration is verified by comparing the original wheel with the declared fields. Before performing this, one must ensure that the source URL has not been compromised. Due to the lack of warranty for the long-term availability of the source and to save resources, an implementation may only do this for recently modified declarations, e.g. ones added or updated by a patch.
Every wheels declared must have their dependencies, including extras, satisfied on every supported platform.
If all checks pass, a new repository release can be published from the declarations.
For a fresh build, every wheel declared is downloaded and placed according to PEP 503:
The parent directory is then recursively added to IPFS. To leverage subsequent builds, IPFS mutable file system (MFS) can be utilized for caching: only new declarations and those not matching the content ID are downloaded and added to the previous repository version. This way, we need not rely on the upstream URLs' availability; however wheels from declarations having been updated or removed must be garbage collected.
Should MFS be used, the following files and directories must be removed to maintain the property of being platform-unique and single-versioned of IPWHL:
The mainline Git trees are hosted on SourceHut:
In ascending order of traffic:
Bugs and requests are tracked on SourceHut TODO, which also supports submitting via email to ~firstname.lastname@example.org.
The maintainers of the floating cheeses can be found on Matrix at #ipwhl:matrix.org. However, there is no concept of an official chat room, so feel free to create ones on other platforms. We will consider bridging them to the Matrix room upon requests.