~boringcactus/arewe1.0yet

f8383e59ceb16a37f3dbea7ec103552168376fa0 — Melody Horn 3 years ago
initial website logic
6 files changed, 274 insertions(+), 0 deletions(-)

A .build.yml
A .gitignore
A LICENSE
A index.html
A requirements.txt
A update.py
A  => .build.yml +20 -0
@@ 1,20 @@
image: alpine/latest
packages:
  - py3-pip
  - rsync
sources:
  - https://git.sr.ht/~boringcactus/arewe1.0yet
environment:
  deploy: services@boringcactus.com
secrets:
  - b5cb9b2b-1461-4486-95e1-886451674a89
tasks:
  - install: |
      cd arewe1.0yet
      python3 -m pip install -r requirements.txt
  - build: |
      cd arewe1.0yet
      python3 update.py
  - deploy: |
      cd arewe1.0yet
      rsync --rsh="ssh -o StrictHostKeyChecking=no" -rlt8hP --del out/ $deploy:/var/www/html/1.0.boringcactus.com/

A  => .gitignore +4 -0
@@ 1,4 @@
/*.tar.gz
/out/
/venv
/.idea

A  => LICENSE +25 -0
@@ 1,25 @@
               GLWT(Good Luck With That) Public License
                 Copyright (c) Everyone, except Author

Everyone is permitted to copy, distribute, modify, merge, sell, publish,
sublicense or whatever they want with this software but at their OWN RISK.

                            Preamble

The author has absolutely no clue what the code in this project does.
It might just work or not, there is no third option.


                GOOD LUCK WITH THAT PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION, AND MODIFICATION

  0. You just DO WHATEVER YOU WANT TO as long as you NEVER LEAVE A
TRACE TO TRACK THE AUTHOR of the original product to blame for or hold
responsible.

IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.

Good luck and Godspeed.

A  => index.html +96 -0
@@ 1,96 @@
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Are We 1.0 Yet?</title>
    <style>
        /* "derived" (stolen) from evenbettermotherfucking.website */
        html {
            margin: 1rem auto;
            background: #f2f2f2;
            color: #444444;
            font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
            font-size: 16px;
            line-height: 1.8;
            text-shadow: 0 1px 0 #ffffff;
            max-width: 60em;
        }
        body {
            margin: 0 1rem;
        }
        a {
            border-bottom: 1px solid #444444;
            color: #444444;
            text-decoration: none;
        }
        a:hover {
            border-bottom-style: dashed;
        }
        blockquote {
            margin-left: 1em;
            border-left: 2px solid #444444;
            padding-left: 1em;
        }

        .highlight {
            background-color: #c2f2c2;
        }
    </style>
</head>
<body>
<main>
    <h1>Are We 1.0 Yet?</h1>
    <p>
        Checking the 360 most downloaded crates of all time on <a href="https://crates.io">crates.io</a> to see which ones have reached version 1.0 yet.
    </p>
    <p>
        <strong>{{ crates | map(attribute='latest_version') | selectattr('is_1_0') | list | length }} / {{ crates | length }}</strong> crates are 1.0 by now.
    </p>
    <table>
        <thead>
        <tr>
            <th>Name</th>
            <th>Latest Version</th>
        </tr>
        </thead>
        <tbody>
        {% for crate in crates %}
            <tr class="{% if crate.latest_version.is_1_0 %}highlight{% endif %}">
                <td><a href="https://crates.io/crates/{{ crate.name }}">{{ crate.name }}</a></td>
                <td>{{ crate.latest_version }}{% if crate.latest_version != crate.latest_pre_release_version %}
                    (& <span class="{% if crate.latest_pre_release_version.is_1_0 %}highlight{% endif %}">{{ crate.latest_pre_release_version }})</span>{% endif %}</td>
            </tr>
        {% endfor %}
        </tbody>
    </table>
</main>
<aside>
    <h2>Why?</h2>
    <p>
        The Rust ecosystem is still very immature.
        Very few critical packages have actually reached version 1.0 yet.
        The <a href="https://semver.org/#spec-item-4">semver spec</a> says "Major version zero (0.y.z) is for initial development... The public API SHOULD NOT be considered stable."
        Additionally, the <a href="https://semver.org/#how-do-i-know-when-to-release-100">FAQ entry</a> for "How do I know when to release 1.0.0?" gives some heuristics that the Rust ecosystem doesn't really abide by at all:
    </p>
    <blockquote>
        <p>
            If your software is being used in production, it should probably already be 1.0.0.
            If you have a stable API on which users have come to depend, you should be 1.0.0.
            If you’re worrying a lot about backwards compatibility, you should probably already be 1.0.0.
        </p>
    </blockquote>
    <p>
        I submit that it would be difficult to build a non-trivial Rust program intended for production without depending on any crates which are not yet at version 1.0.0 or higher.
        This situation is not really ideal.
        I'm not saying the maintainers of these crates are lazy, or owe the community a stable 1.0.0 release, or anything like that.
        I'm just saying ecosystem stability is a good thing to care about, and right now Rust in general doesn't have that.
    </p>
</aside>
<footer>
    Built by <a href="https://www.boringcactus.com/">boringcactus</a>.
    Inspired by <a href="https://pythonwheels.com/">Python Wheels</a>.
    Generator code <a href="https://git.sr.ht/~boringcactus/arewe1.0yet">available</a>.
    Data pulled at {{ metadata.timestamp }}.
</footer>
</body>
</html>
\ No newline at end of file

A  => requirements.txt +1 -0
@@ 1,1 @@
Jinja2

A  => update.py +128 -0
@@ 1,128 @@
import csv
from dataclasses import dataclass
import datetime
from functools import total_ordering
import io
import json
from pathlib import Path
import tarfile
import urllib.request

from jinja2 import Environment, FileSystemLoader, StrictUndefined, select_autoescape


@total_ordering
@dataclass()
class Version:
    major: int
    minor: int
    patch: int
    pre_release: str

    def __init__(self, text):
        text = text.split('+')[0]
        core, self.pre_release = (text.split('-') + [''])[:2]
        self.major, self.minor, self.patch = [int(x) for x in core.split('.')]

    def __str__(self):
        pre_release = self.pre_release
        if len(pre_release) > 0:
            pre_release = '-' + pre_release
        return '{}.{}.{}{}'.format(self.major, self.minor, self.patch, pre_release)

    def __lt__(self, other: 'Version') -> bool:
        if self.major != other.major:
            return self.major < other.major
        if self.minor != other.minor:
            return self.minor < other.minor
        if self.patch != other.patch:
            return self.patch < other.patch
        if self.pre_release == '' and other.pre_release != '':
            return False
        if self.pre_release != '' and other.pre_release == '':
            return True

        def pre_release_lt(a: str, b: str):
            if len(a) == 0 and len(b) != 0:
                return True
            if len(b) == 0:
                return False
            a1, an = (a.split('.', 2) + [''])[:2]
            b1, bn = (b.split('.', 2) + [''])[:2]
            try:
                a1, b1 = int(a1), int(b1)
            except ValueError:
                pass
            if a1 < b1:
                return True
            elif a1 > b1:
                return False
            else:
                return pre_release_lt(an, bn)
        return pre_release_lt(self.pre_release, other.pre_release)

    @property
    def is_1_0(self):
        return self.major >= 1


@dataclass()
class Crate:
    name: str
    downloads: int
    latest_version: Version = None
    latest_pre_release_version: Version = None


today = datetime.date.today().strftime('%Y-%m-%d')

dump_tarball = Path(f'db-dump-{today}.tar.gz')
if not dump_tarball.exists():
    with urllib.request.urlopen('https://static.crates.io/db-dump.tar.gz') as f:
        dump_tarball.write_bytes(f.read())

csv.field_size_limit(69696969)
dump = tarfile.open(dump_tarball)
crates = dict()
metadata = None
for item in dump:
    if item.name.endswith('metadata.json'):
        metadata = json.load(dump.extractfile(item))
    elif item.name.endswith('crates.csv'):
        reader = csv.DictReader(io.TextIOWrapper(dump.extractfile(item), 'UTF-8'))
        for crate in reader:
            crates[crate['id']] = Crate(crate['name'], int(crate['downloads']))
    elif item.name.endswith('versions.csv'):
        assert len(crates) > 0, "versions read before crates!"
        reader = csv.DictReader(io.TextIOWrapper(dump.extractfile(item), 'UTF-8'))
        for version in reader:
            if version['yanked'] == 't':
                continue
            crate = crates[version['crate_id']]
            this_version = Version(version['num'])
            if crate.latest_pre_release_version is None or crate.latest_pre_release_version < this_version:
                crate.latest_pre_release_version = this_version
            if this_version.pre_release == '':
                if crate.latest_version is None or crate.latest_version < this_version:
                    crate.latest_version = this_version
        versions = list(reader)

most_downloaded_crates = sorted(crates.values(), key=lambda x: x.downloads, reverse=True)

crates = most_downloaded_crates[:360]

print('{}/{} crates at or above version 1.0'.format(sum(1 for crate in crates if crate.latest_version.is_1_0),
                                                    len(crates)))

env = Environment(
    loader=FileSystemLoader('.'),
    autoescape=select_autoescape(['html', 'xml']),
    undefined=StrictUndefined
)
index_template = env.get_template('index.html')
rendered_index = index_template.render(crates=crates, metadata=metadata)

out_file = Path('out', 'index.html')
out_file.parent.mkdir(parents=True, exist_ok=True)
with open(out_file, 'w') as f:
    f.write(rendered_index)