~tsileo/blobsfile

ref: 76a89107299fb76b273d92e13602ab1822aa4f76 blobsfile/README.md -rw-r--r-- 1.1 KiB
76a89107Thomas Sileo Bugfixes 2 years ago

#BlobsFile

builds.sr.ht status    Godoc Reference

BlobsFile is an append-only (i.e. no update and no delete) content-addressed blob store (using BLAKE2b as hash function).

It draws inspiration from Facebook's Haystack, blobs are stored in flat files (called BlobFile) and indexed by a small kv database for fast lookup.

BlobsFile is BlobStash's storage engine.

#Features

  • Durable (data is fsynced before returning)
  • Immutable (append-only, can't mutate or delete blobs)
  • Optional compression (Snappy or Zstandard)
  • Extra parity data is added to each BlobFile (using Reed-Solomon error correcting code), allowing the database to repair itself in case of corruption.
    • The test suite is literraly punching holes at random places