ad6c3bff4ba810f46e6e5941b986f14d00879d98 — Sebastien Binet 6 months ago fbed429
2021-11-16-ji-groot: first import
32 files changed, 710 insertions(+), 0 deletions(-)

A 2021/2021-11-16-ji-groot/README.md
A 2021/2021-11-16-ji-groot/_code/big-o-timings.txt
A 2021/2021-11-16-ji-groot/_code/binary-blob.txt
A 2021/2021-11-16-ji-groot/_code/croot-fail.txt
A 2021/2021-11-16-ji-groot/_code/groot-bench-results.txt
A 2021/2021-11-16-ji-groot/_code/hello.cxx
A 2021/2021-11-16-ji-groot/_code/hello.f
A 2021/2021-11-16-ji-groot/_code/hello.py
A 2021/2021-11-16-ji-groot/_code/perf-v0.26.txt
A 2021/2021-11-16-ji-groot/_code/perf-v0.27.txt
A 2021/2021-11-16-ji-groot/_code/storage-1.txt
A 2021/2021-11-16-ji-groot/_code/storage-2.txt
A 2021/2021-11-16-ji-groot/_code/storage-3.txt
A 2021/2021-11-16-ji-groot/_code/streamer-p3.txt
A 2021/2021-11-16-ji-groot/_code/streamers.txt
A 2021/2021-11-16-ji-groot/_code/struct-p3.c
A 2021/2021-11-16-ji-groot/_code/tfile-tkey-1.go
A 2021/2021-11-16-ji-groot/_code/tkey.txt
A 2021/2021-11-16-ji-groot/_code/tree-r.cxx
A 2021/2021-11-16-ji-groot/_code/tree-r.go
A 2021/2021-11-16-ji-groot/_code/tree-w.cxx
A 2021/2021-11-16-ji-groot/_code/tree.cxx
A 2021/2021-11-16-ji-groot/_figs/file-records.png
A 2021/2021-11-16-ji-groot/_figs/file_layout.jpg
A 2021/2021-11-16-ji-groot/_figs/gophereartrumpet.jpg
A 2021/2021-11-16-ji-groot/_figs/groot-bkreader.dia
A 2021/2021-11-16-ji-groot/_figs/groot-bkreader.png
A 2021/2021-11-16-ji-groot/_figs/groot.jpg
A 2021/2021-11-16-ji-groot/_figs/my-python-logo.png
A 2021/2021-11-16-ji-groot/_figs/my-root6splash.png
A 2021/2021-11-16-ji-groot/_figs/ttree.png
A 2021/2021-11-16-ji-groot/talk.slide
A 2021/2021-11-16-ji-groot/README.md => 2021/2021-11-16-ji-groot/README.md +3 -0
@@ 0,0 1,3 @@
# 2021-11-16-ji-groot

`go-present` link: [slides](https://talks.sbinet.org/2021/2021-11-16-ji-groot/talk.slide)

A 2021/2021-11-16-ji-groot/_code/big-o-timings.txt => 2021/2021-11-16-ji-groot/_code/big-o-timings.txt +22 -0
@@ 0,0 1,22 @@
           0.5 ns - CPU L1 dCACHE reference
           1   ns - speed-of-light (a photon) travel a 1 ft (30.5cm) distance
           5   ns - CPU L1 iCACHE Branch mispredict
           7   ns - CPU L2  CACHE reference
          71   ns - CPU cross-QPI/NUMA best  case on XEON E5-46*
         100   ns - MUTEX lock/unlock
         100   ns - own DDR MEMORY reference
         135   ns - CPU cross-QPI/NUMA best  case on XEON E7-*
         202   ns - CPU cross-QPI/NUMA worst case on XEON E7-*
         325   ns - CPU cross-QPI/NUMA worst case on XEON E5-46*
      10,000   ns - Compress 1K bytes with Zippy PROCESS
      20,000   ns - Send 2K bytes over 1 Gbps NETWORK
     250,000   ns - Read 1 MB sequentially from MEMORY
     500,000   ns - Round trip within a same DataCenter
  10,000,000   ns - DISK seek
  10,000,000   ns - Read 1 MB sequentially from NETWORK
  30,000,000   ns - Read 1 MB sequentially from DISK
 150,000,000   ns - Send a NETWORK packet CA -> Netherlands
|   |   |   |
|   |   | ns|
|   | us|
| ms|

A 2021/2021-11-16-ji-groot/_code/binary-blob.txt => 2021/2021-11-16-ji-groot/_code/binary-blob.txt +3 -0
@@ 0,0 1,3 @@
[]byte{42, 42,  1, 154,  2, 0,  0,  0, 4,  0,  0,   0,  0, 0,  0, 24,
       45, 68, 84, 251, 33, 9, 64,

A 2021/2021-11-16-ji-groot/_code/croot-fail.txt => 2021/2021-11-16-ji-groot/_code/croot-fail.txt +2 -0
@@ 0,0 1,2 @@
## may fail because of missing dependencies
$> go get github.com/go-hep/croot

A 2021/2021-11-16-ji-groot/_code/groot-bench-results.txt => 2021/2021-11-16-ji-groot/_code/groot-bench-results.txt +16 -0
@@ 0,0 1,16 @@
## CMS data

name                                  time/op
ReadCMSScalar/GoHEP/Zlib-8            3.92s ± 2%  // only read scalar data
ReadCMSScalar/ROOT-TreeBranch/Zlib-8  7.98s ± 2%  // ditto
ReadCMSScalar/ROOT-TreeReader/Zlib-8  6.60s ± 2%  // ditto

name                                  time/op
ReadCMSAll/GoHEP/Zlib-8               18.4s ± 1%  // read all branches
ReadCMSAll/ROOT-TreeBranch/Zlib-8     30.4s ± 2%  // ditto
ReadCMSAll/ROOT-TreeReader/Zlib-8     [N/A]       // comparison meaningless (b/c of loading-on-demand)

## ATLAS data

1.6 ms/kEvt (1.1 s for 720 kEvts)  [groot v0.27]
2.6 ms/kEvt (1.9 s for 720 kEvts)  [ROOT  v6.20]

A 2021/2021-11-16-ji-groot/_code/hello.cxx => 2021/2021-11-16-ji-groot/_code/hello.cxx +5 -0
@@ 0,0 1,5 @@
#include <iostream>
int main(int, char **) {
  std::cout << "Hello from C++" << std::endl;
  return EXIT_SUCCESS;

A 2021/2021-11-16-ji-groot/_code/hello.f => 2021/2021-11-16-ji-groot/_code/hello.f +6 -0
@@ 0,0 1,6 @@
c     == hello.f ==
      program main
      implicit none
      write ( *, '(a)' ) 'Hello from FORTRAN'

A 2021/2021-11-16-ji-groot/_code/hello.py => 2021/2021-11-16-ji-groot/_code/hello.py +1 -0
@@ 0,0 1,1 @@
print "Hello from python"

A 2021/2021-11-16-ji-groot/_code/perf-v0.26.txt => 2021/2021-11-16-ji-groot/_code/perf-v0.26.txt +2 -0
@@ 0,0 1,2 @@
5.2 ms/kEvt (3.7 s for 720 kEvts)  [groot v0.26]
2.6 ms/kEvt (1.9 s for 720 kEvts)  [ROOT  v6.20]

A 2021/2021-11-16-ji-groot/_code/perf-v0.27.txt => 2021/2021-11-16-ji-groot/_code/perf-v0.27.txt +2 -0
@@ 0,0 1,2 @@
1.6 ms/kEvt (1.1 s for 720 kEvts)  [groot v0.27]
2.6 ms/kEvt (1.9 s for 720 kEvts)  [ROOT  v6.20]

A 2021/2021-11-16-ji-groot/_code/storage-1.txt => 2021/2021-11-16-ji-groot/_code/storage-1.txt +1 -0
@@ 0,0 1,1 @@
[n, d], [n, d], [n, d], [n, d], ...

A 2021/2021-11-16-ji-groot/_code/storage-2.txt => 2021/2021-11-16-ji-groot/_code/storage-2.txt +1 -0
@@ 0,0 1,1 @@
[n, n, n, n, ...], [d, d, d, d, ...]

A 2021/2021-11-16-ji-groot/_code/storage-3.txt => 2021/2021-11-16-ji-groot/_code/storage-3.txt +2 -0
@@ 0,0 1,2 @@
[n, n, n, n, ...], [d.i32, d.i32, d.i32, d.i32, ...], 
[d.i64, d.i64, d.i64, d.i64, ...], [d.f64, d.f64, d.f64, d.f64, ...]

A 2021/2021-11-16-ji-groot/_code/streamer-p3.txt => 2021/2021-11-16-ji-groot/_code/streamer-p3.txt +4 -0
@@ 0,0 1,4 @@
StreamerInfo for "P3" version=1 title=""
 int    Px      offset=  0 type=  3 size=  4  
 double Py      offset=  4 type=  8 size=  8  
 int    Pz      offset= 12 type=  3 size=  4  

A 2021/2021-11-16-ji-groot/_code/streamers.txt => 2021/2021-11-16-ji-groot/_code/streamers.txt +13 -0
@@ 0,0 1,13 @@
StreamerElement              // encodes name, size, shape, min/max, ... 
+-- StreamerBase             // base class
+-- StreamerBasicType        // builtin type (char, uint16, ...)
+-- StreamerBasicPointer     // pointer to a builtin type
+-- StreamerLoop             // repeats of a type
+-- StreamerObject           // a TObject
+-- StreamerObjectAny        // a user class
+-- StreamerObjectAnyPointer // pointer to a user class
+-- StreamerObjectPointer    // pointer to a TObject
+-- StreamerSTL              // STL container (vector/set/map/pair/unordered_xxx/...)
+-- StreamerSTLstring        // std::string
+-- StreamerString           // TString

A 2021/2021-11-16-ji-groot/_code/struct-p3.c => 2021/2021-11-16-ji-groot/_code/struct-p3.c +5 -0
@@ 0,0 1,5 @@
struct P3 {
    int    Px;
    double Py;
    int    Pz;

A 2021/2021-11-16-ji-groot/_code/tfile-tkey-1.go => 2021/2021-11-16-ji-groot/_code/tfile-tkey-1.go +14 -0
@@ 0,0 1,14 @@
f, err := groot.Create("out.root")
if err != nil {
defer f.Close()

for i, evt := range detector.Readout() {
  log.Printf("recording event %d...", i)
  key := fmt.Sprintf("evt-%03d", i)
  err := f.Put(key, evt) // HL
  if err != nil {

A 2021/2021-11-16-ji-groot/_code/tkey.txt => 2021/2021-11-16-ji-groot/_code/tkey.txt +14 -0
@@ 0,0 1,14 @@
| Data Member | Explanation |
|  fNbytes    | Number of bytes for the compressed object and key. |
|  fObjlen    | Length of uncompressed object. |
|  fDatime    | Date/Time when the object was written. |
|  fKeylen    | Number of bytes for the key structure. |
|  fCycle     | Cycle number of the object. |
|  fSeekKey   | Address of the object on file (points to fNbytes).
|             | This is a redundant information used to cross-check the
|             | data base integrity. |
|  fSeekPdir  | Pointer to the directory supporting this object.|
|  fClassName | Object class name. |
|  fName      | Name of the object. |
|  fTitle     | Title of the object. |

A 2021/2021-11-16-ji-groot/_code/tree-r.cxx => 2021/2021-11-16-ji-groot/_code/tree-r.cxx +18 -0
@@ 0,0 1,18 @@
auto f = TFile::Open("out.root", "READ");
auto t = f->Get<TTree>("t");

auto n = int32_t(0);
auto d = struct{
    int32_t i32;
	int64_t i64;
	double  f64;

t->SetBranchAddress("n", &n); // HL
t->SetBranchAddress("d", &d); // HL

for (int64_t i = 0; i < t->GetEntries(); i++) {
	printf("evt=%d, n=%d, d.i32=%d, d.i64=%d, d.f64=%f\n",
			i, n, d.i32, d.i64, d.f64);

A 2021/2021-11-16-ji-groot/_code/tree-r.go => 2021/2021-11-16-ji-groot/_code/tree-r.go +36 -0
@@ 0,0 1,36 @@
package main

import (


func main() {
	f, err := groot.Open("f.root")
	defer f.Close()
	o, err := f.Get("t")

	v := struct {
		N int32 `groot:"n"`
		D struct {
			I32 int32   `groot:"i32"`
			I64 int64   `groot:"i64"`
			F64 float64 `groot:"f64"`
		} `groot:"d"`

	r, err := rtree.NewReader(o.(rtree.Tree), rtree.ReadVarsFromStruct(&v))
	defer r.Close()

	err = r.Read(func(ctx rtree.RCtx) error { // HL
			"evt=%d, n=%d, d.i32=%d, d.i64=%d, d.f64=%v\n",
			ctx.Entry, v.N, v.D.I32, v.D.I64, v.D.F64,
		return nil

A 2021/2021-11-16-ji-groot/_code/tree-w.cxx => 2021/2021-11-16-ji-groot/_code/tree-w.cxx +12 -0
@@ 0,0 1,12 @@
auto t = new TTree("t", "my tree");
auto n = int32_t(0);
auto d = struct{
    int32_t i32;
	int64_t i64;
	double  f64;
t->Branch("n", &n, "n/I"); // HL
t->Branch("d", &d);        // HL

// -> leaf_n = TLeaf<int32_t>(t, "n");
// -> leaf_d = TLeaf<struct> (t, "d");

A 2021/2021-11-16-ji-groot/_code/tree.cxx => 2021/2021-11-16-ji-groot/_code/tree.cxx +26 -0
@@ 0,0 1,26 @@
void write() {
	auto f = TFile::Open("out.root", "RECREATE");
	auto t = new TTree("t", "title"); // HL

	int32_t n = 0;
	double  px = 0;
	double  arr[10];
	double  vec[20];

	t->Branch("n",   &n,  "n/I"); // HL
	t->Branch("px",  &px, "px/D");
	t->Branch("arr", arr, "arr[10]/D");
	t->Branch("vec", vec, "vec[n]/D"); // HL

	for (int i = 0; i < NEVTS; i++) {
		// fill data: n, px, arr, vec with some values
		fill_data(&n, &px, &arr, &vec);

		t->Fill(); // commit data to tree. // HL

	f->Write(); // commit data to disk.


A 2021/2021-11-16-ji-groot/_figs/file-records.png => 2021/2021-11-16-ji-groot/_figs/file-records.png +0 -0
A 2021/2021-11-16-ji-groot/_figs/file_layout.jpg => 2021/2021-11-16-ji-groot/_figs/file_layout.jpg +0 -0
A 2021/2021-11-16-ji-groot/_figs/gophereartrumpet.jpg => 2021/2021-11-16-ji-groot/_figs/gophereartrumpet.jpg +0 -0
A 2021/2021-11-16-ji-groot/_figs/groot-bkreader.dia => 2021/2021-11-16-ji-groot/_figs/groot-bkreader.dia +0 -0
A 2021/2021-11-16-ji-groot/_figs/groot-bkreader.png => 2021/2021-11-16-ji-groot/_figs/groot-bkreader.png +0 -0
A 2021/2021-11-16-ji-groot/_figs/groot.jpg => 2021/2021-11-16-ji-groot/_figs/groot.jpg +0 -0
A 2021/2021-11-16-ji-groot/_figs/my-python-logo.png => 2021/2021-11-16-ji-groot/_figs/my-python-logo.png +0 -0
A 2021/2021-11-16-ji-groot/_figs/my-root6splash.png => 2021/2021-11-16-ji-groot/_figs/my-root6splash.png +0 -0
A 2021/2021-11-16-ji-groot/_figs/ttree.png => 2021/2021-11-16-ji-groot/_figs/ttree.png +0 -0
A 2021/2021-11-16-ji-groot/talk.slide => 2021/2021-11-16-ji-groot/talk.slide +502 -0
@@ 0,0 1,502 @@
# groot: reading ROOT data, with Go, faster than ROOT
16 Nov 2021

Sebastien Binet

## (a brief) History of software in HEP

## 50's-90's: FORTRAN77

.code _code/hello.f

	$ gfortran -c hello.f && gfortran -o hello hello.o
	$ ./hello
	Hello from FORTRAN

  - `FORTRAN77` is the **king**
  - 1964: **CERNLIB**
  - REAP (paper tape measurements), THRESH (geometry reconstruction)
  - SUMX, **HBOOK** (statistical analysis chain)
  - ZEBRA (memory management, I/O, ...)
  - GEANT3, **PAW**

## 90's-...: C++

.code _code/hello.cxx

	$ c++ -o hello hello.cxx && ./hello
	Hello from C++

.image _figs/my-root6splash.png 190 190

  - object-oriented programming (OOP) is the cool kid on the block
  - **ROOT**, POOL, LHC++, AIDA, **Geant4**
  - `C++` takes roots in HEP

## 00's-...: python

.code _code/hello.py

	$ python ./hello.py
	Hello from python

.image _figs/my-python-logo.png 100 250

  - `python` becomes the _de_ _facto_ scripting language in HEP
  - framework data-cards
  - analysis glue, (whole) analyses in `python`
  - **PyROOT**, rootpy
  - numpy, scipy, matplotlib, **IPython/Jupyter**


## ROOT: a file format

[ROOT](https://root.cern) is a set of `C++` libraries and a data analysis framework to analyze High Energy Physics data.

But it also comes with a file format.

Basically, **all** the [LHC](https://home.cern/science/accelerators/large-hadron-collider) data (including the one used to discover the Higgs boson) is stored using ROOT.

- simulation and reconstruction frameworks write data as ROOT files
- analysis frameworks read and write data as ROOT files
- analyses read and write data as ROOT files (`TTree`, `TH1x`, `TH2x`, ...)

_It's TTrees all the way down_

## Reading ROOT data

`C++` is **currently** the _lingua franca_ of HEP software.

`Python` is coming up fast on its heels.

What should another framework or set of libraries, written in an experimental, novative or brand new language, do to read/write `ROOT` data?

To reuse the library, wrap it (manually, or via `SWIG`):

- link to `libRIO.so` (`C++`)
- use it directly (compiled: (old) `PyROOT`, or directly: [Julia](https://github.com/JuliaInterop/Cxx.jl) can do this)


- link to `libRIO.so` (`C++`)
- use it through a `C API`

## Reading ROOT data

I initially tried the latter avenue:

.link https://github.com/go-hep/croot

_Unfortunately_ for [Go](https://golang.org), calling `C` from `Go` is quite expansive (`~10 fct calls`).
Let alone calling `C++` from `C` from `Go`.

It's also quite inconvenient to require to install ROOT (and all its dependencies), and this breaks the seamless experience of installing `Go` code:

.code _code/croot-fail.txt

**⇒ Implement reading/writing `ROOT` data from `Go`, from first principles.**

## Reading ROOT data w/o ROOT

There are actually many attempts/libraries to read (and sometimes write too) `ROOT` data w/o `ROOT`:

- [inexlib](https://github.com/gbarrand/inexlib_rio) (`C++`, header-only, read/write, used by [Geant4](https://geant4.web.cern.ch/), ROOT4)
- [FreeHEP/rootio](http://java.freehep.org/freehep-rootio/) (`Java`, read-only, ROOT3)
- [diana-hep/root4j](https://github.com/diana-hep/root4j) (`Java`, fork of `FreeHEP/rootio`, read-only, ROOT?)
- [scikit-hep/uproot](https://github.com/scikit-hep/uproot4) (`Python`, initially based off `groot`, read/write, ROOT6)
- [cbourjau/root-io](https://docs.rs/root-io) (`Rust`, read-only, ROOT6)
- [UnROOT.jl](https://github.com/tamasgal/UnROOT.jl) (`Julia`, read-only, ROOT6)
- [groot](https://go-hep.org/x/hep/groot) (`Go`, read/write, ROOT6)

The last 4 projects have been sharing tips and knowledge on:

.link https://groups.google.com/g/polyglot-root-io

## How to read ROOT files w/o ROOT?

## ROOT file format

`ROOT` files are binary files:

- one needs to know the [endianness](https://en.wikipedia.org/wiki/Endianness) of multi-bytes data (Little-/Big-endian)
- one needs to know how many bytes to read back
- one needs to know what those bytes mean (_ie:_ their type+name)

These issues are usually addressed by a document specifying the file format.

  - [PNG file format description](https://www.w3.org/TR/PNG/)
  - [CBOR specs](https://cbor.io/)
  - [FITS specs](https://fits.gsfc.nasa.gov/standard40/fits_standard40aa-le.pdf)

Unfortunately, `ROOT` doesn't provide such a clear description of its on-disk format...

## Reading back binary files

Once saved, `float64`, `float32` or your favorite `struct` are all just a sequence of bytes.
How does one chop and chunkify back this stream into structured data?

_ie:_ how to read back:

.code _code/binary-blob.txt

how does one know that it's a `(uint8,uint16,uint32,int64,float64)` ?
and not, _e.g._ `23x(uint8)` ? (or any other combination)

This is usually addressed with:

  - adding delimiters: special magic values that flank "real" data (_e.g.:_ `\0`, `0xcc`, `0xdeadbeef`, `0xbadcaffe`, ...) (_e.g.:_ a C-string)
  - using a [TLV (tag-length-value)](https://en.wikipedia.org/wiki/Type-length-value) encoding scheme (_e.g.:_ a Python string)

One nice property of TLV: allows to skip unknown data.


`ROOT` uses a kind of TLV scheme.

.image _figs/file_layout.jpg

## ROOT file

At a very high-level, a `ROOT` file can be seen as:

	[file header]
	[file footer]

The `ROOT` file header encodes:

  - the `ROOT` magic word (`"root"`) and the file format (`uint32`)
  - the pointer to the first data record (`uint32`, position on file in bytes)
  - the pointer to the end of the last data record (`uint32/uint64`)
  - the pointer to the `streamers` record (+its length)
  - other metadata (file creation time, compression algorithm+level, ...)

The `ROOT` file footer holds the `streamers` (metadata about types).


A `ROOT` record consists of its key (`TKey`) and its associated data (an opaque binary blob).

.image _figs/file-records.png _ 450

## TKey

The on-disk representation of a `TKey` is:

.code _code/tkey.txt

  - `TKey` knows its length and the length of the associated payload
  - `TKey` knows the type of the associated payload (`fClassName`)
  - `TKey` knows the location on file of the associated payload

## Deserialization

Knowing the position+length of the user data to read, `ROOT` knows what to do to be able to unmarshal/deserialize that user data.
The "how-to" part is done by cross-referencing the name of the user data class with the `streamers` list (that is stored within the `ROOT` file footer.)

`ROOT` stores metadata about the types that are stored in a `ROOT` file **within** that `ROOT` file.
_ie:_ `ROOT` files are self-describing. You don't need _a priori_ to know anything about the data between stored inside a `ROOT` file to be able to correctly interpret all its bytes.

One "just" needs to be able to correctly interpret and decode/encode:

  - `TFile` (and `TDirectoryFile`, for hierarchical data)
  - `TKey`
  - `TStreamerInfo` and its `TStreamerElements`

and the bootstrap process is complete.

## TStreamerInfo & TStreamerElement

A `TStreamerInfo` encodes metadata about a class:

  - the class name + checksum
  - the version of that class
  - the list of fields (+base classes) that describes that class (`TStreamerElement`)

_e.g.:_ `ROOT` describes the C++ type `P3` below, as:

.code _code/struct-p3.c


.code _code/streamer-p3.txt


The `streamer` elements vocabulary is quite exhaustive and allows to represent pretty much all of possible C++ classes:

.code _code/streamers.txt

`ROOT` can support reading multiple versions of a type, through this `version` field in the `TStreamerInfo`.

`MyClass` at version 1 may have 2 fields `f1` and `f2` of types `float` and at version 2 have those 2 fields w/ types `float` and `double`.

## Reading ROOT files w/o ROOT

  - decode ROOT file structure (`TFile`, `TDirectoryFile`, `TKey`)
  - decode ROOT metadata structure (`TStreamerInfo`, `TStreamerElements`)
  - generate deserialization routines from the `streamers` informations
  - decode builtins and composites (structs, static arrays, dynamic arrays, ...)

Pretty simple, right? right... (w/o having to reverse engineer the file format, it might have been.)

But that doesn't cover the main physicist use case: `TTrees`.

The (most used) ROOT API to read/write user data is through:

  - `TFile`
  - `TTree` + `TBranch`

`ROOT` stores data in binary files, organized into `TDirectories` and `TKeys`.

## TFile + TKey

With `TFile` and `TKey`, one could already address the following physicist use-case:

  - retrieve data in an incremental fashion (_i.e.:_ event `#1`, `#2`, ...)
  - store it piecewise in the ROOT file.

we could do something like:

.code _code/tfile-tkey-1.go

## TFile + TKey


  - reading back + iterating over all the data would be cumbersome
  - one could store the events in an array (so iteration would be addressed), but this would mean writing/reading **all** events in a single step
  - selection of a subset of the event data would be cumbersome
  - missed opportunities for compression and/or data locality

It's doable (that's more or less what [Python](https://python.org) guys do with [pickles](https://docs.python.org/3/library/pickle.html).)
But it's no _panacea_.

Enters `TTree`...

## TTree

`TTree` is an API to:

  - declare a list of pairs `(name,value)` that are called _"branches"_ (of the tree)
  - "fill" the tree with data for each "entry" in the tree

Once a `TTree` is filled and written to a `TFile`, one can read it back, re-attaching variables to each of its branches (or a subset thereof), to inspect the stored data.

A `TTree` is kind of like a database, where you can store:

  - data row-wise or column-wise
  - C-builtins, (static) C-arrays, var-len C-arrays, C-structs
  - `C++` classes (with inheritance, virtual tables, etc...)
  - `C++` STL containers of all the above


.code _code/tree.cxx


.image _figs/ttree.png _ 480

## TTree (x-ray) scan

`TTree` reuses much of the `TKey + TStreamers` infrastructure.

When one connects a branch with some user data:

  - the type (and shape) of the user data is "introspected" (via C++ templates+overloading)
  - from that type a streamer is associated (via type reflection or via the C++ interpreter `CINT` for ROOT \<=v5, `CLing` nowadays)

at that point, the `TTree` knows how to serialize/deserialize the user data into chunks of bytes, **TBasket** in ROOT speak.

To support arbitrarily nested containers/user-data, ROOT introduces the notion of branches with sub-branches with sub-branches, ... that, _in_ _fine_ have leaves.

This is controlled by the _"split_ _level"_ of a tree.

## TTree writing

.code _code/tree-w.cxx

## TTree writing (modes)

  - row-wise storage:

.code _code/storage-1.txt

  - column-wise storage:

.code _code/storage-2.txt

  - column-wise storage, split-level > 1:

.code _code/storage-3.txt

All these different ways of storing data are, ultimately, represented as `TBaskets` holding the serialized representation of these `[...]` user data as bytes.

Each `TBasket` associated payload, is compressed (or not).
A `TBasket` payload may contain data from multiple entries.

## TTree reading

.code _code/tree-r.cxx

## TTree reading

Once a `TTree` is requested, ROOT needs to locate it on disk and then deserialize it (only the "metadata", not the full associated dataset payload) using the usual ROOT machinery (streamers+`TKey`).

A `TTree` knows:

  - the number of entries it contains
  - the list of branches (and their associated payload type) it contains

A `TBranch` knows:

  - the list of baskets associated to it
  - the entry span of each basket (`[1,` `10],` `[11,` `23],` `...,` `[xxx,` `evtmax]`)
  - the physical location (byte offset, number of bytes) of each basket

## TTree reading

Whenever somebody asks to read entry `n` from disk:

  - for each active branch, read entry `n` 
  - find+retrieve the basket containing the entry `n`
  - (possibly uncompress the basket data)
  - store the deserialized data into the pointer captured by `SetBranchAddress`, using the correct streamer

And voilà, you know how (at a very coarse level) `TTrees` read and present data to users.

## Go-HEP/groot

## groot

[groot](https://go-hep.org/x/hep/groot) is a pure-Go implementation of (a subset of) ROOT.

  - [groot/riofs](https://go-hep.org/x/hep/groot/riofs) provides access to the `TFile` equivalent layer of ROOT
  - [groot/rdict](https://go-hep.org/x/hep/groot/rdict) provides access to the "streamers" layer of ROOT
  - [groot/rtree](https://go-hep.org/x/hep/groot/rtree) provides access to the `TTree` layer of ROOT

.image _figs/groot.jpg _ 600

## groot reading

.code _code/tree-r.go /START/,/END/

## groot reading speed

Reading some ATLAS data, with Go-HEP v0.26, compared to ROOT/C++ 6.20

.code _code/perf-v0.26.txt

"Only" **~2 times slower**, w/o any optimization wrt baskets buffering, TTreeCache, ...

And that was like that since the inception of `groot`.

## groot reading speed

Reading some ATLAS data, with Go-HEP v0.26, compared to ROOT/C++ 6.20

.code _code/perf-v0.26.txt

And that was like that since the inception of `groot`.

Until, `v0.27` (released May-2020):

.code _code/perf-v0.27.txt

**Almost** twice faster than ROOT :)

See, for more informations:

.link https://root-forum.cern.ch/t/gohep-groot-v0-27-0-root-split-groot-faster-than-root/39609
.link https://github.com/go-hep/groot-bench

How come `groot` is faster than ROOT to read ROOT data?

Thanks to Go's lightweight goroutines...

## groot & Go

[Go](https://golang.org) is known to be very fast to compile and relatively fast to execute.
But at the moment, Go binaries are usually slower than a C++ one for number crunching.

How could a Go binary be faster than a C++ one?

  - better (profiling, diagnostic) tools
  - better architecture
  - better concurrency

Reading a `TTree` is basically:

  - locate spans of bytes on disk
  - (uncompress those bytes span)
  - deserialize those (uncompressed) bytes into memory locations
  - repeat

## Latency

.code _code/big-o-timings.txt

## groot rtree Reader

With the new re-engineered `rtree.Reader`, `groot` can infer:

  - the expected entry span to process
  - the expected list of baskets and their byte spans

and thus, for each requested branch:

  - create a goroutine that will serve the baskets in turn
  - pre-fetch the basket bytes on disk
  - pre-launch the uncompression

.image _figs/gophereartrumpet.jpg _ 350

So when one requests entry `N`, everything is already in memory, ready to be used.


.image _figs/groot-bkreader.png _ 800

## groot rtree

An additional concurrency axis (not yet implemented) would be to have `N` concurrent goroutines each requesting/handling one entry of the tree (and filling in turn the user data)...

but being already `~2x` faster than ROOT isn't too bad.

Now, the same kind of optimization should also be applied to writing...

.image _figs/groot.jpg _ 500
.caption That's all folks

## Conclusions & Prospects

It is possible to read `ROOT` data faster than `ROOT` itself:

.code _code/groot-bench-results.txt

_(inspite of the lack of specifications for `ROOT`'s file format)_

## Conclusions & Prospects

`ROOT-7` is slated to ship with a new file format: `RNTuple`.

Specifications are a bit more fleshed out:

.link https://github.com/root-project/root/blob/master/tree/ntuple/v7/doc/specifications.md
.link https://github.com/jblomer/root/blob/ntuple-binary-format-v1/tree/ntuple/v7/doc/specifications.md

and there are plans to provide a `C API` (as well as a non `.root` file format).


[groot](https://go-hep.org/x/hep/groot) will probably try to implement that format as well.