7 files changed, 255 insertions(+), 0 deletions(-)

A 2022/2022-09-22-calorado/README.md
A 2022/2022-09-22-calorado/_figs/collab-calorado.png
A 2022/2022-09-22-calorado/_figs/detector.png
A 2022/2022-09-22-calorado/_figs/grafana-logo.png
A 2022/2022-09-22-calorado/_figs/plot.png
A 2022/2022-09-22-calorado/_figs/screen-crop.png
A 2022/2022-09-22-calorado/talk.slide

@@ 0,0 1,3 @@
+# 2022-09-22-calorado
+
+`go-present` link: [slides](https://talks.sbinet.org/2022/2022-09-22-calorado/talk.slide)

@@ 0,0 1,252 @@
+# CaloRado
+LPC Dev
+22 Sep 2022
+
+Sebastien Binet
+CNRS/IN2P3/LPC-Clermont
+https://github.com/sbinet
+@0xbins
+sebastien.binet@clermont.in2p3.fr
+
+## CaloRado
+
+Since ~June/July, started working on `CaloRado`:
+
+  _"A detector to measure the radioactivity of volcanic gases"_
+
+LPC is working on the development of a gamma calorimeter for continuous radon monitoring in volcanic plumes.
+
+.image _figs/collab-calorado.png 350 _
+
+## Apparatus
+
+.image _figs/detector.png 450 _
+
+## DAQ
+
+Alex tricked me into having a look at the "DAQ".
+
+- detector is sending UDP packets as fast as it can
+- a dedicated LPC machine is listening for those and storing them
+- initial `python` script by Magali was listening for packets, decoding them and storing data as `.csv` files
+- initial `python` script by Magali was missing packets
+
+So:
+- could `Go` do better ?
+
+  Well _of course!_
+  Let me try.
+
+.link https://gitlab.in2p3.fr/sbinet/calorado-daq
+
+## 
+
+The detector is sending UDP binary packets:
+
+
+```
+-- header: 14 bytes (multi-bytes encoded as big-endian)
+ 0 -  4: uint32: type of frames
+ 4 -  6: uint16: rate
+ 6 -  8: uint16: temperature
+ 8 - 14:         [RESERVED]
+-- n-frames of 16 bytes
+ 0 -  4: uint32: boundary (==0xaaaaaaaa)
+ 4 -  8: uint32: trigger number
+ 8 - 10: uint16: amplitude
+10 - 16: uint48: timestamp (relative)
+-- footer: 2 bytes
+ 0 -  2: uint16: boundary (==0xaaaa)
+```
+
+_ie:_ a 14-bytes header, followed by `n` frames of 16-bytes data, and a final 2-bytes footer.
+
+## How it started
+
+```go
+
+conn, err := net.ListenUDP("udp", &net.UDPAddr{
+	Port: port,
+	IP:   net.ParseIP(ip),
+})
+if err != nil {
+	log.Fatalf("could not create UDP listener: %+v", err)
+}
+defer conn.Close()
+
+buf := make([]byte, 1024)
+for {
+	n, err := conn.Read(buf)
+	if err != nil {
+		log.Printf("could not read data from UDP socket: %+v", err)
+		continue
+	}
+
+	err = process(buf[:n])
+	if err != nil {
+		log.Printf("could not process trigger data: %+v", err)
+		continue
+	}
+}
+```
+
+## How it started
+
+Pretty simple:
+
+```shell
+
+$> go build -o calo-daq ./main.go
+$> calo-daq
+```
+
+`calo-daq` is:
+
+- (still) listening on UDP
+- stores data as binary
+- every hour, commits data to a binary file
+- every minute, commits summary/monitoring data to a `JSON` file (timestamp, rate, temperature)
+
+## And now...
+
+## 
+
+A slew of commands to handle the DAQ+Reco workflow:
+
+```
+
+$> go build ./cmd/calo-daq      # DAQ
+$> go build ./cmd/calo-dump     # displays CALO files
+$> go build ./cmd/calo-janitor  # concatenates CALO files (1/day) + gzip
+$> go build ./cmd/calo-mon      # monitors CALO files for missing frames
+$> go build ./cmd/calo-srv      # schedules calo-mon jobs, creates histograms (soon)
+$> go build ./cmd/calo-cnv      # converts CALO files to ROOT
+```
+
+## calo-dump
+
+```shell
+
+$> calo-dump ./2022092100.raw.gz
+Event #0 - 2022-09-21 00:00:00.0791 UTC - rate= 175.77 temp=14.22 kind=1 frames=30
+  217614721    179  12602452132343
+  217614722    611  12602452155334
+  217614723   3692  12602452193631
+  217614724    299  12602452222285
+  217614725    518  12602452284647
+  217614726    112  12602452339360
+  217614727    247  12602452393412
+  217614728    531  12602452411598
+  217614729   1784  12602452426692
+  217614730    445  12602452477254
+  217614731   2520  12602452517602
+[...]
+Event #1 - 2022-09-21 00:00:00.2557 UTC - rate= 175.77 temp=14.22 kind=1 frames=30
+  217614751    480  12602453509912
+  217614752   2012  12602453513271
+  217614753    136  12602453535683
+  217614754    444  12602453558193
+[...]
+```
+
+## calo-mon
+
+`calo-mon` reads in a collection of `CALO` files and analyzes them to find missing frames:
+
+```
+
+  $> calo-mon ./data/20220907*.raw.gz
+  ./data/2022090700.raw: ok (triggers=640530)
+  ./data/2022090701.raw: lost triggers=  30 frames=   1
+  ./data/2022090702.raw: ok (triggers=649170)
+  ./data/2022090703.raw: ok (triggers=653100)
+  ./data/2022090704.raw: ok (triggers=657930)
+  ./data/2022090705.raw: ok (triggers=662490)
+  ./data/2022090706.raw: lost triggers=  30 frames=   1
+  ./data/2022090707.raw: lost triggers=  60 frames=   2
+  ./data/2022090708.raw: ok (triggers=649950)
+  ./data/2022090709.raw: ok (triggers=645240)
+  ./data/2022090710.raw: ok (triggers=636510)
+  ./data/2022090711.raw: ok (triggers=630060)
+  ./data/2022090712.raw: lost triggers=  30 frames=   1
+  ./data/2022090713.raw: ok (triggers=650280)
+  [...]
+```
+
+Analysis of the origin(s) of lost triggers is still pending.
+
+(most probably _"just"_ network effects (UDP))
+
+## calo-cnv
+
+`calo-cnv` reads in a collection of `CALO` files and can produce a single output ROOT file containing a `"calorado"` `TTree`:
+
+```shell
+
+  $> calo-cnv -o out.root ./20220907*.raw.gz
+  calo-cnv: processing "./2022090700.raw.gz"...
+  calo-cnv: processing "./2022090701.raw.gz"...
+  [...]
+  calo-cnv: processing "./2022090722.raw.gz"...
+  calo-cnv: processing "./2022090723.raw.gz"...
+  
+  $> ll out.root
+  -rw-r--r-- 1 binet binet 133M Sep 21 17:58 out.root
+
+  $> root-ls -t ./out.root 
+  === [./out.root] ===
+  version: 62600
+    TTree       calorado              (entries=15473430)
+      trig      "trig/i"      TBranch
+      ampl      "ampl/s"      TBranch
+      timestamp "timestamp/l" TBranch
+      temp      "temp/F"      TBranch
+```
+
+## Grafana
+
+Alex is collecting the `JSON` sampling data files and the `CSV` output of `calo-mon` into [Grafana](https://grafana.com/)
+
+   Grafana allows you to query, visualize, alert on and understand
+   your metrics no matter where they are stored. Create, explore, and
+   share beautiful dashboards with your team and foster a data driven
+   culture.
+
+.link https://grafana.com/oss/
+
+We have a local instance that presents this data.
+
+.image _figs/grafana-logo.png 150 _
+
+##
+
+.image _figs/screen-crop.png 510 _
+
+## Physics
+
+With the produced "reco" ROOT file (`calo-cnv` processes the `CALO` file and reconstruct the absolute timestamp for each frame, from the relative one), one can start actual physics:
+
+- energy calibration of the detector
+
+This will need to:
+
+- build the distribution of the amplitude (in bins of temperature)
+- fit the "obvious" "known" peaks at "known" energies
+- extract a `F(ampl, temp) -> E`
+
+```
+  $> ll ./data/root/202208.root
+  -rw-r--r-- 1 binet binet 5.5G Sep 19 14:28 ./data/root/202208.root
+```
+
+##
+
+.image _figs/plot.png 600 _
+
+## Next steps
+
+- Luca will fit the various peaks and extract a calibration function `F(temp)`
+- `calo-srv` will apply the calibration "on the fly" on the reconstructed data
+- `calo-srv` will serve energy spectrum plots as data quality checks
+
+