~n0mn0m/blog

12f6c9ec1347a4a445b0729c5531b81bf4fabdc7 — n0mn0m 2 months ago d28d254
New articles.

New articles for train all the things and cloudflare. Fix broken link in old article.
M content/blog/h3lou-q1-2020.md => content/blog/h3lou-q1-2020.md +1 -1
@@ 17,7 17,7 @@ Last year we kicked off the first Hardware Happy Hour in Louisville Kentucky, US

This year we wanted to start things off by inviting everybody to build and learn together with the [Circuit Playground Express](https://www.adafruit.com/product/3333). On January 29th we got together [15 makers and hackers](https://flic.kr/s/aHsmL8818Z) to have some fun with Arduino and Circuit Python making LEDs blink and speakers buzz. To kick things off Auystn introduced the group to the Arduino IDE, getting it to recognize your board setup, and receiving feedback via the serial console. As with any workshop we had plenty of fun figuring out why this and that didn't work on whatever OS, but in many ways I think that was exposure for those new to working with the boards and tools that unexpected behavior may occur, but we can find a solution. 

Once everybody had a board up and working [Austyn](https://flic.kr/p/2inNG6V) spent some time getting everybody comfortable with the Arduino syntax and constructs. That turned into showing how to make some [noise](https://github.com/Hardware-Happy-Hour-Louisville/HardwareLou_CircuitPlayground/blob/master/cricket/lightsensor_cricket.ino) followed by a quick on/off switch demo. With a couple more code demos and showing off the Arduino code library we decided to switch gears and look at Circuit Python before having some general open make time.
Once everybody had a board up and working [Austyn](https://flic.kr/p/2inNG6V) spent some time getting everybody comfortable with the Arduino syntax and constructs. That turned into showing how to make some [noise](https://github.com/h3-louisville/HardwareLou_CircuitPlayground/blob/master/cricket/lightsensor_cricket.ino) followed by a quick on/off switch demo. With a couple more code demos and showing off the Arduino code library we decided to switch gears and look at Circuit Python before having some general open make time.

With Circuit Python we had the same demos with a different approach. Instead of using an IDE and editor we showed how you could put the board into bootloader mode and drag and drop the UF2 and code files directly on the board for loading. Along with that we demo'd the ability to use REPL driven development on the boards for quick prototyping and feedback. 


A content/blog/system-status-observer.md => content/blog/system-status-observer.md +136 -0
@@ 0,0 1,136 @@
+++
title = "A Simple Status Page"
description = "Building a status page with cloudflare workers and systemd"
date = 2020-02-18
in_search_index = true

[taxonomies]
tags = ["cloudflare", "systemd", "FaaS", "serverless", "monitoring"]
categories = ["programming"]
+++

I have a bad habit of creating side projects for my side projects.

A couple months ago I switched from running my blog with Pelican and Gitlab Pages to Zola and Cloudflare Workers. I didn't do a write up on it, but if you're interested there's a good [post by Steve Klabnik](https://words.steveklabnik.com/porting-steveklabnik-com-to-workers-sites-and-zola) to get you started. It was a surprisingly easy switch, and gaps between writing haven't been as difficult with the better tools. After getting that setup I read about [Cloudflare Workers KV](https://developers.cloudflare.com/workers/reference/storage), thought it sounded really neat and started to think about what I might build.

On another [project](@/blog/train-all-the-things-planning.md) I need to signal between different systems a simple status. Naturally that lead to me building a status page. I setup a Cloudflare Worker that receives `POST` from `N` systems, stores the date of the last `POST` uses that to provide a status when asked.

```javascript
const setCache = (key, data) => LOCAL_STATUS.put(key, data);
const getCache = key => LOCAL_STATUS.get(key);

function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

function dateToStatus(dateTime) {
    var isoDateNow = Date.now();
    var dateDiff = (isoDateNow - dateTime);
    if (dateDiff < 180000) {
	return 1
    } else {
	return 0
    }
}

async function getStatuses() {
    const cacheKeys = await LOCAL_STATUS.list();
    while (!(cacheKeys.list_complete === true)) {
	sleep(5)
    }

    const numKeys = cacheKeys.keys.length;
    var statuses = [];

    for (var i = 0; i < numKeys; i++) {
	var c = cacheKeys.keys[i];
	var epcDate = await getCache(c.name);
	var data = {date: Number(epcDate), name: c.name};
	data.strDate = new Date(data.date).toISOString();
	data.status = dateToStatus(data.date);
	data.statusIndicator = getStatusIndicator(data.status);
	statuses.push(data);
    }

    const body = html(JSON.stringify(statuses || []));

    return new Response(body, {
	headers: { 'Content-Type': 'text/html' },
    });
}

async function getStatus(cacheKey) {
    var cacheDate = await getCache(cacheKey);

    if (!cacheDate) {
	return new Response('invalid status key', { status: 500 });
    } else {
	var status = dateToStatus(cacheDate);
	return new Response(status, {status: 200});
    }
}

async function updateStatus(cacheKey) {
    try {
	var isoDate = Date.now();
	await setCache(cacheKey, isoDate);
	var strDate = new Date(isoDate).toISOString();
	return new Response((cacheKey + " set at " + strDate + "\n"), { status: 200 });
    } catch (err) {
	return new Response(err, { status: 500 });
    }
}

async function handleRequest(request) {
    let statusKey = new URL(request.url).searchParams.get('service');
    let queryType = new URL(request.url).searchParams.get('query');

    if (request.method === 'POST') {
	return updateStatus(statusKey);
    } else if (queryType === 'simple') {
	return getStatus(statusKey);
    } else {
	return getStatuses();
    }
}

addEventListener('fetch', event => {
    event.respondWith(handleRequest(event.request))
})
```

With that anything that can `POST` can "check in" with the endpoint. You can see it working [here](https://status.unexpectedeof.casa/). I also went ahead and wrote a simple `systemd` service that I can drop on to different machines I want to have report in to the endpoint.

```bash
[Unit]
Description=Regular check in
Wants=check-in.timer

[Service]
Type=oneshot
ExecStart=/usr/bin/curl -X POST https://status.unexpectedeof.casa/?service=JETSON

[Install]
WantedBy=multi-user.target
```

And a timer for the service.

```bash
[Unit]
Description=Run checkin every 2 minutes
Requires=check-in.service

[Timer]
Unit=check-in.service
OnUnitInactiveSec=1m

[Install]
WantedBy=timers.target
```

This was a fun "Serverless/FaaS" experiment that actually let me know my ISP was having an outage one morning before work. I've used other Functions as a service on other cloud platforms and while they all provide slightly different functionality (For instance Cloudflare being a CDN and the V8 isolate setup) Cloudflare Workers has been really easy to work with and a lot of fun to build experiments on. They even have a [web playground](https://cloudflareworkers.com) that you can start with. 

Two things I do wish were easier are interacting with K/V from Rust. This is probably partially related to how new I am to Rust, but working with K/V from JS is super easy, while this [thread](https://www.reddit.com/r/rust/comments/fdmzyh/serverless_rust_i_tried_it_with_cloudflare_workers/) documents another experience with Workers and Rust in more detail. Another mild annoyance is working with different workers from the same machine and how API keys are handled. There are some suggestions for this, but non of them feel ergnomic at this time. Other than that my experience with Workers and K/V has been great and I've already got more ideas for future experiments.

The code, docs, etc for the project can be found [here](https://git.sr.ht/~n0mn0m/system-status). If you have any questions or ideas reach [out](mailto:alexander@unexpextedeof.net).

A content/blog/train-all-the-things-display.md => content/blog/train-all-the-things-display.md +73 -0
@@ 0,0 1,73 @@
+++
title = "Train All the Things: Setting up the display"
description = "Using a PyPortal to let others know when your unavailable."
date = 2020-03-05
in_search_index = true

[taxonomies]
tags = ["hackaday", "machine-learning", "train-all-the-things", "circuitpython"]
categories = ["embedded"]
+++

Continuing my project with things I know the PyPortal display was up next. Last year I spent a few weeks playing with the portal to make a badge at Gen Con and had a lot of fun with it. Since that time [CircuitPython 5](https://circuitpython.org/downloads) has been released and the portal now expects a few new [modules](https://circuitpython.org/libraries) which were easy enough to download and send to the board. The PyPortal makes it incredibly easy to point at an endpoint to fetch data:

```python
import board
from adafruit_pyportal import PyPortal

pyportal = PyPortal(
    url=<your url here>,
    default_bg="green.bmp"
)

status = pyportal.fetch()
print(status)
```

With that small snippet we have our status, and all we need to do is put that in a loop to set the background depending on the bit returned. 

```python
import board
from time import sleep
from adafruit_pyportal import PyPortal

try:
    from secrets import secrets  # noqa
except ImportError:
    print("WiFi secrets are kept in secrets.py, please add them there!")
    raise

pyportal = PyPortal(
    url=secrets["signal"],
    default_bg="green.bmp"
)

current = 0

while True:
    status = int(pyportal.fetch())
    if status == 0 and status == current:
        pass
    elif status == 0 and status != current:
        pyportal.set_background("green.bmp")
        current = 0
    elif status == 1 and status != current:
        pyportal.set_background("red.bmp")
        current = 1
    elif status == 1 and status == current:
        pass
    sleep(30)
```

Even though it's a small snippet I want to point out a couple things. First I'm wrapping the return from `fetch` in a cast to `int`. If you use Python,  but you are new to CircuitPython this may seem odd. If you don't do this and try to compare a `string` to an `int` you're probably not going to get the result you expect. Try it out in a repl and then follow up with [CircuitPython Essentials](https://learn.adafruit.com/circuitpython-essentials/circuitpython-essentials). Also I'm only changing the background if the status we fetch is different than the current status. While repainting the screen is fast, it's noticeable and there's no reason to do it every 30 seconds if nothing is different.

That's it. Now whenever the endpoint receives an update the portal will see that status change and update the display. 

![Train all the Things Display Red](/images/on-air-display-red.jpg "PyPortal displaying a red background with the Hackaday Jolly Wrencher.")
![Train all the Things Display Green](/images/on-air-display-green.jpg "PyPortal displaying a green background with the Hackaday Jolly Wrencher.")

Thanks to Adafruit for publishing the [case](https://www.thingiverse.com/search?q=pyportal&dwh=915e616a3fbda6e) above. The logo on display is the Jolly Wrencher of [Hackaday](https://hackaday.com/about/). 

With the [endpoint](@/blog/train-all-the-things-sighandler.md) and display done I'm off into the unknown. I'll be setting up the ESP-EYE to update the endpoint, training the voice model and finally running it all with FreeRTOS.

The code, docs, images etc for the project can be found [here](https://git.sr.ht/~n0mn0m/on-air) and I'll be posting updates as I continue along to [HackadayIO](https://hackaday.io/project/170228-on-air) and this blog. If you have any questions or ideas reach [out](mailto:alexander@unexpextedeof.net).
\ No newline at end of file

A content/blog/train-all-the-things-planning.md => content/blog/train-all-the-things-planning.md +29 -0
@@ 0,0 1,29 @@
+++
title = "Train All the Things: Planning"
description = ""
date = 2020-02-08
in_search_index = true

[taxonomies]
tags = ["hackaday", "maker", "machine-learning", "planning", "train-all-the-things"]
categories = ["programming"]
+++

Earlier this year Hackaday announced the [Train all the Things](https://hackaday.io/contest/169421-train-all-the-things#j-discussions-title) contest. I immediately knew I wanted to submit something, but figuring out what to build took me a little bit. For my side projects I like to make something that is useful to me, or somebody I know; while also learning something new. A few days after the contest was announced my daughter was in the basement playing outside my office/homelab when I remembered my wife had asked me if there was a way for her to know when I was working with somebody so that they could avoid coming down in the basement. I thought a voice driven display could be a fun solution.

### Choosing my tools

After deciding on the project the next thing I wanted to figure out was what new boards I would need (if any) and how I would build my model. After doing some research I landed on [Tensorflow](https://www.tensorflow.org/lite/microcontrollers) as my path forward for deploying a model to a microcontroller. Having used Tensorflow the barrier for model creation is a bit lower, but I am really curious about Tensorflow Lite and the potential it provides. Additionally a relatively new book [TinyML](https://tinymlbook.com/) looks like a good resource to use along the way.

After settling on TF Lite the next thing was picking a board. Most of my embedded experience has been with CircuitPython and Rust. For this project I thought it would be fun to learn something new. The Espressif ESP-EYE caught my eye as an interesting board known to work with TF Lite. I've seen the ESP32 and 8266 in a lot of other projects, so learning the ESP toolchain seems valuable. Additionally a lot of the Espressif ecosystem seems to be built around FreeRTOS which provides a whole other avenue of learning and hacking.

Finally I will need a way to let somebody know when the model has picked up voice activity, to signal that I'm currently busy in the lab. The ESP32 has a WiFi chip providing the ability to send and receive signals via TCP if we want. The ESP-EYE has that built in, and I happend to have a PyPortal (with an ESP32) that could make a great display checking for a status using WiFi too. To signal from one to the other I decided to have some fun and use Cloudflare Workers K/V to set a bit from the ESP-EYE that would be read by the PyPortal at a given time interval to set the display.

Putting it all together the initial idea looks something like this:

![Train all the Things Project Diagram](/images/on-air-grid.jpg "Event flow between ESP-EYE, Cloudflare and PyPortal.")

Which allows me to have a small board in my homelab listening and the display above the stairwell where somebody can get a status
update before they ever come down.

The code, docs, images etc for the project can be found [here](https://git.sr.ht/~n0mn0m/on-air) and I'll be posting updates as I continue along to [HackadayIO](https://hackaday.io/project/170228-on-air) and this blog. If you have any questions or ideas reach [out](mailto:alexander@unexpextedeof.net).
\ No newline at end of file

A content/blog/train-all-the-things-sighandler.md => content/blog/train-all-the-things-sighandler.md +65 -0
@@ 0,0 1,65 @@
+++
title = "Train All the Things: Signaling"
description = "Capturing a status bit for others to see."
date = 2020-03-01
in_search_index = true

[taxonomies]
tags = ["hackaday", "machine-learning", "train-all-the-things", "edge", "javascript", "serverless", "FaaS"]
categories = ["programming"]
+++

After [figuring out ](@/blog/train-all-the-things-planning.md) what I was going to use for my project I started work with things I know. I already had some experience with Cloudflare workers building a [home system status](@/blog/system-status-observer.md) page, and Workers K/V makes storing and fetching data quick and easy. I ended up with a simple endpoint that I `POST` to set a bit after keyword detection, and the PyPortal retrieves that status to determine what to display:

```javascript
const setCache = (key, data) => SIGNALS.put(key, data);
const getCache = key => SIGNALS.get(key);

async function getStatus(cacheKey) {
    var serviceStat = await getCache(cacheKey);

    if (!serviceStat) {
        return new Response('invalid status key', { status: 500 });
    } else {
        return new Response(serviceStat, {status: 200});
    }
}

async function setStatus(cacheKey, cacheValue) {
    try {
        await setCache(cacheKey, cacheValue);
        return new Response((cacheKey + " set to " + cacheValue + "\n"), { status: 200 });
    } catch (err) {
        return new Response(err, { status: 500 });
    }
}

async function handleRequest(request) {
    var psk = await getCache("PSK")
    let presharedKey = new URL(request.url).searchParams.get('psk');
    let statusKey = new URL(request.url).searchParams.get('service');
    let statusValue = new URL(request.url).searchParams.get('status');

    if (presharedKey === psk) {
        if (request.method === 'POST') {
            return setStatus(statusKey, statusValue);
        } else if (request.method === 'GET' && statusKey) {
            return getStatus(statusKey);
        } else {
            return new Response("\n", { status: 418 });
        }
        
    } else {
        return new Response("Hello")
    }
}


addEventListener('fetch', event => {
    event.respondWith(handleRequest(event.request))
})
```

Nothing tricky happening above, just checking the request, and calling the appropriate function to store or fetch the status bit. With the function deployed to my Cloudflare Worker and verified with some `GET` and `POST` calls I was ready to move on to the [display](@/blog/train-all-the-things-display.md).

The code, docs, images etc for the project can be found [here](https://git.sr.ht/~n0mn0m/on-air) and I'll be posting updates as I continue along to [HackadayIO](https://hackaday.io/project/170228-on-air) and this blog. If you have any questions or ideas reach [out](mailto:alexander@unexpextedeof.net).
\ No newline at end of file

A static/images/on-air-display-green.jpg => static/images/on-air-display-green.jpg +0 -0

A static/images/on-air-display-red.jpg => static/images/on-air-display-red.jpg +0 -0

A static/images/on-air-grid.jpg => static/images/on-air-grid.jpg +0 -0