#hare-neural

Basic tools for working with neural networks in Hare. Currently consisting of:

• ml/grad.ha: A simple scalar-valued DAG-based autograd engine
• ml/perceptron.ha: A simple perceptron implementation
• ml/layer.ha: A layer of perceptrons
• ml/mlp.ha: A set of layers
• ml/train.ha: Some simple training code to do iterations of backpropagation to minimise loss

#MLP Example

You can run make run to run the example code, and tweak the test data in cmd/test/main.ha.

Or, here's an example for how to use the API.

// Training data
let xs: [][]f64 = [
        [ 2.0,  3.0, -1.0],
        [ 3.0, -1.0,  0.5],
        [ 0.5,  1.0,  1.0],
        [ 1.0,  1.0, -1.0],
];
// Training data labels
let ys: []f64 = [1.0, -1.0, -1.0, 1.0];

// Make MLP
const mlp = ml::mlp_make(3, [4, 4, 1]);

// Perform gradient descent
const res = ml::do_grad_descent(mlp, xs, ys, 200, -0.02);
fmt::printfln("loss: {:.5f}", res.loss.data)!;
fmt::printf("predicted labels:")!;
for (let y_pred .. res.ys_pred) {
        fmt::printf(" {:.5f}", y_pred.data)!;
};
fmt::println()!;

Example output:

loss: 0.07261
predicted labels: 0.96868 -0.97724 -0.95116 0.94189

#Autograd Example

use ml;
use fmt;

export fn main() void = {
	let a = ml::add(ml::value_make(0.4), ml::value_make(0.2));
	let b = ml::pow(a, ml::value_make(2.0));
	let c = ml::mul(b, ml::value_make(1.2));
	let d = ml::div(c, ml::value_make(2.0));
	let e = ml::sub(d, ml::value_make(0.1));
	let f = ml::tanh(e);
	ml::graph_backward(f);
	ml::graph_print(f, 0);
};

The above code prints the following graph:

tanh (data 0.11548) (grad 1.00000)
        + (data 0.11600) (grad 0.98666)
                * (data 0.21600) (grad 0.98666)
                        * (data 0.43200) (grad 0.49333)
                                ^ (data 0.36000) (grad 0.59200)
                                        + (data 0.60000) (grad 0.71040)
                                                const (data 0.40000) (grad 0.71040)
                                                const (data 0.20000) (grad 0.71040)
                                        const (data 2.00000) (grad 0.00000)
                                const (data 1.20000) (grad 0.17760)
                        ^ (data 0.50000) (grad 0.42624)
                                const (data 2.00000) (grad -0.10656)
                                const (data -1.00000) (grad 0.00000)
                * (data -0.10000) (grad 0.98666)
                        const (data 0.10000) (grad -0.98666)
                        const (data -1.00000) (grad 0.09867)

#Prior Art

• A Gentle Introduction to torch.autograd
• karpathy/micrograd
• “The spelled-out intro to neural networks and backpropagation: building micrograd” by Andrej Karpathy

#Contributing

Send patches to ~vladh/general@lists.sr.ht using subject prefix [PATCH hare-neural]. For example:

git config sendemail.to '~vladh/general@lists.sr.ht'
git config format.subjectPrefix 'PATCH hare-neural'
git send-email HEAD^

For more information, see git-send-email.io.