## Solving N equations and N unknowns: The Fine Print (Gauss Jordan Elimination)

In basic algebra we were taught that if we have three unknowns (variables), it takes three equations to solve for them. There’s some fine print though that isn’t talked about until quite a bit later. Let’s have a look at … Continue reading

## Neural Network Recipe: Recognize Handwritten Digits With 95% Accuracy

This post is a recipe for making a neural network which is able to recognize hand written numeric digits (0-9) with 95% accuracy. The intent is that you can use this recipe (and included simple C++ code, and interactive web … Continue reading

## Neural Network Gradients: Backpropagation, Dual Numbers, Finite Differences

In the post How to Train Neural Networks With Backpropagation I said that you could also calculate the gradient of a neural network by using dual numbers or finite differences. By special request, this is that post! The post I … Continue reading

## How to Train Neural Networks With Backpropagation

This post is an attempt to demystify backpropagation, which is the most common method for training neural networks. This post is broken into a few main sections: Explanation Working through examples Simple sample C++ source code using only standard includes … Continue reading

## Multivariable Dual Numbers & Automatic Differentiation

In a previous post I showed how to use dual numbers to be able to get both the value and derivative of a function at the same time: Dual Numbers & Automatic Differentiation That post mentions that you can extend … Continue reading

## Incremental Least Squares Surface and Hyper-Volume Fitting

The last post showed how to fit a equation to a set of 2d data points, using least squares fitting. It allowed you to do this getting only one data point at a time, and still come up with the … Continue reading

## Incremental Least Squares Curve Fitting

This Post In Short: Fit a curve of degree N to a data set, getting data points 1 at a time. Storage Required: 3*N+2 values. Update Complexity: roughly 3*N+2 additions and multiplies. Finalize Complexity: Solving Ax=b where A is an … Continue reading

## The Secret to Writing Fast Code / How Fast Code Gets Slow

This is a “soft tech” post. If that isn’t your thing, don’t worry, I’ll be returning to some cool “hard tech” and interesting algorithms after this. I’ve been abusing the heck out of the GPU texture sampler lately, so be … Continue reading

## Minimizing Code Complexity by Programming Declaratively

Writing good code is something all programmers aspire to, but the definition of what actually makes good code can be a bit tricky to pin down. The idea of good code varies from person to person, from language to language, … Continue reading

## Low Tech Homomorphic Encryption

Homomorphic encryption is a special type of encryption that lets you do calculations on encrypted values as if they weren’t encrypted. One reason it’s desired is that secure computing could be done in the cloud, if practical homomorphic encryption were … Continue reading