Solving N equations and N unknowns: The Fine Print (Gauss Jordan Elimination)

In basic algebra we were taught that if we have three unknowns (variables), it takes three equations to solve for them. There’s some fine print though that isn’t talked about until quite a bit later. Let’s have a look at … Continue reading

Orthogonal Projection Matrix Plainly Explained

“Scratch a Pixel” has a really nice explanation of perspective and orthogonal projection matrices. It inspired me to make a very simple / plain explanation of orthogonal projection matrices that hopefully will help them be less opaque for folks and … Continue reading

How to Train Neural Networks With Backpropagation

This post is an attempt to demystify backpropagation, which is the most common method for training neural networks. This post is broken into a few main sections: Explanation Working through examples Simple sample C++ source code using only standard includes … Continue reading

Multivariable Dual Numbers & Automatic Differentiation

In a previous post I showed how to use dual numbers to be able to get both the value and derivative of a function at the same time: Dual Numbers & Automatic Differentiation That post mentions that you can extend … Continue reading

A Geometric Interpretation of Neural Networks

In the 90s before I was a professional programmer / game developer I looked at neural networks and found them interesting but got scared off by things like back propagation, which I wasn’t yet ready to understand. With all the … Continue reading

Raytracing Reflection, Refraction, Fresnel, Total Internal Reflection, and Beer’s Law

This post talks about how to render images like the below in real time using ray tracing. Some realism in the images come from reflection and refraction, but the real icing on the cake comes from Fresnel, total internal reflection … Continue reading

Incremental Least Squares Surface and Hyper-Volume Fitting

The last post showed how to fit a equation to a set of 2d data points, using least squares fitting. It allowed you to do this getting only one data point at a time, and still come up with the … Continue reading

Incremental Least Squares Curve Fitting

This Post In Short: Fit a curve of degree N to a data set, getting data points 1 at a time. Storage Required: 3*N+2 values. Update Complexity: roughly 3*N+2 additions and multiplies. Finalize Complexity: Solving Ax=b where A is an … Continue reading

Evaluating Polynomials with the GPU Texture Sampler

This is an extension of a paper I wrote which shows how to use the linear texture sampling capabilities of the GPU to calculate points on Bezier curves. You store the control points in the texture, then sample along the … Continue reading

Incremental Averaging

This is a super short post about something I want to be able to reference again in the future (: Let’s say that you need to average N things, where you don’t know what the final N will be, but … Continue reading