This post is an attempt to demystify backpropagation, which is the most common method for training neural networks. This post is broken into a few main sections: Explanation Working through examples Simple sample C++ source code using only standard includes … Continue reading

# Category Archives: Math

In a previous post I showed how to use dual numbers to be able to get both the value and derivative of a function at the same time: Dual Numbers & Automatic Differentiation That post mentions that you can extend … Continue reading

In the 90s before I was a professional programmer / game developer I looked at neural networks and found them interesting but got scared off by things like back propagation, which I wasn’t yet ready to understand. With all the … Continue reading

This post talks about how to render images like the below in real time using ray tracing. Some realism in the images come from reflection and refraction, but the real icing on the cake comes from Fresnel, total internal reflection … Continue reading

The last post showed how to fit a equation to a set of 2d data points, using least squares fitting. It allowed you to do this getting only one data point at a time, and still come up with the … Continue reading

This Post In Short: Fit a curve of degree N to a data set, getting data points 1 at a time. Storage Required: 3*N+2 values. Update Complexity: roughly 3*N+2 additions and multiplies. Finalize Complexity: Solving Ax=b where A is an … Continue reading

This is an extension of a paper I wrote which shows how to use the linear texture sampling capabilities of the GPU to calculate points on Bezier curves. You store the control points in the texture, then sample along the … Continue reading

This is a super short post about something I want to be able to reference again in the future (: Let’s say that you need to average N things, where you don’t know what the final N will be, but … Continue reading

I’ve been working on getting a better understanding of the Discrete Fourier Transform. I’ve figured out some things which have really helped my intuition, and made it a lot simpler in my head, so I wanted to write these up … Continue reading

I have another item to add to the pile of ways to calculate sine without trig. Here are the previous ways before we start: Four Ways to Calculate Sine Without Trig A Fifth Way to Calculate Sine Without Trig This … Continue reading