Overview

Most "neural network demos" are a thin UI over a library that does all the real work. NeuroForge is the opposite: the learning algorithm is the project. The forward pass, the gradients, the optimizers, the weight initialization — all hand-written and readable in ~250 lines. Build a network by hand, drop it on a spiral, and watch the boundary bend, the connection weights thicken and fade, and the loss/accuracy curves tick down live. View source, and the math is right there.

Tech Stack

Challenges

• A wrong-but-not-crashing gradient is the worst class of bug — the first run threw no error, but the boundary simply never moved.
• The decision-boundary heatmap is an 80×80 grid — 6,400 forward passes per frame — which destroys the framerate if you recompute it naively.
• Keeping the picture honest: a teaching tool that draws an "approximate" boundary is worse than useless.
• Re-implementing the charts, the RNG and weight-init by hand, because the whole point was zero dependencies.

Solution

The silently-wrong gradient got caught the way safety-critical control work taught me — measure the clever code against ground truth. A central-difference numerical gradient check localized a batch-averaging bug to one line, and now agrees with the analytic backprop to ~2e-10. The heatmap renders into a tiny 80×80 offscreen canvas scaled up by the GPU, and only recomputes when weights actually change. Crucially, the renderer calls the real forward() on real featurized coordinates — so toggle X₁² and the boundary becomes a true circle, because that's the function the net is now fitting, not a visual fake.

Outcome

It trains XOR, the circle, moons and the two-spiral problem to ~100% accuracy at a steady 60fps, with the weights reorganizing live as the optimizer works. If you want to understand backpropagation, reading 250 lines of commented JavaScript that demonstrably solves a spiral beats another diagram — which is exactly why I built it instead of hand-waving.

What I'd do differently

Nested JS arrays mirror the math but leave performance on the table; I'd move training to a Web Worker and swap in Float32Array. The single sigmoid output means binary classification only — softmax + cross-entropy would open up multi-class — and a "replay" scrubber over recorded weight snapshots would let you scrub through training history.