thoughtnet

Jan 22 2024

Noisy is Better — Improving Performance with Gaussian Noise

danielmewes

In my previous post about ThoughtNet, an attention-based neural architecture for variable-compute inference, I highlighted two limitations that I encountered with it: Slow and inconsistent convergence during training time Poor generalization on multiplication tasks, despite great...

machine learning experiments thoughtnet artificial neural networks

Nov 28 2023

Variable-Time Neural Computation

danielmewes

The majority of today’s artificial neural network (ANN) architectures perform a constant amount of computation at inference time regardless of their inputs. This includes all recent GPT-style LLMs1 and other transformer-based architectures. Whether you ask an LLM to complete the series “1, 2,...

machine learning experiments thoughtnet artificial neural networks