On Intelligence
When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now. His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' brains are built upon. The part of the brain that gives us emotions and motivations. The leftover from the reptiles that drive animals to survive, to find pleasure in a full belly,…
One of the most consistent theory about how our brain learns is described in HTM (Hierarchical Temporal Memory), a more biologically inspired neural network. See Jeff Hawkins' "On Intelligence". It is based on: * Input of continuous unlabeled time-based patterns. * Associative Hebbian Learning (when distinct inputs/patterns come together, they are neuron-wired together). Synapses can be modified via experience. See "Hebbian Theory". * The brain is a prediction machine: it is always trying to predict the future based on past learned patterns. Learning happens when reality does not match the…
Would you kindly care to explain how the field should change paradigm? Do you mean that " black boxing" the brain with input/output and hoping something will emerge is a non-productive endeavour, and that the neurophysiology, like a bottom up approach from biology/neuroscience, is the way to go? I'm a comp-sci student too, and considering what possibilities there are for studying BCI, machine intelligence and the brain. But I think that maybe, having approached it from a more biologically oriented angle would have been better. After having read "On intelligence" I'm just more aware how…
Yet, this doesn't answer my question. Human brain is obviously has so much more in it (visual cortex for starters, grid cells etc), and, in terms of neural network, much more sophisticated architecture. But still, there is a big probability that what we call "knowledge" and "reasoning" and "conciousness" are just a result of this sophisticated architecture. I.e. there is no special magical thing for "reasoning" that next generations of prediction models can't replicate. There is a faboulous book by Jeff Hawkins "On Intelligence" (2004) that explores this. I think main premise of it still…
I read/markered "On Intelligence" on my train commute to work and have scribbled a bunch of notes in the book. Pretty interesting and I like the basic idea of the memory-prediction framework, invariant representations, "melodies of patterns", focus on neocortex and the whole same general algorithm for all senses. I haven't had the time to research how far the general idea has gone or if it is relevant at all but the scetched examples were pretty interesting. I also found the random remark of "consciousness = what it feels like to have a neocortex" interesting. Glad to see that some smart…
I'm just going to throw this out there in the hope that someone enlightens me. Ever since I first read the book "On Intelligence" I thought Jeff Hawkings was on the right track with Strong AI. The insights I got from reading the book I thought were invaluable. I don't know how close they are to Strong AI or if they are closer then anybody else but I would expect that if they were making any real head way into the field that Google would already have made an offer to buy them. At least that is what I would do if I wanted to be the first to control the technology. The fact that nobody seems…
No one's mentioned On Intelligence by Jeff Hawkins yet... I'm surprised. Check it out here: http://www.onintelligence.org/ Great read on exactly this topic, and one of the more plausible paths to actually achieving machine intelligence. But it's not through the typical path followed by most AI research up until recently. He argues you need to look at how the neocortex actually works, how it communicates with the senses, stores data and learns patterns, in order to create anything artificial that displays intelligence. I'm inclined to agree. Our brains may seem to map to computer-like…
This doesn't seem like a perspective offering anything new and possibly involving a few problems. Jeff Hawkins' On Intelligence also emphasized the predictive qualities of intelligence but again, this was a new emphasis, not really a new idea but a new emphasis in that it's well known time is implicitly an element of nearly any cognitive process. Moreover, the current deep learning driven AI renaissance is has been most effective in predicting patterns in quite static phenomena; image recognition and competitive video and board games (game involve change over short periods but their context…
I'm a computational PhD student in the field -- I think you'd most enjoy "On Intelligence" by Jeff Hawkins, an inspiring book written by a hacker himself! If you'd like a very excellent, fascinating title on mathematics and art and mind, try "Godel, Escher, Bach" by Hofstadter. Another fun, classical book with more of an AI bend is "Society of Mind" by Marvin Minsky. For more classical neuroscience, "Phantoms of the Brain" by Ramachandran, well-known in perception research, is good. I haven't read it, but "In Search of Memory" is supposed to be excellent by Kandel, a memory researcher…
" With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI- 1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters" That is very impressive. Side note: Superficially reminds me of Hierarchical Temporal Memory from Jeff Hawkins "On Intelligence". Although this doesn't have the sparsity aspect, its hierarchical and temporal aspects are related. https://en.wikipedia.org/wiki/Hierarchical_temporal_memory https://www.numenta.com