How Many Bytes in Human Memory?

by Ralph C. Merkle
(appeared in Foresight Update No. 4, 1988)
([email protected])

Today it is commonplace to compare the human brain to a computer, and the human mind to a program running on that computer. Once seen as just a poetic metaphore, this viewpoint is now supported by most philosophers of human consciousness and most researchers in artificial intelligence. If we take this view literally, then just as we can ask how many megabytes of RAM a PC has we should be able to ask how many megabytes (or gigabytes, or terabytes, or whatever) of memory the human brain has.

Several approximations to this number have already appeared in the literature based on ‘hardware’ considerations (though in the case of the human brain perhaps the term ‘wetware’ is more appropriate). One estimate of 10**20 bits is actually an early estimate (by Von Neumann in ‘The Computer and the Brain’) of all the neural impulses conducted by the brain during a lifetime. This number is almost certainly larger than the true answer. Another method is to estimate the total number of synapses, and then presume that each synapse can hold a few bits. Estimates of the number of synapses have been made in the range from 10**13 to 10**15 — with corresponding estimates of memory capacity.

A fundamental problem with these approaches is that they rely on rather poor estimates of the raw hardware in the system. The brain is highly redundant and not well understood: the mere fact that a great mass of synapses exists does not imply that they are in fact contributing to the memory capacity. This makes the work of Thomas K. Landauer very interesting for he has entirely avoided this hardware guessing game by measuring the actual functional capacity of human memory directly (‘How Much Do People Remember? Some Estimates of the Quantity of Learned Information in Long-term Memory’ in Cognitive Science 10, 477- 493, 1986).

Landauer works at Bell Communications Research — closely affiliated with Bell Labs where the modern study of information theory was begun by C. E. Shannon to analyze the information carrying capacity of telephone lines (a subject of great interest to a telephone company). Landauer naturally used these tools by viewing human memory as a novel ‘telephone line’ that carries information from the past to the future. The capacity of this ‘telephone line line’ can be determined by measuring the information that goes in and the information that comes out — the great power of modern information theory can be applied.

Landauer reviewed and quantitatively analyzed experiments by himself and others in which people were asked to read text; look at pictures; hear words, short passages of music, sentences and nonsense syllables. After delays ranging from minutes to days the subjects were then tested to determine how much they had retained. The tests were quite sensitive (they did not merely ask ‘What do you remember?’) often using true/false or multiple choice questions, in which even a vague memory of the material would allow selection of the correct choice. Often, the differential abilities of a group that had been exposed to the material and another group that had not been exposed to the material were used. The difference in the scores between the two groups was used to estimate the amount actually remembered (to control for the number of correct answers an intelligent human could guess without ever having seen the material). Because experiments by many different experimenters were summarized and analyzed, the results of the analysis are fairly robust; they are insensitive to fine details or specific conditions of one or another experiment. Finally, the amount remembered was divided by the time alloted to memorization to determine the number of bits remembered per second.

The remarkable result of this work was that human beings remembered very nearly two bits per second under ALL the experimental conditions. Visual, verbal, musical, or whatever — two bits per second. Continued over a lifetime, this rate of memorization would produce somewhat over 10**9 bits, or a few hundred megabytes.

While this estimate is probably only accurate to within an order of magnitude, Landauer says ‘We need answers at this level of accuracy to think about such questions as: What sort of storage and retrieval capacities will computers need to mimic human performance? What sort of physical unit should we expect to constitute the elements of information storage in the brain: molecular parts, synaptic junctions, whole cells, or cell-circuits? What kinds of coding and storage methods are reasonable to postulate for the neural support of human capabilities? In modeling or mimicking human intelligence, what size of memory and what efficiencies of use should we imagine we are copying? How much would a robot need to know to match a person?’

What is interesting about Landauer’s estimate is its small size. Perhaps more interesting is the trend — from Von Neumann’s early and very high estimate, to the high estimates based on rough synapse counts, to a better supported and more modest estimate based on information theoretic considerations. While Landauer doesn’t measure everything (he did not measure, for example, the bit rate in learning to ride a bicycle nor does his estimate even consider the size of ‘working memory’) his estimate of memory capacity suggests that the capabilities of the human brain are more approachable than we had thought. While this might come as a blow to our egos, it suggests that we could build a device with the skills and abilities of a human being with little more hardware than we now have — if only we knew the correct way to organize that hardware.

The post How Many Bytes in Human Memory? appeared first on Paranoia Magazine.