Study and Learning

Maxwell's Demon

James Clerk Maxwell was a 19th century mathematician and physicist. He described 'Maxwell's Demon' -- a mythical creature that strives to increase the order of a system (thus decreasing entropy) without a concomitant increase in total net entropy (disorder). However, this defies the second law of thermodynamics, a fundamental law of physics, which states that entropy in a system must increase. Although order can be increased in one place for a short period of time, disorder must also eventually increase. Hence, the demon defies the laws of physics. We should avoid invoking this demon in systematic processes that we develop, including learning processes.[1]

This principle of net entropy applies to learning, because the brain operates via rate-limited biochemical and electrochemical functions subject to the laws of physics. Most learning models are completely oblivious to this principle. Learning requires modification of synaptic interfaces at the dendritic level.[2] There are cell and tissue based regulatory pathways for learning that are subject to gene regulation with protein synthesis.[3]

There are both deterministic and stochastic aspects of the learning process. The deterministic aspect of learning indicates how much information can be retained. For learning to occur, only a finite number of synaptic modifications can be established per unit time. These synaptic modifications cannot be increased by increasing the rate of information delivery.[4] The stochastic aspect of learning involves a decay of learning over time. However, the loss of learned data elements is random, just like another stochastic process -- radioactive decay. One cannot tell precisely what data will be lost over time, but some loss will inevitably occur.[5]

Learning requires information transfer at key points in time. The brain must first process and store short-term memories. The important paleocortical regions involved with short-term memory are the medial temporal lobe structures that include the hippocampus, amygdala, and adjacent cerebral cortex.[6] Transfer of short-term memory to intermediate memory occurs between 20 to 40 minutes following initial formation of memory, through axonal projections from hippocampus to cerebral neocortex. Significant transfer of intermediate memory to long-term memory occurs at 24 to 48 hours. The memories stored in the neocortex eventually become independent of the medial temporal lobe system.[7]

The process of forming memory requires synaptic modification through a process known as long-term potentiation (LTP) in which a long-lasting enhancement of synaptic transmission occurs with repetitive stimulation of excitatory synapses. LTP is a biochemical process that occurs in the hippocampus and is mediated by N-methyl-D-aspartate (NMDA) receptors as well as cyclic AMP-responsive element binding protein (CREB). Aging has been shown to alter the expression and distribution of N-methyl-D-aspartate (NMDA) receptors in many different brain regions, including the hippocampus, and may explain the diminished learning capacity and dementia seen in the elderly [8].

Short-term memory retention begins to decay at 20 to 40 minutes. There is a finite amount of information that can be transferred from short term to long-term memory per unit time, but some of that information will be lost. A reduction in the stochastic process will facilitate long-term retention of information. There are key branch points in learning where reinforcement can reduce the stochastic process. These branch points occur at 20 to 40 minutes (transfer to intermediate memory) and at 24 to 48 hours (transfer to long-term memory). Review of the information in the learning process at these critical branch points will reduce the loss of information.

There is a finite learning rate that is bounded by entropy-driven molecular processes, and this cannot be exceeded, just as Maxwell's demon cannot create order without simultaneously creating disorder. Optimal teaching occurs when increasing order (decreased entropy) in the brain occurs over a short time, so that overall disorder (increased entropy) is also allowed to occur.

In most institutions of higher learning, including medical schools, the timing of teaching exercises is determined by convenient clock intervals and by tradition, subject to those eight fateful words, 'But we have always done it that way.' Thus, a lecture mode of teaching is governed in most institutions by a rigid schedule of one-hour intervals. The lecturer has set amount of material that must be delivered, and if there is an hour available, then an hour will be used. Within that hour, the lecturer may race through the material, oblivious to the deterministic aspect of the learning process. The stochastic aspect is probably recognized, at least in part, as a price to pay in this educational system, though any given lecturer's bias is typically that someone else's material will be the part that is forgotten.

If we were to proactively exclude Maxwell's demon from the learning process, we would break up delivery of information into smaller segments of no more than 20 minutes. We would present the material at a constant pace, without trying to overwhelm ourselves with information. We would stop at intervals to reinforce the material that had been presented. We would make time available in our schedule for periodic review within the next day or two. We would not wait until the night before the examination to 'cram' as much information as possible, only to encounter increasing entropy during the examination period.

References:

1. McClare CW. Chemical machines, Maxwell's demon and living organisms. J Theor Biol. 1971;30(1):1-34.

2. Bhatt DH, Zhang S, Gan WB. Dendritic spine dynamics. Annu Rev Physiol. 2009;71:261-82.

3. Kandel ER. The molecular biology of memory storage: a dialogue between genes and synapses. Science. 2001;294:1030-1038.

4. Cartling B. Dynamics control of semantic processes in a hierarchical associative memory. Biol Cybern. 1996;74(4):385.

5. Hastings HM, Pekelney R. Stochastic information processing biological systems. Biosystems. 1982;15(2):155-168.

6. Pasquier F, Hamon M, Lebert F, Jacob B, Pruvo JP, Petit H. Medial temporal lobe atrophy in memory disorders. J Neurol. 1997;244(3):175-181.

7. Squire LR, Zola-Morgan S. The medial temporal lobe memory system. Science. 1991;253(5026):1380-1386.

8. Clayton DA, Grosshans DR, Browning MD. Aging and surface expression of hippocampal NMDA receptors. J Biol Chem 2002;26;277(17):14367-14369.