High capacity associative memories and connection constraints

• Published in: Connection science Vol. 16; no. 1; pp. 47 - 65
• Main Authors: Davey, Neil, Adams, Rod
• Format: Journal Article
• Language: English
• Published: London Taylor & Francis Group 01-03-2004; Taylor & Francis; Taylor & Francis Ltd
• Subjects: Applied sciences; Artificial intelligence; associative memory; Biological and medical sciences; Cognition. Intelligence; Computer science; control theory; systems; dilution; Exact sciences and technology; Fundamental and applied biological sciences. Psychology; General aspects; General aspects. Models. Methods; Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects); Miscellaneous; neural network; Neural networks; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; symmetry; Vertebrates: nervous system and sense organs; weight signs; Engineering; Associative memory; Selection problem; Perceptron; Memory capacity; Connection; Hebbian learning; Neural network; Cognitive science; Artificial intelligence
• ISSN: 0954-0091; 1360-0494
• DOI: 10.1080/09540090310001659981

High capacity associative neural networks can be built from networks of perceptrons, trained using simple perceptron training. Such networks perform much better than those trained using the standard Hopfield one-shot Hebbian learning. An experimental investigation into how such networks perform when the connection weights are not free to take any value is reported. The three restrictions investigated are: a symmetry constraint, a sign constraint and a dilution constraint. The selection of these constraints is motivated by both engineering and biological considerations.