As I learned more about Cognitive Science, I became very determined to discover what was known and what wasn't yet known about the mind and how we think. There were still some very big mysteries to solve. And many, many questions to be asked to properly explain the workings of the human mind. I was truly shocked to discover there wasn't even a unified definition of a concept, let alone an agreed-upon theory of how the entire mind actually worked. Perception seemed fairly well understood; how we get information from the world around us into our minds. But then what happens? How is that information encoded in our minds? There are many theories. And after the information is encoded, how is it remembered? Again, there are many theories for this. And what about the ability to be creative? How did this happen cognitively?
Cognitive Science seems largely divided into two main camps (or attitudes) that represent two quite different approaches. A student in cognitive science had to make a critical decision. Was your research orientation going to be 'hard' or 'fuzzy'? Was it 'top-down' or 'bottom-up'? The top-down approach encompassed artificial intelligence and rule-based systems. It was about developing an overriding structure, or framework, and organizing data within this framework. This approach had worked remarkably well to store and retrieve data (and is now used in contemporary computer operating systems), but it certainly isn't flexible enough to represent the way the mind actually works. The other approach - bottom-up - encompasses neural networks. This approach has some intrinsic advantages, because it starts out simple and evolves into higher levels of complexity through adaptive learning. This dynamic is very similar to real human learning. The somewhat perplexing disadvantage of this approach is its inability to evolve a syntax of any kind. At what point does structure appear? How does a framework evolve? How are rules determined?
A simple way to think about these two approaches is to imagine a solid cookie cutter and a piece of rising dough. The top-down approach is to take the cookie cutter and press it into the dough. Any dough outside the cookie cutter is simply not included. If the dough wants to rise beyond the boundary or top of the cookie cutter, it can't. The bottom-up approach is to watch the dough rise up to (somehow) assume the shape of the cookie cutter, before it even reaches the cookie cutter. How does the dough know to self-organize itself into the cookie cutter shape? Quite simply, it doesn’t. It's needs some guidance. It needs some foresight or some preliminary instruction.
There are problems with both of these approaches. The top-down approach superimposes a structure. It does not adapt to information that was not originally anticipated in the initial design. It doesn't learn. The bottom-up approach tries to evolve a structure. The bottom-up approach has a problem emerging some type of overriding framework, structure or form, within which to contain itself. A superimposed structure is too rigid. And a self-evolving structure is so chaotic as to be practically non-existent. It doesn’t have any rules. The truth must lie somewhere in between.
I started to wonder if something was happening in between these two approaches. Some middle process. Some transitional layer that could tie these two approaches together. There had to be some form of cognitive representation that was missing, that people just weren’t thinking about, even though clues about the existence of this mysterious area seemed to be everywhere, in different forms, in many different disciplines.
I started looking for clues. And in a very gradual, roundabout fashion, the clues began to point to an area that may answer many questions, and perhaps begin to unify the many micro-theories about cognition.
This is my idea. I think there is an area of knowledge representation in cognition that I have named 'protoglyphic'. Proto: ‘indicating the first or earliest or original’. Glyph: ‘a symbol, icon, pictograph, etc. that is used as a graphic representation’. I think the mind may have a set of protoglyphs that it uses at the very roots of cognition. These protoglyphs may be encoded at the very deepest level in the mind, just above the level of the neurons. Protoglyphs may be patterns that act as a categories and subcategories to give ‘structure’ or shape to information. Protoglyphs are not visual patterns, necessarily, but are better thought of as SPATIAL form patterns. Information is mapped onto these patterns. Protoglyphs act as the encoded structure for information. These patterns can be represented as a visual pattern, or a physical structure, or a temporal structure. There is a linear quality to protoglyphs. They are perceived over time in a linear fashion. Whether through visual sampling, tactile sampling, or auditory sampling, the perceived samples are ‘pieced together’ into a larger pattern. Protoglypic patterns are spatial. They are created by linear data sampling, and then superimposing (sharp and fuzzy) data samples together to form a pattern. This re-creation of the sampled bits relies on memory.
What are protoglyphs like?
I think protoglyphs are partly fuzzy, partly solid; like outline drawings of very basic patterns such as a spiral, a matrix (grid) or an arrangement of descending or ascending horizontal or vertical bars (some people theorize that these types of primary patterns are hard-wired in the brain), or protoglyphs can be made of more complex pattern fields, such as hexagonal shapes that are interlocking, or triangular shapes (Similar to the images seen in quilts!!). I think protoglyphic patterns are the fundamental underpinnings of cognition. These patterns are the structure onto which we map our knowledge categories. And oddly enough, I think the highest level mathematics echoes these low level patterns. As above, so below. Strangely enough, a mind must advance to a certain level of development, must acquire the ability to abstract, before it can ‘look back’ on itself, to examine its own patterns. By externalizing mathematical formulas, and applying them to other things in our world, we are essentially applying our own mental patterns that are originally derived from nature, back onto nature. Our mind makes an internal blueprint (from its observations and experiences with nature), abstracts the blueprint (in terms of principles and formulas), and then applies that blueprint back onto the external world to produce artifacts. Truly a strange process indeed, to be human.
My personal protoglyphs are like wireframe images. They’re just lines that represent a pattern. They can move in space, or be static. They’re like a set of templates and I map information onto them. Most people think the only way to organize information is using a tree structure, a DAG (directed acyclic graph). But some information, especially if it is associatively or temporally organized, does not easily map onto a tree structure. A tree structure cannot show simultaneity very well (or things happening in parallel) ??.
Some protoglyphs may be hard-wired, and some may be individually crafted. There is evidence to support both types. Some visual patterns seem common to all people. Others are entirely unique. For example, a concept of time. Some people think of time as a line. Some think of it as a shifting window, or moving interval. I think of time as a spiral, where each ‘loop’ is one year and the months map around the loop with January to the east and June to the west. My spiral starts at the bottom, with a small loop and gets larger every year as it moves upwards. I've spoken to other people who also represent time as a spiral, but theirs begins at the top and moves downward. Some people might call this a mental model of time. And it is. The protoglyph is the spiral pattern shape that is independent of any information mapping.
So what’s the difference between a ‘mental model’ and a protoglyph, you ask? Ah! Protoglyphs may be smaller components of mental models. Protoglyphs may act as a sort of alphabet for mental models. And mental models may act as the syntax for protoglyphs. Or, the protoglyph may be the actual pattern itself, independent of any information. It is a prototypical graphical pattern structure onto which information is mapped.
The reason I think protoglyphs exist is based on my research into the two primary ways we represent cognitive information. The most common way is known as top-down knowledge representation. It is hierarchical and rule-based. This is how computer software is designed. The rules are designed from the top-down. Nested hierarchies of rules that call each others’ functions. The second most common way to represent knowledge is known as bottom-up. Fuzzy and incomplete knowledge is cycled through a neural network until a prototype or ‘best instance’ can be formed. This form is predominately associative (relational) and based on fuzzy data clusters that produce category prototypes.
There are clues as to why something like the Protoglyphic Layer must exist in cognitive representation. Whether the protoglyphs are a layer in cognition, or a relay switch between two different systems, I don’t know right now. If you look at the circumstantial evidence, the clues I’ve included below, maybe you too will conclude what I’ve concluded. There is an area in cognition, the protoglyphic layer, that we need to investigate and research, if we are ever to truly understand the way in which our minds really work.
Clue No. 1 - Human learning is concrete, then abstract
We know that humans first learn associatively, and then evolve their learning to more rational forms that enable abstraction and complex symbol manipulation. And we know that this process takes about 12 years. There is a learning continuum in cognition that we all experience from a concrete one-to-one literal mapping/correlation with real world objects, to the ability to later abstract internally. We go from outside, to inside. It takes a while for our internal ‘world’ to develop. It is not fully developed until we are 11 or 12. And then it’s as if we quite literally do have our own personal world mapped out inside our head.
Clue No. 2 - A visual aspect to cognition
Scientists and thinkers the world over have indicated there is some visual aspect of thinking, some internal manipulation of ‘hidden forms or references’ where connections are made and rearranged at some deep level. There’s the famous discovery of the Benzene ring, the snake biting its tail dream. Einstein said there was a visual aspect to his thinking. Visual metaphors somehow imply that there is a deep level that represents knowledge, that can be accessed through visualization, dreams, somehow associating two seemingly un-related things. A deep-level visual structure explains how a person can think of a sunset and it is evocative of a song, or a poem, or even a mathematical formula that describes a physical descent, all as items that possess a similar structure.
Clue No. 3 - Underlying similarities in Art, Music, and Math
Common structures and patterns underpin all knowledge disciplines. Its as if the ‘bedrock’ is the same, at the deepest level, for all disciplines, for all forms of human knowledge. This implies that there is a common form of representation at some deep level in the human mind.
Clue No. 4 - Limitations of traditional software
Traditional software does not properly model the workings of human thought. It is not creative, adaptive or predictive.This implies that a hierarchical top-down approach is not the whole story of cognition. Neural network software is good for image/pattern recognition, but it offers no method to represent relationships between objects.
Clue No. 5 The artifacts of art, crafts and technology
The products produced by women and men seem to reflect predominantly different thinking styles. (Perhaps the focus here should be more on the different thinking styles and less on the genders that embody these styles.) As with most things, both genders exemplify both styles of thinking in varying degrees, both the logical and the creative. This has been proven in research, behavioural and physiological. It is perhaps best shown by the artifacts produced by the different genders. An interesting thing to note is that whenever there is a technological boom where the men are making profound advances in science and technology, there is a similar boom in the arts and crafts of women. You might think this is a backlash effect, but I think it is something else. I think classic, fundamental protoglyphic patterns have been recurrent throughout the history of peoplekind, predominantly in the arts of women. I think the artifacts of mena and women are parallel activities, offering expressions through different modes.
Clue No. 6 - Fractals in nature
What we know about fractals is that a simple formula that is reproduced can create a beautiful form that is identical at different resolutions or scales. Ferns, shells, trees, branches, flowers, mountains, clouds; forms in nature seem to be based on the repetition of a single, relatively simple mathematical formula.
Clue No. 7 - Nature does not have straight lines
Nature does not have straight lines. But mathematics (the notation of humans to quantify their world) does. A line is an ‘average’. It does not exist anywhere in nature. This is because a purely straight line cannot emerge, cannot grow in a world that is interconnected and constantly changing and subjected to cycles and seasons. Imagine a plant growing in the ground. Each day it is subjected to fluctuations in temperature, sunlight, rainfall, wind, the random meanderings of insects and animal, some predators, some egg-layers. And all the while it is attempting to grow upwards. Its growth cannot be straight, as it adapts to all of these outside influences. You will never, ever see a straight plant! Or tree, or river. Look around you. Nothing in nature is straight. (Note: I attribute this interesting point to my uncle Dr. Stephen Vonder Haar, a hydrogeologist in Berkeley, CA) If you think about this a bit more, you’ll realize that it’s rather odd that people come from a natural environment that is not straight, is not geometric, but we build geometric things. Where is this geometrical ordering coming from? Is it our mind’s unique way to efficiently record and organize information? I think it is. I think the Platonic view of the world is not something that exists ‘out there’ for us to discover. I think it is an efficient system our mind has evolved to make sense of fuzzy data out there in the natural, moving world.
Clue No. 8 - The Laws of Thermodynamics and Minimal Lexical Attachment.
All systems tend to disorder. All systems tend to lose energy. The natural state of a system with a high state of order (or energy) is to become disordered and disorganized. This dovetails nicely with the interworkings of nature. I think this is more an attribute of nature trying to establish equilibrium amongst. In cognition, there is a theory in language called Minimal Lexical Attachment. Given two ambiguous paths in a sentence structure, this theory describes how the mind will always recall the path that costs the least amount of energy to ‘traverse’. The best example is when a sentence has an ambiguous structure, when it can be grouped in two different ways with two different prepositional phrases. The mind will choose the shortest path as the most obvious way to interpret the sentence. The important point here is that the organization of knowledge within the mind is done with efficiency in mind. Nature, by nature, is not wasteful with energy. And neither are our minds. Our minds organize information in a way that is efficient and straightforward.
Clue No 9 - Modality independence
While vision is our dominant modality for getting information from our environment, we also have auditory and tactile information (along with olfactory, of course). Note here: how the deaf use sign language to create ‘forms’ in space that convey information. If you were to photograph the ‘trails’ of the hands over time through space during deaf signing, they create a 3-dimensional shape. A protoglyph is not a visual pattern, necessarily, although if one is pre-dominantly visual, it will manifest itself as a visual pattern. If one is blind, it may manifest as an auditory, tactile or olfactory pattern. Other clues here are in synaesthesia (The Man who tasted shapes)
Clue No 10 - Autism
Autism and Temple Grandin. Her explanation of the strengths and weaknesses of being autistic. The video player in her head. The primitive emotions. Visual thinking. A bottom-up approach. How she forms prototypes and categories.
Clue No 11 - We're pattern makers.
We’re not just pattern recognizers. We’re also prolific pattern makers. I think this ability may be attributed to the fact that we are pattern encoders – spatial pattern encoders. Because we have protoglyphic patterns encoded in our minds, we can produce and reproduce patterns. A pattern is a module that is repeated. Repetition is an action that occurs over time. Patterns are a function of time. We are pattern-makers.
Clue No. 12
Humor is an indication of the way the brain may represent patterns. Normally we go along the main track and suddenly the punch-line makes sense and we’re taken to the sidetrack and in hindsight it’s perfectly logical. The essence of humour is you’re taken to a different position and from that position looking backwards, it’s perfectly logical.
(Sharon Mascall/Susan Mackie – Are we hardwired for creativity? DeBono Institute)
Clue No. 13
Mental Models (from Wikipedia)
A mental model is an explanation in someone's thought process for how something works in the real world. It is a kind of internal symbol or representation of external reality, hypothesised to play a major part in cognition. The idea is believed to have been originated by Kenneth Craik in his 1943 book The Nature of Explanation. After the early death of Craik in a bicycle accident, the idea was not elaborated on until much later. Two books, both titled Mental Models, appeared in 1983 [1]. One was by Philip Johnson-Laird, a psychology professor at Princeton University. The other was a collection of articles edited by Dedre Gentner and Albert Stevens. See Mental Models (Gentner-Stevens book). Since then there has been much discussion and use of the idea in human computer interaction and usability by people such as Donald Norman and by Steve Krug in his book Don't Make Me Think. Walter Kintsch and Teun A. van Dijk, using the term situation model (in their book Strategies of Discourse Comprehension, 1983), showed the relevance of mental models for the production and comprehension of discourse. These are just a couple of examples among many, many others.
Clue No. 14
The work of artists...like Paul Klee. Why do his paintings in the Guggenheim museum that are celebrated as masterpieces look like handmade quilts made by women? Klee was searching to represent ‘the reality behind visible things’. So why then does so much of his work look like the arts and crafts of women throughout history?The rugs, patchwork quilts, paintings made by the Australian aboriginals?
