Selected Newsgroup Message
From: "Sergio Navega" <snavega@ibm.net>
Subject: Re: Analog vs Digital (reposted)
Date: 28 Jan 1999 00:00:00 GMT
Message-ID: <36b0d3f8@news3.ibm.net>
References: <78np62$7b4@ux.cs.niu.edu> <78okri$pen@journal.concentric.net>
<78ot9t$924@ux.cs.niu.edu> <78qcjd$9aa@journal.concentric.net>
X-Notice: should be reported to postmaster@ibm.net
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: postmaster@ibm.net
X-Trace: 28 Jan 1999 21:17:44 GMT, 200.229.242.9
Organization: SilWis
Newsgroups: comp.ai.philosophy
Finally, nuts and bolts!
modlin@concentric.net wrote in message
<78qcjd$9aa@journal.concentric.net>...
>
>[big snip]
>For direct sensory cells, the function of the cell is to produce a
>series of firings at a rate proportional to the magnitude of the
>stimulus, which is a very analogish transform. More incoming energy
>produces more firing. Simple.
>
I agree that this is a reasonable way to see the things, but I'm
a little bit concerned with the kind of reasoning it can derive:
we discuss a methodology to treat a particular form of codification
(that may even be very close to the real thing) but we risk to
be devoured by the details and forget the core questions.
I'll propose one way of discussing things without the need to
define architectures. We must establish all the principles that
the system must obey, or else we may lose a great chance of
finding our way home.
What are the principles we must obey? Here's an incomplete list:
a) The need of sensory input
Whatever the architecture we imagine, I guess it will have to
accept some kind of external input. So our architecture will
have to have something that translates what the world informs
into something meaningful for processing. It can be a keyboard,
it can be a video camera, it can be a microphone.
b) Innate feature detection
Evolution provided us with something that pre-processes what
has been captured by sensory input in a significant manner.
In our system, this can be an ASCII to binary converter or
a string detection mechanism or a counting subroutine or
a similarity judge.
c) Recognition of repeating patterns
This architecture will have to recognize patterns. Not only
to annotate the repetitions, but to be able to do two more
things:
c.1) To allow perception of patterns that are "similar"
(using one of the innate feature detectors)
c.2) To allow prediction (pattern completion)
(using another of the innate stuff)
d) Categorization
From c.1 and c2 above, we should be able to assemble mechanisms
that categorizes patterns, transforming whole "sentences" of
input signals into the relation of a few "concepts".
When the system learns what are the fundamental aspects that
must be remembered about categories, it is able to
*recognize* it in the future. I ascribe to this recognition
ability one of the most important preconditions to understand
natural language (with all its nuances and metaphors).
e) Assembly of Causal Models
Based on categorization and coupled with the same mechanisms
of pattern detection, the system should be able to recognize
causal links between categories, informed by events of the
world, interaction and tutors (teachers).
There's more items (f, g,..) and there's some aspects of
real world knowledge and *grounding* that (although fundamental)
I'll not address now because I want to go to a more
practical aspect, and avoid boring the readers.
I wrote the following text some days ago, it is a
symbolic way to see the problem but I guess its ideas
can be duplicated using connectionist schemes. I appeal
to the patience of the readers to see the point I want
to make:
--------------------------------------------------------
Suppose we want to design one agent capable of interpreting simple
images. How can we teach this agent to recognize a tree? I'll
reduce the problem here to fit the space of this message.
Imagine that the following pattern is a symbolic representation of
the digitization of the edge of an image of a trunk of a tree
(taken as a segment of a horizontal scan, each line is more or less
the equivalent of a spike train of the optic nerve):
a
aaa
aa
aaaa
a
aaa
aa
aaaa
a
aaa
aa
aaaa
I'm concerned with the process we may be using to look for
regularities in this pattern. Eventually, our system will
find that this pattern is comprised of one irreducible
sequence that I will call EEL (Edge Element) but the system may
call it, say, #SYM3242:
a
EEL -+
aaa |
aa |
aaaa -----+
a EEL -+
aaa |
aa |
aaaa -----+
a EEL -+
aaa |
aa |
aaaa -----+
So this pattern can be subsumed into this sequence:
EEL
EEL
EEL
which is pretty regular. This is the first transformation
that the system applies: found a regularity? *Give it a name*.
The sequence of three EELs is also redundant. What to do?
Replace it by a name. Say #SYM9982. But we can inform the
system that this is a "concept" with a *public* name: we
call it EDGE (this public name will not affect the performance
of the system, it will only ease our communication). Now we
must go on and present to the system *another* example of
EDGE:
a
aaa
aa
aaaa
a
aaa
aa
aaaa
As has been learned, the system will *recognize* that this
pattern is EEL EEL. So its conclusion is "EEL EEL is also
an EDGE". His concept of EDGE will receive this added
information (it will keep both exemplars as separate
instances). We move on with another exemplar of EDGE:
a
aaa
aa
aaaa
a
aaa
aa
aaaa
a
aaa
aa
aaaa
a
aaa
aa
aaaa
a
aaa
aa
aaaa
a
aaa
aa
aaaa
"EEL EEL EEL EEL EEL EEL is also an EDGE". But this is
the time for the system to conclude (inductively) that
EDGE is something comprised of a variable number of EELs:
[n:EEL] -> EDGE { n occurrences of EEL }
and that typical values for n are in the range of 2 to 6 (it
may revise this in the future when presented to other instances
of EDGEs).
Now imagine that the following pattern is a horizontal scan of the
*central* part of a trunk of a tree:
aa
a
aa
a
aa
a
aa
a
By a similar process (and using more exemplars), our system will
conclude that central trunk is composed of:
[n:CT] -> TRUNK
Where CT stands for "aa a", TRUNK is the public name (given by us)
of the internal concept found by the system.
It is time to tell the system that a horizontal scan of a tree is
composed of two edges with a central trunk in the middle:
[TREE] -> [EDGE] [TRUNK] [EDGE]
This is pretty much similar to a "definition", in natural language,
of what is a scan of a tree. Keep in mind that this process, although
artificially simple in our example, may be what happens during the
learning of more "complex" concepts. Notice that we can have
complex concepts assembled through the use of other concepts up to
the basic "sensory components", that are the initial grounding of
the system. Also notice that we could have started with the
definition of [TREE], without defining what is an [EDGE] or
[TRUNK]. In fact I believe that children learn most of the things
this way (adults also do this when they study a new subject; it
takes some time until they can "pave" the new concepts all the
way toward more basic ones).
What would we have now if we left our system analyze the following
sequence (taken, say, from a natural scene)?
a
aaa
aa
aaaa
a
aaa
aa
aaaa
a
aaa
aa
aaaa
aa
a
aa
a
aa
a
aa
a
aa
a
aa
a
a
aaa
aa
aaaa
a
aaa
aa
aaaa
a
aaa
aa
aaaa
a
aaa
aa
aaaa
Seen for the first time, this would not be obvious neither for the
system neither for us. But the system already knows about some concepts.
So besides initiating a thread doing the "hard work" (trying to find all
regularities from the start), it will *also* start *other* threads
in which *previously acquired concepts* are put to use. Eventually,
one of the threads (taking the resulting processing from several
others, which means being *spread activated by others*) will settle
discovering the following sequence:
EEL EEL EEL CT CT CT CT EEL EEL EEL EEL
[EDGE] [TRUNK] [EDGE]
[ T R E E ] !!!
Because the amount of work to reach this conclusion is much less
than the amount of work of the "hard approach" (because the
latter started from zero and because of the parallelism of
other threads recognizing elements very fast), this conclusion
will *pop out* much faster and the system will be able to recognize
the feature in a minimum amount of time. I guess this is
not only efficient, but also neuropsychologically plausible
(remember, our brain is parallel!).
Obviously, this "method" has dozens of unaccounted details
but I find it appealing. One of the things I like most is
its "Gestalt" aspect.
Say you're given a phone number to dial. Here it is:
(321) 212-298
When you fix your eyes in that number, something odd surfaces.
After a couple of seconds, you may wonder if there's a digit
missing, because your brain is used to find things like:
(xxx) xxx-xxxx
and not
(xxx) xxx-xxx
For me, this is an unconscious and automatic mechanism that
works independently of our will, alerting us of everything
that "doesn't fit" the rules (by the way, I think this is
related to Implicit Learning). Doesn't fit means is not easily
explained by the *rules* and methods we derived from the
learning that occurred because of previous experiences.
Some rules may be pretty fancy, with conditional situations,
specific exemplars, generalizations, etc. But they are very
fast, because they are processed in parallel with other
"thoughts" and most of the time only the relevant ones
are brought into action, because of the spreading activation
effect.
Some time ago I posted a message about a subdiscipline of
Psychology called Implicit Learning that deals with some
similar situations. One of the experiments is done with
"artificial grammars" that confirm experimentally the
ability that our brain has of unconsciously capturing
"hidden rules and regularities". We are cheated
by our brain because this happens "under the rug", we don't
have conscious access to the mechanism that are running. I see
no other way to understand the problem unless to figure out
how this mechanism works in a way to choose the best architecture
and the best algorithms to *functionally* duplicate it
in computers.
Regards,
Sergio Navega.
Back to Menu of Messages Sergio's Homepage
Any Comments? snavega@attglobal.net