Selected Newsgroup Message
From: "Sergio Navega" <snavega@attglobal.net>
Subject: Re: Rationality and Formalizability
Date: 04 Nov 1999 00:00:00 GMT
Message-ID: <38216ee3_3@news3.prserv.net>
References: <3819F05E.203FF675@ix.netcom.com>
<s1pmvur9iqi90@corp.supernews.com> <uVIdOGmUwIEcwDiAPPz6X2XfvGpb@4ax.com>
<s1r24adgqof72@corp.supernews.com> <fpceOD=yc5Gx4DyILIKb3uPgOIHb@4ax.com>
<7vnhov$tlf$1@nntp6.atl.mindspring.net>
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: abuse@prserv.net
X-Trace: 4 Nov 1999 11:32:51 GMT, slip-32-102-103-40.sp.br.prserv.net
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.philosophy
Stephen Harris wrote in message <7vnhov$tlf$1@nntp6.atl.mindspring.net>...
>
>I am under the impression that a system that cannot be formalized
>can also not be rendered as an effective procedure(algorithm).
>The *they above refers to a lack of predictability which is
>equivalent to no knowable effective procedure(which produces
>the same results whenever initialized).
>
Stephen, I guess that you touched a central point here.
I'll try to say some words about a system that I suppose
is not formalizable but *can* be implemented.
First, let me rewrite your ideas using different words,
to see if I got it right. You seem to imply that unless we
come up with a proper formalization, we won't be able to
implement it. This is ok for most of our programming
activities, but it may not be the case for the human mind.
I suggest that we don't need to "formalize" the human mind
in order to implement it (and look, I'm fond of
computationalists!).
I suggest that, in reality, the human mind is not "formalizable",
not because it goes against some silly mathematical principles,
but because the mind transcends the mere description of algorithms.
Now that is a dangerous paragraph, which may be easily misunderstood,
so I'll try to clarify it a bit (it is *not* what you're thinking
I'm thinking).
I suspect that the "algorithm" for the mind is, indeed,
very, very simple. I mean, simple if compared with, say,
the size of the executable for the Microsoft Word. Word
appears to be a much more complex program than the "program"
of the human mind. How come?
It is often assumed that the human mind is the product of
a special program running in the biological hardware substrate.
So what we (AI researchers) must discover is that program.
This is, IMO, the wrong way to see things, as it conducts us
to focus our attention on the discovery of that "program",
leaving everything else below the rug. What appears to be
important for a mind is the kind of constructs it builds
*from* the "data" it receives of the world. So, abusing
again this awful analogy, it is not just a question of
program, it is also (if not preponderantly) a question
of "data".
What seems complex in the human mind is not the "basic"
algorithm, but the information structure it builds. It takes
a lifetime to build a suitable structure like that. When
someone tries to make a program to answer to:
All humans are mortals
Socrates is human.
Is Socrates mortal?
What is at stake here is *not* the mere logic of it: it is a
real lot of things, below it. Just "simulating" or formalizing
what appears to be the chain of thought in this example will
inevitably conduct us to the impression that the "program"
must know how to do, for instance, modus ponens. In my way to
see the things, no human is born knowing modus ponens. When
we think, we don't use such constructs, it seems to be a
side-effect of another kind of process.
I find that we will be able to (eventually) formalize this
basic algorithm, which I expect to be relatively (probably
deceptively) simple. But the content of the mind (which
emerges when this algorithm is ran in the "brain" and its
body is put in contact with the world) will be much more
than its simple definition, it will be the overall
"status quo" of the system. Add to that the complication of
the probable modification of this basic algorithm as new
experiences come through.
All this was just to say that it is misleading to look for
logical alternatives for formalization of intelligence.
Although there are lots of AI researchers still pursuing
this path, I find no success in its endeavors.
Regards,
Sergio Navega.
From: Neil W Rickert <rickert+nn@cs.niu.edu>
Subject: Re: Rationality and Formalizability
Date: 04 Nov 1999 00:00:00 GMT
Message-ID: <7vsmi6$kas@ux.cs.niu.edu>
References: <7vnhov$tlf$1@nntp6.atl.mindspring.net>
<38216ee3_3@news3.prserv.net>
Organization: Northern Illinois University
Newsgroups: comp.ai.philosophy
"Sergio Navega" <snavega@attglobal.net> writes:
>I suggest that, in reality, the human mind is not "formalizable",
>not because it goes against some silly mathematical principles,
>but because the mind transcends the mere description of algorithms.
>Now that is a dangerous paragraph, which may be easily misunderstood,
>so I'll try to clarify it a bit (it is *not* what you're thinking
>I'm thinking).
I don't think "transcends" needs to be brought in here. It tends to
suggest something mystical.
What mathematicians do is formal or presumably formalizable (well,
perhaps not as I have argued elsewhere, but let's assume it as a
starting point).
Similarly, what theoretical physicists do has a large formal
component. But, if you look at the experimental physicist, you would
have to say that what he/she is doing is empirical, rather than
formal. The experimental physicist spends time tweaking the
instruments, perhaps building new measuring instruments and
calibrating them, perhaps even developing new measuring systems
(measuring conventions).
The brain is there to solve empirical problems, not to solve formal
problems. So we should expect it to be doing things along the lines
of what the experiment physicist does. That is, it should be tuning,
tweaking, calibrating, developing new measuring standards.
Much of what goes under the name of AI (and under the name of
epistemology or cognitive science) is posturing on how to solve the
wrong problem (a formal problem), while completely ignoring the real
empirical problems that the brain must solve to enable us to be
intelligent.
Searle was right to argue that logic without intentionality is not
sufficient. Searle was wrong to argue that intentionality come from
some unexplained causal powers of the brain. The empirical work of
the experimental physicist is important because it does establish
intentionality (meaningfulness) for scientific terms. Without the
measuring standards and instrumentation, and the careful calibration
of instruments, physics would be as pointless as astrology or
epistemology. Newton's work was important, not because he discovered
some great new pattern, but because he constructed an extensible
framework for conquering nature in the intentional sense of being
able to meaningfully apply terms such as "force", "work" and
"mass"
to a wide range of phenomena. In this respect, his invention of
calculus and his demonstration of how to use it in physics, are
actually far more important than were his laws of motion.
From: "Sergio Navega" <snavega@attglobal.net>
Subject: Re: Rationality and Formalizability
Date: 05 Nov 1999 00:00:00 GMT
Message-ID: <3822e6af_2@news3.prserv.net>
References: <7vnhov$tlf$1@nntp6.atl.mindspring.net>
<38216ee3_3@news3.prserv.net> <7vsmi6$kas@ux.cs.niu.edu>
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: abuse@prserv.net
X-Trace: 5 Nov 1999 14:16:15 GMT, 129.37.182.149
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.philosophy
Neil W Rickert wrote in message <7vsmi6$kas@ux.cs.niu.edu>...
>"Sergio Navega" <snavega@attglobal.net> writes:
>
>>I suggest that, in reality, the human mind is not "formalizable",
>>not because it goes against some silly mathematical principles,
>>but because the mind transcends the mere description of algorithms.
>>Now that is a dangerous paragraph, which may be easily misunderstood,
>>so I'll try to clarify it a bit (it is *not* what you're thinking
>>I'm thinking).
>
>I don't think "transcends" needs to be brought in here. It tends to
>suggest something mystical.
>
Hi, Neil.
I knew that my choice of words would be seen as inappropriate :-)
For the record, I'm a happy member of the Skeptics Society.
>What mathematicians do is formal or presumably formalizable (well,
>perhaps not as I have argued elsewhere, but let's assume it as a
>starting point).
>
>Similarly, what theoretical physicists do has a large formal
>component. But, if you look at the experimental physicist, you would
>have to say that what he/she is doing is empirical, rather than
>formal. The experimental physicist spends time tweaking the
>instruments, perhaps building new measuring instruments and
>calibrating them, perhaps even developing new measuring systems
>(measuring conventions).
>
It often amazes me the amount of predictive power that certain
theoretical inferences obtain. For instance, the case of Einstein
predicting the deviation of light of stars when its path goes
near massive objects such as the sun. This was experimentally
confirmed only years after the theoretical prediction. I find
that this kind of "success" is used (perhaps inappropriately)
by theoretical researchers to develop that idea of a "formal
universe" out there that is waiting to be discovered. They may
use this notion to support the activity of doing theoretical
research with no contact with empirical results. It is not easy
to argue with one of those theoretical guys. I find this kind of
thinking very close to that of philosophers.
On the other side of the question, where I (and certainly you)
are, we find that we have only partial, vague and imperfect
visions of the universe; that our theorization may often
follow the empirical results but frequently goes in diverging
directions; that our formal notions of this universe doesn't
mean that there are any formal order hidden in that universe.
Then it is reasonable to make that analogy: classical, logical
AI is just like theoretical physics. Without concern (and origin
from) the real world, this idea will produce an intelligence
which can only "run" in an idealized, platonic universe where
the frame problem is not an issue.
>The brain is there to solve empirical problems, not to solve formal
>problems. So we should expect it to be doing things along the lines
>of what the experiment physicist does. That is, it should be tuning,
>tweaking, calibrating, developing new measuring standards.
>
I agree with the "specification" of the problem to solve that you
give. I'm only reluctant to see why it is a process of calibration.
I see this activity as being the result of two kind of processes
(here I go again).
The first is, I now grant, a process in which the "dreaded" patterns
are statistically acquired and learned. This would put me in the
side of the connectionist guys (with Bill Modlin being our nearest
reference). But this level is not very thick and certainly have
problems dealing with high-level cognition. In particular, this
level is not adequate to explain a lot of our abilities (story
understanding and memorization being one of them). So there
must be another level on top of that.
This second level is more elusive. I propose that it is not really
the level that traditional "symbolic" AI proposes, although
it is closer to it than the preceding level. It is still somewhat
connectionist, but the basic "nodes" are not individual
neurons anymore. It is clusters of neurons, which are able to
oscillate according to certain rules.
Each group of neurons (of a population) were "trained" with
statistical regularities (first level) found in low level signals.
Through those oscillations, they will "invite" other groups to
oscillate, spreading the activity to many other areas. More: they
will often "learn" how to enter into synchronously oscillating
patterns with other groups of neurons (these other groups may
be processing other low-level regularities) based on supervised
learning conditions. For instance, the first group may oscillate
because of the concept "sweet" and the second because of the
concept "apple". This is a learned association, different from
the statistical correlation of visual shape or the pattern of
taste (one exemplar is enough to establish this association).
I suggest that this is how episodic memory works. Our ability
to rapidly establish connections between totally distinct things
is one of the events I expect to be explained by such mechanisms
(when someone finally discover them).
We leave a movie theatre being able to recite the story
we had just seen with great detail. But the details we
remember are not about the low level aspects of the scenes
(we could see the same movie recorded with a different
camera on a slightly different position and we will not
notice it), but we remember the high-level details of the plot.
Our statistical level (the first one) is not significantly
altered by that experience (seeing an apple in that film
will not add much to our previous visual concept of apples).
But the high-level details of the story will be remembered.
And these are the details that may be used generatively, if
one is a writer and uses "pieces" of that story as source
of inspiration (or just plain plagiarism).
It is that generative ability of the "second" level that
I see as being particularly important to language. So if
language is hard to be processed by statistical methods
with neural networks, that is not surprising to me. It
should be processed by this second level, in which the
processing is different. But it requires the first level,
to handle the concepts of things such as "apples" and
"sweeteness".
Well, enough of my ramblings.
Regards,
Sergio Navega.
From: Neil W Rickert <rickert+nn@cs.niu.edu>
Subject: Re: Rationality and Formalizability
Date: 05 Nov 1999 00:00:00 GMT
Message-ID: <7vvged$prn@ux.cs.niu.edu>
References: <7vsmi6$kas@ux.cs.niu.edu> <3822e6af_2@news3.prserv.net>
Organization: Northern Illinois University
Newsgroups: comp.ai.philosophy
"Sergio Navega" <snavega@attglobal.net> writes:
>Neil W Rickert wrote in message <7vsmi6$kas@ux.cs.niu.edu>...
>>What mathematicians do is formal or presumably formalizable (well,
>>perhaps not as I have argued elsewhere, but let's assume it as a
>>starting point).
>>Similarly, what theoretical physicists do has a large formal
>>component. But, if you look at the experimental physicist, you would
>>have to say that what he/she is doing is empirical, rather than
>>formal. The experimental physicist spends time tweaking the
>>instruments, perhaps building new measuring instruments and
>>calibrating them, perhaps even developing new measuring systems
>>(measuring conventions).
>It often amazes me the amount of predictive power that certain
>theoretical inferences obtain. For instance, the case of Einstein
>predicting the deviation of light of stars when its path goes
>near massive objects such as the sun.
However, theoretical physics does not exist in a vacuum. Most of the
theoreticians are in close contact with the experimental scientists,
so that a lot of empirical data influences their interpretation of
theoretical models.
>On the other side of the question, where I (and certainly you)
>are, we find that we have only partial, vague and imperfect
>visions of the universe; that our theorization may often
>follow the empirical results but frequently goes in diverging
>directions; that our formal notions of this universe doesn't
>mean that there are any formal order hidden in that universe.
>Then it is reasonable to make that analogy: classical, logical
>AI is just like theoretical physics.
But it really isn't. Some of the work on artificial life might be
like theoretical physics, because there is the same background of
empirical work. But most of the theoretical work in AI does not
emerge from an empirical background -- rather it comes out of an
abstract background of mathematics and philosophy.
>
Without concern (and origin
>from) the real world, this idea will produce an intelligence
>which can only "run" in an idealized, platonic universe where
>the frame problem is not an issue.
I agree with that. I guess we disagree a little about theoretical
physics, which I see as far closer to empirical work than is
theoretical AI.
>>The brain is there to solve empirical problems, not to solve formal
>>problems. So we should expect it to be doing things along the lines
>>of what the experiment physicist does. That is, it should be tuning,
>>tweaking, calibrating, developing new measuring standards.
>I agree with the "specification" of the problem to solve that you
>give. I'm only reluctant to see why it is a process of calibration.
>I see this activity as being the result of two kind of processes
>(here I go again).
>The first is, I now grant, a process in which the "dreaded" patterns
>are statistically acquired and learned. This would put me in the
>side of the connectionist guys (with Bill Modlin being our nearest
>reference). But this level is not very thick and certainly have
>problems dealing with high-level cognition. In particular, this
>level is not adequate to explain a lot of our abilities (story
>understanding and memorization being one of them). So there
>must be another level on top of that.
The idea of using statistical methods to find patterns is fine. But
you won't find patterns where there are none.
I get up one morning, and measure the height
of my bedroom
window.
The next morning I again measure it, but with a
different
answer.
The following morning, I measure it again, with
yet another
answer.
Three different answers. There is no pattern.
Why were there three different answers? Perhaps I measured in inches
the first day, in centimeters the second day, and with the
logarithmic scale of a slide rule on the third day.
If I develop measuring standards, and calibrate all of my instruments
according to those standards, then there might be a pattern in the
data. Without standards and calibration, there are no patterns.
I look at a flower from a distance of 3 feet.
I look at the same flower from a distance of 5 feet.
There will be very little relationship between the raw visual signals
received in the two instances. However, if my brain has been
developing measuring standards, and cross calibrating everything so
that I can compensate for the distance of the object viewed, then the
appropriately scaled and standardized data derived from the raw
signals might be similar in the two cases.
My point is that, contrary to the conventional wisdom of the Machine
Learning crowd, there are at most very weak patterns in the raw
data. The important patterns are not those in the raw data, but
those that we create by means of how we use (organize) the raw data.
In other words,
patterns are created, not discovered.
Learning is then not a process of statistically discovering patterns,
but is a process of experimenting with a variety of ways of
organizing the raw data, until we find systems of organization which
result in useful patternings of the organized data.
As an example, take the theory of evolution. Most people (other than
creationists) will say that we can see that evolution is true
by looking at patterns in natural data. And generally, Darwin is
credited as being the father of Evolution.
In my opinion, the real father of evolution was Carl Linneaus. He
devised the system of biological classification which we still use
(in somewhat modified form). I am suggesting that the patterns from
which we derive evolution arise out of the system of organization
that Linneaus gave us. If we still looked at species in the way they
were viewed at the time of Aristotle, then the only patterns we would
see were those of many individual kinds, the type of patterns
described by the creationists. Before the Linneaus system was
adopted, no statistical patterns relevant to evolution would have
been discovered in the data, because any patterns were far too weak
to be detected.
Go back to my earlier example of measuring windows. The idea that we
should develop measuring standards, and use them to calibrate those
different rulers, is really just a way of saying that we should
organize our data so that the data obtained with one ruler can be
usefully compared with the data obtained with another ruler.
>This second level is more elusive. I propose that it is not really
>the level that traditional "symbolic" AI proposes, although
>it is closer to it than the preceding level. It is still somewhat
>connectionist, but the basic "nodes" are not individual
>neurons anymore. It is clusters of neurons, which are able to
>oscillate according to certain rules.
>Each group of neurons (of a population) were "trained" with
>statistical regularities (first level) found in low level signals.
>Through those oscillations, they will "invite" other groups to
>oscillate, spreading the activity to many other areas. More: they
>will often "learn" how to enter into synchronously oscillating
>patterns with other groups of neurons (these other groups may
>be processing other low-level regularities) based on supervised
>learning conditions. For instance, the first group may oscillate
>because of the concept "sweet" and the second because of the
>concept "apple". This is a learned association, different from
>the statistical correlation of visual shape or the pattern of
>taste (one exemplar is enough to establish this association).
You are probably describing something similar to what Modlin
proposes, even at your higher levels.
I am saying that you have it backwards.
The approach appears to be:
Discover statistical patterns.
These patterns will predict how the true
organization of the
data. Now start organizing. Then
you will find higher
patterns.
Whereas I am arguing:
Patterns depend on organization. You
won't find patterns
in unorganized data. If there are weak
patterns in raw
data, they reflect that fact that there is some
organization
present even in the raw data.
There is no "true organization" of
the data. There are a
zillion ways to organize the data. They
each have their
advantages and disadvantages. The choice
of organization is
not a judgement of truth. It is a
pragmatic judgement -- I
need to organize the data in a way that will
help satisfy my
internal drives (hunger, thirst, etc).
The starting point has to be a system with
internal drives,
taking actions so as to discover ways of
satifying those
internal drives.
The Modlin and ML approach wants a purely passive learning system,
and rejects my requirement of action and internal drives. For that
matter, much of AI is based on the illusion that you could have some
kind of pure absolute objective knowledge independent of practical
subjective concerns such as the need to eat. For this they look
to logic.
In a way, they are right. You could indeed have an absolute
objective knowledge independent of practical subjective concerns.
But the cost is that this will be knowledge about nothing at all.
That is, it will be totally abstract, disconnected from reality. For
it is our inner drives, such as the need to eat and breathe, that
force us to stay in touch with reality.
From: "Sergio Navega" <snavega@attglobal.net>
Subject: Re: Rationality and Formalizability
Date: 06 Nov 1999 00:00:00 GMT
Message-ID: <38243a97_1@news3.prserv.net>
References: <7vsmi6$kas@ux.cs.niu.edu> <3822e6af_2@news3.prserv.net>
<7vvged$prn@ux.cs.niu.edu>
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: abuse@prserv.net
X-Trace: 6 Nov 1999 14:26:31 GMT, 32.100.0.35
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.philosophy
Neil W Rickert wrote in message <7vvged$prn@ux.cs.niu.edu>...
>"Sergio Navega" <snavega@attglobal.net> writes:
>
>>The first is, I now grant, a process in which the "dreaded" patterns
>>are statistically acquired and learned. This would put me in the
>>side of the connectionist guys (with Bill Modlin being our nearest
>>reference). But this level is not very thick and certainly have
>>problems dealing with high-level cognition. In particular, this
>>level is not adequate to explain a lot of our abilities (story
>>understanding and memorization being one of them). So there
>>must be another level on top of that.
>
>The idea of using statistical methods to find patterns is fine. But
>you won't find patterns where there are none.
>
> I get up one morning, and measure the height of my bedroom
> window.
> The next morning I again measure it, but with a different
> answer.
> The following morning, I measure it again, with yet another
> answer.
>
>Three different answers. There is no pattern.
>
>Why were there three different answers? Perhaps I measured in inches
>the first day, in centimeters the second day, and with the
>logarithmic scale of a slide rule on the third day.
>
>If I develop measuring standards, and calibrate all of my instruments
>according to those standards, then there might be a pattern in the
>data. Without standards and calibration, there are no patterns.
>
> I look at a flower from a distance of 3 feet.
>
> I look at the same flower from a distance of 5 feet.
>
>There will be very little relationship between the raw visual signals
>received in the two instances. However, if my brain has been
>developing measuring standards, and cross calibrating everything so
>that I can compensate for the distance of the object viewed, then the
>appropriately scaled and standardized data derived from the raw
>signals might be similar in the two cases.
>
>My point is that, contrary to the conventional wisdom of the Machine
>Learning crowd, there are at most very weak patterns in the raw
>data. The important patterns are not those in the raw data, but
>those that we create by means of how we use (organize) the raw data.
>In other words,
>
> patterns are created, not discovered.
>
>Learning is then not a process of statistically discovering patterns,
>but is a process of experimenting with a variety of ways of
>organizing the raw data, until we find systems of organization which
>result in useful patternings of the organized data.
>
I mostly agree with this idea (in particular with the brilliant
notion of experimenting with ways to organize raw data!). I just
use a different way to get there. I agree that much of what is
captured by the senses consists of patternless structures, with
very few intrinsic regularities. But the fact is that we are able
to come up with "interpretations" of these data that show repeating
patterns. As you say, this is a creative process, not a discovery
process and that puts the whole responsibility of pattern finding
in the efforts of the brain.
Modlin's ideas appears to suggest that there are patterns lurking
in the data and a purely statistical process can reveal them all.
In a way, my idea is between both hypotheses. It is obvious
that sensory data must have some kind of information, but it is
also true that we must exert some effort in order to obtain
those valuable structures, often interacting with the environment
in order to discern what's relevant from what's not.
So my proposition is a mixture of some points between these two
(apparently) irreconcilable points. I'll try to write more about
my approach not as I have done in the last post (which focused
on proposed mechanisms), but on the informational activity
that's going on. I know this is a can of worms, but I guess
I have a contribution to give.
>As an example, take the theory of evolution. Most people (other than
>creationists) will say that we can see that evolution is true
>by looking at patterns in natural data. And generally, Darwin is
>credited as being the father of Evolution.
>
>In my opinion, the real father of evolution was Carl Linneaus. He
>devised the system of biological classification which we still use
>(in somewhat modified form). I am suggesting that the patterns from
>which we derive evolution arise out of the system of organization
>that Linneaus gave us. If we still looked at species in the way they
>were viewed at the time of Aristotle, then the only patterns we would
>see were those of many individual kinds, the type of patterns
>described by the creationists. Before the Linneaus system was
>adopted, no statistical patterns relevant to evolution would have
>been discovered in the data, because any patterns were far too weak
>to be detected.
>
>Go back to my earlier example of measuring windows. The idea that we
>should develop measuring standards, and use them to calibrate those
>different rulers, is really just a way of saying that we should
>organize our data so that the data obtained with one ruler can be
>usefully compared with the data obtained with another ruler.
>
I agree with this idea of organizing the data, although I reluct
to see it as a process of calibrating (which appears to be too
analogical). Rather I see it as the attempt to try different ways
of "grouping" elements in order to produce "valuable" patterns.
I'll try to refine later what is it that I call valuable patterns.
>>This second level is more elusive. I propose that it is not really
>>the level that traditional "symbolic" AI proposes, although
>>it is closer to it than the preceding level. It is still somewhat
>>connectionist, but the basic "nodes" are not individual
>>neurons anymore. It is clusters of neurons, which are able to
>>oscillate according to certain rules.
>
>>Each group of neurons (of a population) were "trained" with
>>statistical regularities (first level) found in low level signals.
>>Through those oscillations, they will "invite" other groups to
>>oscillate, spreading the activity to many other areas. More: they
>>will often "learn" how to enter into synchronously oscillating
>>patterns with other groups of neurons (these other groups may
>>be processing other low-level regularities) based on supervised
>>learning conditions. For instance, the first group may oscillate
>>because of the concept "sweet" and the second because of the
>>concept "apple". This is a learned association, different from
>>the statistical correlation of visual shape or the pattern of
>>taste (one exemplar is enough to establish this association).
>
>You are probably describing something similar to what Modlin
>proposes, even at your higher levels.
>
>I am saying that you have it backwards.
>
>The approach appears to be:
>
> Discover statistical patterns.
>
> These patterns will predict how the true organization of the
> data. Now start organizing. Then you will find higher
> patterns.
>
This partially reflects what I said before. I still think this is
an important way to see things, although I agree it is not the whole
story.
>Whereas I am arguing:
>
> Patterns depend on organization. You won't find patterns
> in unorganized data. If there are weak patterns in raw
> data, they reflect that fact that there is some organization
> present even in the raw data.
>
That's very reasonable. I'm sure, however, that Modlin would
say that higher order statistics of that same data would reveal
other weak patterns (different than the formers) and that the sum
of the discoveries would be the full "informational content" of
the input signal. I follow only partially this idea, because
it leaves unanswered the question of complexity of this task.
> There is no "true organization" of the data. There are a
> zillion ways to organize the data. They each have their
> advantages and disadvantages. The choice of organization is
> not a judgement of truth. It is a pragmatic judgement -- I
> need to organize the data in a way that will help satisfy my
> internal drives (hunger, thirst, etc).
That's agreeable. Perhaps what this appears to say is that given
a complex signal there are so many ways to come up with regular
patterns that it would not be possible for a brain like ours
to obtain meaningful things (after all, two years after birth
any child is a extraordinary object recognizer). In summary, the
problem is much more complex than purely unsupervised learning
would be capable to solve using the limited computational
resources (and time) available.
>
> The starting point has to be a system with internal drives,
> taking actions so as to discover ways of satifying those
> internal drives.
>
>The Modlin and ML approach wants a purely passive learning system,
>and rejects my requirement of action and internal drives. For that
>matter, much of AI is based on the illusion that you could have some
>kind of pure absolute objective knowledge independent of practical
>subjective concerns such as the need to eat. For this they look
>to logic.
>
>In a way, they are right. You could indeed have an absolute
>objective knowledge independent of practical subjective concerns.
>But the cost is that this will be knowledge about nothing at all.
>That is, it will be totally abstract, disconnected from reality. For
>it is our inner drives, such as the need to eat and breathe, that
>force us to stay in touch with reality.
>
I'm sympathetic with this vision, as it puts survivability as a
fundamental concern of brains. But I often question whether this
would *entirely* explain the emergence of complex information
processors such as human's (or mammal's) brains. It appears that
our brain is only partially dedicated to the goal of improving
survivability.
Big brained mammals are exceptions among other animals, they
appear to be "deviances" of the "standard" (if this weren't the
case, then the whole animal kingdom would grow toward bigger
brains, and that's not happening). So, what would be the "drive"
that pushed our brains into existence?
I don't have a clear answer to this (other than pure chance),
but I find it interesting to analyze the question of intelligence
in informational terms. These ideas follow the 'Principle of
Cognitive Economy'.
I suggest that the main task that (unconscious) brains develop is
that of looking for invariances. All experiments, all
transformations, all manipulations are done to reveal invariant
things. Only the processes that produce invariance are remembered.
This idea is, I believe, compatible with your hypothesis of
successive calibration, as the purpose of brains would be to
look to ways of "messing" with the data in order to produce
regularity, obtaining the same outcomes over and over.
Given an input signal, the brain would put its neural mechanisms
to look for repeating structures. It is often a philosophical
question that in order to find these structures one has to have
a comparison mechanism. I say yes, this initial level has a
genetically determined, innate mechanism which "bootstraps" the
whole process. In the visual cortex, a small part of its
self-organization occurs even in the absence of external
signals.
However, the question of "finding patterns in signals" is terribly
complex, one that approaches intractability. It is here that I
abandon part of Modlin's suppositions, because in order to
find anything meaningful it is not enough to look to higher
order statistic regularities. It is necessary to simplify this
whole problem, and I propose that this is done through a
"darwinian-like" process, in which candidates fight with each
other and the winner is the one who can gets most resources
over time (which could also be equated with successive
reinforcement). This is what appears to happen in ocular
dominance and orientation selectivity in the visual cortex.
The task of generating these "candidates" is my hunch of a
process that can resemble your idea of progressive calibration.
The brain would "try" random ways to assess the data and the
ones with better success (in reducing complexity, which means
showing greater regularity, invariance) would survive.
In my previous post I mentioned a two-level architecture.
The process I just sketched belongs to the first level. And that's
the level that appears to develop preponderantly during initial
babyhood: it is a time where the baby's brain is maturing, with
an enormous number of new synaptic connections growing (while
others are being prunned) and where exposition to diverse
environmental factors can alter dramatically the evolution
of future perceptual abilities of the child.
With time, this neural self-organization would be less important,
because the neural connections would have "settled" around the
most important ways to process the incoming data. This is, then,
the way I propose that the brain would become a "specialist"
in the analyses of low level signals. The result of this process
is a brain wired to rapidly extract the relevant features from the
signals (although I said that this evolves particularly during
childhood, I propose that this occurs also in adults, as when
we learn to ride a bicycle; the coordination of motor patterns
required to do that precisely appears to be done as the result
of "slight" experiments which learn to detect and associate the
best sense/action pairs).
When one is relatively done with this level, then what starts to
be important is the second level I proposed earlier, where the
essential activity is around networks of populations of neurons.
At this level, it is not statistic correlations of the signals
that are important (these are already processed by the relatively
stable neural circuits which evolved to be dedicated to the
resolution of the initial problem), but the rule-like aspects
of what is discovered by that initial level. Language cannot be
used before one has a solid auditory processing (or visual
processing for sign language, in the case of deafs; or touch
processing in the case of deaf-blind).
Although this is all merely speculative, I can't resist doing
this kind of inference, as it seems to unify one of the greatest
battles of AI so far, that of symbolicists and connectionists.
This also appears compatible with 'symbol grounding' philosophies.
Regards,
Sergio Navega.
From: Neil W Rickert <rickert+nn@cs.niu.edu>
Subject: Re: Rationality and Formalizability
Date: 06 Nov 1999 00:00:00 GMT
Message-ID: <80208d$bu@ux.cs.niu.edu>
References: <7vvged$prn@ux.cs.niu.edu> <38243a97_1@news3.prserv.net>
Organization: Northern Illinois University
Newsgroups: comp.ai.philosophy
"Sergio Navega" <snavega@attglobal.net> writes:
>Neil W Rickert wrote in message <7vvged$prn@ux.cs.niu.edu>...
[Much deleted for brevity]
>>My point is that, contrary to the conventional wisdom of the Machine
>>Learning crowd, there are at most very weak patterns in the raw
>>data. The important patterns are not those in the raw data, but
>>those that we create by means of how we use (organize) the raw data.
>>In other words,
>> patterns are created, not discovered.
>>Learning is then not a process of statistically discovering patterns,
>>but is a process of experimenting with a variety of ways of
>>organizing the raw data, until we find systems of organization which
>>result in useful patternings of the organized data.
>I mostly agree with this idea (in particular with the brilliant
>notion of experimenting with ways to organize raw data!). I just
>use a different way to get there. I agree that much of what is
>captured by the senses consists of patternless structures, with
>very few intrinsic regularities. But the fact is that we are able
>to come up with "interpretations" of these data that show repeating
>patterns. As you say, this is a creative process, not a discovery
>process and that puts the whole responsibility of pattern finding
>in the efforts of the brain.
>Modlin's ideas appears to suggest that there are patterns lurking
>in the data and a purely statistical process can reveal them all.
That is my understanding of Modlin's position.
>In a way, my idea is between both hypotheses. It is obvious
>that sensory data must have some kind of information, but it is
>also true that we must exert some effort in order to obtain
>those valuable structures, often interacting with the environment
>in order to discern what's relevant from what's not.
You have to first decide what is information. Many people, including
Modlin, go for an objective notion of information. But they have not
come up with a convincing definition of what this objective
information is, and I doubt that there is such a thing.
Consider two sentences:
Dinner is on the table.
King Xzzyottmx of the kingdom of Vvbbngw on
the planet
Bzzeryy of the andromeda galaxy has committed
suicide.
The second of these contains many more words. If we measure in bits
of objective information, it presumably has about 4 times as much as
the first sentence. However, I don't see the second sentence as
containing any information at all. It might be completely phony
(well it is, because I just made it up). I just made up the first
sentence too. But the first sentence is something that I can check,
and which if true might cause me to modify my behavior. The second
sentence doesn't do anything for me. So I would say that the second
sentence has zero information and zero misinformation, while the
first sentence is rich in either information or misinformation.
My point is that it is not information unless we can act on it. So
any idea of information has to be tied up with possible actions. I
have had this debate with Modlin, and he insists that action has to
wait until later, and that learning can proceed without any action.
What it boils down to, is that we need a more subjective notion of
information, where the quantity of information is not to be measured
by the number of bits to write it out, but by the usefulness of the
information. This usefulness criterion can actually appear to be not
too far from objective, since we humans are mostly in sufficiently
similar situations that what is useful for one is usually useful for
another.
My view, then, is that inner drives enable us to judge what is
useful, and this is what energizes the learning process. Roughly
speaking, an intelligent system is one which can make intelligent
judgements. Learning is then the process that starts with the
judgement that come from our inner drives, and finds ways of
projecting that ability of judging onto the external world so that we
can make judgements about the external world.
Logic by itself cannot make judgements. But it can transform
judgements. In typical computer programming, the judgement of the
programmer is transformed into judgments made by the programmed
machine. If you want to think of the brain as a logic system, I am
saying that you should think of it as transforming the judgements of
our inner drives, so that they can be applied to the external world.
>So my proposition is a mixture of some points between these two
>(apparently) irreconcilable points. I'll try to write more about
>my approach not as I have done in the last post (which focused
>on proposed mechanisms), but on the informational activity
>that's going on. I know this is a can of worms, but I guess
>I have a contribution to give.
>>As an example, take the theory of evolution. Most people (other than
>>creationists) will say that we can see that evolution is true
>>by looking at patterns in natural data. And generally, Darwin is
>>credited as being the father of Evolution.
>>In my opinion, the real father of evolution was Carl Linneaus. He
>>devised the system of biological classification which we still use
>>(in somewhat modified form). I am suggesting that the patterns from
>>which we derive evolution arise out of the system of organization
>>that Linneaus gave us. If we still looked at species in the way they
>>were viewed at the time of Aristotle, then the only patterns we would
>>see were those of many individual kinds, the type of patterns
>>described by the creationists. Before the Linneaus system was
>>adopted, no statistical patterns relevant to evolution would have
>>been discovered in the data, because any patterns were far too weak
>>to be detected.
>>Go back to my earlier example of measuring windows. The idea that we
>>should develop measuring standards, and use them to calibrate those
>>different rulers, is really just a way of saying that we should
>>organize our data so that the data obtained with one ruler can be
>>usefully compared with the data obtained with another ruler.
>I agree with this idea of organizing the data, although I reluct
>to see it as a process of calibrating (which appears to be too
>analogical). Rather I see it as the attempt to try different ways
>of "grouping" elements in order to produce "valuable" patterns.
>I'll try to refine later what is it that I call valuable patterns.
Go back to my comment on Linneaus. It wouldn't do us a lot of good
if biologists all decided to use the Linneaus idea, but
independently, so that the classifications by two different
biologists would be incompatible. So our biology students are
trained, so that they can be consistent. Calibration is really
nothing more than establishing procedures to ensure the consistency
of interpretation. Suppose that my brain can estimate distance of an
object by means of the light entering the left eye, the light
entering the right eye, the sound made by the object entering the
right ear, and the sound entering the left ear. It is not very
useful if these four estimates are inconsistent. There need to be
processes in the brain to cross calibrate all sensory processes, in
order to assure consistency of interpretation.
Maybe the idea of calibration doesn't fit easily into your way
of looking at things. But I think it essential.
Every so often you take your automobile to the mechanics for a tune
up. They adjust the ignition timing, and the spark plug gaps which
affect the intensity of the ignition spark. The tuneup parameters
might be different in the mountains of Colorado, than on the plains
of the midwest. This tuneup really amounts to calibration of the
ignition system.
I am arguing that the brain needs a tune up as much as the
automobile, and that there are processes in the brain actively tuning
it up as we use it.
Roughly speaking, I am saying that an infant is born with a brain
that is badly out of tune, and that learning for that child results
from the continuing tuneup process. But of course the tuning
required for a mathematician will be different from the tuning
required for a sculptor. The tuning processes in the brain will be
tuning up the brain for the type of environment in which the person
puts that brain. Thus we all learn different things.
>>Whereas I am arguing:
>> Patterns depend on organization. You won't find patterns
>> in unorganized data. If there are weak patterns in raw
>> data, they reflect that fact that there is some organization
>> present even in the raw data.
>That's very reasonable. I'm sure, however, that Modlin would
>say that higher order statistics of that same data would reveal
>other weak patterns (different than the formers) and that the sum
>of the discoveries would be the full "informational content" of
>the input signal. I follow only partially this idea, because
>it leaves unanswered the question of complexity of this task.
I am sure you are right about what Modlin would say. I have
had that debate with him.
>> The starting point has to be a system with internal drives,
>> taking actions so as to discover ways of satifying those
>> internal drives.
>>The Modlin and ML approach wants a purely passive learning system,
>>and rejects my requirement of action and internal drives. For that
>>matter, much of AI is based on the illusion that you could have some
>>kind of pure absolute objective knowledge independent of practical
>>subjective concerns such as the need to eat. For this they look
>>to logic.
>>In a way, they are right. You could indeed have an absolute
>>objective knowledge independent of practical subjective concerns.
>>But the cost is that this will be knowledge about nothing at all.
>>That is, it will be totally abstract, disconnected from reality. For
>>it is our inner drives, such as the need to eat and breathe, that
>>force us to stay in touch with reality.
>I'm sympathetic with this vision, as it puts survivability as a
>fundamental concern of brains. But I often question whether this
>would *entirely* explain the emergence of complex information
>processors such as human's (or mammal's) brains. It appears that
>our brain is only partially dedicated to the goal of improving
>survivability.
I think you have to look at the type of life we live. A plant in a
tropical rain forest can survive pretty easily, so it can do without
a brain. We have evolved to have no fixed source of nutrition, and
so that we have to be able to find nutrition where it happens to be
available. For that we need the ability to identify opportunities
where they happen to arise. Curiosity is thus an important internal
drive, and I suspect that much of human cognition depends on that
innate curiosity.
>Big brained mammals are exceptions among other animals, they
>appear to be "deviances" of the "standard" (if this weren't the
>case, then the whole animal kingdom would grow toward bigger
>brains, and that's not happening). So, what would be the "drive"
>that pushed our brains into existence?
I think there is no "standard." Whatever works well enough can
survive. The enormous variety of biological life shows that there
are many roads to survival.
The path taken by insects is that of low cost units, with the high
reproduction rate that is possible when the reproduced units are low
cost, and with the expectation that most units will be throwaways but
enough will survive to perpetuate the species. The path taken by
mammals is one of high cost units, with low reproductive rates since
it is expensive to reproduce high-cost units. The expectation is
that these high cost units will have many built in protective
mechanisms, so that most will survive. There are, of course plenty
of other possible strategies, so that there is a wide range in what
is actually used by biological systems.
But of course, evolution is just the same type of learning as is
human learning. Using random processes, biological system experiment
with many different forms of organization. There is not just one
possible way of organizing. Each successful species is successful
because the method it uses for organizing its world is capable of
supporting that species.
From: "Sergio Navega" <snavega@attglobal.net>
Subject: Re: Rationality and Formalizability
Date: 08 Nov 1999 00:00:00 GMT
Message-ID: <3826cd8b_3@news3.prserv.net>
References: <7vvged$prn@ux.cs.niu.edu> <38243a97_1@news3.prserv.net>
<80208d$bu@ux.cs.niu.edu>
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: abuse@prserv.net
X-Trace: 8 Nov 1999 13:18:03 GMT, slip-32-102-102-167.sp.br.prserv.net
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.philosophy
Neil W Rickert wrote in message <80208d$bu@ux.cs.niu.edu>...
>"Sergio Navega" <snavega@attglobal.net> writes:
>[snip agreed stuff]
>
>>In a way, my idea is between both hypotheses. It is obvious
>>that sensory data must have some kind of information, but it is
>>also true that we must exert some effort in order to obtain
>>those valuable structures, often interacting with the environment
>>in order to discern what's relevant from what's not.
>
>You have to first decide what is information. Many people, including
>Modlin, go for an objective notion of information. But they have not
>come up with a convincing definition of what this objective
>information is, and I doubt that there is such a thing.
>
>Consider two sentences:
>
> Dinner is on the table.
>
> King Xzzyottmx of the kingdom of Vvbbngw on the planet
> Bzzeryy of the andromeda galaxy has committed suicide.
>
>The second of these contains many more words. If we measure in bits
>of objective information, it presumably has about 4 times as much as
>the first sentence. However, I don't see the second sentence as
>containing any information at all. It might be completely phony
>(well it is, because I just made it up). I just made up the first
>sentence too. But the first sentence is something that I can check,
>and which if true might cause me to modify my behavior. The second
>sentence doesn't do anything for me. So I would say that the second
>sentence has zero information and zero misinformation, while the
>first sentence is rich in either information or misinformation.
>
I agree with you about the notion of objective/subjective information
and if we keep our eyes on just those two sentences, your point is
correct. But I propose that sentence 2 must be analyzed in the
context of thousands of related others. Taken in isolation, sentence
2 is obviously meaningless. But suppose we listen to a third sentence,
following those:
Iottkye was elected as king of of the kingdom of Vvbbngw
on the planet Bzzeryy of the andromeda galaxy.
(can't believe I wrote this)
Again, this sentence may be said to contain zilch information.
But taken in *conjunction* with the previous sentence, we notice
that there are repeating names, and that these names appear to
be surrounded by relatively constant structures:
Vvbbngw
kingdom
Bzzeryy
planet
[Iottkye, Xzzyottmx] king
My point here is that this "kind" of nonsensical phrases is *similar*
(in "nonsensicalness") to what a baby listens. For a baby, everything
is like a kingdom in the andromeda galaxy. There are strange
references to "unknown" entities such as "ball", "puppy",
"teddy bear",
etc. The baby will start to acquire the meaning of each word by
noticing their repetition (often in a context of pointing or grasping
or playing). So the kind of information that these apparently
meaningless phrases have is due to their *structural repetition over
time*. I extend this same idea of information from repetition to all
kinds of sensory signals.
>My point is that it is not information unless we can act on it. So
>any idea of information has to be tied up with possible actions. I
>have had this debate with Modlin, and he insists that action has to
>wait until later, and that learning can proceed without any action.
>
I'm in your side here, as I agree that without action one will not
be able to acquire the kind of semantic platform that we need in order
to support, for example, language. But this process appears to require
a starting point (which I believe is what Modlin is proposing), an
initial level responsible for the treatment of the signals from
"startup" (we can call this "the BIOS of the baby"). Although
Modlin appears to see one mechanism for this level, I see two. My
second part is what Modlin proposes. But my first has no correspondent
in Modlin's vision.
I have no other way to see it other than innate mechanisms "designed"
by evolution. Let me say this in other words. Although I have a
strong position against all kinds of innate mechanisms for language
and high-level knowledge (domain specific!), I have no other way to
see this "startup level" other than being composed of neural circuitry
with a strong genetically determined nature. I guess Modlin would
say that this level is not necessary, that the network would self-
adjust without the need of specific constructs. I see that we have
indeed a pre-wired circuitry and that everything that happens
later is strongly influenced by the characteristics of this level.
>What it boils down to, is that we need a more subjective notion of
>information, where the quantity of information is not to be measured
>by the number of bits to write it out, but by the usefulness of the
>information. This usefulness criterion can actually appear to be not
>too far from objective, since we humans are mostly in sufficiently
>similar situations that what is useful for one is usually useful for
>another.
>
>My view, then, is that inner drives enable us to judge what is
>useful, and this is what energizes the learning process. Roughly
>speaking, an intelligent system is one which can make intelligent
>judgements. Learning is then the process that starts with the
>judgement that come from our inner drives, and finds ways of
>projecting that ability of judging onto the external world so that we
>can make judgements about the external world.
>
I think I understand what you mean here, agreeing that much of the
performance of an intelligent system depends on this judgement
process, fed by inner drives. But this would explain our high-level
behavior. I doubt it would shed much light on the low-level aspects,
which is certainly an important part of the problem. I may concede
that part of our perceptual abilities is influenced by reinforcements,
which are a function of our action/reaction decisions (and judgements).
But much of our perception happens automatically and mindlessly,
independent of our will.
>
>>I agree with this idea of organizing the data, although I reluct
>>to see it as a process of calibrating (which appears to be too
>>analogical). Rather I see it as the attempt to try different ways
>>of "grouping" elements in order to produce "valuable"
patterns.
>>I'll try to refine later what is it that I call valuable patterns.
>
>Go back to my comment on Linneaus. It wouldn't do us a lot of good
>if biologists all decided to use the Linneaus idea, but
>independently, so that the classifications by two different
>biologists would be incompatible. So our biology students are
>trained, so that they can be consistent. Calibration is really
>nothing more than establishing procedures to ensure the consistency
>of interpretation. Suppose that my brain can estimate distance of an
>object by means of the light entering the left eye, the light
>entering the right eye, the sound made by the object entering the
>right ear, and the sound entering the left ear. It is not very
>useful if these four estimates are inconsistent. There need to be
>processes in the brain to cross calibrate all sensory processes, in
>order to assure consistency of interpretation.
>
>Maybe the idea of calibration doesn't fit easily into your way
>of looking at things. But I think it essential.
>
This is perhaps the strongest version of your point. There's no
doubt that this perceptual integration is something that resembles
a calibration process, which is obviously learned. In fact, recent
experiments with the barn owl (Knudsen, Eric I. (1998) Capacity
for Plasticity in the Adult Owl Auditory System Expanded by
Juvenile Experience. Science 1998 Mar 6;279(5356):1531-3) showed
how this process seems to occur, and it appears to support your
hypothesis.
This is closely related to the biggest and most important mystery
that I'm anxious to see solved: that of binding, or sensory
integration. Without this problem solved, I doubt that we could
have the necessary insights in order to propose how language
emerges from perception.
The neural mechanisms of binding would, certainly, suggest how
we are able to maintain in our memory extensive "scripts" of
relatively unrelated concepts. Just think about, for instance,
Alice's Adventures in the Wonderland. After telling the story
to a child, she may be able to recall a pretty good version of
it. On the other hand, to rote memorize a list of words is a
difficult task. Our abilities to memorize interconnected
sequences of concepts is amazing and I bet that the neural
mechanism that supports this ability will be the biggest
discovery of the (next) century.
Regards,
Sergio Navega.
From: Neil W Rickert <rickert+nn@cs.niu.edu>
Subject: Re: Rationality and Formalizability
Date: 08 Nov 1999 00:00:00 GMT
Message-ID: <807nl3$7pk@ux.cs.niu.edu>
References: <80208d$bu@ux.cs.niu.edu> <3826cd8b_3@news3.prserv.net>
Organization: Northern Illinois University
Newsgroups: comp.ai.philosophy
"Sergio Navega" <snavega@attglobal.net> writes:
>Neil W Rickert wrote in message <80208d$bu@ux.cs.niu.edu>...
>>Consider two sentences:
>> Dinner is on the table.
>> King Xzzyottmx of the kingdom of Vvbbngw on the planet
>> Bzzeryy of the andromeda galaxy has committed suicide.
>>The second of these contains many more words. If we measure in bits
>>of objective information, it presumably has about 4 times as much as
>>the first sentence. However, I don't see the second sentence as
>>containing any information at all. It might be completely phony
>>(well it is, because I just made it up). I just made up the first
>>sentence too. But the first sentence is something that I can check,
>>and which if true might cause me to modify my behavior. The second
>>sentence doesn't do anything for me. So I would say that the second
>>sentence has zero information and zero misinformation, while the
>>first sentence is rich in either information or misinformation.
>I agree with you about the notion of objective/subjective information
>and if we keep our eyes on just those two sentences, your point is
>correct. But I propose that sentence 2 must be analyzed in the
>context of thousands of related others. Taken in isolation, sentence
>2 is obviously meaningless. But suppose we listen to a third sentence,
>following those:
>Iottkye was elected as king of of the kingdom of Vvbbngw
>on the planet Bzzeryy of the andromeda galaxy.
>(can't believe I wrote this)
>Again, this sentence may be said to contain zilch information.
>But taken in *conjunction* with the previous sentence, we notice
>that there are repeating names, and that these names appear to
>be surrounded by relatively constant structures:
I agree that there are now some interesting patterns. But there is
still zero information. However there is a puzzle. Is the
repetition an arbitrary coincidence, or is their a reason? Now we
can develop a research program to try to track this down.
Once again, you see me disagreeing with the Modlin approach where
learning has to do with a statistical analysis of passively received
signals. Instead, I am insisting that the system must interact with
the world, and carry out experiments to discover whether anything
useful can be done with these apparent coincidences. If we find
something useful we can do with this, then the inputs become
informative.
>>My point is that it is not information unless we can act on it. So
>>any idea of information has to be tied up with possible actions. I
>>have had this debate with Modlin, and he insists that action has to
>>wait until later, and that learning can proceed without any action.
>I'm in your side here, as I agree that without action one will not
>be able to acquire the kind of semantic platform that we need in order
>to support, for example, language. But this process appears to require
>a starting point (which I believe is what Modlin is proposing), an
>initial level responsible for the treatment of the signals from
>"startup" (we can call this "the BIOS of the baby"). Although
>Modlin appears to see one mechanism for this level, I see two. My
>second part is what Modlin proposes. But my first has no correspondent
>in Modlin's vision.
I agree that a starting point is needed. I see the starting point as
an infant's psychology, its innate system of emotions and drives.
>I have no other way to see it other than innate mechanisms "designed"
>by evolution. Let me say this in other words. Although I have a
>strong position against all kinds of innate mechanisms for language
>and high-level knowledge (domain specific!), I have no other way to
>see this "startup level" other than being composed of neural circuitry
>with a strong genetically determined nature. I guess Modlin would
>say that this level is not necessary, that the network would self-
>adjust without the need of specific constructs. I see that we have
>indeed a pre-wired circuitry and that everything that happens
>later is strongly influenced by the characteristics of this level.
I don't have any problem with "designed by evolution." Nor do I have
any problem with innate mechanisms, if they are internal. I don't
see evidence of substantial innate knowledge of the external world.
But a newborn infant is well supplied with internal drives, emotions,
and personality, as any mother of a newborn will attest.
>>My view, then, is that inner drives enable us to judge what is
>>useful, and this is what energizes the learning process. Roughly
>>speaking, an intelligent system is one which can make intelligent
>>judgements. Learning is then the process that starts with the
>>judgement that come from our inner drives, and finds ways of
>>projecting that ability of judging onto the external world so that we
>>can make judgements about the external world.
>I think I understand what you mean here, agreeing that much of the
>performance of an intelligent system depends on this judgement
>process, fed by inner drives. But this would explain our high-level
>behavior. I doubt it would shed much light on the low-level aspects,
>which is certainly an important part of the problem. I may concede
>that part of our perceptual abilities is influenced by reinforcements,
>which are a function of our action/reaction decisions (and judgements).
>But much of our perception happens automatically and mindlessly,
>independent of our will.
I do think that curiosity is one of those internal drives, and is
probably one of the most important in terms of human cognition.
>>Go back to my comment on Linneaus. It wouldn't do us a lot of good
>>if biologists all decided to use the Linneaus idea, but
>>independently, so that the classifications by two different
>>biologists would be incompatible. So our biology students are
>>trained, so that they can be consistent. Calibration is really
>>nothing more than establishing procedures to ensure the consistency
>>of interpretation. Suppose that my brain can estimate distance of an
>>object by means of the light entering the left eye, the light
>>entering the right eye, the sound made by the object entering the
>>right ear, and the sound entering the left ear. It is not very
>>useful if these four estimates are inconsistent. There need to be
>>processes in the brain to cross calibrate all sensory processes, in
>>order to assure consistency of interpretation.
>>Maybe the idea of calibration doesn't fit easily into your way
>>of looking at things. But I think it essential.
>This is perhaps the strongest version of your point. There's no
>doubt that this perceptual integration is something that resembles
>a calibration process, which is obviously learned. In fact, recent
>experiments with the barn owl (Knudsen, Eric I. (1998) Capacity
>for Plasticity in the Adult Owl Auditory System Expanded by
>Juvenile Experience. Science 1998 Mar 6;279(5356):1531-3) showed
>how this process seems to occur, and it appears to support your
>hypothesis.
>This is closely related to the biggest and most important mystery
>that I'm anxious to see solved: that of binding, or sensory
>integration. Without this problem solved, I doubt that we could
>have the necessary insights in order to propose how language
>emerges from perception.
Yes, I agree that they are related.
From: "Sergio Navega" <snavega@attglobal.net>
Subject: Re: Rationality and Formalizability
Date: 09 Nov 1999 00:00:00 GMT
Message-ID: <38282983_3@news3.prserv.net>
References: <80208d$bu@ux.cs.niu.edu> <3826cd8b_3@news3.prserv.net>
<807nl3$7pk@ux.cs.niu.edu>
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: abuse@prserv.net
X-Trace: 9 Nov 1999 14:02:43 GMT, 129.37.182.200
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.philosophy
Neil W Rickert wrote in message <807nl3$7pk@ux.cs.niu.edu>...
>"Sergio Navega" <snavega@attglobal.net> writes:
>>I agree with you about the notion of objective/subjective information
>>and if we keep our eyes on just those two sentences, your point is
>>correct. But I propose that sentence 2 must be analyzed in the
>>context of thousands of related others. Taken in isolation, sentence
>>2 is obviously meaningless. But suppose we listen to a third sentence,
>>following those:
>
>>Iottkye was elected as king of of the kingdom of Vvbbngw
>>on the planet Bzzeryy of the andromeda galaxy.
>
>>(can't believe I wrote this)
>>Again, this sentence may be said to contain zilch information.
>>But taken in *conjunction* with the previous sentence, we notice
>>that there are repeating names, and that these names appear to
>>be surrounded by relatively constant structures:
>
>I agree that there are now some interesting patterns. But there is
>still zero information. However there is a puzzle. Is the
>repetition an arbitrary coincidence, or is their a reason? Now we
>can develop a research program to try to track this down.
>
>Once again, you see me disagreeing with the Modlin approach where
>learning has to do with a statistical analysis of passively received
>signals. Instead, I am insisting that the system must interact with
>the world, and carry out experiments to discover whether anything
>useful can be done with these apparent coincidences. If we find
>something useful we can do with this, then the inputs become
>informative.
I agree. It is reasonable to see the appearance of a lot of
not meaningful coincidences during this process of checking for
patterns and I guess we have two ways to check their validity:
a) we "keep one eye" in their rate of repetition b) we try to
act on the world, changing some conditions and assessing its
effect (or not) on the chosen parameter(s).
I'd like to emphasize that what we're practicing in *both*
cases is the search for invariances. What appears to be valuable
are the circumstances in which something appears not to vary as
a function of different situations (intentionally provoked by
the organism or just passively perceived).
>
>>I think I understand what you mean here, agreeing that much of the
>>performance of an intelligent system depends on this judgement
>>process, fed by inner drives. But this would explain our high-level
>>behavior. I doubt it would shed much light on the low-level aspects,
>>which is certainly an important part of the problem. I may concede
>>that part of our perceptual abilities is influenced by reinforcements,
>>which are a function of our action/reaction decisions (and judgements).
>>But much of our perception happens automatically and mindlessly,
>>independent of our will.
>
>I do think that curiosity is one of those internal drives, and is
>probably one of the most important in terms of human cognition.
>
I also put curiosity as one of the internal drives of most animals.
And I relate this trait with our "wish" to look for invariances.
Extending this idea a bit, I'd say that curiosity is something
that rewards the organism while it is in the process of discovery.
Once the invariance is found, then the reward tends to zero,
forcing the organism to look for another "uncommon" situation.
Once another of that irregular situations is found (which means,
the surprise of finding something not predicted), this will
prompt the organism again to look for an "interpretation" of
that irregularity (here enters the process of creative manipulation
of the data, through novel ways of arrangement) that turns
it into an invariant, regular event, which again reduces reward
to zero and so on.
Regards,
Sergio Navega.
From: daryl@cogentex.com (Daryl McCullough)
Subject: Re: Rationality and Formalizability
Date: 08 Nov 1999 00:00:00 GMT
Message-ID: <8071l5$hi9@edrn.newsguy.com>
References: <7vvged$prn@ux.cs.niu.edu> <38243a97_1@news3.prserv.net>
<80208d$bu@ux.cs.niu.edu>
Organization: CoGenTex, Inc.
Newsgroups: comp.ai.philosophy
Neil says...
>>> Patterns depend on organization. You won't find patterns
>>> in unorganized data. If there are weak patterns in raw
>>> data, they reflect that fact that there is some organization
>>> present even in the raw data.
I don't understand the difference between "some organization
is present even in the raw data" and "some patterns are present
in the raw data".
Daryl McCullough
CoGenTex, Inc.
Ithaca, NY
From: "Sergio Navega" <snavega@attglobal.net>
Subject: Re: Rationality and Formalizability
Date: 08 Nov 1999 00:00:00 GMT
Message-ID: <382724f1_2@news3.prserv.net>
References: <7vvged$prn@ux.cs.niu.edu> <38243a97_1@news3.prserv.net>
<80208d$bu@ux.cs.niu.edu> <8071l5$hi9@edrn.newsguy.com>
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: abuse@prserv.net
X-Trace: 8 Nov 1999 19:30:57 GMT, 166.72.21.125
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.philosophy
Daryl McCullough wrote in message <8071l5$hi9@edrn.newsguy.com>...
>Neil says...
>
>>>> Patterns depend on organization. You won't find patterns
>>>> in unorganized data. If there are weak patterns in raw
>>>> data, they reflect that fact that there is some organization
>>>> present even in the raw data.
>
>I don't understand the difference between "some organization
>is present even in the raw data" and "some patterns are present
>in the raw data".
>
An interesting question that led me to comment from a special
viewpoint (sorry, not much related to the question itself).
We cannot, in principle, look to a signal (say, a senoidal tone
shown in an oscilloscope) and say that it contains a pattern,
without using some kind of preconceived notion of hierarchically
lower elements to compare with. I guess you had exposed this
very same question several weeks ago: where do comparison criteria
come from?
This is sort of that philosophically fundamental idea that we
can't derive inductive assessments about something without
using previously acquired notions. Or can we?
Now lets look it upside down, from a different angle. Suppose
we are following a spike train, a sequence of neural firings,
temporally distributed, with identical timing among the spikes.
They are all the same, under all aspects we analyze.
We check the response of the organism (behavior) and it is a
function of the presence or not of that regular spike train.
We're tempted to call that train a "pattern", because it is
regular in all aspects (time and voltage levels). But they aren't.
Each spike is the result of an action potential, which is
produced by several synaptic vesicles popping. The number
of synaptic vesicles popped in each action potential is variable.
Each spike in the train was produced by a different number of
vesicles popping and each vesicle contained a variable number
of molecules of neurotransmitters. No two spikes are alike: they
differ in minimum aspects like voltage, exact timing, waveshape,
etc.
Nevertheless, the neural machinery act over that spike train
as if each spike was mathematically equal to the others (that's
the luck of computational neuroscientists!). The machinery
is relatively insensitive to these small variations, which
allows us to say that that machinery sees (or ascribes)
similarity even when there isn't exact correspondence. This
machinery is not able to compare spikes with precision
above a certain level (which is obvious, due to the nature
of our universe). I put this as the first "comparison
criterion" that we have built-in, and this is, I hypothesize,
the root of all our other judgements of similarity.
It follows that pattern is an idea that starts to be
"meaningful" as the result of the evolutionarily designed
sensitivity of the physical construction of the neurons.
Some spikes are similar, others aren't.
Thus, some patterns may be present in raw data, in the sense
that the "spike coding" correspondent to that data possess an
interpretation of similarity, as given by that neural
machinery. For all practical purposes, they are the same,
they are redundant, they are a pattern.
But I guess Neil's point (which I agree) is that there are
*other* patterns that can be obtained from this very same raw
data, once we discover how to "filter" adequately. Whether
this can be seen as a calibration process, is another matter.
This filtering process appears to be a creative process involving
hundreds of neurons, one that demands experimentation with
several ways of "organizing" the clusters of data to compare
(recall that now we already have at least *one* criterion of
comparison). I propose that this creative mechanism is relatively
random at first and that this happens as the result of synaptic
plasticity, whose peek is at 18 months of age in the case
of humans. The brain adapts itself to strengthen the most
valuable connections, those who reveal *more* invariances in
the raw data (the goal is to reduce the complexity of the data).
Once these connections are developed and solidified, then most
incoming data is rapidly processed, with relevant features
identified by "almost dedicated circuits" (which work very fast
because they work in parallel), evolved predominantly during
this babyhood phase.
Regards,
Sergio Navega.
From: Neil W Rickert <rickert+nn@cs.niu.edu>
Newsgroups: comp.ai.philosophy
Subject: Re: Rationality and Formalizability
Date: 10 Nov 1999 13:25:35 -0600
Organization: Northern Illinois University
Lines: 60
Message-ID: <80cgrf$ipa@ux.cs.niu.edu>
References: <807nl3$7pk@ux.cs.niu.edu> <38282983_3@news3.prserv.net>
NNTP-Posting-Host: ux.cs.niu.edu
Path:
news3.prserv.net!newsfeed.us.ibm.net!ibm.net!news.maxwell.syr.edu!howland.erols.net!vixen.cso.uiuc.edu!husk.cso.niu.edu!not-for-mail
Xref: news3.prserv.net comp.ai.philosophy:68809
"Sergio Navega" <snavega@attglobal.net> writes:
>Neil W Rickert wrote in message <807nl3$7pk@ux.cs.niu.edu>...
>I'd like to emphasize that what we're practicing in *both*
>cases is the search for invariances. What appears to be valuable
>are the circumstances in which something appears not to vary as
>a function of different situations (intentionally provoked by
>the organism or just passively perceived).
I agree that we are searching for invariances. I think we are on the
same track here. But let me restate my position, in case there is
some miscommunication.
I am differing from the Modlin approach in two ways. Firstly, I am
saying that you might have to change the way you collect the data to
find some of the invariances, and experiment with different changes
till you find ways that work well. Secondly, I am saying that you
cannot know whether something is an invariant unless you take actions
to try and disturb the external world, so as to test under what
conditions it is invariant.
>>I do think that curiosity is one of those internal drives, and is
>>probably one of the most important in terms of human cognition.
>I also put curiosity as one of the internal drives of most animals.
>And I relate this trait with our "wish" to look for invariances.
Yes, I agree. J. J. Gibson described perception in terms of
invariances. In my view, by looking for invariances, we are looking
for ways to enhance perception by increasing the amount of
information that we can pickup from the environment.
However, I want to emphasize that the invariances don't tell the
whole story. Something that is perfectly invariant is a constant.
And there isn't very much information about a changing world if you
only look at the things which are constant. What is important about
invariants, is that they establish a framework from which you can
more readily observe what varies.
>Extending this idea a bit, I'd say that curiosity is something
>that rewards the organism while it is in the process of discovery.
>Once the invariance is found, then the reward tends to zero,
>forcing the organism to look for another "uncommon" situation.
>Once another of that irregular situations is found (which means,
>the surprise of finding something not predicted), this will
>prompt the organism again to look for an "interpretation" of
>that irregularity (here enters the process of creative manipulation
>of the data, through novel ways of arrangement) that turns
>it into an invariant, regular event, which again reduces reward
>to zero and so on.
I agree. But this helps to point out something else that is wrong
with the Modlin approach. For it is based on inductionism. It
assumes that induction basically works, and acquiring knowledge is
finding out the circumstances where it works. However as your point
on the importance of surprise points out, it is often the failures of
induction (the somethings not predicted), rather than the successes,
that lead us to expand our knowledge.
From: "Sergio Navega" <snavega@attglobal.net>
Newsgroups: comp.ai.philosophy
References: <807nl3$7pk@ux.cs.niu.edu> <38282983_3@news3.prserv.net>
<80cgrf$ipa@ux.cs.niu.edu>
Subject: Re: Rationality and Formalizability
Date: Wed, 10 Nov 1999 18:14:16 -0200
Lines: 111
Organization: Intelliwise Research and Training
X-Newsreader: Microsoft Outlook Express 4.71.1712.3
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
NNTP-Posting-Host: 166.72.29.196
Message-ID: <3829e0f4_4@news3.prserv.net>
X-Trace: 10 Nov 1999 21:17:40 GMT, 166.72.29.196
X-Complaints-To: abuse@prserv.net
Path: news3.prserv.net!166.72.29.196
Xref: news3.prserv.net comp.ai.philosophy:68819
Neil W Rickert wrote in message <80cgrf$ipa@ux.cs.niu.edu>...
>"Sergio Navega" <snavega@attglobal.net> writes:
>>Neil W Rickert wrote in message <807nl3$7pk@ux.cs.niu.edu>...
>
>>I'd like to emphasize that what we're practicing in *both*
>>cases is the search for invariances. What appears to be valuable
>>are the circumstances in which something appears not to vary as
>>a function of different situations (intentionally provoked by
>>the organism or just passively perceived).
>
>I agree that we are searching for invariances. I think we are on the
>same track here. But let me restate my position, in case there is
>some miscommunication.
>
>I am differing from the Modlin approach in two ways. Firstly, I am
>saying that you might have to change the way you collect the data to
>find some of the invariances, and experiment with different changes
>till you find ways that work well.
I agree entirely, with only one observation: I add, as another method
to experiment for invariances, random attempts of correlating data
that does not appear to correlate (and a subsequent memory to receive
reinforcements). I find this strategy required because there's no
other way to face the complexity of this initial learning other
than using random and exaustive (brute-force) methods. This is not
costly for a highly parallel brain such as ours and appears to be
related to the highly plastic period of the brain of babies.
> Secondly, I am saying that you
>cannot know whether something is an invariant unless you take actions
>to try and disturb the external world, so as to test under what
>conditions it is invariant.
Although I agree with part of this idea, I don't think this can
fit all situations. I'm sure Modlin would say (Bill, are you there?)
that several of these disturbances would occur naturally in the
world and a passive observer could find plenty of useful
information just by peeking at the data. Obviously, having the
chance of experimenting with the world would augment significantly
the range of discoveries, because of the benefit of redirecting
one's attention to a particular subset of the state space. I put
this interactive experimentation as something fundamental to
the learning process.
But purely unsupervised schemes have another big problem to
solve: without interacting with the world, the organism will not
be able to capture the perception/action loops (associations)
that will, later, be used to support such things as language.
I see the explanations of the origin of language as the most
difficult challenges to Modlin's ideas.
>
>>>I do think that curiosity is one of those internal drives, and is
>>>probably one of the most important in terms of human cognition.
>
>>I also put curiosity as one of the internal drives of most animals.
>>And I relate this trait with our "wish" to look for invariances.
>
>Yes, I agree. J. J. Gibson described perception in terms of
>invariances. In my view, by looking for invariances, we are looking
>for ways to enhance perception by increasing the amount of
>information that we can pickup from the environment.
>
>However, I want to emphasize that the invariances don't tell the
>whole story. Something that is perfectly invariant is a constant.
>And there isn't very much information about a changing world if you
>only look at the things which are constant. What is important about
>invariants, is that they establish a framework from which you can
>more readily observe what varies.
>
I agree entirely.
>>Extending this idea a bit, I'd say that curiosity is something
>>that rewards the organism while it is in the process of discovery.
>>Once the invariance is found, then the reward tends to zero,
>>forcing the organism to look for another "uncommon" situation.
>
>>Once another of that irregular situations is found (which means,
>>the surprise of finding something not predicted), this will
>>prompt the organism again to look for an "interpretation" of
>>that irregularity (here enters the process of creative manipulation
>>of the data, through novel ways of arrangement) that turns
>>it into an invariant, regular event, which again reduces reward
>>to zero and so on.
>
>I agree. But this helps to point out something else that is wrong
>with the Modlin approach. For it is based on inductionism. It
>assumes that induction basically works, and acquiring knowledge is
>finding out the circumstances where it works. However as your point
>on the importance of surprise points out, it is often the failures of
>induction (the somethings not predicted), rather than the successes,
>that lead us to expand our knowledge.
>
Well, you found a way to open a big hole in my ideas about inductionism.
After this, I can't say that induction is the origin of knowledge.
But I can say that without induction (which means, without a "memory"
of past events and the "hope" that they will repeat often) we wouldn't
have a clue to alert us when something valuable occurs (a deviation
from the expected). Then, induction may not be the origin of the
knowledge per se (which leaves us in peace with Hume), but it
surely is a fundamental auxiliary participating in the strategy
we use to derive knowledge from sensory data.
Regards,
Sergio Navega.
Back to Menu of Messages Sergio's Homepage
Any Comments? snavega@attglobal.net