Selected Newsgroup Message
From: "Sergio Navega" <snavega@ibm.net>
Subject: Re: NN formats
Date: 17 May 1999 00:00:00 GMT
Message-ID: <3740344e@news3.us.ibm.net>
References: <3739A199.9E2B8209@erols.com> <3739B919.B4A6AF36@clickshop.com>
<3739C62F.527BAF91@erols.com> <3739E816.4BF79A4B@clickshop.com>
<373A143C.6A467BFF@erols.com> <373C1480.7482A5B5@tig.com.au>
<373C58B7.E700B8F6@erols.com> <373EC872.80D8C02D@tig.com.au>
<7hoift$1ie$1@nnrp1.deja.com>
X-Notice: should be reported to postmaster@ibm.net
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: postmaster@ibm.net
X-Trace: 17 May 1999 15:22:54 GMT, 129.37.183.189
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.nat-lang,bionet.neuroscience,comp.ai.neural-nets
Jiri Donat wrote in message <7hoift$1ie$1@nnrp1.deja.com>...
>
>To me, the biggest difference between natural NN and ANN is that every
>digital simulation of ANN network has a discrete set of states (however
>large the set is). This "limitation" (if we understand this feature of
>digital representations of ANNs on today's computers as a limitation -
>and some theories do) is inherited in our existing tools for ANN
>simulations - in digital computers.
>
I'm not sure I understand you here. In fact, biological neurons
seen from "outside" are just things that fire or don't fire, in a
purely discrete manner. There don't seem to be any other meaningful
characteristic (such as waveshape or voltage) from the output of a
biological neuron, just the presence or not of the pulse. But our
models fail to account for what happens *inside* the neuron, as the
operation to fire or not fire seems to be the result of a *much*
more complicated process in the biological neurons than the simple
weight summing/thresholding of our ANNs.
>Every "artificial neurone" is an exactly defined unit which is
>originally defined using general mathematical functions, but in its
>computer realisation (if we don't use an analogue computer for its
>simulation) has a discrete set of states - however complex the original
>description is. So it can be generally described as a multidimensional
>table (=combinations of inputs and the output, or outputs, depending on
>the ANN model). Generally speaking, this table evolves over the time
>(as the network "learns" and "lives"), but in the majority of ANN
>models this table could be just extended by some additional columns
>(weights, threshold) and then it is *static* over the whole life of
>ANN. So most of today's ANNs are reducible to Cellular Automata.
Which turns ANNs into things like Turing machines, and then susceptible
to be processed by a "symbolic machine". I think that all these
facts put ANNs into the class of useful things that humans have
created without much relevance to the "real neuron". What worries
me is that some research in cognitive science is being conducted by
using connectionist systems as models and maybe this is worse than a
rough approximation of the real thing. In particular, the behavior
of populations of neurons (with temporal synchrony, ensenble "coding",
etc) may be a more relevant thing to model than ANNs.
Regards,
Sergio Navega.
From: juola@mathcs.duq.edu (Patrick Juola)
Subject: Re: NN formats
Date: 17 May 1999 00:00:00 GMT
Message-ID: <7hpk1p$u4$1@quine.mathcs.duq.edu>
References: <3739A199.9E2B8209@erols.com> <373EC872.80D8C02D@tig.com.au>
<7hoift$1ie$1@nnrp1.deja.com> <3740344e@news3.us.ibm.net>
Organization: Duquesne University, Pittsburgh PA USA
Newsgroups: comp.ai.nat-lang,bionet.neuroscience,comp.ai.neural-nets
In article <3740344e@news3.us.ibm.net>, Sergio Navega <snavega@ibm.net>
wrote:
>Jiri Donat wrote in message <7hoift$1ie$1@nnrp1.deja.com>...
>>
>>To me, the biggest difference between natural NN and ANN is that every
>>digital simulation of ANN network has a discrete set of states (however
>>large the set is). This "limitation" (if we understand this feature of
>>digital representations of ANNs on today's computers as a limitation -
>>and some theories do) is inherited in our existing tools for ANN
>>simulations - in digital computers.
>>
>
>
>I'm not sure I understand you here. In fact, biological neurons
>seen from "outside" are just things that fire or don't fire, in a
>purely discrete manner.
Seen from *FAR* outside, yes. In a similar fashion, from sufficiently
far outside, the Solar System is a point mass.
> There don't seem to be any other meaningful
>characteristic (such as waveshape or voltage) from the output of a
>biological neuron, just the presence or not of the pulse.
This is incorrect. What's usually regarded as more important than
the presence or absence of a pulse is the firing *rate*, measured
as a scalar quantity. Different neurons respond at different rates
depending on the circumstances, &c.
Viewed in this light, the (scalar) activation level present at
the output of an ANN unit is a model of the scalar activation level
of the output of a real neuron. In more sophisticated models, this
time course can be explicitly taken into account -- Birkbeck College,
Univ. of London and Cal-tech both have active research groups looking
at this sort of model.
-kitten
From: "Sergio Navega" <snavega@ibm.net>
Subject: Re: NN formats
Date: 17 May 1999 00:00:00 GMT
Message-ID: <37408107@news3.us.ibm.net>
References: <3739A199.9E2B8209@erols.com> <373EC872.80D8C02D@tig.com.au>
<7hoift$1ie$1@nnrp1.deja.com> <3740344e@news3.us.ibm.net>
<7hpk1p$u4$1@quine.mathcs.duq.edu>
X-Notice: should be reported to postmaster@ibm.net
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: postmaster@ibm.net
X-Trace: 17 May 1999 20:50:15 GMT, 129.37.182.124
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.nat-lang,bionet.neuroscience,comp.ai.neural-nets
Patrick Juola wrote in message <7hpk1p$u4$1@quine.mathcs.duq.edu>...
>In article <3740344e@news3.us.ibm.net>, Sergio Navega <snavega@ibm.net>
wrote:
>>Jiri Donat wrote in message <7hoift$1ie$1@nnrp1.deja.com>...
>>>
>>>To me, the biggest difference between natural NN and ANN is that every
>>>digital simulation of ANN network has a discrete set of states (however
>>>large the set is). This "limitation" (if we understand this feature
of
>>>digital representations of ANNs on today's computers as a limitation -
>>>and some theories do) is inherited in our existing tools for ANN
>>>simulations - in digital computers.
>>>
>>
>>
>>I'm not sure I understand you here. In fact, biological neurons
>>seen from "outside" are just things that fire or don't fire, in a
>>purely discrete manner.
>
>Seen from *FAR* outside, yes. In a similar fashion, from sufficiently
>far outside, the Solar System is a point mass.
>
My point exactly. If we're studying the kinematic behavior of the
milky way, it doesn't pay off to know the mass of Mars.
>> There don't seem to be any other meaningful
>>characteristic (such as waveshape or voltage) from the output of a
>>biological neuron, just the presence or not of the pulse.
>
>This is incorrect. What's usually regarded as more important than
>the presence or absence of a pulse is the firing *rate*, measured
>as a scalar quantity. Different neurons respond at different rates
>depending on the circumstances, &c.
>
Although this is an open issue (not all neuroscientists agree with
firing rate), your assertion does not invalidate what I've said.
In other words, I said that each element of a spike train does
not seem to be differentiable by such characteristics as waveshape or
voltage, but only by the discrete presence or absence of the pulse
(while composing a scalar, mean firing rate or by means of timing
among each spike is something I didn't mention).
>Viewed in this light, the (scalar) activation level present at
>the output of an ANN unit is a model of the scalar activation level
>of the output of a real neuron. In more sophisticated models, this
>time course can be explicitly taken into account -- Birkbeck College,
>Univ. of London and Cal-tech both have active research groups looking
>at this sort of model.
>
I agree with that, but this is part of the open issue. Apparently
what is being settled is that neurons close to sensory inputs seem
to work considering the mean firing rate and that neurons of more inner
portions of the cortex care more for the "individual spikes". For
instance, it has been demonstrated by Steveninck and Bialek that
single spikes of the visual system of the fly contain significant
information about the stimulus.
But I think there's an additional contestant here, and this is
related to the proposals that put a role on the synchrony of
populations of neurons (Wolf Singer is an important name in this
regard). So although all these results are not meant to dismiss
current models of ANNs, I would certainly assume that what ANNs
model is a very different kind of thing than biological neurons.
Regards,
Sergio Navega.
From: "Sergio Navega" <snavega@ibm.net>
Subject: Re: NN formats
Date: 17 May 1999 00:00:00 GMT
Message-ID: <3740344b@news3.us.ibm.net>
References: <3739A199.9E2B8209@erols.com> <3739B919.B4A6AF36@clickshop.com>
<3739C62F.527BAF91@erols.com> <3739E816.4BF79A4B@clickshop.com>
<373A143C.6A467BFF@erols.com> <373C1480.7482A5B5@tig.com.au>
<373C58B7.E700B8F6@erols.com> <373EC872.80D8C02D@tig.com.au>
X-Notice: should be reported to postmaster@ibm.net
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: postmaster@ibm.net
X-Trace: 17 May 1999 15:22:51 GMT, 129.37.183.189
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.nat-lang
Anton Funk Tism Trees wrote in message <373EC872.80D8C02D@tig.com.au>...
>[snip]
>Additionally, human brains seem to come with a certain amount of
>structure built in thanks to evolution. Language abilities are
>foremost in the list of such inherited structure, but there are
>also a whole range of things like reflexes (smiling, crying,
>sucking), and recognition of human faces.
>
Anton, I enjoyed the remaining of your post, but in this particular
paragraph you touched a matter that, in my vision, could be
interpreted differently. In particular, native language mechanisms,
although assumed to be present by the majority of the cognitive
scientists of today, is finding serious opposition from recent
neurobiological evidences. In my vision, the question of innateness
of language is open, and in this regard I'd stay on the side
of those who say that language is learned, not innate.
Regards,
Sergio Navega.
From: juola@mathcs.duq.edu (Patrick Juola)
Subject: Re: NN formats
Date: 17 May 1999 00:00:00 GMT
Message-ID: <7hpkan$ul$1@quine.mathcs.duq.edu>
References: <3739A199.9E2B8209@erols.com> <373C58B7.E700B8F6@erols.com>
<373EC872.80D8C02D@tig.com.au> <3740344b@news3.us.ibm.net>
Organization: Duquesne University, Pittsburgh PA USA
Newsgroups: comp.ai.nat-lang
In article <3740344b@news3.us.ibm.net>, Sergio Navega <snavega@ibm.net>
wrote:
>
>Anton Funk Tism Trees wrote in message <373EC872.80D8C02D@tig.com.au>...
>>[snip]
>>Additionally, human brains seem to come with a certain amount of
>>structure built in thanks to evolution. Language abilities are
>>foremost in the list of such inherited structure, but there are
>>also a whole range of things like reflexes (smiling, crying,
>>sucking), and recognition of human faces.
>>
>
>Anton, I enjoyed the remaining of your post, but in this particular
>paragraph you touched a matter that, in my vision, could be
>interpreted differently. In particular, native language mechanisms,
>although assumed to be present by the majority of the cognitive
>scientists of today, is finding serious opposition from recent
>neurobiological evidences. In my vision, the question of innateness
>of language is open, and in this regard I'd stay on the side
>of those who say that language is learned, not innate.
And you'd be grossly and obviously wrong.
There is clearly *SOMETHING* innate about language and language
capacity -- why else does every known (non-pathological) example
of H. sap. have the capacity for "human-level" language, while no
known examples of G. gorilla or P. troglodytes do?
The research questions -- and this coming from a card-carrying member
of the "language is learned" camp -- is not whether language is
learned or innate, but how much or how little support *must* be
in the inherited neurobiological structure. No one would claim
that language mechanisms are not innate; the question is whether
language mechanisms *as postulated by Chomskian linguistics* are
innate.
-kitten
From: "Sergio Navega" <snavega@ibm.net>
Subject: Re: NN formats
Date: 17 May 1999 00:00:00 GMT
Message-ID: <3740810a@news3.us.ibm.net>
References: <3739A199.9E2B8209@erols.com> <373C58B7.E700B8F6@erols.com>
<373EC872.80D8C02D@tig.com.au> <3740344b@news3.us.ibm.net>
<7hpkan$ul$1@quine.mathcs.duq.edu>
X-Notice: should be reported to postmaster@ibm.net
X-MimeOLE: Produced By Microsoft MimeOLE V4.71.1712.3
X-Complaints-To: postmaster@ibm.net
X-Trace: 17 May 1999 20:50:18 GMT, 129.37.182.124
Organization: Intelliwise Research and Training
Newsgroups: comp.ai.nat-lang
Patrick Juola wrote in message <7hpkan$ul$1@quine.mathcs.duq.edu>...
>In article <3740344b@news3.us.ibm.net>, Sergio Navega <snavega@ibm.net>
wrote:
>>
>>Anton Funk Tism Trees wrote in message <373EC872.80D8C02D@tig.com.au>...
>>>[snip]
>>>Additionally, human brains seem to come with a certain amount of
>>>structure built in thanks to evolution. Language abilities are
>>>foremost in the list of such inherited structure, but there are
>>>also a whole range of things like reflexes (smiling, crying,
>>>sucking), and recognition of human faces.
>>>
>>
>>Anton, I enjoyed the remaining of your post, but in this particular
>>paragraph you touched a matter that, in my vision, could be
>>interpreted differently. In particular, native language mechanisms,
>>although assumed to be present by the majority of the cognitive
>>scientists of today, is finding serious opposition from recent
>>neurobiological evidences. In my vision, the question of innateness
>>of language is open, and in this regard I'd stay on the side
>>of those who say that language is learned, not innate.
>
>And you'd be grossly and obviously wrong.
>
Well, you go directly to the point, don't you? :-)
I like that.
>There is clearly *SOMETHING* innate about language and language
>capacity -- why else does every known (non-pathological) example
>of H. sap. have the capacity for "human-level" language, while no
>known examples of G. gorilla or P. troglodytes do?
>
>The research questions -- and this coming from a card-carrying member
>of the "language is learned" camp -- is not whether language is
>learned or innate, but how much or how little support *must* be
>in the inherited neurobiological structure. No one would claim
>that language mechanisms are not innate; the question is whether
>language mechanisms *as postulated by Chomskian linguistics* are
>innate.
>
It is obvious that something about language must be innate,
comparable, for instance, with our ability to perceive color. You
wrote correctly when you raised that the main point is whether
Chomskian theory of innate knowledge of grammatical structures
is tenable or not. It is this that deserves refuting.
But we must be cautious about the issue of language in non-human
animals. Some time ago, few would accept that other primates
could develop syntactically sophisticated forms of communication.
Today, this has been proven a point to discuss, in light of
experiments such as Savage-Rumbaugh's Kanzi bonobo monkey.
It is easy to say that bonobo apes may have difficulty in
creating symbolic-like languages by themselves, without
external training. This would explain why languages of this
kind did not appear naturally in these species so far. But
that does not mean they can't handle simplified symbolic
structures.
It seems to be more an effect of a lack of preconditions for
its *natural emergence* in their community. What's lacking
appears to be a small "detail" that bootstraps the whole
process. Maybe cortex size. Some anthropologists say
that this small detail, in case of humans, was the development
of a more capable phonological apparatus, about 40.000 years ago.
I think that when we "join the pieces" of the puzzle with more
information from these diverse disciplines it will be easier
to justify anti-chomskian positions.
It is a pity that you're a 'card-carrying member of language
is learned camp', because so do I. For quite some time
I was looking forward to discuss with a strong nativist some
of their more cherished reasons to follow Chomsky and Fodor.
Regards,
Sergio Navega.
Back to Menu of Messages Sergio's Homepage
Any Comments? snavega@attglobal.net