Greetings,

I had very much enjoyed the first two chapters of Chaitin's philosophical book
Omega ( http://www.cs.auckland.ac.nz/CDMTCS/chaitin/omega.html ), which I had
read some time ago, and I think they are very sound and informative, as well
as thought provoking. The part where three proofs of the infinite number of
primes is given is plain wonderful. I don't have much to say about these
chapters. On the more philosophical chapters, however, I would like to
communicate my views.

Some comments on Chaitin's "Meta Math! The Quest for Omega", Chapters II - VI
follow. I think you will find my remarks in Chapter VI the most interesting.
In the text below, Chaitin is addressed by the second personal pronoun.

Regards,

-----------------------------------------------------------------------------

Chapter II
----------

LISP, an Elegant Programming Language

A question I liked asking, and nobody wanted to answer of course, was "How
would you know if you ran on LISP?". One of the most important properties of
LISP is "reflective simplicity", ie. low H(sim(LISP)) in LISP... I think
there might be some information-theoretic criteria for choosing one language
over the other for doing a job, (For LISP, this might be certain reflective
tasks such as self-modifying codes) I suspect complex systems like the mind
must necessarily be towers of Babylon. (No need to say that this must have
something to do with the irreducible essence of the world, are not
programming languages really virtual machine specifications, to build
machines in which programs are run? There are so many kinds of virtual
machines in an African jungle or inside an ameoba!)


Chapter III
-----------

Who was Leibniz:

Leibniz, and his master Spinoza, gave us also the most plausible theory of
mind: multism, as I dub it. His is the only theory that tries to explain what
subjective experience is like, in my opinion. I learnt that they had thought
of exactly the same thing after I formed the idea of multism and especially
digital multism:

The theory is very simple. Each algorithmically independent program in your
mind constitutes an irreducible part of "qualia". Red feels like red, because
it is processed in a certain, unique way. Reasoning feels like reasoning, and
not red, because these algorithms are quite independent... So overall, there
are n bits of Omega worth of qualia packed into your genes, but that does not
mean you cannot create new ones, it depends on how plastic your nervous
system is. I cut out crucial parts, my reasoning comes from my motivation to
find abstract analogues: "algorithmically independent" and "qualitatively
different" are too similar to ignore! (e.g. even the ideas of geometry and
integers are qualitatively different! they are not as "large" as the color of
an object in our experience "sphere", because their computational loads are
much smaller... they are at a compressed dimension, to give a physical
metaphor...)


Leibniz on Complexity and Randomness

Indeed Leibniz's ideas about the goodness of a scientific theory are much more
articulated than those attributed to William of Occam. Would a reference to
Solomonoff's and Schmidhuber's work be relevant here? They have worked on
universal distributions, too.

I agree that compression is a good model of understanding, that highlighted
paragraph was a nice read.


Compressing digital pictures and video:

Off topic, but I want to mention here that majority of the image compression
schemes are lossy, they model the sensory limits of our vision system (ie.
energy in a band) or the nature of image sequences (little change from frame
to frame), rather than intelligent codings that can find redundancies, in my
opinion. In fact, I am more or less convinced that there is no such thing as
a proper image coding at the moment. We are basically using 1-dimensional
simple coding schemes (based on dictionaries, etc.) for lossless coding of
images. I believe we need algorithmic codes for images, but it seems to me
harder than it might sound at first.

This of course also brings out the roles of lossy decompression in definition
of Kolmogorov complexity. I tried to define it, but since I was so busy with
some realworld tasks, I couldn't finish it. The latest place I came was that
it would be wrong to define "loss" as bit error rate, it should be defined as
algorithmic similarity. [Which is rather obvious and trivial, I know.]


What is self-delimiting information?


I liked this part a lot, I had never seen this subject written in such an
entertaining and actually readable way.

More on Information Theory & Biology
Gas/Crystal
In intuitive terms, I think Crystal is highly organized, while the Gas has
chaotic behavior. However, they are both not _adaptive_.

Bach, Shostakovich:
One can actually show that this is the case, by using specific
decompressors... Could some real numbers be interesting?


Chapter IV: Intermezzo
----------------------

The Parable of the Rose

I think the essence of the image of the rose actually lies outside the image
data, e.g. since the 2d image is a rendering from a 3d world... If one knows
how the rendering is done, and how to trace it back (computer vision) then
one could come up with more concise 3d models, but I think this is
conditional entropy (as I discuss below)...

Literary texts: already done in the work of Vitanyi et al. by using a specific
decompressor instead of a universal computer.
http://citeseer.ist.psu.edu/689388.html
http://www.cwi.nl/~paulv/papers/informationdistance.pdf
http://citeseer.ist.psu.edu/li03similarity.html

Also for music http://arxiv.org/abs/cs.SD/0303025
They have reported some nice results I think.

I think there is a missing point here that may be discussed. It is common to
both image and text data, which humans use other information to cognize. In
the mind, we can say that all sensory information is compressed together, to
produce a model of the world. If it is a good compression of the time series,
then, it also becomes a really nice tool for predicting the future. (As in
the work of Schmidhuber and Hutter)

In giving an interpretation to the image of the rose, we do not consider only
a single rose, but the idea of a rose. The common sense concept of a rose is
comprised of sensory data fusion (smells like *this*), and abstraction
(categories/mereology: has leaves) at least. This can be modeled with a
program that can recognize or regenerate the sensory data of a rose.
Likewise, for literary texts, the algorithmic information of two sentences
can be quite close, but the meaning can be the opposite e.g. "p exists" vs.
"p doesn't exist", Consider "p will never be one of the things you will
encounter", which is a constructive semantics for "p doesn't exist". Its
syntactic distance, say, max{H(a/b),H(b/a)} can be large, but it means
exactly the same thing (according to a certain *theory* of semantics). So,
information content of *syntax* alone does not tell the whole story, as can
be evidenced by the effect of synonym and dictionary definition substitutions
on the raw information content of a text. Furthermore, a universal
decompressor may actually deceive us into thinking that two texts are similar
although they are not by detecting algorithmic redundancies that are not
carried into meaning, such as by use of similar propositions or,
morphologically close words that are semantically very far apart. [I think it
can be argued that these might also be said to happen with images]

So, all of this should be considered, in my opinion, in semantic terms. When
we see a rose, we give it an interpretation, which itself is learnt from
scratch. The interpretation is later fed back into the model and takes
corrective action (it could add or remove a subroutine). Ideally, the entire
action might be seen as compressing the entire sensory history of the
organism. The better the compression is, the better accuracy in
classification of data into parts. Roughly, there is a model of the world p
in the brain. The sensory data s is cognized by constructing a conditional
program p approximating |p*| = H(s/m), we can call this perception (when |p*|
is too large, we almost certainly fail in perceiving, it's meaningless for us
because we cannot find any good program to get from the model to the sense
data) I will not expound on the learning strategy, I don't think it is known,
but in very simplistic terms, it would boil down to combining empirical
trials to enhance algorithmic models. [Here, an assumption is that, given
large enough data, the 3d programmatic model of the sensory history will
actually be a minimal one, too]

The semantics of the text might be similarly conceived. Let's think that
semantics function meaning = phi(text) can take the text to a *program* in
the language of whatever it is that the common sense might be using. (e.g. a
language of thought). This would again be constructed as a minimal
conditional program with H(phi(text)/m) entropy, because the references must
not contradict the world model (experience), and in fact, a principle of
induction is at work, we are guessing the meaning of the text, and for that
we might want to consider smaller conditional programs as more likely (and
indeed less complex phi(text)...). But, we cannot just set H(phi(text)/m) =
c, a small constant. We have to for instance consider that there is a
consistent relation between H(text) and H(phi(text)), and that things like
principle of compositionality might actually hold, e.g. it's more likely that
H(phi(a,b)) = H(phi(a)) + H(phi(b)/phi(a)) + O(1) than not... [Still, this is
a very lacking picture of the whole process, because we can use language to
interact with the world, there is no communication and use in the above
prototheory, therefore it's very very inadequate]

I believe that there may be many more extensions to such trains of thought
which might help us reason about the limits of intelligence, one day. (One of
the requirements, in my simplistic scientist's mind, is to define the meaning
of "abstract", e.g. is it a kind of lossy compression?)

"And my ultimate ambition, which hopefully somebody will someday achieve,
would be to prove that life, intelligence and consciousness must with high
probability evolve in one of these toy worlds. "

Such a mathematical theory of evolution is highly desirable. Could it be as
simple as saying that a program-size or efficiency bias is present in the
universe (a property of the very *architecture* of _the_ computer?), or would
it actually require us to go the experimental route, collect a lot of data
from the real world, and show just how well our algorithms model change and
adaptivity? I nowadays tend to believe the latter is the case. Maybe, the
logical reality is that, the more general a theory of intelligence is, the
less efficient (smart!) a program it describes.


Theoretical Physics & Digital Philosophy

I cannot say how much this relates to Zeno's argument, which harbors an
apparent logical contradiction to my mind. Zeno did not have the tool of
"limit" , e.g. Leibniz's infinitesmal calculus, and hence the error.

Planck scale. The uncertainty principle does seem to imply that nothing might
exist beneath the planck scale, neither in space nor time, that is one of the
common interpretations. I have been also told of a lattice theory in physics
which analyzes quantum configurations in this fashion. Many physicist have
told me that there is absolutely no physical property which can store a real
number, to this date I remain somehow convinced, but I do not believe 100%
because I do not in general endorse faith. (I mean our knowledge seems to be
quite limited at the moment)

On sci.physics.discrete, we had discussed the possibilities for strong
continuum theories. One of the weirdest possibilities is that the universe is
a fractal continuous mechanism, which contains submechanisms that are
identical to itself, or has submechanisms as complex as itself. This is
basically the high school fantasy that there are universes beneath the Planck
scale, there might be a whole lot of possibilities that are basically
unobservable. On the other hand, Juergen Schmidhuber makes interesting
remarks about a nonhalting program that simply runs every program, his idea
is that this has appeal due to its algorithmic simplicity: we expect the
nature to be simple. (Why?) His "algorithmic theories of everything" is an
interesting read, and I think it is much relevant to the issues you raise in
"Omega".



Chapter V Labyrinth of the Continuum
-----------------------------------

The idea of a number with a countably infinite number of digits is, in my
opinion, a property of reflective thought. Let's partition a brain into an A
brain and a B brain (like in Minsky's SOM). A brain is a reactive brain, it
experiences the outside world, and acts according to a program. B's job is to
watch A's computation and program it, a reflective brain. B would write good
programs for A only if it knew a bit about theory of computation. I have
argued elsewhere that mathematics arises at a third level of reflection at
least, however, let's collapse all these extra levels at B. I think the
important idea here is that B knows that there are nonhalting computations
that can be represented with symbols (like a = 0.0101010101010101...) [e.g.
Leibniz became linguistically aware of this capability, he also had a way to
represent these theories of computation and modify them. In a way, he
transcended himself, into a higher order of complexity. In my opinion, for a
computational mind the concept of infinity is "programmatic capability to
construct loops, and run them up to a bound"]

On the highlighted sentence about "God's transcendence": Nowadays, a nice
argument about human mind is that it is capable of numerically simulating
continuous systems, so it might be working like the numerical analysis
researcher next door who is writing parallel FORTRAN programs to crunch big
matrices. That way, the mind may be aspiring to continuous computations.

Cantor: Did not he actually get mad in the end?

5. unnameable: I did not understand this part. I am perhaps thinking too
philosophically, but I guess that more explanation would be needed for this
item.

Highlighted passage:
> Why should I believe in a real number if I can't calculate it, if I can't
> prove what its bits are, and if I can't even refer to it? And each of these
> things happens with probability one! The real line from 0 to 1 looks more
> and more like a Swiss cheese, more and more like a stunningly black
> high-mountain sky studded with pin-pricks of light.

I like the writing here, it's a little dramatic.


The "Unutterable" and the Pythagorean School.

I think the better conclusion that Pythagoreans had to draw was that the space
is not continuous...

Re: Kronecker, I think his forceful statements play a large role in digital
philosophy, today, but in my futile mind, a better way to view numbers is
psychological. An integer is an abstract concept, it exists only in heads. A
bitstring or computation, however, may exist, directly... (But abstract
concepts are part of programs, therefore they follow rules, which corresponds
to a logical reality. That is of course not to say that particular programs
exist *before* they are written!)


Cantor's Continuum Problem: Kronecker might have commented that "It makes no
sense to ask the extent of two objects that do not exist".


Borel's number: Yes, believing in existence of a Borel number is identical to
a faith in ominous knowledge of a God.

Reals are uncomputable with probability one: " It's possible that you get a
computable real, but it's infinitely unlikely. ", would it then be precise to
say that it's possible?

Reals are un-nameable with probability one!

I sense some trouble in this definition of name. Omega has a name, but it is a
random real. For a name to exist, it suffices that we can describe it in a
system (e.g. provide a way to resolve its reference), not that it is chosen
from a countable infinity of names, in my opinion. Omega can have a name,
because in our virtual machine (e.g. your mind), there is only one name,
"Omega", that is fixed on mathematical descriptions of a oracle for halting
problem, (e.g. it does not bother naming all integers, for instance)



Chapter VI---Complexity, Randomness & Incompleteness
----------------------------------------------------

Irreducible Truths and the Greek Ideal of Reason

Would not it be a better compromise to say that we do not yet know the
compressibility of "interesting" or "useful" theorems ? (e.g. those that
will, for instance, help us write an AI) Omega is maximally unknowable, so
perhaps it makes no sense to try to prove it. (In fact, I believe such ideas
ultimately lead to a digital interpretation of Wittgenstein)


Coin Tosses, Randomness vs. Reason, True for no Reason, Unconnected Facts

My first impression is that Leibniz's and your arguments are essentially
identical to my mental irreducibility argument, however you do not make it
quite explicit that we must accept as a premise that minds are discrete
computers for the logical conclusion.
(See
http://groups.google.com/groups?selm=fa69ae35.0408310328.3f527e3e%40posting.goog\
le.com )

Also of interest is Godel's disjunctive proposition discussed in Lucas's
article:
http://users.ox.ac.uk/~jrlucas/Godel/implgoed.html

I argue that combining this proposition with your results imply that minds are
computers:

(
http://groups.google.com/groups?selm=fa69ae35.0409021551.38b91a56%40posting.goog\
le.com )

So, according to Godel, you have proven that minds are computers.


Kurt Godel: By your remarks about rationalism, it might seem to some curious
readers that you describe Kurt Godel as a logical positivist. I think he
considered himself in objection to such ideas like conventionalism and
verificationism, and definitely outside the Vienna Circle. However, he was a
realist foremost, and since he preferred to work in the framework of logic,
perhaps a logicist as well.


Creativity/randomness: While randomness has not much to do with free will as
Hume explains, it must have something to do with creativity. I suspect our
brains routinely gather entropy from the environment for use in some rather
fundamental randomized algorithms that work well (for things like random
generation of programs of a certain-length, etc. A good way to do this might
be to perceive things, collect the percepts in a temporary storage, and then
compress the programs further, and use them as as bitstrings elsewhere).
These random programs are somewhat observable in behavior of infants with
less sophisticated control programs, in my opinion. That is, I agree with
your view in general. However, many other things can be turned into
complexity: such as time and communication. I think there may be further
relations to be worked out between program-size, time and communication.
(Basically, I think interaction with the world must be modelled as
communication of two computer systems.)


Mind at the molecular level: A biologist friend of mine explained to me that
the chemical gradients in cells have a precise mechanical organization, which
might lead us to suspect that cellular complexity might be much greater than
we had previously assumed, e.g. a neural circuit model might be insufficient
to explain the mind.


Zeitgeist: It is dangerous because it wants us to be like rather badly
designed machines, instead of building smart machines. Logic alone can be
evil, because it can serve as the justification of unethical decisions, as it
happens today ... (One of the things you are proving may be that logic is not
the only or the best possible mode of reasoning... Maybe we like it because
it has a simple rule, not because it is the way of God.)


--
Eray Ozkural (exa) <erayo@...>
Comp. Sci. Dept., Bilkent University, Ankara KDE Project: http://www.kde.org
http://www.cs.bilkent.edu.tr/~erayo http://www.soundclick.com/malfunct
GPG public key fingerprint: 360C 852F 88B0 A745 F31B EA0F 7C07 AE16 874D 539C