The Universe as a Computer II

I like to use the topic of my last post and tie it back to the very first post I made to this blog.

In my first post I claimed that life is defined as something that resists the 2nd Law of Thermodynamics. Now this is more of a poetic statement than a scientific one so allow me to clarify. The 2nd law is a statistical fact that must be true about any large collection of discrete things (such as atoms and molecules or even the items on top of my desk). It basically says that there is a higher probability of such collections moving to a state of increased disorder (entropy) than increased order. So when I say that life resists the second law I am really stating that life has the property of expending energy to resist decay. It does this, of course, at the expense of those things in its immediate environment (including other life forms) so that on the whole the 2nd law is not violated.

Returning to the ideas expressed in my previous post and the New Scientist article, I think it is safe to say that if any law must be true in all possible universes, the 2nd law is as good a candidate as you are going to get. Given this, one must not ask why our universe is so tuned to support life. Such a statement implicitly assumes life as that which is composed of atoms and molecules. My definition of life is much more general. It does not require that there be such a thing as electric charge, for instance. It only requires that there be:

1) Some collection of discrete things
2) Some way for those discrete things to interact (i.e. at least one force)
3) Some emergent complex dynamics that can arise due to combinatorial configurations of discrete things and forces (this probably means the force must not be too weak or too strong and that it vary with distance).

Given such a system, I believe there is a high probability that in the course of large spans of time a configuration could evolve that resists the second law through actions such as replication and metabolism. It may even be inevitable for a much larger class of systems than we can consider simply by permuting the laws or constants of our own universe. For example, there may be deserts of non-life in the immediate vicinity of our universe's configuration of laws and constants but a a majestic bounty of life forms in the space of all possible laws.

Here again computers provide a wonderful analogy. Imagine a piece of working software. Almost any mild permutation of that piece of working software will lead to a broken piece of software. In fact, the probability of a crashing a program by flipping a single bit in its executable section is fairly high. However, it does not follow form this observation that all working programs must look almost exactly like this particular program. There are an infinite number of amazingly rich and varied programs in the vast space of all possible programs. So too, I believe there are a vast richness of life forms in the space of all possible universes.

The law that entropy always increases, holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell's equations — then so much the worse for Maxwell's equations. If it is found to be contradicted by observation — well, these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation. ”
--Sir Arthur Stanley Eddington, The Nature of the Physical World (1927)

The Universe as a Computer

Here is a quote from the latest issue of New Scientist. You can read the part of the article here (or the whole thing if you are a subscriber).

There is, however, another possibility: relinquish the notion of immutable, transcendent laws and try to explain the observed behaviour entirely in terms of processes occurring within the universe. As it happens, there is a growing minority of scientists whose concept of physical law departs radically from the orthodox view and whose ideas offer an ideal model for developing this picture. The burgeoning field of computer science has shifted our view of the physical world from that of a collection of interacting material particles to one of a seething network of information. In this way of looking at nature, the laws of physics are a form of software, or algorithm, while the material world - the hardware - plays the role of a gigantic computer.

This is by no means a new idea. However, it is gaining more traction and I believe it will become prevailing viewpoint in my lifetime or at least that of the generation of physicists  who grow up emerged in worlds such as Second Life.

The problem I have with the article is that it mentions how we need to explain why our universe is so tuned to support life. It is certainly true that life as we understand it could not arise if some of the fundamental constants of nature were altered just a tad. However, it does not follow that these alternate realities would not support complex systems for which we can have little understanding from our vantage point. I think if Stephen Wolfram's work on NKS showed anything at all it showed that complex dynamics can arise from quite simple initial ingredients.

Archetypes Redux

In an earlier post I wrote about how archetypes can be specified by a set of prototypical vectors with weights. A better explanation of this setup is that an Archetype is a set of vectors and the weights are the fuzzy membership of the vector in the set. Under this setup a vector with the characteristics of a boulder can exist in the rock archetype with a fuzzy membership value less than 1.0.

Vision Science

I just picked up Vision Science: Photons to Phenomenology by Stephen E. Palmer. Without exaggeration this is the best book on cognitive science I have ever read and the best book on science in general that I have read in a while.

As the title suggests this book covers vision from the physics of photons all the way through to the phenomenology of experience. The chapter on color is one of the most complete treatments I have ever seen in one book.

I think what a like most about the book is that it is authoritative, well researched and scholarly yet reads almost as easily as a pop science book. The book carries a hefty price tag ($82.00) but at 800 pages it is well worth it and you can find used editions at a significant discount.

Archetypes revisited

I partly addressed my displeasure with the prior post on archetypes, so if you are interested you may want to read the latest version.

The role of Archetypes in Semantic modeling

In a previous post I introduced the notion of Semantic Vectors. These are vectors (in the sense of the mathematical notion of a Vector Space) that can be used to model knowledge about the world. It is not yet clear to me how vectors, in and of themselves, can model much of what needs to be molded in a knowledge based system (at least without complicating the notion of a vector space so it only vaguely resembled its mathematical counterpart). This post is about one aspect of this challenge that I have begun working on in earnest. I have some hope that this challenge can be met by the model.

Imagine, if you will, a rock. If my notion of a semantic vector space has any value at all it should be able to model knowledge about a rock. Presumably a rock would be modeled as vector with explicit dimensions such as mass, density, hardness, etc. At the moment, it is not my intent to propose a specific set of dimensions sufficient to model something like a rock so this is only meant to give you a rough idea of the vector concept.

When I asked you to imagine a rock, which particular rock did you imagine?

Was it this one?

Or this one?Chances are you had a much more vague idea in your mind. Although the idea you had was vague it was probably not the idea of "Mount Everest" or "the tiniest pebble" even though theses have something rocky about them.

A system that purports to model knowledge of specific things must also model knowledge of general things. In fact, most truly intelligent behavior manifests itself as the fluid way we humans can deal with the general.

I use the term archetype to denote what must exist in a semantic model for it to effectively deal with generality.

At the moment, I will not be very specific about what archetypes are but rather I will talk about what they must do.

An archetype must place constraints on what can be the case for an x to be an instance of an archetype X. In other words if you offer rock23 as an instance of archetype ROCK there should be a well defined matching process that determines if this is the case.

An archetype must allow you to instantiate an instance of itself. Thus the ROCK archetype acts as a kind of factory for particular rocks that the system is able to conceive.

An archetype must specify what semantic dimensions are immutable and which are somewhat constrained and which are totally free. For instance, a rock is rigid, so although rocks can come in many shapes, once a particular rock is instantiated it will not typically distort without breaking into smaller pieces (lets ignore what might happen under extreme pressure or temperature for the moment). In contrast, the archetype for rock would not constrain where the rock can be located. I can imagine few places that you can put a rock where it would cease to be a rock (again lets ignore places like inside a volcano or a black hole, for now).

An archetype must model probabilities, at least in a relative sort of way. For example, there should be a notion that a perfectly uniform fire engine-red rock is less likely than a grayish-blackish-greenish rock with tiny silverish specks.

Archetypes also overlap. A BOULDER archetype overlaps a ROCK archetype and they system should know that a rock becomes a boulder by the application of the adjective BIG.

An intelligent entity must be able to reason about particular things and general classes of things. It would be rather odd and awkward, in my opinion, if the system had distinctly different ways to deal with specific things and general things. It would be nice if the system had a nice universal representation for both. Certainly, the fluid way in which humans can switch back and forth between the general and the specific lends credence to the existence of a uniform representational system. If a knowledge representation proposal (like my semantic vector concept) fails to deliver these characteristics then it should be viewed as implausible.

I am only just beginning to think in earnest out how the vector model can deal with archetypes. I have some hope but nothing that I am willing to commit to at the moment. Presently I am working with the idea that an archetype is nothing more than as set of pairs consisting of a vector and a weight. The vector provides an exemplar of an element of the archetype and the weight provides some information as to the likelihood. The nice thing about vectors is that, give two of them, a new vector can be produced that lies in the middle. Hence the vector model provides away of flushing out a sparsly populated archetype. Further, membership in the archetype can be tested using the distance metric of the semantic space.

The limitation of this approach has to do with the notion of the flexibility of various dimensions that I mentioned above. It would seem that the vector model, in and of itself, does not have an obvious way to represent such constraints. Perhaps this simply means that the model must be expanded but expansion always leads to complexity and a semantic modeler should always prefer economy. There is some hope for a solution here. The basic idea is to provide a means by which constraints can be implied by the vectors themselves but elaboration of this idea will have to wait for a future post.

Color your consciousness

Is my experience of red the same as yours? Or maybe when you experience red you experience something more like this relative to my experience? Can we ever say anything definitive here? It would seem hopeless.

Here is an interesting paper that adds some color to these ideas.