Reality by numbers

Here is an expert from an article by Max Tegmark in New Scientist that I read today. I find this sought of thinking amoung mainstream science encouraging.

So here is the crux of my argument. If you believe in an external reality independent of humans, then you must also believe in what I call the mathematical universe hypothesis: that our physical reality is a mathematical structure. In other words, we all live in a gigantic mathematical object - one that is more elaborate than a dodecahedron, and probably also more complex than objects with intimidating names like Calabi-Yau manifolds, tensor bundles and Hilbert spaces, which appear in today's most advanced theories. Everything in our world is purely mathematical - including you.

See Mathematical cosmos: Reality by numbers (requires subscription).

Alchemy and AI

For over four millennia the alchemists sought to transmute the elements. It is only from the modern vantage point provided by chemistry and physics that the we can clearly see how foolhardy their quest was. The alchemists believed the secret to the success that eluded them was a philosophers stone. It was thought that such a stone would allow the base elements to combine to achieve their goals of producing silver and gold (and eternal youth, to boot).

Although, given their methods, their goal was impossible, they did develop quite a few useful results (gun powder, paints, ceramics, and booze, to name a few).

The folly of the alchemists was clearly that they were operating at the wrong granularity. They worked at the level of atoms and molecules while their quest could only be achieved by the manipulation of protons and neutrons. However, no one can blame them for starting with the se most obvious ingredients. These were the things they could see, smell, taste and touch.

AI and Ontology are presently operating under a similar dilemma. Here the goal is the mastery of intelligence via endowing it to machines. Like the alchemists, practitioners of AI and semantics have largely dealt with the most obvious ingredients of thought - symbols. However, it is clear, at least to me, that symbols are at the wrong level. Symbols and symbol manipulation are the end game of intelligence; they are not the elementary particles.

If symbols and symbol manipulation were the end game, it seems clear to me that symbols would be much more pervasive throughout the animal kingdom. You would certainly see other intelligent creatures (rats, apes, dolphins) engaging in symbolic reasoning. If symbols were primary then there would be an obvious way to translate the cacophony of our brain's neural firings into symbolic thought. At present, this has not been the case.

If symbols are not elementary then what is? I think the only answer can be numbers. Now, before blasting me with the ridicule that is so obvious to anyone who has studied modern mathematics, allow me a moment to explain.

Yes, it is quite clear that modern mathematics is symbol manipulation. So numbers are symbols. To claim that numbers are more primitive than symbols while also acknowledging numbers as symbols would seem to place me on the shakiest grounds. Fully aware of my peril, I shall continue forth.

Symbols are used in mathematics (number theory, arithmetic, algebra, etc.) because they are the only vehicle open to humans. Just as protons and neutrons were out of reach of the alchemists, so to the true nature of numbers is out of our reach. What is this true nature? 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 are just glyphs. They have no special status in nature. That much is uncontroversial. Given appropriate rules (called mathematics) we can use them to build models that are useful in describing things that we measure. However, what we measure are magnitudes. Our brains perceive magnitudes across various modalities and we are trained thorough the study of mathematics to represent those magnitudes as numbers (symbols). But the magnitudes are more fundamental than the numbers used to model them.

The key property of magnitudes is that they stand in relation to other magnitudes. Differences in magnitudes can be perceived. Further, magnitudes of one modality (say, hue perception) can be discriminated from magnitudes of other modalities (say, temperature perception). This is not true of the symbols "red", "blue", "warm" and "cold". Yet, it is by using using various equivalent formalism for manipulating symbols - all reducible to first (or higher) order logic - that modern AI and ontology operate. Like the alchemists, important results are achieved but the true nature of intelligence and consciousness remain elusive.

I must now kindly ask my reader for a bit of sympathy toward my plight. I am suggesting something to be the case without having the proper tools to show it is in fact the case. It is not unlike the problem faced by the first atomists. Woe is me. However, it is at the root of these difficulties and seeming contradictions that my intuition tells me the answers to the mysteries of intelligence and consciousness lie. None of the alchemists lived to see atoms of lead split and reconstituted inside of accelerators to produce gold. Based on the acceleration of man's progress in our modern era, I am hopeful that I will live to see the "splitting of symbolic intelligence" to its more primitive state.

I wish software squeaked.

We all know that the software industry has been in trouble since its inception. Books like the Mythical Man Month by Brooks and the infamous 1968/69 NATO Software Engineering Reports were the first articulations of the so called software crisis. More recently there was the year 2000 fiasco, which, through a mixture of over exaggeration and tons of money spent on corrective action, turned out to be not that big of a crisis after all. In fact, the whole software crisis has never really reached crisis proportions. Sure, there have been some well documented software disasters, but every industry has its share of these. To me, a crisis implies that something is at the brink of collapse. I don't recall the software industry being on such a brink. The riches of the software industry show there has certainly been no financial collapse. Software only gets more remarkable as time marches on. New companies, whose livelihood depends 100% on software, emerge at a steady pace, go public, and create billionaires.

Yet everyone in the industry knows that there are big issues with software development. Its more of a relative crisis than an absolute one. Software engineers lament that software engineering is nothing like other forms of engineering. It is far less controlled, it has far less agreed upon norms, it relies too much on subjective taste, and its practitioners differ in talent by at least an order of magnitude.

The problem is that software does not squeak.

If a machinist machined a ball bearing or other part even a few thousandths of an inch off tolerance then, when deployed, the device would squeak, vibrate or otherwise do noticeably ugly things that would get progressively worse with time. Poorly engineered mechanical parts don't only squeak, they wear. And they do so rapidly.

In my job as a consultant for major corporations who can afford to pay for the best talent, I have seen lots of software that would squeak if it was in software's nature to do so. Hell, I've written some myself. If it only could squeak, how great it would be!!

Have you ever been in the position of explaining to a CEO, CFO or non-technical manager that the software they were entrusting their company's livelihood was really horribly engineered? What if the political climate at the company was not receptive to such dire news? What if, to make matters worse, the software seemed to work basically fine? Oh sure, a small outage here, a dropped customer order there, well these things happen. The business behind the software is complex, after all. Time to market is paramount. Yada, yada, yada.

But what CEO, no matter how technologically ignorant, would put a machine into the market that squeaked. He would look like an utter fool. Oh, how I wish software squeaked!! For if it did, the CEO would never find out about it, the engineers would be too embarrassed to ever let it out of the shop. Oh dear Turing, why don't your machines squeak!

I wish I could end this essay with the news that I have discovered a way to make poorly engineered software squeak. Sadly, no. I can't make it squeak and it probably never will. Some have made attempts at the equivalent of a squeak. Things like cyclometric complexity analyzers and the like. But the value of these metrics are highly contested among software professionals and there is slim hope they would sway a reluctant CEO into action. Squeaks are incontrovertible, metrics are not.

The best I can offer, and I'll be the first to admit its inadequacy, is to engineer your software as if it could squeak. And don't try to fix it with the equivalent of a glob of grease!

Communicating Sequential Processes

My recent interest in Erlang has motivated me to reread C.A.R. Hoare's classic Communicating Sequential Processes . If you are interested in software development and concurrency then I implore you to read (and re-read) this important work. If you won't take my word for it then consider the words of Edsger W. Dijkstra.

The most profound reason [the manuscript was eagerly awaited] , however, was keenly felt by those who had seen earlier drafts of his manuscript, which shed with surprising clarity new light on what computing science could—or even should—be. To say or feel that the computing scientist’s main challenge is not to get confused by
the complexities of his own making is one thing; it is quite a different
matter to discover and show how a strict adherence to the tangible and quite
explicit elegance of a few mathematical laws can achieve this lofty goal. It
is here that we, the grateful readers, reap to my taste the greatest benefits
from the scientific wisdom, the notational intrepidity, and the manipulative
agility of Charles Antony Richard Hoare.

Who Knew the Best Selling Book of all Time was about AI!

This little diddy showed up on the Erlang mailing list recently because the author had some equally unbelievable claims about his programming language that is "better than Erlang". This would make one of those great articles that get published on April Fools day, except in this case I am afraid the author is serious.

Artificial Intelligence From the Bible!


Time to go reorganize my book shelf. :-D

Professor Victor Raskin's Talk

This past Friday (8/24/07) Professor Raskin of Purdue University and Hakia gave a talk at the New York Semantic Web Meetup What follows is a summary of Raskin's points and my own thoughts on the topic.

Summary Of Key Points

• Conceptually, the Semantic Web (SW) is a good and noble vision.
• The present proposal for the SW by Tim Berners-Lee (et. al.) will fail.
• Formalisms like OWL don't capture meaning. Tagging is not representation of meaning (shallow semantics = no semantics).
• The average web author (or even above average) is not skilled enough to tag properly. Semantics requires well-trained ontologists.
• Manually tagging can be used to deceive search engines.
• Formalisms, in and of themselves, are useless. The meaning of the formalism is what counts.
• Ontology (like steal-making) is something that requires highly skilled practitioners. Ontology is not for the masses.
• Most computer scientists know next to nothing about language or semantics.
• Statistical and syntactic techniques are useless if one is after meaning.
• Native speakers are experts in using their language but are highly ignorant about their language (i.e., how language works).
• Meaning is language independent, so even though ontologies use symbols that look like words, they are really tokens for language-independent concepts.
• Raskin's Ontologic formalism is called Text Meaning Representation (TMR).
• TMR uses a frame like construct where the slots store case roles, constraints and other information like style modality, references, etc. (See http://ontologicalsemantics.com/tmr-new.pdf).
• The Semantic Web does not need OWL or any other tagged based system because web authors will not need to tag once a full Ontological Model (and other related tools, like lexicons, semantic parsers, etc.) are available.
• Ontological Search Engine will be able to index pages by meaning without the help of web authors.
• This is what Hakia is working on.

My Impressions of the Presentation

Professor Raskin is a very good presenter with a unique and humorous style (think of a cross between Jackie Mason, David Letterman and Albert Einstein). His points resonated well with my own impressions of the present architecture and direction of the Semantic Web. However, I thought that his presentation was too unbalanced. There were far too many slides critical of the SW and Tim Berners-Lee, in particular and far too little on Ontological Semantics.

My Thoughts on Raskin's Points

• I could not agree more with Raskin on the inadequacy of the present architecture of the SW.
• I also believe it is primarily the job of automated software tools to extract semantic information. However, I think web authors could help these tools be more efficient. My earlier post speaks to this point somewhat but after hearing Raskin's presentation, I plan to refine these thoughts in a future post.
• Raskin's point on the difficulty of "the masses" creating ontologies does not bode well for my vision of a Wisdi. However I am not the least bit discouraged by his bias toward expertly trained ontologists.
• Pre-Linux, experts in operating systems would have claimed that a commercial grade operating system could never be constructed by a loose band of programmer-enthusiasts.
• Pre-Wikipedia, intellectuals would have thumbed their nose at the idea of a competitive encyclopedia being authored by "the masses".
• The success of these projects stem from three major ingredients:
1. The involvement of some expert individuals
2. The involvement of many many enthusiastic but not necessarily expert participants.
3. Unending rounds of testing and refinement (ala Agile Methods and Extreme Programming).
• So I believe that a Wisdi model can ultimately kill an elitist approach because the elitist-expert approach can get too expensive. Information, knowledge and meaning do not remain static so ontologies must change and grow to remain relevant. I think an open collaborative approach is a good model for this endeavor. If you agree, I'd love to hear from you (flames equally welcome!).

References

http://ontologicalsemantics.com/

Ontological Semantics Book

The Whys and Hows of Ontological Semantics

Ambiguity, Disambiguation and KISS

My recent work on the Wisdi Project has me thinking quite a bit about ambiguity. Evolution has obviously provided us humans with an amazing ability to function quite well in the face of ambiguity. In fact, we often fail to perceive ambiguity until it is specifically brought to our attention. Ambiguity can arise in many different contexts and it is instructive to review some of these contexts, although you probably will not find them to be unfamiliar.

Human ability to deal with ambiguity has had some undesirable consequences. Our skill at disambiguation has left a legacy of ambiguous content spewed across the web. While almost all the content of the web was targeted for human consumption, its present vastness and continued exponential growth has made it paramount that machines come to our aid in dealing with it. Unfortunately, ambiguity is the bane of the information architects, knowledge engineers, ontologists and software developers who seek to distill knowledge from the morass of HTML.

Of all the forms of ambiguity mentioned in the above referenced Wikipedia article, word sense ambiguity is probably the most relevant to further development of search engines and other tools. You may find it instructive to read a survey of the state of the art in Word Sense Disambiguation (circa 1998). There is also a more recent book on the topic here.

An important goal, although certainly not the only goal, of the Semantic Web initiative is to eliminate ambiguity from online content via various ontology technologies such as Topic Maps, RDF, OWL, DAML+OIL. These are fairly heavy-handed technologies and perhaps it is instructive to consider how far we can proceed with a more light weight facility.

Keep It Simple Silly

Consider the history of the development of HTML. There are clearly many reasons why HTML was successful however simplicity was clearly a major one. This quote from Raggett on HTML 4 says it all.

What was needed was something very simple, at least in the beginning. Tim demonstrated a basic, but attractive way of publishing text by developing some software himself, and also his own simple protocol - HTTP - for retrieving other documents' text via hypertext links. Tim's own protocol, HTTP, stands for Hypertext Transfer Protocol. The text format for HTTP was named HTML, for Hypertext Mark-up Language; Tim's hypertext implementation was demonstrated on a NeXT workstation, which provided many of the tools he needed to develop his first prototype. By keeping things very simple, Tim encouraged others to build upon his ideas and to design further software for displaying HTML, and for setting up their own HTML documents ready for access.

Although I have great respect for Tim Berners-Lee, it is somewhat ironic that his proposals for the semantic web seemingly ignores the tried and true principles of KISS that made the web the success it is today. Some may argue that the over simplicity of the original design of HTML was what got us into this mess, but few who truly understand the history of computing would buy that argument. For better or worse, worse is better (caution, this link is a bit off topic, but interesting none the less)!

So, circling back to the start of this post, I have been doing a lot of thinking about ambiguity and disambiguation. The Wisdi Sets subproject hinges on the notion that an element of a set must be unambiguous (referentially transparent). This has me thinking about the role knowledge bases can play in improving the plight of those whose mission it is to build a better web. Perhaps, a very simple technology is all that is needed at the start.

Consider the exceedingly useful HTML span tag. The purpose of this tag is to group inline elements so that they can be stylized. Typically, this is done in conjunction with CSS technology. Why not also allow span (or a similar tag) to be used to provide the contextual information needed to reduce ambiguity? There are numerous ways this could be accomplished, but to make this suggestion concrete I'll simply propose a new span attribute called context.

I had a great time at the <span context="http://wisdi.net/ctx/rockMusic">rock</span> concert. My favorite moment was when Goth <span context="http://wisdi.net/ctx/surname">Rock</span>
climbed on top of the large <span context="http://wisdi.net/ctx/rock">rock</span>
and did a guitar solo.

It should not be to difficult to guess the intent of the span tags. They act as disambiguation aids for software, like a search engine's web crawler, that might process this page. The idea being that an authoritative site is used to provide standardized URL's for word disambiguation. Now one can argue that authors of content would not take the time to add this markup (and this is essentially the major argument against the Semantic Web) but clearly the simplicity of this proposal leads to ease of automation. A web authoring tool or service could easily flag words with ambiguous meaning and the author would simply point and click to direct the tool to insert needed tags.

One can debate the merits of overloading the span tag in this way but the principle is more important than the implementation. The relevant points are:

1. Familiar low-tech HTML facilities are used.
2. URL's provide the semantic context via an external service that both search engines and authoring tools can use.
3. We need not consider here what content exists at those URL's, they simply need to be accepted as definitive disambiguation resources by all parties.
4. This facility can not do everything that more sophisticated ontology languages can do, but who cares. Worse is better, after all.