Prof. Ray C. Dougherty's Research

A few weeks ago I attended a Wolfram Research event called Mathematica Publishers Day. The goal of the event was to highlight the capabilities of Mathematica 6 as a platform for technical publishing. I really enjoyed this event but was also pleasantly surprised by a talk that did not quite fit into the overall theme of the event but nevertheless was quite fascinating to me.

The presenter was Prof. Ray C. Dougherty, NYU Linguistics researcher. He used Mathematica to model all possible sine wave based communications systems. The presentation is available via Wolfram. Unfortunately, as with many interesting presentations, you needed to hear the talk to get the most out of it. Here are some interesting excerpts that I remember:

• The Cochlea is computing the second derivative of the auditory input.
• The most mathematically complex communication system is one where the transmitter and receiver have the same anatomy (e.g., wings of insects).
• Bats can hear phase changes because they can rotate their ears. A human can not hear a change in the rotation of a tuning fork but a bat can.
• Prof Dougherty believes he has a Chomsky generative grammar that enumerates all possible animal communication systems.
• He also believes he can map each possible system onto the integers in a natural way.
• From this he concludes that evolution must proceed in jumps.
• He relates this idea to the evolution of all possible Tic Tac Toe Games to illustrate the notion that all such games are not unique and similarly the space of all possible communication systems contains many redundant systems as well.
• He goes on to visualising distributions of the primes to illustrate that there are systems that are not random but whose patterns are too complex for us to model in a simple fashion. Explains how this is related to the ideas in Stephen Wolfram's NKS.

Stephen Wolfram on Software Design and Naming

In this post Stephen Wolfram talks about design reviews and the importance of naming things correctly. I sought of envy him for working in a company where one can afford to spend a total of 10,000 hours in design review and where you sweat over the details of naming every last function. As much as I can see the merits of Agile Development, I have never worked on an Agile project that creating anything nearly as elegant as Mathematica. Sure, Agile focuses on working software and "good enough" software and the economics of most software projects makes this mode of development necessary. But it would be nice to one day work on a project where time to market was less important than the end result.

Ubiquitous Eigenvectors and Quantum Computing

Recently I have been studying Quantum Computation (QC). If you want to get anywhere with QC you need to master Linear Algebra and Vector Spaces since QC is baically an exercise in applied vector space theory.

Being a bit rusty in the topic myself, I decided to pick up the book Finite-Dimensional Vector Spaces by Paul Richard Halmos. Although Halmos is one of my favorite math authors, I picked this book primarily because it has a Kindle edition. It turns out that another one of Halmos's books, Linear Algebra Problem Book, is a far better choice for the non-mathematcian. The later book walks you step by step through bite sized problems and provides hints (and also all the answers if you get stuck). A free resource with answers can also be found here.

One of the central mathematical techniques at the heart of Linear Algebra is the concept of Eigenvectors and Eigenvalues. The term Eigen is derived from German and means "characteristic". An Eigen Decomposition is a method of reducing a square matrix to into a constant (eigenvalue) and a vector (eigenvector). This decompostion is central to many problems in physics.

It turns out that the study of Eigen decompostion can yield deep insight into problems that are in the realm of computer science. Consider, for instance, this paper about Google's page rank algorithm and the Eigenface technique for facial recognition.

Coming to grips with the mathematics behind Vector Spaces is one of the single most rewarding experiences for anyone interested in advanced problems in computer science. It is a must if you ever want to graduate from the comprehension of clasical algorithms to the comprehension of quantum algorithms. However, if are curious about QC but the thought of learning advanced linear algebra sounds like too big of a comitment, then you might want to check out Quantum Computation explained to my Mother. This is the most approachable paper I have ever read on the topic that is also mathematically accurate.

Reality by numbers

Here is an expert from an article by Max Tegmark in New Scientist that I read today. I find this sought of thinking amoung mainstream science encouraging.

So here is the crux of my argument. If you believe in an external reality independent of humans, then you must also believe in what I call the mathematical universe hypothesis: that our physical reality is a mathematical structure. In other words, we all live in a gigantic mathematical object - one that is more elaborate than a dodecahedron, and probably also more complex than objects with intimidating names like Calabi-Yau manifolds, tensor bundles and Hilbert spaces, which appear in today's most advanced theories. Everything in our world is purely mathematical - including you.

See Mathematical cosmos: Reality by numbers (requires subscription).

Alchemy and AI

For over four millennia the alchemists sought to transmute the elements. It is only from the modern vantage point provided by chemistry and physics that the we can clearly see how foolhardy their quest was. The alchemists believed the secret to the success that eluded them was a philosophers stone. It was thought that such a stone would allow the base elements to combine to achieve their goals of producing silver and gold (and eternal youth, to boot).

Although, given their methods, their goal was impossible, they did develop quite a few useful results (gun powder, paints, ceramics, and booze, to name a few).

The folly of the alchemists was clearly that they were operating at the wrong granularity. They worked at the level of atoms and molecules while their quest could only be achieved by the manipulation of protons and neutrons. However, no one can blame them for starting with the se most obvious ingredients. These were the things they could see, smell, taste and touch.

AI and Ontology are presently operating under a similar dilemma. Here the goal is the mastery of intelligence via endowing it to machines. Like the alchemists, practitioners of AI and semantics have largely dealt with the most obvious ingredients of thought - symbols. However, it is clear, at least to me, that symbols are at the wrong level. Symbols and symbol manipulation are the end game of intelligence; they are not the elementary particles.

If symbols and symbol manipulation were the end game, it seems clear to me that symbols would be much more pervasive throughout the animal kingdom. You would certainly see other intelligent creatures (rats, apes, dolphins) engaging in symbolic reasoning. If symbols were primary then there would be an obvious way to translate the cacophony of our brain's neural firings into symbolic thought. At present, this has not been the case.

If symbols are not elementary then what is? I think the only answer can be numbers. Now, before blasting me with the ridicule that is so obvious to anyone who has studied modern mathematics, allow me a moment to explain.

Yes, it is quite clear that modern mathematics is symbol manipulation. So numbers are symbols. To claim that numbers are more primitive than symbols while also acknowledging numbers as symbols would seem to place me on the shakiest grounds. Fully aware of my peril, I shall continue forth.

Symbols are used in mathematics (number theory, arithmetic, algebra, etc.) because they are the only vehicle open to humans. Just as protons and neutrons were out of reach of the alchemists, so to the true nature of numbers is out of our reach. What is this true nature? 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 are just glyphs. They have no special status in nature. That much is uncontroversial. Given appropriate rules (called mathematics) we can use them to build models that are useful in describing things that we measure. However, what we measure are magnitudes. Our brains perceive magnitudes across various modalities and we are trained thorough the study of mathematics to represent those magnitudes as numbers (symbols). But the magnitudes are more fundamental than the numbers used to model them.

The key property of magnitudes is that they stand in relation to other magnitudes. Differences in magnitudes can be perceived. Further, magnitudes of one modality (say, hue perception) can be discriminated from magnitudes of other modalities (say, temperature perception). This is not true of the symbols "red", "blue", "warm" and "cold". Yet, it is by using using various equivalent formalism for manipulating symbols - all reducible to first (or higher) order logic - that modern AI and ontology operate. Like the alchemists, important results are achieved but the true nature of intelligence and consciousness remain elusive.

I must now kindly ask my reader for a bit of sympathy toward my plight. I am suggesting something to be the case without having the proper tools to show it is in fact the case. It is not unlike the problem faced by the first atomists. Woe is me. However, it is at the root of these difficulties and seeming contradictions that my intuition tells me the answers to the mysteries of intelligence and consciousness lie. None of the alchemists lived to see atoms of lead split and reconstituted inside of accelerators to produce gold. Based on the acceleration of man's progress in our modern era, I am hopeful that I will live to see the "splitting of symbolic intelligence" to its more primitive state.

I wish software squeaked.

We all know that the software industry has been in trouble since its inception. Books like the Mythical Man Month by Brooks and the infamous 1968/69 NATO Software Engineering Reports were the first articulations of the so called software crisis. More recently there was the year 2000 fiasco, which, through a mixture of over exaggeration and tons of money spent on corrective action, turned out to be not that big of a crisis after all. In fact, the whole software crisis has never really reached crisis proportions. Sure, there have been some well documented software disasters, but every industry has its share of these. To me, a crisis implies that something is at the brink of collapse. I don't recall the software industry being on such a brink. The riches of the software industry show there has certainly been no financial collapse. Software only gets more remarkable as time marches on. New companies, whose livelihood depends 100% on software, emerge at a steady pace, go public, and create billionaires.

Yet everyone in the industry knows that there are big issues with software development. Its more of a relative crisis than an absolute one. Software engineers lament that software engineering is nothing like other forms of engineering. It is far less controlled, it has far less agreed upon norms, it relies too much on subjective taste, and its practitioners differ in talent by at least an order of magnitude.

The problem is that software does not squeak.

If a machinist machined a ball bearing or other part even a few thousandths of an inch off tolerance then, when deployed, the device would squeak, vibrate or otherwise do noticeably ugly things that would get progressively worse with time. Poorly engineered mechanical parts don't only squeak, they wear. And they do so rapidly.

In my job as a consultant for major corporations who can afford to pay for the best talent, I have seen lots of software that would squeak if it was in software's nature to do so. Hell, I've written some myself. If it only could squeak, how great it would be!!

Have you ever been in the position of explaining to a CEO, CFO or non-technical manager that the software they were entrusting their company's livelihood was really horribly engineered? What if the political climate at the company was not receptive to such dire news? What if, to make matters worse, the software seemed to work basically fine? Oh sure, a small outage here, a dropped customer order there, well these things happen. The business behind the software is complex, after all. Time to market is paramount. Yada, yada, yada.

But what CEO, no matter how technologically ignorant, would put a machine into the market that squeaked. He would look like an utter fool. Oh, how I wish software squeaked!! For if it did, the CEO would never find out about it, the engineers would be too embarrassed to ever let it out of the shop. Oh dear Turing, why don't your machines squeak!

I wish I could end this essay with the news that I have discovered a way to make poorly engineered software squeak. Sadly, no. I can't make it squeak and it probably never will. Some have made attempts at the equivalent of a squeak. Things like cyclometric complexity analyzers and the like. But the value of these metrics are highly contested among software professionals and there is slim hope they would sway a reluctant CEO into action. Squeaks are incontrovertible, metrics are not.

The best I can offer, and I'll be the first to admit its inadequacy, is to engineer your software as if it could squeak. And don't try to fix it with the equivalent of a glob of grease!

Communicating Sequential Processes

My recent interest in Erlang has motivated me to reread C.A.R. Hoare's classic Communicating Sequential Processes . If you are interested in software development and concurrency then I implore you to read (and re-read) this important work. If you won't take my word for it then consider the words of Edsger W. Dijkstra.

The most profound reason [the manuscript was eagerly awaited] , however, was keenly felt by those who had seen earlier drafts of his manuscript, which shed with surprising clarity new light on what computing science could—or even should—be. To say or feel that the computing scientist’s main challenge is not to get confused by
the complexities of his own making is one thing; it is quite a different
matter to discover and show how a strict adherence to the tangible and quite
explicit elegance of a few mathematical laws can achieve this lofty goal. It
is here that we, the grateful readers, reap to my taste the greatest benefits
from the scientific wisdom, the notational intrepidity, and the manipulative
agility of Charles Antony Richard Hoare.