Professor John McCarthy
Father of AI

Appearance and Reality

A Puzzle and a Challenge to Machine Learning and Pattern Recognition

30 August 2006

Our senses give us only partial information about the objects present in our vicinity. In particular, vision gives only a 2-dimensional view of 3-dimensional objects. Our visual systems and our brains use sense information to learn about the 3-dimensional objects.

Also humans and dogs can represent objects that are not presently visible. (The evidence about dogs is that if a thrown ball goes out of sight, the dog will look for it.) Humans can infer the existence of objects that are out of sight, and human learning from experience often involves learning about the hidden reality behind some phenomenon. This is what science is usually about, but it occurs in common sense reasoning as well.

Machine learning research, to my knowledge, has so far involved classifying appearances, and has not involved inferring reality behind experience. Classifying experience is inadequate as a model of human learning and also inadequate for robotic applications. I offer a puzzle that would involve inference about the hidden reality. Another way of looking at it is that we use observations to recognize patterns in the world, not just patterns in the observations. [2001 August: I have found members to the pattern recognition community rather narrow minded about this. To put it sharply, they recognize patterns in appearance but not patterns in the spatio-temporal world behind appearance.]

Below, if your browser is up to it, you will see a circle of 13 letters, although some positions may have no letter. To the left and right of each position is a tab. The only action you can do is to click on a tab which makes the appearance change.

Actually the clicks affect the reality of which you can see the appearance.

PROBLEM: What's the reality behind the appearance?

As you will see, the reality of this problem is not the grand reality of Plato and other philosophers, but something quite mundane and common to human experience.

This needs a browser that supports java. Please direct your attention to Netscape or Microsoft Thanks anyways.

You may find this alternate version which uses colors instead of letters to be easier. After you have looked at the solution, ask yourself how to make a computer program that can infer reality behind appearance as you have had to do. I don't have a definite answer. Human ability to solve the problem involves some aspects of common sense physics, and you might want to provide your program with those concepts. Stephen Muggleton and Ramon Otero have made considerable progress using inductive logic programming.

The Java applet is by Tim McCarthy, after a Lisp program by Tom Costello and John McCarthy. We also thank Barton Smith and Eyal Amir.