When I was in my early 20's, just out of college, I wanted to make an extreme move somewhere, so I went to Chicago to work at the Chicago Mercantile Exchange, in the commodities business. It was a job and place for which I was ridiculously unsuited, given that my highest ambition at the time was to find the 1990s equivalent of Ken Kesey's Merry Pranksters and hop on whatever contemporary version of Furthur was heading across the country.
The one thing that being alone in Chicago in the dead of winter (what better time could there be to move to one of the coldest cities in America than January?) did for me was allow me to spend some time reading things that I would not otherwise have the time or mental energy to read. I read several of Thomas Mann's longest novels, including Buddenbrooks and Faust and Pynchon's Gravity's Rainbow among other things.
But the book that affected my thinking the most, by far, was Douglas Hofstadter's Goedel, Escher, Bach: An Eternal Golden Braid. I bought it blind from a cool little used bookstore in Hyde Park; I can't remember what attracted to me but it was probably the psychedelic-looking Escher drawings on the book jacket.
The book, if approached the way that Hofstadter would like you to, is like a college textbook - more like a college course in what appear to be a collection of topics that interest Hofstadter. It's not just a book you read, the book assigns you homework. Though Escher and Bach are an important part of H's thinking about the idea of recursiveness, Godel is really the core. The focus of the first half of the book is really that one gets the best possible understanding that a non-mathematician can get of the principles behind Godel's Incompleteness Theorem. I am the last person to be qualified to explain the I.T. to anyone, but the best explanation of it that I am capable of is this: Godel proves logically that any mathematical system devisable could be jerry-rigged with a logical time bomb of the form "this statement cannot be proved," meaning that no system can be both complete (prove everything that's true), and consistent (not prove anything that isn't). Specifically he was responding to Bertrand Russell's insanely complex Principia Mathematica, which famously took 360 pages to prove that 1 + 1 = 2. (Followed by the comment "The above proposition is occasionally useful." A real card, that Russell.) But Godel went on from plunging the sword of logic into the heart of the greatest work of axiomatic set theory ever to go on and make the general case.
WARNING: I AM NOT A MATHEMATICIAN, I AM A HIGH SCHOOL ALGEBRA TEACHER. DON'T BASE ANY UNDERSTANDING OF GODEL ON WHAT I WROTE; IF YOU'RE CURIOUS, RESEARCH HIS WORK THROUGH A RESPECTED AUTHORITY.
Of course, my mind was blown, and like nearly everyone first exposed to the IT for the first time, I got completely the wrong message from it.
The second half of the book is entirely about getting the reader to understand why the message I and so many other people get ("Mathematics can't explain anything! Two plus two is five! The human soul is impenetrable to reason!") is the wrong message. The more I have talked to people about the book, the more I have come to the conclusion that I am one of about five people who have actually finished the thing. And honestly, if I wasn't stuck in a freezing hellhole a thousand miles from anyone that I had any personal relationship with, I might not have either.
Of course, there are stupid and smart versions of what Hofstadter would call the misinterpretations of the IT. The stupid ones are in the form of the first two examples I wrote above. The smart version, and the interpretation that I think Hofstadter is specifically answering, is best formed by Roger Penrose, a brilliant mathematician and physicist approximately a bazillion times smarter than me I am sure.
Penrose concludes from the Incompleteness Theorem, roughly, that there are certain kinds of problems that logical systems cannot solve, but people can. From there he goes on to conclude that a computer (which is just a really, really complex logical system based on a Turing machine), can never achieve human intelligence, or what scientists call consciousness. Let me restate my earlier warning; if you really want to understand Penrose's strong AI skepticism, you need to read The Emperor's New Mind, as well as the original form of the argument as made by J.R. Lucas in The Freedom of the Will I am only giving you my best interpretation of what he argues.
Hofstadter's case is that consciousness is based on recursive functions, or what he calls "strange loops": functions that repeatedly refer back to themselves. Recursive functions are actually quite common in programming (many timer functions work this way), but Hofstadter's strange loops call themselves in a much more complex fashion. Essentially Hofstadter argues that when such loops get deep enough (we're talking 30 million neurons a second deep), the result is consciousness. In later works he suggests that a certain kind of controlled randomness (sort of Bayesian) helps, too.
What has dismayed me in reading so much about Strong AI since is that, while many people refer to what is sometimes called the Lucas-Penrose Thesis, Hofstadter's answer to it is not so much dismissed as never even mentioned. I would read with interest if a qualified mathematician or logician tore the arguments in GEB to pieces through reasoning, and I'd be curious if they even dismissed them as not worth answering. But it's literally as if the book was never written. It's not as if it's obscure; the book won the Pulitzer in 1980 after all.
I think that the problem with GEB is that the way the book is designed misses nearly all of Hofstadter's intended audience. I once asked a fairly important mathematician about the book, and his response was a somewhat condescending, "yes, well, the world does need popularizations, after all," assuming that the book was nothing more than Godel for Dummies. A former girlfriend, a doctor in piano from Rochester, said that she thought he got some things about Bach wrong (didn't say what), and so she didn't read any further. A third friend, an intelligent but druggy poet type, looked at a few pages that I showed him and said "that makes my head hurt." All three of them, needless to say, never got within a mile of the point that DH was trying to make.
GEB is a rare book because DH is saying "You won't understand the point I want to make about artificial intelligence unless you first understand Godel's Incompleteness Theorem, the form of a Bach fugue, the nature of Escher's art, and some stuff about quantum theory, DNA coding and entemology. So I'm going to them to you, with help from parabolic dialogues by some amusing characters invented by Lewis Carroll." Unfortunately, that calls for a rare type of reader. The people who already understand some of these "prerequisites" stop reading on the assumption, "feh, I already know this." The people who don't stop because it's just too much damn work.
I'm told that DH, realizing the problem, has restated his thesis in a far simpler new book called I am a Strange Loop, which was just released last year. Of course I'll read it as soon as I get my hands on it. But I know it won't be anything like the cerebral-cortex detonation that first reading GEB was. And until some of the arguments that Hofstadter made get the kind of attention and response they deserve, it's just hard to take anything anyone says about strong AI seriously.