Beyond AI: Creating the Conscience of the Machine [Format Kindle]

The first two thirds of this book survey current knowledge of AI and make some guesses about when and how it will take off. This part is more eloquent than most books on similar subjects, and its somewhat different from normal perspective makes it worth reading if you are reading several books on the subject. But ease of reading is the only criterion by which this section stands out as better than competing books.
The last five chapters that are surprisingly good, and should shame most professional philosophers whose writings by comparison are a waste of time.
His chapter on consciousness, qualia, and related issues is more concise and persuasive than anything else I've read on these subjects. It's unlikely to change the opinions of people who have already thought about these subjects, but it's an excellent place for people who are unfamiliar with them to start.
His discussions of ethics using game theory and evolutionary pressures is an excellent way to frame ethical discussions.
My biggest disappointment was that he starts to recognize a possibly important risk of AI when he says "disparities among the abilities of AIs ... could negate the evolutionary pressure to reciprocal altruism", but then seems to dismiss that thoughtlessly ("The notion of one single AI taking off and obtaining hegemony over the whole world by its own efforts is ludicrous").
He probably has semi-plausible grounds for dismissing some of the scenarios of this nature that have been proposed (e.g. the speed at which some people imagine an AI would take off is improbable). But if AIs with sufficiently general purpose intelligence enhance their intelligence at disparate rates for long enough, the results would render most of the book's discussion of ethics irrelevant. The time it took humans to accumulate knowledge didn't give Neanderthals much opportunity to adapt. Would the result have been different if Neanderthals had learned to trade with humans? The answer is not obvious, and probably depends on Neanderthal learning abilities in ways that I don't know how to analyze.
Also, his arguments for optimism aren't quite as strong as he thinks. His point that career criminals are generally of low intelligence is reassuring if the number of criminals is all that matters. But when the harm done by one relatively smart criminal can be very large (e.g. Mao), it's hard to say that the number of criminals is all that matters.
Here's a nice quote from Mencken which this book quotes part of:
Moral certainty is always a sign of cultural inferiority. The more uncivilized the man, the surer he is that he knows precisely what is right and what is wrong. All human progress, even in morals, has been the work of men who have doubted the current moral values, not of men who have whooped them up and tried to enforce them. The truly civilized man is always skeptical and tolerant, in this field as in all others. His culture is based on 'I am not too sure.'
Another interesting tidbit is the anecdote that H.G. Wells predicted in 1907 that flying machines would be built. In spite of knowing a lot about attempts to build them, he wasn't aware that the Wright brothers had succeeded in 1903.
If an AI started running in 2003 that has accumulated the knowledge of a 4-year old human and has the ability to continue learning at human or faster speeds, would we have noticed? Or would the reports we see about it sound too much like the reports of failed AIs for us to pay attention?

Hollywood and the science fiction literature has indulged themselves over the last few decades in the prospect of highly intelligent machines either taking over human affairs or in fact acting to completely destroy human civilization. Oddly, these scenarios presuppose that entities that possess high intelligence would engage in this type of control or violence. Possessing high intelligence is assumed to be uncorrelated with possessing a conscience, or even being inversely related to it. Such biases make for excellent movies and science fiction novels, but there is no evidence as yet that would support the notion that conscience is independent of intelligence, nor has there been a careful scientific study of the connection between the two. But even in some quarters in the field of artificial intelligence (AI), there are those who are worried about the prospects of intelligent machines unleashing havoc on human civilization. Are these worries justified, and if so, what can be done to thwart the construction of these kinds of machines? Can one indeed build a machine with a conscience or should such machines be built?

This book gives a realistic appraisal of progress in artificial intelligence and sheds considerable light on these questions. It is careful to distinguish between fact and fiction, between what has been accomplished and what has not, and it does so without falling into the trap of extreme skepticism, the latter of which seems to happen to so many who are deeply involved in AI research. Indeed, after an initial period of extreme confidence in research results, and a designation as "intelligent", the confidence wanes until it is eventually viewed as a "trivial" discovery or merely a "program." There are many indications from the historical accounts of AI research that this pattern is repeated often.

This author though takes the general reader through this history and also takes a view of future developments in artificial intelligence, discussing at various places in the book the possibility of a technological "singularity" sometime in the next fifty years. Readers who are curious about the status of machine intelligence will find an understandable overview in this book, but it can still be of interest to those, such as this reviewer, who are working "in the trenches" of applied artificial intelligence and are interested in the opinions of researchers affiliated with the academy. The author does not delve deeply into the technologies, algorithms, and mathematics for this type of reader, but there are some new ideas within the covers that definitely make the book worth reading.

Machine intelligence has advanced, the author argues, and he gives many examples. Robots for example, can currently navigate with the same adeptness as a three-year old child, which is astounding considering what was possible just ten years ago. Readers who own and develop AIBO robot dogs will understand this claim, as their navigation abilities are impressive. If one thinks qualitatively, then one can project with a fair degree of confidence that robots will be able to interact with the environment with the same adeptness as an adult human within the next two decades. This prediction would however be difficult to put on a quantitative foundation, one must arrive at a measurable definition of robot-environment interaction. The lack of quantitative measures of progress has plagued the AI community since its inception in the early 1950's, especially the lack of a general, measurable definition of intelligence. In the opinion of this reviewer, the field of cybernetics and control theory, its generalization, has formulated the best quantitative notion of intelligence to date. Cybernetics is discussed in some detail in this book (along with its "death"). The author seems to believe though that it is information theory that promises the best measurable definition of intelligence. He discusses some reasons for this view, but does not elaborate with any detail, except brief commentary on how it can be used to measure, by using the concept of entropy, the predictive power of theories.

Particularly interesting in the book is the discussion of the "ELIZA effect" that refers to the program invented by Joseph Weizenbaum in the early 1960's that was designed to converse with a human subject, with the intent of fooling the subject into believing that the program understood what she was saying. The author scoffs at any imputation of understanding by ELIZA, and uses the "ELIZA effect" to describe any effort or claim by AI researchers that their work is a significant advance, and not just a bag of tricks that can easily mislead. But there is a serious problem with the author's use of the "ELIZA effect", in that significant advances may indeed have been made, but then after they are studied and understood they are then viewed as insignificant, and the discoverers are then labeled as falling prey to the ELIZA effect. As an example of how this scenario might be played out, consider an English professor who retired ten years before the advent of sophisticated spell checkers and real-time English grammar. She then comes out of retirement and decides to write a novel, and discovers this spelling/grammar checker, marveling at its abilities and definitely convinced that it displays intelligence. But sometime thereafter an AI expert reveals to her how it actually works and she then begins to accept it as merely a software program, no different really than some of the crude writing software she used years earlier.

But the author's belief in the ELIZA effect does not mean that he does not believe that intelligent machines (or "software") have not been achieved. However, this intelligence has only been able to operate in specific domains. As an example, he discusses the SHRDLU system invented by Terry Winograd, which was able to converse about a tabletop on which were placed a set of children's blocks. The author believes that SHRDLU was able to achieve genuine understanding, albeit in a very specific domain: the blocks world. This domain-specificity has been the hallmark of all of the commercial successes of artificial intelligence, since business are primarily concerned with automating tasks in very specific domains, such as managing and analyzing networks, collecting and interpreting information from competitors, or finding profitable financial opportunities by sifting through mountains of data. Machines that are able to think in many different domains may not be useful in this regard. A machine that is able to troubleshoot a network would be useful to a network manager, but if it had expertise in chess playing and decided to do this instead, this would raise the ire of the network manager.

For this reviewer, the most interesting part of the book was the discussion on `autogenous systems' because of its novelty and because it is related to the efforts to build machines that possess general intelligence. The author defines such a system as one that is able to extend itself arbitrarily, and thus go beyond preconceived limits. An autogenous machine will therefore be able to confront new and innovative situations or problems without excessive fiddling by the designer. Its cognitive structure, as one might call it, can build concepts and engage in learning on-the-fly without external intervention. At the present time, such systems are the holy grail of AI and there are concentrated efforts to build them. If they are built, and this reviewer is confident that this will be the case, will they fall into the usual pattern of first being viewed as major breakthroughs, and then latter as merely "programs?" If history is a guide this will happen, but such machines will be the tour de force of the twenty-first century, possibly bringing about a "singularity" as the author discusses, but also serving as an example of what can be accomplished with that low-voltage mass of biological matter called the human brain, which is the most impressive machine, and maybe indeed a universal one, that has yet arisen.

At some point we will realize all the chatter we hear about "I don't want things to change" or "I'll never augment my mind or body" or "Maybe we'll return to the good old days" is just that - enpty chatter. There is no turning back from that wondrous, terrifying future and the remarkable element about this whole discussion (as noted in the book) is the incredibly short time it has taken to arrive at this plateau. Starting with simple instinctal behavior, humans began to augment themselves - through culture, rough tools, domesticated animals and then reworking nature itself. The Industrial Age accelerated this push and now, 300 years later, we are on the cusp of an undefinable, magical future.

This is a fascinating read - in fact it should be a must read. We are not in the realm of science fiction any more and anyone who follows the daily "Kurzweil Technology Advancement Board" (my own term) realizes that the rate of advancement is ever-increasing almost daily as one discovery leads to an invention that allows yet another discovery - the wheel turns endlessly. What I especially liked were the serious discussions without exaggeration or wish-fulfillment, conservative time frames and realistic expectations. Implicit to any discussion of AI is an assessment of its past. One of the most informative parts of the whole book was the history of AI which is as much an evolution of ideas as it is technique, methods and machinery. We have gone form hoping AI could mimic humans to expecting it to surpass humankind. We read about both "hard" and "soft" AI, software and then that most important subject - the human qualities of AI.

This last area may prove the most difficult in both design and concept since human opinions are both universal and individualistic and human emotions are nebulous. Those emotions may be - as the fundamentalist Darwinisns insist - simplay a result of trillions of "yes/no" switches but they interplay with our health, location, past events, future expectations, primal urges and societal forces. Our sense of morality derives from distinctly human edifices - culture, society, family and religion. Where would conscious machines learn morality? Will machines know that lying and disguise are a primary component of being human, of existing in the world. The problem is that we cannot conceive of what hyper intelligence thinks or even the dizzying pathways it employs for thinking or even if it "thinks" at all.

There is an unnamed fear, the unspoken elephant in the room. If the Universe is teeming with life as many suggest and if sentience exists elsewhere than Earth, why haven't we seen evidence? Societies millions of years older should be traveling galazies, building wormholes, harnessing the very forces of the universe yet the continual silence and lack of anything remotely intelligent indicates we are the first. The alternative is more frightening - they surpassed their own "humanity" and by doing so, ensured their own extinction. Apparently even the machines vanished as well. It is a lesson we will only recognize on hindsight. My Grade - A