Our Final Invention: Artificial Intelligence and the End of the Human Era (Anglais) Relié – 1 octobre 2013

If you don't know much about real-world AI research and/or you're totally unfamiliar with the nonfiction concerns about the risks it poses, then this book is a quick and easy read that will make you aware of the basics. However, the author is himself clearly non-technical and has a sensationalist style that feels too much like tabloid writing.

When I started reading it, I began bookmarking pages with passages that struck me as problematic. I thought I might write a short review on my wife's tech blog, or perhaps for LessWrong. But as I read further, I realized there were so many problem areas that I'd never bother to sit down and address them individually. Again, these problems would only matter to a technical audience -- experienced programmers, people with a more-than-passing-interest in AI, and so on.

This is my big problem with the book: It's a critically important subject which deserves better treatment than this. Barrat seems to understand the basic problem well enough, but much of the time I had the feeling his primary goal was hitting a page-count target. For example, most of the section about malware is largely irrelevant to the real problem, but it felt like one of the longer chapters in the book (I didn't bother to confirm this, that's just my impression). His TV documentary background shows at the start of each paragraph: each time I felt like I was coming back from a commercial break. He'll shoot somebody down in one chapter, then use that same person to support his argument in the next. He tosses around concepts like cognitive bias and logical fallacies apparently without realizing the book is mostly one big appeal to authority. There is a very good, very important story here waiting to be told. This book only scratches the surface.

I've been a programmer for 36 years. I played around AI-related things back in the late 80s, and I recently became interested in it again. I believe it has great promise, but I do agree that it is also terrifyingly dangerous (in the "existential-threat" sense), and that insufficient attention and respect is being given to the problem. For that reason I'm giving this three stars -- it is a tremendously important subject. If it wasn't for that, I'd probably be one of those "drive-by" one- or two-star "spammers" Barrat likes to rant about in his replies to less-than-fawning reviews.

If you're non-technical, buy it and read it, and don't stop here. If you're a technical type, hit up the LessWrong website as a good jumping-off point for learning more about what is really going on today. Many more technical people need to be thinking about this, concerned about this, and ultimately *doing something* about it.

You are peering inside a black hole at a "point" beyond which you cannot see and where no one knows what exists. The point represents a period of time technologically known as Singularity. Even light cannot escape from the point and on the other side it is known only that there is a profound self replicating intelligence greater than our own, all driven by artificial super intelligence (ASI).
Physicist Stephen Hawking writes that "In contrast with our intellect, computers double their performance every eighteen months. So the danger is real that they could develop intelligence and take over the world".
Computer scientist and professor Vernon Vinge writes that "Within 30 years, we will have the technological means to create super human intelligence. Shortly after the human era will be ended".
Our Final Invention is 267 pages of authoritative manuscript that is compelling, fascinating and beyond the fright stage.
The book's author on numerous occasions refers to "we" as if there exists a unified collective engaged in artificial general intelligence(AGI) or artificial super intelligence (ASI). The reality is that some 56 nations are currently in different stages of arcane artificial intelligence designs. They include antagonists such as North Korea, Iran and suicide regimes from the Middle East. Russia, China and the U.S. are the biggest players as is Israel.
The author believes that super computers fueled by nanotechnology will combine to produce ASI trillions of times more powerful than any human academic or intellectual resources. ASI has the potential to eliminate hunger, poverty, disease and even mortality but disruptions of global economies and politics will be in evidence as balance of powers are shifted. Unemployment dynamics will infect bank tellers, retail clerks,
travel agents, loan officers stock brokers....
Computer software designs are so complex, even incomprehensible, that failures are inevitable. The 1986 Chernobyl meltdown, Three Mile Island, and Fukushima were all designed by highly qualified professionals but with complex infrastructures. Under Singularity as computer speeds double with frequency while human intelligence is unchanged, perhaps the musings of Hawking and Vinge will prove to be prescient.
Our Final Invention is 267 pages of a very dark subject which not even a trace of a happy Betty Grable ending is to be found. My time has expired. Perhaps the final words were well expressed by Jaan Tallin, cofounder of Skype: 'A hard-hitting book about the most important topic of this century and possibly beyond---the issue of whether our species can survive. I wish it was science fiction but I know it's not'!

James Barrat, as a filmmaker and journalist, has layman's style that makes the topic accessible to non-experts. Some reviewers have commented that his lack of subject matter expertise is reflected in the weakness of his arguments. I disagree, Barrat went to great pains to interview many of the big thinkers on AI and its potential impact on society and humanity - including optimistic boosters, skeptics, and doomsday alarmists. While he ends up closer to the alarmists, I feel he got there by the strength of the arguments he heard along the way. If a machine can attain human level intelligence and is able to recursively improve its own code and therefore the rate at which it learns and becomes more intelligent, there are reasons to believe such a machine could exceed our intelligence many times over in a very short period of time. Well just unplug it, you say? The AI already thought of that and stored itself all over the place. It is not a machine anymore, it is an unbounded sentience. If we had it contained on one system completely cut-off from all networks, what strategies might it devise to convince us to connect it to the wider world? Promise a cure for cancer? The solution to clean cheap energy? Reversing climate change? if only it had a little more computing power.... Also consider this - what would a super-intelligence want, what would be its ambition? Just survival? Expansion? As Barrat cleverly puts it, maybe it has different uses for our human molecules than we do. And finally, consider that probably the most aggressive investments in AI are happening largely in the dark, by DARPA and defense contractors in the US and by God knows who in other countries. So why would we assume that the first sentient human-level AI will be friendly? These are the kinds of questions Barrat grapples with, and it is a heck of a ride.

I've read Ray Kurzweil and have worked in technology for 20 years. I see the impact of automation on jobs and how fast we are moving but never really thought of the bad side of AI. This book and one by Nick Bostrom take the other side of our technology achievements. Barrat's book is a quick read from a journalist point of view though he does a lot of research and interviewing of the people in AI. His conclusion is that once we reach Artificial General Intelligent (AGI) machines, the game is over. They will very, very rapidly become superintelligent and when they do, humans will be like ants or mice are to us. He doesn't expect our computer scientists to build in "friendly AI" or really consider it until its too late. I think he has a point. Our history is to race ahead and not worry about naysayers. Think Climate Change. If there is money or power to be gained, risks will be down-played until some major, major disaster. The question is, can man build a self-learning, general intelligent machine? Last year was a big year for AI and machine learning. Corporations are rushing to retool to take advantage of cloud computing, big data and learning algorithms. Just check out how many people are taking classes in data science and machine learning on Udacity, Udemy or Coursera. Google, AT&T, IBM, Amazon... are all investing heavily in AI. Personally, I think Barrat has good reason to be concerned by human hubris.

"Our Final Invention" is a fascinating and well-written look at the risks posed by artificial super-intelligence. As other reviewers have pointed out, this book offers a relatively pessimistic take on the subject, but there is a lot of value in that perspective. There are plenty of other books, by Ray Kurzweil and others, that offer the optimistic viewpoint.

The danger highlighted by the book is that an intelligent machine would turn its energies toward building even better versions of itself--creating an accelerating feedback loop that could culminate in a machine THOUSANDS of times more intelligent than any human. Once such an intelligence "escaped from its box" there would be no way to protect ourselves.

This book focuses entirely on the long term risk of super-intelligence and does not touch at all on the near term consequences of less advanced and more specialized AI. For example, millions of routine jobs will be lost and the economy will be transformed, and this could happen quite soon.

In the longer run, the points raised in Our Final Invention are well worth thinking about. Some experts feel that an advanced AI would be controlled by programming in "friendliness" right from the start. Just as humans have basic drives (food, shelter, sex, etc.) a machine might be programmed to have an essential need to help humanity. As the author points out, however, in humans these basic drives often produce unpleasant and unexpected consequences -- like for example suicide bombers. A truly advanced, alien intelligence might exhibit some qualifies that are not unlike mental illness in humans. A machine might by nature be a sociopath.

As the author says it is naive to think that just because we create a super-intelligent machine, that intelligence will care about us. If you found out you were created by mice, would that make you devote your life to improving the welfare of mice? Questions like this may turn to be among the most important we ever ask...and this book does a good job of presenting them.