Think Tank To Study The Risk Of A Genocidal Robot Uprising

By on November 28, 2012 in News 8

Good to know that we may finally have an answer on this. The BBC reports:

Cambridge researchers at the Centre for the Study of Existential Risk (CSER) are to assess whether technology could end up destroying human civilisation. The scientists said that to dismiss concerns of a potential robot uprising would be “dangerous”.

Fears that machines may take over have been central to the plot of some of the most popular science fiction films. But despite being the subject of far-fetched fantasy, researchers said the concept of machines outsmarting us demanded mature attention. “The seriousness of these risks is difficult to assess, but that in itself seems a cause for concern, given how much is at stake,” the researchers write.

The CSER project has been co-founded by Cambridge philosophy professor Huw Price, cosmology and astrophysics professor Martin Rees and Skype co-founder Jaan Tallinn. Prof. Price said that as robots and computers become smarter than humans, we could find ourselves at the mercy of “machines that are not malicious, but machines whose interests don’t include us”.

, , , , , , ,

  • Anarchy Pony

    Are these sods actually getting paid for this? I’m in the wrong line of work.

  • marssuit

    Here’s your answer. Artificially intelligent robots commiting genocide is NOT NEARLY AS LIKELY as a genocidal scenerio resulting from the Singularity. Humans who cannot die because of technology are more likely to view normal humans as lesser beings whos larger populations may be seen as needing containment or extinction. I would worry more about human robots than robot robots.

    • alizardx

      Human enhancements in terms of life-extension technology or neural implant mental enhancement are going to be so expensive that they’re going to be restricted to elites-only regardless of what Transhumanists or Singulatarians think.

      Anyone who knows the cost of medicine and developing FDA-approved biological implants will have no trouble with my guesses as to what early generation “SmarterHUman” tech is going to cost when it’s COTS (Common Off The Shelf – A Thing One Can Buy)… think $250,000 implant hardware and $1,000,000 neurosurgical sessions + ICU stays for installation… and I strongly recommend buying the $200K/year “extended warranty” package. While the hardware cost can drop in time given adequate demand, neurosurgical costs don’t scale much. What will ordinary mortals be able to afford? Places like Google might have health plans covering neural interface to your smartphone based on increased employee efficiency.

      Even if the cost quoted above is off by a factor of 5 and the average cost is only $250K, this is still way above what the current health care delivery system can provide the public for a “medically unnecessary” “experimental” procedure. Though it’s more likely that the costs will drop to this in time.

      Will Obamacare pay for this? Will even a good corporate health plan cover this? Access to this is going to be elites-only and top researchers for which buying this kind of tech is likely to enhance corporate profit if the “free” market is left to itself. IOW, this is going to be restricted to billionaires and their kids, their top-level enablers like Romney, and the few individuals on their payrolls for which these procedures might enhance corporate profits. And a few sort-of-rich people willing to mortgage and sell off all their assets to make it possible for their kids to compete. (the kind of people who now spend tens of thousands of dollars for tutoring programs to get their kids into the right pre-schools)

      To provide access to this for everyone who wants it, a national health care program in the US comparable to the Canadian program is necessary with coverage for neural enhancement, and initial academic research into the technology has to be taxpayer-funded. When demand is in the hundreds of millions, economies of scale for the devices kick in and it might be possible to roboticize these specific procedures to reduce the cost of high-priced medical labor. What we’d get out of this as a nation? I suspect that a nation whose citizens are literally smarter than everyone else’s might have certain competitive advantages. How can this be paid for?

      Precondition to solution? “Government is the ultimate crowdfunding mechanism”. If the billionaire class is willing to accept post-WWII marginal tax rates, maybe these problems can be solved. So where is the Transhuman / Singularity political agenda telling us We Must Do This? LOL. Funding for this movement comes from Silicon Valley tech money whose real futurist vision seems to be “No New Taxes” http://tinyurl.com/bccfpe4 and using the scene for self-glorification to create somewhat inaccurate PR images of themselves as benevolent visionary philosopher-kings.

      Futurist Model of The Future:
      1. cool science (largely slated for budget axe)
      2. cool technology (from the majority of VC-funded startups which will fail)
      3. “FREE” MARKET!!!
      4. “FREE” MARKET!!!
      5. ?
      6. ?
      7. Nerdvana with nerdgasms for all, even for people who don’t want them.

      So it isn’t surprising that what we hear from them comes down to mindless cheerleading for science at the press release stage and tech at the startup-funding stage (most VC-funded startups fail) and technology-based capitalism and capitalists. And nothing about the models that describe how all this wonderful technology and science (one of the expected casualties in the budget debates in progress – lots of panicking post-docs) is going to get to us.

      You can buy an iPhone 10 in The Future when it becomes available at a retail store. Life extension and neural enhancement implants are something that only the wealthy will be able to buy from major medical institutions. I like science and technology, too, but just because there’s an article about it at io9 doesn’t mean any mere mortals will ever be able to afford it or that it will ever happen.

      Issues involved with mass-access life-extension tech get even uglier. Imagine a world where the average lifespan is 150 years and average consumption is at EU level. This means that ALL of the problems that threaten the existence of human technological civilization have to be solved, resources, wealth differentials (imagine being an American where 150 year lifespan is a citizenship right going into the Third World – think these people hate us *now*?), transition to renewables.

      Being able to live 150 years is useless if one doesn’t have a habitable planet to live on or food is unavailable at ANY price. The solutions to these problems are political, not technological, problem is being willing to pay for the scientific and technological developments to deal with them. This means HIGHER TAXES.

      Where is the enhanced profit for the superwealthy funding this movement coming from if this movement achieved its announced goals? The result would be much larger numbers of people competing for them with “their” resources and having increased brainpower to compete with. in the “apolitical” ideological framework for this movement, the “free” market is supposed to provide all these expensive things with the money to pay for them materializing out of thin air. Is there some reason why we are supposed to take this seriously?

      Given this, it’s really surprising that even small advocacy movements for this sort of thing exist in the USA regardless of the amount of astroturf in their DNA. Anyone who doesn’t expect to have $100M in the bank by the time these become Things You Can Buy is going to be on the outside looking in regardless of their fervent beliefs if the centrist upward transfer of wealth ideology of the Transhumanist / Singularity movement’s major financial backers continues to dominate the political process.

      The people leading the advocacy movements are NOT going to be of sufficient value to the people paying the bills for these movements to motivate the funders to want to keep them alive or make them smarter.

      Ironic that the most deeply corrupted part of the hipster “counterculture” is the futurist movement which is their leading edge.

      The SMI2E ideas of Timothy Leary and Robert Anton Wilson from which they ultimately derived were NOT funded by the corporate wealthy to begin with. Those ideas in part were what got Leary sent to prison.

      What changed? The parts of their messages compatible with putting the minds of creatives to sleep were kept, the part about understanding the world around one to make it possible to understand the future “What the hell is really going on?” mysteriously disappeared.

      While I would not presume to speak for a dead Robert Anton Wilson, I think if he were alive and functioning today, he would denounce the modern futurist movement the way Barry Goldwater denounced modern conservativism towards the end of his own life, and for the same reasons. Both conservativism and futurism got driven off the deep end by the money behind them.

      I’d call myself a paleo-futurist in the same way people who believe in Goldwater’s actual ideas call themselves paleo-conservatives. Freethinker works equally well, I suppose. Heretic is a matter of viewpoint.

      Amazing that the pocket universe containing self-referential “think” tanks and blogs and twitter feeds and institutions with a top-down driven message repeated by so many well-meaning and sincere people looks so much like a smaller version of that of the conservative movement. Stay inside the echo chamber and never see unpleasant reality until it sees you.

      But since it, too was bought and paid for by wealthy people with an existing model to work from, how could it look like anything else? Not a lot of original thinking, good journalism, or good scholarship in either place. Why is there almost never critical thinking about new tech or new tech companies or the words of their superrich backers on their sites? Why do descriptions of interesting new tech look like slightly edited press releases?

      The implicit assumption seems to be “In The Future, Murphy’s Law will be repealed.”

      Not cool to ask unpleasant questions inside this narrow unreality tunnel where the high-tech side of the free market will magically solve all of our future problems and create wealth and abundance for everyone, not just the investors. Anyone who’s worked in a high-tech startup knows that the idea behind funding a startup is enriching the investors, founders who get rich along with them if it succeeds are both smart and lucky. Public interest? You’re kidding, right?

      Amanda Palmer “You can’t fuck the system and expect the system to crown you” – a lesson Transhumanists and Singulatarians internalized as they got assimilated. Next stage of assimilation: Thiel’s Singularity University is buying up Singularity properties and events he didn’t already own. If you own one and get a chance to sell out, take the money and run.

      For bonus points, which Futurist alt-culture event is sponsored by the ultra-right wing pro-fossil fuel / pro-global warming / anti-science / anti-tax / pro-government subsidies (for themselves) Koch Brothers?

      There is no “The Future” no matter what Futurists think. Just a sheaf of probability futures ALL of which come from the present, like the “fan-shaped destiny” discussed in Illuminatus. The high-probability futures driven by the “no new taxes” agenda that bought modern Futurism all look like grim meathook future followed by equal-opportunity dieoff. Other futures are possible, but modern Futurists are irrelevant to them.

      Those who want to see reality can look at THIS: http://t.co/FEMJ43ko Not all the news is bad: Exit economics: The relatively low cost of Germany’s nuclear phase-out http://bos.sagepub.com/content/68/6/42.full – but what does this real future news have to do with futurism? The first thing a person who wants to see what kind of future can come out of the present must be to walk out of the futurist unreality tunnel and look around at the NOW.

      To get what they claim to want will require them to think about the real world, how to get from where they are to where they Will to be. They are afraid to think outside their corporate-funded box.

      I believe that people should be smarter and live longer. I can’t be part of a movement that claims to believe in that and whose spokespeople are servants providing PR for wealthy people who want the opposite for humanity.

      Not surprising young people aren’t buying into this. They’re too busy trying to survive in the real world where they can’t even aspire to become a well-paid creative unless they have parents rich enough to buy them a place in that world via subsidizing unpaid internships to buy into fantasies which will improve neither their present nor their and our future. Minority groups have even less reason to buy into this fantasy future. The hipsters at home in this movement have already got theirs and they expect their entitlements to continue (I think they’re wrong about that.).

      The main questions of Transhumanism, not that anyone in the scene is interested in asking them, are about public policy, not technology. The basic research and technological development required will happen when somebody is willing to pay for them. Enough money to make mass access to biotech enhancements possible is a society-level priority.

      Transhumanism without a coherent political agenda to achieve its goals is a fantasy LARP.

      Without answers to the political questions and coherent plans to deal with them, the US Transhumanist / Singularity unreality tunnel leads into a brick wall. The Russian Transhumanist movement is just starting to figure this out and talking about political organizing.

      The idea that billionaires can have longer lives and neural enhancements the rest of us can’t get needs no advocacy, just people willing to write checks to cover the research if they can’t get academic institutions supported by taxpayers to pay for it.

      Your concerns are not unreasonable, enhanced intelligence in the service of short-term greed I discuss here could be extremely dangerous to EVERYONE. The people who will be able to buy the enhancements and the money that’s going to pay for their development can be described accurately this way. And the enhanced elite and especially, their kids will be part of EVERYONE. If you’ve read Jared Diamond’s “Collapse”, you know what it means when a society-controlling elite collectively screws up. Lemming scenario with the elite driving everybody into the sea drowning last. Or parasites more efficiently killing their hosts.

  • alizardx

    Based on the latest IBM “whole human brain” simulation platform which runs at 1/1532 of human computational speed using an entire data center full of state-of-the-art custom 17 core CPUs based on 45 nm fab tech… we are around 20 years based on Moore’s Law projections from having a human-equivalent AI platform. While the actual projection is longer than that, I am assuming speed increases due to network architecture tweaks and higher internal bandwidth as well as straight-up number-crunching speed.

    Discussions of “sentient AI beings that want to kill all humanity” are at the same level as discussions of “the future of aviation” circa 1880, i.e. before power plants with sufficient horsepower to weight existed to drive heavier-than-air airplanes. That’s why all those discussions back in the 1880s talked about bigger and better lighter-than-air airships. For how accurate those projections were, go to your nearest airport and count the dirigibles.

    Taking worry about killer-AI seriously demonstrates that one does not have a clue as to what one is talking about, watching science fiction is no substitute for knowing computer hardware state of the art and what advances it in the real world. Anyone who doesn’t know what Moore’s Law is without looking it up has no business disputing whether I’m right or wrong. Also note that once one digs into the article, they talk about “this or the next century”.

    Discussions of this are decidedly premature. Until we actually have sentient AIs running at even hamster level, we simply are not going to know enough to have the remotest clue as to what questions we should be asking. In fact,. these discussions could be potentially harmful.

    Imagine a future where there’s an academic establishment whose members get their rice bowls filled according to their agreement with the consensus as to “What Killer AI Must Be” based on theories that don’t match where the software actually went that gets collectively blindsided when it appears. Hopefully, those “killer AIs” will do us a favor and clean out that academic community first.

    “It is a capital mistake to theorize before you have all the evidence. It biases the judgment.”
    Sherlock Holmes

    We don’t have ANY of the evidence.

    I don’t even have any particular doubt that there will be “killer AIs” sooner or later once sentient machines are achieved. People have been trying to figure out how to program developing human beings in a way which will induce them to never commit murder (as in killings not authorized by “the state”) for a very long time, without notable success.

    But I see neural acceleration of human beings as a parallel development, and “killer AIs” will exist in a sea of drastically enhanced human beings coming from an evolutionary line which has a million-year head start in staying alive in hostile environments. That sea will be full of predators. Anyone who’s dealt with venture capitalists knows what I mean, I suspect that we will also see developments in networked human consciousnesses once hooking minds up reliably is simply a matter of plugging in the right network cables. My prediction for future “killer AIs” is that they will exist very, very briefly. Before they get shut down, isolated, and disassembled to figure out what went wrong.

    These artificial consciousnesses are not going to spring fully grown into the world (argue this when you can show me a consciousness appearing instantly out of nowhere), they can be reasonably expected to go through developmental stages just like any other conscious being does. An AI toddler that goes around breaking shit is unlikely to be allowed to grow up to be released into the wild. Especially if it was created under the auspices of a profit-making corporation that has to worry about things like legal liability.

    if we’re still around in 2030 or 2050 (just because the hardware can do it doesn’t mean that the software can be written to support it), argue with me then as to whether I’m right or CSER is right. While my speculation about killer AI vs accelerated human consciousness (I don’t speculate on the nature of that consciousness other than that it’ll process data a lot faster) is also premature, I think it at least has some sort of rational basis.

  • http://www.facebook.com/profile.php?id=742104313 Adam Goodwin

    CSER job qualifications: compulsive worrywarting, hypochondria, existential angst, perpetual pessimism, and low self-esteem..

    • alizardx

      and getting one’s tech ideas from Hollywood and not experience in computer electronics or commercial software development. Modern futurists en masse forget that the future is a nonlinear outgrowth of the present and what’s in it is based on what the big money wants to pay for. The line of thinking that’s led to modern futurism is largely derived from Robert Anton Wilson’s ideas.

      That the future comes out of the present is something RAW NEVER forgot. That’s why so much of his work was based on digging further into “what the hell is really going on” than the mass media of his time wanted to tell us. And why he was also known as a “conspiracy theorist”.

      Strange that this mindset never survived into futurism as we know it today.

  • IG

    I’m not really worried about the robots. We’ll run out of useful, plentiful, easy to acquire energy long before most of that comes to pass. There’s no time for our machines to become self aware.

    • alizardx

      Politics, not technology. The top 0.001% owns a big chunk of the world;s fossil fuel inventory. Transition to renewables would result in a massive writedown of their personal net worth. So the politicians they own at most, talk platitudes instead of phased closing of coal-fired power plants.