ad-free
Channel / Source:
TEDx Talks
Published: 2013-12-06
Source: https://www.youtube.com/watch?v=CK5w3wh4G-M
world I'm gonna talk about my research on the long term future of artificial intelligence and in particular I want to tell you about a very important phenomenon called intelligence explosion there are two reasons that I work on intelligence explosion and that I think it's worth sharing the first is that it's a phenomenon of immense theoretical interest for those who want to understand intelligence on a fundamental
level the second reason is practical to do with the effects that intelligence explosion could have depending on the conditions under which an intelligence explosion could arise and on the dynamics that it exhibits it could mean that a I changes very rapidly from a safe technology relatively easy to handle to a volatile technology that's difficult to handle safely and in order to navigate this hazard we need
to understand intelligence explosion intelligence explosion is a theoretical phenomenon in that sense it's a bit like a hypothetical particle in particle physics there are arguments that explain why it should exist but we haven't been able to experimentally confirm it yet nevertheless the thought experiment that explains what intelligence explosion would look like is relatively simple because like that's suppose we had a machine that was much more
capable than today's computers this machine given a task could form hypotheses from observations use those hypotheses to make plans execute the plans and observe the outcomes relative to the task and do it all efficiently within a reasonable amount of time this kind of machine could be given science and engineering tasks to do on its own autonomously and this is the key step in the thought experiment
this machine could even be tasked with performing a I research designing faster better machines what's their machine goes to work and after awhile produces blueprints for a second generation of a I that's more efficient more capable and more general than the first this second generation can be tossed once again with designing improved machines leading to a third generation a fourth a fifth and so an outside
observer would see a very large and very rapid increase in the abilities of these machines and it's this large rapid increase that we call intelligence explosion know if it's the case that in order to undergo an intelligence explosion many new pieces of heart hardware need to be built or new manufacturing tech then an explosion will be more slow although still quite fast by historical standards however
looking at the history of algorithmic improvement it turns out that just as much improvement tends to come from new software as from new hardware this is true in areas like physics simulation game playing image recognition and many parts of machine learning what this means is that are outside observer may not see physical changes in the machines that are undergoing an intelligence explosion they may just see
a series of programs writing successively more capable programs it stands to reason that this process could give rise to programs that are much more capable at any number of intellectual task then any humans just as we now build machines that are much stronger and faster and more precise and all kinds of physical tasks it's certainly possible to build machines that are more efficient at intellectual tasks
the human brain is not at the upper end of computational efficiency and it goes further than this there's no particular reason to define our scale by the abilities of a single human or a single brain the largest thermonuclear bomb this release more energy in less than a second then the human population of earth dies in a day it's not out of the question to think that
machines designed to perform intellectual tasks and then honed over many generations of proven could similarly outperform the productive thinking of the human race this is the theoretical phenomenon called intelligence explosion we don't have a good theory of intelligence explosion yet but there's reason to think that it could happen at software speed and could reach a level of capability that's far greater than any human or group
of humans at any number of intellectual tasks the first time I encountered this argument I more or less ignored it looking back it seems crazy for me someone who takes a I seriously to walk away from intelligence explosion and I'll give you two reasons for that the first reason is a theorists reason theorists should be interested in the large scale features of their field in contours
of their phenomena of choice as determined by the fundamental forces I or interactions are building blocks of their subject as someone who aspires to be a good theorist of intelligence I can't in good faith ignore intelligence explosion as a major feature of many simple straightforward theories of intelligence what intelligence explosion means is that intelligence improvement is not uniform there's a threshold below which improvements tend to
Peter out but above that threshold intelligence grows like compound interest increasing more and more this threshold would have to emerge from any successful theory of intelligence the way phase transitions emerged from thermodynamics intelligence would effectively have a boiling point seen this way exploring intelligence explosion is exactly the kind of thing a theorist wants to do especially in a field like a I where we're trying to
move from our current state %HESITATION partial theories pseudo theories arguments and thought experiments toward a fully fledged predictive theory of intelligence this is the intelligence explosion in its most basic form it relies on a simple premise that AI research is not so different from other intellectual tasks but can be performed by machine we don't have a good understanding yet but there's reason to think they could
it could happen that software speed and reach levels of capability far exceeding any human or group of humans the second reason which I alluded to at the start of the talk is that intelligence explosion could change a I very suddenly from being a benign technology to being a volatile technology that requires significant thought into safety before use or even develop today's a I by contrast is
not called I don't mean that AI systems can cause harm weaponization of A. I. is on going and accidental harms can arise from unanticipated systemic effects or from faulty assumptions but on the whole these sorts of harm should be manageable today's a I is not so different from today's other technology intelligence explosion however highlights an important fact a I will become more general more capable and
more efficient perhaps very quickly and could become more so than any human or group of humans this kind of a I will require a radically different approach to be used safely and small incidents could plausibly escalate to cause large amounts of to understand how a I could be hazardous let's consider an analogy to micro organisms they're two traits that make microorganisms more difficult to handle safely
than say a simple talks microorganisms are goal oriented and they're what I'm going to call chain reactive goal oriented means that a microorganisms behaviors tend to push toward some certain result in their case that's more copies of themselves chain reactive means that we don't expect a group of microorganisms to stay put we expect their zone of influence to grow and we expect their population to spread
hazards can arise because microorganisms values don't often align well with human goals and values I don't have particular use for an infinite number of clones of this guy chain reactivity can make this problem worse since small releases of a micro organism can balloon into large population spanning pandemics very advanced AI such as could arise from intelligence explosion could be quite similar in some ways to a
microorganism most AI systems are task oriented they're designed by humans tactic to to complete a task capable a eyes will use many different kinds of actions and many types of plans to accomplish their task and flexible a eyes will be able to learn to thrive that is to make accurate predictions and effective plans in a wide variety of environments since a eyes will act to accomplish
their task as well as possible they will also be chain reactive they'll have use for more resources they'll want to improve themselves spread to other computer systems to make backup copies of themselves in order to make sure that their task gets done because of their task orientation and chain reactivity sharing an environment with this kind of a I would be hazardous they may use some of
the things we care about our raw materials and our stuff to accomplish their ends and there's no task that has been yet been devised that is compatible with human safety under these circumstances this hazard is made worse by intelligence explosion in which very volatile a I could arise quickly from benign and I instead of a gradual learning period in which we come to terms with the
power of very efficient and I we could be thrust suddenly into a world where I am a I is much more powerful than it is today this scenario is not inevitable it's mostly dependent upon some research group or company or government walking into intelligence explosion blindly if we can understand intelligence explosion and if we have sufficient will and self control as a society then we should
be able to avoid an a I outbreak there's still the problem of chain reactivity though it would only take one group to release a I into the world even if nearly all groups are careful one group walking into intelligence explosion accidentally or on purpose without taking proper precautions could release in a I that was self improve and cause immense amounts of harm to everyone else I'd
like to close with four questions these are questions that I'd like to see answered because they'll tell us more about the theory of artificial intelligence and that theory is what will let us understand intelligence explosion well enough to mitigate the risks that it poses some of these questions are being actively pursued by researchers at my home institution the future of humanity institute at Oxford and by
others like the machine intelligence research institute my first question is can we get a precise predictive theory of intelligence explosion of what happens when a I starts to do a I research in particular I'd like to know how fast software can improve in its intellectual capabilities many of the most volatile scenarios we've examined include a rapid self contained takeoff such as could only happen under software
improvements circumstance if there's some key resource that limits software improvement or if it's the case that such improvement is impossible below a certain threshold of capability of these would be very useful facts from a state safety stand question to what are our options political our technological for dealing with the potential harms from super efficient artificial intelligence one option of course is to not build them in
the first place but this would require exceedingly good cooperation between many governments commercial entities and even research groups and that copper cooperation and that level of understanding isn't easy to come by it would also depend to some extent on an answer to question one so that we know how to prevent intelligence explosion another option would be to make sure that everyone knows how to devise safe
tasks it's intuitively plausible that there some kinds of tasks that can be assigned by safety conscious team without causing too much risk it's another question entirely how would these kinds of safety standards could be implied applied uniformly and reliably enough all over the world to prevent serious harm this leads into question three very capable allies if they can be programmed correctly should be able to determine
what is valuable by modeling human preferences and philosophical arguments is it possible to assign the task of learning what is valuable and then acting to pursue that and this turns out to be a highly technical problem some of the groundwork has been laid by researchers like Eliezer you'd kowski Nick Bostrom Paul Cristiano and myself but we still have a long way to go my final question
as a machine self improves it may make mistakes even if the first AI is programmed to pursue valuable ends later ones may not be designing a stable and reliable self improvement process turns out to involve some open problems in logic and in decision theory these problems are being actively pursued at research workshops held by the machine intelligence research institute those are my four questions I've only
been able to cover the basics in this talk I feel like to know more about the long term future of A. I. about the intelligence explosion I can recommend David Chalmers is excellent paper the singularity of philosophical analysis as well as a book forthcoming in twenty fourteen called super intelligence by Nick Bostrom and of course there are links and references on my website I believe that
