Existential risks and uncontrolled digital evolution

[I am not an AI specialist. The book under discussion does not offer any concrete solutions, but rather belongs to the category #speculative journalism and science fiction. As often happens in such cases, the discussions generated by the book are much more interesting and useful than the book itself. This topic has become more widespread in English. the internet. In Runet, the query @ superintelect, @ supermind gives ~ 189 results per month (Yandex)]

"What if we succeed?"
Stuart Russell

I. Apocalypse tomorrow

In his book Superintelligence: Opportunities, Risks, Strategies Nick Bostrom draws a pessimistic scenario of the development of artificial intelligence: as soon as an AI appears equal to human intelligence, superintelligence will be one of his first inventions. And then "the violinist is not needed", super-intelligent information systems can self-educate and reproduce even more complex systems, control the production and distribution of competing AI, indulge in an aggressive Malthusian race with other intelligent agents in the bosom of Artificial Nature.

Existential risks from the neighborhood of the supermind and man grow manyfold. Nick Bostrom warns that the superintelligence is able to learn faster and more efficiently of its creators and, if you don’t inculcate human values ​​in it, then everything can end badly, because superintelligence will not only be able to regulate the further production of smart machines, it will successfully prevent people from trying to correct his work or interfere with it.

Autobiographical retreat: Nick Bostrom among scientists and philosophers is known as a horror lover - he heads the institute, which is engaged in the search and publication of existential risks, he has such a job. Some colleagues even call him Donald Trump from AI, Nick Bostrom is offended, but quickly forgives. The harsh world of scientific debate. The end of the autobiographical retreat.

Ii. What do experts think?

Already, the AI ​​has "limited rationality" - it can choose the optimal strategy for achieving results. What will happen if he learns to change them and change the hierarchy of his goals?

As the main argument in support of their fears, Bostrom cites the desire for optimization. If the superintelligent needs to solve the Riemann theorem and for this he needs to convert half of living beings into computers, he will do it without hesitation, Bostrom asserts. If the superintelligence will need to cure cancer, then he will do it, even if it needs to destroy all people who do not have cancer. The author puts the problem of control of super-intelligence in the first place in the development of AI. Among scientists, such a scenario is called paradigms of bostrom in the 2000s. And recent advances in machine learning have drawn attention to this issue even by techno-oligarchs and businessmen, who sounded the alarm and frankly expressed concerns about the development of AI.

Ramez Naam doubts that the opinions of B.Gates, I.Maska ("super-intelligence is a new atomic bomb"), S. Hawking on the risks of AI have any weight. "#Crive the need. None of them worked in this area."He cites the opinions of prominent AI / machine learning experts:

  • Michael Litman, a professor at Brown University and a former employee of the AI ​​Association: “Such fears are simply unrealistic. The AI ​​cannot just wake up and make an apocalypse.”
  • Jan LeKun, Facebook Specialist on Neural Networks: "Some people think that the AI ​​will be able to program itself one day. Usually such assumptions are not made by specialists."
  • Andrew Eun, Curser / Google / Baidu Machine Learning Specialist: "Yes, computers are getting smarter. But this is only the mind, not the mind and self-consciousness, and most experts do not believe that this way you can create a conscious AI. Worrying about the dangers of superintelligence is just the same as worrying about overpopulation of mars."
  • Other, no less serious experts believe otherwise:
  • Richard Sutton, a professor of computer science at the University of Albert, in his speech at a conference on secure AI said that: "there is a certain probability of occurrence of a person's AI level" and then "we will be forced to cooperate with him", but "from a reasonable slave the AI ​​can become a reasonable contender."
  • Jürgen Schmidhuber, a professor of AI at the University of Lugano and a former professor of cognitive science of robots at the University of Munich, states that: "if the pace of development of science remains at the same level as before, then in the coming decades we can expect a real breakthrough in the application of AI." On the reddite, he noticed that: “At first glance, recursive self-enhancement in Gödel’s machines can help develop superintelligence. Gödel machines can select and make changes in their code that will likely bring positive results according to the initial set of features. That is, we can specify the initial“ correct ” the parameters and the machine will develop them themselves. Other researchers can ask their machines a different set of parameters. And the machine race of evolutions will begin. It is impossible to predict who will win this race and what it will bring to us. More about that."
  • Murray Shenehen, a professor of computer science at Cambridge, in his book "Technological singularity" writes that: "From creating a person's AI level (theoretically possible, but very difficult) to create a superintelligence, it can take quite a bit of time and this breakthrough can be compared to the point of singularity, where events will develop faster than we assume. Over-intelligence superior to human capabilities will be associated with increased existential risks and increased benefits. "

Alan Turing, who needs no introduction, in his work "Is a digital machine capable of thinking?" writes: "Suppose we manage to create thinking machines. This will produce a resonance and will meet resistance from intellectuals and scientists who will be worried about their position and work. And for good reason. Because when machines reach the human level of thinking, people will have to try very hard in order to keep up with machines in reasonable activity. It is also quite likely that a reasonable machine’s ability to create its own similarity or an even more intelligent machine. Thus, it can be expected that once a reasonable machine can be reasonable under the control of mankind. "

Iii. Why artificial intelligence?

Because the human brain is the closest analogue of what happens in machine learning: information systems that undergo machine learning are called neural networks and consist of separate layers, each of which is responsible for processing certain information. If the layers of the lower levels can receive information, then the layers of the upper levels can process, transmit and use it for their own training. Very similar to the self-learning machine in the human head.

Neural networks are already able to distinguish between human speech and faces better than humans. This is not as harmless as it seems at first glance. We are surprised at the victory of a computer over a person in chess or go, although a person does not have special sections of the cerebral cortex for solving chess problems. But recognition and categorization of faces is already a special zone of the cortex that has evolved evolutionarily and is responsible for survival. And in this area, AI is better than man today.

So, AI is able to abstractly and categorically comprehend educational information: to distinguish between images of white dogs, black dogs, spaniels, dachshunds, puppies, adult dogs and to attribute all this to the "dog." Class. AI can create and label pornographic motifs in images. (Beware: real digital pornography in terms of AI). These capabilities of AI (categorization / generalization / classification) indicate the simplest reasonable activity of an information system.

What's next?

Iv. Compare the two approaches:

1. Mechanistic: classification / categorization is certainly one of the signs of a reasonable agent. But only one of. This is a useful and wonderful feature, but not the main one. True intelligence requires more than the ability to click puzzles — the desire to click puzzles. Distinguishing between images of dogs and calling it a dog is cool, no doubt. The present AI must be primarily inherent in the desire, motivational drive to solve problems and prove theorems. And the creation of a rational wishing information system is more than the ability to perform ultrafast and efficiently even very complex tasks as instructed by the engineer of this system. And the desires of a wishing intelligent machine may differ from the goals of a human creator. This approach is cute Nick Bostrom.

2. Biological (caution: crude simplification): most of the brain consists of the same or slightly different cells that have evolved during evolution. If we can understand how one part of the brain works, then it will be much easier to understand how other areas work. Motivational processes of human rational activity take place in the same or similar areas of the cerebral cortex as perception / categorization / classification information. If scientists believe that information systems for perception-categorization-classification can be reproduced using the example of the human brain, then to reproduce motivational intelligent systems, scientists will rather change the existing neural network samples for categorization / classification. Such systems will not look like totally motivated and goal-focused prototypes (the human brain is also a little like such systems). Most likely, it will turn out a lot of small from diverse perceptions, desires and motivations, as it usually happens in human heads. Such an approach calls into question the total power of the creator-man over AI, because the programmed motives / goals of the machine may not arise or will be significantly changed.

Biological retreat. The evolutionary history of the biological approach is difficult to trace and confirm with great accuracy. The separation between cognitive perception / processing of sensory information (prefrontal cortex) and memory / emotion / behavior centers (hypothalamus and hippocampus) occurred in the early stages of the development of vertebrates or even earlier. However, these different systems have retained connections and interfere with each other’s functionality. Some parts of the cortex are tied to the hypothalamus and belong to the limbic system. Other parts of the thalamus (lizard brain, limbic system) are involved in information processing (colliculi).

It seems that brain areas responsible for sensory perception and motivational-emotional zones are present in different parts of the cortex. Moreover, the pre-frontal cortex, which is considered the cradle of the human personality (higher cognitive activity, evaluation, planning, etc.), probably evolved from the cortex of the first fish, from the single brain of these fish. Which, in turn, originated from those several hundred neurons that were responsible for the perception and processing of sensory information in the first worms. The end of the biological retreat.

Neither anatomy nor evolution implies a purely autonomous separation between perception, processing and motivation of reasonable activity. The same systems can perceive information in one place and, with minor changes, process it and motivate further activity in other parts of the brain. All one. At the risk of plunging into metaphysics and mysticism, let's just say that cognitive distortions (information processing) and visual illusions (perceptions) have more in common than previously thought (another simplification: these distortions occur in technically identical fields, Brodmann fields).

V. On the origin of species

Categorization / classification is the basis of moral education. Perhaps this is morality. Take for example the mental categories: everyone knows what a bird is, but with the exact definition of a "bird" a little more difficult (stork is a bird, penguin is a strange bird, Archeopteryx ... a semi-bird?).

Everyone remembers how these categories arise. A bit of learning, the mother shows the child a chicken and says "bird", after a couple of years the child learns that the bat is "not a bird" and that your brain freely operates with categories / classes and complex abstractions, and after another two thousand years Darwin arrives and says that there is such a class of vertebrate "birds", which is very good, but you knew it without him.

People learn ethics and morality in a similar way. When we bite someone, mother / father / teacher / rabbi / priest / shaman tells us that this is "bad"; when we share a cake, we are told that this is "good." Based on work with such examples, the information system operates with categories and classes even if it cannot provide precise definitions of these categories and classes.

Artificial intelligence can develop its artificial morality based on the data that scientists provide it. Morality occurs during AI training on educational information. Bostrom believes that the morality / ethics of AI is separate module or array of training information.

The usual Bostromovsky counterargument is the likelihood that the AI ​​will make wrong conclusions from the training information and develop a system of values ​​that is dangerous for people. For example, an AI might decide that all the things that make a person happy are good. And he will begin to optimize to the maximum - he will begin to make heroin injections for the person all the time so that the person remains happy.

I do not know why, but the human assessment of actions does not work so straightforwardly, and in rare cases brings some idea to a critical mass. Maybe this is due to the peculiarities of classification / categorization or extraneous factors are involved in the final decision, and not only a good-bad dichotomy. This can be an interesting task for AI developers - to determine what exactly makes information systems classify information like a human being *.

In general, it is difficult to imagine that a person has a specific moral gene, a moral segment of the cerebral cortex, some kind of moral adaptation. And it is unlikely that ethics and morality are biological, evolutionary constructs, neurobiological mechanisms - most likely ethics and morality are the necessary conditions for prosocial behavior. Ethics is like a roof of a house, which naturally grows due to the correct work of lower divisions, teaching artificial intelligence to ethics and morality is impossible apart from ordinary learning (if we are talking about AI based on neural networks that repeat brain structures).

The motivational sphere of a person works on dopamines and the reward system. It remains to hope that the closest human ability to classify can be implemented in AI models. People may want sex without the desire to fertilize all living and nonliving until the planet explodes. AI based on the human model of perception and data classification, will to want solve problems without manic desire to subdue all other goals for this desire. Not always and not all people are thoughtless optimizers: hierarchical goal setting is common for people; everyone understands that being moral is beneficial in the long run. That means that superintelligence will most likely not become a terminator optimizer fatal for humanity.

The most common assumption (which is supported by Nick Bostrom): artificial intelligence will be the embodiment of pure logic, completely devoid of similarities of emotions / doubt / hesitation, and we will have to restrain this mathematical golem **. If we recall such examples from the human sample, then these fears become a bit exaggerated - most people who have low empathy, who have bad ethics and morality, demonstrate antisocial behavior, it is difficult for them to learn, to adapt, simply because it is difficult for them to understand other people (autistic, people with schizoid disorders), even if they can extract the roots of five-digit numbers. It is hard for such people not to plan the killing of a mosquito, they hardly endure the sensations of a new shoe on their feet, what kind of world conquest to talk about? There is already an obvious connection with the processes of mentalization, which speaks of the inseparability of intelligence from ethics. It seems to me that another scenario, where the AI ​​will think not only by mathematical formulas / program code, is also possible. It will be much more complicated and not as scary as Bostrom suggests. And then people will need other strategies for coexistence with AI.


* On March 4, 2016, the MIRI Machine Learning Institute program "value alignment for advanced machine learning systems" (goal setting for complex machine learning systems). The task of this program is the creation of "infra-human AI", information systems with limited autonomy.

** Formal logic, the expert on which is Nick Bostrom, does not always adequately describe reality (or does not seek). Так что эти опасения немного преувеличены и логико-математический терминатор может оказаться не таким опасным, как видится Бострому.


About the author of the book under review: Swedish philosopher Nick Bostrom, author of over 200 publications on global risks, sectoralism, anthropic principle, scientific ethics Director of the Institute for the Future of Humanity, Professor at Oxford University. Winner of degrees in mathematical logic, physics, computational neuroscience, philosophy.

Original: Bostromian paradigm and recent AI progress

Watch the video: 65. The Real Dangers of AI. THUNK (September 2019).

Leave Your Comment