The rise of machines: Has technology evolved beyond human control?

Your Ads Here

Netease Technology News, June 21, the "Guardian" website wrote an article today detailing the rise of machines. Technology is beginning to operate in smart and unpredictable ways that even its creators can't predict. As machines increasingly change global events, how can we regain control of them?

The following is the main content of the article:

The smart speaker in the corner of your bedroom suddenly laughed wildly and sent a recording of your pillow whispers to a colleague. Your toddler watched the cartoon "Peppa Pig" on YouTube, but unexpectedly saw the blood and death scenes. The social network you used to keep in touch with your old classmates was found to affect election results and was used to incite a coup.

There has been a strange change in our way of thinking-therefore, stranger things have happened in this world. We began to believe that everything is calculable and can be solved through the application of new technologies. But these technologies are not neutral facilitators: they embody our political tendencies and prejudices, they transcend the boundaries of national and legal jurisdiction, and even more and more beyond the understanding of their creators. Therefore, as these powerful technologies have more control over our daily lives, we become less and less understanding of the world.

In science and society, in politics and education, in war and commerce, new technologies not only enhance our capabilities, they are also actively shaping and guiding them, for better or worse. If we do not understand how complex technologies work, their potential will be more easily exploited by selfish elites and companies. The result will be seen everywhere around us. There is a causal relationship between the complex and opaque systems we encounter every day and the global problems of inequality, violence, populism, and fundamentalism.

We seem to have entered a new dark age full of more bizarre and unforeseen events, rather than a utopian future where technological advancement will bring hope and liberation to the world. The idea of ​​the Enlightenment to spread more information more widely does not allow us to promote mutual understanding and world peace. On the contrary, it seems to promote social division, mistrust, conspiracy theories and post-truth politics. To understand what is happening, it is necessary to understand how our technologies work and how we become so confident in them.

The cloud is the core metaphor of the Internet: a powerful, almost elusive global system.

In the 1950s, a new symbol began to enter the diagrams drawn by electrical engineers to describe the system they built: a fuzzy circle, or a puffball, or a thought bubble. In the end, its shape became the shape of a cloud. No matter what the engineer is doing, it can connect to this cloud, and that's all you need to know. Another cloud may be a power system, or a data exchange system, or another computer network. Whatever it is, it doesn't matter. Cloud is a way to reduce complexity, it allows you to focus on the problem at hand. Over time, as the network becomes larger and more interconnected, the cloud becomes more and more important. It became a business buzzword and a selling point. It has become more than an engineering symbol; it has become a metaphor.

Today, the cloud is the core metaphor of the Internet: a global system with immense power and energy, but it still retains the taste of something elusive and almost impossible to grasp. We work in it; we store and retrieve things from it; it is something that we have been in contact with but do not really understand. But there is a problem with this metaphor: the cloud is not some magical and remote place: it is composed of water vapor and radio waves, and everything on it is working effectively. It is a kind of physical infrastructure, including telephone lines, fiber optics, satellites, submarine cables, and huge warehouses full of computers, which consume a lot of water and energy. What is absorbed by the cloud are many buildings that were previously important in the public domain: where we shop, get banking services, socialize, borrow books, and vote. Therefore, after being shielded, they become less visible and less susceptible to criticism, investigation, and supervision.

In the past few decades, trading floors around the world have fallen into silence because people have been replaced by a large number of automated trading computers. Digitization means that transactions within and between stock exchanges will become faster and faster. When a transaction falls into the hands of a machine, the machine can react to market conditions almost immediately. High-frequency trading (HFT) algorithms designed by former PhD students in physics entered the market, and traders gave them names such as "The Knife" because they can take advantage of the millisecond time advantage. They can earn a cent from each transaction, and they can do it millions of times a day.

In these massively accelerated and opaque markets, something very strange happened. On May 6, 2010, the Dow Jones Index opened lower and fell slowly in the following hours due to news of the Greek debt crisis. But at 2:42 pm, the index began to fall rapidly. In less than five minutes, the index was erased by 600 points. At its lowest point, the index was nearly 1,000 points lower than the average level of the previous day, which was almost 10% of its total value, and also the largest one-day drop in market history. By 3:07 pm, in just 25 minutes, it had recovered almost all 600 points, which was the largest and fastest shock ever.

In those 25 minutes of chaos, 2 billion stocks worth $56 billion changed hands. What's more worrying is that many orders are executed at what the Securities and Exchange Commission calls "irrational prices": as low as one cent and as high as $100,000. This event is called the "flash crash", and it is still in a vortex of investigation and debate many years later.

Traders may be long-term, and the machine will choose to exit the market as soon as possible in the face of uncertainty.

A report by the regulator found that high-frequency traders exacerbated price volatility. In many high-frequency trading programs, many have hard-coded selling prices: the price set by the program to sell the stock immediately. As the price began to fall, many programs were triggered to sell at the same time. As each price level was skipped, the subsequent price drop triggered another set of algorithms to automatically sell their stocks, which created a feedback effect. Therefore, the speed of price decline exceeds the reaction speed of human traders. Experienced market participants may be able to deal with the crash and stabilize the situation by taking a long-term approach, while the machine will choose to exit the market as soon as possible in the face of uncertainty.

Other theories accuse these algorithms of causing a crisis. One technique identified from the data is that high-frequency trading programs send a large number of "unexecutable" instructions to the exchange-that is, the price of buying or selling stocks in the instructions is far beyond the normal price range, so they will be Ignore. The purpose of this instruction is not to truly convey information or make money, but to deliberately create chaos in the system so that other more valuable transactions can be executed in the chaos. Many orders that were never intended to be executed have actually been executed, which caused drastic market volatility.

Flash crashes are now a recognized feature of markets that adopt new technologies, but people still have a very poor understanding of them. In October 2016, the algorithm reacted to negative headlines about Brexit negotiations. In less than two minutes, the pound exchange rate against the U.S. dollar fell by 6% and then rebounded almost immediately. It is almost impossible to know which headline news or which algorithm caused this crash. In October 2012, a messy algorithm began to place and cancel orders, causing 4% of the trading traffic in the US stock market to be swallowed. In response, a commentator said sarcastically, "The motive of this algorithm is unclear."

At 1:07 pm on April 23, 2013, the Associated Press sent a tweet to its 2 million followers: "Breaking news: Two explosions occurred in the White House and Barack Obama was injured." This The message was sent because the Associated Press account was hacked, and the Syrian Electronic Army, which belongs to Syrian President Bashar al-Assad, later claimed responsibility. The Associated Press and other media reporters quickly issued a warning on the website, saying that the news was false. However, in the face of breaking news, the algorithm does not have this discrimination ability. At 1:8 pm, the Dow Jones Index plummeted. Before most people saw the fake tweet, the index dropped 150 points in two minutes and then rebounded to its previous level. During the period, the market value of the stock market evaporated 136 billion US dollars.

Computing technology is increasingly being cross-layered and hidden in objects in our daily lives. As it expands, opacity and unpredictability will increase. One big selling point of Samsung's "Smart Refrigerator" series launched in 2015 is that they are integrated with Google's calendar service, allowing users to arrange grocery delivery directly in the kitchen. This also means that hackers can read the user's Gmail account password if they can access a machine that was not secure enough at the time. Researchers in Germany have found a way to embed malicious code in Philips WiFi-enabled Hue bulbs, and then the malicious code can spread from one bulb device to another spreading device, spreading to entire buildings and even entire cities. Quick opening and closing-where possible-can also cause photosensitive epilepsy. This is the method favored by Byron, the light bulb in Thomas Pynchon's "Gravity's Rainbow," which is a great uprising of the small machine against the tyranny of its creator. The once-fictional possibility of technological violence is being realized through the Internet of Things.

In Kim Stanley Robinson's novel "Aurora" (Aurora), an intelligent spacecraft transports a crew of human crews from the earth to a distant star. This journey takes several lifetimes, so one of the spacecraft’s jobs is to ensure that humans take care of themselves. When their fragile society is falling apart and the execution of the mission is threatened, the spacecraft will implement control by deploying security systems: it can see any place through sensors, it can open or seal the doors on the ship at will, and it can make a loud noise through its communication equipment Talking can cause physical pain, and a fire extinguishing system can be used to reduce the oxygen level in a specific space.

This is roughly similar to the current operation of the equipment package from Google and its partners: a network of network cameras for home security, smart door locks, a thermostat that can adjust the temperature of a single room, fire and intruder detection that emits harsh alarms system. This means that any successful hacker can have the same ability as Aurora who controls the crew at will or Byron who resists the hateful master.

Before refuting such scenes from the fantasies of science fiction writers, consider the rogue algorithms in the stock exchange. These are not isolated events, but daily events in a complex system. The question then becomes, what would a rogue algorithm or flash crash look like in the broader reality?

For example, will it be like the malware Mirai? On October 21, 2016, Mirai paralyzed a blockbuster internet for several hours. When the researchers delved into Mirai, they discovered that it was targeting those connected devices with insufficient security — from security cameras to digital video recorders — and then turned them into a swarm of robots. In just a few weeks, Mirai infected 500,000 devices. It only needs to infect a small number of devices to paralyze important networks for hours.

In fact, Mirai does not look like Stuxnet at all, which is another virus found in the industrial control systems of hydroelectric power plants and factory assembly lines in 2010. Stuxnet is a military-grade cyber weapon; during the unpacking analysis, it was found to be specifically targeted at Siemens centrifuges. When it encounters a facility with a specific number of such machines, it will explode. That number corresponds to a specific facility: the Natanz nuclear facility in Iran. Once activated, the program will quietly degrade the key components of the centrifuges, causing them to collapse and undermining Iran's uranium enrichment program.

The attack was clearly a partial success, but the impact of this attack on other infected facilities is unknown. To this day, despite obvious suspicions, no one knows where Stuxnet came from or who made it. No one knows who developed Mirai, or where its next version might come from, but it might be in the CCTV camera in your office or in a WiFi-enabled kettle placed in the corner of your kitchen.

Or maybe the attack will look like this: a series of blockbuster films have become catered to right-wing conspiracy and survivalist fantasies, from quasi-fascist superheroes ("Captain America" ​​and the Batman series) to defenses of torture and assassination ("Assassination Ben. Bin Laden, "American Sniper"). In Hollywood, film companies run their scripts through the neural network of a company called Epagogix. The Epagogix system is trained on the implicit preference data formed by millions of movie viewers over the past few decades, with the purpose of predicting what lines will most resonate with people—that is, the most profitable. . Algorithm engines are enhanced with data from Netflix, Hulu, YouTube, and other websites, and can understand the real-time preferences of millions of video viewers, so they can gain a certain degree of cognitive insight, which is completely impossible with previous mechanisms. of. By directly nurturing consumers' desire to chase drama, that kind of network itself becomes excited: reflecting on, strengthening and elevating the inherent paranoia of the system.

Under the guidance of the A/B test interface and real-time monitoring of player behavior, game developers enter an endless loop of game updates and in-app purchases. They have such a meticulous grasp of the neural pathways that humans produce dopamine, so that seriously addicted teenagers play those games around the clock, and eventually die in front of the computer because of exhaustion.

Or maybe the flash crash looks like a nightmare live broadcast that everyone can see online? In the summer of 2015, the sleep disorder clinic at a hospital in Athens was busier than ever: the debt crisis in Greece was in its most turbulent period. There are many high-level government officials and civil servants among the patients, but the machines that monitor their breathing, movement, and even aloud sleep talk at night send those information and their personal medical information back to the equipment manufacturer’s diagnostic data center in Northern Europe. What kind of whispers will escape the tracking of these such facilities?

Users are encouraged to put their phones on the bed in order to record their sleep patterns. But where did all this data go?

We can record all aspects of our daily life by attaching technology to the surface of the body, and technology providers have convinced us that we can also optimize and upgrade like our equipment. Smart bracelets and smartphone applications integrate step counters and skin sensor monitors. They can not only track our location, but also our breathing and heartbeat, and even our brain wave patterns. Users are encouraged to put their phones by their beds at night so that they can record their sleep patterns. Where did all this data go, who owns it, and when might it be leaked? Data about our dreams, our night terrors, and morning sweats (the nature of our subconscious selves) becomes the fuel that further drives the ruthless and incomprehensible system.

Or, the actual flash crash may look like everything we are experiencing now: increasing economic inequality, increasing global surveillance, shrinking personal freedom, the triumph of multinational corporations and neurocognitive capitalism, far-right organizations and local communities The rise of socialist ideology, the deterioration of the natural environment...These are not the direct results of new technologies, but all of these are caused by the general inability to perceive the broader and networked effects of individual and corporate actions, and those actions are positive It is facilitated by an opaque, technologically enhanced complex system.

In 1997, the world chess champion Garry Kasparov (Garry Kasparov) played against Deep Blue for the second time in New York. Deep Blue is a computer specially designed by IBM to defeat this master chess player. After losing the game, he claimed that some of Deep Blue's moves were very smart and creative, and they must be the result of human intervention. But we understand why Deep Blue makes those moves: its decision-making process is a brute force cracking, behind which is a massively parallel architecture composed of 14,000 custom-designed chess chips, capable of analyzing 200 million pieces per second. Board position. Kasparov did not lose in intelligence, but only in "firepower".

When Google's AlphaGo software powered by Google Brain competed with South Korean professional Go player Lee Sedol in 2016, a certain change occurred. In the second round of the five-game match, AlphaGo played a move that surprised Li Shishi: a pawn was placed at the far end of the board. "That move is very strange." said a commentator. "I think this was a mistake." Another commentator said. Six months ago, the experienced Go player Fan Hui became the first professional player to lose to this machine. In his opinion, "Humans don't do this. I have never seen anyone do this."

AlphaGo later won that round and the entire series. AlphaGo’s engineers trained the software by inputting data covering millions of moves of master-level Go players to the neural network, and then letting it constantly play against itself and develop strategies that surpass human players. But the strategies it shows are illegible: we can see how it goes, but we cannot see the decision-making process behind it.

The late famous writer Iain Banks (Iain Banks) called the place where these moves came up as "Infinite Fun Space". In Banks’ science fiction, his cultural civilization is governed by a benevolent, super-intelligent artificial intelligence (Mind). Although Minds were originally created by humans, they have already redesigned and reconstructed themselves and become omnipotent beings. When controlling ships and planets, when commanding wars and caring for billions of humans, Mind also has their own fun. Some Minds have the ability to simulate the operation of the entire universe in their imagination, and they will forever retreat to the bliss space-a field with the possibility of meta-mathematics that only superhuman artificial intelligence can enter.

In 2016, Google’s three networks developed a private form of encryption. Those machines are learning to keep secrets.

Many of us are familiar with Google Translate, which was launched in 2006 and uses a technique called statistical linguistic inference. The system does not try to understand the actual mechanism of the language, but absorbs a large amount of existing translation materials: parallel texts with the same content in different languages. By simply mapping words in one language to words in other languages, it eliminates the need to acquire human-like comprehension, and instead relies on data-driven correlation.

Google Translate is well-known for its ridiculous errors, but in 2016, the system began to use the neural network developed by Google Brain, and its capabilities have grown exponentially. The network does not simply cross-reference piles of text, but builds its own model of the world. The result of this is not a set of two-dimensional connections between words, but a whole layout. In this new architecture, words are coded in a meaning grid based on their distance from other words—a grid that only computers can understand.

Although humans can easily draw a line between "tank" (tank) and "water" (water), it is necessary to draw the boundary between "tank" and "revolution" on a map, The line between "water" and "liquidity" and all the emotions and inferences that arise from those connections suddenly become impossible. Therefore, this map is multi-dimensional, extending in more directions than the human brain can understand. . As a Google engineer said, when a reporter asked for an image of such a system, he said: "I usually don't like trying to visualize a thousand-dimensional vector in a three-dimensional space." This is machine learning system understanding The invisible space where the meaning of language lies. In addition, we cannot understand the space we cannot imagine.

In the same year, other researchers at Google Brain established three networks called Alice, Bob? and Eve. Their task is to learn how to encrypt information. Both Alice and Bob knew a number—a key in cryptographic terms—but Eve didn't. Alice performs some operations on a string of text and then sends it to Bob and Eve. If Bob can decode the information, Alice’s score will increase; but if Eve can do it, Alice’s score will decrease.

After thousands of iterations, Alice and Bob learned to communicate without Eve cracking their passwords: they developed a form of private encryption that is similar to the encryption tools now used for private mail. But the point is that we don’t understand how this encryption works. Its operation is blocked by the deep structure of the network. What Eve doesn't know, we don't know either. Those machines are learning to keep secrets.

"We can't help thinking about that kind of network; we can only think about it thoroughly and think about it."

How we understand and think about our place in the world and our relationship with others and machines will ultimately determine where our technology will take us. We have to think about that kind of network; we can only think about it thoroughly and think about it. The technologies that affect and change our current view of reality will not disappear, and in many cases, we should not want them to disappear. On this planet of 7.5 billion people, our current life support system depends on them. Understanding those systems and the conscious choices we make in their design is still completely within our capabilities. We are not powerless. We just need to think, think again, think continuously. The network—we, our machines, and what we think and discover together—needs us to do that.

As a tool, computing systems emphasize the most powerful aspect of mankind: our ability to act effectively in the world and to shape the world according to our desires. But it is our privilege to reveal and clarify those desires and ensure that they do not demean, deny, erase or eliminate the desires of others.

After being defeated in 1997, Kasparov did not give up the game. A year later, he returned to the arena in a new form: advanced chess, or centaur chess. In advanced chess, humans cooperate with machines, not compete. We soon understood that this model brought interesting results. Even a middle-level chess computer can defeat most masters; and an ordinary chess player can beat the most advanced supercomputer with an ordinary computer—this combination of different ways of thinking The gameplay completely changed this game. Whether human-machine cooperation is feasible-and whether it will be permitted in the future-remains to be seen. After all, there are various complex machines and management systems under development, but compared to obfuscation and one-party rule, humans and machines understand and think together Provides a more promising way out.

Our technology is an extension of ourselves, encoded into machines and infrastructure, using a framework of knowledge and action. Computers do not exist to give us answers to all questions, but to allow us to ask new questions to the universe in new ways.

Your Ads Here

Your Ads Here

Your Ads Here

Your Ads Here