In chapter 11 of the Bible, Old Testament, Genesis, after the Great Flood receded, the people of this world were descendants of Noah and spoke the same language. At that time, mankind began to cooperate and build a tower called Babel. Tower of the Towers. This move alerted God, so God allowed humans in the world to begin to have different languages. Since then humans have been unable to work together. The plan to build the Sky Tower ended in failure, and language differences have become the largest Obstacles. Perhaps there is still a dream in the blood to rebuild the Tower of Babel. Therefore, translation has become a key cultural project for the continuous evolution of mankind over the past thousand years.
The linguistic barrier is not so easy to break. In particular, it is necessary to understand the same concept across languages. For the first time in human history, a cross-language parallel corpora was produced by Rosetta Stone, made in 196 BC. The ancient Egyptian language, ancient Greek, and local colloquial texts were used to record the engraved scriptures of King Ptolemy of the ancient Egyptian king. This is also a major milestone in translation.
Rule-based machine translation
As for the origin of machine translation, which dates back to 1949, information theory researcher Warren Weave officially proposed the concept of machine translation. Five years later, in 1954, IBM and Georgetown University in the United States announced the first translation machine in the world. IBM-701. It was able to translate Russian into English, not to mention that it had a huge body. Actually it had only six grammar rules and 250 words built in. But even so, it was still a major technological breakthrough. At that time, humans began to feel that they should be able to quickly break the wall of language.
It was possible that God had noticed something different and poured a bucket of cold water on the plan for human reconstruction of the Tower of Babel. In 1964, the American Academy of Sciences established the Automatic Language Processing Advisory Committee (ALPAC). Two years later, in the Committee In the report presented, it is considered that machine translation is not worth continuing to invest, because this report caused the United States to almost completely stop the machine translation study in the next ten years.
From the birth of IBM's first translation machine to the 1980s, the technological mainstream at that time was rule-based machine translation. The most common method is to directly translate words according to the dictionary, although some people later proposed to add syntax rules to correct them. But to be honest, the results turned out to be very frustrating, because it looks stupid. Therefore, by the 1980s such practices have disappeared.
Why can't languages apply rules? Because languages are extremely complex and vague systems, from word ambiguity to rhetoric, it is impossible to exhaust all rules. But interestingly, many recent innovations in natural language The company, still trying to solve the Chinese semantics with exhaustive rules, but this idea will definitely end in failure.
I'll give an example to illustrate why the rules are not feasible. Don't mention the complexity of translation in two languages. Just from the Chinese perspective, the concept of express delivery is fast. How many kinds of teachings can you think of? 10 kinds or 100 kinds? According to the natural language statistics we have done before, there may be 3,600 kinds of teachings in total, and this number should increase over time. A sentence with such a simple concept can be so For a complex system of rules, if you use translations, I am afraid that the amount of rules will be an astonishing astronomical number. Therefore, the rule-based machine translation idea will become a yellow flower yesterday.
Instance-based machine translation
While the whole world has fallen into the low phase of machine translation, there is a country that has strong obsessions for machine translation. That is Japan. The Japanese have poor English proficiency and therefore have a strong rigid demand for machine translation.
Professor Nagao Shinretsu of Kyoto University in Japan proposed an example-based machine translation, that is, stop thinking about letting machines translate from scratch. We only need to store enough example sentences. Even if we encounter sentences that do not match perfectly, we You can also compare example sentences by simply replacing the translation of different words. This sort of naive thinking is certainly not much better than rule-based machine translation, so it did not cause a wave. But soon, the hope of human reconstruction of the Tower of Babel Seems to see the dawn again.
Statistical machine translation
The detonation of the statistical machine translation boom is still IBM. In the paper "Mathematical Theory of Machine Translation" published in 1993, five word-based statistical models were proposed, namely "IBM Model 1" to "IBM Model 5".
The idea of the statistical model is to treat translation as a probability problem. In principle, it is necessary to use parallel corpus and then perform statistics on a word-by-word basis. For example, although the machine does not know what “knowledge” is in English, it will be found after most corpus statistics. As long as there is a sentence with knowledge, the word “Knowledge” appears in the corresponding English example sentence. In this way, even if the dictionary and grammar rules are not maintained manually, the machine can understand the meaning of the word.
This concept is not new, because Warren Weave first proposed a similar concept, but then there was not enough parallel corpus and the ability to limit the calculator at the time was too weak and therefore not put into practice. Modern statistical machine translation from Where can we find the "modern Rosetta Stone"? The main source is the United Nations. Because the resolutions and announcements of the United Nations will all be in the language versions of various member countries, but in addition to this, we must produce parallel corpus by ourselves. Now the cost of human translation translates to knowing that this cost is astonishingly high.
In the past ten years, everyone is familiar with the Google translation is based on statistical machine translation. Hearing this, it should be clear that the statistical translation model is unable to accomplish the great cause of the tower. In your prints, the machine translation only stays in The degree of "useful" rather than "useful".
Neural network machine translation
By 2014, machine translation ushered in the most revolutionary change in history - "deep learning"!
Neural networks are not new. In fact, neural network inventions have been around for more than 80 years. However, deep learning has continued since Geoffrey Hinton (deep study of the three great gods) improved the fatal shortcomings of neural network optimization in 2006. Various miracles-like results have frequently appeared in our lives. In 2015, the machine for the first time realized image recognition beyond humanity; in 2016, Alpha Go defeated the world chess king; in 2017, speech recognition surpassed human stenographers; in 2018, Machine English reading comprehension goes beyond humans for the first time. Of course, this area of machine translation has also begun to flourish because of the deep learning of this super fertilizer.
Yoshua Bengio of the deep learning God in the 2014 paper, for the first time laid the basic structure of deep learning technology for machine translation. He mainly uses a sequence-based recurrent neural network (RNN), so that the machine can automatically capture sentences The word feature, which in turn can be automatically translated into another language's translation result. This article shows that Google has won the treasure. Soon after, Google provided ample gunpowder and Great God's blessing, Google officially announced in 2016 that All statistical machine translations were off the shelf, neural network machine translations became the absolute mainstream of modern machine translation.
The biggest feature of Google’s neural network machine translation is the addition of Attention. In fact, the attention mechanism is to sweep through the eyes first when simulating human translation, and then to pick out a few key words to confirm the semantics. Process (figure 2). Sure enough, with the attention mechanism blessing, the power has greatly increased. Google claims that in the English-French, English-Chinese, and English-Western languages, the error rate has changed. The statistical machine translation system is reduced by 60%.
Although the neural network can learn from the existing parallel corpus and understand the subtle linguistic features of the sentence, it is not perfect. The biggest problem arises from the large amount of data needed and its incomprehensibility as a black box. That is to say, There is no way to make mistakes, but only to provide more correct corpus to correct “deep learning”. Therefore, the same sentence pattern can have very different translation results.
In February 2018, Microsoft made new moves to make machine language understanding beyond humanity. On March 14, researchers from Microsoft Research Asia and Redmond Research Institute announced that their R&D machine translation system was The news report test set Newstest2017's Chinese-English translation test set has reached a level comparable to that of human translation. This is naturally a major victory for the machine translation of neural networks. Of course, there are also many innovations in the architecture, of which the most noteworthy. It is joined with Dual Learning and Deliberation Networks.
Dual learning has to solve the problem of limited parallel corpus. In general, deep learning must be provided to the machine at the same time. In this way, the machine can continuously modify and improve according to the difference between its translation result and answer. As for the stimulating network, it is also a process of imitating human translation. Usually, human translators will do a rough translation first, and then adjust the content to the exact second translation result. In fact, you can find that no matter how smart the neural network is, you still have to refer to the smartest creature on the surface. For humanity we.
Language cannot be used out of context
The development of machine translation does not mean that people in the translation industry will have no food in the future. It can be noted that Microsoft's publication has emphasized the “Chinese-English translation test set” of the “Universal News Report Test Set Newstest 2017”. A good performance may not be equal to universality, which can also explain why Tencent's translator Jun Mingming has a good reputation, but why the real-time interpretation in Boao has been inaccurate.
Real-time interpreting can be said to be the culmination of a translation task. In addition to having a correct listening comprehension of the original sentence, it must be converted to other languages within a limited time. And remember that the speaker will not give any time for the translation, so it is equivalent to speech recognition. Machine translation must be processed synchronously, together with noise on the spot, speaker's expression, interjections of modal words, etc., all of which may cause misjudgment by the machine.
From my point of view, Tencent’s translation of the king may be blamed on the point that it may not be enough work, and the key proper nouns are not entered. This will result in a “classic mistake” of “a highway and a belt”.
An interesting difference can also be seen in Fig. 3. Why is Western machine translation misplaced, but machine translation in the home country is almost always in control? This is because the language cannot exist without departing from human use scenarios. That is, we often learn Chinese. Context, which comes from our past culture, consists of memories that were common in the past. Google, who has not read Tang poetry, naturally cannot understand the essence of this poem. Language can be the last human barrier in the age of artificial intelligence because Languages will constantly change due to the use of humans. This is a very difficult substitute for machines.
With the advancement of technology, one day, machine translation will change from being "useful" to "useful" and then evolving to "useful." But as I have always argued, machines will not rob people of their work. It is only ourselves that human beings are unemployed. How to make good use of artificial intelligence to become your own tool, and to withdraw yourself from tedious work, this is the correct posture for the future.