At present, Qualcomm is actively promoting its terminal AI strategy in areas such as smart phones, the Internet of Things, and smart cars, and at the same time accelerating the research and development of AI's innovative technologies to build an AI ecosystem. At the Qualcomm Artificial Intelligence Summit held in recent days, the heads of relevant departments shared The development strategy, progress and use cases of Qualcomm technology in support of AI presented Qualcomm's panorama in the AI field.
Smartphone AI focuses on visual, speech, AR three aspects
As a giant in the field of mobile communications, Qualcomm has made tremendous contributions to the development of mobile communications technology and smart terminals. After more than ten years of development, although the growth rate of the smart phone market is slowing, but with the Internet of Things, 5G era The arrival of the smart phone's core position once again highlights.
It is estimated that in 2018-2022, the cumulative shipments of smart phones will exceed 8.6 billion. This means that AI technology has a wide range of applications in the field of smart phones. Smartphones are the largest platform for artificial intelligence to make relevant innovations. Today, The killing popularity of mobile phone manufacturers in the AI field can already illustrate this point.
According to Gary Brotman, Director of Product Management at Qualcomm, Qualcomm focuses on three areas in the AI field of smartphones:
The first is the visual AI field. From the current situation of mobile phone manufacturers applying AI, the most AI in visual aspects, such as facial recognition, unlocking, payment, and smart camera, beauty, etc., on the one hand, visual AI technology is relatively mature, on the other hand , Mobile phone users experience high demand in this area.
The second is AI voice, using voice and intelligent terminals to communicate dialogue and better interaction.
The third is Augmented Reality (AR), which requires a high density of terminal-intensive computing capabilities. Through AR, using the surrounding environment to achieve common support for all sensors on a smartphone, providing entertainment in real time by integrating virtual and reality, education And remind services.
In the past three years, Qualcomm has promoted AI technology evolution for three generations based on the AI platform on smartphones. In this process, as the architecture of AI algorithms continues to evolve, the capabilities and capabilities of the platform have also become rich and powerful. Qualcomm is also gradually building the openness of AI ecology.
The first-generation AI platform is Xiaolong 820. Researchers and commercial developers can run neural networks on the CPU, and typically run Caffe. Caffe is the most mainstream architecture at that time, and it is the basis of many related R&D in the commercial field. It can be directly Put neural network training on the terminal CPU.
The second generation is Snapdragon 835, which introduces the neural processing engine SDK. Through in-depth cooperation with Google and Facebook, the framework provided for both is optimized, supporting TensorFlow and Caffe2 on the terminal side. This gives the developer a great deal. Convenience, according to their needs, choose the right hardware core to support AI operations.
In December of last year, the third-generation AI technology was integrated in Qualcomm's Xiaolong 845, which expanded the computing power of each core, and also extended its support for more neural network frameworks. By working directly with the neural network framework vendors, on the other hand, the ONNX exchange format is also supported, which can easily provide developers with more flexibility without having to worry about what kind of network the underlying hardware supports.
Qualcomm artificial intelligence engine AI Engine is composed of a series of hardware and software components, designed to enable terminal-side AI to provide developers with high energy efficiency and flexibility. On the hardware, by optimizing the CPU, GPU and DSP, can meet different AI applications Scene requirements for power consumption, neural networks, workloads, and energy efficiency.
In terms of software, the Android NN environment can be supported through the SDK. It also supports Hexagon NN. If the developer chooses to use the Hexagon DSP for development, the Hexagon NN library can be optimized specifically for a certain kernel to achieve the optimal energy efficiency ratio. All this brings great flexibility to developers and customers and helps them to maximize performance.
From the industry point of view, only HiSilicon's Qilin 750 set up the AI unit NPU separately, and Qualcomm and MediaTek still use the distributed computing approach. Gary Brotman believes that the smart phone AI experience cannot rely solely on a specific core, but more important It requires multiple architectures and multiple tools. At present, distributed architecture can fully meet the needs of smart phones for AI functions.
AI Voice Personal Assistant: Transformative User Interface
Many important use cases of artificial intelligence require the support of various aspects. After the maturity of visual AI, the next step of AI is considered to be AI voice. Personal digital assistants have also become one of the important use cases, such as mobile assistants, smart speakers, etc. .
According to Hou Jilei, director of the Qualcomm artificial intelligence research project and senior director of engineering technology, real-time is a very important feature of personal assistants to provide services to users at any time. From the perspective of energy efficiency, the characteristics of personal assistants' always-online model are highly effective. There are very high demands. In terms of personalization, how to provide very personalized service at the level of hearing, intention and behavior is also an important personal need for personal assistants. In terms of learning, we hope that personal assistants will continue to understand personal behaviors. , Constantly adjusting the model, and learning and training on the terminal side. From the system architecture perspective, context awareness is an important characteristic of artificial intelligence assistants, and promoting sensor multimodal learning and multimodal fusion is an important manifestation of its future capabilities. angle.
Voice interaction is a very important part of the personal assistant. Voice is the revolutionary user interaction interface that we have been looking forward to, especially in the application scenarios of hands-free (without using hands to touch the terminal). The voice interface is to create a virtual assistant. It is very important. The voice interaction interface can support four very important features. First, it is always on, it needs to be always online, and it is always ready to provide services. Second, the conversational type, personal assistant does not memorize commands, but can do Very natural and smooth, multiple rounds of normal communication. Third, personalization, how personal assistants recognize words and phrases, and a clear understanding of intentions, which is also a very important aspect of the voice interaction interface. Fourth, privacy, do not put the data Go to the cloud, but do a lot of processing in the terminal, which is also an important direction for the future development of the voice interactive interface.
In fact, as a research direction, speech interaction has been in existence for more than 50 years. Why did it suddenly become hot in recent years? Hou Jilei pointed out that about 20 years ago, the way of machine learning was traditional machine learning, not today’s. Deep learning, at that time through the Gaussian mixture model, has been able to achieve certain performance indicators. With the emergence and mining of deep learning, convolutional neural networks and recursive neural networks are constantly being applied to the speech interactive interface scene. There has been a very The important trend is: Speech recognition performance indicators will soon approach or even exceed human accuracy.
'When this important threshold was breached, a large number of voice interactive application scenarios and business models, such as consumer, enterprise, and industrial, would soon be excavated.'
Another very important trend is that the voice interaction function is rapidly migrating from the cloud to the terminal. Today, from the commercial perspective and not from the R&D perspective, voice interaction is still more of a cloud-centric architecture. Even in this case Due to low-power consumption and real-time considerations, some functions required for voice interaction such as voice noise reduction and voice activation have been processed on the terminal side. The migration from the cloud to the terminal side is a gradual process that will soon include More features in speech recognition, natural language comprehension and text-to-speech (TTS) will gradually evolve to be centered on the terminal side. The end-to-end solution driven by machine learning is driving the trend of voice interaction to the terminal side. Houying Hou indicated that the evolution of voice interaction from the cloud to the terminal lies in the advantages of privacy, timely response, etc. The long-term development direction of voice interaction in the future should be the close integration of the cloud and the terminal. Model training, model updating, knowledge base application and some more general Services, processing in the cloud can be better complemented with the terminal.
AI Brings Opportunities and Challenges to Internet of Things and Smart Cars
Today, many Internet of Things terminals that support terminal-side artificial intelligence have been introduced to the market, including smart speakers, smart assistants, smart cameras, home hubs, smart vacuum cleaners, etc. For example, in home hubs, smart speakers and smart assistants have applied voice intelligence. Application of image classification, object classification and face recognition in networked cameras. In terms of robots, for example, smart vacuum cleaners are used to avoid obstacles. Overall, terminal-side artificial intelligence is rapidly developing to provide a strong driving force for the Internet of Things.
According to Shardul Brahmbhatt, senior product manager of Qualcomm, Qualcomm's technical support terminal-side artificial intelligence use cases for the Internet of Things are generally divided into three categories: vision, audio and sensor processing.
Brahmbhatt introduced two use cases. First, in the field of enterprise security, person detection, face recognition, and face detection were used to identify employees and non-employees entering the company's building to ensure that the security system's warning signal could be used on the terminal side. Send it out without going back to the cloud for processing. Another use case is Smart City. In this use case, terminal-side artificial intelligence can help with license plate recognition, collision accident warning, and traffic condition monitoring.
Today, whether it is a traditional depot, or many new Internet companies, and many emerging unicorn enterprises want to be able to grasp the future of the car market. About the future of cars, Qualcomm has identified three major directions. First, to make cars and all things connected. Whether today's 4G or the future 5G, more than 70% of cars will support Internet of Vehicles by 2021. Many domestic and foreign large vehicle manufacturers will achieve 100% coverage of car networking this year or next year. Second, change drivers and passengers. The user experience. Over the years, the car's infotainment platform has grown rapidly from no-screen to large-screen and multi-screen, enabling full digital and multiple operating systems, while providing rich cloud-based services and content interaction. Third, automatic for the future Driving paving the way.
According to Ye Zhiping, senior director of product marketing at Qualcomm, at present, major automakers use Qualcomm's technologies and solutions. In terms of connection technology, Qualcomm is the world's largest supplier of in-vehicle information processing and automotive Bluetooth, with more than 10 years of experience. .
'Qualcomm's leading next-generation top-level infotainment solution has been adopted by a variety of cars that will be mass-produced in 2019-2020, from large screens to multi-screens and multiple operating systems. In fiscal year 2017, Qualcomm won 25 new vehicle types. Information processing and infotainment design. Fourteen top global automakers have already selected 14 brands to use the Opteron platform in their automotive infotainment designs and mass production in their next generation cars. 2019 and 2020 The first generation of Xiaolong 820 car processor will enter mass production. ' Ye Zhiping said.
At the same time, Ye Zhiping pointed out that the challenges and requirements of a series of automotive artificial intelligence need to be met. The first is security, the car will have many user privacy data, such as face or fingerprint. Second, the artificial intelligence of the car must be able to respond immediately, such as In the automatic driving state, the delay must be greatly reduced. Third, practicality and reliability, artificial intelligence must be able to operate in any state, especially in some areas without network coverage. Finally, there are still challenges of thermal efficiency and energy efficiency.
Ye Zhiping said that the power consumption usually involves artificial intelligence will be up to 100 watts, the car is more than 60 watts, the server (sever box) design requirements exceed 60 watts must be equipped with a cooling system. For the display of the current design is possible, and To achieve a true mass production of self-driving cars with reasonable cost-effectiveness and without wasting space in the car, the current technology cannot be realized. With more and more electric cars starting to be on the market, power consumption and battery mileage are also a relatively large consideration. .