What is the "AI chip" that is being installed in smartphones?
In 2017, the flow of "incorporating artificial intelligence (AI) processing support logic into smartphones" was born.
In September, Huawei, familiar with the Mate series, announced the "Kirin 970" as an SoC for smartphones (a processor that integrates all the functions necessary for computer operation such as CPU, GPU, and memory interface into one chip). Although we announced it, we revealed that it integrates AI processing support logic called Neural Processing Unit (NPU).
This Kirin 970 is used in Huawei's latest smartphone "Mate 10 Pro".
At about the same time, Apple also announced that the main processor "A11" of the new smartphone "iPhone X" was integrated with "Nerual Engine" as AI processing support logic.
Google and NVIDIA have also developed the Tensor Processing Unit as AI processing support logic and are implementing it in their own processors.
These AI processing support logics are called "AI chips" as an abstract name.
When you think of an AI chip, you might think that "artificial intelligence that thinks for yourself" fits into one chip, but the structure of an actual processor is similar to that of a GPU, and it is nothing more than an integrated vector computer.
Of course. The processor you put on your mobile phone can't be the brain of an elaborate robot like the one in science fiction movies.
The mainstream of the current AI architecture is machine learning type AI, but in practicing these, a large amount of matrix multiplication and summation is performed.
Therefore, the AI chip mentioned above can be said to be similar to a processor that can perform a large number of 4x4 matrix multiplications and summaries at the same time.
An illustration of the integration of a 4x4 matrix. AI chips can perform many of these calculations in parallel at once.
Some may have recalled, "Machine learning AI should have been popular in graphics processors (GPUs)."
exactly.
Certainly, GPU can also perform 4x4 matrix multiplication and summing at high speed, but since GPU was originally developed for graphics drawing, the calculation accuracy was designed based on 32-bit floating point. increase.
Besides, the GPU also contains logic specific to texture access and logic around the display output.
Calculations performed with machine learning AI do not require the graphics drawing function, and the calculation accuracy does not actually need to be as high as that of the GPU.
That's why 16-bit floating point numbers (FP16) and 8-bit integers are sufficient for operational accuracy.
In other words, the AI chip built into the smartphone omits the graphics-related functions installed in the GPU, and simplifies the calculation accuracy and the scale of the calculation unit.
That's why you can put it on your smartphone.
You may be wondering, "Since GPUs are also installed in smartphones, you can do the same by using them, right?"
It's pretty sharp.
That's true, but the GPU installed in the smartphone has to mainly perform graphics processing, and when AI processing is being executed, scrolling of the smartphone screen may slow down. The operation feeling is not good, isn't it?
So, at present, GPU and AI processing support logic are separate.
The AI processing support logic installed in NVIDIA GPUs and Tegra is called TENSOR CORE.The main calculation to be practiced is the product-sum calculation of a 4 × 4 matrix.
By the way, NVIDIA's SoC Tegra series has an architecture that allows AI processing support to be practiced on the GPU core before Tegra X2, but in the next generation TEGRA (development code name "Xavier"), like Kirin 970 and A11. , It seems that it will be changed to a design that implements dedicated AI processing support logic.
If the GPU has better performance than it is now, it will be possible to concurrently support AI processing, but now the policy is that it is more efficient to implement it separately.
In 2018, it seems that various SoCs equipped with AI processing support logic called "AI chip" will come out, and smartphones that adopt those SoCs should come out.
I don't know if this trend will spread to the entry class by the end of 2018, but at least it is expected that adoption will proceed at such a momentum that it will be "must have" for high-end smartphones.
Various stand-alone VR-HMDs are likely to come out!
Next is the topic of "Virtual Reality" (VR), which has a feeling that it is no longer unrelated to smartphones.
In 2017, Google's "Daydream View" was released in Japan as well, and although it is secret, I feel that a platform called "Smartphone VR" has been launched, which can be used by inserting a smartphone into a VR-HMD.
Daydream View
And in 2018, a new style will emerge in earnest in VR-compatible head-mounted displays (HMDs).
That is the "stand-alone type" VR-HMD.
Oculus Rift, HTC VIVE, and Sony's PSVR required a separate host computer, a Windows PC, and a game console, PS4, to enjoy VR content.
The stand-alone VR-HMD is the one that integrates this host computer into the VR-HMD.
"HoloLens" released by Microsoft and Daydream View, which is converted to VR-HMD by inserting the smartphone mentioned above, can be said to be a kind of stand-alone VR.
In the latter half of 2017, the announcement of the appearance of this stand-alone VR-HMD came one after another.
Oculus has announced Oculus GO, HTC has announced VIVE FOCUS, and Google has also announced Daydream Standalone, announcing the release in 2018.
Oculus GO
VIVE FOCUS
Daydream standalone teaser movie. Lenovo Daydream standalone headset coming out
Oculus Go seems to be compatible with Gear VR, and VIVE FOCUS will develop its own framework called "VIVE WAVE".
Daydream standalone is, of course, based on the Daydream platform.
Based on these three platforms, it is expected that various VR-HMDs with built-in host computers will appear in 2018.
Microsoft is also focusing on Windows Mixed Reality (Windows MR), so a Windows 10-based standalone "MR" -HMD may come out.
Oculus Go, VIVE FOCUS, and Daydream standalone don't seem to be that expensive, so hopefully they'll become popular with the general public and be recognized as "wearable mobile devices."
Why is the LCD of VR-HMD video panels becoming more and more liquid crystal?
I would like to provide another topic in connection with HMD.
This is a topic related to display panel technology, which is my other specialty.
In 2016, it was touted as "the first year of VR", and Sony released PSVR as a VR system for PS4. Oculus Rift and HTC VIVE have been released for Windows PC.
All three "VR-HMD Misanke" adopted organic EL panels.
Of the standalone VR-HMDs Oculus Go, VIVE FOCUS, and Daydream standalone mentioned above, Oculus Go uses an LCD panel.
VIVE FOCUS is an organic EL panel.
Daydream Standalone has not been officially announced so far, but it is expected to become a liquid crystal panel because it was reported in May 2017 that "Google will adopt Sharp's liquid crystal panel for its own VR-HMD".
Even so, why did you come here and the trend toward liquid crystal display of HMDs occurred?
Those who wondered, "Is the LCD panel cheaper?" Are half correct.
But there are also some deeper reasons.
In the first place, the VR-HMD Misanke adopted the organic EL panel because the response speed of the pixels is an order of magnitude faster.
Rift and VIVE have two 3.5-inch organic EL panels (for both eyes) of 1,080 x 1,200 pixels per eye, and PSVR has a single 1,920 x 1,080 pixel 5.7-inch organic EL panel that can be seen half by two eyes. It was.
The binocular resolution is about 2.6 million pixels for Rift and VIVE with 2,160 x 1,200 pixels, and about 2.07 million pixels for PSVR with 1,920 x 1,080 pixels.
The ppi value representing the pixel density was 461ppi for Rift and VIVE, and 386ppi for PSVR.
At first glance, the Rift and VIVE ones seem to have higher ppi values, but in fact, the Rift and VIVE OLED panels are called pentile (staggered) arrangement, which is a design that emphasizes cost rather than image quality, and is a blue and red sub. The pixels were only about half the nominal overall resolution.
So, so to speak, the effective ppi was only about 307ppi, and PSVR had better video expressiveness. In fact, this was the main reason why it was said that "PSVR has better images than Rift and VIVE".
Difference in sub-pixel structure of the video panel. The "RGB stripe" type on the left has excellent resolution because all RGB (red, green, and blue) are the same number.In the "Pentile RGB" type on the right, only G has full resolution, and the R subpixel and B subpixel are shared by multiple G subpixels (driven by the average value), so the resolution is poor.
Although it was a very popular Misaki VR-HMD, there were quite a few opinions that the display image was "insufficient in resolution".
This is a problem that comes from the current structure of VR-HMD, which is to magnify the video panel.
To improve this, you need to use a higher resolution video panel than ever before.
In other words, you need a video panel with a high ppi value.
GPUs for PCs that will appear after 2018 are expected to achieve the performance of drawing 4K resolution at 90 fps or higher.
In that case, the binocular resolution of the new generation VR-HMD will match about 4K.
Even if the GPU built into the SoC for smartphones cannot render CG (≒ game graphics) in 4K in the near future, it will be possible to handle live-action VR content with 4K resolution in the near future. I will come.
The 5G mobile phone network, which is aiming for practical use around 2020, aims to deliver 8K VR content in real time using smartphone VR. The photo is the prototype model released at CEATEC 2017.The need for high-resolution video panels is increasing even in smartphone VR and stand-alone VR-HMDs.
However, it is not desirable to increase the body size of the VR-HMD just because the resolution is high.
In other words, there is a need for a video panel with a higher resolution that is equal to or smaller than the current body size.
Considering that the video panel of the Misanke VR-HMD had a definition of about 400ppi, at least twice that, 800ppi class is required.
In fact, it is difficult to manufacture a high-definition panel that achieves 800 ppi with an organic EL panel.
At present, the only option for video panels that can achieve this pixel density is liquid crystal display.
Of course, due to technological innovation, it is possible that higher-definition OLED panels will appear, but at least in the short term, we will manufacture OLED panels with a binocular resolution of 4K (3,840 x 2,160 pixels) at 800 ppi. Is difficult.
Therefore, there is a growing trend toward the adoption of LCD panels for high-resolution HMD video panels.
The competition to develop high-ppi LCD panels is accelerating.
After all, LCD panel manufacturers are now focusing on the development of "LCD panels specialized for HMDs".
For example, in December 2017, JDI announced a dedicated LCD panel for VR-HMD.
The size is 3.6 inches and the resolution is 1,920 x 2,160 pixels. The definition is 803ppi.
Since the panel size and aspect ratio are almost the same as Rift and VIVE, it was appealed that it is possible to configure an HMD with 4K binocular resolution while keeping the conventional HMD body size.
The orientation mode of the liquid crystal is IPS type, but it seems that it is a newly developed ultra-high speed response speed IPS type panel.
Based on this technology, JDI also announced the release of a VR-HMD dedicated LCD panel with over 1,000 ppi by 2019.
A VR-HMD prototype using a VR-HMD dedicated LCD panel announced by JDI. Supports 90fps drive
Sharp is also famous when it comes to LCD panel manufacturers.
In 2017, Sharp released a prototype VR-HMD using a 1,920 x 2,160 pixel VR-HMD dedicated LCD panel with the same resolution as the JDI panel.
This has a panel size of 2.87 inches. Since the definition is 1,007ppi, it exceeds that of JDI.
Sharp's proud IGZO is used for the semiconductor that drives the liquid crystal display.
There are also reports that Google is already moving toward adoption.
LCD panel for VR-HMD announced by Sharp. The panel itself was announced in 2016, but in 2017, the prototype VR-HMD jointly developed with NTT DoCoMo was released.
Generally speaking, "LCD vs. OLED" is often associated with television, but in fact, even in HMD products, this "composition of battle" is becoming more intense.
The technical theme predicted this time is only a part of the elemental technologies that make up smartphones, but in any case, smartphones should continue to evolve this year.
Last year, I bought the iPad Pro (12.9-inch model) as the first Apple product in my life, and although it has not been reported in this series yet, I replaced it with a new smartphone and "Mobile Tech that surrounds me personally" has evolved. bottom.
I hope that we will be able to meet various new technologies this year as well.
Next time, I would like to talk about the new smartphone.
