GPUs are now used pretty much everywhere for running the training of deep learning models as this involves sustaining some serious processing power usually over a period of several days. However, once the model is trained it can be deployed on various other architectures that can run the inference much more efficiently and in situations where standard GPUs would not be feasible.
Recent advancements in chip technology gave birth to novel architectures designed to perform incredibly complex computations using very little power. This in turn enabled novel solutions for ever more popular domain of edge computing. Unlike cloud computing, edge computing is capable of processing data right at its source resulting in real-time or near real-time data analysis, lower operating costs, reduced network traffic and improved application performance.
One such architecture, which I decided to evaluate, was Intel Movidius Myriad 2 chip offering high-performance at ultra-low power. I came across news about a very new mini-PCI Express module developed by UP featuring Myriad 2 processor. I have previously used Intel Movidius Neural Compute Stick that has the same processor but having a mini PCI-e board opens up a whole new list of possibilities, so I decided to test it out. This new PCI-e module called AI Core enables Artificial Intelligence at the Edge. Thanks to its form factor it can be seamlessly integrated in most of the industrial computing edge devices. It is powered by ultra-low power high-performance Intel® Movidius™ Myriad™ 2 2450 VPU with 512MB.
For this demonstration, a single AI Core module was plugged into a mini-PCI Express port of UP Squared, the world’s fastest x86 maker board based on latest Intel platforms Apollo Lake and the successor of 2015 Kickstarter supported UP board. UP Squared was setup with Ubuntu 16.04 Linux distribution and the AI Core module was used to run convolutional neural networks trained to recognise and localise various objects. These neural networks are based on the Caffe implementation of the mobilenet-ssd architecture. The following video shows three different models running on this setup in sequence at around 10 frames per second. None of this model is particularly well trained or optimised. The main point of the video was to demonstrate how this setup can be used for various edge applications.
As a Head of Innovation at Cortexica, I look into emerging applications that I believe hold a lot of promise for the future. I am convinced that Edge Computing and particularly the Intel Myriad family chips on the AI Core module will enable many exciting applications. Cortexica is already working with Cisco on some cool projects (https://emear.thecisconetwork.com/site/content/lang/en/id/8533) I already see countless applications of edge computing in our business.