OpenCV AIKit (OAK)

HTC is back for summer session. Lot's of fun high-tech developments and research to cover before we need to lock ourselves in the code dungeon again.

And the first thing to focus on is a very new in-development open source inexpensive neural net AI embedded hardware platform.  Introducing the OAK, specifically OAK-1 and OAK-D.

These 2 very inexpensive OAK hardware boards are tiny artificial intelligence (AI) and computer vision (CV) powerhouses, with OAK-D providing spatial AI leveraging stereo depth in addition to the 4K/30 12MP camera that both models share. They are supposedly absurdly easy to use, up and running in under 30 seconds. You can program your OAK's by just plugging them into your Mac or Ubuntu or Windows computer's USB port. OAK's modular, FCC/CE-approved, open-source hardware ecosystem can also be directly integrated into your products if you want to use them as embedded hardware in systems that you sell.

OAK is currently a brand new kickstarter program.  Funded in the first 20 minutes no less, and with current pledges continuing to climb it is one of the most successful embedded hardware boards ever crowd funded. Here's an interview with OAK's creator Brandon Gilles to reassure everyone that you are in good hands with this particular kickstarter project (not always the case).

Here's some more information about what OAK is all about. OAK comes in 2 different versions

OAK-1: The standard OpenCV AI board that can perform neural network inference, object tracking, April Tags recognition, feature detection, and basic image processing operations.

OAK-D: Everything in OAK-1, but with a stereo depth camera, 3D object localization, and tracking objects in a 3D space.

You can think of these 2 boards as very smart cameras.  But smart cameras that return structured data, data that be much more sophisticated then just pixel data, more like intelligent metadata.  You can certainly use them to capture 2 or 3D images or videos. But you can also use them to tell you things like the position and name categories of different objects in the scene they are viewing (again 2D or 3D positions), or where all the strawberries are in physical space it is viewing, their approximate ripeness, types of insect damage they have, etc.  They can also tell you if someone in the scene is wearing a Covid-19 mask or not.

You can program the OAKs in C++ or Python.

The color camera resolution is 4056 x 3040 pixels.
The depth camera resolution is 1280 x 720 pixels.  There are 2 image sensors (left and right view) in the depth camera on OAK-D.

Note: You would currently have to program the calibration mapping to map the depth image into the higher res color camera image. They have plans to do this at some point and include the calibration data in the OAK's onboard EEPROM memory. It's apparently a little bit tricky right now due to the auto-focus in the color camera.

The left or right depth camera views can be output at full frame rate (optionally H264 or H265 coded if you want that rather then raw pixels).

The OAKs are based on Myraid X,  Intel's Movidius Myriad X Vision Processing Unit.  You may recall that we discussed the Intel Movidius system before in our previous look at embedded deployment of deep learning neural net systems.  Brandon makes an interesting point in the interview article that the usb stick solution we looked at previously only uses 1/7 of the Movidius's computer capability, and that this was why Brandon looked into developing a different more efficient solution.

The design goals for OAK include:
- Real-time neural inference (covering nearly every type of network)
- Stereo depth
- Feature tracking including Motion Estimation and platform-motion estimation
- Object tracking
- H.265/H.264 encoding (for recording incidents and/or generally just recording)
- JPEG encoding
- 16x efficient vector processors for combining these capabilities together and running accelerated CV functions (think of these as link an embedded vision-specific GPU)

And now you get a sense of why this kickstarter program was funded so quickly.  Truly a very smart camera.

Comments

Post a Comment

Popular posts from this blog

CycleGAN: a GAN architecture for learning unpaired image to image transformations

Image
 Like the Doublemint Twins touting the joys of Doublemint gum, 2 GANs are surely better than 1 GAN.  Especially if we package them together inside of one meta GAN module.  And this is exactly what the CycleGAN architecture does. Have you ever harbored dark secrets of turning a horse into a zebra?  The CycleGAN was developed to do just that. Learn how to turn a horse into a zebra.  And more. Now right away you can notice a difference between the image to image transformation GAN architectures we've been discussing over the last few posts.  Those last few posts described systems that learn from a database of matched input-output image pairs. And if your goal is to turn an edge representation into a nicely filled in continuous tone image, it's easy to build your database of matched input-output image pairs that your GAN system can then learn off of.  Take a continuous tone image (which will be the output of the database pair entry), then run it through an edge detector algorithm. 
Read more

Pix2Pix: a GAN architecture for image to image transformation

Image
 I thought following up yesterday's TraVelGAN post with a Pix2Pix GAN post would be useful to compare what is going on in the 2 architectures.  Two different approaches to the same problem. I stole this Pix2Pix Overview slide below from an excellent deeplearning.ai GAN course (note that they borrowed it from the original paper) because it gives you a good feel for what is going on inside of the Pix2Pix architecture.    Note how the Generator part is very much like an auto-encoder architecture, but rebuilt using the U-Net architecture features (based on skip-connections) that fastai has been discussing in their courses for several years before it became more widely known to the deep learning community at large  (and which originally came from an obscure medical image segmentation paper) . So the Generator in this Pix2Pix GAN is really pretty sophisticated, consisting of a whole image to image auto-encoder network with U-Net skip connections to generate better image quality at highe
Read more

Smart Fabrics

Image
"What makes smart fabrics revolutionary is that they have the ability to do many things that traditional fabrics cannot, including communicate, transform, conduct energy and even grow."  This is a quote from Rebeccah Pailes-Friedman, who is a professor at Pratt with a research focus on developing wearable technology and smart textiles. Enhancing textiles can be viewed through different lens.  Enhancements can be aesthetic, or performance boosting, or durability boosting. 
Read more