Loading and Visualizing Kinect Depth images in Matlab(用matlab将深度图显示成彩色图)(转）

傷城~ 2023-06-13 11:11 19阅读 0赞

In the [previous post][], we developed an application for saving the acquired RGB and depth images from a Kinect device connected to the USB port using MATLAB. In this post, we will see how the saved depth images can be loaded in MATLAB, and visualized as a point cloud.  
Depth images from Kinect v2 can be 8-bit (0-255) or 16-bit (0-65535) PNG images with the depth data in millimetres up to 4500 or 8000 mm, depending on how the data has been saved. The “Depth Basics” example provided in the Kinect SDK saves the images in a 8-bit PNG format. This means that the depth information is lost as it is saved in a grayscale intensity image. (Please see: Saving Kinect’s Depth image in 16-bit PNG format in C\#)  
Both of these types of images can be read into MATLAB and the resulting point cloud can be visualized in 3D. While the depth images can be loaded using normal surf() command in MATLAB to generate 3D surfaces, but it is not convenient if one wants to actually see the independent points in the point cloud. A much better method is by using the [fantastic plot3k() function][fantastic plot3k_ function], which basically is a 3D scatter plot of a Cartesian data set (in our case, the image). Not only is it faster for plotting larger point clouds, it also colour codes the points according to an upper and lower limit that can be specified. So lets go on and see how it can be done.

# Loading 8-BIT PNG depth images #

Since the 8-bit depth images can only store values upto 0-255, the actual depth data is lost when the image is saved. This means that since the maximum value that the Kinect can return is 8,000mm , to convert it into grayscale, the values are scaled to 256 parts. So each grayscale value represents 8000/255 = 31.25 mm of depth. So even though the actual depth value is lost when saving 8-bit depth images, the depth values can be re-calculated from the loaded image as follows:

MATLAB

<table> 
 <tbody> 
  <tr> 
   <td> <p>1</p> </td> 
   <td> <p>imgDepth = 31.25*uint16(imread('Q2.png'));</p> </td> 
  </tr> 
 </tbody> 
</table>

Note, that if only imread() is used, then the workspace variable will only be an 8-bit matrix, with maximum values upto 255. So it has to be converted into uint16 before multiplying.

# Loading 16-BIT PNG depth images #

For loading 16-bit depth images, the standard imread can be used to load the images.

MATLAB

<table> 
 <tbody> 
  <tr> 
   <td> <p>1</p> </td> 
   <td> <p>imgDepth = imread('TEST.png');</p> </td> 
  </tr> 
 </tbody> 
</table>

The loaded workspace variables can be seen in the picture below:

![format_png][]

# Plotting the depth data using plot3k() #

Once the images have been loaded, its time to visualize them in Matlab. Make sure that you have [downloaded the plot3k() file ][fantastic plot3k_ function]and placed it in the current working directory.  
First step is to create a figure and show the depth image that has been loaded:

MATLAB

<table> 
 <tbody> 
  <tr> 
   <td> <p>1</p> <p>2</p> <p>3</p> <p>4</p> <p>5</p> </td> 
   <td> <p>[m,n] = size(imgDepth);</p> <p>gcf = figure(1);</p> <p>subplot(4,1,1);</p> <p>imshow(imgDepth); % truesize;</p> <p>title('Input image');</p> </td> 
  </tr> 
 </tbody> 
</table>

Next we plot the depth image along with a depth scale on the right side. This will also color code the depth values according to the colormap sepected, which can be any one of the [default ones in matlab][].

MATLAB

<table> 
 <tbody> 
  <tr> 
   <td> <p>1</p> <p>2</p> <p>3</p> <p>4</p> <p>5</p> </td> 
   <td> <p>subplot(4,1,2);</p> <p>fig = imagesc(imgDepth); axis off; % truesize;</p> <p>title('Depth in color map');</p> <p>colormap parula;</p> <p>c = colorbar; c.Label.String = 'Distance in mm';</p> </td> 
  </tr> 
 </tbody> 
</table>

Next we use the plot3k() function to plot a 3D scatter plot of the depth image.

MATLAB

<table> 
 <tbody> 
  <tr> 
   <td> <p>1</p> <p>2</p> <p>3</p> </td> 
   <td> <p>subplot(4,1,3);</p> <p>plot3k(imgDepth); grid minor;</p> <p>title('Raw point cloud');</p> </td> 
  </tr> 
 </tbody> 
</table>

Lastly, it is always good to visualize the large amounts of depth data in a simpler and easily understandable format. I personally prefer Histogram of the depth data. This way, we can see where the maximum amount of the depth in the scene lies. Here I have used the depth upto 4000 mm (as the scene was relatively close to Kinect) but it can be varied as per your own scene depth.

MATLAB

<table> 
 <tbody> 
  <tr> 
   <td> <p>1</p> <p>2</p> <p>3</p> <p>4</p> <p>5</p> </td> 
   <td> <p>[counts,binLocations] = imhist(imgDepth,4000);</p> <p>binLocations = binLocations(1:4000,1);</p> <p>subplot(4,1,4);</p> <p>bar( binLocations,counts); xlim([450 4000])</p> <p>title('Histogram of depth values');</p> </td> 
  </tr> 
 </tbody> 
</table>

The final result can be seen in the following window.

[![format_png 1][]][format_png 1 1]

And the complete code can be downloaded [HERE ][HERE](including a 16-bit and 8-bit depth image).

[previous post]: http://www.atifanwer.xyz/blog/saving-kinect-depth-images-to-disk-in-matlab/
[fantastic plot3k_ function]: https://www.mathworks.com/matlabcentral/fileexchange/9519-color-coded-3d-scatterplot
[format_png]: https://imgconvert.csdnimg.cn/aHR0cDovL3d3dy5hdGlmYW53ZXIueHl6L3dwLWNvbnRlbnQvdXBsb2Fkcy8yMDE3LzA3LzE2Qml0LnBuZw?x-oss-process=image/format,png
[default ones in matlab]: https://www.mathworks.com/help/matlab/ref/colormap.html
[format_png 1]: https://imgconvert.csdnimg.cn/aHR0cDovL3d3dy5hdGlmYW53ZXIueHl6L3dwLWNvbnRlbnQvdXBsb2Fkcy8yMDE3LzA3L1BvaW50Q2xvdWQtMTYyeDMwMC5wbmc?x-oss-process=image/format,png
[format_png 1 1]: http://www.atifanwer.xyz/wp-content/uploads/2017/07/PointCloud.png
[HERE]: http://jmp.sh/I8Q3krg