Class Ptcloud

Method Details

• loadPointCloud

  public static void loadPointCloud(String filename, Mat vertices, Mat normals, Mat rgb)

  Loads a point cloud from a file. The function loads point cloud from the specified file and returns it. If the cloud cannot be read, throws an error. Vertex coordinates, normals and colors are returned as they are saved in the file even if these arrays have different sizes and their elements do not correspond to each other (which is typical for OBJ files for example) Currently, the following file formats are supported: - [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file) - [Polygon File Format *.ply](https://en.wikipedia.org/wiki/PLY_(file_format))

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

  normals - per-vertex normals, each value contains 3 floats

  rgb - per-vertex colors, each value contains 3 floats

• loadPointCloud

  public static void loadPointCloud(String filename, Mat vertices, Mat normals)

  Loads a point cloud from a file. The function loads point cloud from the specified file and returns it. If the cloud cannot be read, throws an error. Vertex coordinates, normals and colors are returned as they are saved in the file even if these arrays have different sizes and their elements do not correspond to each other (which is typical for OBJ files for example) Currently, the following file formats are supported: - [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file) - [Polygon File Format *.ply](https://en.wikipedia.org/wiki/PLY_(file_format))

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

  normals - per-vertex normals, each value contains 3 floats

• loadPointCloud

  public static void loadPointCloud(String filename, Mat vertices)

  Loads a point cloud from a file. The function loads point cloud from the specified file and returns it. If the cloud cannot be read, throws an error. Vertex coordinates, normals and colors are returned as they are saved in the file even if these arrays have different sizes and their elements do not correspond to each other (which is typical for OBJ files for example) Currently, the following file formats are supported: - [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file) - [Polygon File Format *.ply](https://en.wikipedia.org/wiki/PLY_(file_format))

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

• savePointCloud

  public static void savePointCloud(String filename, Mat vertices, Mat normals, Mat rgb)

  Saves a point cloud to a specified file. The function saves point cloud to the specified file. File format is chosen based on the filename extension.

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

  normals - per-vertex normals, each value contains 3 floats

  rgb - per-vertex colors, each value contains 3 floats

• savePointCloud

  public static void savePointCloud(String filename, Mat vertices, Mat normals)

  Saves a point cloud to a specified file. The function saves point cloud to the specified file. File format is chosen based on the filename extension.

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

  normals - per-vertex normals, each value contains 3 floats

• savePointCloud

  public static void savePointCloud(String filename, Mat vertices)

  Saves a point cloud to a specified file. The function saves point cloud to the specified file. File format is chosen based on the filename extension.

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

• loadMesh

  public static void loadMesh(String filename, Mat vertices, List<Mat> indices, Mat normals, Mat colors, Mat texCoords)

  Loads a mesh from a file. The function loads mesh from the specified file and returns it. If the mesh cannot be read, throws an error Vertex attributes (i.e. space and texture coodinates, normals and colors) are returned in same-sized arrays with corresponding elements having the same indices. This means that if a face uses a vertex with a normal or a texture coordinate with different indices (which is typical for OBJ files for example), this vertex will be duplicated for each face it uses. Currently, the following file formats are supported: - [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file) (ONLY TRIANGULATED FACES) - [Polygon File Format *.ply](https://en.wikipedia.org/wiki/PLY_(file_format))

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

  indices - per-face list of vertices, each value is a vector of ints

  normals - per-vertex normals, each value contains 3 floats

  colors - per-vertex colors, each value contains 3 floats

  texCoords - per-vertex texture coordinates, each value contains 2 or 3 floats

• loadMesh

  public static void loadMesh(String filename, Mat vertices, List<Mat> indices, Mat normals, Mat colors)

  Loads a mesh from a file. The function loads mesh from the specified file and returns it. If the mesh cannot be read, throws an error Vertex attributes (i.e. space and texture coodinates, normals and colors) are returned in same-sized arrays with corresponding elements having the same indices. This means that if a face uses a vertex with a normal or a texture coordinate with different indices (which is typical for OBJ files for example), this vertex will be duplicated for each face it uses. Currently, the following file formats are supported: - [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file) (ONLY TRIANGULATED FACES) - [Polygon File Format *.ply](https://en.wikipedia.org/wiki/PLY_(file_format))

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

  indices - per-face list of vertices, each value is a vector of ints

  normals - per-vertex normals, each value contains 3 floats

  colors - per-vertex colors, each value contains 3 floats

• loadMesh

  public static void loadMesh(String filename, Mat vertices, List<Mat> indices, Mat normals)

  Loads a mesh from a file. The function loads mesh from the specified file and returns it. If the mesh cannot be read, throws an error Vertex attributes (i.e. space and texture coodinates, normals and colors) are returned in same-sized arrays with corresponding elements having the same indices. This means that if a face uses a vertex with a normal or a texture coordinate with different indices (which is typical for OBJ files for example), this vertex will be duplicated for each face it uses. Currently, the following file formats are supported: - [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file) (ONLY TRIANGULATED FACES) - [Polygon File Format *.ply](https://en.wikipedia.org/wiki/PLY_(file_format))

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

  indices - per-face list of vertices, each value is a vector of ints

  normals - per-vertex normals, each value contains 3 floats

• loadMesh

  public static void loadMesh(String filename, Mat vertices, List<Mat> indices)

  Loads a mesh from a file. The function loads mesh from the specified file and returns it. If the mesh cannot be read, throws an error Vertex attributes (i.e. space and texture coodinates, normals and colors) are returned in same-sized arrays with corresponding elements having the same indices. This means that if a face uses a vertex with a normal or a texture coordinate with different indices (which is typical for OBJ files for example), this vertex will be duplicated for each face it uses. Currently, the following file formats are supported: - [Wavefront obj file *.obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file) (ONLY TRIANGULATED FACES) - [Polygon File Format *.ply](https://en.wikipedia.org/wiki/PLY_(file_format))

  Parameters:

  filename - Name of the file

  vertices - vertex coordinates, each value contains 3 floats

  indices - per-face list of vertices, each value is a vector of ints

• saveMesh

  public static void saveMesh(String filename, Mat vertices, List<Mat> indices, Mat normals, Mat colors, Mat texCoords)

  Saves a mesh to a specified file. The function saves mesh to the specified file. File format is chosen based on the filename extension.

  Parameters:

  filename - Name of the file.

  vertices - vertex coordinates, each value contains 3 floats

  indices - per-face list of vertices, each value is a vector of ints

  normals - per-vertex normals, each value contains 3 floats

  colors - per-vertex colors, each value contains 3 floats

  texCoords - per-vertex texture coordinates, each value contains 2 or 3 floats

• saveMesh

  public static void saveMesh(String filename, Mat vertices, List<Mat> indices, Mat normals, Mat colors)

  Saves a mesh to a specified file. The function saves mesh to the specified file. File format is chosen based on the filename extension.

  Parameters:

  filename - Name of the file.

  vertices - vertex coordinates, each value contains 3 floats

  indices - per-face list of vertices, each value is a vector of ints

  normals - per-vertex normals, each value contains 3 floats

  colors - per-vertex colors, each value contains 3 floats

• saveMesh

  public static void saveMesh(String filename, Mat vertices, List<Mat> indices, Mat normals)

  Saves a mesh to a specified file. The function saves mesh to the specified file. File format is chosen based on the filename extension.

  Parameters:

  filename - Name of the file.

  vertices - vertex coordinates, each value contains 3 floats

  indices - per-face list of vertices, each value is a vector of ints

  normals - per-vertex normals, each value contains 3 floats

• saveMesh

  public static void saveMesh(String filename, Mat vertices, List<Mat> indices)

  Saves a mesh to a specified file. The function saves mesh to the specified file. File format is chosen based on the filename extension.

  Parameters:

  filename - Name of the file.

  vertices - vertex coordinates, each value contains 3 floats

  indices - per-face list of vertices, each value is a vector of ints

• triangleRasterize

  public static void triangleRasterize(Mat vertices, Mat indices, Mat colors, Mat colorBuf, Mat depthBuf, Mat world2cam, double fovY, double zNear, double zFar, TriangleRasterizeSettings settings)

  Renders a set of triangles on a depth and color image Triangles can be drawn white (1.0, 1.0, 1.0), flat-shaded or with a color interpolation between vertices. In flat-shaded mode the 1st vertex color of each triangle is used to fill the whole triangle. The world2cam is an inverted camera pose matrix in fact. It transforms vertices from world to camera coordinate system. The camera coordinate system emulates the OpenGL's coordinate system having coordinate origin in a screen center, X axis pointing right, Y axis pointing up and Z axis pointing towards the viewer except that image is vertically flipped after the render. This means that all visible objects are placed in z-negative area, or exactly in -zNear > z > -zFar since zNear and zFar are positive. For example, at fovY = PI/2 the point (0, 1, -1) will be projected to (width/2, 0) screen point, (1, 0, -1) to (width/2 + height/2, height/2). Increasing fovY makes projection smaller and vice versa. The function does not create or clear output images before the rendering. This means that it can be used for drawing over an existing image or for rendering a model into a 3D scene using pre-filled Z-buffer. Empty scene results in a depth buffer filled by the maximum value since every pixel is infinitely far from the camera. Therefore, before rendering anything from scratch the depthBuf should be filled by zFar values (or by ones in INVDEPTH mode). There are special versions of this function named triangleRasterizeDepth and triangleRasterizeColor for cases if a user needs a color image or a depth image alone; they may run slightly faster.

  Parameters:

  vertices - vertices coordinates array. Should contain values of CV_32FC3 type or a compatible one (e.g. cv::Vec3f, etc.)

  indices - triangle vertices index array, 3 per triangle. Each index indicates a vertex in a vertices array. Should contain CV_32SC3 values or compatible

  colors - per-vertex colors of CV_32FC3 type or compatible. Can be empty or the same size as vertices array. If the values are out of [0; 1] range, the result correctness is not guaranteed

  colorBuf - an array representing the final rendered image. Should containt CV_32FC3 values and be the same size as depthBuf. Not cleared before rendering, i.e. the content is reused as there is some pre-rendered scene.

  depthBuf - an array of floats containing resulting Z buffer. Should contain float values and be the same size as colorBuf. Not cleared before rendering, i.e. the content is reused as there is some pre-rendered scene. Empty scene corresponds to all values set to zFar (or to 1.0 in INVDEPTH mode)

  world2cam - a 4x3 or 4x4 float or double matrix containing inverted (sic!) camera pose

  fovY - field of view in vertical direction, given in radians

  zNear - minimum Z value to render, everything closer is clipped

  zFar - maximum Z value to render, everything farther is clipped

  settings - see TriangleRasterizeSettings. By default the smooth shading is on, with CW culling and with disabled GL compatibility

• triangleRasterize

  public static void triangleRasterize(Mat vertices, Mat indices, Mat colors, Mat colorBuf, Mat depthBuf, Mat world2cam, double fovY, double zNear, double zFar)

  Renders a set of triangles on a depth and color image Triangles can be drawn white (1.0, 1.0, 1.0), flat-shaded or with a color interpolation between vertices. In flat-shaded mode the 1st vertex color of each triangle is used to fill the whole triangle. The world2cam is an inverted camera pose matrix in fact. It transforms vertices from world to camera coordinate system. The camera coordinate system emulates the OpenGL's coordinate system having coordinate origin in a screen center, X axis pointing right, Y axis pointing up and Z axis pointing towards the viewer except that image is vertically flipped after the render. This means that all visible objects are placed in z-negative area, or exactly in -zNear > z > -zFar since zNear and zFar are positive. For example, at fovY = PI/2 the point (0, 1, -1) will be projected to (width/2, 0) screen point, (1, 0, -1) to (width/2 + height/2, height/2). Increasing fovY makes projection smaller and vice versa. The function does not create or clear output images before the rendering. This means that it can be used for drawing over an existing image or for rendering a model into a 3D scene using pre-filled Z-buffer. Empty scene results in a depth buffer filled by the maximum value since every pixel is infinitely far from the camera. Therefore, before rendering anything from scratch the depthBuf should be filled by zFar values (or by ones in INVDEPTH mode). There are special versions of this function named triangleRasterizeDepth and triangleRasterizeColor for cases if a user needs a color image or a depth image alone; they may run slightly faster.

  Parameters:

  vertices - vertices coordinates array. Should contain values of CV_32FC3 type or a compatible one (e.g. cv::Vec3f, etc.)

  indices - triangle vertices index array, 3 per triangle. Each index indicates a vertex in a vertices array. Should contain CV_32SC3 values or compatible

  colors - per-vertex colors of CV_32FC3 type or compatible. Can be empty or the same size as vertices array. If the values are out of [0; 1] range, the result correctness is not guaranteed

  colorBuf - an array representing the final rendered image. Should containt CV_32FC3 values and be the same size as depthBuf. Not cleared before rendering, i.e. the content is reused as there is some pre-rendered scene.

  depthBuf - an array of floats containing resulting Z buffer. Should contain float values and be the same size as colorBuf. Not cleared before rendering, i.e. the content is reused as there is some pre-rendered scene. Empty scene corresponds to all values set to zFar (or to 1.0 in INVDEPTH mode)

  world2cam - a 4x3 or 4x4 float or double matrix containing inverted (sic!) camera pose

  fovY - field of view in vertical direction, given in radians

  zNear - minimum Z value to render, everything closer is clipped

  zFar - maximum Z value to render, everything farther is clipped with CW culling and with disabled GL compatibility

• triangleRasterizeDepth

  public static void triangleRasterizeDepth(Mat vertices, Mat indices, Mat depthBuf, Mat world2cam, double fovY, double zNear, double zFar, TriangleRasterizeSettings settings)

  Overloaded version of triangleRasterize() with depth-only rendering

  Parameters:

  vertices - vertices coordinates array. Should contain values of CV_32FC3 type or a compatible one (e.g. cv::Vec3f, etc.)

  indices - triangle vertices index array, 3 per triangle. Each index indicates a vertex in a vertices array. Should contain CV_32SC3 values or compatible

  depthBuf - an array of floats containing resulting Z buffer. Should contain float values and be the same size as colorBuf. Not cleared before rendering, i.e. the content is reused as there is some pre-rendered scene. Empty scene corresponds to all values set to zFar (or to 1.0 in INVDEPTH mode)

  world2cam - a 4x3 or 4x4 float or double matrix containing inverted (sic!) camera pose

  fovY - field of view in vertical direction, given in radians

  zNear - minimum Z value to render, everything closer is clipped

  zFar - maximum Z value to render, everything farther is clipped

  settings - see TriangleRasterizeSettings. By default the smooth shading is on, with CW culling and with disabled GL compatibility

• triangleRasterizeDepth

  public static void triangleRasterizeDepth(Mat vertices, Mat indices, Mat depthBuf, Mat world2cam, double fovY, double zNear, double zFar)

  Overloaded version of triangleRasterize() with depth-only rendering

  Parameters:

  vertices - vertices coordinates array. Should contain values of CV_32FC3 type or a compatible one (e.g. cv::Vec3f, etc.)

  indices - triangle vertices index array, 3 per triangle. Each index indicates a vertex in a vertices array. Should contain CV_32SC3 values or compatible

  depthBuf - an array of floats containing resulting Z buffer. Should contain float values and be the same size as colorBuf. Not cleared before rendering, i.e. the content is reused as there is some pre-rendered scene. Empty scene corresponds to all values set to zFar (or to 1.0 in INVDEPTH mode)

  world2cam - a 4x3 or 4x4 float or double matrix containing inverted (sic!) camera pose

  fovY - field of view in vertical direction, given in radians

  zNear - minimum Z value to render, everything closer is clipped

  zFar - maximum Z value to render, everything farther is clipped with CW culling and with disabled GL compatibility

• triangleRasterizeColor

  public static void triangleRasterizeColor(Mat vertices, Mat indices, Mat colors, Mat colorBuf, Mat world2cam, double fovY, double zNear, double zFar, TriangleRasterizeSettings settings)

  Overloaded version of triangleRasterize() with color-only rendering

  Parameters:

  vertices - vertices coordinates array. Should contain values of CV_32FC3 type or a compatible one (e.g. cv::Vec3f, etc.)

  indices - triangle vertices index array, 3 per triangle. Each index indicates a vertex in a vertices array. Should contain CV_32SC3 values or compatible

  colors - per-vertex colors of CV_32FC3 type or compatible. Can be empty or the same size as vertices array. If the values are out of [0; 1] range, the result correctness is not guaranteed

  colorBuf - an array representing the final rendered image. Should containt CV_32FC3 values and be the same size as depthBuf. Not cleared before rendering, i.e. the content is reused as there is some pre-rendered scene.

  world2cam - a 4x3 or 4x4 float or double matrix containing inverted (sic!) camera pose

  fovY - field of view in vertical direction, given in radians

  zNear - minimum Z value to render, everything closer is clipped

  zFar - maximum Z value to render, everything farther is clipped

  settings - see TriangleRasterizeSettings. By default the smooth shading is on, with CW culling and with disabled GL compatibility

• triangleRasterizeColor

  public static void triangleRasterizeColor(Mat vertices, Mat indices, Mat colors, Mat colorBuf, Mat world2cam, double fovY, double zNear, double zFar)

  Overloaded version of triangleRasterize() with color-only rendering

  Parameters:

  vertices - vertices coordinates array. Should contain values of CV_32FC3 type or a compatible one (e.g. cv::Vec3f, etc.)

  indices - triangle vertices index array, 3 per triangle. Each index indicates a vertex in a vertices array. Should contain CV_32SC3 values or compatible

  colors - per-vertex colors of CV_32FC3 type or compatible. Can be empty or the same size as vertices array. If the values are out of [0; 1] range, the result correctness is not guaranteed

  colorBuf - an array representing the final rendered image. Should containt CV_32FC3 values and be the same size as depthBuf. Not cleared before rendering, i.e. the content is reused as there is some pre-rendered scene.

  world2cam - a 4x3 or 4x4 float or double matrix containing inverted (sic!) camera pose

  fovY - field of view in vertical direction, given in radians

  zNear - minimum Z value to render, everything closer is clipped

  zFar - maximum Z value to render, everything farther is clipped with CW culling and with disabled GL compatibility

• registerDepth

  public static void registerDepth(Mat unregisteredCameraMatrix, Mat registeredCameraMatrix, Mat registeredDistCoeffs, Mat Rt, Mat unregisteredDepth, Size outputImagePlaneSize, Mat registeredDepth, boolean depthDilation)

  Registers depth data to an external camera Registration is performed by creating a depth cloud, transforming the cloud by the rigid body transformation between the cameras, and then projecting the transformed points into the RGB camera. uv_rgb = K_rgb * [R | t] * z * inv(K_ir) * uv_ir Currently does not check for negative depth values.

  Parameters:

  unregisteredCameraMatrix - the camera matrix of the depth camera

  registeredCameraMatrix - the camera matrix of the external camera

  registeredDistCoeffs - the distortion coefficients of the external camera

  Rt - the rigid body transform between the cameras. Transforms points from depth camera frame to external camera frame.

  unregisteredDepth - the input depth data

  outputImagePlaneSize - the image plane dimensions of the external camera (width, height)

  registeredDepth - the result of transforming the depth into the external camera

  depthDilation - whether or not the depth is dilated to avoid holes and occlusion errors (optional)

• registerDepth

  public static void registerDepth(Mat unregisteredCameraMatrix, Mat registeredCameraMatrix, Mat registeredDistCoeffs, Mat Rt, Mat unregisteredDepth, Size outputImagePlaneSize, Mat registeredDepth)

  Registers depth data to an external camera Registration is performed by creating a depth cloud, transforming the cloud by the rigid body transformation between the cameras, and then projecting the transformed points into the RGB camera. uv_rgb = K_rgb * [R | t] * z * inv(K_ir) * uv_ir Currently does not check for negative depth values.

  Parameters:

  unregisteredCameraMatrix - the camera matrix of the depth camera

  registeredCameraMatrix - the camera matrix of the external camera

  registeredDistCoeffs - the distortion coefficients of the external camera

  Rt - the rigid body transform between the cameras. Transforms points from depth camera frame to external camera frame.

  unregisteredDepth - the input depth data

  outputImagePlaneSize - the image plane dimensions of the external camera (width, height)

  registeredDepth - the result of transforming the depth into the external camera

• depthTo3dSparse

  public static void depthTo3dSparse(Mat depth, Mat in_K, Mat in_points, Mat points3d)

  Parameters:

  depth - the depth image

  in_K -

  in_points - the list of xy coordinates

  points3d - the resulting 3d points (point is represented by 4 chanels value [x, y, z, 0])

• depthTo3d

  public static void depthTo3d(Mat depth, Mat K, Mat points3d, Mat mask)

  Converts a depth image to 3d points. If the mask is empty then the resulting array has the same dimensions as depth, otherwise it is 1d vector containing mask-enabled values only. The coordinate system is x pointing left, y down and z away from the camera

  Parameters:

  depth - the depth image (if given as short int CV_U, it is assumed to be the depth in millimeters (as done with the Microsoft Kinect), otherwise, if given as CV_32F or CV_64F, it is assumed in meters)

  K - The calibration matrix

  points3d - the resulting 3d points (point is represented by 4 channels value [x, y, z, 0]). They are of the same depth as depth if it is CV_32F or CV_64F, and the depth of K if depth is of depth CV_16U or CV_16S

  mask - the mask of the points to consider (can be empty)

• depthTo3d

  public static void depthTo3d(Mat depth, Mat K, Mat points3d)

  Converts a depth image to 3d points. If the mask is empty then the resulting array has the same dimensions as depth, otherwise it is 1d vector containing mask-enabled values only. The coordinate system is x pointing left, y down and z away from the camera

  Parameters:

  depth - the depth image (if given as short int CV_U, it is assumed to be the depth in millimeters (as done with the Microsoft Kinect), otherwise, if given as CV_32F or CV_64F, it is assumed in meters)

  K - The calibration matrix

  points3d - the resulting 3d points (point is represented by 4 channels value [x, y, z, 0]). They are of the same depth as depth if it is CV_32F or CV_64F, and the depth of K if depth is of depth CV_16U or CV_16S

• rescaleDepth

  public static void rescaleDepth(Mat in, int type, Mat out, double depth_factor)

  If the input image is of type CV_16UC1 (like the Kinect one), the image is converted to floats, divided by depth_factor to get a depth in meters, and the values 0 are converted to std::numeric_limits<float>::quiet_NaN() Otherwise, the image is simply converted to floats

  Parameters:

  in - the depth image (if given as short int CV_U, it is assumed to be the depth in millimeters (as done with the Microsoft Kinect), it is assumed in meters)

  type - the desired output depth (CV_32F or CV_64F)

  out - The rescaled float depth image

  depth_factor - (optional) factor by which depth is converted to distance (by default = 1000.0 for Kinect sensor)

• rescaleDepth

  public static void rescaleDepth(Mat in, int type, Mat out)

  If the input image is of type CV_16UC1 (like the Kinect one), the image is converted to floats, divided by depth_factor to get a depth in meters, and the values 0 are converted to std::numeric_limits<float>::quiet_NaN() Otherwise, the image is simply converted to floats

  Parameters:

  in - the depth image (if given as short int CV_U, it is assumed to be the depth in millimeters (as done with the Microsoft Kinect), it is assumed in meters)

  type - the desired output depth (CV_32F or CV_64F)

  out - The rescaled float depth image

• warpFrame

  public static void warpFrame(Mat depth, Mat image, Mat mask, Mat Rt, Mat cameraMatrix, Mat warpedDepth, Mat warpedImage, Mat warpedMask)

  Warps depth or RGB-D image by reprojecting it in 3d, applying Rt transformation and then projecting it back onto the image plane. This function can be used to visualize the results of the Odometry algorithm.

  Parameters:

  depth - Depth data, should be 1-channel CV_16U, CV_16S, CV_32F or CV_64F

  image - RGB image (optional), should be 1-, 3- or 4-channel CV_8U

  mask - Mask of used pixels (optional), should be CV_8UC1, CV_8SC1 or CV_BoolC1

  Rt - Rotation+translation matrix (3x4 or 4x4) to be applied to depth points

  cameraMatrix - Camera intrinsics matrix (3x3)

  warpedDepth - The warped depth data (optional)

  warpedImage - The warped RGB image (optional)

  warpedMask - The mask of valid pixels in warped image (optional)

• warpFrame

  public static void warpFrame(Mat depth, Mat image, Mat mask, Mat Rt, Mat cameraMatrix, Mat warpedDepth, Mat warpedImage)

  Warps depth or RGB-D image by reprojecting it in 3d, applying Rt transformation and then projecting it back onto the image plane. This function can be used to visualize the results of the Odometry algorithm.

  Parameters:

  depth - Depth data, should be 1-channel CV_16U, CV_16S, CV_32F or CV_64F

  image - RGB image (optional), should be 1-, 3- or 4-channel CV_8U

  mask - Mask of used pixels (optional), should be CV_8UC1, CV_8SC1 or CV_BoolC1

  Rt - Rotation+translation matrix (3x4 or 4x4) to be applied to depth points

  cameraMatrix - Camera intrinsics matrix (3x3)

  warpedDepth - The warped depth data (optional)

  warpedImage - The warped RGB image (optional)

• warpFrame

  public static void warpFrame(Mat depth, Mat image, Mat mask, Mat Rt, Mat cameraMatrix, Mat warpedDepth)

  Warps depth or RGB-D image by reprojecting it in 3d, applying Rt transformation and then projecting it back onto the image plane. This function can be used to visualize the results of the Odometry algorithm.

  Parameters:

  depth - Depth data, should be 1-channel CV_16U, CV_16S, CV_32F or CV_64F

  image - RGB image (optional), should be 1-, 3- or 4-channel CV_8U

  mask - Mask of used pixels (optional), should be CV_8UC1, CV_8SC1 or CV_BoolC1

  Rt - Rotation+translation matrix (3x4 or 4x4) to be applied to depth points

  cameraMatrix - Camera intrinsics matrix (3x3)

  warpedDepth - The warped depth data (optional)

• warpFrame

  public static void warpFrame(Mat depth, Mat image, Mat mask, Mat Rt, Mat cameraMatrix)

  Warps depth or RGB-D image by reprojecting it in 3d, applying Rt transformation and then projecting it back onto the image plane. This function can be used to visualize the results of the Odometry algorithm.

  Parameters:

  depth - Depth data, should be 1-channel CV_16U, CV_16S, CV_32F or CV_64F

  image - RGB image (optional), should be 1-, 3- or 4-channel CV_8U

  mask - Mask of used pixels (optional), should be CV_8UC1, CV_8SC1 or CV_BoolC1

  Rt - Rotation+translation matrix (3x4 or 4x4) to be applied to depth points

  cameraMatrix - Camera intrinsics matrix (3x3)

• findPlanes

  public static void findPlanes(Mat points3d, Mat normals, Mat mask, Mat plane_coefficients, int block_size, int min_size, double threshold, double sensor_error_a, double sensor_error_b, double sensor_error_c, int method)

  Find the planes in a depth image

  Parameters:

  points3d - the 3d points organized like the depth image: rows x cols with 3 channels

  normals - the normals for every point in the depth image; optional, can be empty

  mask - An image where each pixel is labeled with the plane it belongs to and 255 if it does not belong to any plane

  plane_coefficients - the coefficients of the corresponding planes (a,b,c,d) such that ax+by+cz+d=0, norm(a,b,c)=1 and c < 0 (so that the normal points towards the camera)

  block_size - The size of the blocks to look at for a stable MSE

  min_size - The minimum size of a cluster to be considered a plane

  threshold - The maximum distance of a point from a plane to belong to it (in meters)

  sensor_error_a - coefficient of the sensor error. 0 by default, use 0.0075 for a Kinect

  sensor_error_b - coefficient of the sensor error. 0 by default

  sensor_error_c - coefficient of the sensor error. 0 by default

  method - The method to use to compute the planes.

• findPlanes

  public static void findPlanes(Mat points3d, Mat normals, Mat mask, Mat plane_coefficients, int block_size, int min_size, double threshold, double sensor_error_a, double sensor_error_b, double sensor_error_c)

  Find the planes in a depth image

  Parameters:

  points3d - the 3d points organized like the depth image: rows x cols with 3 channels

  normals - the normals for every point in the depth image; optional, can be empty

  mask - An image where each pixel is labeled with the plane it belongs to and 255 if it does not belong to any plane

  plane_coefficients - the coefficients of the corresponding planes (a,b,c,d) such that ax+by+cz+d=0, norm(a,b,c)=1 and c < 0 (so that the normal points towards the camera)

  block_size - The size of the blocks to look at for a stable MSE

  min_size - The minimum size of a cluster to be considered a plane

  threshold - The maximum distance of a point from a plane to belong to it (in meters)

  sensor_error_a - coefficient of the sensor error. 0 by default, use 0.0075 for a Kinect

  sensor_error_b - coefficient of the sensor error. 0 by default

  sensor_error_c - coefficient of the sensor error. 0 by default

• findPlanes

  public static void findPlanes(Mat points3d, Mat normals, Mat mask, Mat plane_coefficients, int block_size, int min_size, double threshold, double sensor_error_a, double sensor_error_b)

  Find the planes in a depth image

  Parameters:

  points3d - the 3d points organized like the depth image: rows x cols with 3 channels

  normals - the normals for every point in the depth image; optional, can be empty

  mask - An image where each pixel is labeled with the plane it belongs to and 255 if it does not belong to any plane

  plane_coefficients - the coefficients of the corresponding planes (a,b,c,d) such that ax+by+cz+d=0, norm(a,b,c)=1 and c < 0 (so that the normal points towards the camera)

  block_size - The size of the blocks to look at for a stable MSE

  min_size - The minimum size of a cluster to be considered a plane

  threshold - The maximum distance of a point from a plane to belong to it (in meters)

  sensor_error_a - coefficient of the sensor error. 0 by default, use 0.0075 for a Kinect

  sensor_error_b - coefficient of the sensor error. 0 by default

• findPlanes

  public static void findPlanes(Mat points3d, Mat normals, Mat mask, Mat plane_coefficients, int block_size, int min_size, double threshold, double sensor_error_a)

  Find the planes in a depth image

  Parameters:

  points3d - the 3d points organized like the depth image: rows x cols with 3 channels

  normals - the normals for every point in the depth image; optional, can be empty

  mask - An image where each pixel is labeled with the plane it belongs to and 255 if it does not belong to any plane

  plane_coefficients - the coefficients of the corresponding planes (a,b,c,d) such that ax+by+cz+d=0, norm(a,b,c)=1 and c < 0 (so that the normal points towards the camera)

  block_size - The size of the blocks to look at for a stable MSE

  min_size - The minimum size of a cluster to be considered a plane

  threshold - The maximum distance of a point from a plane to belong to it (in meters)

  sensor_error_a - coefficient of the sensor error. 0 by default, use 0.0075 for a Kinect

• findPlanes

  public static void findPlanes(Mat points3d, Mat normals, Mat mask, Mat plane_coefficients, int block_size, int min_size, double threshold)

  Find the planes in a depth image

  Parameters:

  points3d - the 3d points organized like the depth image: rows x cols with 3 channels

  normals - the normals for every point in the depth image; optional, can be empty

  mask - An image where each pixel is labeled with the plane it belongs to and 255 if it does not belong to any plane

  plane_coefficients - the coefficients of the corresponding planes (a,b,c,d) such that ax+by+cz+d=0, norm(a,b,c)=1 and c < 0 (so that the normal points towards the camera)

  block_size - The size of the blocks to look at for a stable MSE

  min_size - The minimum size of a cluster to be considered a plane

  threshold - The maximum distance of a point from a plane to belong to it (in meters)

• findPlanes

  public static void findPlanes(Mat points3d, Mat normals, Mat mask, Mat plane_coefficients, int block_size, int min_size)

  Find the planes in a depth image

  Parameters:

  points3d - the 3d points organized like the depth image: rows x cols with 3 channels

  normals - the normals for every point in the depth image; optional, can be empty

  mask - An image where each pixel is labeled with the plane it belongs to and 255 if it does not belong to any plane

  plane_coefficients - the coefficients of the corresponding planes (a,b,c,d) such that ax+by+cz+d=0, norm(a,b,c)=1 and c < 0 (so that the normal points towards the camera)

  block_size - The size of the blocks to look at for a stable MSE

  min_size - The minimum size of a cluster to be considered a plane

• findPlanes

  public static void findPlanes(Mat points3d, Mat normals, Mat mask, Mat plane_coefficients, int block_size)

  Find the planes in a depth image

  Parameters:

  points3d - the 3d points organized like the depth image: rows x cols with 3 channels

  normals - the normals for every point in the depth image; optional, can be empty

  mask - An image where each pixel is labeled with the plane it belongs to and 255 if it does not belong to any plane

  plane_coefficients - the coefficients of the corresponding planes (a,b,c,d) such that ax+by+cz+d=0, norm(a,b,c)=1 and c < 0 (so that the normal points towards the camera)

  block_size - The size of the blocks to look at for a stable MSE

• findPlanes

  public static void findPlanes(Mat points3d, Mat normals, Mat mask, Mat plane_coefficients)

  Find the planes in a depth image

  Parameters:

  points3d - the 3d points organized like the depth image: rows x cols with 3 channels

  normals - the normals for every point in the depth image; optional, can be empty

  mask - An image where each pixel is labeled with the plane it belongs to and 255 if it does not belong to any plane

  plane_coefficients - the coefficients of the corresponding planes (a,b,c,d) such that ax+by+cz+d=0, norm(a,b,c)=1 and c < 0 (so that the normal points towards the camera)