Intrinsic Parameters

Use Cases

Point Cloud Calculation from Depth Map

Depth map is a basic output calculated by Photoneo devices, transferred to PhoXi Control, and used for point cloud calculation. In some applications, the users might calculate the point cloud from the depth map by themselves and not rely on the calculation done by PhoXi Control. Some information about how the 2D camera “sees” the world is necessary for this procedure. This information is carried either by intrinsic camera matrix or by reprojection map.

In Photoneo scanning devices the texture and depth map have a direct relationship → each pixel in the texture has corresponding depth information in the depth map. Therefore operations like segmenting out the only object of interest in the texture and then calculating only the 3D points that were segmented are possible.

The point cloud can be calculated from depth map using these approaches:

• Using intrinsic camera matrix and triangle similarity described in pinhole camera model together with distortion coefficients to calculate point cloud for each pixel

• Using reprojection map to multiply the depth of given pixel by a reprojection vector corresponding to that pixel

Image Processing on Undistorted Data

Texture, ColorCameraImage and a DepthMap from Photoneo devices are distorted. This means that straight lines at some parts of the picture appear bent or round-shaped objects appear to be oval-shaped in the texture. Therefore image processing for such geometric shapes is not possible on distorted data and we have to undistort them first.

To undistort, we first apply the inverse of distortion function to each pixel, for example by using the OpenCV method. Once we have the undistorted texture, we can identify the feature points of our object. Then there are two possibilities:

• To undistort the depth map as well and use the intrinsic parameters to calculate a new, undistorted point cloud

• Distort the feature points back to see to which camera pixels they belong and then use this information to identify them in the point cloud

Intrinsic Parameters Provided by Photoneo Devices

All provided intrinsic parameters can be retrieved by PhoXi API or via Chunk Data Control in GigEV protocol on devices with FW 1.14.0 or higher. There are two different types of intrinsic parameters:

• Intrinsic camera matrix and distortion coefficients

• Reprojection map

All available intrinsic parameters can be now retrieved via PhoXi API through corresponding pho::api::PhoXiCoordinatesSettings class members:

• CurrentCamera: Current (effective) camera settings; depends on CameraSpace setting selector.

• CurrentPrimaryCamera: Parameters of the camera which the original depth map (before reprojection) was created with.

• CurrentColorCamera: Parameters of the color camera with which the color texture was created.

Intrinsic Camera Matrix and Distortion Coefficients

Intrinsic camera matrix describes the transformation from 3D coordinates to 2D coordinates on an image plane using the pinhole camera model. It is a 3x3 matrix containing focal length - the distance between the pinhole and the image plane and principal point offset - the pixel offset from the principal point.

Distortion coefficients describe the properties of the camera lens and how it distorts the real shape of an object to the shape on the image. OpenCV compatible coefficients (k1, k2, p1, p2, k3) are used to describe the distortion introduced by the camera lens in Photoneo devices.

• Intrinsic camera matrix and distortion coefficient can be retrieved using PhoXi API by two different approaches:

• Global method: PhoXiDevice → CalibrationSettings:

  • provides the Camera Matrix and Distortion Coefficients at maximum device resolution:

    • PhoXi 3D Scanner → 2064x1544

    • MotionCam-3D → Scanner mode 1680x1200

• PerFrame method: FrameInfo → Camera

  • provides the Camera Matrix and Distortion Coefficients for current device resolution

    • PhoXi 3D Scanner → 2064x1544 or 1032x772

    • MotionCam-3D → respects current Operation mode and Output topology. Empty for Raw and Irregular grid topologies

Global Method

See GetISCalibParams API example:

void printCalibParams(pho::api::PPhoXi &PhoXiDevice)
{
    pho::api::PhoXiCalibrationSettings CalibrationSettings = PhoXiDevice->CalibrationSettings;
    std::cout << "CalibrationSettings: " << std::endl;
    std::cout << " FocusLength: " << CalibrationSettings.FocusLength << std::endl;
    std::cout << " PixelSize: " << CalibrationSettings.PixelSize.Width << " x " << CalibrationSettings.PixelSize.Height << std::endl;

    printMatrix("CameraMatrix", CalibrationSettings.CameraMatrix);
    std::cout << " DistortionCoefficients: " << std::endl;
    std::cout << " Format is the following: " << std::endl;
    std::cout << " (k1, k2, p1, p2[, k3[, k4, k5, k6[, s1, s2, s3, s4[, tx, ty]]]])" << std::endl;

    std::vector<double> distCoeffs = CalibrationSettings.DistortionCoefficients;
    std::stringstream currentDistCoeffsSS;
    std::size_t brackets = 0;

    currentDistCoeffsSS << "(";
    currentDistCoeffsSS << distCoeffs[0];
    for (std::size_t i = 1; i < distCoeffs.size(); ++i)
    {
        if (i == 4 || i == 5 || i == 8 || i == 12 || i == 14)
        {
            currentDistCoeffsSS << "[";
            ++brackets;
        }
        currentDistCoeffsSS << ", " << distCoeffs[i];
    }

    for (std::size_t j = 0; j < brackets; ++j)
    {
        currentDistCoeffsSS << "]";
    }
    currentDistCoeffsSS << ")";

    std::cout << " " << currentDistCoeffsSS.str() << std::endl;
}

Per Frame Method

See FullAPI example:

void FullAPIExample::PrintFrameInfo(const pho::api::PFrame &Frame)
{
    const pho::api::FrameInfo &FrameInfo = Frame->Info;
    std::cout << " Frame params: " << std::endl;
    std::cout << " Frame Index: " << FrameInfo.FrameIndex << std::endl;
    std::cout << " Frame Timestamp: " << FrameInfo.FrameTimestamp << " s" << std::endl;
    std::cout << " Frame Acquisition duration: " << FrameInfo.FrameDuration << " ms" << std::endl;
    std::cout << " Frame Computation duration: " << FrameInfo.FrameComputationDuration << " ms" << std::endl;
    std::cout << " Frame Transfer duration: " << FrameInfo.FrameTransferDuration << " ms" << std::endl;
    std::cout << " Sensor Position: ["
        << FrameInfo.SensorPosition.x << "; "
        << FrameInfo.SensorPosition.y << "; "
        << FrameInfo.SensorPosition.z << "]"
        << std::endl;

    PrintMatrix("Camera calibration matrix", FrameInfo.CameraMatrix);
    PrintDistortionCoefficients("Frame Distortion Coefficients", FrameInfo.DistortionCoefficients);
    std::cout << " Camera binning height: " << FrameInfo.CameraBinning.Height << std::endl;
    std::cout << " Camera binning width: " << FrameInfo.CameraBinning.Width << std::endl;
    std::cout << " Total scan count: " << FrameInfo.TotalScanCount << std::endl;
}

Reprojection Map

Reprojection map is a field of vectors that describes the direction from which each pixel gathers data. Each pixel has its own vector, therefore when we multiply the depth information by the vector from the reprojection map we get a point cloud.

Reprojection map can only be retrieved by one method:

• Global method: PhoXiDevice → ReprojectionMap

  • Provides current reprojection map that respect current device settings

    • PhoXi 3D Scanner → both resolutions

    • MotionCam-3D both operations modes and all output topologies

Global Method

See PointCloudCalculation API example:

bool calculatePointCloud(pho::api::PointCloud32f& pointCloud,
                         pho::api::DepthMap32f depth,
                         pho::api::PhoXiReprojectionMap reprojection) {
    if (pointCloud.Size != reprojection.Map.Size) {
        return false;
    }
    for (int row = 0; row < pointCloud.Size.Height; ++row) {
        for (int col = 0; col < pointCloud.Size.Width; ++col) {
            if (depth.At(row, col) > 0.0f) {
                pointCloud.At(row, col).x = depth.At(row, col) * reprojection.Map.At(row, col).x;
                pointCloud.At(row, col).y = depth.At(row, col) * reprojection.Map.At(row, col).y;
                pointCloud.At(row, col).z = depth.At(row, col) * reprojection.Map.At(row, col).z;
            } else {
                pointCloud.At(row, col).x = 0.0f;
                pointCloud.At(row, col).y = 0.0f;
                pointCloud.At(row, col).z = 0.0f;
            }
        }
    }
    return true;
}

pho::api::PhoXiReprojectionMap map = PhoXiDevice->ReprojectionMap;
PhoXiDevice->StartAcquisition();
if (!PhoXiDevice->isAcquiring())
{
    std::cout << "Your device could not start acquisition!" << std::endl;
    return 0;
}

int id = PhoXiDevice->TriggerFrame();
if (id < 0) {
    std::cout << "Trigger frame unsuccessful" << std::endl;
}

auto frame = PhoXiDevice->GetSpecificFrame(id);
if (frame) {
    auto depth = frame->DepthMap;
    pho::api::PointCloud32f pointCloud;
    pointCloud.Resize(depth.Size);
    calculatePointCloud(pointCloud, depth, map);
    //pointCloud now contains your point cloud data
}

return 0;

Overview

Retrieval Methods by Device and Mode

Device and Mode	Camera Matrix & Distortion Coefficients	Reprojection Map
PhoXi 3D Scanner (Full Resolution)	Global or PerFrame	Global
PhoXi 3D Scanner (Half Resolution)	PerFrame only	Global
MotionCam-3D (Scanner Mode)	Global or PerFrame	Global
MotionCam-3D (Camera Mode)	PerFrame only (Note: Only applicable for Output Topology = Regular Grid)	Global
Alpha 3D Scanner (Full Resolution)	Global or PerFrame	Global