InstantMeshing

class InstantMeshing

The PhoXi instant meshing API.

For proper unconstrained use in a multithreaded environment please consider switching to headless mode which supports multithreading without limitations.

Warning

The methods of this class are expected to be called from either the main GUI thread or the same thread where this class was instantiated and initialized. Additionally it is required that all threads, that use the methods of this class, stay alive until the end of the program.

Lifecycle

explicit phoim::InstantMeshing::InstantMeshing(const std::unordered_map<DeviceId, DeviceParameters> &devices, const Settings &settings, const MessageCallback &msg_callback = nullptr)

The constructor of InstantMeshing class.

Parameters:

  • devices – Map of all devices that will be used later in AddScan.

  • settings – InstantMeshing scene settings (not related to devices).

  • msg_callback – Callback that is called for log messages and critical events.

Throws:

  • InvalidLicenseException – When a valid license is not available.

  • Exception – For other errors:

    • PHOIM_ERROR_DEVICE_PARAMS: Empty device ID, invalid depth parameters (min_depth >= max_depth), invalid sweet_spot_depth (not within min/max depth range), unsupported projection mode, missing distortion coefficients.

    • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for the specified devices and settings.

    • PHOIM_ERROR_GPU_NOT_FOUND: No GPU detected or invalid GPU ID specified.

    • PHOIM_ERROR_INTERNAL: Internal initialization errors.

explicit phoim::InstantMeshing::InstantMeshing(const std::vector<std::unordered_map<DeviceId, DeviceParameters>> &device_batches, const Settings &settings, const MessageCallback &msg_callback = nullptr)

The constructor of the InstantMeshing class for devices that are mechanically bound together and has constant relative transformation (defined by calibration_matrix for each device).

Parameters:

  • device_batches – Vector of device batches that will be used later in AddScanBatch. One batch could represent for example one robotic arm with any number of devices.

  • settings – InstantMeshing scene settings (not related to devices).

  • msg_callback – Callback that is called for log messages and critical events.

Throws:

  • InvalidLicenseException – When a valid license is not available.

  • Exception – For other errors:

    • PHOIM_ERROR_DEVICE_PARAMS: Empty device ID, duplicate device ID in batch, devices in batch have different resolutions, mixed tracking enabled/disabled in batch, texture tracking enabled in batch, invalid depth parameters, unsupported projection mode.

    • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for the specified device batches and settings.

    • PHOIM_ERROR_GPU_NOT_FOUND: No GPU detected or invalid GPU ID specified.

    • PHOIM_ERROR_INTERNAL: Internal initialization errors.

void phoim::InstantMeshing::Clear()

Removes all data that has been added, but keeps cropping volume and all added devices.

Throws:
void phoim::InstantMeshing::ResetTracking()

Rests tracking, which results in all data being aligned to the Camera space of the next added scan.

Note

Already added scans will remain in the same space as they were added and aligned.

Throws:

Exception – For errors returned by the C API.

Scan Integration

glm::mat4 phoim::InstantMeshing::AddScan(const pho::api::PFrame &scan, const DeviceId &device_id, const glm::mat4 &transformation = INVALID_MATRIX)

Integrates additional scan to the volumetric structure.

Note

All matrices (input transformation and returned transformation) are in column-major format.

Parameters:

  • scan – Scan data. If it is in other than camera space, the transformation from camera space can be handled in two ways: a) If the transformation is constant in all added scans with the same device_id - this is equivalent to using of calibration_matrix in DeviceParams. If the actual calibration_matrix from calibration_matrix is also defined (non-identity), the calibration_matrix will be applied as a second one. b) If the transformation is different in each scan - this is equivalent to using the transformation parameter of this method.

  • device_id – If tracking is enabled, it will start from the transformation found for the last added scan with the same device id.

  • transformation – Optional transformation matrix that will be applied on the scan before further processing. It overrides any transformation from camera space given implicitly by the input scan (CurrentCameraPosition, CurrentCameraXAxis, CurrentCameraYAxis and CurrentCameraZAxis). This transformation is applied before calibration_matrix, so it is in the camera space of the first scan added with the specific device_id and will be completely ignored if it is equal to INVALID_MATRIX. It can be handled in two ways: a) If the matrix is used for the first AddScan for a specific device_id, it will be equivalent to using of calibration_matrix in DeviceParams. If the actual calibration_matrix from calibration_matrix is also defined (non-identity), the calibration_matrix will be applied as a second one. b) If the matrix is used in any other frame, it is considered a starting point for tracking or a final scan transformation if the tracking is disabled.

Throws:

Exception – For errors:

  • PHOIM_ERROR_INVALID_ARGUMENT: Invalid transformation matrix, camera axes not unit vectors or not orthogonal.

  • PHOIM_ERROR_DEVICE_PARAMS: Unknown device_id.

  • PHOIM_ERROR_INTERNAL: Internal processing errors.

Returns:

Transformation matrix from Camera space of input scan to the mesh space. If tracking is enabled but unsuccessful, returns INVALID_MATRIX.

glm::mat4 phoim::InstantMeshing::AddScanBatch(const ScanBatch &scans)

Integrates additional batch of scans to the volumetric structure.

Note

All matrices (input transformations and returned transformation) are in column-major format.

Parameters:

scans – Batch of scans that has to correspond to one of the batches of devices added in the constructor. For more details, see documentation of ScanFromBatch. The transformation supplied with the scan is used either for direct integration of the input scan if tracking is disabled, or as a starting position for the tracking. NOTE: calibration_matrix of device supplied in the constructor is ignored.

Throws:

Exception – For errors:

  • PHOIM_ERROR_INVALID_ARGUMENT: Invalid transformation matrix (contains NaN or infinity), camera axes are not unit vectors, camera axes are not orthogonal, empty scans vector.

  • PHOIM_ERROR_DEVICE_PARAMS: Unknown device_id in any scan.

  • PHOIM_ERROR_INTERNAL: Internal batch processing errors.

Returns:

Transformation matrix from batch space of input scans to the mesh space. The matrix needs to be multiplied by the relative transformation of a specific device to achieve the final transformation of a specific scan. If tracking is enabled but unsuccessful, it returns INVALID_MATRIX.

Virtual Scanning

Scan phoim::InstantMeshing::GetVirtualScan(const CameraParams &virtual_camera_params, bool camera_space)

Creates a virtual perspective 3D scan of meshing internal volumetric structure from an arbitrary view, without the necessity for mesh computation. NOTE: Uses extra GPU memory beyond the max_gpu_memory_usage setting.

Parameters:

  • virtual_camera_params – Virtual camera parameters with camera position, basis, resolution, etc.

  • camera_space – If true, the resulting scan will be in the camera space defined by virtual_camera_params or in the mesh space otherwise.

Throws:

Exception – For errors:

  • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for virtual scan generation.

Returns:

3D scan of volumetric structure which contains all added scans. NOTE: filtering by min_voxel_consensus, which is used in the GetMesh, is not applied here.

Scan phoim::InstantMeshing::GetVirtualScan(const CameraParamsOrthogonal &virtual_camera_params, bool camera_space)

Creates a virtual orthogonal 3D scan of meshing internal volumetric structure from an arbitrary view, without the necessity for mesh computation. NOTE: Uses extra GPU memory beyond the max_gpu_memory_usage setting.

Parameters:

  • virtual_camera_params – Virtual camera parameters with camera position, basis, resolution, etc.

  • camera_space – If true, the resulting scan will be in the camera space defined by virtual_camera_params or in the mesh space otherwise.

Throws:

Exception – For errors:

  • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for virtual scan generation.

Returns:

3D scan of volumetric structure which contains all added scans. NOTE: filtering by min_voxel_consensus, which is used in the GetMesh, is not applied here.

Output Extraction

Mesh phoim::InstantMeshing::GetMesh(const ProgressCallback &progress_callback = nullptr)

Returns union of all added scans as a single triangle mesh.

Parameters:

progress_callback – Optional callback to track progress of mesh generation. Progress values are in range [0.0, 1.0].

Throws:

Exception – For errors:

  • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for mesh extraction.

  • PHOIM_ERROR_INTERNAL: Internal mesh generation errors.

Returns:

Returns mesh.

Mesh phoim::InstantMeshing::GetMesh(const CroppingVolume &cropping_volume, const ProgressCallback &progress_callback = nullptr)

Returns union of all added scans inside cropping volume as a single triangle mesh.

Parameters:

  • cropping_volume – Allowed 3D volume.

  • progress_callback – Optional callback to track progress of mesh generation. Progress values are in range [0.0, 1.0].

Throws:

Exception – For errors:

  • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for mesh extraction.

  • PHOIM_ERROR_INTERNAL: Internal mesh generation errors.

Returns:

Returns mesh.

bool phoim::InstantMeshing::ExportMesh(const std::string &file_path, const ProgressCallback &progress_callback = nullptr)

Save union of all added scans as triangle mesh to a file.

Note

Supported output file extensions: .stl, .fbx, .ply, .obj, .dae, .cogs.

Parameters:

  • file_path – Full directory including filename and file extension.

  • progress_callback – Optional callback to track progress of mesh generation. Progress values are in range [0.0, 1.0].

Throws:

Exception – For errors:

  • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for mesh extraction.

  • PHOIM_ERROR_INTERNAL: Internal mesh generation or export errors.

Returns:

Return true if the mesh was successfully exported, false otherwise.

bool phoim::InstantMeshing::ExportMesh(const std::string &file_path, const CroppingVolume &cropping_volume, const ProgressCallback &progress_callback = nullptr)

Save union of all added scans inside cropping volume as triangle mesh to a file.

Note

Supported output file extensions: .stl, .fbx, .ply, .obj, .dae, .cogs.

Parameters:

  • file_path – File path with the file extension.

  • cropping_volume – Allowed 3D volume.

  • progress_callback – Optional callback to track progress of mesh generation. Progress values are in range [0.0, 1.0].

Throws:

Exception – For errors:

  • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for mesh extraction.

  • PHOIM_ERROR_INTERNAL: Internal mesh generation or export errors.

Returns:

Returns if point clout was successfully exported.

PointCloud phoim::InstantMeshing::GetUnion(const std::vector<CameraParams> &camera_views)

Generates virtual scans based on input camera views and returns them as single point cloud.

Note

Mind the number of camera_views as this method is quite CPU memory heavy.

Parameters:

camera_views – Camera views based on which are generated virtual scans.

Throws:

Exception – For errors:

  • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for virtual scan generation.

  • PHOIM_ERROR_INTERNAL: Internal point cloud generation errors.

Returns:

Returns non-organized point cloud.

PointCloud phoim::InstantMeshing::GetUnion(const std::vector<CameraParamsOrthogonal> &camera_views)

Generates virtual scans based on input orthogonal camera views and returns them as single point cloud.

Note

Mind the number of camera_views as this method is quite CPU memory heavy.

Parameters:

camera_views – Orthogonal camera views based on which are generated virtual scans.

Throws:

Exception – For errors:

  • PHOIM_ERROR_GPU_MEMORY: Insufficient GPU memory for virtual scan generation.

  • PHOIM_ERROR_INTERNAL: Internal point cloud generation errors.

Returns:

Returns non-organized point cloud.

static bool phoim::InstantMeshing::ExportUnion(const PointCloud &union_pc, const std::string &file_path)

Exports a point cloud generated by the GetUnion method.

Note

Supported output file extensions: .ply

Parameters:

  • union_pc – Non-organized point cloud.

  • file_path – File path with the file extension.

Returns:

Returns if point clout was successfully exported.

Cropping Volume

void phoim::InstantMeshing::SetCroppingVolume(const CroppingVolume &cropping_volume)

Sets 3D volume outside of which will be all newly added data discarded.

Note

The comparison of newly added data to the cropping volume will take place after the alignment to the union. This will not affect existing data.

Parameters:

cropping_volume – Allowed 3D volume.

Throws:

Exception – For errors:

  • PHOIM_ERROR_INVALID_ARGUMENT: Unsupported cropping volume type.

void phoim::InstantMeshing::ResetCroppingVolume()

Removes any cropping volume that has been added.

Throws:

Exception – For errors returned by the C API.

Static Helpers

static std::optional<std::pair<float, float>> phoim::InstantMeshing::GetDeviceDepthParams(const pho::api::PPhoXi device)

Get the device min/max depth parameters based on the model and variant, if available.

Warning

This method is not thread safe-safe and does not work for file cameras.

Parameters:

device – Device to get parameters from.

Returns:

A pair of floats indicating min/max depth in mm or std::nullopt if the method is unable to detect the depth parameters of a device.