vitis::ai::PointPainting

Base class for pointpating.

Input is points data and related params.

Output is a struct of detection results, named PointPaintingResult.

Sample code : 

  ...
  std::string anno_file_name = "./sample_pointpainting.info";
  PointsInfo points_info;  
  std::vector<cv::Mat> images;
  read_inno_file_pointpainting(anno_file_name, points_info, 5, points_info.sweep_infos, 16, images); 
  std::string seg_model = "semanticfpn_nuimage_576_320_pt";
  std::string model_0 = "pointpainting_nuscenes_40000_64_0_pt";
  std::string model_1 = "pointpainting_nuscenes_40000_64_1_pt";
  auto pointpainting = vitis::ai::PointPainting::create(
       seg_model, model_0, model_1);
  auto ret = pointpainting->run(images, points_info);
  ...
please see the test sample for detail.

Quick Function Reference

The following table lists all the functions defined in the vitis::ai::PointPainting class:

Table 1. Quick Function Reference
TypeNameArguments
std::unique_ptr< PointPainting >create
  • const std::string & seg_model_name
  • const std::string & pp_model_name_0
  • const std::string & pp_model_name_1
intgetInputWidth
  • void
intgetInputHeight
  • void
size_tget_pointpillars_batch
  • void
size_tget_segmentation_batch
  • void
PointPaintingResultrun
  • const std::vector< cv::Mat > & input_images
  • const vitis::ai::pointpillars_nus::PointsInfo & points_info
std::vector< PointPaintingResult >run
  • const std::vector< std::vector< cv::Mat >> & batch_input_images
  • const std::vector< vitis::ai::pointpillars_nus::PointsInfo > & batch_points_info
std::vector< cv::Mat >runSegmentation
  • std::vector< cv::Mat > batch_images
std::vector< float >fusion
  • const std::vector< cv::Mat > & seg_images
  • const vitis::ai::pointpillars_nus::PointsInfo & points_info
vitis::ai::pointpillars_nus::PointsInforunSegmentationFusion
  • const std::vector< cv::Mat > & input_images
  • const vitis::ai::pointpillars_nus::PointsInfo & points
PointPaintingResultrunPointPillars
  • const vitis::ai::pointpillars_nus::PointsInfo & points_info
std::vector< PointPaintingResult >runPointPillars
  • const std::vector< vitis::ai::pointpillars_nus::PointsInfo > & batch_points_info

create

Factory function to get an instance of derived classes of class PointPainting.

Prototype

std::unique_ptr< PointPainting > create(const std::string &seg_model_name, const std::string &pp_model_name_0, const std::string &pp_model_name_1, bool need_preprocess=true);

Parameters

The following table lists the create function arguments.

Table 2. create Arguments
Type Name Description
const std::string & seg_model_name Segmentation model name
const std::string & pp_model_name_0 The first pointpillars nuscenes model name
const std::string & pp_model_name_1 The second pointpillars nuscenes model name

Returns

An instance of PointPainting class.

getInputWidth

Function to get input width of the first model of pointpainting (segmentation model).

Prototype

int getInputWidth() const =0;

Returns

Input width of the first model (segmentation model).

getInputHeight

Function to get input height of the first model of pointpainting (segmentation model).

Prototype

int getInputHeight() const =0;

Returns

Input height of the first model (segmentation model).

get_pointpillars_batch

Function to get the number of pointpillars inputs processed by the DPU at one time.

Note: Batch size of different DPU core may be different, it depends on the IP used. For pointpainting class, segmentation model and pointpillars models may be running on different DPU cores.

Prototype

size_t get_pointpillars_batch() const =0;

Returns

Batch size of pointpillars model.

get_segmentation_batch

Function to get the number of segmentation inputs processed by the DPU at one time.

Note: Batch size of different DPU core may be different, it depends on the IP used. For pointpainting class, segmentation model and pointpillars models may be running on different DPU cores.

Prototype

size_t get_segmentation_batch() const =0;

Returns

Batch size of segmentation model.

run

Function of get result of the pointpainting full flow.

Prototype


            PointPaintingResult run(const std::vector< cv::Mat > &input_images, const vitis::ai::pointpillars_nus::PointsInfo &points_info)=0;

Parameters

The following table lists the run function arguments.

Table 3. run Arguments
Type Name Description
const std::vector< cv::Mat > & input_images Images from different cameras for segmentation .
const vitis::ai::pointpillars_nus::PointsInfo & points_info points data and camera related params.

Returns

PointPaintingResult.

run

Function of get result of the pointpainting full flow in batch mode.

Prototype

std::vector< PointPaintingResult > run(const std::vector< std::vector< cv::Mat >> &batch_input_images, const std::vector< vitis::ai::pointpillars_nus::PointsInfo > &batch_points_info)=0;

Parameters

The following table lists the run function arguments.

Table 4. run Arguments
Type Name Description
const std::vector< std::vector< cv::Mat >> & batch_input_images Batch input of images from different cameras for segmentation. The size should be equal to the result of get_pointpillars_batch.
const std::vector< vitis::ai::pointpillars_nus::PointsInfo > & batch_points_info Batch input of points datas and camera related params.The size should be equal to the result of get_pointpillars_batch.

Returns

A Vector of PointPaintingResult.

runSegmentation

Function of get result of the segmentation in batch mode.

Prototype

std::vector< cv::Mat > runSegmentation(std::vector< cv::Mat > batch_images)=0;

Parameters

The following table lists the runSegmentation function arguments.

Table 5. runSegmentation Arguments
Type Name Description
std::vector< cv::Mat > batch_images Batch input of images from different cameras for segmentation. The size should be equal to the result of get_segmentation_batch.

Returns

A Vector of segmentation result(cv::Mat).

fusion

Function of get result points fusion.

Prototype

std::vector< float > fusion(const std::vector< cv::Mat > &seg_images, const vitis::ai::pointpillars_nus::PointsInfo &points_info)=0;

Parameters

The following table lists the fusion function arguments.

Table 6. fusion Arguments
Type Name Description
const std::vector< cv::Mat > & seg_images Segmentation result images.
const vitis::ai::pointpillars_nus::PointsInfo & points_info Points data and camera related params.

Returns

Points data after fusion.

runSegmentationFusion

Function of get result of segmentation and points fusion.

Prototype


            vitis::ai::pointpillars_nus::PointsInfo runSegmentationFusion(const std::vector< cv::Mat > &input_images, const vitis::ai::pointpillars_nus::PointsInfo &points)=0;

Parameters

The following table lists the runSegmentationFusion function arguments.

Table 7. runSegmentationFusion Arguments
Type Name Description
const std::vector< cv::Mat > & input_images Images for segmentation
const vitis::ai::pointpillars_nus::PointsInfo & points Points data and camera related params.

Returns

an instance of PointsInfo with points data result

runPointPillars

Function of get result of pointpillars nuscenes neural network.

Prototype


            PointPaintingResult runPointPillars(const vitis::ai::pointpillars_nus::PointsInfo &points_info)=0;

Parameters

The following table lists the runPointPillars function arguments.

Table 8. runPointPillars Arguments
Type Name Description
const vitis::ai::pointpillars_nus::PointsInfo & points_info Points data and camera related params.

Returns

PointPaintingResult(same as PointPillarsNuscenesResult).

runPointPillars

Function of get result of pointpillars nuscenes neural network in batch mode.

Prototype

std::vector< PointPaintingResult > runPointPillars(const std::vector< vitis::ai::pointpillars_nus::PointsInfo > &batch_points_info)=0;

Parameters

The following table lists the runPointPillars function arguments.

Table 9. runPointPillars Arguments
Type Name Description
const std::vector< vitis::ai::pointpillars_nus::PointsInfo > & batch_points_info A batch of Points data and camera related params.

Returns

A Vector of PointPaintingResult(same as PointPillarsNuscenesResult).