Running Examples

If you want to do optimizations to achieve high performance, here are some suggestions:

• Rearrange the thread pipeline structure so that every DPU thread has its own "DPU" runner object.
• Optimize display thread so that when DPU FPS is higher than display rate, skipping some frames. 200 FPS is too high for video display.
• Pre-decoding. The video file might be H.264 encoded. The decoder is slower than the DPU and consumes a lot of CPU resources. The video file has to be first decoded and transformed into raw format.
• The batch mode on Alveo boards needs special consideration as it may cause video frame jittering. ZCU102 has no batch mode support.
• OpenCV cv::imshow is slow, so you need to use libdrm.so. This is only for local display, not through X server.

The following table below describes these Vitis AI examples.

The typical code snippet to deploy models with Vitis AI unified C++ high-level APIs is as follows:

// get dpu subgraph by parsing model file
auto runner = vart::Runner::create_runner(subgraph, "run");
// get input scale and output scale,
// they are used for fixed-floating point conversion
auto outputTensors = runner->get_output_tensors();
auto inputTensors = runner->get_input_tensors();
auto input_scale = get_input_scale(inputTensors[0]);
auto output_scale = get_output_scale(outputTensors[0]);
// do the image pre-process, convert float data to fixed point data
// populate input/output tensors
auto job_id = runner->execute_async(inputsPtr, outputsPtr);
runner->wait(job_id.first, -1);
// process outputs, convert fixed point data to float data 

The typical code snippet to deploy models with Vitis AI unified extension C++ high-level APIs is as follows:

// get dpu subgraph by parsing model file
std::unique_ptr<vart::RunnerExt> runner =
          vart::RunnerExt::create_runner(subgraph, attrs.get());
// get input & output tensor buffers
auto input_tensor_buffers = runner->get_inputs();
auto output_tensor_buffers = runner->get_outputs();
// get input scale and output scale,
// they are used for fixed-floating point conversion
auto input_tensor = input_tensor_buffers[0]->get_tensor();
auto output_tensor = output_tensor_buffers[0]->get_tensor();
auto input_scale = get_input_scale(input_tensor);
auto output_scale = get_output_scale(output_tensor);
// do the image pre-process, convert float data to fixed point data
setImageBGR(images[batch_idx], (void*)data_in, input_scale);
// sync data for input
input->sync_for_write(0, input->get_tensor()->get_data_size() /
                         input->get_tensor()->get_shape()[0]); 
// populate input/output tensors
auto v = runner->execute_async(input_tensor_buffers, output_tensor_buffers);
auto status = runner->wait((int)v.first, -1);
// sync data for output
output->sync_for_read(0, output->get_tensor()->get_data_size() /
                         output->get_tensor()->get_shape()[0]);
// process outputs, conver fixed point data to float data 

The typical code snippet to deploy models with Vitis AI unified Python high-level APIs is shown below:

dpu_runner = runner.Runner(subgraph，"run")
# populate input/output tensors
jid = dpu_runner.execute_async(fpgaInput, fpgaOutput)
dpu_runner.wait(jid)
# process fpgaOutput

Before running Vitis™ AI examples on Edge or on Cloud, download the vitis_ai_runtime_r1.4.0_image_video.tar.gz from here. The images and videos used in the following example can be found in the package.

To improve the user experience, the Vitis AI Runtime packages, VART samples, Vitis-AI-Library samples and models have been built into the board image, and the examples are pre-compiled. You can directly run the example program on the target.