C++ APIs

Class 1

The class name is vart::Runner. The following table lists all the functions defined in the vitis::vart::Runner class.

Table 1. Quick Function Reference
Return Type	Name	Arguments
std::unique_ptr<Runner>	create_runner	const xir::Subgraph* subgraph const std::string& mode
std::vector<std::unique_ptr<Runner>>	create_runner	const std::string& model_directory
std::pair<uint32_t, int>	execute_async	const std::vector<TensorBuffer>& input const std::vector<TensorBuffer>& output
int	wait	int jobID int timeout
TensorFormat	get_tensor_format
std::vector<const xir::Tensor*>	get_input_tensors
std::vector<const xir::Tensor*>	get_output_tensors

Class 2

The class name is vart::TensorBuffer. The following table lists all the functions defined in the vart::TensorBuffer class.

Table 2. Quick Function Reference
Return Type	Name	Arguments
location_t	get_location
const xir::Tensor*	get_tensor
std::pair<std::uint64_t, std::size_t>	data	const std::vector<std::int32_t> idx = {}
std::pair<uint64_t, size_t>	data_phy	const std::vector<std::int32_t> idx
void	sync_for_read	uint64_t offset, size_t size
void	sync_for_write	uint64_t offset, size_t size
void	copy_from_host	size_t batch_idx, const void* buf, size_t size, size_t offset
void	copy_to_host	size_t batch_idx, void* buf, size_t size, size_t offset
void	copy_tensor_buffer	vart::TensorBuffer* tb_from, vart::TensorBuffer* tb_to

Class 3

The class name is vart::RunnerExt. The following table lists all the functions defined in the vart::RunnerExt class.

Table 3. Quick Function Reference
Return Type	Name	Arguments
std::vector<vart::TensorBuffer*>	get_inputs
std::vector<vart::TensorBuffer*>	get_outputs

Class 4

The class name is vitis::ai::GraphRunner. The following table lists all the functions defined in the vitis::ai::GraphRunner class.

Table 4. Quick Function Reference
Return Type	Name	Arguments
std::unique_ptr<vart::RunnerExt>	create_graph_runner	const xir::Graph* graph, xir::Attrs* attrs

create_runner

Creates an instance of CPU/SIM/DPU runner by subgraph. This is a factory method.

Prototype

std::unique_ptr<Runner> create_runner(const xir::Subgraph* subgraph,
                                       const std::string& mode = "");

Parameters

The following table lists the create_runner function arguments.

Table 5. create_runner Arguments
Type	Name	Description
const xir::Subgraph*	subgraph	XIR Subgraph
const std::string&	mode	3 mode supported: 'ref' - CPU runner 'sim' - Simulation 'run' - DPU runner

Returns

An instance of CPU/SIM/DPU runner.

create_runner

Creates a DPU runner by model_directory.

Prototype

std::vector<std::unique_ptr<Runner>> create_runner(const std::string& model_directory);

Parameters

The following table lists the create_runner function arguments.

Table 6. create_runner Arguments
Type	Name	Description
conststd::string&	model_directory	The directory name which contains meta.json

Returns

A vector of DPU runner.

execute_async

Executes the runner. This is a block function.

Prototype

virtual std::pair<uint32_t, int> execute_async(
      const std::vector<TensorBuffer*>& input,
      const std::vector<TensorBuffer*>& output) = 0;

Parameters

The following table lists the execute_async function arguments.

Table 7. execute_async Arguments
Type	Name	Description
conststd::vector<TensorBuffer*>&	input	Vector of the input Tensor buffers containing the input data for inference.
conststd::vector<TensorBuffer*>&	output	Vector of the output Tensor buffers which will be filled with output data.

Returns

pair<jobID, status> status 0 for exit successfully, others for customized warnings or errors.

wait

Waits for the end of DPU processing. This is a block function.

Prototype

int wait(int jobid, int timeout)

Parameters

The following table lists the wait function arguments.

Table 8. wait Arguments
Type	Name	Description
int	jobid	job id, neg for any id, others for specific job id
int	timeout	timeout, neg for block for ever, 0 for non-block, pos for block with a limitation(ms).

Returns

Status 0 for exit successfully, others for customized warnings or errors.

get_tensor_format

Gets the tensor format of runner.

Prototype

TensorFormat get_tensor_format();

Parameters

None

Returns

TensorFormat: NHWC / HCHW

get_input_tensors

Gets all input tensors of runner.

Prototype

std::vector<const xir::Tensor*> get_input_tensors()

Parameters

None

Returns

All input tensors. A vector of raw pointer to the input tensor.

get_output_tensors

Gets all output tensors of runner.

Prototype

std::vector<const xir::Tensor*> get_output_tensors()

Parameters

None

Returns

All output tensors. A vector of raw pointer to the output tensor.

get_location

Get where the tensor buffer located.

Prototype

location_t get_location();

Parameters

None.

Returns

The tensor buffer location, a location_t enum type value.

The following table lists the location_t enum type.

Table 9. location_t enum type
Name	Value	Description
HOST_VIRT	0	Only accessible by the host.
HOST_PHY	1	Continuous physical memory, shared among host and device.
DEVICE_0	2	Only accessible by device_*.
DEVICE_1	3
DEVICE_2	4
DEVICE_3	5
DEVICE_4	6
DEVICE_5	7
DEVICE_6	8
DEVICE_7	9

get_tensor

Get Tensor of TensorBuffer.

Prototype

const xir::Tensor* get_tensor()

Parameters

None.

Returns

A pointer to the Tensor.

data

Get the data address of the index and the left accessible data size.

Prototype

std::pair<std::uint64_t, std::size_t> data(const std::vector<std::int32_t> idx = {});

Parameters

The following table lists the data function arguments.

Table 10. data Arguments
Type	Name	Description
const std::vector<std::int32_t>	idx	The index of the data to be accessed, its dimension same to the Tensor shape

Returns

A pair of the data address of the index and the left accessible data size in byte unit.

data_phy

Get the data physical address of the index and the left accessible data size.

Prototype

std::pair<uint64_t, size_t> data_phy(const std::vector<std::int32_t> idx);

Parameters

The following table lists the data_phy function arguments.

Table 11. data_phy Arguments
Type	Name	Description
const std::vector<std::int32_t>	idx	The index of the data to be accessed, its dimension same to the tensor shape

Returns

A pair of the data physical address of the index and the left accessible data size in byte unit.

sync_for_read

Invalid cache for reading before read. It is no-op in case get_location() returns DEVICE_ONLY or HOST_VIRT.

Prototype

void sync_for_read(uint64_t offset, size_t size) {};

Parameters

The following table lists the sync_for_read function arguments.

Table 12. sync_for_read Arguments
Type	Name	Description
uint64_t	offset	The start offset address
size_t	size	The data size

Returns

None.

sync_for_write

Flush cache for writing after write. It is no-op in case get_location() returns DEVICE_ONLY or HOST_VIRT.

Prototype

void sync_for_write (uint64_t offset, size_t size) {};

Parameters

The following table lists the sync_for_write function arguments.

Table 13. sync_for_write Arguments
Type	Name	Description
uint64_t	offset	The start offset address
size_t	size	The data size

Returns

None.

copy_from_host

Copy data from the source buffer.

Prototype

void copy_from_host(size_t batch_idx, const void* buf, size_t size, size_t offset);

Parameters

The following table lists the copy_from_host function arguments.

Table 14. copy_from_host Arguments
Type	Name	Description
size_t	batch_idx	The batch index
const void*	buf	Source buffer start address
size_t	size	Data size to be copied
size_t	offset	The start offset to be copied

Returns

None.

copy_to_host

Copy data to the destination buffer.

Prototype

void copy_to_host(size_t batch_idx, void* buf, size_t size, size_t offset);

Parameters

The following table lists the copy_to_host function arguments.

Table 15. copy_to_host Arguments
Type	Name	Description
size_t	batch_idx	The batch index
void*	buf	Destination buffer start address
size_t	size	Data size to be copied
size_t	offset	The start offset to be copied

Returns

None.

copy_tensor_buffer

Copy TensorBuffer from one to another.

Prototype

static void copy_tensor_buffer(vart::TensorBuffer* tb_from, vart::TensorBuffer* tb_to);

Parameters

The following table lists the copy_tensor_buffer function arguments.

Table 16. copy_tensor_buffer Arguments
Type	Name	Description
vart::TensorBuffer*	tb_from	The source TensorBuffer
vart::TensorBuffer*	tb_to	The destination TensorBuffer

Returns

None.

get_inputs

Gets all input TensorBuffers of RunnerExt.

Prototype

std::vector<vart::TensorBuffer*> get_inputs();

Parameters

None.

Returns

All input TensorBuffers. A vector of raw pointer to the input TensorBuffer.

get_outputs

Gets all output TensorBuffers of RunnerExt.

Prototype

std::vector<vart::TensorBuffer*> get_outputs();

Parameters

None.

Returns

All output TensorBuffers. A vector of raw pointer to the output TensorBuffer.

create_graph_runner

Factory function to create an instance of runner by graph and attributes.

Prototype

static std::unique_ptr<vart::RunnerExt> create_graph_runner(const xir::Graph* graph, xir::Attrs* attrs);

Parameters

The following table lists the create_graph_runner function arguments.

Table 17. create_graph_runner Arguments
Type	Name	Description
`const xir::Graph*`	graph	XIR Graph
`xir::Attrs*`	attrs	XIR attrs object, this object is shared among all runners on the same graph.

Returns

An instance of runner.