Getting Started with SDSoC

This chapter provides the information you need to bring up your design using the xfOpenCV library functions.

Prerequisites

This section lists the prerequisites for using the xfOpenCV library functions on ZCU104 based platforms. The methodology holds true for ZC702 and ZC706 reVISION platforms as well.
  • Download and install the SDx development environment according to the directions provided in SDSoC Environments Release Notes, Installation, and Licensing Guide (UG1294). Before launching the SDx development environment on Linux, set the $SYSROOT environment variable to point to the Linux root file system if using terminal to build project, delivered with the reVISION platform. For example:
    export SYSROOT = <local folder>/zcu104_rv_ss/sw/a53_linux/a53_linux/sysroot/aarch64-xilinx-xilinx
  • Download the Zynq® UltraScale+™ MPSoC Embedded Vision Platform zip file and extract its contents. Create the SDx development environment workspace in the zcu104_rv_ss folder of the extracted design file hierarchy. For more details, see the reVISION Getting Started Guide.
  • Set up the ZCU104 evaluation board. For more details, see the reVISION Getting Started Guide.
  • Download the xfOpenCV library. This library is made available through github. Run the following git clone command to clone the xfOpenCV repository to your local disk: 
    git clone https://github.com/Xilinx/xfopencv.git

Migrating HLS Video Library to xfOpenCV

The HLS video library will soon be deprecated .All the functions and most of the infrastructure available in HLS video library are now available in xfOpenCV with their names changed and some modifications. These HLS video library functions ported to xfOpenCV support SDSoc build flow also.

This section provides the details on using the C++ video processing functions and the infrastructure present in HLS video library.

Infrastructure Functions and Classes

All the functions imported from HLS video library now take xf::Mat (in sync with xfOpenCV library) to represent image data instead of hls::Mat. The main difference between these two is that the hls::Mat uses hls::stream to store the data whereas xf::Mat uses a pointer. Therefore, hls:: Mat cannot be exactly replaced with xf::Mat for migrating.

Below table summarizes the differences between member functions of hls::Mat to xf::Mat.

Table 1. Infrastructure Functions and Classes
Member Function hls::Mat (HLS Video lib) Xf::Mat (xfOpenCV lib)
channels() Returns the number of channels Returns the number of channels
type() Returns the enum value of pixel type Returns the enum value of pixel type
depth() Returns the enum value of pixel type Returns the depth of pixel including channels
read() Readout a value and return it as a scalar from stream Readout a value from a given location and return it as a packed (for multi-pixel/clock) value.
operator >> Similar to read() Not available in xfOpenCV
operator << Similar to write() Not available in xfOpenCV
Write() Write a scalar value into the stream Writes a packed (for multi-pixel/clock) value into the given location..

Infrastructure files available in HLS Video Library hls_video_core.h, hls_video_mem.h, hls_video_types.h are moved to xf_video_core.h, xf_video_mem.h, xf_video_types.h in xfOpenCV Library and hls_video_imgbase.h is deprecated. Code inside these files unchanged except that these are now under xf::namespace.

Classes

Memory Window Buffer
hls::window is now xf::window. No change in the implementation, except the namespace change. This is located in “xf_video_mem.h” file.
Memory Line Buffer
hls::LineBuffer is now xf::LineBuffer. No difference between the two, except xf::LineBuffer has extra template arguments for inferring different types of RAM structures, for the storage structure used. Default storage type is “RAM_S2P_BRAM” with RESHAPE_FACTOR=1. Complete description can be found here xf::LineBuffer. This is located in xf_video_mem.h file.

Funtions

OpenCV interface functions
These functions covert image data of OpenCV Mat format to/from HLS AXI types. HLS Video Library had 14 interface functions, out of which, two functions are available in xfOpenCV Library: cvMat2AXIvideo and AXIvideo2cvMat located in “xf_axi.h” file. The rest are all deprecated.
AXI4-Stream I/O Functions
The I/O functions which convert hls::Mat to/from AXI4-Stream compatible data type (hls::stream) are hls::AXIvideo2Mat, hls::Mat2AXIvideo. These functions are now deprecated and added 2 new functions xf::AXIvideo2xfMat and xf:: xfMat2AXIvideo to facilitate the xf::Mat to/from conversion. To use these functions, the header file "xf_infra.h" must be included.

xf::window

A template class to represent the 2D window buffer. It has three parameters to specify the number of rows, columns in window buffer and the pixel data type.

Class definition

template<int ROWS, int COLS, typename T>
class Window {
public:
    Window() 
   /* Window main APIs */
    void shift_pixels_left();
    void shift_pixels_right();
    void shift_pixels_up();
    void shift_pixels_down();
    void insert_pixel(T value, int row, int col);
    void insert_row(T value[COLS], int row);
    void insert_top_row(T value[COLS]);
    void insert_bottom_row(T value[COLS]);
    void insert_col(T value[ROWS], int col);
    void insert_left_col(T value[ROWS]);
    void insert_right_col(T value[ROWS]);
    T& getval(int row, int col);
    T& operator ()(int row, int col);
    T val[ROWS][COLS];
#ifdef __DEBUG__
    void restore_val();
    void window_print();
    T val_t[ROWS][COLS];
#endif
};

Parameter Descriptions

The following table lists the xf::Window class members and their descriptions.

Table 2. Window Function Parameter Descriptions
Parameter Description
Val 2-D array to hold the contents of buffer.

Member Function Description

Table 3. Member Function Description
Function Description
shift_pixels_left() Shift the window left, that moves all stored data within the window right, leave the leftmost column (col = COLS-1) for inserting new data.
shift_pixels_right() Shift the window right, that moves all stored data within the window left, leave the rightmost column (col = 0) for inserting new data.
shift_pixels_up() Shift the window up, that moves all stored data within the window down, leave the top row (row = ROWS-1) for inserting new data.
shift_pixels_down() Shift the window down, that moves all stored data within the window up, leave the bottom row (row = 0) for inserting new data.
insert_pixel(T value, int row, int col) Insert a new element value at location (row, column) of the window.
insert_row(T value[COLS], int row) Inserts a set of values in any row of the window.
insert_top_row(T value[COLS]) Inserts a set of values in the top row = 0 of the window.
insert_bottom_row(T value[COLS]) Inserts a set of values in the bottom row = ROWS-1 of the window.
insert_col(T value[ROWS], int col) Inserts a set of values in any column of the window.
insert_left_col(T value[ROWS]) Inserts a set of values in left column = 0 of the window.
insert_right_col(T value[ROWS]) Inserts a set of values in right column = COLS-1 of the window.
T& getval(int row, int col) Returns the data value in the window at position (row,column).
T& operator ()(int row, int col) Returns the data value in the window at position (row,column).
restore_val() Restore the contents of window buffer to another array.
window_print() Print all the data present in window buffer onto console.

Template Parameter Description

Table 4. Template Parameter Description
Parameter Description
ROWS Number of rows in the window buffer.
COLS Number of columns in the window buffer.
T Data type of pixel in the window buffer.
Sample code for window buffer declaration
Window<K_ROWS, K_COLS, unsigned char> kernel;

xf::LineBuffer

A template class to represent 2D line buffer. It has three parameters to specify the number of rows, columns in window buffer and the pixel data type.

Class definition

template<int ROWS, int COLS, typename T, XF_ramtype_e MEM_TYPE=RAM_S2P_BRAM, int RESHAPE_FACTOR=1>
 class LineBuffer {
public:
    LineBuffer()
       /* LineBuffer main APIs */
    /* LineBuffer main APIs */
    void shift_pixels_up(int col);
    void shift_pixels_down(int col);
    void insert_bottom_row(T value, int col);
    void insert_top_row(T value, int col);
    void get_col(T value[ROWS], int col);
    T& getval(int row, int col);
    T& operator ()(int row, int col);

    /* Back compatible APIs */
    void shift_up(int col);
    void shift_down(int col);
    void insert_bottom(T value, int col);
    void insert_top(T value, int col);
    T val[ROWS][COLS];
#ifdef __DEBUG__
    void restore_val();
    void linebuffer_print(int col);
    T val_t[ROWS][COLS];
#endif
};

Parameter Descriptions

The following table lists the xf::LineBuffer class members and their descriptions.

Table 5. Line Buffer Function Parameter Descriptions
Parameter Description
Val 2-D array to hold the contents of line buffer.

Member Functions Description

Table 6. Member Functions Description
Function Description
shift_pixels_up(int col) Line buffer contents Shift up, new values will be placed in the bottom row=ROWS-1.
shift_pixels_down(int col) Line buffer contents Shift down, new values will be placed in the top row=0.
insert_bottom_row(T value, int col) Inserts a new value in bottom row= ROWS-1 of the line buffer.
insert_top_row(T value, int col) Inserts a new value in top row=0 of the line buffer.
get_col(T value[ROWS], int col) Get a column value of the line buffer.
T& getval(int row, int col) Returns the data value in the line buffer at position (row, column).
T& operator ()(int row, int col); Returns the data value in the line buffer at position (row, column).

Template Parameter Description

Table 7. Template Parameter Description
Parameter Description
ROWS Number of rows in line buffer.
COLS Number of columns in line buffer.
T Data type of pixel in line buffer.
MEM_TYPE Type of storage element. It takes one of the following enumerated values: RAM_1P_BRAM, RAM_1P_URAM, RAM_2P_BRAM, RAM_2P_URAM, RAM_S2P_BRAM, RAM_S2P_URAM, RAM_T2P_BRAM, RAM_T2P_URAM.
RESHAPE_FACTOR Specifies the amount to divide an array.
Sample code for line buffer declaration: 
LineBuffer<3, 1920, XF_8UC3, RAM_S2P_URAM,1>     buff;

Video Processing Functions

The following table summarizes the video processing functions ported from HLS Video Library into xfOpenCV Library along with the API modifications.

Table 8. Video Processing Functions
Functions HLS Video Library -API xfOpenCV Library-API
addS 
template<int ROWS, int COLS, int SRC_T, typename _T, int DST_T>
void AddS(Mat<ROWS, COLS, SRC_T>&src,Scalar<HLS_MAT_CN(SRC_T), _T> scl, Mat<ROWS, COLS, DST_T>& dst)
 
template<int POLICY_TYPE, int SRC_T, int ROWS, int COLS, int NPC =1>
void addS(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
AddWeighted 
template<int ROWS, int COLS, int SRC1_T, int SRC2_T, int DST_T, typename P_T>
void AddWeighted(Mat<ROWS, COLS, SRC1_T>& src1,P_T alpha,Mat<ROWS, COLS, SRC2_T>& src2,P_T beta, P_T gamma,Mat<ROWS, COLS, DST_T>& dst)

template< int SRC_T,int DST_T, int ROWS, int COLS, int NPC = 1> 
void addWeighted(xf::Mat<SRC_T, ROWS, COLS, NPC> & src1,float alpha, xf::Mat<SRC_T, ROWS, COLS, NPC> & src2,float beta, float gama, xf::Mat<DST_T, ROWS, COLS, NPC> & dst)
Cmp 
template<int ROWS, int COLS, int SRC1_T, int SRC2_T, int DST_T>
void Cmp(Mat<ROWS, COLS, SRC1_T>& src1,Mat<ROWS, COLS, SRC2_T>& src2,
Mat<ROWS, COLS, DST_T>& dst,int cmp_op)

template<int CMP_OP, int SRC_T, int ROWS, int COLS, int NPC =1>
void compare(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1, xf::Mat<SRC_T, ROWS, COLS, NPC> & _src2,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
CmpS 
template<int ROWS, int COLS, int SRC_T, typename P_T, int DST_T>
void CmpS(Mat<ROWS, COLS, SRC_T>& src,  P_T value, Mat<ROWS, COLS, DST_T>& dst, int cmp_op)

template<int CMP_OP, int SRC_T, int ROWS, int COLS, int NPC =1>
void compare(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
Max 
template<int ROWS, int COLS, int SRC1_T, int SRC2_T, int DST_T>
void Max(Mat<ROWS, COLS, SRC1_T>& src1,
        Mat<ROWS, COLS, SRC2_T>& src2,
        Mat<ROWS, COLS, DST_T>& dst)

template<int SRC_T, int ROWS, int COLS, int NPC =1>
void Max(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1, xf::Mat<SRC_T, ROWS, COLS, NPC> & _src2,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
MaxS 
template<int ROWS, int COLS, int SRC_T, typename _T, int DST_T>
void MaxS(Mat<ROWS, COLS, SRC_T>& src,
_T value, Mat<ROWS, COLS, DST_T>& dst)

template< int SRC_T, int ROWS, int COLS, int NPC =1>
void max(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1,  unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
Min 
template<int ROWS, int COLS, int SRC1_T, int SRC2_T, int DST_T>
void Min(Mat<ROWS, COLS, SRC1_T>& src1,
        Mat<ROWS, COLS, SRC2_T>& src2,
        Mat<ROWS, COLS, DST_T>& dst)

template< int SRC_T, int ROWS, int COLS, int NPC =1>
void Min(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1, xf::Mat<SRC_T, ROWS, COLS, NPC> & _src2,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
MinS 
template<int ROWS, int COLS, int SRC_T, typename _T, int DST_T>
void MinS(Mat<ROWS, COLS, SRC_T>& src,
        _T value,Mat<ROWS, COLS, DST_T>& dst)

template< int SRC_T, int ROWS, int COLS, int NPC =1>
void min(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1,  unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
PaintMask 
template<int SRC_T,int MASK_T,int ROWS,int COLS>
void PaintMask(
Mat<ROWS,COLS,SRC_T>   &_src,
Mat<ROWS,COLS,MASK_T>&_mask, 
Mat<ROWS,COLS,SRC_T>&_dst,Scalar<HLS_MAT_CN(SRC_T),HLS_TNAME(SRC_T)> _color)

template< int SRC_T,int MASK_T, int ROWS, int COLS,int NPC=1>
void paintmask(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src_mat, xf::Mat<MASK_T, ROWS, COLS, NPC> & in_mask, xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst_mat, unsigned char _color[XF_CHANNELS(SRC_T,NPC)])
Reduce 
template<typename INTER_SUM_T, int ROWS, int COLS, int SRC_T, int DST_ROWS, int DST_COLS, int DST_T>
void Reduce(
         Mat<ROWS, COLS, SRC_T> &src,
         Mat<DST_ROWS, DST_COLS, DST_T> &dst,
         int dim,
         int op=HLS_REDUCE_SUM)

template< int REDUCE_OP, int SRC_T,int DST_T, int ROWS, int COLS,int ONE_D_HEIGHT, int ONE_D_WIDTH, int NPC=1>
void reduce(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src_mat,  xf::Mat<DST_T, ONE_D_HEIGHT, ONE_D_WIDTH, 1> & _dst_mat, unsigned char dim)
Zero 
template<int ROWS, int COLS, int SRC_T, int DST_T>
void Zero(Mat<ROWS, COLS, SRC_T>& src,
          Mat<ROWS, COLS, DST_T>& dst)

template< int SRC_T, int ROWS, int COLS, int NPC =1>
void zero(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
Sum 
template<typename DST_T, int ROWS, int COLS, int SRC_T>
Scalar<HLS_MAT_CN(SRC_T), DST_T> Sum(
        Mat<ROWS, COLS, SRC_T>& src)

template< int SRC_T, int ROWS, int COLS, int NPC = 1>
void sum(xf::Mat<SRC_T, ROWS, COLS, NPC> & src1, double sum[XF_CHANNELS(SRC_T,NPC)] )
SubS 
template<int ROWS, int COLS, int SRC_T, typename _T, int DST_T>
void SubS(Mat<ROWS, COLS, SRC_T>& src,
        Scalar<HLS_MAT_CN(SRC_T), _T> scl,
        Mat<ROWS, COLS, DST_T>& dst)

template<int POLICY_TYPE, int SRC_T, int ROWS, int COLS, int NPC =1>
void SubS(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1,  unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
SubRS 
template<int ROWS, int COLS, int SRC_T, typename _T, int DST_T>
void SubRS(Mat<ROWS, COLS, SRC_T>& src,
        Scalar<HLS_MAT_CN(SRC_T), _T> scl,
        Mat<ROWS, COLS, DST_T>& dst)

template<int POLICY_TYPE, int SRC_T, int ROWS, int COLS, int NPC =1>
void SubRS(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1, unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
Set 
template<int ROWS, int COLS, int SRC_T, typename _T, int DST_T>
void Set(Mat<ROWS, COLS, SRC_T>& src,
        Scalar<HLS_MAT_CN(SRC_T), _T> scl,
        Mat<ROWS, COLS, DST_T>& dst)

template< int SRC_T, int ROWS, int COLS, int NPC =1>
void set(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1,  unsigned char _scl[XF_CHANNELS(SRC_T,NPC)],xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
Absdiff 
template<int ROWS, int COLS, int SRC1_T, int SRC2_T, int DST_T>
void AbsDiff(
        Mat<ROWS, COLS, SRC1_T>& src1,
        Mat<ROWS, COLS, SRC2_T>& src2,
        Mat<ROWS, COLS, DST_T>& dst)

template<int SRC_T, int ROWS, int COLS, int NPC =1>
void absdiff(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1,xf::Mat<SRC_T, ROWS, COLS, NPC> & _src2,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
And 
template<int ROWS, int COLS, int SRC1_T, int SRC2_T, int DST_T>
void And(
        Mat<ROWS, COLS, SRC1_T>& src1,
        Mat<ROWS, COLS, SRC2_T>& src2,
        Mat<ROWS, COLS, DST_T>&  dst)

template<int SRC_T, int ROWS, int COLS, int NPC = 1>
void bitwise_and(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1, xf::Mat<SRC_T, ROWS, COLS, NPC> & _src2, xf::Mat<SRC_T, ROWS, COLS, NPC> &_dst)
Dilate 
template<int Shape_type,int ITERATIONS,int SRC_T, int DST_T, typename KN_T,int IMG_HEIGHT,int IMG_WIDTH,int K_HEIGHT,int K_WIDTH>
void Dilate(Mat<IMG_HEIGHT, IMG_WIDTH, SRC_T>&_src,Mat<IMG_HEIGHT, IMG_WIDTH, DST_T&_dst,Window<K_HEIGHT,K_WIDTH,KN_T>&_kernel)

template<int BORDER_TYPE, int TYPE, int ROWS, int COLS,int K_SHAPE,int K_ROWS,int K_COLS, int ITERATIONS, int NPC=1> 
void dilate (xf::Mat<TYPE, ROWS, COLS, NPC> & _src, xf::Mat<TYPE, ROWS, COLS, NPC> & _dst,unsigned char _kernel[K_ROWS*K_COLS])
Duplicate 
template<int ROWS, int COLS, int SRC_T, int DST_T>
void Duplicate(Mat<ROWS, COLS, SRC_T>& src,Mat<ROWS, COLS, DST_T>& dst1,Mat<ROWS, COLS, DST_T>& dst2)

template<int SRC_T, int ROWS, int COLS,int NPC>
void duplicateMat(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src, xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst1,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst2)
EqualizeHist 
template<int SRC_T, int DST_T,int ROW, int COL>
void EqualizeHist(Mat<ROW, COL, SRC_T>&_src,Mat<ROW, COL, DST_T>&_dst)

template<int SRC_T, int ROWS, int COLS, int NPC = 1>
void equalizeHist(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src,xf::Mat<SRC_T, ROWS, COLS, NPC> & _src1,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst)
erode 
template<int Shape_type,int ITERATIONS,int SRC_T, int DST_T, typename KN_T,int IMG_HEIGHT,int IMG_WIDTH,int K_HEIGHT,int K_WIDTH>
void Erode(Mat<IMG_HEIGHT, IMG_WIDTH, SRC_T>&_src,Mat<IMG_HEIGHT,IMG_WIDTH,DST_T>&_dst,Window<K_HEIGHT,K_WIDTH,KN_T>&_kernel)

template<int BORDER_TYPE, int TYPE, int ROWS, int COLS,int K_SHAPE,int K_ROWS,int K_COLS, int ITERATIONS, int NPC=1>
void erode (xf::Mat<TYPE, ROWS, COLS, NPC> & _src, xf::Mat<TYPE, ROWS, COLS, NPC> & _dst,unsigned char _kernel[K_ROWS*K_COLS])
FASTX 
template<int SRC_T,int ROWS,int COLS>
void FASTX(Mat<ROWS,COLS,SRC_T> &_src,
Mat<ROWS,COLS,HLS_8UC1>&_mask,HLS_TNAME(SRC_T)_threshold,bool _nomax_supression)

template<int NMS,int SRC_T,int ROWS, int COLS,int NPC=1>
void fast(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src_mat,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst_mat,unsigned char _threshold)
Filter2D 
template<int SRC_T, int DST_T, typename KN_T, typename POINT_T,
int IMG_HEIGHT,int IMG_WIDTH,int K_HEIGHT,int K_WIDTH>
void Filter2D(Mat<IMG_HEIGHT, IMG_WIDTH, SRC_T> &_src,Mat<IMG_HEIGHT, IMG_WIDTH, DST_T>  &_dst,Window<K_HEIGHT,K_WIDTH,KN_T>&_kernel,Point_<POINT_T>anchor)

template<int BORDER_TYPE,int FILTER_WIDTH,int FILTER_HEIGHT, int SRC_T,int DST_T, int ROWS, int COLS,int NPC>
void filter2D(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src_mat,xf::Mat<DST_T, ROWS, COLS, NPC> & _dst_mat,short int filter[FILTER_HEIGHT*FILTER_WIDTH],unsigned char _shift)
GaussianBlur 
template<int KH,int KW,typename BORDERMODE,int SRC_T,int DST_T,int ROWS,int COLS>
void GaussianBlur(Mat<ROWS, COLS, SRC_T>
&_src, Mat<ROWS, COLS, DST_T>	  
&_dst,double sigmaX=0,double sigmaY=0)

template<int FILTER_SIZE, int BORDER_TYPE, int SRC_T, int ROWS, int COLS,int NPC = 1>
void GaussianBlur(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src, xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst, float sigma)
Harris 
template<int blockSize,int Ksize,typename KT,int SRC_T,int DST_T,int ROWS,int COLS>
void Harris(Mat<ROWS, COLS, SRC_T>	    
&_src,Mat<ROWS, COLS, DST_T>&_dst,KT k,int threshold

template<int FILTERSIZE,int BLOCKWIDTH, int NMSRADIUS,int SRC_T,int ROWS, int COLS,int NPC=1,bool USE_URAM=false>
void cornerHarris(xf::Mat<SRC_T, ROWS, COLS, NPC> & src,xf::Mat<SRC_T, ROWS, COLS, NPC> & dst,uint16_t threshold, uint16_t k)
CornerHarris 
template<int blockSize,int Ksize,typename KT,int SRC_T,int DST_T,int ROWS,int COLS>
void CornerHarris(
Mat<ROWS, COLS, SRC_T>&_src,Mat<ROWS, COLS, DST_T>&_dst,KT k)

template<int FILTERSIZE,int BLOCKWIDTH, int NMSRADIUS,int SRC_T,int ROWS, int COLS,int NPC=1,bool USE_URAM=false>
void cornerHarris(xf::Mat<SRC_T, ROWS, COLS, NPC> & src,xf::Mat<SRC_T, ROWS, COLS, NPC> & dst,uint16_t threshold, uint16_t k
HoughLines2 
template<unsigned int theta,unsigned int rho,typename AT,typename RT,int SRC_T,int ROW,int COL,unsigned int linesMax>
void HoughLines2(Mat<ROW,COL,SRC_T> &_src,
Polar_<AT,RT> (&_lines)[linesMax],unsigned int threshold)

template<unsigned int RHO,unsigned int THETA,int MAXLINES,int DIAG,int MINTHETA,int MAXTHETA,int SRC_T, int ROWS, int COLS,int NPC>
void HoughLines(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src_mat,float outputrho[MAXLINES],float outputtheta[MAXLINES],short threshold,short linesmax)
Integral 
template<int SRC_T, int DST_T,
int ROWS,int COLS>
void Integral(Mat<ROWS, COLS, SRC_T>&_src,
        Mat<ROWS+1, COLS+1, DST_T>&_sum )

template<int SRC_TYPE,int DST_TYPE, int ROWS, int COLS, int NPC>
void integral(xf::Mat<SRC_TYPE, ROWS, COLS, NPC> & _src_mat, xf::Mat<DST_TYPE, ROWS, COLS, NPC> & _dst_mat)
Merge 
template<int ROWS, int COLS, int SRC_T, int DST_T>
void Merge(
        Mat<ROWS, COLS, SRC_T>& src0,
        Mat<ROWS, COLS, SRC_T>& src1,
        Mat<ROWS, COLS, SRC_T>& src2,
        Mat<ROWS, COLS, SRC_T>& src3,
        Mat<ROWS, COLS, DST_T>& dst)

template<int SRC_T, int DST_T, int ROWS, int COLS, int NPC=1>
void merge(xf::Mat<SRC_T, ROWS, COLS, NPC> &_src1, xf::Mat<SRC_T, ROWS, COLS, NPC> &_src2, xf::Mat<SRC_T, ROWS, COLS, NPC> &_src3, xf::Mat<SRC_T, ROWS, COLS, NPC> &_src4, xf::Mat<DST_T, ROWS, COLS, NPC> &_dst)
MinMaxLoc 
template<int ROWS, int COLS, int SRC_T, typename P_T>
void MinMaxLoc(Mat<ROWS, COLS, SRC_T>& src,
P_T* min_val,P_T* max_val,Point& min_loc,
Point& max_loc)

template<int SRC_T,int ROWS,int COLS,int NPC=0>
void minMaxLoc(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src,int32_t *min_value, int32_t *max_value,uint16_t *_minlocx, uint16_t *_minlocy, uint16_t *_maxlocx, uint16_t *_maxlocy )
Mul 
template<int ROWS, int COLS, int SRC1_T, int SRC2_T, int DST_T>
void Mul(Mat<ROWS, COLS, SRC1_T>& src1,
        Mat<ROWS, COLS, SRC2_T>& src2,
        Mat<ROWS, COLS, DST_T>& dst)

 template<int POLICY_TYPE, int SRC_T, int ROWS, int COLS, int NPC = 1>
void multiply(xf::Mat<SRC_T, ROWS, COLS, NPC> & src1, xf::Mat<SRC_T, ROWS, COLS, NPC> & src2, xf::Mat<SRC_T, ROWS, COLS, NPC> & dst,float scale)
Not 
template<int ROWS, int COLS, int SRC_T, int DST_T>
void Not(Mat<ROWS, COLS, SRC_T>& src,
        Mat<ROWS, COLS, DST_T>& dst)

template<int SRC_T, int ROWS, int COLS, int NPC = 1>
void bitwise_not(xf::Mat<SRC_T, ROWS, COLS, NPC> & src, xf::Mat<SRC_T, ROWS, COLS, NPC> & dst)
Range 
template<int ROWS, int COLS, int SRC_T, int DST_T, typename P_T>
void Range(Mat<ROWS, COLS, SRC_T>& src,
        Mat<ROWS, COLS, DST_T>& dst,
        P_T start,P_T end)

template<int SRC_T, int ROWS, int COLS,int NPC=1>
void inRange(xf::Mat<SRC_T, ROWS, COLS, NPC> & src,unsigned char lower_thresh,unsigned char upper_thresh,xf::Mat<SRC_T, ROWS, COLS, NPC> & dst)
Resize 
template<int SRC_T, int ROWS,int COLS,int DROWS,int DCOLS>
void Resize (
        Mat<ROWS, COLS, SRC_T> &_src,
        Mat<DROWS, DCOLS, SRC_T> &_dst,
        int interpolation=HLS_INTER_LINEAR )

template<int INTERPOLATION_TYPE, int TYPE, int SRC_ROWS, int SRC_COLS, int DST_ROWS, int DST_COLS, int NPC, int MAX_DOWN_SCALE> 
void resize (xf::Mat<TYPE, SRC_ROWS, SRC_COLS, NPC> & _src, xf::Mat<TYPE, DST_ROWS, DST_COLS, NPC> & _dst)
sobel 
template<int XORDER, int YORDER, int SIZE, int SRC_T, int DST_T, int ROWS,int COLS,int DROWS,int DCOLS>
void Sobel (Mat<ROWS, COLS, SRC_T>
&_src,Mat<DROWS, DCOLS, DST_T> &_dst)

template<int BORDER_TYPE,int FILTER_TYPE, int SRC_T,int DST_T, int ROWS, int COLS,int NPC=1,bool USE_URAM = false>
void Sobel(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src_mat,xf::Mat<DST_T, ROWS, COLS, NPC> & _dst_matx,xf::Mat<DST_T, ROWS, COLS, NPC> & _dst_maty)
split 
template<int ROWS, int COLS, int SRC_T, int DST_T>
void Split(
        Mat<ROWS, COLS, SRC_T>& src,
        Mat<ROWS, COLS, DST_T>& dst0,
        Mat<ROWS, COLS, DST_T>& dst1,
        Mat<ROWS, COLS, DST_T>& dst2,
        Mat<ROWS, COLS, DST_T>& dst3)

template<int SRC_T, int DST_T, int ROWS, int COLS, int NPC=1> 
void extractChannel(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src_mat, xf::Mat<DST_T, ROWS, COLS, NPC> & _dst_mat, uint16_t _channel)
Threshold 
template<int ROWS, int COLS, int SRC_T, int DST_T>
void Threshold(
        Mat<ROWS, COLS, SRC_T>& src,
        Mat<ROWS, COLS, DST_T>& dst,
        HLS_TNAME(SRC_T) thresh,
        HLS_TNAME(DST_T) maxval,
        int thresh_type)

template<int THRESHOLD_TYPE, int SRC_T, int ROWS, int COLS,int NPC=1>
void Threshold(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src_mat,xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst_mat,short int thresh,short int maxval )
Scale 
template<int ROWS, int COLS, int SRC_T, int DST_T, typename P_T>
void Scale(Mat<ROWS, COLS, SRC_T>& src,Mat<ROWS, COLS, DST_T>& dst, P_T scale=1.0,P_T shift=0.0)

template< int SRC_T,int DST_T, int ROWS, int COLS, int NPC = 1>
void scale(xf::Mat<SRC_T, ROWS, COLS, NPC> & src1, xf::Mat<DST_T, ROWS, COLS, NPC> & dst,float scale, float shift)
InitUndistortRectifyMapInverse 
template<typename CMT, typename DT, typename ICMT, int ROWS, int COLS, int MAP1_T, int MAP2_T, int N>
void InitUndistortRectifyMapInverse (
 Window<3,3, CMT> cameraMatrix,DT(&distCoeffs)[N],Window<3,3, ICMT> ir, Mat<ROWS, COLS, MAP1_T>  &map1,Mat<ROWS, COLS, MAP2_T>  &map2,int noRotation=false)

template< int CM_SIZE, int DC_SIZE, int MAP_T, int ROWS, int COLS, int NPC >
void InitUndistortRectifyMapInverse (
		ap_fixed<32,12> *cameraMatrix,
		ap_fixed<32,12> *distCoeffs,
		ap_fixed<32,12> *ir,
		xf::Mat<MAP_T, ROWS, COLS, NPC> &_mapx_mat,xf::Mat<MAP_T, ROWS, COLS, NPC> &_mapy_mat,int _cm_size, int _dc_size)
Avg, mean, AvgStddev 
template<typename DST_T, int ROWS, int COLS, int SRC_T>
DST_T Mean(Mat<ROWS, COLS, SRC_T>& src)

template<int SRC_T,int ROWS, int COLS,int NPC=1>void meanStdDev(xf::Mat<SRC_T, ROWS, COLS, NPC> & _src,unsigned short* _mean,unsigned short* _stddev)
CvtColor 
template<typename CONVERSION,int SRC_T, int DST_T,int ROWS,int COLS>
void CvtColor(Mat<ROWS, COLS, SRC_T>	&_src,
        Mat<ROWS, COLS, DST_T>	&_dst)
Color Conversion
Note: All the functions except Reduce can process N-pixels per clock where N is power of 2.

Using the xfOpenCV Library

This section describes using the xfOpenCV library in the SDx development environment.

Note: The instructions in this section assume that you have downloaded and installed all the required packages. For more information, see the Prerequisites.

include folder constitutes all the necessary components to build a Computer Vision or Image Processing pipeline using the library. The folders common and core contain the infrastructure that the library functions need for basic functions, Mat class, and macros. The library functions are categorized into three folders, features, video and imgproc based on the operation they perform. The names of the folders are self-explanatory.

To work with the library functions, you need to include the path to the The xfOpenCV library is structured as shown in the following table. The include folder in the SDx project. You can include relevant header files for the library functions you will be working with after you source the include folder’s path to the compiler. For example, if you would like to work with Harris Corner Detector and Bilateral Filter, you must use the following lines in the host code: 


#include “features/xf_harris.hpp” //for Harris Corner Detector
#include “imgproc/xf_bilateral_filter.hpp” //for Bilateral Filter
#include “video/xf_kalmanfilter.hpp”

After the headers are included, you can work with the library functions as described in the xfOpenCV Library API Reference using the examples in the examples folder as reference.

The following table gives the name of the header file, including the folder name, which contains the library function.

Table 9. xfOpenCV Library Contents
Function Name File Path in the include folder
xf::accumulate imgproc/xf_accumulate_image.hpp
xf::accumulateSquare imgproc/xf_accumulate_squared.hpp
xf::accumulateWeighted imgproc/xf_accumulate_weighted.hpp
The xfOpenCV library is structured as shown in the following table.xf::absdiff, xf::add, xf::subtract, xf::bitwise_and, xf::bitwise_or, xf::bitwise_not, xf::bitwise_xor,xf::multiply ,xf::Max, xf::Min, xf::compare, xf::zero, xf::addS, xf::SubS, xf::SubRS ,xf::compareS, xf::MaxS, xf::MinS, xf::set core/xf_arithm.hpp
xf::addWeighted imgproc/xf_add_weighted.hpp
xf::bilateralFilter imgproc/xf_histogram.hpp
xf::boxFilter imgproc/xf_box_filter.hpp
xf::boundingbox imgproc/xf_boundingbox.hpp
xf::Canny imgproc/xf_canny.hpp
xf::Colordetect imgproc/xf_colorthresholding.hpp, imgproc/xf_bgr2hsv.hpp, imgproc/xf_erosion.hpp, imgproc/xf_dilation.hpp
xf::merge imgproc/xf_channel_combine.hpp
xf::extractChannel imgproc/xf_channel_extract.hpp
xf::convertTo imgproc/xf_convert_bitdepth.hpp
xf::crop imgproc/xf_crop.hpp
xf::filter2D imgproc/xf_custom_convolution.hpp
xf::nv122iyuv, xf::nv122rgba, xf::nv122yuv4, xf::nv212iyuv, xf::nv212rgba, xf::nv212yuv4, xf::rgba2yuv4, xf::rgba2iyuv, xf::rgba2nv12, xf::rgba2nv21, xf::uyvy2iyuv, xf::uyvy2nv12, xf::uyvy2rgba, xf::yuyv2iyuv, xf::yuyv2nv12, xf::yuyv2rgba,xf::rgb2iyuv,xf::rgb2nv12,xf::rgb2nv21,xf::rgb2yuv4,xf::rgb2uyvy,xf::rgb2yuyv,xf::rgb2bgr,xf::bgr2uyvy,xf::bgr2yuyv,xf::bgr2rgb,xf::bgr2nv12,xf::bgr2nv21,xf::iyuv2nv12,xf::iyuv2rgba,xf::iyuv2rgb,xf::iyuv2yuv4,xf::nv122uyvy,xf::nv122yuyv,xf::nv122nv21,xf::nv212rgb,xf::nv212bgr,xf::nv212uyvy,xf::nv212yuyv,xf::nv212nv12,xf::uyvy2rgb,xf::uyvy2bgr,xf::uyvy2yuyv,xf::yuyv2rgb,xf::yuyv2bgr,xf::yuyv2uyvy,xf::rgb2gray,xf::bgr2gray,xf::gray2rgb,xf::gray2bgr,xf::rgb2xyz,xf::bgr2xyz... imgproc/xf_cvt_color.hpp
xf::dilate imgproc/xf_dilation.hpp
xf::demosaicing imgproc/xf_demosaicing.hpp
xf::erode imgproc/xf_erosion.hpp
xf::fast features/xf_fast.hpp
xf::GaussianBlur imgproc/xf_gaussian_filter.hpp
xf::cornerHarris features/xf_harris.hpp
xf::calcHist imgproc/xf_histogram.hpp
xf::equalizeHist imgproc/xf_hist_equalize.hpp
xf::HOGDescriptor imgproc/xf_hog_descriptor.hpp
xf::Houghlines imgproc/xf_houghlines.hpp
xf::inRange imgproc/xf_inrange.hpp
xf::integralImage imgproc/xf_integral_image.hpp
xf::densePyrOpticalFlow video/xf_pyr_dense_optical_flow.hpp
xf::DenseNonPyrLKOpticalFlow video/xf_dense_npyr_optical_flow.hpp
xf::LUT imgproc/xf_lut.hpp
xf::KalmanFilter video/xf_kalmanfilter.hpp
xf::magnitude core/xf_magnitude.hpp
xf::MeanShift imgproc/xf_mean_shift.hpp
xf::meanStdDev core/xf_mean_stddev.hpp
xf::medianBlur imgproc/xf_median_blur.hpp
xf::minMaxLoc core/xf_min_max_loc.hpp
xf::OtsuThreshold imgproc/xf_otsuthreshold.hpp
xf::phase core/xf_phase.hpp
xf::paintmask imgproc/xf_paintmask.hpp
xf::pyrDown imgproc/xf_pyr_down.hpp
xf::pyrUp imgproc/xf_pyr_up.hpp
xf::reduce imgrpoc/xf_reduce.hpp
xf::remap imgproc/xf_remap.hpp
xf::resize imgproc/xf_resize.hpp
xf::scale imgproc/xf_scale.hpp
xf::Scharr imgproc/xf_scharr.hpp
xf::SemiGlobalBM imgproc/xf_sgbm.hpp
xf::Sobel imgproc/xf_sobel.hpp
xf::StereoPipeline imgproc/xf_stereo_pipeline.hpp
xf::sum imgproc/xf_sum.hpp
xf::StereoBM imgproc/xf_stereoBM.hpp
xf::SVM imgproc/xf_svm.hpp
xf::Threshold imgproc/xf_threshold.hpp
xf::warpTransform imgproc/xf_warp_transform.hpp

The different ways to use the xfOpenCV library examples are listed below:

Downloading and Using xfOpenCV Libraries from SDx GUI

You can download xfOpenCV directly from SDx GUI. To build a project using the example makefiles on the Linux platform:

  1. From SDx IDE, click Xilinx and select SDx Libraries.
  2. Click Download next to the Xilinx xfOpenCV Library.

    Figure: SDx Libraries

    The library is downloaded into <home directory>/Xilinx/SDx/2019.1/xfopencv. After the library is downloaded, the entire set of examples in the library are available in the list of templates while creating a new project.
    Note: The library can be added to any project from the IDE menu options.
  3. To add a library to a project, from SDx IDE, click Xilinx and select SDx Libraries.
  4. Select Xilinx xfOpenCV Library and click Add to project. The dropdown menu consists of options of which project the libraries need to be included to.

All the headers as part of the include/ folder in xfOpenCV library would be copied into the local project directory as <project_dir>/libs/xfopencv/include. All the settings required for the libraries to be run are also set when this action is completed.

Building a Project Using the Example Makefiles on Linux

Use the following steps to build a project using the example makefiles on the Linux platform:

  1. Open a terminal.
  2. When building for revision platform, set the environment variable SYSROOT to <the path to platform folder>/sw/a53_linux/a53_linux/sysroot/aarch64-xilinx-linux.
  3. Change the platform variable to point to the downloaded platform folder in makefile. Ensure that the folder name of the downloaded platform is unchanged.
  4. When building for revision platform , change IDIRS and LDIRS variables in the Makefile as follows:
    IDIRS = -I. -I${SYSROOT}/usr/include -I ../../include
LDIRS = --sysroot=${SYSROOT} -L=/lib -L=/usr/lib -Wl,-rpath-link=${SYSROOT}/lib,-rpath-link=${SYSROOT}/usr/lib
  5. Change the directory to the location where you want to build the example.
    cd <path to example>
  6. When building for revision platform , add #include"opencv2/imgcodecs/imgcodecs.hpp" in xf_headers.h file ,both in if and else part.
  7. Set the environment variables to run SDx development environment.
    • For c shell:
      source <SDx tools install path>/settings.csh
    • For bash shell:
      source <SDx tools install path>/settings.sh
  8. Type the make command in the terminal. The sd_card folder is created and can be found in the <path to example> folder.
Note: Ignore 2,4 and 6 steps when building for Non revision platforms.

Using reVISION Samples on the reVISION Platform

Use the following steps to run a unit test for bilateral filter on zcu104_rv_ss:

  1. Launch the SDx development environment using the desktop icon or the Start menu.

    The Workspace Launcher dialog appears.

  2. Click Browse to enter a workspace folder used to store your projects (you can use workspace folders to organize your work), then click OK to dismiss the Workspace Launcher dialog.
    Note: Before launching the SDx IDE on Linux, ensure that you use the same shell that you have used to set the $SYSROOT environment variable. This is usually the file path to the Linux root file system.

    The SDx development environment window opens with the Welcome tab visible when you create a new workspace. The Welcome tab can be closed by clicking the X icon or minimized if you do not wish to use it.

  3. Select File > New > Xilinx SDx Project from the SDx development environment menu bar.

    The New Project dialog box opens.

  4. Specify the name of the project. For example Bilateral.
  5. Click Next.

    The the Choose Hardware Platform page appears.

  6. From the Choose Hardware Platform page, click the Add Custom Platform button.
  7. Browse to the directory where you extracted the reVISION platform files. Ensure that you select the zcu104_rv_ss folder.
  8. From the Choose Hardware Platform page, select zcu104_rv_ss (custom).
  9. Click Next.

    The Templates page appears, containing source code examples for the selected platform.

  10. From the list of application templates, select bilateral - File I/O and click Finish.
  11. add #include"opencv2/imgcodecs/imgcodecs.hpp" in xf_headers.h file present under project/src/examples/bilateral/,both in if and else part.
  12. Click the Active build configurations drop-down from the SDx Project Settings window, to select the active configuration or create a build configuration.

    The standard build configurations are Debug and Release. To get the best runtime performance, switch to use the Release build configuration as it uses a higher compiler optimization setting than the Debug build configuration.

    Figure: SDx Project Settings - Active Build Configuration



  13. Set the Data motion network clock frequency (MHz) to the required frequency, on the SDx Project Settings page.
  14. Right-click the project and select Build Project or press Ctrl+B keys to build the project, in the Project Explorer view.
  15. Copy the contents of the newly created sd_card folder to the SD card. The sd_card folder contains all the files required to run designs on the ZCU104 board.
  16. Insert the SD card in the ZCU104 board card slot and switch it ON.
    Note: A serial port emulator (Teraterm/ minicom) is required to interface the user commands to the board.
  17. Upon successful boot, run the following command in the Teraterm terminal (serial port emulator.)
    #cd /media/card
#remount
  18. Run the .elf file for the respective functions.

    For more information, see the Using the xfOpenCV Library Functions on Hardware.

Using the xfOpenCV Library on a non-reVISION Platform

This section describes using the xfOpenCV library on a non-reVISION platform, in the SDx™ development environment. The examples in xfOpenCV require OpenCV libraries for successful compilation. As non-reVISION platform may or may not contain opencv libs, as a perquisites it is required to install/compile opencv libraries(with compatible libjpeg.so).

Note: The instructions in this section assume that you have downloaded and installed all the required packages. For more information, see the Prerequisites.

Use the following steps to import the xfOpenCV library into a SDx project and execute it on a custom platform:

  1. Launch the SDx development environment using the desktop icon or the Start menu.

    The Workspace Launcher dialog appears.

  2. Click Browse to enter a workspace folder used to store your projects (you can use workspace folders to organize your work), then click OK to dismiss the Workspace Launcher dialog.

    The SDx development environment window opens with the Welcome tab visible when you create a new workspace. The Welcome tab can be closed by clicking the X icon or minimized if you do not wish to use it.

  3. Select File > New > Xilinx SDx Project from the SDx development environment menu bar.

    The New Project dialog box opens.

  4. Specify the name of the project. For example Test.

  5. Click Next.

    The the Choose Hardware Platform page appears.

  6. From the Choose Hardware Platform page, select a suitable platform. For example, zcu102.

  7. Click Next.

    The Choose Software Platform and Target CPU page appears.

  8. From the Choose Software Platform and Target CPU page, select an appropriate software platform and the target CPU. For example, select A9 from the CPU dropdown list for ZC702 and ZC706 reVISION platforms.
  9. Click Next. The Templates page appears, containing source code examples for the selected platform.

  10. From the list of application templates, select Empty Application and click Finish.

    The New Project dialog box closes. A new project with the specified configuration is created. The SDx Project Settings view appears. Notice the progress bar in the lower right border of the view, Wait for a few moments for the C/C++ Indexer to finish.

  11. The standard build configurations are Debug and Release. To get the best run-time performance, switch to use the Release build configuration as it uses a higher compiler optimization setting than the Debug build configuration.

    Figure: SDx Project Settings - Active Build Configuration



  12. Set the Data motion network clock frequency (MHz) to the required frequency, on the SDx Project Settings page.
  13. Select the Generate bitstream and Generate SD card image check boxes.
  14. Right-click on the newly created project in the Project Explorer view.

  15. From the context menu that appears, select C/C++ Build Settings.

    The Properties for <project> dialog box appears.

  16. Click the Tool Settings tab.
  17. Expand the SDS++ Compiler > Directories tree.

  18. Click the icon to add the "<xfopencv_location>\include" and "<OpenCV_location>\include" folder locations to the Include Paths list.

    Note: The OpenCV library is not provided by Xilinx for custom platforms. You are required to provide the library. Use the reVISION platform in order to use the OpenCV library provided by Xilinx.

    Figure: SDS++ Compiler Settings



  19. In the same page, under SDS++ Compiler > Inferred Options > Software Platform, specify "-hls-target 1" in the Software Platform Inferred Flags.
  20. Click Apply.
  21. Expand the SDS++ Linker > Libraries tree.
  22. Click the icon and add the following libraries to the Libraries(-l) list. These libraries are required by OpenCV.
    • opencv_core
    • opencv_imgproc
    • opencv_imgcodecs
    • opencv_features2d
    • opencv_calib3d
    • opencv_flann
    • opencv_video
    • opencv_videoio

  23. Click the icon and add <opencv_Location>/lib folder location to the Libraries search path (-L) list.

    Note: The OpenCV library is not provided by Xilinx for custom platforms. You are required to provide the library. Use the reVISION platform in order to use the OpenCV library provided by Xilinx.

    Figure: SDS++ Linker Settings



  24. Click Apply to save the configuration.
  25. Click OK to close the Properties for <project> dialog box.
  26. Expand the newly created project tree in the Project Explorer view.
  27. Right-click the src folder and select Import. The Import dialog box appears.
  28. Select File System and click Next.
  29. Click Browse to navigate to the <xfopencv_Location>/examples folder location.

  30. Select the folder that corresponds to the library that you desire to import. For example, accumulate.

    Figure: Import Library Example Source Files



  31. Right-click the library function in the Project Explorer view and select Toggle HW/SW to move the function to the hardware.

    Figure: Moving a Library Function to the Hardware



  32. Right-click the project and select Build Project or press Ctrl+B keys to build the project, in the Project Explorer view.

    The build process may take anytime between few minutes to several hours, depending on the power of the host machine and the complexity of the design. By far, the most time is spent processing the routines that have been tagged for realization in hardware.

  33. Copy the contents of the newly created .\<workspace>\<function>\Release\sd_card folder to the SD card. The sd_card folder contains all the files required to run designs on a board.

  34. Insert the SD card in the board card slot and switch it ON.

    Note: A serial port emulator (Teraterm/ minicom) is required to interface the user commands to the board.
  35. Upon successful boot, navigate to the ./mnt folder and run the following command at the prompt:
    #cd /mnt
    Note: It is assumed that the OpenCV libraries are a port of the root filesystem. If not, add the location of OpenCV libraries to LD_LIBRARY_PATH using the $ export LD_LIBRARY_PATH=<location of OpenCV libraries>/lib command.
  36. Run the .elf executable file. For more information, see the Using the xfOpenCV Library Functions on Hardware.

Changing the Hardware Kernel Configuration

Use the following steps to change the hardware kernel configuration:
  1. Update the <path to xfOpenCV git folder>/xfOpenCV/examples/<function>/xf_config_params.h file.
  2. Update the makefile along with the xf_config_params.h file:
    1. Find the line with the function name in the makefile. For bilateral filter, the line in the makefile will be xf::BilateralFilter<3,1,0,1080,1920,1>.
    2. Update the template parameters in the makefile to reflect changes made in the xf_config_params.h file. For more details, see the xfOpenCV Library API Reference.

Using the xfOpenCV Library Functions on Hardware

The following table lists the xfOpenCV library functions and the command to run the respective examples on hardware. It is assumed that your design is completely built and the board has booted up correctly.

Table 10. Using the xfOpenCV Library Function on Hardware
Example Function Name Usage on Hardware
accumulate xf::accumulate ./<executable name>.elf <path to input image 1> <path to input image 2>
accumulatesquared xf::accumulateSquare ./<executable name>.elf <path to input image 1> <path to input image 2>
accumulateweighted xf::accumulateWeighted ./<executable name>.elf <path to input image 1> <path to input image 2>
addS xf::addS ./<executable name>.elf <path to input image>
arithm xf::absdiff, xf::add, xf::subtract, xf::bitwise_and, xf::bitwise_or, xf::bitwise_not, xf::bitwise_xor ./<executable name>.elf <path to input image 1> <path to input image 2>
addweighted xf::addWeighted ./<executable name>.elf <path to input image 1> <path to input image 2>
Bilateralfilter xf::bilateralFilter ./<executable name>.elf <path to input image>
Boxfilter xf::boxFilter ./<executable name>.elf <path to input image>
Boundingbox xf::boundingbox ./<executable name>.elf <path to input image> <No of ROI's>
Canny xf::Canny ./<executable name>.elf <path to input image>
channelcombine xf::merge ./<executable name>.elf <path to input image 1> <path to input image 2> <path to input image 3> <path to input image 4>
Channelextract xf::extractChannel ./<executable name>.elf <path to input image>
Colordetect xf::bgr2hsv, xf::colorthresholding, xf:: erode, and xf:: dilate ./<executable name>.elf <path to input image>
compare xf::compare ./<executable name>.elf <path to input image 1> <path to input image 2>
compareS xf::compareS ./<executable name>.elf <path to input image>
Convertbitdepth xf::convertTo ./<executable name>.elf <path to input image>
Cornertracker xf::cornerTracker ./exe <input video> <no. of frames> <Harris Threshold> <No. of frames after which Harris Corners are Reset>
crop xf::crop ./<executable name>.elf <path to input image>
Customconv xf::filter2D ./<executable name>.elf <path to input image>
cvtcolor IYUV2NV12 xf::iyuv2nv12 ./<executable name>.elf <path to input image 1> <path to input image 2> <path to input image 3>
cvtcolor IYUV2RGBA xf::iyuv2rgba ./<executable name>.elf <path to input image 1> <path to input image 2> <path to input image 3>
cvtcolor IYUV2YUV4 xf::iyuv2yuv4 ./<executable name>.elf <path to input image 1> <path to input image 2> <path to input image 3> <path to input image 4> <path to input image 5> <path to input image 6>
cvtcolor NV122IYUV xf::nv122iyuv ./<executable name>.elf <path to input image 1> <path to input image 2>
cvtcolor NV122RGBA xf::nv122rgba ./<executable name>.elf <path to input image 1> <path to input image 2>
cvtcolor NV122YUV4 xf::nv122yuv4 ./<executable name>.elf <path to input image 1> <path to input image 2>
cvtcolor NV212IYUV xf::nv212iyuv ./<executable name>.elf <path to input image 1> <path to input image 2>
cvtcolor NV212RGBA xf::nv212rgba ./<executable name>.elf <path to input image 1> <path to input image 2>
cvtcolor NV212YUV4 xf::nv212yuv4 ./<executable name>.elf <path to input image 1> <path to input image 2>
cvtcolor RGBA2YUV4 xf::rgba2yuv4 ./<executable name>.elf <path to input image>
cvtcolor RGBA2IYUV xf::rgba2iyuv ./<executable name>.elf <path to input image>
cvtcolor RGBA2NV12 xf::rgba2nv12 ./<executable name>.elf <path to input image>
cvtcolor RGBA2NV21 xf::rgba2nv21 ./<executable name>.elf <path to input image>
cvtcolor UYVY2IYUV xf::uyvy2iyuv ./<executable name>.elf <path to input image>
cvtcolor UYVY2NV12 xf::uyvy2nv12 ./<executable name>.elf <path to input image>
cvtcolor UYVY2RGBA xf::uyvy2rgba ./<executable name>.elf <path to input image>
cvtcolor YUYV2IYUV xf::yuyv2iyuv ./<executable name>.elf <path to input image>
cvtcolor YUYV2NV12 xf::yuyv2nv12 ./<executable name>.elf <path to input image>
cvtcolor YUYV2RGBA xf::yuyv2rgba ./<executable name>.elf <path to input image>
Demosaicing xf::demosaicing ./<executable name>.elf <path to input image>
Difference of Gaussian xf:: GaussianBlur, xf:: duplicateMat, xf:: delayMat, and xf::subtract ./<exe-name>.elf <path to input image>
Dilation xf::dilate ./<executable name>.elf <path to input image>
Erosion xf::erode ./<executable name>.elf <path to input image>
Fast xf::fast ./<executable name>.elf <path to input image>
Gaussianfilter xf::GaussianBlur ./<executable name>.elf <path to input image>
Harris xf::cornerHarris ./<executable name>.elf <path to input image>
Histogram xf::calcHist ./<executable name>.elf <path to input image>
Histequialize xf::equalizeHist ./<executable name>.elf <path to input image>
Hog xf::HOGDescriptor ./<executable name>.elf <path to input image>
Houghlines xf::HoughLines ./<executable name>.elf <path to input image>
inRange xf::inRange ./<executable name>.elf <path to input image>
Integralimg xf::integralImage ./<executable name>.elf <path to input image>
Lkdensepyrof xf::densePyrOpticalFlow ./<executable name>.elf <path to input image 1> <path to input image 2>
Lknpyroflow xf::DenseNonPyrLKOpticalFlow ./<executable name>.elf <path to input image 1> <path to input image 2>
Lut xf::LUT ./<executable name>.elf <path to input image>
Kalman Filter xf::KalmanFilter ./<executable name>.elf
Magnitude xf::magnitude ./<executable name>.elf <path to input image>
Max xf::Max ./<executable name>.elf <path to input image 1> <path to input image 2>
MaxS xf::MaxS ./<executable name>.elf <path to input image>
meanshifttracking xf::MeanShift ./<executable name>.elf <path to input video/input image files> <Number of objects to track>
meanstddev xf::meanStdDev ./<executable name>.elf <path to input image>
medianblur xf::medianBlur ./<executable name>.elf <path to input image>
Min xf::Min ./<executable name>.elf <path to input image 1> <path to input image 2>
MinS xf::MinS ./<executable name>.elf <path to input image>
Minmaxloc xf::minMaxLoc ./<executable name>.elf <path to input image>
otsuthreshold xf::OtsuThreshold ./<executable name>.elf <path to input image>
paintmask xf::paintmask ./<executable name>.elf <path to input image>
Phase xf::phase ./<executable name>.elf <path to input image>
Pyrdown xf::pyrDown ./<executable name>.elf <path to input image>
Pyrup xf::pyrUp ./<executable name>.elf <path to input image>
reduce xf::reduce ./<executable name>.elf <path to input image>
remap xf::remap ./<executable name>.elf <path to input image> <path to mapx data> <path to mapy data>
Resize xf::resize ./<executable name>.elf <path to input image>
scale xf::scale ./<executable name>.elf <path to input image>
scharrfilter xf::Scharr ./<executable name>.elf <path to input image>
set xf::set ./<executable name>.elf <path to input image>
SemiGlobalBM xf::SemiGlobalBM ./<executable name>.elf <path to left image> <path to right image>
sobelfilter xf::Sobel ./<executable name>.elf <path to input image>
stereopipeline xf::StereoPipeline ./<executable name>.elf <path to left image> <path to right image>
stereolbm xf::StereoBM ./<executable name>.elf <path to left image> <path to right image>
subRS xf::SubRS ./<executable name>.elf <path to input image>
subS xf::SubS ./<executable name>.elf <path to input image>
sum xf::sum ./<executable name>.elf <path to input image 1> <path to input image 2>
Svm xf::SVM ./<executable name>.elf
threshold xf::Threshold ./<executable name>.elf <path to input image>
warptransform xf::warpTransform ./<executable name>.elf <path to input image>
zero xf::zero ./<executable name>.elf <path to input image>