C/C++ Kernels

Arbitrary precision integer data types are defined by ap_int or ap_uint for signed and unsigned integer respectively inside the header file ap_int.h. To use arbitrary precision integer data type:

• Add header file ap_int.h to the source code.
• Change the bit types to ap_int<N> or ap_uint<N>, where N is a bit-size from 1 to 1024.

The following example shows how the header file is added and the two variables are implemented to use 9-bit integer and 10-bit unsigned integer.

#include "ap_int.h" 
ap_int<9> var1 // 9 bit signed integer
ap_uint<10> var2 // 10 bit unsigned integer

Some existing applications use floating point data types as they are written for other hardware architectures. However, fixed-point data types are a useful replacement for floating point types which require many clock cycles to complete. When choosing to implement floating-point versus fixed-point arithmetic for your application and accelerators, carefully evaluate trade-offs in power, cost, productivity, and precision.

As discussed in Reduce Power and Cost by Converting from Floating Point to Fixed Point (WP491), using fixed-point arithmetic instead of floating point for applications can increase power efficiency, and lower the total power required. Unless the entire range of the floating-point type is required, the same accuracy can often be implemented with a fixed-point type, resulting in the same accuracy with smaller and faster hardware.

Fixed-point data types model the data as an integer and fraction bits. The fixed-point data type requires the ap_fixed header, and supports both a signed and unsigned form as follows:

Header file

ap_fixed.h

Signed fixed point

ap_fixed<W,I,Q,O,N>

Unsigned fixed point

ap_ufixed<W,I,Q,O,N>

• W = Total width < 1024 bits
• I = Integer bit width. The value of I must be less than or equal to the width (W). The number of bits to represent the fractional part is W minus I. Only a constant integer expression can be used to specify the integer width.
• Q = Quantization mode. Only predefined enumerated values can be used to specify Q. The accepted values are:
  • AP_RND: Rounding to plus infinity.
  • AP_RND_ZERO: Rounding to zero.
  • AP_RND_MIN_INF: Rounding to minus infinity.
  • AP_RND_INF: Rounding to infinity.
  • AP_RND_CONV: Convergent rounding.
  • AP_TRN: Truncation. This is the default value when Q is not specified.
  • AP_TRN_ZERO: Truncation to zero.
• O = Overflow mode. Only predefined enumerated values can be used to specify O. The accepted values are:
  • AP_SAT: Saturation.
  • AP_SAT_ZERO: Saturation to zero.
  • AP_SAT_SYM: Symmetrical saturation.
  • AP_WRAP: Wrap-around. This is the default value when O is not specified.
  • AP_WRAP_SM: Sign magnitude wrap-around.
• N = The number of saturation bits in the overflow WRAP modes. Only a constant integer expression can be used as the parameter value. The default value is zero.

In the example code below, the ap_fixed type is used to define a signed 18-bit variable with 6 bits representing the integer value above the binary point, and by implication, 12 bits representing the fractional value below the binary point. The quantization mode is set to round to plus infinity (AP_RND). Because the overflow mode and saturation bits are not specified, the defaults AP_WRAP and 0 are used.

#include <ap_fixed.h>
...
  ap_fixed<18,6,AP_RND> my_type;
...

When performing calculations where the variables have different numbers of bits (W), or different precision (I), the binary point is automatically aligned. For more information on using fixed-point data types, see C++ Arbitrary Precision Fixed-Point Types in the Vitis HLS User Guide (UG1399).

Memory mapped interfaces are inferred from pointer parameters. They allow kernels to read and write data in global memory, which is the memory that is shared between kernels and the host application. Therefore, memory mapped interfaces are a convenient way of sharing data across different elements of the accelerated application, but interfaces are only allowed for sequential and pipelined kernel execution modes as described in Kernel Execution Modes.

To customize the default interfaces assigned by the Vitis tools during compilation, you can use the INTERFACE pragma. For optimal performance, Xilinx recommends performing burst transfers, if possible up to the AXI protocol limit of 4 KB per transfer.

If the data is accessed sequentially, a streaming interface can be used. This interface enables direct streaming of data from the host to kernel, and from the kernel to host without the need to migrate the data through the global memory as an intermediate step. The streaming interface can also be used between two kernels where one kernel is streaming data as a producer to another kernel acting as a consumer. This transfer also occurs directly and without making use of global memory. For more information, refer to Streaming Data Transfers.

By default, every iteration of a loop only starts when the previous iteration has finished. In the loop example below, a single iteration of the loop adds two variables and stores the result in a third variable. Assume that in hardware this loop takes three cycles to finish one iteration. Also, assume that the loop variable len is 20, that is, the vadd loop runs for 20 iterations in the kernel. Therefore, it requires a total of 60 clock cycles (20 iterations * 3 cycles) to complete all the operations of this loop.

vadd: for(int i = 0; i < len; i++) {
  c[i] = a[i] + b[i];
}

vadd: for(int i = 0; i < len; i++) {
  #pragma HLS PIPELINE
  c[i] = a[i] + b[i];
}

In the example above, it is assumed that every iteration of the loop takes three cycles: read, add, and write. Without pipelining, each successive iteration of the loop starts in every third cycle. With pipelining the loop can start subsequent iterations of the loop in fewer than three cycles, such as in every second cycle, or in every cycle.

The number of cycles it takes to start the next iteration of a loop is called the initiation interval (II) of the pipelined loop. So II = 2 means each successive iteration of the loop starts every two cycles. An II = 1 is the ideal case, where each iteration of the loop starts in the very next cycle. When you use pragma HLS PIPELINE, the compiler always tries to achieve II = 1 performance.

The following figure illustrates the difference in execution between pipelined and non-pipelined loops. In this figure, (A) shows the default sequential operation where there are three clock cycles between each input read (II = 3), and it requires eight clock cycles before the last output write is performed.

If there are data dependencies inside a loop, as discussed in Loop Dependencies, it might not be possible to achieve II = 1, and a larger initiation interval might be the result.

vadd: for(int i = 0; i < 20; i++) {
  #pragma HLS UNROLL
  c[i] = a[i] + b[i];
}

In the preceding example, you can see pragma HLS UNROLL has been inserted into the body of the loop to instruct the compiler to unroll the loop completely. All 20 iterations of the loop are executed in parallel if that is permitted by any data dependency.

array_sum:for(int i=0;i<4;i++){
  #pragma HLS UNROLL factor=2
  sum += arr[i];
}

array_sum_unrolled:for(int i=0;i<4;i+=2){
  // Manual unroll by a factor 2
  sum += arr[i];
  sum += arr[i+1];
}

Data dependencies in loops can impact the results of loop pipelining or unrolling. These loop dependencies can be within a single iteration of a loop or between different iterations of a loop. The straightforward method to understand loop dependencies is to examine an extreme example. In the following code example, the result of the loop is used as the loop continuation or exit condition. Each iteration of the loop must finish before the next can start.

Minim_Loop: while (a != b) { 
  if (a > b) 
    a -= b; 
  else 
    b -= a;
}

This loop cannot be pipelined. The next iteration of the loop cannot begin until the previous iteration ends.

Dealing with various types of dependencies with the Vitis compiler is an extensive topic requiring a detailed understanding of the high-level synthesis procedures underlying the compiler. For more information, refer to "Dependencies with Vitis HLS" in Vitis HLS User Guide (UG1399).

Coding with nested loops is a common practice. Understanding how loops are pipelined in a nested loop structure is key to achieving the desired performance.

If the HLS PIPELINE pragma is applied to a loop nested inside another loop, the v++ compiler attempts to flatten the loops to create a single loop, and apply the PIPELINE pragma to the constructed loop. The loop flattening helps in improving the performance of the kernel.

The compiler is able to flatten the following types of nested loops:

1. Perfect nested loop:
   • Only the inner loop has a loop body.
   • There is no logic or operations specified between the loop declarations.
   • All the loop bounds are constant.
2. Semi-perfect nested loop:
   • Only the inner loop has a loop body.
   • There is no logic or operations specified between the loop declarations.
   • The inner loop bound must be a constant, but the outer loop bound can be a variable.

The following code example illustrates the structure of a perfect nested loop:

ROW_LOOP: for(int i=0; i< MAX_HEIGHT; i++) {
  COL_LOOP: For(int j=0; j< MAX_WIDTH; j++) {
    #pragma HLS PIPELINE
    // Main computation per pixel
  }
}

The above example shows a nested loop structure with two loops that performs some computation on incoming pixel data. In most cases, you want to process a pixel in every cycle, hence, PIPELINE is applied to the nested loop body structure. The compiler is able to flatten the nested loop structure in the example because it is a perfect nested loop.

The nested loop in the preceding example contains no logic between the two loop declarations. No logic is placed between the ROW_LOOP and COL_LOOP; all of the processing logic is inside the COL_LOOP. Also, both the loops have a fixed number of iterations. These two criteria help the v++ compiler flatten the loops and apply the PIPELINE constraint.

void adder(unsigned int *in, unsigned int *out, int inc, int size) {

  unsigned int in_internal[MAX_SIZE];
  unsigned int out_internal[MAX_SIZE];
  mem_rd: for (int i = 0 ; i < size ; i++){
    #pragma HLS PIPELINE
    // Reading from the input vector "in" and saving to internal variable
    in_internal[i] = in[i];
  }
  compute: for (int i=0; i<size; i++) {
  #pragma HLS PIPELINE
    out_internal[i] = in_internal[i] + inc;
  } 

  mem_wr: for(int i=0; i<size; i++) {
  #pragma HLS PIPELINE
    out[i] = out_internal[i];
  }
}

In the previous example, three sequential loops are shown: mem_rd, compute, and mem_wr.

• The mem_rd loop reads input vector data from the memory interface and stores it in internal storage.
• The main compute loop reads from the internal storage and performs an increment operation and saves the result to another internal storage.
• The mem_wr loop writes the data back to memory from the internal storage.

This code example is using two separate loops for reading and writing from/to the memory input/output interfaces to infer burst read/write.

By default, these loops are executed sequentially without any overlap. First, the mem_rd loop finishes reading all the input data before the compute loop starts its operation. Similarly, the compute loop finishes processing the data before the mem_wr loop starts to write the data. However, the execution of these loops can be overlapped, allowing the compute (or mem_wr) loop to start as soon as there is enough data available to feed its operation, before the mem_rd (or compute) loop has finished processing its data.

The loop execution can be overlapped using dataflow optimization as described in Dataflow Optimization.

In the dataflow coding example you should notice the following:

1. The pragma HLS dataflow is applied to instruct the compiler to enable dataflow optimization. This is not a data mover, which deals with interfacing between the PS and PL, but instead addresses how the data flows through the accelerator.
2. The stream class is used as a data transferring channel between each of the functions in the dataflow region.
   TIP: The stream class infers a first-in first-out (FIFO) memory circuit in the programmable logic. This memory circuit, which acts as a queue in software programming, provides data-level synchronization between the functions and achieves better performance.

void compute_kernel(ap_int<256> *inx, ap_int<256> *outx, DTYPE alpha) {
  hls::stream<unsigned int>inFifo;
  #pragma HLS STREAM variable=inFifo depth=32
  hls::stream<unsigned int>outFifo; 
  #pragma HLS STREAM variable=outFifo depth=32

  #pragma HLS DATAFLOW
  read_data(inx, inFifo);
  // Do computation with the acquired data
  compute(inFifo, outFifo, alpha);
  write_data(outx, outFifo);
  return;
}

The following behaviors can prevent the Vitis compiler from performing dataflow optimizations:

1. Single producer-consumer violations.
2. Bypassing tasks.
3. Feedback between tasks.
4. Conditional execution of tasks.
5. Loops with multiple exit conditions or conditions defined within the loop.

If any of the above conditions occur inside the dataflow region, you might need to re-architect your code to successfully achieve dataflow optimization.