Specialized Graph Constructs

The AI Engine compiler automatically allocates the desired number of buffers for each memory connection. There are several different cases.

• Lookup tables are always allocated as single buffers because they are expected to be read-only and private to a kernel. No locks are needed to synchronize lookup table accesses because they are expected to be accessed in an exclusive manner.
• Window connections are usually assigned double buffers if the producer and consumer kernels are mapped to different processors or if the producer or the consumer is a DMA. This enables the two agents to operate in a pipelined manner using ping-pong synchronization with two locks. The AI Engine compiler automatically generates this synchronization in the respective processor main functions.
• If the producer and consumer kernels are mapped to the same processor, then the window connection is given only one buffer and no lock synchronization is needed because the kernels are executed sequentially.
• Run-time parameter connections can be assigned double buffers (default) along with a selector word to choose the next buffer to be accessed.

Run-time parameter connections can also be assigned single buffers. Sometimes, with window connections, it is desirable to use only single buffer synchronization instead of double buffers. This is useful when the local data memory is at a premium and the performance penalty of using a single buffer for data transfer is not critical. This can be achieved using the single_buffer(port<T>&) constraint.

single_buffer(first.in[0]); //For window input or RTP input
single_buffer(first.inout[0]); //For RTP output

The AI Engine compiler supports C++ kernel classes. The following example shows how to set filter coefficients and the number of samples of a FIR filter class through a constructor. The C++ kernel class allows internal states for each kernel instance to be encapsulated within the corresponding class object. In the following code, you can see an example of this where the filter coefficients (coeffs) are specified through the constructor. This resolves the problem of using file scope variable, global variable, or static function scope variable to store the internal states of a C function kernel. When multiple instances of such a kernel are mapped to the same core, the internal state variables are shared across multiple instances and cause conflicts.

//fir.h
#pragma once
#include "adf.h"
#define NUM_COEFFS 12

class FIR
{
private:
    int32 coeffs[NUM_COEFFS];
    int32 tapDelayLine[NUM_COEFFS];
    uint32 numSamples;

public:
    FIR(const int32(&coefficients)[NUM_COEFFS], uint32 samples);
    void filter(input_window_int32* in, output_window_int32* out);
    static void registerKernelClass()
    {
        REGISTER_FUNCTION(FIR::filter);
    }
};

You are required to write the static void registerKernelClass() method in the header file. Inside the registerKernelClass() method, you need to call the REGISTER_FUNCTION macro. This macro is used to register the class run method to be executed on the AI Engine core to perform the kernel functionality. In the preceding example FIR::filter is registered using this macro. The kernel class constructor and run method should be implemented inside a separate source file. The implementation of a run method of a kernel class is the same as writing a kernel function described in previous chapters.

//fir.cpp
//implementation in this example is not optimized and is for illustration purpose
#include "fir.h"

FIR::FIR(const int32(&coefficients)[NUM_COEFFS], uint32 samples)
{
    for (int i = 0; i < NUM_COEFFS; i++)
        coeffs[i] = coefficients[i];

    for (int i = 0; i < NUM_COEFFS; i++)
        tapDelayLine[i] = 0;

    numSamples = samples;
}

void FIR::filter(input_window_int32* in, output_window_int32* out)
{
    for (int i = 0; i < numSamples; i++)
    {
        for (int j = NUM_COEFFS-1; j > 0; j--)
            tapDelayLine[j] = tapDelayLine[j - 1];

        tapDelayLine[0] = window_readincr(in);

        int32 y = 0;
        for (int j = 0; j < NUM_COEFFS; j++)
        {
            y += coeffs[j] * tapDelayLine[j];
        }

        window_writeincr(out, y);
    }
}

//graph.h
#pragma once
#include "adf.h"
#include "fir.h"
using namespace adf;

class mygraph : public graph
{
public:
    input_port in1, in2;
    output_port out1, out2;
    kernel k1, k2;

    mygraph()
    {
        //see lab8.3 for narrow filter coefficients
        k1 = kernel::create_object<FIR>(std::vector<int>({ 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 }), 8);
        runtime<ratio>(k1) = 0.1;
        source(k1) = "src/fir.cpp";

        //see lab8.3 for wide filter coefficients
        k2 = kernel::create_object<FIR>(std::vector<int>({ -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539 }), 8);
        runtime<ratio>(k2) = 0.1;
        source(k2) = "src/fir.cpp";

        connect<window<32>>(in1, k1.in[0]);
        connect<window<32>>(in2, k2.in[0]);

        connect<window<32>>(k1.out[0], out1);
        connect<window<32>>(k2.out[0], out2);
    }
};

For a kernel class with a non-default constructor, you can specify the constructor parameter values in the arguments of kernel::create_object, when creating a representation of a kernel instance. In the previous example, two FIR filter kernels (k1 and k2) are created using kernel::create_object<FIR>. k1 has filter coefficients { 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 } and k2 has filter coefficients { -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539 }. Both of them consume eight samples for each invocation.

The following code shows the AI Engine compiler generated program. The two FIR kernel objects are instantiated with the proper constructor parameters.

//Work/aie/x_y/src/x_y.cc
...
FIR i4({180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504}, 8);
FIR i5({-21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539}, 8);

int main(void) {
    ...
    // Kernel call : i4:filter
    i4.filter(get_input_window_int32(window_buf0_buf0d),get_output_window_int32(window_buf2_buf2d));
    ...
    // Kernel call : i5:filter
    i5.filter(get_input_window_int32(window_buf1_buf1d),get_output_window_int32(window_buf3_buf3d));
    ...
}

A kernel class can have a member variable occupying a significant amount of memory space that might not fit into program memory. The location of the kernel class member variable can be controlled. The AI Engine compiler supports array reference member variables that allow the compiler to allocate or constrain the memory space while passing the reference to the object.

//fir.h
#pragma once
#include "adf.h"
#define NUM_COEFFS 12

class FIR
{
private:
    int32 (&coeffs)[NUM_COEFFS];
    int32 tapDelayLine[NUM_COEFFS];
    uint32 numSamples;

public:
    FIR(int32(&coefficients)[NUM_COEFFS], uint32 samples);
    void filter(input_window_int32* in, output_window_int32* out);
    static void registerKernelClass()
    {
        REGISTER_FUNCTION(FIR::filter);
        REGISTER_PARAMETER(coeffs);
    }
};

//fir.cpp
#include "fir.h"
FIR::FIR(int32(&coefficients)[NUM_COEFFS], uint32 samples)
    : coeffs(coefficients)
{
    for (int i = 0; i < NUM_COEFFS; i++)
        tapDelayLine[i] = 0;

    numSamples = samples;
}

void FIR::filter(input_window_int32* in, output_window_int32* out)
{
...
}

The previous example shows a slightly modified version of the FIR kernel class. Here, member variable coeffs is a int32 (&)[NUM_COEFFS] data type. The constructor initializer coeffs(coefficients) initializes coeffs to the reference to an array allocated externally to the class object. To let the AI Engine compiler know that the coeffs member variable is intended to be allocated by the compiler, you must use REGISTER_PARAMETER to register an array reference member variable inside the registerKernelClass

The use of kernel::create_object to create a representation of a FIR kernel instance and to specify the initial value of the constructor parameters is the same as in the previous example. See the following code.

//graph.h
...
class mygraph : public graph
{
...
    mygraph()
    {
        k1 = kernel::create_object<FIR>(std::vector<int>({ 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 }), 8);
        ...
        k2 = kernel::create_object<FIR>(std::vector<int>({ -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539 }), 8);
        ...
    }
};

The following code shows the corresponding AI Engine compiler generated program. The memory spaces for int32 i4_coeffs[12] and int32 i5_coeffs[15] are outside the kernel object instances and are passed into the FIR objects by reference.

//Work/aie/x_y/src/x_y.cc
int32 i4_coeffs[12] = {180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504};
FIR i4(i4_coeffs, 8);
int32 i5_coeffs[12] = {-21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539};
FIR i5(i5_coeffs, 8);

int main(void) {
    ...
    // Kernel call : i4:filter
    i4.filter(get_input_window_int32(window_buf0_buf0d),get_output_window_int32(window_buf2_buf2d));
    ...
    // Kernel call : i5:filter
    i5.filter(get_input_window_int32(window_buf1_buf1d),get_output_window_int32(window_buf3_buf3d));
    ...
}

Because the memory space for an array reference member variable is allocated by the AI Engine compiler, the location constraint can be applied to constrain the memory location of these arrays, as shown in the following example code. The REGISTER_PARAMETER macro allows kernel::create_object to create a parameter handle for an array reference member variable, like k1.param[0] and k2.param[0], and the location<parameter> constraint can be applied.

//graph.h
...
class mygraph : public graph
{
...
    mygraph()
    {
        k1 = kernel::create_object<FIR>(std::vector<int>({ 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 }), 8);
        ...
        k2 = kernel::create_object<FIR>(std::vector<int>({ -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539 }), 8);
        ...

        location<parameter>(k1.param[0]) = address(…);
        location<parameter>(k2.param[0]) = bank(…);
    }
};

The C++ kernel class header files and the C++ kernel function template (see C++ Template Support) should not contain single-core specific intrinsic APIs and pragmas. This is the same programming guideline as writing regular C function kernels. This is because these header files are included in the graph header file and can be cross-compiled as part of the PS program. The Arm® cross-compiler cannot understand single-core intrinsic APIs or pragmas. Single-core specific programming content must be kept inside the source files.

A template is a powerful tool in C++. By passing the data type as a parameter, you eliminate the need to rewrite code to support different data types. Templates are expanded at compile time, like macros. The difference is that the compiler performs type checking before template expansion. The source code contains template functions and class definitions, but the compiled code can contain multiple copies of same function or class. Type parameters, non-type parameters, default arguments, scalar parameters, and template parameters can be passed to a template, where the compiler instantiates the function or class accordingly.

• Support for general C++ template features.
• Supported data types (T) and connection types between kernels:
  • Data type (T): int8, uint8, int16, uint16, cint16, int32, uint32, cint32, int64, uint64, float, cfloat
    IMPORTANT: acc48 and cacc48 data types are not supported in template stream connections.
  • Function parameter type: input_window<T>, output_window<T>, input_stream<T>, output_stream<T>
• The compiler does not support pre-compiled headers for template kernels.

Function Templates

Function template source code defines a generic function that can be used for different data types. Example function template:

// add.h
template<typename ELEMENT_TYPE, int FACTOR, size_t NUM_SAMPLES> void add(input_window<ELEMENT_TYPE>* in, 
 output_window<ELEMENT_TYPE>* out);

// add.cpp
template<typename ELEMENT_TYPE, int FACTOR, size_t NUM_SAMPLES> void add(input_window<ELEMENT_TYPE>* in, 
 output_window<ELEMENT_TYPE>* out)
{
    for (int i=0; i<NUM_SAMPLES; i++)
    {
        ELEMENT_TYPE value = window_readincr(in);
        value += FACTOR;
        window_writeincr(out, value);
    }
}

// graph.h
mygraph()
{
    k[0] = kernel::create(add<int32, 6, 8>);
    k[1] = kernel::create(add<int16, 3, 8>);
    for (int i=0; i<NUM_KERNELS; i++)
    {
        runtime<ratio>(k[i]) = 0.3;
        source(k[i]) = "src/add.cpp";
    }

    connect<window<32>>(in[0], k[0].in[0]);
    connect<window<32>>(k[0].out[0], out[0]);

    connect<window<16>>(in[1], k[1].in[0]);
    connect<window<16>>(k[1].out[0], out[1]);
}

where:

• add.h defines a template add() function.
• add.cpp defines the code for the template add() function.
• graph.h uses the template add() function within mygraph class.

Class Templates

Like function templates, class templates are useful when a class defines an object that is independent of a specific data type. Example class template:

// fir.h
...
template<size_t NUM_COEFFS, typename ELEMENT_TYPE> class FIR
{
private:
    ELEMENT_TYPE (&coeffs)[NUM_COEFFS];
    ELEMENT_TYPE tapDelayLine[NUM_COEFFS];
    uint32 numSamples;

public:
    FIR(ELEMENT_TYPE(&coefficients)[NUM_COEFFS], uint32 samples);

    void filter(input_window<ELEMENT_TYPE>* in, output_window<ELEMENT_TYPE>* out);

    //user needs to write this function to register necessary info
    static void registerKernelClass()
    {
        REGISTER_FUNCTION(FIR::filter);
        REGISTER_PARAMETER(coeffs);
    }
};

// fir.cpp
...
template<size_t NUM_COEFFS, typename ELEMENT_TYPE> FIR<NUM_COEFFS, ELEMENT_TYPE>::FIR(ELEMENT_TYPE(&coefficients)[NUM_COEFFS], uint32 samples):coeffs(coefficients)
{
    ...
}

template<size_t NUM_COEFFS, typename ELEMENT_TYPE> void FIR<NUM_COEFFS, ELEMENT_TYPE>::filter(input_window<ELEMENT_TYPE>* in, output_window<ELEMENT_TYPE>* out)
{
    ...
}

// graph.h
...
mygraph()
{
    k1 = kernel::create_object<FIR<12, int32>>(std::vector<int>({ 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 }), 8);
    runtime<ratio>(k1) = 0.1;
    source(k1) = "src/fir.cpp";
    headers(k1) = { "src/fir.h" };

    k2 = kernel::create_object<FIR<15, int32>>(std::vector<int>({ -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539, 0, 0, 0 }), 8);
    runtime<ratio>(k2) = 0.1;
    source(k2) = "src/fir.cpp";
    headers(k2) = { "src/fir.h" };
    ...
}

where:

• fir.h defines a class template where class FIR is declared.
• fir.cpp contains class FIR implementation and the class FIR member function filter implementation.
• graph.h demonstrates the template class FIR instantiation within the mygraph class.

Various multicast scenarios are supported in the graph, such as from a window to multiple windows, from stream to multiple streams, from PLIO to multiple windows etc. This section lists the supported types of multicast from a single source to multiple destinations. For more details on PLIO, FileIO and GMIO, see Using a Virtual Platform.

Note the following.

• All source and destination windows in the multicast connections are required to have the same size.
• RTP and packet switching are not covered in this section.
• If the multicast type is supported, the destination number is not limited if it can fit into the hardware.

When multiple streams are connected to the same source, the data is sent to all the destination ports at the same time and is only sent when all destinations are ready to receive data. This might cause stream stall or design hang if the FIFO depth of the stream connections are not deep enough. Refer to the examples in AI Engine Kernel Coding Best Practices Guide (UG1079) for more information about the stream stalls and potential solutions.