Working with Backends¶
What is a backend?¶
In the nGraph Compiler stack, what we call a backend is responsible for function execution and value allocation. A backend can be used to carry out computations from a framework on a CPU, GPU, or ASIC; it can also be used with an Interpreter mode, which is primarily intended for testing, to analyze a program, or to help a framework developer customize targeted solutions.
nGraph also provides a way to use the advanced tensor compiler PlaidML as a backend; you can learn more about this backend and how to build it from source in our documentation: Building nGraph-PlaidML from source.
|Backend||Current nGraph support||Future nGraph support|
|Intel® Architecture Processors (CPUs)||Yes||Yes|
|Intel® Nervana™ Neural Network Processor™ (NNPs)||Yes||Yes|
Each backend must define a function
that registers a backend constructor function and ensures that
initializations are performed. An example that includes initializations
can be found in the
ngraph/src/runtime/cpu/cpu_backend.cpp file. See
also: Backend APIs.
How to use?¶
- Create a
Backend; think of it as a compiler.
Backendcan then produce an
- A single iteration of the executable is executed by calling the
callmethod on the
The nGraph execution API for
Executable objects is a simple, five-method
interface; each backend implements the following five functions:
create_tensor()method allows the bridge to create tensor objects in host memory or an accelerator’s memory.
read()methods are used to transfer raw data into and out of tensors that reside in off-host memory.
compile()method instructs the backend to prepare an nGraph function for later execution.
- And, finally, the
call()method is used to invoke an nGraph function against a particular set of tensors.
When specified as the generic backend – either manually or automatically
from a framework –
NGRAPH defaults to CPU, and it also allows for
additional device configuration or selection.
Because nGraph can select backends, specifying the
backend as a runtime environment variable also works if one is
present in your system:
An axpy.py example is optionally available to test; outputs will vary depending on the parameters specified.
NGRAPH_TF_BACKEND="INTELGPU" python3 axpy.py
NGRAPH_INTELGPU_DUMP_FUNCTION– dumps nGraph’s functions in dot format.
OpenCL is only needed for the PlaidML from nGraph; if you have only a CPU backend, it is not needed.
Install the latest Linux driver for your system. You can find a list of drivers at https://software.intel.com/en-us/articles/opencl-drivers; You may need to install OpenCL SDK in case of an
Any user added to “video” group:
sudo usermod –a –G video <user_id>
may, for example, be able to find details at the