Working with Backends¶
What is a backend?¶
In the nGraph Compiler stack, what we call a backend is responsible for function execution and value allocation. A backend can be used to carry out computations from a framework on a CPU, GPU, or ASIC; it can also be used with an Interpreter mode, which is primarily intended for testing, to analyze a program, or to help a framework developer customize targeted solutions.
|Backend||Current nGraph support||Future nGraph support|
|Intel® Architecture Processors (CPUs)||Yes||Yes|
|Intel® Nervana™ Neural Network Processor™ (NNPs)||Yes||Yes|
|NVIDIA* CUDA (GPUs)||Yes||Some|
How to use?¶
- Create a
Backend; think of it as a compiler.
Backendcan then produce an
- A single iteration of the executable is executed by calling the
callmethod on the
The nGraph execution API for
Executable objects is a simple, five-method
interface; each backend implements the following five functions:
create_tensor()method allows the bridge to create tensor objects in host memory or an accelerator’s memory.
read()methods are used to transfer raw data into and out of tensors that reside in off-host memory.
compile()method instructs the backend to prepare an nGraph function for later execution.
- And, finally, the
call()method is used to invoke an nGraph function against a particular set of tensors.
Additional resources for device or framework-specific configurations:
OpenCL is needed for the PlaidML from nGraph; this is not needed if you have only a CPU backend.
Install the latest Linux driver for your system. You can find a list of drivers at https://software.intel.com/en-us/articles/opencl-drivers; You may need to install OpenCL SDK in case of an
Any user added to “video” group:
sudo usermod –a –G video <user_id>
may, for example, be able to find details at the
When specified as the generic backend – either manually or automatically
from a framework –
NGRAPH defaults to CPU, and it also allows for
additional device configuration or selection.
Because nGraph can select backends, specifying the
backend as a runtime environment variable also works if one is
present in your system:
An axpy.py example is optionally available to test; outputs will vary depending on the parameters specified.
NGRAPH_TF_BACKEND="INTELGPU" python3 axpy.py
NGRAPH_INTELGPU_DUMP_FUNCTION– dumps nGraph’s functions in dot format.