Interact with Backends

Backend

Backends are responsible for function execution and value allocation. They can be used to carry out a programmed computation from a framework by using a CPU or GPU; or they can be used with an Interpreter mode, which is primarily intended for testing, to analyze a program, or for a framework developer to develop a custom UI or API.

../_images/runtime.png
class Backend

Interface to a generic backend.

Backends are responsible for function execution and value allocation.

Subclassed by ngraph::runtime::cpu::CPU_Backend, ngraph::runtime::gpu::GPU_Backend, ngraph::runtime::interpreter::INTBackend

Public Functions

virtual std::shared_ptr<ngraph::runtime::TensorView> create_tensor(const ngraph::element::Type &element_type, const Shape &shape, void *memory_pointer) = 0

Return a handle for a tensor for given mem on backend device.

Public Static Functions

shared_ptr<runtime::Backend> create(const std::string &type)

Create a new Backend object.

Return
shared_ptr to a new Backend or nullptr if the named backend does not exist.
Parameters
  • type: The name of a registered backend, such as “CPU” or “GPU”. To select a subdevice use “GPU:N” where sN is the subdevice number.

vector<string> get_registered_devices()

Query the list of registered devices.

Return
A vector of all registered devices.

TensorView

class TensorView

Subclassed by ngraph::runtime::cpu::CPUTensorView, ngraph::runtime::gpu::GPU_TensorView, ngraph::runtime::HostTensorView

Public Functions

virtual void write(const void *p, size_t tensor_offset, size_t n) = 0

Write bytes directly into the tensor.

Parameters
  • p: Pointer to source of data
  • tensor_offset: Offset into tensor storage to begin writing. Must be element-aligned.
  • n: Number of bytes to write, must be integral number of elements.

virtual void read(void *p, size_t tensor_offset, size_t n) const = 0

Read bytes directly from the tensor.

Parameters
  • p: Pointer to destination for data
  • tensor_offset: Offset into tensor storage to begin reading. Must be element-aligned.
  • n: Number of bytes to read, must be integral number of elements.