Backend APIs

Each backend BACKEND needs to define the macro ${BACKEND}_API appropriately to import symbols referenced from outside the library, and to export them from within the library. See any of the ${backend}_backend_visibility header files for an example; see also What is a backend?

class Backend

Interface to a generic backend.

Backends are responsible for function execution and value allocation.

Subclassed by ngraph::runtime::cpu::CPU_Backend, ngraph::runtime::dynamic::DynamicBackend, ngraph::runtime::gcpu::GCPUBackend, ngraph::runtime::interpreter::INTBackend, ngraph::runtime::nop::NOPBackend, ngraph::runtime::plaidml::PlaidML_Backend

Public Types

enum Property

A set of properties supported by a backend.

Values:

memory_attach

New tensor can use attached memory.

Public Functions

virtual std::shared_ptr<ngraph::runtime::Tensor> create_tensor(const ngraph::element::Type &element_type, const Shape &shape) = 0

Create a tensor specific to this backend.

Return
shared_ptr to a new backend-specific tensor
Parameters
  • element_type: The type of the tensor element
  • shape: The shape of the tensor

virtual std::shared_ptr<ngraph::runtime::Tensor> create_tensor(const ngraph::element::Type &element_type, const Shape &shape, void *memory_pointer) = 0

Create a tensor specific to this backend.

Return
shared_ptr to a new backend-specific tensor
Parameters
  • element_type: The type of the tensor element
  • shape: The shape of the tensor
  • memory_pointer: A pointer to a buffer used for this tensor. The size of the buffer must be sufficient to contain the tensor. The lifetime of the buffer is the responsibility of the caller.

template<typename T>
std::shared_ptr<ngraph::runtime::Tensor> create_tensor(const Shape &shape)

Create a tensor of C type T specific to this backend.

Return
shared_ptr to a new backend specific tensor
Parameters
  • shape: The shape of the tensor

std::shared_ptr<ngraph::runtime::Tensor> create_dynamic_tensor(const ngraph::element::Type &element_type, const PartialShape &shape)

Create a dynamic tensor specific to this backend, if the backend supports dynamic tensors.

Return
shared_ptr to a new backend-specific tensor
Parameters
  • element_type: The type of the tensor element
  • shape: The shape of the tensor
Exceptions
  • std::invalid_argument: if the backend does not support dynamic tensors

virtual bool supports_dynamic_tensors()

Return
true if this backend supports dynamic tensors, else false.

virtual std::shared_ptr<Executable> compile(std::shared_ptr<Function> func, bool enable_performance_data = false) = 0

Compiles a Function.

Return
compiled function or nullptr on failure
Parameters
  • func: The function to compile

std::shared_ptr<runtime::Executable> compile(std::shared_ptr<Function> func, ngraph::pass::PassConfig &pass_config, bool enable_performance_data = false)

Compiles a Function.

Return
compiled function or nullptr on failure
Parameters
  • func: The function to compile
  • pass_config: Configuration object for defining compilation options

std::shared_ptr<runtime::Executable> load(std::istream &input_stream)

Loads a previously saved Executable object from a stream.

Return
A compiled function or throws an exception on error
Parameters
  • input_stream: the opened input stream containing the saved Executable

bool is_supported(const Node &node) const

Test if a backend is capable of supporting an op.

Return
true if the op is supported, false otherwise.
Parameters
  • node: is the op to test.

bool is_supported_property(const Property prop) const

Test if a backend particular property is supported.

Return
true if the property is supported, false otherwise.
Parameters
  • prop: is the feature to test.

std::shared_ptr<ngraph::Node> get_backend_op(const std::string &op_name, ...)

Return a backend specific op (that is not a core ngraph op). The string op_name is the requested op, which a backend may or may not implement. If unsupported, nullptr is returned, else a backend op is returned. The variadic input is used to pass inputs that the op constructor might take.

Return
a shared pointer to the op if found, else nullptr
Parameters
  • op_name: is the name of the backend specific op

bool set_config(const std::map<std::string, std::string> &config, std::string &error)

Allows sending backend specific configuration. The map contains key, value pairs specific to a particluar backend. The definition of these key, value pairs is defined by each backend.

Return
true if the configuration is supported, false otherwise. On false the error parameter value is valid.
Parameters
  • config: The configuration map sent to the backend
  • error: An error string describing any error encountered

virtual Allocator *get_host_memory_allocator()

Returns memory allocator used by backend for host allocations.

virtual void set_host_memory_allocator(Allocator *allocator)

Set the host memory allocator to be used by the backend.

Parameters
  • allocator: is pointer to host memory allocator object

virtual Allocator *get_device_memory_allocator()

Returns memory allocator used by backend for device allocations.

bool is_device_memory(void *ptr)

method for each supported backend to determine if the passed pointer is in device pinned memory or not

Parameters
  • ptr: pointer to the memory to determine if its in device memory or not

virtual std::string get_version() const

Get the version of the backend The default value of 0.0.0 is chosen to be a parsable version number.

Public Static Functions

std::shared_ptr<runtime::Backend> create(const std::string &type, bool must_support_dynamic = false)

Create a new Backend object.

Return
shared_ptr to a new Backend or nullptr if the named backend does not exist.
Parameters
  • type: The name of a registered backend, such as “CPU” or “GPU”. To select a subdevice use “GPU:N” where sN is the subdevice number.
  • must_support_dynamic: If true, the returned Backend object will support dynamic tensors. If the underlying backend has native support for dynamic tensors, then that backend object will be returned directly. Otherwise, it will be wrapped with DynamicWrapperBackend. This feature is EXPERIMENTAL.

vector<string> get_registered_devices()

Query the list of registered devices.

Return
A vector of all registered devices.