nGraph is able to import and execute ONNX models. Models are converted to
nGraph’s Intermediate Representation and converted to
objects, which can be compiled and executed with nGraph backends.
You can use nGraph’s Python API to run an ONNX model and nGraph can be used as a backend to ONNX with the add-on package nGraph ONNX.
In order to support ONNX, nGraph must be built with the
NGRAPH_ONNX_IMPORT_ENABLE flag. See Building nGraph-ONNX for more
information. All nGraph packages published on PyPI are built with ONNX
Importing an ONNX model¶
You can download models from the ONNX Model Zoo. For example, ResNet-50:
$ wget https://s3.amazonaws.com/download.onnx/models/opset_9/resnet50.tar.gz $ tar -xzvf resnet50.tar.gz
Use the following Python commands to convert the downloaded model to an nGraph
# Import ONNX and load an ONNX file from disk >>> import onnx >>> onnx_protobuf = onnx.load('resnet50/model.onnx') # Convert ONNX model to an ngraph model >>> from ngraph.impl.onnx_import import import_onnx_model >>> ng_function = import_onnx_model(onnx_protobuf.SerializeToString()) # The importer returns a list of ngraph models for every ONNX graph output: >>> print(ng_function) <Function: 'resnet50' ([1, 1000])>
This creates an nGraph
Function object, which can be used to execute a
computation on a chosen backend.
Running a computation¶
You can now create an nGraph
Runtime backend and use it to compile your
Function to a backend-specific
Computation object. Finally, you can
execute your model by calling the created
Computation object with input
# Using an nGraph runtime (CPU backend) create a callable computation object >>> import ngraph as ng >>> runtime = ng.runtime(backend_name='CPU') >>> resnet_on_cpu = runtime.computation(ng_function) >>> print(resnet_on_cpu) <Computation: resnet50(Parameter_269)> # Load an image (or create a mock as in this example) >>> import numpy as np >>> picture = np.ones([1, 3, 224, 224], dtype=np.float32) # Run computation on the picture: >>> resnet_on_cpu(picture) [array([[2.16105007e-04, 5.58412226e-04, 9.70510227e-05, 5.76671446e-05, 7.45318757e-05, 4.80892748e-04, 5.67404088e-04, 9.48728994e-05, ...
Find more information about nGraph and ONNX in the nGraph ONNX GitHub* repository.