Build and Test¶
There are a few common paths to take when manually building the nGraph Compiler stack from source code. Today nGraph supports various developers working on all parts of the Deep Learning stack, and the way you decide to build or install components ought to depend on the capabilities of your hardware, and how you intend to use it.
A “from scratch” source-code build of the nGraph Library enables the CPU,
Interpreter
, and unit tests by default. See Building nGraph from source
for more detail.
A “from scratch” source-code build that defaults to the PlaidML backend
contains rich algorithm libraries akin to those that were previously available
only to developers willing to spend extensive time writing, testing, and
customizing kernels. An NGRAPH_PLAIDML
dist can function like a framework
that lets developers compose, train, and even deploy DL
models in their preferred language on neural networks of any size. This is
a good option if, for example, you are working on a laptop with a high-end
GPU that you want to use for compute. See Building nGraph-PlaidML from source
for instructions on how to build.
In either case, there are some prerequisites that your system will need to build from sources.
Prerequisites¶
Operating System | Compiler | Build System | Status | Additional Packages |
---|---|---|---|---|
CentOS 7.4 64-bit | GCC 4.8 | CMake 3.9.0 | supported | wget zlib-devel ncurses-libs ncurses-devel patch diffutils gcc-c++ make git perl-Data-Dumper |
Ubuntu 16.04 or 18.04 (LTS) 64-bit | Clang 3.9 | CMake 3.5.1 + GNU Make | supported | build-essential cmake clang-3.9 clang-format-3.9 git curl zlib1g zlib1g-dev libtinfo-dev unzip autoconf automake libtool |
Clear Linux* OS for Intel® Architecture version 28880 | Clang 8.0 | CMake 3.14.2 | experimental | bundles machine-learning-basic c-basic python-basic python-basic-dev dev-utils |
Building nGraph from source¶
Important
The default cmake procedure (no build flags) will
install ngraph_dist
to an OS-level location like /usr/bin/ngraph_dist
or /usr/lib/ngraph_dist
. Here we specify how to build locally to the
location of ~/ngraph_dist
with the cmake target -DCMAKE_INSTALL_PREFIX=~/ngraph_dist
.
All of the nGraph Library documentation presumes that ngraph_dist
gets
installed locally. The system location can be used just as easily by
customizing paths on that system. See the ngraph/CMakeLists.txt
file to change or customize the default CMake procedure.
Ubuntu LTS build steps¶
The process documented here will work on Ubuntu* 16.04 (LTS) or on Ubuntu 18.04 (LTS).
Ensure you have installed the Prerequisites for Ubuntu*.
Clone the NervanaSystems
ngraph
repo:$ git clone https://github.com/NervanaSystems/ngraph.git $ cd ngraph
Create a build directory outside of the
ngraph/src
directory tree; somewhere likengraph/build
, for example:$ mkdir build && cd build
Generate the GNU Makefiles in the customary manner (from within the
build
directory). This command enables ONNX support in the library and sets the target build location at~/ngraph_dist
, where it can be found easily.$ cmake .. -DNGRAPH_ONNX_IMPORT_ENABLE=ON -DCMAKE_INSTALL_PREFIX=~/ngraph_dist
Other optional build flags – If running
gcc-5.4.0
orclang-3.9
, remember that you can also appendcmake
with the prebuilt LLVM option to speed-up the build. Another option if your deployment system has Intel® Advanced Vector Extensions (Intel® AVX) is to target the accelerations available directly by compiling the build as follows during the cmake step:-DNGRAPH_TARGET_ARCH=skylake-avx512
.$ cmake .. [-DNGRAPH_USE_PREBUILT_LLVM=OFF] [-DNGRAPH_TARGET_ARCH=skylake-avx512]
Run
$ make
andmake install
to installlibngraph.so
and the header files to~/ngraph_dist
:$ make # note: make -j <N> may work, but sometimes results in out-of-memory errors if too many compilation processes are used $ make install
(Optional, requires doxygen, Sphinx, and breathe). Run
make html
inside thedoc/sphinx
directory of the cloned source to build a copy of the website docs locally. The low-level API docs with inheritance and collaboration diagrams can be found inside the/docs/doxygen/
directory. See the Contributing to documentation for more details about how to build documentation for nGraph.
CentOS 7.4 build steps¶
The process documented here will work on CentOS 7.4.
Ensure you have installed the Prerequisites for CentOS*, and update the system with yum.
$ sudo yum update
Install Cmake 3.4:
$ wget https://cmake.org/files/v3.4/cmake-3.5.0.tar.gz $ tar -xzvf cmake-3.5.0.tar.gz $ cd cmake-3.5.0 $ ./bootstrap --system-curl --prefix=~/cmake $ make && make install
Clone the NervanaSystems
ngraph
repo via HTTPS and use Cmake 3.5.0 to build nGraph Libraries to~/ngraph_dist
. This command enables ONNX support in the library (optional).$ cd /opt/libraries $ git clone https://github.com/NervanaSystems/ngraph.git $ cd ngraph && mkdir build && cd build $ ~/cmake/bin/cmake .. -DCMAKE_INSTALL_PREFIX=~/ngraph_dist -DNGRAPH_ONNX_IMPORT_ENABLE=ON $ make && sudo make install
Building nGraph-PlaidML from source¶
The following instructions will create the ~/ngraph_plaidml_dist
locally:
Ensure you have installed the Prerequisites for your OS.
Install the prerequisites for the backend. Our hybrid
NGRAPH_PLAIDML
backend works best with Python3 versions. We recommend that you use a virtual environment, due to some of the difficulties that users have seen when trying to install outside of a venv.$ sudo apt install python3-pip $ pip install plaidml $ plaidml-setup
Clone the source code, create and enter your build directory:
$ git clone https://github.com/NervanaSystems/ngraph.git $ cd ngraph && mkdir build && cd build
Prepare the CMake files as follows:
$ cmake .. -DCMAKE_INSTALL_PREFIX=~/ngraph_plaidml_dist -DNGRAPH_CPU_ENABLE=OFF -DNGRAPH_PLAIDML_ENABLE=ON
Run make and
make install
. Note that if you are building outside a local or user path, you may need to runmake install
as the root user.$ make $ make install
This should create the shared library
libplaidml_backend.so
and nbench. Note that if you built in a virtual environment and runmake check
from it, the Google Test may report failures. Full tests can be run when PlaidML devices are available at the machine level.
For more about working with the PlaidML backend from nGraph, see our API documentation PlaidML from nGraph.
macOS* development¶
Note
Although we do not currently offer full support for the macOS platform, some configurations and features may work.
The repository includes two scripts (maint/check-code-format.sh
and
maint/apply-code-format.sh
) that are used respectively to check adherence
to libngraph
code formatting conventions, and to automatically reformat code
according to those conventions. These scripts require the command
clang-format-3.9
to be in your PATH
. Run the following commands
(you will need to adjust them if you are not using bash):
$ brew install [email protected] automake
$ mkdir -p $HOME/bin
$ ln -s /usr/local/opt/[email protected]/bin/clang-format $HOME/bin/clang-format-3.9
$ echo 'export PATH=$HOME/bin:$PATH' >> $HOME/.bash_profile
Testing the build¶
We use the googletest framework from Google for unit tests. The cmake
command automatically downloaded a copy of the needed gtest
files when
it configured the build directory.
To perform unit tests on the install:
Create and configure the build directory as described in our Build and Test guide.
Enter the build directory and run
make check
:$ cd build/ $ make check