You can now incorporate NCCL in your GPU-accelerated application. Step 6: Create symbolic link for NCCL header file $ sudo ln -s /usr/local/nccl-2.1/include/nccl.h /usr/include/nccl.h You can read more about parameter expansion in this article. However, when $LD_LIBRARY_PATH is already set, the parameter value will be replaced with “ :$LD_LIBRARY_PATH” (without the quotes). If $LD_LIBRARY_PATH is unset, the parameter value is an empty string. Step 5: Add NCCL library into LD_LIBRARY_PATH $ vi ~/.profile. Step 4: Copy the extracted directory to /usr/local $ sudo mkdir -p /usr/local/nccl-2.1 Step 2: Download NCCL deb package from NVIDIA website ( ) and put into the installer directoryįrom the download options, ensure that the OS agnostic installer is chosen as shown in the picture below.įor NCCL 2.1.15 and CUDA 9.1, the tar file name is nccl_2.1.15-1+cuda9.1_x86_64.txz $ tar -xvf nccl_2.1.15-1+cuda9.1_x86_64.txz We will create a new directory named installers/nccl in the user’s home directory. Step 1: Select or create a directory to store the installer file. To install NCCL for use by TensorFlow on Ubuntu 16.04, the following procedure can be used as a reference. TensorFlow can also be configured with NCCL for improved performance. The communication patterns are described in the following picture. NCCL supports various collective communication patterns that include: Hadoop, for example, supports MapReduce programming model that splits a compute job into chunks that are spread into the slave nodes and collected back by the master to produce the final output. People with background or experience in distributed system, such as Hadoop, may immediately relate this concept with similar model applied in the traditional distributed system.
With multiple GPUs working together, the task will be completed in less time, rendering a more performing system. This is especially helpful when the computation is complex. The idea is to enable GPUs to collectively work to complete certain computing task. NVIDIA Collective Communications Library (NCCL) is a library developed to provide parallel computation primitives on multi-GPU and multi-node environment.
0 kommentar(er)
