Compiling Qbics

Download Source Code
Preparations
One-Key Build: Quick Way
One-Key Build: More Options
Build from Lean Source Code
- Modify Makefile
- Third-Party Libraries
Build GPU Version of Qbics
Build MPI Version of Qbics
Additional Modules of Plumed
Possible Issues

To compile Qbics, please follow the instructions step-by-step.

Download Source Code 

Just visit https://zhjun-sci.com/qbics, you can find 2 source code files:

Qbics Source Code The is the lean source code of Qbics. To compile it, you may have to compile some libraries and set optimization, link options by yourself. The bonus is that you can optimize Qbics at a better level on your machine.
Qbics Source Code with environment This is the Qbics source code with compilation environment. In principle, on most Linux and macOS systems, you can compile it with just one command. This is recommended if you want to compile it rapidly.

Preparations 

The minimal requirement for compiling Qbics is:

gcc, version >= 8.0;
g++, version >= 8.0;
gfortran, version >= 8.0;
make, version >= 3.8;
cmake, version >= 3.16;

Below are some others:

Intel MKL, version >= 2021.4.0, if MKL is to be used;
OpenMPI, version >= 3.0, if MPI version of Qbics is required;
nvcc, version >= 10.0, if GPU version of Qbics is required.

If you do not want to build third-party libraries by yourself, you can download Qbics source code with compilation environment from https://zhjun-sci.com/qbics, which is named as qbics-source-env.tar.gz. In most cases, you can compile it with just a few commands:

$ tar -xvzf qbics-source-env.tar.gz
$ cd qbics-source-env
$ ./build-qbics.sh

On most Linux machines, this can be done smoothly and excutables will be available in qbics-Release/bin.

If you want to rebuild Qbics, then:

$ ./build-qbics.sh clean
Type [yes] to make sure to delete *all* cache files: yes
$ ./build-qbics.sh

One-Key Build: More Options 

Parallel Compilation 

If you want to use more CPU cores to compile Qbics, open build-qbics.sh and find

build-qbics.sh

########################################################################
numCores=64                            # Number of cores to compile.
export CC=gcc FC=gfortran              # Compilers.

Just change numCores to the number you like.

Use Intel MKL 

Intel MKL can significantly improve the performance of Qbics. The default option of building Qbics is NOT using MKL since not all machine has MKL installed. However, if you have MKL on your machine, you can simple activae MKL by:

$ tar -xvzf qbics-source-env.tar.gz
$ cd qbics-source-env
$ ./build-qbics.sh mkl

Note that, these commands assume that Intel MKL is installed on /opt/intel. Open build-qbics.sh and you can see:

build-qbics.sh

if test $1 = "mkl"; then
    source /opt/intel/oneapi/setvars.sh
    useMKL=1
fi

Open qbics-source/Makefile and you can also see:

qbics-source/Makefile

# Third-party library.
# ... omitted ...
ifdef eigen_mkl
    LIBBLAS = -L/opt/intel/oneapi/mkl/2021.4.0/lib/intel64 -Wl,--no-as-needed -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm
else
    LIBBLAS = ../third-party/OpenBLAS-0.3.28/lib/libopenblas.a
endif

# Third-party include.
# ... omitted ...
ifdef eigen_mkl
    INCBLAS = -I/opt/intel/oneapi/mkl/2021.4.0/include
else
    INCBLAS =
endif

So, if you install MKL to another path, like /usr/intel, you have to change all /opt/intel shown above to /usr/intel to enable compilers to find MKL.

You can see that all linear algebras are done down with OpenBLAS if Intel MKL is not used.

Attention

Here, we compile Qbics with GNU compiler and link with Intel MKL library. We do NOT use Intel compiler here.

Aggressive Optimizations 

If you are pursuing extreme performance, you can also try:

Link to Intel MKL, like mentioned above. This is the most effective way.
In qbics-source/Makefile, you can find the following lines:

qbics-source/Makefile

CXX     = g++
CXXFLAG = -O2 --std=c++17 -fopenmp -ffast-math -fno-finite-math-only -fexpensive-optimizations -Wall -mavx2 -mfma
CC      = gcc
CCFLAG  = -O2 -fopenmp -Wall
FC      = gfortran
FCFLAG  = -O2 -fopenmp -Wall

In CXXFLAG, we have given optimal options. However, you can replace -O2 to -O3, and add options like -march=native -mtune=native. This may or may NOT improve performance, and sometimes it may decrease performance. Please do benchmark to confirm your options.

According to our benchmark, using clang or intel compiler do NOT improve the performance as compared with GNU compiler, but you can try if they do on your machine.
For third-party libraries, you can also try to modify their compilation options. Please refer to their documentations.

Build from Lean Source Code 

Modify Makefile 

If you have your own third-party libraries and want to build only Qbics source code, just visit https://zhjun-sci.com/qbics and download qbics-source.tar.gz. After decompressing the package, find Makefile:

Makefile

# Code information.
SHELL=/bin/bash
CODEINFO = -DCodeCommit="\"`git rev-parse HEAD`\"" \
    -DCodeFlag="\"beta testing\"" \
    -DCodeMajorVer=0 \
    -DCodeMinorVer=4 \
    -DCodeUser="\"`whoami | sed 's/\\\\/\\\\\\\\/g'`\"" \
    -DCodeMachine="\"`hostname`\""
CODENAME = qbics
# ...omitted...
# Third-party library.
LIBGNU    = -lgfortran -lquadmath
LIBXC     = ../third-party/libxc-6.2.2/lib/libxc.a
LIBDFTD   = ../third-party/dftd3-0.9/libdftd3.a
ifeq ($(MAKECMDGOALS),mac-cpu)
    LIBPLUMED = ../third-party/plumed-2.9.2/lib/libplumed.dylib
    LIBXTB    = ../third-party/xtb-6.5.0/lib/libxtb.a
else
    LIBPLUMED = ../third-party/plumed-2.9.2/lib/libplumed.a -lz
    LIBXTB    = ../third-party/xtb-6.5.0/lib/libxtb.a ../third-party/xtb-6.5.0/lib/libmctc-lib.a #../third-party/xtb-6.5.0/lib/results_patch.o
endif
LIBFFTW3  = ../third-party/fftw-3.3.10/lib/libfftw3.a ../third-party/fftw-3.3.10/lib/libfftw3_omp.a
LIBDLFIND = ../third-party/dl-find/libdlf.a
ifdef eigen_mkl
    LIBBLAS = -L/opt/intel/oneapi/mkl/2021.4.0/lib/intel64 -Wl,--no-as-needed -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm
else
    LIBBLAS = ../third-party/OpenBLAS-0.3.28/lib/libopenblas.a
endif
LIBCUDA   = /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudart_static.a -lrt
LIBSPONGE =

# Third-party include.
INCBOOST  = -I../third-party/boost_1_78_0
INCEIGEN  = -I../third-party/eigen-3.4.0
INCLIBXC  = -I../third-party/libxc-6.2.2/include
INCDFTD   = -I../third-party/dftd3-0.9
INCPLUMED = -I../third-party/plumed-2.9.2/include
INCFFTW3  = -I../third-party/fftw-3.3.10/include
INCXTB    = -I../third-party/xtb-6.5.0/include
INCDLFIND = -I../third-party/dl-find
ifdef eigen_mkl
    INCBLAS = -I/opt/intel/oneapi/mkl/2021.4.0/include
else
    INCBLAS =
endif
INCCUDA   = -I/usr/local/cuda-11.4/targets/x86_64-linux/include/

You need to change these lines:

Line 4: -DCodeFlag can be your unique flag, like Specific for XX Lab;
Line 12-30: The paths to third-party libraries. For example, LIBXC is the path to libxc library libxc.a;
Line 33-46: The paths to header files. For example, INCLIBXC is the path to libxc header flies like xc.h. In addition, for dftd3-lib and DL-Find library, INCDFTD and INCDLFIND should be the paths to their module files like dftd3_api.mod and dlf_allocate.mod, respectively.
Please pay attention that the options for some libraries depends on operating system or if Intel MKL is used, like LIBBLAS or INCBLAS.

Of course, you can also change C++, C, and Fortran compilers with CXX, CC, and FC, respectively.

Now, you can compile Qbics. Below are some examples:

$ make linux-cpu -j 32             # On Linux, wihtout Intel MKL, with 32 CPU cores
$ make linux-cpu -j 32 eigen_mkl=1 # On Linux, wiht Intel MKL, with 32 CPU cores
$ make mac-cpu -j 32               # On macOS, wihtout Intel MKL, with 32 CPU cores

If the compilation succeeds, the executable bin/qbics-linux-cpu or bin/qbics-mac-cpu can be found.

Attention

This Makefile can also be used to compile Qbics on Windows:

$ make win-cpu -j 8 # On Windows, wihtout Intel MKL, with 8 CPU cores

However, you will have to install MSYS2 and MinGW64 on Windows, which may be quite complicated. I recommend you just download Windows executable from https://zhjun-sci.com/qbics.

Third-Party Libraries 

Here, third-party libraries mean libraries that are not present on a native Linux or macOS machine. All third-party libraries needed by Qbics are:

Boost: https://www.boost.org/ A de facto standard library for extending C++ functionalities.
Eigen: https://eigen.tuxfamily.org/ A C++ linear algebra library.
libxc: https://libxc.gitlab.io/ A library for density functionals.
dftd3-lib: https://github.com/dftbplus/dftd3-lib A library for Grimme dispersion correction.
plumed: https://github.com/plumed/plumed2/releases/ A library for enhanced sampling.
libfftw3: https://www.fftw.org/download.html A library for fast Frourier transformation.
xTB: https://github.com/grimme-lab/xtb A powerful semi-empirical quantum chemical method library.
DL-Find: https://chemshell.org/dl-find/ An open-source geometry optimisation library.
OpenBLAS: https://github.com/OpenMathLib/OpenBLAS The high performance linear algebra library backend for Eigen. It can be replaced by Intel MKL.
HDF5: https://www.hdfgroup.org/ The implementation of the HDF5 data model.
(Optional) Intel MKL: https://www.intel.cn/content/www/cn/zh/developer/tools/oneapi/onemkl-download.html High performance math kernel library.
(Optional) OpenMPI: https://www.open-mpi.org/ An implementation for MPI protocal to do parallelzation.
(Optional) CUDA: https://developer.nvidia.com/cuda-toolkit A development environment for creating high-performance, GPU-accelerated applications.

Build GPU Version of Qbics 

To build GPU version of Qbics, the first thing is to make sure that CPU version can be smoothly built. If you use Qbics Source Code with environment, you should have run ./build-qbics.sh to successfully build a CPU version of Qbics.

After this, open Makefile to find these lines:

Makefile

# ...omitted...
LIBCUDA   = /usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudart_static.a -lrt
LIBSPONGE =

# Third-party include.
# ...omitted...
INCCUDA   = -I/usr/local/cuda-11.4/targets/x86_64-linux/include/

You should modify LIBCUDA and INCCUDA to enable compiler to find CUDA library libcudart_static.a and CUDA head files, then you can compile GPU version of Qbics:

$ make linux-gpu -j 32 # On Linux, GPU version, with 32 CPU cores

If the compilation succeeds, the executable bin/qbics-linux-gpu an be found. Currently, GPU version of Qbics is only available on Linux.

Build MPI Version of Qbics 

To build GPU version of Qbics, the first thing is to make sure that CPU version can be smoothly built. If you use Qbics Source Code with environment, you should have run ./build-qbics.sh to successfully build a CPU version of Qbics.

After this, make sure that your system has an MPI implementation, like OpenMPI or MPICH. We recommend OpenMPI. To compile MPI version of Qbics:

$ make linux-cpu-mpi -j 32 # On Linux, MPI version, with 32 CPU cores

If the compilation succeeds, the executable bin/qbics-linux-cpu-mpi an be found. Currently, MPI version of Qbics is only available on Linux.

Additional Modules of Plumed 

As shown above, Qbics can be compiled with plumed library. However, some additional modules of plumed are not included in the default plumed library (before 2025). For example, OPES algorithm or libtorch support are not compiled by default. If you want to use these features, you have to compile plumed with additional option like --enable-modules=+module1+module2+... when you compile plumed.

For all modules, pleasse refer to https://www.plumed.org/doc-v2.9/user-doc/html/mymodules.html.

For example, if you want to use the OPES algorithm and the PIV collective variable, you can compile plumed with:

$ ./configure --enable-modules=+opes+piv

This can be added to build-qbics.sh. If you do not want to recompile Qbics, you can use this to recompile plumed only and link it to Qbics.

Possible Issues 

In compiling Qbics, you may encounter some issues. Here are some common issues and solutions:

C++: fatal error: Killed signal terminated program cc1plus

Reason: This is because the compilation of Qbics and related third-party libraries may take a long time and a large amount of memory. If you are working on a login node in some clusters, the system may kill your job due to the exceed of user quotas.

Solution: In build-qbics.sh, set numnCores=1 to compile Qbics with only one CPU core. If this still does not work, you can try to submit a job to compile Qbics on computing nodes.

undefined reference to 'gsl_vector_get'

Reason: In the compilation of Plumed, the GSL library is used in some old versions of Linux. In the newer versions of Linux, the GSL library is not needed anymore.

Solution: Open Makefile and find the following line:

Makefile

ifeq ($(MAKECMDGOALS),mac-cpu)
    LIBPLUMED = ../third-party/plumed-2.9.2/lib/libplumed.dylib
    LIBXTB    = ../third-party/xtb-6.5.0/lib/libxtb.a
else
    LIBPLUMED = ../third-party/plumed-2.9.2/lib/libplumed.a -lz
    LIBXTB    = ../third-party/xtb-6.5.0/lib/libxtb.a ../third-party/xtb-6.5.0/lib/libmctc-lib.a #../third-party/xtb-6.5.0/lib/results_patch.o
endif

In LIBPLUMED, add -lgsl, like this:

Makefile

ifeq ($(MAKECMDGOALS),mac-cpu)
    LIBPLUMED = ../third-party/plumed-2.9.2/lib/libplumed.dylib
    LIBXTB    = ../third-party/xtb-6.5.0/lib/libxtb.a
else
    LIBPLUMED = ../third-party/plumed-2.9.2/lib/libplumed.a -lz -lgsl
    LIBXTB    = ../third-party/xtb-6.5.0/lib/libxtb.a ../third-party/xtb-6.5.0/lib/libmctc-lib.a #../third-party/xtb-6.5.0/lib/results_patch.o
endif

Error: Rank mismatch in argument 'coords' at (1) (rank-l and scalar)

Reason: This is because the coordinates of the system are not in the correct format.

Solution: Check the coordinates of the system and make sure that they are in the correct format.

Error: Rank mismatch in argument 'coords' at (1)(rank-local and scalar)

Reason: This is due to the different treatment of DL-Find code by different versions of Fortran compiler.

Solution: Open third-party/dl-find-src/Makefile.qbics and find the following line:

third-party/dl-find-src/Makefile.qbics

ifeq ($(arch),gfortran)
  F90        = gfortran
  F90FLAGS   =  -C -M $(OBJDIR)
  F90FLAGS   =   -fimplicit-none  -fbounds-check -std=gnu -Wconversion \
                   -Wsurprising #-fallow-argument-mismatch

Delete # to make -fallow-argument-mismatch to be used:

third-party/dl-find-src/Makefile.qbics

ifeq ($(arch),gfortran)
  F90        = gfortran
  F90FLAGS   =  -C -M $(OBJDIR)
  F90FLAGS   =   -fimplicit-none  -fbounds-check -std=gnu -Wconversion \
                   -Wsurprising -fallow-argument-mismatch