linpack基准测试（HPL+MKL+IMPI）安装

　

HPL

参考http://blog.csdn.net/icefromair/article/details/51524476

1、系统环境

ubuntu server 16.04

软件：    

hpl-2.2.tar.gz
l_mkl_2018.1.163.tgz
l_mpi_2018.1.163.tgz

2、解压安装mkl

tar命名解压安装即可，然后进入目录，直接执行install.sh文件，根据提示安装即可。

node1@node1:~$ tar -zxvf l_mkl_2018.1.163.tgz
node1@node1:~$ cd l_mkl_2018.1.163/
node1@node1:~/l_mkl_2018.1.163$ ./install.sh#后面根据提示进行就可以了

3、解压安装mpi

 同mkl一样解压安装即可。

node1@node1:~$ tar -zxvf l_mpi_2018.1.163.tgz 
node1@node1:~$ cd l_mpi_2018.1.163/
node1@node1:~/l_mpi_2018.1.163$ ./install.sh #同样根据提示进行就可以

4、编译安装hpl

4.1、解压hpl文件

node1@node1:~$ tar -zxvf hpl-2.2.tar.gz
node1@node1:~$ cd hpl-2.2/

node1@node1:~/hpl-2.2$ cp setup/Make.linux_PII_CBLAS .

把setup目录里面的Make.Linux_PII_CBLAS文件复制到top目录也就是setup的上级目录，hpl-2.2目录下，然后命名一下

我的命名是Make.ubuntu_mkl_intel，文件里面arch=Make.ubuntu_mkl_intel，这两个是一样的。

修改后文件主要有下面内容（去掉了注释行）：

SHELL        = /bin/sh
CD           = cd
CP           = cp
LN_S         = ln -s
mkdir        = mkdir
RM           = /bin/rm -f
TOUCH        = touch
ARCH         = ubuntu_mkl_intel
TOPdir       = $(HOME)/hpl-2.2
INCdir       = $(TOPdir)/include
BINdir       = $(TOPdir)/bin/$(ARCH)
LIBdir       = $(TOPdir)/lib/$(ARCH)
HPLlib       = $(LIBdir)/libhpl.a 
MPdir        = /usr/local/intel/impi/2018.1.163/intel64
MPinc        = -I$(MPdir)/include
MPlib        = $(MPdir)/lib/libmpi.a
LAdir        = -L/usr/local/intel/mkl/lib/intel64 -L/usr/local/intel/lib/intel64#这里也可以通过导入链接库路径也最好这样，
LAinc        = -I/usr/local/intel/mkl/include #不然像我这样，最后执行xhpl文件会找不到链接库，要添加到LD_LIBRARY_PATH中才可以
LAlib        = -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lm#
F2CDEFS      =
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAdir) $(LAinc) $(MPinc) -lpthread
HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib) -lpthread
HPL_OPTS     = -DHPL_CALL_CBLAS
HPL_DEFS     = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
CC           = /usr/local/intel/impi/2018.1.163/bin64/mpicc -lpthread
CCNOOPT      = $(HPL_DEFS)
CCFLAGS      = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops
LINKER       = /usr/local/intel/impi/2018.1.163/bin64/mpicc -lpthread
LINKFLAGS    = $(CCFLAGS)
ARCHIVER     = ar
ARFLAGS      = r
RANLIB       = echo

接下来编译：

node1@node1:~/hpl-2.2$ make arch=ubuntu_mkl_intel
#如果没有报错，在当前目录下有个bin目录会看到ubuntu_mkl_intel目录，里面就有可以这行的文件了

5、运行测试：

node1@node1:~/hpl-2.2/bin/ubuntu_mkl_intel$ mpirun -n 4 ./xhpl 
./xhpl: ERROR while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory
./xhpl: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory
./xhpl: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory
./xhpl: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory

出现了我说上面说的问题，没有添加没有把共享库路径写进LD_LIBRARY_PATH，因为这个共享库没有在默认搜索路径下，所以现在添加到LD_LIBRARY_PATH就可以。

node1@node1:/etc/profile.d$ sudo vim mkl.sh
#写入export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/intel/mkl/lib/intel64:/usr/local/intel/lib/intel64
然后重新加载这个文件
node1@node1:/etc/profile.d$ source mkl.sh
#你也可以不写在这里。能加入环境变量即可

重新运行文件：

node1@node1:~/hpl-2.2/bin/ubuntu_mkl_intel$ ldd xhpl
#可以看到动态链接库都找得到了
node1@node1:~/hpl-2.2/bin/ubuntu_mkl_intel$ mpirun -n 4 ./xhpl

#就可以看到一大堆的运行结果了，部分如下
--------------------------------------------------------------------------------
||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=        0.0167027 ...... PASSED
================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR00R2R4          35     4     4     1               0.00              2.786e-01
HPL_pdgesv() start time Wed Mar  7 22:46:45 2018

HPL_pdgesv() end time   Wed Mar  7 22:46:45 2018

--------------------------------------------------------------------------------
||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=        0.0234356 ...... PASSED
================================================================================

Finished    864 tests with the following results:
            864 tests completed and passed residual checks,
              0 tests completed and failed residual checks,
              0 tests skipped because of illegal input values.
--------------------------------------------------------------------------------

End of Tests.
===============================================================================

6、总结

这几天装各种不同数学库和mpi版本出现各种问题，希望借此能帮助一些同样装hpl遇到困难的人。

7、附带一些关于mpich+atlas、mpich+openblas、mpich+gotoblas2数学库的配置文件

MPICH + atlas

#MPICH+Atlas
SHELL        = /bin/sh
CD           = cd
CP           = cp
LN_S         = ln -s
MKDIR        = mkdir
RM           = /bin/rm -f
TOUCH        = touch
ARCH         = ubuntu_atlas_mpich
TOPdir       = $(HOME)/hpl-2.2
INCdir       = $(TOPdir)/include
BINdir       = $(TOPdir)/bin/$(ARCH)
LIBdir       = $(TOPdir)/lib/$(ARCH)
HPLlib       = $(LIBdir)/libhpl.a 
MPdir        = /usr/local/mpich
MPinc        = -I$(MPdir)/include
MPlib        = $(MPdir)/lib/libmpi.a
LAdir        = /usr/local/atlas/lib
LAinc        =
LAlib        = $(LAdir)/libcblas.a $(LAdir)/liBATlas.a
F2CDEFS      =
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) -lpthread
HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib) -lpthread
HPL_OPTS     = -DHPL_CALL_CBLAS
HPL_DEFS     = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
CC           = /usr/local/mpich/bin/mpicc -lpthread
CCNOOPT      = $(HPL_DEFS)
CCFLAGS      = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops
LINKER       = /usr/local/mpich/bin/mpicc
LINKFLAGS    = $(CCFLAGS)
ARCHIVER     = ar
ARFLAGS      = r
RANLIB       = echo

MPICH + openblas

SHELL        = /bin/sh
CD           = cd
CP           = cp
LN_S         = ln -s
MKDIR        = mkdir
RM           = /bin/rm -f
TOUCH        = touch
ARCH         = ubuntu_open_mpich
TOPdir       = $(HOME)/hpl-2.2
INCdir       = $(TOPdir)/include
BINdir       = $(TOPdir)/bin/$(ARCH)
LIBdir       = $(TOPdir)/lib/$(ARCH)
HPLlib       = $(LIBdir)/libhpl.a 
MPdir        = /usr/local/mpich
MPinc        = -I$(MPdir)/include
MPlib        = $(MPdir)/lib/libmpi.a
LAdir        = /usr/local/openblas/lib
LAinc        =
LAlib        = $(LAdir)/libopenblas.a
F2CDEFS      =
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) -lpthread
HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib) -lpthread
HPL_OPTS     = -DHPL_CALL_CBLAS
HPL_DEFS     = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
CC           = /usr/local/mpich/bin/mpicc -lpthread
CCNOOPT      = $(HPL_DEFS)
CCFLAGS      = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops
LINKER       = /usr/local/mpich/bin/mpicc
LINKFLAGS    = $(CCFLAGS)
ARCHIVER     = ar
ARFLAGS      = r
RANLIB       = echo

MPI + GotoBlas2

SHELL        = /bin/sh
CD           = cd
CP           = cp
LN_S         = ln -s
MKDIR        = mkdir
RM           = /bin/rm -f
TOUCH        = touch
ARCH         = ubuntu_goto_mpich
TOPdir       = $(HOME)/hpl-2.2
INCdir       = $(TOPdir)/include
BINdir       = $(TOPdir)/bin/$(ARCH)
LIBdir       = $(TOPdir)/lib/$(ARCH)
HPLlib       = $(LIBdir)/libhpl.a 
MPdir        = /usr/local/mpich
MPinc        = -I$(MPdir)/include
MPlib        = $(MPdir)/lib/libmpi.a
LAdir        = $(HOME)/GotoBLAS2
LAinc        =
LAlib        = $(LAdir)/libgoto2.a
F2CDEFS      =
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) -lpthread
HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib) -lpthread
HPL_OPTS     = -DHPL_CALL_CBLAS
HPL_DEFS     = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
CC           = /usr/local/mpich/bin/mpicc -lpthread
CCNOOPT      = $(HPL_DEFS)
CCFLAGS      = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops
LINKER       = /usr/local/mpich/bin/mpicc
LINKFLAGS    = $(CCFLAGS)
ARCHIVER     = ar
ARFLAGS      = r
RANLIB       = echo

相关阅读