libmkl/ intel compiler compatibility issue

Submitted by jphilbin01 on Mon, 08/22/2016 - 12:36

Forums 

Installation

Hello,

I am having trouble installing Berkeley GW. I am getting an error from what seems to be a compatibility issue between the libmkl libraries and the intel compilers I am trying to use - see 'File in wrong format' message in the output below. Here is the output and my arch.mk file:

output:
cd Epsilon && make all
make[1]: Entering directory `/home/jphilbin/programs/BerkeleyGW-1.0.6/Epsilon'
mpiifort-intel -fc=ifort -free -O3 -o epsilon.cplx.x ../Common/bessel.o ../Common/subgrp.o ../Common/check_inversion.o ../Common/svninfo.o ../Common/sort.o ../Common/scalapack.o ../Common/misc.o ../Common/gmap.o ../Common/inversion.o ../Common/write_program_header.o ../Common/write_matrix.o ../Common/read_matrix.o ../Common/fftw.o ../Common/fullbz.o ../Common/irrbz.o ../Common/fft_parallel.o ../Common/minibzaverage.o ../Common/vcoul_generator.o ../Common/trunc_cell_wire.o ../Common/trunc_cell_box.o ../Common/trunc_cell_box_d.o ../Common/trunc_scell_box_d.o ../Common/checkbz.o ../Common/eqpcor.o ../Common/createpools.o ../Common/random.o ../Common/input_utils.o ../Common/wfn_rho_vxc_io.o ../Common/blas.o ../Common/lapack.o ../Common/splines.o ../Common/global.o ../Common/typedefs.o ../Common/nrtype.o ../Common/push_pop.o ../Common/message.o ../Common/peinfo.o ../Common/timing.o ../Common/system.o rqstar.o input.o input_q.o inread.o genwf_disk.o genwf_mpi.o mtxel.o epsinv.o mtxelmultiply.o extrapolar.o epsilon_main.o /share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic/libmkl_scalapack_lp64.a -L/home/jphilbin/programs/FFTW-2.1.5/fftw/lib -lfftw -Wl,--start-group /share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic/libmkl_intel_lp64.a /share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic/libmkl_sequential.a /share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic/libmkl_core.a /share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic/libmkl_blacs_intelmpi_lp64.a -Wl,--end-group -lpthread
ld: /share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic/libmkl_scalapack_lp64.a(pdgesv.o): Relocations in generic ELF (EM: 181)
ld: /share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic/libmkl_scalapack_lp64.a(pdgesv.o): Relocations in generic ELF (EM: 181)
/share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic/libmkl_scalapack_lp64.a: could not read symbols: File in wrong format
make[1]: *** [epsilon.cplx.x] Error 1
make[1]: Leaving directory `/home/jphilbin/programs/BerkeleyGW-1.0.6/Epsilon'
make: *** [epsilon-all] Error 2

And here is my arch.mk file:

# arch.mk for BerkeleyGW codes
#
# Suitable for Ubuntu 10.10 in parallel.
# Install packages: liblapack-dev, fftw-dev, gfortran, g++, mpi-default-dev
# Build BLACS according to http://www.open-mpi.org/faq/?category=mpi-apps#blacs
# Build ScaLAPACK according to http://www.open-mpi.org/faq/?category=mpi-apps#scalapack,
#
# D. Strubbe
# January 2011, UCB

# Precompiler option
#
COMPFLAG = -DINTEL
PARAFLAG = -DMPI
MATHFLAG = -DUSESCALAPACK
DEBUGFLAG = -DDEBUG -DVERBOSE

FCPP = cpp -ansi
F90free = mpiifort-intel -fc=ifort -free
#F90free = /usr/bin/mpif90 -ffree-form -ffree-line-length-none -fbounds-check -fbacktrace -ffpe-trap=invalid,zero,overflow
# denormal not supported, according to runtime error
# precision is flagged by cputime: http://gcc.gnu.org/onlinedocs/gcc-4.5.3/gfortran/Debugging-Options.html
# underflow is flagged by dlamch from lapack
LINK = mpiifort-intel -fc=ifort -free
FOPTS = -O3
#FOPTS = -g -C
FNOOPTS = $(FOPTS)
MOD_OPT = -module
INCFLAG = -I

C_PARAFLAG = -DPARA -DMPICH_IGNORE_CXX_SEEK
CC_COMP = mpiCC-intel
C_COMP = mpicc-intel
C_LINK = mpiCC-intel
C_OPTS = -O3

REMOVE = /bin/rm -f

# Math Libraries
#
FFTWLIB = -L/home/jphilbin/programs/FFTW-2.1.5/fftw/lib -lfftw
FFTWINCLUDE = /home/jphilbin/programs/FFTW-2.1.5/fftw/include
#MKLPATH = /opt/intel/composer_xe_2015.1.133/mkl/lib/ia32
MKLPATH = /share/apps/intel/composer_xe_2015.1.133/mkl/lib/mic
LAPACKLIB = -Wl,--start-group $(MKLPATH)/libmkl_intel_lp64.a $(MKLPATH)/libmkl_sequential.a \
$(MKLPATH)/libmkl_core.a $(MKLPATH)/libmkl_blacs_intelmpi_lp64.a -Wl,--end-group -lpthread
SCALAPACKLIB = $(MKLPATH)/libmkl_scalapack_lp64.a

#LAPACKLIB = -L/usr/lib/ -llapack
#BLACSDIR = /usr/lib
#BLACS = $(BLACSDIR)/blacs_MPI-LINUX-1.a $(BLACSDIR)/blacsF77init_MPI-LINUX-1.a $(BLACSDIR)/blacs_MPI-LINUX-1.a
#SCALAPACKLIB = /usr/lib/libscalapack-1.a $(BLACS)

#Ubuntu BLACS/SCALAPACK packages don't work properly sometimes
#BLACS = -lblacs-openmpi -lblacsF77init-openmpi -lblacs-openmpi
#SCALAPACK = -lscalapack-openmpi $(BLACS)

#MKL ScaLAPACK fails completely in complex version
#MKLPATH = /auto/opt/intel/mkl/lib/intel64
#LAPACKLIB = -Wl,--start-group $(MKLPATH)/libmkl_gf_lp64.a $(MKLPATH)/libmkl_sequential.a \
$(MKLPATH)/libmkl_core.a $(MKLPATH)/libmkl_blacs_openmpi_lp64.a -Wl,--end-group -lpthread
#SCALAPACKLIB = $(MKLPATH)/libmkl_scalapack_lp64.a

#need to export MPIEXEC=/usr/bin/mpirun if this is not default in `which mpiexec`
TESTSCRIPT = make check-parallel

Any suggestions would be greatly appreciated.

Thank you!

jornada's picture

Submitted by jornada on Wed, 08/31/2016 - 18:27

Hi John,

you are trying to cross-compile BerkeleyGW using mkl libraries compiled for a MIC (eg: Intel Xeon Phi), but the compiler is probably set to compile to stander amd64 architecture. Either (1) link against regular mkl libraries libraries for the host processor, (2) configure your compiler to cross-compile to a mic, or (3) ssh to the MIC and compile BerkeleyGW there.

If you don't know what a MIC is, follow solution (1), i.e., fix the path to mkl.