SIMGMS on FPGA

---------------------7/27 running example of openarc

 export openarc=/home/umunipala/openARC

export OPENARC_ARCH=3

/home/umunipala/openARC/bin/openarc matmul.c

2days july last week for converting to openacc and learning about openarc

8-07-2017---------------------------------------
reading about openacc : still translating to openACC

run gprof: not necessory


----8-9-2017 still trying to figure out how to correctly annotate openarc and give options correct.

/home/umunipala/openARC/bin/openarc -AccPrivatization=1 AccReduction=1 macro=_N_=__inputSize__,VERIFICATION=1 addIncludePath=__openarcrt__ assumeNoAliasingAmongKernelArgs /home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simgms_scf__c.c >& openARCout

--------------------------------8-14-2017
We are not touching simgms directory.
All changes with openacc and openarc are in this directory below.
/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC>>

changed paths in subdir.mk and changed for main.c
changed main.cpp into main.c
complains about extern: so commented it
generated libsimgms

copy lulesh makefile and build scritpt into simgmsCoAC and change paths to make.template and make.header
---------------------------------------------------------------------
now compile c programs and get main executablell

 gcc -Wl,--start-group -Wall -g -O -O1 -o main -lgfortran -fopenmp -lgomp ../main.c -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm

got errros::

g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.c -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm

g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.c -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm

-------------------------------------------------------------------

error compiling.
[umunipala@newark Debug]$ g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.c -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm
/tmp/ccYSzEib.o: In function `main':
/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug/../main.c:35: undefined reference to `simgms_scf_c(double const*, double const*, double const*, double const*, double const*, double const*, double const*, double const*, long const*, long const*, long const*, long const*, long const*, long, long, long, long, long, double const*, double const*, double)'


For openarc we need to add the correct files and modify makefiles.

g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.o -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm

------------------------8-15-2017
openarc supports ansi c older than 99
need to remove // comments 11 to 12.21


------------08-23 Try to recreate fpga file structure in simgms dir

/home/umunipala/gamess/gamess-kris-simgms/simgms/Debug
g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.cpp -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm
-------------Aug 24----------------------
rebuilt simint and simgms
can not generate make executable simgms_c throws an error.
reason: we changed simgms_scf for openacc. when openacc converted the simgms c file to cpp file it needed different path + to define extern c in a few files.
1. simgms_scf.h
2.simgms_init.h and some more (around 2 3 more - Forgot! Work from saturday, Aug 22)

main.o has to be in Debug directory.

Realized that we do not need the main executable anymore. If we need that it is already there in simgms path.
Undoing and trying to run the exe with openarc as before.
For some reason I again got the illegal instruction.
When I looked at simint library it was an old one. So recompiled simint.
Then copy the library to simint/simint/lib path : this is what is in OpenARC makefile settings for Simgms project in simgmsCoAC directory.


This should fix the error.
Then proceed to adding more openACC
then add timers