Aug 25, 2017

Run example matmul on FPGA
Don't change the makefile. Default is on device

1. Run script O2build.script
   This creates the host code and the kernel

To run on device:
Makefile flags

OMP ?= 0
MODE ?= normal
AOCL_BOARD ?= p385_hpc_d5
#AOCL_BOARD ?= p385_hpc_a7
#AOCL_FLAGS ?= -march=emulator
#AOCL_FLAGS ?= -march=emulator -c
AOCL_FLAGS ?= -v -c --report
#AOCL_FLAGS ?= --report


2. Run make
This creates the aoc file




After this bin and cetus look like this

[umunipala@newark cetus_output]$ ls
matmul.cpp  openarc_kernel  openarc_kernel.aoco  openarc_kernel.cl
[umunipala@newark cetus_output]$ cd ../bin
[umunipala@newark bin]$ ls
Timer  binBuilder_opencl  matmul_ACC  openarc_kernel.cl

now goto cetus output > (where aoc file is)


aoc openarc_kernel.cl -o openarc_kernel_p385_hpc_d5PCIe385n.aocx --board p385_hpc_a7 - here a7 is the second board; need to correct to a5

This is similar to 
(Compile kernel for emulation) 
aoc -v --board device/hello_world.cl -o bin/hello_world.aocx ( ) aoc -march=emulator -v --board device/hello_world.cl -o bin/hello_world_emulation.aocx 

aoc -v openarc_kernel.cl -o ../bin/openarc_kernel_p385_hpc_d5PCIe385n.aocx --board p385_hpc_d5

(on oswald
aoc -v openarc_kernel.cl -o ../bin/openarc_kernel_p510t_sch_ax115PCIe510n.aocx --board p510t_sch_ax115 
)



about 2+ hours later...




goto cetus_output>>
Then run ./SIMGMS_ACC this takes device in default.



#################################################################################Running on FPGA device on oswald??
1. create the aocx from aoco first

aoc -v openarc_kernel.cl -o ../bin/openarc_kernel_p510t_sch_ax115PCIe510n.aocx --board p510t_sch_ax115

Then run ./SIMGMS_ACC this takes device in default.

Aug 24, 2017

SIMGMS on FPGA



---------------------7/27 running example of openarc

 export openarc=/home/umunipala/openARC

export OPENARC_ARCH=3

/home/umunipala/openARC/bin/openarc matmul.c

2days july last week for converting to openacc and learning about openarc

8-07-2017---------------------------------------
reading about openacc : still translating to openACC

run gprof: not necessory


----8-9-2017 still trying to figure out how to correctly annotate openarc and give options correct.

/home/umunipala/openARC/bin/openarc -AccPrivatization=1 AccReduction=1 macro=_N_=__inputSize__,VERIFICATION=1 addIncludePath=__openarcrt__ assumeNoAliasingAmongKernelArgs /home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simgms_scf__c.c >& openARCout

--------------------------------8-14-2017
We are not touching simgms directory.
All changes with openacc and openarc are in this directory below.
/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC>>

changed paths in subdir.mk and changed for main.c
changed main.cpp into main.c
complains about extern: so commented it
generated libsimgms

copy lulesh makefile and build scritpt into simgmsCoAC and change paths to make.template and make.header
---------------------------------------------------------------------
now compile c programs and get main executablell

 gcc -Wl,--start-group -Wall -g -O -O1 -o main -lgfortran -fopenmp -lgomp ../main.c -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm

got errros::

g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.c -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm

g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.c -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm

-------------------------------------------------------------------

error compiling.
[umunipala@newark Debug]$ g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.c -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm
/tmp/ccYSzEib.o: In function `main':
/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug/../main.c:35: undefined reference to `simgms_scf_c(double const*, double const*, double const*, double const*, double const*, double const*, double const*, double const*, long const*, long const*, long const*, long const*, long const*, long, long, long, long, long, double const*, double const*, double)'


For openarc we need to add the correct files and modify makefiles.

g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.o -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm

------------------------8-15-2017
openarc supports ansi c older than 99
need to remove // comments 11 to 12.21


------------08-23 Try to recreate fpga file structure in simgms dir

/home/umunipala/gamess/gamess-kris-simgms/simgms/Debug
g++ -Wl,--start-group -Wall -g -O -O1 -std=c++11 -o main -lgfortran -fopenmp -lgomp ../main.cpp -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/Debug -lsimgms -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/lib -lsimint -L/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/oed -loed -loedlib -L/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64 -lmkl_mc -lmkl_sequential -lmkl_gf_lp64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lmkl_blas95_lp64 -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/simint/simint/include -I/home/umunipala/gamess/gamess-kris-simgms/simgmsCoAC/include -I/opt/intel/composer_xe_2013.5.192/mkl/include -Wl,--end-group -ldl -lpthread -lm
-------------Aug 24----------------------
rebuilt simint and simgms
can not generate make executable simgms_c throws an error.
reason: we changed simgms_scf for openacc. when openacc converted the simgms c file to cpp file it needed different path + to define extern c in a few files.
1. simgms_scf.h
2.simgms_init.h and some more (around 2 3 more - Forgot! Work from saturday, Aug 22)

main.o has to be in Debug directory.

Realized that we do not need the main executable anymore. If we need that it is already there in simgms path.
Undoing and trying to run the exe with openarc as before.
For some reason I again got the illegal instruction.
When I looked at simint library it was an old one. So recompiled simint.
Then copy the library to simint/simint/lib path : this is what is in OpenARC makefile settings for Simgms project in simgmsCoAC directory.


This should fix the error.
Then proceed to adding more openACC
then add timers

Followers