File: CoreMark Topic: Welcome Copyright © 2009 EEMBC All rights reserved. CoreMark is a trademark of EEMBC and EEMBC is a registered trademark of the Embedded Microprocessor Benchmark Consortium. CoreMark’s primary goals are simplicity and providing a method for testing only a processor’s core features. For more information about EEMBC's comprehensive embedded benchmark suites, please see www.eembc.org. Topic: Building and running Download the release files from the www.coremark.org. You can verify the download using the coremark_.md5 file > md5sum -c coremark_.md5 Unpack the distribution (tar -vzxf coremark_.tgz && tar -vzxf coremark__docs.tgz) then change to the coremark_ folder. To build and run the benchmark, type > make Full results are available in the files run1.log and run2.log. CoreMark result can be found in run1.log. For self hosted Linux or Cygwin platforms, a simple make should work. Cross Compile: For cross compile platforms please adjust , (and possibly ) according to the specific platform used. When porting to a new platform, it is recommended to copy one of the default port folders (e.g. mkdir && cp linux/* ), adjust the porting files, and run > make PORT_DIR= Systems without make: The following files need to be compiled: - - - - - - / For example > gcc -O2 -o coremark.exe core_list_join.c core_main.c core_matrix.c core_state.c core_util.c simple/core_portme.c -DPERFORMANCE_RUN=1 -DITERATIONS=1000 > ./coremark.exe > run1.log The above will compile the benchmark for a performance run and 1000 iterations. Output is redirected to run1.log. Make targets: run - Default target, creates run1.log and run2.log. run1.log - Run the benchmark with performance parameters, and output to run1.log run2.log - Run the benchmark with validation parameters, and output to run2.log run3.log - Run the benchmark with profile generation parameters, and output to run3.log compile - compile the benchmark executable link - link the benchmark executable check - test MD5 of sources that may not be modified clean - clean temporary files ITERATIONS: By default, the benchmark will run between 10-100 seconds. To override, use ITERATIONS=N > make ITERATIONS=10 Will run the benchmark for 10 iterations. It is recommended to set a specific number of iterations in certain situations e.g.: - Running with a simulator - Measuring power/energy - Timing cannot be restarted Minimum required run time: Results are only valid for reporting if the benchmark ran for at least 10 secs! XCFLAGS: To add compiler flags from the command line, use XCFLAGS e.g. > make XCFLAGS="-g -DMULTITHREAD=4 -DUSE_FORK=1" o CORE_DEBUG Define to compile for a debug run if you get incorrect CRC. > make XCFLAGS="-DCORE_DEBUG=1" o Parallel Execution Use XCFLAGS=-DMULTITHREAD=N where N is number of threads to run in parallel. Several implementations are available to execute in multiple contexts, or you can implement your own in . > make XCFLAGS="-DMULTITHREAD=4 -DUSE_PTHREAD" Above will compile the benchmark for execution on 4 cores, using POSIX Threads API. REBUILD: To force rebuild, add the flag REBUILD to the command line > make REBUILD=1 Check core_portme.mak for more important options. Run parameters for the benchmark executable: Coremark executable takes several parameters as follows (if main accepts arguments). 1st - A seed value used for initialization of data. 2nd - A seed value used for initialization of data. 3rd - A seed value used for initialization of data. 4th - Number of iterations (0 for auto : default value) 5th - Reserved for internal use. 6th - Reserved for internal use. 7th - For malloc users only, ovreride the size of the input data buffer. The run target from make will run coremark with 2 different data initialization seeds. Alternative parameters: If not using malloc or command line arguments are not supported, the buffer size for the algorithms must be defined via the compiler define TOTAL_DATA_SIZE. TOTAL_DATA_SIZE must be set to 2000 bytes (default) for standard runs. The default for such a target when testing different configurations could be ... > make XCFLAGS="-DTOTAL_DATA_SIZE=6000 -DMAIN_HAS_NOARGC=1" Topic: Documentation When you unpack the documentation (tar -vzxf coremark__docs.tgz) a docs folder will be created. Check the file docs/html/index.html and the website http://www.coremark.org for more info. Topic: Submitting results CoreMark results can be submitted on the web. Open a web browser and go to http://www.coremark.org/benchmark/index.php?pg=benchmark Select the link to add a new score and follow the instructions. Topic: Run rules What is and is not allowed. Required: 1 - The benchmark needs to run for at least 10 seconds. 2 - All validation must succeed for seeds 0,0,0x66 and 0x3415,0x3415,0x66, buffer size of 2000 bytes total. o If not using command line arguments to main: > make XCFLAGS="-DPERFORMANCE_RUN=1" REBUILD=1 run1.log > make XCFLAGS="-DVALIDATION_RUN=1" REBUILD=1 run2.log 3 - If using profile guided optimization, profile must be generated using seeds of 8,8,8, and buffer size of 1200 bytes total. > make XCFLAGS="-DTOTAL_DATA_SIZE=1200 -DPROFILE_RUN=1" REBUILD=1 run3.log 4 - All source files must be compiled with the same flags. 5 - All data type sizes must match size in bits such that: o ee_u8 is an 8 bits datatype. o ee_s16 is an 16 bits datatype. o ee_u16 is an 16 bits datatype. o ee_s32 is an 32 bits datatype. o ee_u32 is an 32 bits datatype. Allowed: - Changing number of iterations - Changing toolchain and build/load/run options - Changing method of acquiring a data memory block - Changing the method of acquiring seed values - Changing implementation in core_portme.c - Changing configuration values in core_portme.h - Changing core_portme.mak Not allowed: - Changing of source file other then core_portme* (use make check to validate) Topic: Reporting rules How to report results on a data sheet? CoreMark 1.0 : N / C [/ P] [/ M] N - Number of iterations per second with seeds 0,0,0x66,size=2000) C - Compiler version and flags P - Parameters such as data and code allocation specifics - This parameter *may* be omitted if all data was allocated on the heap in RAM. - This parameter *may not* be omitted when reporting CoreMark/MHz M - Type of parallel execution (if used) and number of contexts This parameter may be omitted if parallel execution was not used. e.g. > CoreMark 1.0 : 128 / GCC 4.1.2 -O2 -fprofile-use / Heap in TCRAM / FORK:2 or > CoreMark 1.0 : 1400 / GCC 3.4 -O4 If reporting scaling results, the results must be reported as follows: CoreMark/MHz 1.0 : N / C / P [/ M] P - When reporting scaling results, memory parameter must also indicate memory frequency:core frequency ratio. - If the core has cache and cache frequency to core frequency ratio is configurable, that must also be included. e.g. > CoreMark/MHz 1.0 : 1.47 / GCC 4.1.2 -O2 / DDR3(Heap) 30:1 Memory 1:1 Cache Topic: Log File Format The log files have the following format (start example) 2K performance run parameters for coremark. (Run type) CoreMark Size : 666 (Buffer size) Total ticks : 25875 (platform dependent value) Total time (secs) : 25.875000 (actual time in seconds) Iterations/Sec : 3864.734300 (Performance value to report) Iterations : 100000 (number of iterations used) Compiler version : GCC3.4.4 (Compiler and version) Compiler flags : -O2 (Compiler and linker flags) Memory location : Code in flash, data in on chip RAM seedcrc : 0xe9f5 (identifier for the input seeds) [0]crclist : 0xe714 (validation for list part) [0]crcmatrix : 0x1fd7 (validation for matrix part) [0]crcstate : 0x8e3a (validation for state part) [0]crcfinal : 0x33ff (iteration dependent output) Correct operation validated. See readme.txt for run and reporting rules. (*Only when run is successful*) CoreMark 1.0 : 6508.490622 / GCC3.4.4 -O2 / Heap (*Only on a successful performance run*) (end example) Topic: Legal See LICENSE.txt or the word document file under docs/LICENSE.doc. For more information on your legal rights to use this benchmark, please see http://www.coremark.org/download/register.php?pg=register Topic: Credits Many thanks to all of the individuals who helped with the development or testing of CoreMark including (Sorted by company name) o Alan Anderson, ADI o Adhikary Rajiv, ADI o Elena Stohr, ARM o Ian Rickards, ARM o Andrew Pickard, ARM o Trent Parker, CAVIUM o Shay Gal-On, EEMBC o Markus Levy, EEMBC o Ron Olson, IBM o Eyal Barzilay, MIPS o Jens Eltze, NEC o Hirohiko Ono, NEC o Ulrich Drees, NEC o Frank Roscheda, NEC o Rob Cosaro, NXP o Shumpei Kawasaki, RENESAS