[753] | 1 | File: CoreMark |
---|
| 2 | |
---|
| 3 | Topic: Welcome |
---|
| 4 | Copyright © 2009 EEMBC All rights reserved. |
---|
| 5 | CoreMark is a trademark of EEMBC and EEMBC is a registered trademark of the Embedded Microprocessor Benchmark Consortium. |
---|
| 6 | |
---|
| 7 | CoreMarks primary goals are simplicity and providing a method for testing only a processors core features. |
---|
| 8 | |
---|
| 9 | For more information about EEMBC's comprehensive embedded benchmark suites, please see www.eembc.org. |
---|
| 10 | |
---|
| 11 | Topic: Building and running |
---|
| 12 | Download the release files from the www.coremark.org. |
---|
| 13 | You can verify the download using the coremark_<version>.md5 file |
---|
| 14 | > md5sum -c coremark_<version>.md5 |
---|
| 15 | |
---|
| 16 | Unpack the distribution (tar -vzxf coremark_<version>.tgz && tar -vzxf coremark_<version>_docs.tgz) |
---|
| 17 | then change to the coremark_<version> folder. |
---|
| 18 | |
---|
| 19 | To build and run the benchmark, type |
---|
| 20 | > make |
---|
| 21 | Full results are available in the files run1.log and run2.log. |
---|
| 22 | CoreMark result can be found in run1.log. |
---|
| 23 | |
---|
| 24 | For self hosted Linux or Cygwin platforms, a simple make should work. |
---|
| 25 | |
---|
| 26 | Cross Compile: |
---|
| 27 | For cross compile platforms please adjust <core_portme.mak>, <core_portme.h> (and possibly <core_portme.c>) |
---|
| 28 | according to the specific platform used. |
---|
| 29 | When porting to a new platform, it is recommended to copy one of the default port folders |
---|
| 30 | (e.g. mkdir <platform> && cp linux/* <platform>), adjust the porting files, and run |
---|
| 31 | > make PORT_DIR=<platform> |
---|
| 32 | |
---|
| 33 | Systems without make: |
---|
| 34 | The following files need to be compiled: |
---|
| 35 | - <core_list_join.c> |
---|
| 36 | - <core_main.c> |
---|
| 37 | - <core_matrix.c> |
---|
| 38 | - <core_state.c> |
---|
| 39 | - <core_util.c> |
---|
| 40 | - <PORT_DIR>/<core_portme.c> |
---|
| 41 | |
---|
| 42 | For example |
---|
| 43 | > gcc -O2 -o coremark.exe core_list_join.c core_main.c core_matrix.c core_state.c core_util.c simple/core_portme.c -DPERFORMANCE_RUN=1 -DITERATIONS=1000 |
---|
| 44 | > ./coremark.exe > run1.log |
---|
| 45 | The above will compile the benchmark for a performance run and 1000 iterations. Output is redirected to run1.log. |
---|
| 46 | |
---|
| 47 | Make targets: |
---|
| 48 | run - Default target, creates run1.log and run2.log. |
---|
| 49 | run1.log - Run the benchmark with performance parameters, and output to run1.log |
---|
| 50 | run2.log - Run the benchmark with validation parameters, and output to run2.log |
---|
| 51 | run3.log - Run the benchmark with profile generation parameters, and output to run3.log |
---|
| 52 | compile - compile the benchmark executable |
---|
| 53 | link - link the benchmark executable |
---|
| 54 | check - test MD5 of sources that may not be modified |
---|
| 55 | clean - clean temporary files |
---|
| 56 | |
---|
| 57 | ITERATIONS: |
---|
| 58 | By default, the benchmark will run between 10-100 seconds. |
---|
| 59 | To override, use ITERATIONS=N |
---|
| 60 | > make ITERATIONS=10 |
---|
| 61 | Will run the benchmark for 10 iterations. |
---|
| 62 | It is recommended to set a specific number of iterations in certain situations e.g.: |
---|
| 63 | - Running with a simulator |
---|
| 64 | - Measuring power/energy |
---|
| 65 | - Timing cannot be restarted |
---|
| 66 | |
---|
| 67 | Minimum required run time: |
---|
| 68 | Results are only valid for reporting if the benchmark ran for at least 10 secs! |
---|
| 69 | |
---|
| 70 | XCFLAGS: |
---|
| 71 | To add compiler flags from the command line, use XCFLAGS e.g. |
---|
| 72 | > make XCFLAGS="-g -DMULTITHREAD=4 -DUSE_FORK=1" |
---|
| 73 | |
---|
| 74 | o CORE_DEBUG |
---|
| 75 | |
---|
| 76 | Define to compile for a debug run if you get incorrect CRC. |
---|
| 77 | > make XCFLAGS="-DCORE_DEBUG=1" |
---|
| 78 | |
---|
| 79 | o Parallel Execution |
---|
| 80 | |
---|
| 81 | Use XCFLAGS=-DMULTITHREAD=N where N is number of threads to run in parallel. |
---|
| 82 | Several implementations are available to execute in multiple contexts, |
---|
| 83 | or you can implement your own in <core_portme.c>. |
---|
| 84 | > make XCFLAGS="-DMULTITHREAD=4 -DUSE_PTHREAD" |
---|
| 85 | Above will compile the benchmark for execution on 4 cores, using POSIX Threads API. |
---|
| 86 | |
---|
| 87 | REBUILD: |
---|
| 88 | To force rebuild, add the flag REBUILD to the command line |
---|
| 89 | > make REBUILD=1 |
---|
| 90 | |
---|
| 91 | Check core_portme.mak for more important options. |
---|
| 92 | |
---|
| 93 | Run parameters for the benchmark executable: |
---|
| 94 | Coremark executable takes several parameters as follows (if main accepts arguments). |
---|
| 95 | 1st - A seed value used for initialization of data. |
---|
| 96 | 2nd - A seed value used for initialization of data. |
---|
| 97 | 3rd - A seed value used for initialization of data. |
---|
| 98 | 4th - Number of iterations (0 for auto : default value) |
---|
| 99 | 5th - Reserved for internal use. |
---|
| 100 | 6th - Reserved for internal use. |
---|
| 101 | 7th - For malloc users only, ovreride the size of the input data buffer. |
---|
| 102 | |
---|
| 103 | The run target from make will run coremark with 2 different data initialization seeds. |
---|
| 104 | |
---|
| 105 | Alternative parameters: |
---|
| 106 | If not using malloc or command line arguments are not supported, the buffer size |
---|
| 107 | for the algorithms must be defined via the compiler define TOTAL_DATA_SIZE. |
---|
| 108 | TOTAL_DATA_SIZE must be set to 2000 bytes (default) for standard runs. |
---|
| 109 | The default for such a target when testing different configurations could be ... |
---|
| 110 | > make XCFLAGS="-DTOTAL_DATA_SIZE=6000 -DMAIN_HAS_NOARGC=1" |
---|
| 111 | |
---|
| 112 | Topic: Documentation |
---|
| 113 | When you unpack the documentation (tar -vzxf coremark_<version>_docs.tgz) a docs folder will be created. |
---|
| 114 | Check the file docs/html/index.html and the website http://www.coremark.org for more info. |
---|
| 115 | |
---|
| 116 | Topic: Submitting results |
---|
| 117 | CoreMark results can be submitted on the web. |
---|
| 118 | |
---|
| 119 | Open a web browser and go to http://www.coremark.org/benchmark/index.php?pg=benchmark |
---|
| 120 | Select the link to add a new score and follow the instructions. |
---|
| 121 | |
---|
| 122 | Topic: Run rules |
---|
| 123 | What is and is not allowed. |
---|
| 124 | |
---|
| 125 | Required: |
---|
| 126 | 1 - The benchmark needs to run for at least 10 seconds. |
---|
| 127 | 2 - All validation must succeed for seeds 0,0,0x66 and 0x3415,0x3415,0x66, |
---|
| 128 | buffer size of 2000 bytes total. |
---|
| 129 | o If not using command line arguments to main: |
---|
| 130 | > make XCFLAGS="-DPERFORMANCE_RUN=1" REBUILD=1 run1.log |
---|
| 131 | > make XCFLAGS="-DVALIDATION_RUN=1" REBUILD=1 run2.log |
---|
| 132 | 3 - If using profile guided optimization, profile must be generated using seeds of 8,8,8, |
---|
| 133 | and buffer size of 1200 bytes total. |
---|
| 134 | > make XCFLAGS="-DTOTAL_DATA_SIZE=1200 -DPROFILE_RUN=1" REBUILD=1 run3.log |
---|
| 135 | 4 - All source files must be compiled with the same flags. |
---|
| 136 | 5 - All data type sizes must match size in bits such that: |
---|
| 137 | o ee_u8 is an 8 bits datatype. |
---|
| 138 | o ee_s16 is an 16 bits datatype. |
---|
| 139 | o ee_u16 is an 16 bits datatype. |
---|
| 140 | o ee_s32 is an 32 bits datatype. |
---|
| 141 | o ee_u32 is an 32 bits datatype. |
---|
| 142 | |
---|
| 143 | Allowed: |
---|
| 144 | - Changing number of iterations |
---|
| 145 | - Changing toolchain and build/load/run options |
---|
| 146 | - Changing method of acquiring a data memory block |
---|
| 147 | - Changing the method of acquiring seed values |
---|
| 148 | - Changing implementation in core_portme.c |
---|
| 149 | - Changing configuration values in core_portme.h |
---|
| 150 | - Changing core_portme.mak |
---|
| 151 | |
---|
| 152 | Not allowed: |
---|
| 153 | - Changing of source file other then core_portme* (use make check to validate) |
---|
| 154 | |
---|
| 155 | Topic: Reporting rules |
---|
| 156 | How to report results on a data sheet? |
---|
| 157 | |
---|
| 158 | CoreMark 1.0 : N / C [/ P] [/ M] |
---|
| 159 | |
---|
| 160 | N - Number of iterations per second with seeds 0,0,0x66,size=2000) |
---|
| 161 | C - Compiler version and flags |
---|
| 162 | P - Parameters such as data and code allocation specifics |
---|
| 163 | - This parameter *may* be omitted if all data was allocated on the heap in RAM. |
---|
| 164 | - This parameter *may not* be omitted when reporting CoreMark/MHz |
---|
| 165 | M - Type of parallel execution (if used) and number of contexts |
---|
| 166 | This parameter may be omitted if parallel execution was not used. |
---|
| 167 | |
---|
| 168 | e.g. |
---|
| 169 | > CoreMark 1.0 : 128 / GCC 4.1.2 -O2 -fprofile-use / Heap in TCRAM / FORK:2 |
---|
| 170 | or |
---|
| 171 | > CoreMark 1.0 : 1400 / GCC 3.4 -O4 |
---|
| 172 | |
---|
| 173 | If reporting scaling results, the results must be reported as follows: |
---|
| 174 | |
---|
| 175 | CoreMark/MHz 1.0 : N / C / P [/ M] |
---|
| 176 | |
---|
| 177 | P - When reporting scaling results, memory parameter must also indicate memory frequency:core frequency ratio. |
---|
| 178 | - If the core has cache and cache frequency to core frequency ratio is configurable, that must also be included. |
---|
| 179 | |
---|
| 180 | e.g. |
---|
| 181 | > CoreMark/MHz 1.0 : 1.47 / GCC 4.1.2 -O2 / DDR3(Heap) 30:1 Memory 1:1 Cache |
---|
| 182 | |
---|
| 183 | |
---|
| 184 | Topic: Log File Format |
---|
| 185 | The log files have the following format |
---|
| 186 | (start example) |
---|
| 187 | 2K performance run parameters for coremark. (Run type) |
---|
| 188 | CoreMark Size : 666 (Buffer size) |
---|
| 189 | Total ticks : 25875 (platform dependent value) |
---|
| 190 | Total time (secs) : 25.875000 (actual time in seconds) |
---|
| 191 | Iterations/Sec : 3864.734300 (Performance value to report) |
---|
| 192 | Iterations : 100000 (number of iterations used) |
---|
| 193 | Compiler version : GCC3.4.4 (Compiler and version) |
---|
| 194 | Compiler flags : -O2 (Compiler and linker flags) |
---|
| 195 | Memory location : Code in flash, data in on chip RAM |
---|
| 196 | seedcrc : 0xe9f5 (identifier for the input seeds) |
---|
| 197 | [0]crclist : 0xe714 (validation for list part) |
---|
| 198 | [0]crcmatrix : 0x1fd7 (validation for matrix part) |
---|
| 199 | [0]crcstate : 0x8e3a (validation for state part) |
---|
| 200 | [0]crcfinal : 0x33ff (iteration dependent output) |
---|
| 201 | Correct operation validated. See readme.txt for run and reporting rules. (*Only when run is successful*) |
---|
| 202 | CoreMark 1.0 : 6508.490622 / GCC3.4.4 -O2 / Heap (*Only on a successful performance run*) |
---|
| 203 | (end example) |
---|
| 204 | |
---|
| 205 | Topic: Legal |
---|
| 206 | See LICENSE.txt or the word document file under docs/LICENSE.doc. |
---|
| 207 | For more information on your legal rights to use this benchmark, please see |
---|
| 208 | http://www.coremark.org/download/register.php?pg=register |
---|
| 209 | |
---|
| 210 | Topic: Credits |
---|
| 211 | Many thanks to all of the individuals who helped with the development or testing of CoreMark including (Sorted by company name) |
---|
| 212 | o Alan Anderson, ADI |
---|
| 213 | o Adhikary Rajiv, ADI |
---|
| 214 | o Elena Stohr, ARM |
---|
| 215 | o Ian Rickards, ARM |
---|
| 216 | o Andrew Pickard, ARM |
---|
| 217 | o Trent Parker, CAVIUM |
---|
| 218 | o Shay Gal-On, EEMBC |
---|
| 219 | o Markus Levy, EEMBC |
---|
| 220 | o Ron Olson, IBM |
---|
| 221 | o Eyal Barzilay, MIPS |
---|
| 222 | o Jens Eltze, NEC |
---|
| 223 | o Hirohiko Ono, NEC |
---|
| 224 | o Ulrich Drees, NEC |
---|
| 225 | o Frank Roscheda, NEC |
---|
| 226 | o Rob Cosaro, NXP |
---|
| 227 | o Shumpei Kawasaki, RENESAS |
---|