source: soft/giet_vm/applications/coremark/readme.txt @ 792

Last change on this file since 792 was 753, checked in by cfuguet, 9 years ago

Introducing the coremark benchmark

File size: 9.2 KB
Line 
1File: CoreMark
2
3Topic: Welcome
4Copyright © 2009 EEMBC All rights reserved.
5CoreMark is a trademark of EEMBC and EEMBC is a registered trademark of the Embedded Microprocessor Benchmark Consortium.
6
7CoreMark’s primary goals are simplicity and providing a method for testing only a processor’s core features.
8
9For more information about EEMBC's comprehensive embedded benchmark suites, please see www.eembc.org.
10
11Topic: Building and running
12        Download the release files from the www.coremark.org.
13        You can verify the download using the coremark_<version>.md5 file
14        > md5sum -c coremark_<version>.md5
15       
16        Unpack the distribution (tar -vzxf coremark_<version>.tgz && tar -vzxf coremark_<version>_docs.tgz)
17        then change to the coremark_<version> folder.
18       
19        To build and run the benchmark, type
20        > make
21        Full results are available in the files run1.log and run2.log.
22        CoreMark result can be found in run1.log.
23       
24        For self hosted Linux or Cygwin platforms, a simple make should work.
25       
26        Cross Compile:
27        For cross compile platforms please adjust <core_portme.mak>, <core_portme.h> (and possibly <core_portme.c>)
28        according to the specific platform used.
29        When porting to a new platform, it is recommended to copy one of the default port folders
30        (e.g. mkdir <platform> && cp linux/* <platform>), adjust the porting files, and run
31        > make PORT_DIR=<platform>
32       
33        Systems without make:
34        The following files need to be compiled:
35        - <core_list_join.c>
36        - <core_main.c>
37        - <core_matrix.c>
38        - <core_state.c>       
39        - <core_util.c>
40        - <PORT_DIR>/<core_portme.c>
41       
42        For example
43        > gcc -O2 -o coremark.exe core_list_join.c core_main.c core_matrix.c core_state.c core_util.c simple/core_portme.c -DPERFORMANCE_RUN=1 -DITERATIONS=1000
44        > ./coremark.exe > run1.log
45        The above will compile the benchmark for a performance run and 1000 iterations. Output is redirected to run1.log.
46       
47        Make targets:
48        run - Default target, creates run1.log and run2.log.
49        run1.log - Run the benchmark with performance parameters, and output to run1.log
50        run2.log - Run the benchmark with validation parameters, and output to run2.log
51        run3.log - Run the benchmark with profile generation parameters, and output to run3.log
52        compile - compile the benchmark executable
53        link - link the benchmark executable
54        check - test MD5 of sources that may not be modified
55        clean - clean temporary files
56       
57        ITERATIONS:
58        By default, the benchmark will run between 10-100 seconds.
59        To override, use ITERATIONS=N
60        > make ITERATIONS=10
61        Will run the benchmark for 10 iterations.
62        It is recommended to set a specific number of iterations in certain situations e.g.:
63        - Running with a simulator
64        - Measuring power/energy
65        - Timing cannot be restarted
66       
67        Minimum required run time:
68        Results are only valid for reporting if the benchmark ran for at least 10 secs!
69       
70        XCFLAGS:
71        To add compiler flags from the command line, use XCFLAGS e.g.
72        > make XCFLAGS="-g -DMULTITHREAD=4 -DUSE_FORK=1"
73       
74        o CORE_DEBUG
75       
76        Define to compile for a debug run if you get incorrect CRC.
77        > make XCFLAGS="-DCORE_DEBUG=1"
78       
79        o Parallel Execution
80       
81        Use XCFLAGS=-DMULTITHREAD=N where N is number of threads to run in parallel.
82        Several implementations are available to execute in multiple contexts,
83        or you can implement your own in <core_portme.c>.
84        > make XCFLAGS="-DMULTITHREAD=4 -DUSE_PTHREAD"
85        Above will compile the benchmark for execution on 4 cores, using POSIX Threads API.
86       
87        REBUILD:
88        To force rebuild, add the flag REBUILD to the command line
89        > make REBUILD=1
90       
91        Check core_portme.mak for more important options.
92
93        Run parameters for the benchmark executable:
94        Coremark executable takes several parameters as follows (if main accepts arguments).
95        1st - A seed value used for initialization of data.
96        2nd - A seed value used for initialization of data.
97        3rd - A seed value used for initialization of data.
98        4th - Number of iterations (0 for auto : default value)
99        5th - Reserved for internal use.
100        6th - Reserved for internal use.
101        7th - For malloc users only, ovreride the size of the input data buffer.
102       
103        The run target from make will run coremark with 2 different data initialization seeds.
104
105        Alternative parameters:
106        If not using malloc or command line arguments are not supported, the buffer size
107        for the algorithms must be defined via the compiler define TOTAL_DATA_SIZE.
108        TOTAL_DATA_SIZE must be set to 2000 bytes (default) for standard runs.
109        The default for such a target when testing different configurations could be ...
110        > make XCFLAGS="-DTOTAL_DATA_SIZE=6000 -DMAIN_HAS_NOARGC=1"
111       
112Topic: Documentation
113        When you unpack the documentation (tar -vzxf coremark_<version>_docs.tgz) a docs folder will be created.
114        Check the file docs/html/index.html and the website http://www.coremark.org for more info.
115       
116Topic: Submitting results
117        CoreMark results can be submitted on the web.
118       
119        Open a web browser and go to http://www.coremark.org/benchmark/index.php?pg=benchmark
120        Select the link to add a new score and follow the instructions.
121       
122Topic: Run rules
123        What is and is not allowed.
124       
125        Required:
126        1 - The benchmark needs to run for at least 10 seconds.
127        2 - All validation must succeed for seeds 0,0,0x66 and 0x3415,0x3415,0x66,
128                buffer size of 2000 bytes total.
129                o If not using command line arguments to main:
130                > make XCFLAGS="-DPERFORMANCE_RUN=1" REBUILD=1 run1.log
131                > make XCFLAGS="-DVALIDATION_RUN=1" REBUILD=1 run2.log
132        3 - If using profile guided optimization, profile must be generated using seeds of 8,8,8,
133                and buffer size of 1200 bytes total.
134                > make XCFLAGS="-DTOTAL_DATA_SIZE=1200 -DPROFILE_RUN=1" REBUILD=1 run3.log
135        4 - All source files must be compiled with the same flags.
136        5 - All data type sizes must match size in bits such that:
137                o ee_u8 is an 8 bits datatype.
138                o ee_s16 is an 16 bits datatype.
139                o ee_u16 is an 16 bits datatype.
140                o ee_s32 is an 32 bits datatype.
141                o ee_u32 is an 32 bits datatype.
142       
143        Allowed:
144        - Changing number of iterations
145        - Changing toolchain and build/load/run options
146        - Changing method of acquiring a data memory block
147        - Changing the method of acquiring seed values
148        - Changing implementation in core_portme.c
149        - Changing configuration values in core_portme.h
150        - Changing core_portme.mak
151       
152        Not allowed:
153        - Changing of source file other then core_portme* (use make check to validate)
154
155Topic: Reporting rules
156        How to report results on a data sheet?
157
158        CoreMark 1.0 : N / C [/ P] [/ M]
159       
160        N - Number of iterations per second with seeds 0,0,0x66,size=2000)
161        C - Compiler version and flags
162        P - Parameters such as data and code allocation specifics
163                - This parameter *may* be omitted if all data was allocated on the heap in RAM.
164                - This parameter *may not* be omitted when reporting CoreMark/MHz
165        M - Type of parallel execution (if used) and number of contexts
166                This parameter may be omitted if parallel execution was not used.
167
168        e.g.
169        > CoreMark 1.0 : 128 / GCC 4.1.2 -O2 -fprofile-use / Heap in TCRAM / FORK:2
170        or
171        > CoreMark 1.0 : 1400 / GCC 3.4 -O4
172       
173        If reporting scaling results, the results must be reported as follows:
174       
175        CoreMark/MHz 1.0 : N / C / P [/ M]
176       
177        P - When reporting scaling results, memory parameter must also indicate memory frequency:core frequency ratio.
178                - If the core has cache and cache frequency to core frequency ratio is configurable, that must also be included.
179       
180        e.g.
181        > CoreMark/MHz 1.0 : 1.47 / GCC 4.1.2 -O2 / DDR3(Heap) 30:1 Memory 1:1 Cache
182
183       
184Topic: Log File Format
185        The log files have the following format
186(start example)
1872K performance run parameters for coremark.     (Run type)
188CoreMark Size           : 666                                   (Buffer size)
189Total ticks                     : 25875                                 (platform dependent value)
190Total time (secs)       : 25.875000                             (actual time in seconds)
191Iterations/Sec          : 3864.734300                   (Performance value to report)
192Iterations                      : 100000                                (number of iterations used)
193Compiler version        : GCC3.4.4                              (Compiler and version) 
194Compiler flags          : -O2                                   (Compiler and linker flags)
195Memory location         : Code in flash, data in on chip RAM
196seedcrc                         : 0xe9f5                                (identifier for the input seeds)
197[0]crclist                      : 0xe714                                (validation for list part)
198[0]crcmatrix            : 0x1fd7                                (validation for matrix part)
199[0]crcstate                     : 0x8e3a                                (validation for state part)
200[0]crcfinal                     : 0x33ff                                (iteration dependent output)
201Correct operation validated. See readme.txt for run and reporting rules.  (*Only when run is successful*)
202CoreMark 1.0 : 6508.490622 / GCC3.4.4 -O2 / Heap                                                  (*Only on a successful performance run*)             
203(end example)
204
205Topic: Legal
206See LICENSE.txt or the word document file under docs/LICENSE.doc.
207For more information on your legal rights to use this benchmark, please see
208http://www.coremark.org/download/register.php?pg=register       
209
210Topic: Credits
211Many thanks to all of the individuals who helped with the development or testing of CoreMark including (Sorted by company name)
212o Alan Anderson, ADI
213o Adhikary Rajiv, ADI
214o Elena Stohr, ARM
215o Ian Rickards, ARM
216o Andrew Pickard, ARM
217o Trent Parker, CAVIUM
218o Shay Gal-On, EEMBC
219o Markus Levy, EEMBC
220o Ron Olson, IBM
221o Eyal Barzilay, MIPS
222o Jens Eltze, NEC
223o Hirohiko Ono, NEC
224o Ulrich Drees, NEC
225o Frank Roscheda, NEC
226o Rob Cosaro, NXP
227o Shumpei Kawasaki, RENESAS
Note: See TracBrowser for help on using the repository browser.