How to run SPEC CPU2017 and disable checksum

1. Environment Preparation

​Install Dependencies:

• For Ubuntu/Debian (ARM cross-compilation example):

sudo apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu gfortran-aarch64-linux-gnu

• Ensure ulimit is set to unlimited:

ulimit -s unlimited

2. Install SPEC CPU2017

• ​Mount the ISO File:

mount -o loop cpu2017-1.1.5.iso /mnt  # Mount ISO to /mnt
mkdir /home/speccpu
cp /mnt/* -r /home/speccpu  # Copy files to installation directory
chmod -R 755 /home/speccpu  # Set permissions

• Run Installation Script:

cd /home/speccpu
./install.sh  # Follow prompts and confirm with "yes"

3. Configure the Test Environment

• ​Modify the Configuration File:
  • Copy the default config (e.g., Example-gcc-linux-aarch64.cfg) and adjust:

vi /home/speccpu/config/Example-gcc-linux-aarch64.cfg

• Key parameters to update:
  • copies and threads: Set to match your CPU core count (e.g., 48 cores → 48 copies/threads) 
  • Compiler paths: Ensure gcc_dir points to the correct GCC installation (e.g., /usr/bin/gcc) 
  • Optimization flags (e.g., -Ofast -march=armv8-a) for performance tuning
  • Cross-compilation setup (for ARM):

SPECLANG = /usr/bin/aarch64-linux-gnu-  # Cross-compiler prefix

• ​Load Environment Variables:

source /home/speccpu/shrc  # Activate SPEC environment

4. Run Benchmark Tests

• ​Basic Commands:

cd /home/speccpu/bin
# Run individual tests:
./runcpu -c ../config/Example-gcc-linux-aarch64.cfg intrate   # Integer rate
./runcpu -c ../config/Example-gcc-linux-aarch64.cfg fpspeed   # Floating-point speed

• ​Run All Tests:

./runcpu -c ../config/Example-gcc-linux-aarch64.cfg intrate intspeed fprate fpspeed

Expected runtime: ~25 hours for all tests on a 48-core system.

5. Analyze Results

• ​Output Location:
  Results are stored in /home/speccpu/result/ as .txt files (e.g., CPU2017.003.intrate.refrate.txt).
• ​Key Metrics:
  • ​Rate tests: Measure throughput (multiple copies). Higher values indicate better performance.
  • ​Speed tests: Measure single-threaded performance.

Performance Tuning Tips

• ​BIOS Settings: Enable NUMA, set Power Policy to Performance, and disable Transparent Huge Pages.
• ​OS Configuration:
  • Use large pages for 4K page systems:

echo 100000 > /proc/sys/vm/nr_hugepages
export HUGETLB_MORECORE=yes

Troubleshooting

• ​Permission Issues: Use sudo for restricted operations.
• ​Compilation Errors: Verify compiler paths in the config file.

For detailed configuration examples or cross-compilation setups, refer to the official SPEC documentation or community guides.

---

---

---

Run specific benchmark of cpu2017

cd /home/speccpu/bin

# Run only 600.perlbench_s (intspeed suite):
./runcpu --config=myconfig.cfg \
  --noreportable \          # Skip reportable run requirements (faster)
  --iterations=1            # Run once (default is 3 for reportable runs) \
  --action=run 600.perlbench_s

---

---

---

How do I disable SPEC CPU checksum?

Explanation

For SPEC CPU 2017, check out the config file options for runcpu. It lists two options that may be of interest that you can put in a header section: strict_rundir_verify and verify_binaries. I pasted their descriptions below.

strict_rundir_verify=[yes|no]:

When set, the tools will verify that the file contents in existing run directories match the expected checksums. Normally, this should always be on, and reportable runs will force it to be on. Turning it off might make the setup phase go a little faster while you are tuning the benchmarks.

Developer notes: setting strict_rundir_verify=no might be useful when prototyping a change to a workload or testing the effect of differing workloads. Note, though, that once you start changing your installed tree for such purposes it is easy to get lost; you might as well keep a pristine tree without modifications, and use a second tree that you convert_to_development.

verify_binaries=[yes|no]:

runcpu uses checksums to verify that executables match the config file that invokes them, and if they do not, runcpu forces a recompile. You can turn that feature off by setting verify_binaries=no.

Warning: It is strongly recommended that you keep this option at its default, yes (that is, enabled). If you disable this feature, you effectively say that you are willing to run a benchmark even if you don’t know what you did or how you did it — that is, you lack information as to how it was built!

The feature can be turned off because it may be useful to do so sometimes when debugging (for an example, see env_vars), but it should not be routinely disabled.

Since SPEC requires that you disclose how you build benchmarks, reportable runs (using the command-line switch --reportable or config file setting reportable=yes) will cause verify_binaries to be automatically enabled. For CPU 2017, this field replaces the field check_md5.

For SPEC CPU 2006, these two options also exist, but note that verify_binaries used to be called check_md5.

Example

Example. I recently built the SPEC CPU 2017 binaries, patched them (in their respective exe directories), and then performed a (non-reportable) run. To do this, I put the following in the “global options” header section of my configuration file:

#--------- Global Settings ----------------------------------------------------
...
reportable           = 0
verify_binaries      = 0
...

before building, patching, and running (with the --nobuild flag) the suite.