Standard Performance Evaluation Corporation
The Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse standardized benchmarks and tools to evaluate performance and energy efficiency for the newest generation of computing systems. SPEC develops benchmark suites and also reviews and publishes submitted results from our member organizations and other benchmark licensees.
06/27/2017: SPEC’s High-Performance Group (SPEC/HPG) has released a new version of its SPEC ACCEL software that adds a suite of OpenMP applications for measuring the performance of systems using hardware accelerator devices and supporting software. Other major updates in SPEC ACCEL 1.2 include the ability to add and change directives within the OpenMP and OpenACC suites to expose more parallelism for peak performance testing, and revised applications in the OpenACC suite to allow for successful GNU compiling.
06/20/2017: SPEC announces the release of SPEC CPU2017. an all-new version of the benchmark suite for evaluating compute-intensive performance across a wide range of hardware systems. CPU2017 contains 43 individual benchmarks organized into four suites. Among the new features are the option to use OpenMP for several of the SPECspeed 2017 benchmarks and the ability to optionally measure and report power statistics. The suite is available now, with licensees of earlier versions of the CPU suites eligible for a discounted upgrade price.
06/20/2017: Due to a known issue in Microsoft Windows Server 2016 memory management, SERT will not be supported on that operating system until a fix is made available.�
06/20/2017: SPEC cannot provide support on SPECpower_ssj2008 for test failures using Microsoft Windows Server 2016 due to an issue with large pages and memory management in the operating system.� Please contact Microsoft technical support for assistance.� Completed runs on Windows Server 2016 will be processed for review as usual.
06/08/2017: The 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018) has issued a call for contributions. Topics of interest include: performance modeling of software; performance and software development processes/paradigms; performance measurement, monitoring and analysis; benchmarking; run-time performance management; power and performance, energy efficiency; performance modeling and evaluation in different environments and application domains; and other topics related to performance of software and systems. ICPE2018 will take place April 9-13, 2018 in Berlin, Germany.
06/05/2017: SPECpower has released a new version of PTDaemon. the software that allows power measurement devices to be incorporated into performance evaluation software. PTDaemon 1.8.1 includes new support for Windows USBTMC using Keysight drivers and the Hioki PW3335 (single channel) power measurement device. Users of SPECpower_ssj2008 and the SERT suite are encouraged to download and apply the update.
05/23/2017: SPEC’s Research Group has issued a call for nominations for the Kaivalya Dixit Distinguished Dissertation Award recognizing outstanding doctoral dissertations in the field of computer benchmarking, performance evaluation, and experimental system analysis. The award is open to dissertations that have been defended between October 2016 and September 2017.
05/22/2017: In an exclusive interview. Klaus-Dieter Lange, SPECpower chair, discusses the origin of SERT, the challenges of creating an international standard, the prospect of continued improvements in server efficiency, and his own personal growth in leading a diverse group of technologists.
05/22/2017: SPEC’s High-Performance Group (SPEC/HPG ) is offering rewards of up to $5,000 and a free benchmark license for application code and datasets accepted under its new SPEC MPI Accelerator Benchmark Search Program. Applications that make it through the entire search program will be incorporated into a new benchmark currently called SPEC MPI ACCEL. The new benchmark will combine components of the current SPEC MPI and SPEC ACCEL benchmark suites.
04/18/2017: Version 2.0 of the Chauffeur Workload Development Kit (WDK) has been released. This major release is based on the SERT 2.0 infrastructure and includes significant enhancements to the hardware detection, customization options of generating HTML reports, and developer documentation. It has now reduced memory requirements for the Director when signing results files and reduced the size of the result output for large systems or clusters. There is also an updated list of supported operating systems including Ubuntu (14.04 LTS and 16.04 LTS) as well as current versions of Windows Server, RHEL, SLES, AIX, and Solaris.
04/04/2017: A patch has been released for SPECvirt_sc2013 V1.1 which addresses an issue encountered when running the webserver workload with TLS encryptions which would cause any result to be marked as invalid. Any result submissions using TLSv1, TLSv1.1, or TLSv1.2 encryptions for the webserver workload must use this patch.
04/04/2017: Scott Beamer III from the University of California, Berkeley is the winner of the 2016 SPEC Kaivalya Dixit Distinguished Dissertation Award with his dissertation on the characterization and treatment of fundamental performance bottlenecks in graph processing and the contribution of the Graph Algorithm Platform (GAP) benchmark suite.
03/28/2017: SPEC announces the release of the Server Efficiency Rating Tool (SERT) 2.0. which adds a single-value metric, reduces runtime, improves automation and testing, and broadens device and platform support. Designed to be simple to configure and use via a comprehensive graphical user interface, the SERT suite uses a set of synthetic worklets to test discrete system components such as processors, memory and storage, providing detailed power consumption data at different load levels.
03/22/2017: Walter Bays steps down as SPEC president after presiding over 12 years of growth; he is succeeded by David Reiner, a SPEC 12-year veteran.
[Older items can be found in the archive .]