SERT Evaluation Programme

Introduction

As part of the European Commission’s technical assistance project to support standardisation-related tasks for DG ENTR Lot 9 products (enterprise servers and data storage), a European Evaluation Programme for the SERT suite is being launched in coordination with the Standard Performance Evaluation Corporation (SPEC).

The Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers.

The Lot 9 study has identified SERT as a useful tool for measuring server efficiency.  It is normally necessary to purchase a license of the SERT software in order to use it for testing purposes.  However, the Lot 9 study has reached an agreement with SPEC to offer EU stakeholders a temporary free-of-charge Evaluation version of the SERT tool for trial purposes.

Server Efficiency Rating Tool (SERT)

The SERT suite is intended to measure server energy efficiency.  Development began in collaboration with the U.S. Environmental Protection Agency (EPA) ENERGY STAR for Computer Servers program. Designed to be simple to configure and use via a comprehensive graphical user interface, the SERT suite uses a set of synthetic “worklets” to test discrete system components such as processors, memory and storage, providing detailed power demand data at different load levels. Results are provided in both machine- and human-readable forms, enabling automatic submission to government-sponsored certification programs as well as both summary and detail reports for use by potential customers.

In principle, SERT aims to achieve the following goals:

  • Energy assessment across a wide spectrum of server configurations
    • Scores that scale with available hardware resources
    • Worklets stress multiple subsystems
  • Easy to use, with minimal equipment and skill requirements
    • Automatic discovery of hardware / software
  • Agnostic
    • Architectural, OS, Platform, Implementation Languages
  • Repeatability, Extensibility, and Scalability

Objectives of this Evaluation Programme

It is hoped that this trial will provide the Lot 9 study with insights on ease of use, procedural refinements/clarifications and insights on potential directions for development of an EU metric based on SERT, should such an approach be deemed appropriate. Focus areas include:

  • Ease of set-up and use
  • Execution experiences
  • Run rule clarifications
  • Load characteristics;
  • Measurement characteristics
  • Bug discovery.

Want to take part?

In order to optimise the impact of the trial programme, the project can benefit from stakeholder involvement. Stakeholders can register their interest using the form below, and should ensure (before registration) that they are able to allocate the following resources to the Evaluation test:

  • Power Analyzer and Temperature Sensor from the Accepted Measurement Devices List (http://www.spec.org/power/docs/SPECpower-Device_List.html)
  • Computer Systems: Two are required, one is the Server Under Test (SUT) and a second system for control and data collection functions.
  • Sufficient engineering resource to complete the tests by end April 2016, in order that the results can be considered in the context of the EC Technical Assistance Study (the license itself will be valid until the beginning of September 2016).

Once you have registered your interest we will contact you to collect information on the type of computer system/server to be used for the test and an anticipated test plan. This will be forwarded to SPEC for them to prepare a dedicated kit and grant a license to successful applicants.

Technical Support

The following SERT documentation is available for support purposes:

Technical Support will be provided by SPEC in both English and German languages – Issues installing/running the SERT can be reported via: http://www.spec.org/sert/feedback/issuereport.html.

Closure of the Evaluation Test

The SERT Full Disclosure Report (FDR) of all measurements will be collected by Evaluation programme administrators in .xml format from all participants (plus accompanying readable .html and .txt documents). The FDR is the automatically generated SERT results output, which contains the configuration information as well as the actual measured power and performance values.

Once the SERT Evaluation Program has closed, contributors will be required to confirm that they have followed the license agreement and destroyed all SERT Evaluation material.

Important Notice

THIS NOTICE MUST APPEAR UNALTERED IN ALL EVALUATION PROGRAMME RESULTS.

“The SERT Evaluation license agreement prohibits the use of numerical information from this report to make public comparisons promoting the use of one product over another.”

Register

Organisation Name (required)

Position held in Organisation (required)

First Name (required)

Last Name (required)

Email (required)

I confirm that I have (all required):

 A suitable Power Analyzer and Temperature Sensor from the Accepted Measurement Devices List (http://www.spec.org/power/docs/SPECpower-Device_List.html)

 Computer Systems: Two are required, one is the Server Under Test (SUT) and a second system for control and data collection functions.

 Sufficient engineering resource to complete the tests by end April 2016.

Power Analyser Make and Model (required)
Temp Sensor Make and Model (required)

I have read (all required):
 Design Document - http://www.spec.org/sert/docs/designdocument.pdf

 User Guide - http://www.spec.org/sert/docs/userguide.pdf

 Run and Reporting Rules - http://www.spec.org/sert/docs/runrules.pdf

Final confirmation and agreement to conditions (all required):

 I agree to provide information on the type of computer system/server to be used for the test and an anticipated test plan. Information to be provided as part of the licence application.

 I accept to provide the SERT Full Disclosure Report (FDR) and accompanying automatically generated .html and .txt documents to the Evaluation Test Programme administrators. In line with the license agreements, I accept to destroy all SERT Evaluation material once the SERT Evaluation Programme has closed.