Data Center Efficiency Case Studies

There is a common misunderstanding that increasing data center operating temperatures will result in degradation of IT equipment reliability and service availability. Airflow management optimization, rearrangement of air vent tiles, and wider operational temperature controls are such factors that can enable an overall reduction in energy consumption. In addition, operational cost savings of 4-5% can be achieved for every 1° F. increase in server inlet temperature.

Currently, the optimal operational temperature for IT equipment in most data centers consist of a thermal range of 68-72° F. As per the guidelines published by ASHRAE in 2008, there are new agreed upon Recommended and Allowable thermal and humidity ranges for IT equipment. The Recommended range is 65-80° F. with up to 60% relative humidity (RH). The Allowable range is 50-95° F. with up to 20-80% RH. The difference between the recommended and allowable range is the acceptance of potential reliability risks at the allowable range, such as corrosion and electrostatic discharge (ESD). These reliability risks can occur as a result of low or high humidity levels, however, effective operational protocols that combines cooling operation at both above and below 68° F. could ensure no change in the overall likelihood of equipment failure.

There are no known increases or decreases in temperature based on server utilization, both during or outside of school hours. This means that there could be excess cooling loads that are not necessary and potential areas of concern such as hot spots that have not been resolved. By examining data center improvement case studies conducted by Google, Intel, and Cisco, a similar solution can be implemented in order to achieve optimal efficiency and energy savings.

Google has a smaller data room, classified as Points of Presence (POPs), which achieved a power usage efficiency (PUE) from 2.4 to 1.5 after a series of small scale improvements. The PUE is a metric used to identify the efficiency of energy delivery from a building to the IT equipment located indoors. With an ideal PUE of 1.0, there is no facility overload energy (energy used by cooling, lighting, and power distribution), therefore every watt of power is going directly to the IT equipment. The POP configuration consisted of double-conversion uninterrupted power supplies (USPs) with an IT load of 250 kW, four 111 kW computer room air conditioners (CRACs) with a direct expansion (DX) cooling coil, and racks that contained third-party equipment such as network routers, power supplies, load balancers, and optical switches. The initial analysis showed that the data room was overcooled, overpowered, and overused.

The baseline airflow model of Google’s POP indicated an inefficient distribution of cold air over machines before reaching the hot aisle.

The baseline airflow model of Google’s POP indicated an inefficient distribution of cold air over machines before reaching the hot aisle.

After installing temperature monitors, thermal models using computational fluid dynamics (CFD) were created as to simulate airflow. Next, air vent tiles were rearranged so that the side of the room with high power racks had matching airflow with IT power. The temperature and relative humidity settings were then increased from 71° F. at 40% RH to 81° F. at 20-80% RH. Other improvements included: placement of refrigerator curtains to block off ends of cold aisles, installation of blanking plates and side panels to block cold air passing through cold racks, 48” sheet metal box extensions to all CRAC air returns, and a new CRAC controller to enable dynamic control of all CRACs. Lastly, the capital investment of $25,000 for these improvements led to a yearly energy savings of over 670MWh and $67,000. The financial payback of each improvement were all under a year.

After implementing changes to the Google POP, the final airflow model indicated an efficient cooling of machines and the network room.

After implementing changes to the Google POP, the final airflow model indicated an efficient cooling of machines and the network room.

Intel has also tested and evaluated servers in a 3,000 square feet data center to see how high-temperature ambient (HTA) operation conditions affect power consumption. Their primary goal was to find the highest potential cooling power savings and identify what effects can be encountered with the continuous operation of IT equipment at high temperatures. The initial baseline inlet server air temperature was 61/64° F and were raised to set points of 75° F., 82° F., 91° F., and 100° F. and server loads of 25, 50, 75, and 100% utilization. The results showed the following: 1) power consumption of servers began to increase on all models at temperatures above 82° F., 2) the optimal set point was found by staying within an increase of 50° F. up to a temperature of 82° F., 3) server functionality was not affected by HTA conditions of up to 100° F., and 4) at least 9% potential yearly power savings of total power. Intel also stated components of an overall power efficiency strategy in data centers: HTA design practices, air containment, airside economizers for cooling efficiency, software tools that provide analytics and dynamic management to optimize the server environment, and power-efficient servers.

Four labs at the Cisco San Jose building achieved energy savings of 13-21% for raising set points from 68-72° F. to 76-80° F. The chilled water set point was also increased from 44-48° F. Note that Cisco equipment has been tested to withstand 104° F. while non-Cisco equipment can withstand 95° F. The average power consumption of the building consisted of 2,700 kW total, 800kW cooling, and 1,800 kw IT. Similar to Google’s POP, Cisco installed power meters and temperature sensors, ran simulation models, and implemented solutions to help optimize airflow and cooling efficiency. Blanking panels were installed to reduce the amount of short-circuiting air and floor grills were placed where needed in the labs to achieve uniform temperature in aisles. Also, the elimination of cold and hot areas enabled the ability to raise the temperature set point without causing overheating issues. As a result, there were at least 910,000 kWh in overall energy savings.

It is evident that increasing the temperature set points at any data center will enable greater energy savings without compromising the functionality, reliability, or service availability of IT equipment. Given a comprehensive analysis of current IT/data rooms (temperature monitoring and simulation models), any inefficiencies such as airflow and hot spots can be identified, along with dynamic control of increased temperature set points. 

References:

  1. Data Center Efficiency and IT Equipment Reliability at Wider Operating Temperature and Humidity Ranges:http://www.thegreengrid.org/~/media/WhitePapers/WP50-Data%20Center%20Efficiency%20and%20IT%20Equipment%20Reliability%20at%20Wider%20Operating%20Temperature%20and%20Humidity%20Ranges.pdf?lang=en
  2. Google Server Room Case Study: https://www.energystar.gov/sites/default/files/asset/document/Google_Server_Room_Case_Study_0.pdf
  3. High-temperature hosting data center : https://www-ssl.intel.com/content/dam/www/public/us/en/documents/white-papers/high-temp-hosting-data-center-paper.pdf
  4. Cisco Lab Setpoint Increase: http://svlg.org/wp-content/uploads/2012/12/Cisco_cs.pdf