We are currently generating more data than ever before. Why is that?
Unstructured data is becoming more prevalent thanks in part to our tendency to convert text, pictures, or music into a digital format for computer processing, social networking, search engine inquiries, and real-time streaming. These assist in creating a large volume of digital data that needs to be stored and may not be accessed later.
In addition, modern enterprises are continuously generating and accumulating data. This includes routine activities across enterprise systems, machines, sensors, and demand-side digitalisation, placing tremendous stress on our data centres. According to IDC, worldwide enterprise data storage is rising at a 27 per cent compound annual growth rate (CAGR).”
So, is dark data the problem here?
Dark data has both pros and cons. The disadvantages are frequently more obvious than the benefits, and they are legitimately serious. Storing such a high percentage of concealed data, just like dark matter, may be difficult to analyse, as dark data far outnumbers the amount of visible data. While visible data may be easily accessed in databases, dark data requires a more sophisticated extraction process before being actively utilised. Its storage and security usually comes at a higher cost and poses unique risks to the data itself – including theft, malware and DDoS attacks.
Moreover, storing a massive amount of dark data on a conventional hard disk drive (HDD) and solid-state drive (SSD) storage type wastes a significant amount of energy, most of which is powered by non-renewable resources, to keep stored data alive. This ultimately increases CO2 emissions.
Will increased digitisation hinder our net zero plans?
Increased storage drives power demand which is carbon intensive and is largely generated from non-renewable sources. For example, in the United States, power consumption due to data centre data storage was estimated to be at 14 billion kWh in 2020 resulting in almost 6.5 metric tons of CO2 emissions. According to Veritas Global, the storage power required to hold and process global dark or redundant data is estimated to emit 5.8 metric tons of CO2 annually. Organisations also need to consider the infrastructure impact of storage cooling.
Although it is impossible to precisely calculate emissions, the entire ICT sector is estimated to account for about 1.4 per cent of CO2 emissions globally. Therefore, business leaders need to be considering the unnecessary waste associated with data storage and begin to generate power from renewable sources. Efforts have been made to address energy efficiency and curb emissions however, this growth has slowed considerably. In fact, workloads (and data) are now hosted in multiple mirrored sites due to demand for increased resiliency. More needs to be done, we simply cannot continue the way we are going.
Can this be addressed at a global or local scale through effective policy?
Huge strides in energy efficiency including a shift to efficient ‘hyperscale’ data centres have helped to limit data centre electricity demand growth globally. At the local level, however, these large hyperscale data centres represent huge electricity demands, adding pressure to electricity grids and increasing the challenge of energy transitions, especially in smaller countries. For example, a recent report by Eirgrid, forecast that data centres' energy consumption is expected to increase by 9 TWh by 2030 – with estimates ranging from 23-31 percent of total grid supply in Ireland alone.
Achieving sustainability means addressing environmental considerations during solution design as well as during the build. Solutions must meet pre-defined and agreed environmental sustainability criteria. Sustainable data centres require designing according to energy conservation, energy efficiency and renewable energy sources.
Carbon neutral data centres should be powered by purely renewable energy. The Climate Neutral Data Centre Pact states that data centre electricity demand will be matched by 100 per cent renewable energy or hourly carbon-free energy by 2030. Whilst these areas indeed ambitious objectives, they must be addressed at local level. It will take the collective effort of regulators, policy makers and business leaders to make this work and establish meaningful change.”
How can IT/data centre leaders operate more sustainably?
Businesses and data centres should start by simply taking control of data storage, assessing the storage rules, and ensuring they are not holding data that is no longer needed. Eliminating data centres unstructured data improves regulatory compliance and can lowers maintenance costs, which in turn curbs their electricity output, reducing their emissions. It is essential that leaders improve their data management policies, identifying which data is in fact valuable, and eliminating any dark or redundant data from their data centres to avoid emissions spiralling out of control and avoid unnecessary digital waste.
Leaders should also enquire about their data centre provider’s multi-site footprint and its ability to enable distributed network availability and dynamic load placement. Working to identify workloads that can be transferred from peak demand periods to off-peak hours ultimately results in lower costs for all parties. They should also explore new air con approaches such as single-phase, two phase and direct chip to cooling immersion liquid for components and racks. Aside from low carbon construction, they should also consider phase change materials in building thermal management.
In theory, data centres and server rooms could be powered by purely renewable energy, but there is a long way to go yet. In the meantime, data centres and server rooms must strive for a high standard for energy efficiency, demonstrated through aggressive power use effectiveness (PUE) targets. Increasing efficiency will go a long way to mitigating the amount of energy needed by energy-hogging data centres. Similarly, data centres continue to rely on vast amounts of water for computer cooling. Business leaders have a responsibility to reduce this to minimum levels by setting ambitious water conservation targets.
How can this be addressed at a higher level? Are there any key target areas? Does it rely upon our own moral responsibility?
Moving forward, organisations across the globe should have a moral obligation to filtering dark data and removing unnecessary information from storage. An international standard on data archiving is needed to avoid unnecessary ‘digital landfilling’ and mountains of dark data storage, which will naturally bring about meaningful changes to other services across the ICT sector. Of course, these things take time, but it will inevitably reduce data centre energy, carbon emissions, as well as water and land footprints.
Aoife Foley is an IEEE Senior Member and Professor in the School of Mechanical and Aerospace Engineering at Queen’s University Belfast