Carbon-aware Computing

Globally, emissions from the world’s growing fleet of data centres are an emerging concern. In Australia, total DC power consumption will rise from 170MW in 2020 to a forecast of 224MWh by 2026, a 32% increase over 6 years. Under this scenario, corresponding annual C02 (measured with Australia’s average C02-e per kWh of 0.78kg from electrical generation sources, and using current best practice air-cooled DC PUE of 1.3) could rise from 880 kilotonnes to up to 1.1 million tonnes by 2026.

As young Australians concerned with our planet’s future, we recognised not only was the current growth in compute and carbon unsustainable, but we would need a breakthrough in technology to achieve the rapid shift in emissions so sorely needed.

Our world-leading immersion cooling system, with one of the lowest possible PUEs in the world at 1.03, is critical to this shift. So too is where this would be located, and how it would be used. When we looked at Australia’s energy market for inspiration, we came across Tasmania as a potential site.

Tasmania has bountiful natural resources capable of producing clean renewable energy far in excess of its local needs, in November 2020 reaching an important milestone of generating 100% of on-island demand from renewable sources (Premier of Tasmania – Tasmania surges to 100% renewable energy).

Tasmania isn’t stopping there, however, with an ambitious program called Battery of the Nation, which aims to double current local renewable production for export to mainland Australia. You can read more about BoTN and how Firmus is helping Tasmania achieve its renewable objectives at our piece on Energy Services.

Firmus’ immersion-cooled data centre, despite being one of the most energy-efficient in the world, will still require large amounts of power to host the clouds of tomorrow. Tasmania, with abundant renewable energy, was the perfect home to start building Australia’s next big innovation in energy efficiency.

Data Centre Basics

Using Tassie renewable energy is important, however, it is our breakthrough in cooling efficiency that makes Firmus’ approach to building data centres a true solution to carbon aware computing for the clouds of tomorrow.

Noted above, data centres – hosting the world’s most powerful compute and storage clouds used by all of us – consume vast amounts of energy. Estimates of global consumption up to 416 TW/h in 2016 alone equated to around 3% of global power usage and 2% of global carbon emissions attributable to large data centres. (Global warming: Data centres to consume three times as much energy in next decade, experts warn | The Independent | The Independent).

By 2025, up to 4.5% of global electricity is expected to be consumed by data centres alone. ((PDF) Total Consumer Power Consumption Forecast (researchgate.net))

Data centres, essential to powering cloud services, present a real and present challenge to the global community and our response to climate change. Demand for cloud services – powering almost every experience in our daily lives – keeps on growing.  COVID-19, requiring a rapid shift to flexible work and keeping many of us away from the office, has served to speed up this growth even further.

The servers powering these services live in data centres, and like all ICT equipment, produce a lot of excess heat when operating. One of the primary jobs of a data centre is to keep the servers from overheating, so they can operate reliably and safely.

Since the dawn of the computer age, air has been the method of choice to cool computers, with refrigerated air pumped over the surface of servers, moving away as much heat as possible to keep the servers cool.

The problem is, air is actually not very good at this job! Naturally, air can transfer heat at a rate of around 2.5 watts per m2. The Firmus solution, which we’ll get to in a bit, uses a non-conductive cooling fluid to perform the same job. Our fluid can wick away heat at around 2,500 watts per m2, and up to 1,000 x improvement over air.

To overcome the inefficiency of air as a method to remove heat, data centres invest in costly and complicated refrigeration systems, requiring all kinds of engineering tricks to eke out the most of these systems. From ‘hot aisle/cool aisle’ layouts, portable CRAC units, raised floor piping, direct feed cooling, the list goes on. All this is costly to build and costly to the environment.

Power Usage Effectiveness

An easy way to measure a data centre’s energy efficiency is through a ratio known as Power Usage Effectiveness or PUE. PUE is defined as the amount of power entering a data centre divided by the power routed to the physical information infrastructure (eg, servers). The PUE ratio improves as the quotient decreases towards the minimum value of 1.00.

Becoming more efficient in cooling systems is one of the easiest ways to reduce total energy usage. The world’s global fleet of data centres, as measured by an annual survey from 2020, has a PUE of 1.59. With a PUE of 1.59, for every 100 kw of servers hosted, an additional 59kw is required to run the DC’s systems, mostly comprised of the cooling infrastructure.

This is a huge overhead and a central challenge point of Firmus’ approach to carbon aware computing. Driving that number down towards 1 is a key objective for all of us in the ICT industry, like an LED lightbulb is to an incandescent – reducing waste, in this case electrical, is a key human response to climate change.

With our system, we sit at a stable 1.03 PUE, requiring only 3 kw for cooling systems for the same 100kw of compute. It’s a total game-changer.

The Firmus Solution

To build a breakthrough, we had to be bold. It was time for air to go.

Firmus selected a synthetic, non-conductive, highly stable cooling fluid with a 2,500w/m2 heat coefficient to move the heat in our DC.

The next step is how to do this, which is where it gets a little bit tricky! The most efficient method is to take a standard air-cooled computer, and physically immerse it in cooling fluid within a specially designed tank that can accommodate standard 19” RU servers (we call this an Immersion Rack).

The fluid, pumped in cool at the bottom of the Immersion Rack, flows up through the servers, passing over hot components and, using that high heat transfer coefficient, efficiently wicks away the heat from the chassis. Once at the top of the Rack, the hot fluid is sucked out, passes through a closed-loop heat exchanger, before being pumped in anew as cool fluid ready for another round.

Firmus’ cooling fluid has a 20-year lifespan, so there is no waste or need to replace it for a very long time!

Aside from being energy-efficient, we are land efficient too. Each Immersion Rack is capable of hosting up to 100KW of servers, far greater than the maximum possible with air-cooled, at around 20-25kw per rack. This density advantage, combined with a simpler plant design, means that Firmus uses up to 75% less land than an air-cooled DC for the same amount of compute.

Fitting Right In

It’s great to build efficient technology, but it’s no good if others can’t use it. From the beginning, we set ourselves the challenge of building a system that would be compatible with today’s data centre standards, minimising any friction in adapting to this immersion design.

We have worked hard to ensure that standards are met, starting with the Rack itself. Our Immersion Rack is an industry-standard 42RU, 19” server rack, however – on its back so that servers sit vertically. When presented this way, Firmus can host up to 100KW of computers per 42RU immersion tank, a four-fold increase in the capacity of today’s best air-cooled DC systems – all at a world’s best PUE to boot!

Networking and power delivery have been tweaked for immersion, however again presented in a way compatible with today’s servers.

Finally – and we’re pretty proud of this one – all of this is delivered in a fully redundant N+1 design for guaranteed uptime, just like leading air-cooled DCs. That means redundancy in power systems, cooling systems, networking, and all infrastructure required to keep the servers on and operating.

Being Carbon Aware is Good Business

De-carbonising the economy is good business, despite some of the narratives out there. Our robust, low-carbon immersion system is not only cheaper to build, but materially cheaper to run.

That amazing fluid heat transfer coefficient, at the heart of our system, requires a significantly less complex mechanical cooling system to run, as against air. This leads to a build cost nearly 4 times lower per megawatt than an air-cooled DC, for a fully redundant N+1 design.

Then in operation, the ultra-low PUE – so good for the environment – is also good for the pocket. Against the global standard, we use up to 95% less power in mechanical systems.

Combining a materially lower cost of capital with substantially lower operating costs is a potent mix. Being carbon aware truly is good business.