Measuring Power Consumption of RAN Automation Infrastructure

Energy Efficiency (EE) is an emerging topic in 5G and 6G networks. However, the discussions usually concentrate on the energy savings at the network equipment level provided by the RAN automation algorithms like Cell On/Off Switching, or RF Channel Reconfiguration. These algorithms also run on specific hardware (HW) and contribute to the overall power consumption (PC) due to performing computationally-intensive calculations or utilizing power-hungry AI workloads.  Things are getting even more complicated if we take into account that these algorithms, along with the Service Management and Orchestration (SMO) or Non-Real Time RAN Intelligent Controller (Non-RT RIC), are often deployed in a virtualized environment. This creates a significant challenge in evaluating their contribution to the E2E EE of the mobile network.

Rimedo Labs, together with i14y Lab, synthesized the standardization efforts from 3GPP, ETSI, and O-RAN ALLIANCE into the unified O-RAN E2E EE Testing Framework (see White Paper). The proposed framework brings SMO/Non-RT RIC and rApps into the equation, enabling evaluation of the tradeoff between running RAN automation algorithms and their impact on RAN.

In this blog post, we demonstrate the implementation of the E2E EE Testing Framework in practice. We cover the PC measurements of multiple Virtual Network Functions (VNFs) (see Testing VNF power consumption). We present the relevant tools and discuss the results obtained in i14y Lab while using a Dell R750 server hosting rApp Lab (a “Lightweight Non-RIC”) and communicating with Keysight RICTest emulating the RAN.

Measurement Tools: Three-Tiered Approach

As in the virtualized environment, multiple functions, like RIC or rApps, share the same HW resources, one of the challenges is to evaluate their contribution to the overall PC. For instance, this is not possible while using a single meter for the entire server. Therefore, to provide a comprehensive evaluation of the PC of certain VNFs, or to compare the HW/servers used to host those VNFs, we need a methodology and proper tools.  They must provide insights into the PC of HW components like the fan or storage. They also need to enable measuring the PC of individual VNFs.

In our approach to get insights into the individual elements of the stack in a virtualized environment, we propose a three-tiered approach using measurement tools as depicted in Fig. 1:

• The electric socket meter can provide information about the total PC of the server. While this is not sufficient for assessing individual components’ consumption, it is important for calibration purposes. For instance, the values provided by other measurement tools, like Redfish, must result in the same total PC as that from the electric socket.
• Redfish (see REDFISH)is a standardized API dedicated to managing enterprise-grade servers. Among the variety of supported metrics, some of them enable monitoring of PC related to the different HW components, like: power supply loss, memory, storage, fan power, remaining power, and total CPU power.
  
  During our evaluation, we observed that, when using Redfish Telemetry, the results are properly calibrated against the values from the electric socket, making this approach recommended in our testing methodology.
  
• Kepler (see Understanding Kepler)is a tool that enables going deeper than simply getting the total CPU PC values provided by Redfish. Using internal analytics, Kepler exposes information on the PC of CPU and DRAM, further split into processes, Kubernetes PODs, and Containers.
  
  This enables diving into the PC of individual VNFs, e.g., rApps or components of SMO/Non-RT RIC. Note that Kepler should be calibrated against Redfish to check whether their reported total CPU power is the same.

Virtualized RAN Environment: Test Setup

Once we had the tools dedicated to PC measurement in place, we performed lab tests. These were conducted in the 14y Lab using the setup presented in Fig. 2.

For conducting the tests, we used two separate servers. The first one is dedicated to RAN and O1 interface emulation by Keysight RICTest (as discussed later). The second machine is a DELL R750 enterprise-grade server, which is the unit under test for PC measurements. It is equipped with two 28-core Intel Xeon Gold CPUs, 512GB of RAM, 6 network interface cards, and runs Ubuntu 20.04.6 LTS as the operating system.

Apart from the standard set of applications required for its operation, the DELL R750 server is hosting 4 components, subject to further discussion:

• rApp Lab: a lightweight Non-RT RIC based on Docker Compose, developed by Rimedo Labs. It provides an O1 interface termination enabling the capture of both Performance Management (PMs) metrics and Configuration Management (CM).
• COOS-rApp: is a Cell On/Off Switching rApp (COOS-rApp) running on top of the rApp Lab. It analyzes the cells’ load and modifies their energy-saving state through the O-CES Management Function (see How can Energy Saving and Traffic Steering cooperate in O-RAN?).
• Power Consumption Measurement Tools: is an implementation of the three-tiered approach discussed in the previous section. The DELL R750 comes with a Redfish API, enabling the PC telemetry of its HW components, and deployed Kepler for monitoring PC of the individual rApp Lab components and COOS-rApp. Both Redfish and Kepler are calibrated as described in the previous section. Their collected data is exposed by Prometheus (see What Prometheus is?) to the external services, e.g., for graphical representation.
• Visualization with Grafana (see Grafana OSS and Enterprise): consumes the data exposed by the PC Measurement Tools through Prometheus and provides the live-monitoring capabilities through the dedicated dashboards, both for HW components (based on Redfish data) and software (SW)/VNF components based on Kepler data.

Once we have the test setup and tools, throughout the next two sections, we will dive into the actual PC measurements.

Virtualized RAN Environment: Test Setup

First, we focused on breaking down the PC of the DELL R750 server into its HW subcomponents using the Redfish telemetry. The example of a related Grafana dashboard is depicted in Fig. 3.

To obtain these results, DELL R750 is running the necessary applications and the rApp Lab (without RAN data processing).  In this case, the total system’s PC is 618 W (see the top left corner of Fig 3). The green gauges on the bottom right indicate the PC of individual HW components, and the plot on the bottom left summarizes their contribution to the overall PC. The key observation here is that the fan and CPU are consuming most of the power – 258 W (about 41%) and 211 W (about 33%), respectively.

The above results were obtained with the server’s CPU load of 16%. Now, the question arises about how the PC of HW components depends on the CPU load. We evaluated this aspect by using dedicated software (see Ubuntu Stress Tool) to enforce a certain static CPU load along with Redfish telemetry. The obtained results are presented in Fig. 4. There are two key observations:

• Except for the CPU, all other components consume almost a static amount of power. This creates a natural question for future optimization, e.g., if some savings can be achieved while adjusting fan speed.
• The CPU load-dependent PC component is relatively small, i.e., under the idle state of about 15% load, DELL R750 consumes about 620 W, while under 100% CPU load, about 760 W. This can lead to the conclusion that these machines are somewhat energy-efficient while operating at their high load.

Evaluation of the Contribution of Individual VNFs to the Overall Power Consumption

To have a full insight into PC in a virtualized environment, the HW-related metrics provided by Redfish Telemetry alone are not sufficient, as they are not reported per single VNF. Thus, to get a detailed overview of the PC related to CPU and DRAM split among individual VNFs, we used Kepler. During the next phase of the measurements, we focused on the PC related to the rApp Lab components, e.g., RAN O1 notification, Performance Management measurements collection, COOS-rApp, and Influx/Redis databases.  

For these measurements, to create a realistic operation environment for rApp Lab and COOS-rApp, we needed to feed it with the RAN data. For this purpose, we used Keysight RICTest RAN emulator, placed on a separate machine, i.e., not affecting the PC of the DELL R750. Keysight RICTest emulates RAN and exposes the O-RAN-compliant O1 interface terminated by rApp Lab. The related network setup, along with a traffic profile, is depicted in the left part of Fig. 5. There are 12 cells in total: 3 coverage and 9 small cells, providing service to 12 groups of 10 UEs. We follow the varying network traffic depicted in Fig. 3 on the right to allow for COOS-rApp actions. In other words, while having a traffic load varying in time, we can trigger COOS-rApp decisions that potentially affect the PC of the CPU and DRAM.   

While having rApp Lab and COOS-rApp running, fed with RAN data, we dive into their power consumption contribution to the server. The representative example of the related data visualization is depicted in Fig. 6.

The dashboard is split into the DRAM-related PC on the left and the CPU-related PC on the right. Each of them provides gauges enabling PC breakdowns into the individual VNF components of the rApp Lab, and COOS-rApp deployed as Docker Containers:

• RAN O1 Notification
• Network Configuration
• Performance Management
• Influx DB
• Redis DB
• COOS-rApp

In the gauges, we can see that the most power is consumed by the Redis DB and Performance Management. Also, the COOS-rApp consumes power only when some periodic calculations are triggered.

However, the most interesting observations are these presented in the middle panel of Fig. 6. This is the PC vs time related to rApp Lab and COOS rApp. In this plot, we can see periodic spikes of increased PC. These are occurring every 2 minutes.

We can try to correlate this observation with RAN processes to get an insight into what operations use the most power. In this case, in network nodes are configured to report PM data every 2 minutes. This happens in a synchronised way. In other words, all cells are sending their PM data at the same time. This creates a short period when rApp Lab, or more specifically, its Performance Management VNF, utilizes more resources to process them. As a result, the DRAM and CPU PC spikes occur. Although they are only about 3 W, keep in mind that this is a small setup – 12 cells in total. If we increase the number of cells, the power consumption related to the App’s actions would increase proportionally. Using this methodology, we can provide estimations of how much power the VNF would consume in large/realistic network setups.

These Kepler-related measurements show that there is a relation between what is happening in the RAN and PC of certain VNFs, making it a beneficial tool for potential network diagnostics and optimization. In the case of RIC and rApps, a potential bottleneck can be the processing of PMs.

Conclusions

In our E2E EE Testing framework, we propose a three-tiered approach utilizing electric socket measurements for calibration, Redfish Telemetry for insights into PC of HW components, and Kepler for investigating PC of VNFs, e.g., Non-RT RIC  and rApps. We have validated the methodology by running lab tests in i14y Lab in Berlin, using a DELL R750 server, and Keysight RICTest for RAN emulation.

The key takeaways are as follows:

• The CPU and fan are the two most power-hungry parts of a server, which should be taken into account for optimization.
• The PC of an enterprise-grade server is mostly static, with only the CPU’s PC increasing with its load.
• While considering E2E measurements of PC, we should also take into account HW infrastructure hosting SMO/Non-RT RIC, and rApps.
• The proposed testing methodology using Kepler enables correlating RAN events (O1 PM data reception) with the PC of given VNFs. This creates potential for optimization on the SW side.
• While in this setup, 12 cells were tested, the proposed methodology, along with the RAN emulator, can be used to increase the number of cells in a controlled way and formulate estimations of PC of SMO/RIC in a network of hundreds of cells.
• In addition, the obtained results can be extrapolated to formulate predictions of the PC of country-wide commercial-grade networks.
• In terms of PC, the computational workload related to the processing of PM data can be a potential bottleneck for SMO/Non-RT RIC.
• The proposed testing methodology can be used to compare the benefits and costs of running SMO/Non-RT RIC, and rApps from different vendors, as well as PC between HW setups used for hosting them.

The same Kepler and Redfish Tools can be utilized for EE Testing of the full O-RAN stack of virtualized components, including O-CU, O-DU, and physical O-RU, as well as benchmarking of a general-purpose virtualized environment.

---

References

[1] M. Hoffmann „O-RAN Network Energy Saving: Cell Switching On/Off” Rimedo Labs Blog, [Online], Available: https://rimedolabs.com/blog/o-ran-network-energy-saving-cell-switching-on-off/

[2] M. Hoffmann „O-RAN Network Energy Saving: RF Channel Switching” Rimedo Labs Blog, [Online], Available: https://rimedolabs.com/blog/o-ran-network-energy-saving-rf-channel-switching/

[3] M. Hoffmann, et al. „i14y Energy Efficiency Testing Framework” i14y Lab, Rimedo Labs, White Paper, [Online], Available: https://www.i14y-lab.com/file/show/1233/529344/i14yLab-iEETF-Whitepaper.pdf

[4] N. Sharma, et al. „A comprehensive framework to evaluate energy efficiency in 5G Disaggregated / Open RAN networks.” i14y Lab, Dell Technologies, Rimedo Labs, White Paper, [Online], Available: https://www.i14y-lab.com/file/show/1446/fe4906/Comprehensive_Energy_Efficiency_Testing_Framework.pdf

[5] „i14y E2E Energy Efficiency Testing Framework: Testing VNF power consumption” i14y Lab, Blog, [Online], Available: https://www.i14y-lab.com/article/testing-vnf-power-consumption-a-step-towards-the-i14y-e2e-energy-efficiency-testing-framework

[6] REDFISH – Redfish®, [Online], Available: https://www.dmtf.org/standards/redfish

[7] Understanding Kepler Power Attribution, [Online], Available: https://sustainable-computing.io/kepler/usage/power-attribution/

[8] What is Prometheus?, [Online], Available: https://prometheus.io/docs/introduction/overview/

[9] M. Hoffmann „How can Energy Saving and Traffic Steering cooperate in O-RAN?” Rimedo Labs Blog, [Online], Available: https://rimedolabs.com/blog/how-can-energy-saving-and-traffic-steering-cooperate-in-o-ran/

[10] Grafana OSS and Enterprise, [Online], Available: https://grafana.com/docs/grafana/latest/

[11] Ubuntu Stress Tool, [Online], Available: https://manpages.ubuntu.com/manpages/bionic/man1/stress.1.html#description

---

Related links – i14y Lab Energy Efficiency Testing Framework (iEETF) project

Energy Efficiency Testing Framework – A White Paper

A comprehensive framework to evaluate energy efficiency in 5G Disaggregated/Open RAN networks

i14y E2E Energy Efficiency Testing Framework: Testing VNF power consumption

---

Author Bio