The exponential growth in data volume and variety across various domains has led to significant challenges in efficiently processing, storing, and managing large-scale datasets. Remote sensing, with its increasingly diverse and complex datasets and demanding computational requirements, exemplifies these challenges as a prominent example of big data processing needs. Cloud computing offers a promising solution for addressing the scalability and resource allocation needs of big data processing by providing a distributed environment where resources can be dynamically managed. A typical cloud-based big data processing platform encompasses infrastructure orchestration, distributed processing frameworks, data access mechanisms, and user interfaces. While this approach enables efficient handling of large-scale data, it also raises concerns regarding energy consumption and carbon footprints. This presentation will delve into the proposed methods and tools aimed at optimizing energy consumption for big data processing within a cloud environment, using remote sensing big data as a representative example. The discussion will be organized around three interrelated topics: establishing an energy-aware benchmarking framework, optimizing infrastructure orchestration for energy efficiency, and implementing energy-efficient task scheduling. Firstly, the benchmarking framework includes applications, data, and monitoring toolkits for collecting and analyzing performance, resource utilization, and more importantly, energy metrics within distributed big data systems. By benchmarking these metrics, we can identify key areas for improvement in energy efficiency. Secondly, optimizing infrastructure orchestration involves proposing resource allocation strategies such as automatic scaling of clusters, container consolidation, and prioritizing workloads, considering energy efficiency as the main criterion. These strategies aim to reduce energy consumption without compromising performance as much as possible, allowing for the benefits to be applied across various big data applications without requiring changes to the existing codebase. Thirdly, a multi-objective task scheduling strategy is introduced to minimize energy consumption while maintaining acceptable execution times at the computing task level. The output of this research includes software components specifically designed to be integrated into widely used remote sensing big data platforms to measure and improve the energy efficiency of distributed big data processing. Additionally, the research will engage the broader community through workshops and mini symposia to disseminate the findings and methodologies developed. By focusing on these strategies, we aim to advance the field of big data processing by providing tools that can be adapted across various domains and promote sustainable practices in cloud-based big data applications.

Energy efficiency in computing becomes more and more important as climate change impacts intensify, energy costs increase, and efforts to reach the Sustainable Development Goals to tackle environmental, social and economic issues gain momentum. This is especially true for large-scale computing for research, including cloud computing, which underpin advancements and breakthroughs in many scientific domains. Achieving significant improvements in energy efficiency for such resource-intensive computing tasks, especially artificial intelligence methods including machine learning, deep learning and the new generation of large language models, is a complex challenge that requires a coordinated effort across hardware and software domains, as well as a large group of actors including infrastructure providers, system administrators, software developers, and researchers. Despite the rapid pace of technological development, many existing tools and platforms still lack essential features, such as detailed energy consumption metrics and granular task-level monitoring capabilities. Optimization of research computing workflows considering energy efficiency is also mostly overlooked. This panel discussion seeks to unite a diverse array of experts from the Netherlands, and potentially from the international community, to shed light on the current practices and challenges of energy-efficient computing, with a special focus on research. We aim to explore how energy efficiency is currently addressed within advanced computing, identify critical gaps, and discuss actionable steps to enhance collaboration and drive forward more energy-efficient practices. By fostering this exchange of ideas and experiences of different actors, we hope to contribute to the development of computing infrastructures and workflows that are not only powerful but also environmentally sustainable.

Panelists:
- Prof. Dr. Rob van Nieuwpoort (Leiden University)
- Gilles Tourpe (Amazon Web Services)
- Gijs van den Oord (Netherlands eScience Centre)
- Sagar Dolas (SURF)
- Adhitya Adhitya Bhawiyuga (University of Twente)