diff options
Diffstat (limited to 'content/intro.tex')
| -rw-r--r-- | content/intro.tex | 119 |
1 files changed, 77 insertions, 42 deletions
diff --git a/content/intro.tex b/content/intro.tex index 69322d4..85ae205 100644 --- a/content/intro.tex +++ b/content/intro.tex @@ -1,7 +1,9 @@ \chapter{Introduction}\label{s:intro} Currently, computer and network systems play a crucial part in the digital industry. -The transport, education and government sectors largely depend on digital services, which are hosted in datacenters~\cite{DBLP:journals/corr/IosupKLVG22}. -To address the recent rise in demand for computation, due to the advancements in Artificial Intelligence, managers expand datacenters with new components and more heterogeneous architectures (\eg GPUs, NPUs)~\cite{DBLP:conf/date/MilojicicFDR21}. +For example, the transport, education and government sectors largely depend on digital services, which are hosted in datacenters~\cite{DBLP:journals/corr/IosupKLVG22}. +Datacenters are complex facilities, housing thousands of servers. +Moreover, the advancements in Artificial Intelligence have sped up data datacenter expansion. +To address the recent rise in demand for computation, datacenter managers add new components and more heterogeneous architectures (\eg GPUs, NPUs)~\cite{DBLP:conf/date/MilojicicFDR21} to the already complex warehouses. However, in return datacenter complexity increases significantly. To make better operational decisions despite the massive scale, promising technologies arise such as Datacenter Digital Twins~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. @@ -12,50 +14,60 @@ Since many public services continue to move online (\eg online administration an % What is changing? In the modern \gls{ai} economy, datacenters need diverse and scalable server architectures, because inference-based workloads require more heterogeneous server components (GPUs, TPUs, NPUs \etc) to perform well. -Nowadays, datacenter operators try to meet AI-customer expectations by adding more specialized hardware~\cite{DBLP:conf/date/MilojicicFDR21}, at the cost of increased system complexity. +%Nowadays, datacenter operators try to meet AI-customer expectations by adding more specialized hardware~\cite{DBLP:conf/date/MilojicicFDR21}, at the cost of increased system complexity. In return, operating a modern datacenter warehouse with thousands of diversified servers presents a difficult challenge that requires fast and well-informed decisions from on-site engineers. -The computational requirements of \gls{ai} are expected to increase in the future~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. -Datacenter complexity will continue to grow, and it will become more difficult to manage~\cite{DBLP:conf/icdcs/IosupUVAEHTBT18}. +The computational requirements of \gls{ai} are expected to increase in the future. +\Cref{fig:explosive_growth_in_ai_computational_requirements} presents the speed and scale which drives the compute requirements of \gls{ai} models. +From 2012 to 2024, the compute power needed to train the models grew by $10^{7}$, a trend that is expected to continue~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. +Because of this, datacenter complexity will continue to grow, and it will become more difficult to manage~\cite{DBLP:conf/icdcs/IosupUVAEHTBT18}. Future servers and software related services to them will include even more specialized hardware, which, while improving datacenter performance, will exhibit behaviour that is harder to predict. Already the rapid expansion of datacenters has increased the presence of service failures across all cloud services~\cite{DBLP:conf/acsos/TalluriOVTI21}. Preventing failure-caused outages in advance could help datacenter operators reduce operational costs, as over 20\% of all reported outages amount to more than 1 million US\$~\cite{DBLP:report/AnnualOutageAnalysis2025}. %Moreover, datacenter outages can have catastrophic consequences, cite Fabian. -In short, the high computational demand of \gls{ai} and the end of Dennard's scaling have resulted in the rise of larger and more heterogeneous datacenter architectures~\cite{DBLP:conf/date/MilojicicFDR21}. +%In short, the +The high computational demand of \gls{ai} and the end of Dennard's scaling have resulted in the rise of larger and more heterogeneous datacenter architectures~\cite{DBLP:conf/date/MilojicicFDR21}. Both events create a need for more careful datacenter management to tackle the unprecedented complexity and ensure availability of all cloud services. To address this new problem a concept of a datacenter \gls{dt} was proposed~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. +% Explain DT here, very briefly +A \gls{dt} is a virtual model of an intended or actual real-world system that serves as its counterpart for purposes such as simulation, integration, testing, monitoring and maintenance. +The digital twin replicates the physical system to predict failures, prescribe real-time actions for mitigating unexpected events, observing and evaluating the behaviour of the system~\cite{Wikipedia:article/DigitalTwin}. +% +This thesis explores the use of digital twins to cope with increasing operational complexity in datacenters, by enabling both historical and online data to pass through the digital twin's analytical and predictive loops. -\begin{figure} +\begin{figure}[t] + \hspace{-1.75cm} \centering - \includegraphics[width=0.8\linewidth]{images/simple_dt.pdf} - \caption[Elements of the digital twin ecosystem.]{Elements of the digital twin ecosystem~\cite{DBLP:modsim24/presentation/Iosup2024} include: the insights and decisions coming from the digital twin (\myCircled{A}), the physical infrastructure (\myCircled{B}), the data coming from the physical twin telemetry (\myCircled{C}), and the digital counterpart to the physical twin (\myCircled{D}). - This thesis focuses on components (\myCircled{A}), (\myCircled{C}), and (\myCircled{D}) in this ecosystem, proposing design improvements to (\myCircled{D}, \myCircled{C}), and the feedback loop (\myCircled{A}). - } - \label{fig:simple_dt} -\end{figure} + \includegraphics[width=0.8\linewidth]{images/ai_timeline.pdf} + \caption[Explosive growth in AI computational requirements.]{Explosive growth in AI computational requirements drives datacenter upgrades (source: NVIDIA Analysis: reproduction with NVIDIA permission by~\cite{DBLP:journals/computer/AthavaleBBMMPS24}). \emph{X}-axis presents the year, and the \emph{Y}-axis presents the training compute requirement (in petaFLOPs). The plot shows the one of the reasons behind why the complexity of datacenters is rising.} + \label{fig:explosive_growth_in_ai_computational_requirements} +\end{figure} \section{Context}\label{s:context} % A digital twin is often called a virtual twin. % The communication between a physical entity and the digital twin is referred to as a digital thread. -A \gls{dt} is a virtual model of an intended or actual real-world system that serves as its counterpart for purposes such as simulation, integration, testing, monitoring and maintenance. -The digital twin replicates the physical system to predict failures, prescribe real-time actions for mitigating unexpected events, observing and evaluating the behaviour of the system~\cite{WIKI:page/DigitalTwin}. +% +% A \gls{dt} is a virtual model of an intended or actual real-world system that serves as its counterpart for purposes such as simulation, integration, testing, monitoring and maintenance. +% The digital twin replicates the physical system to predict failures, prescribe real-time actions for mitigating unexpected events, observing and evaluating the behaviour of the system~\cite{Wikipedia:article/DigitalTwin}. -Most modern \gls{dt} usages are related to prognostics and system health management~\cite{DBLP:conf/cirp/TAO2018169}. +Much modern \gls{dt} usages are related to prognostics and system health management~\cite{DBLP:conf/cirp/TAO2018169}. For example, in aerospace engineering, the \gls{dt} analyzes operational data (\eg temperature, vibration) to predict when a airplane component is likely to fail. The \gls{dt} can reliably manage the health of the physical entity by detecting fatigue cracks on aircraft wings or damage to the wind turbine blades~\cite{DBLP:journal/IJAE/Teugel2011}. This allows maintenance to be scheduled proactively, reducing unplanned downtime and preventing catastrophic failures. Forecasting future maintenance and managing the physical health of an object or facility are the prime purpose of many \gls{dt}s used in practice~\cite{DBLP:conf/AIAA/Teugel2012}. -The first mention of a \gls{dt} dates back to 2003, when Dr. Michael Grieves of Dassault Syst\'emes introduced the 3 core components of a \gls{dt}: the virtual entity, physical entity and the two-way connection (see Figure \ref{fig:simple_dt}). +The concept of a \gls{dt} began in 1960s, at the \gls{nasa}~\cite{Nature:article/Görtz2026}. +\gls{nasa} pioneered the concept to debug issues with its spacecraft. +However, the term ``digital-twin'' dates back to 2003, when Dr. Michael Grieves of Dassault Syst\'emes introduced the 3 core components of a \gls{dt}: the virtual entity, physical entity and the two-way connection (see Figure \ref{fig:simple_dt}). Due to insufficient technological foundations, little work is available on \gls{dt}s between 2003 and 2018, and it is only with the rapid growth of cloud computing, \gls{iot} and Big Data analytics that \gls{dt}s have re-emerged. Today, research is focused on bridging the gap between the long-established foundations of \gls{dt}s and new, novel applications in academia and industry, such as the \gls{dcdt}~\cite{DBLP:conf/cirp/TAO2018169, DBLP:journals/computer/AthavaleBBMMPS24}. -A \gls{dcdt} mirrors the structure, context and behaviour of a datacenter~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. +A \gls{dcdt} mirrors the structure, context, and behaviour of a datacenter~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. The foundation to any digital twin is good monitoring and sensing capabilities in the physical entity. Datacenters, meet this requirement easily because they already connect hundreds of monitoring sensors. -With hundreds of gigabytes of useful information coming from distributed \gls{iot} sensors inside the warehouse, we can gain insight into failure patterns, energy usage, heat dissipation \etc +With hundreds of gigabytes of useful information coming from distributed \gls{iot} sensors inside the warehouse, we can gain insight into failure patterns, energy usage, heat dissipation, \etc What remains challenging is to connect the physical and virtual spaces with a bi-directional connection and to use the monitoring insights and data analysis results for autonomous decision-making. Crucial to \gls{dcdt} operation are predictive capabilities and the continuous interaction with the real-world datacenter. @@ -63,7 +75,7 @@ Crucial to \gls{dcdt} operation are predictive capabilities and the continuous i There already exist \gls{dcdt} deployments. For example, ExaDigiT~\cite{DBLP:conf/sc/BrewerMKWBHSGGW24} is a framework for digital twin development of supercomputers. It has been demonstrated at the Frontier supercomputer and it facilitates virtual prototyping and system optimization. - +% Nonetheless, existing \gls{dcdt}'s are still very limited in their capabilities as the definition and scope of a \gls{dcdt} concept is shallow and unclear. After all, only recently did the hardware capabilities needed to continuously simulate a datacenter become available~\cite{DBLP:conf/cirp/TAO2018169}. Many \gls{dcdt} frameworks still lack critical data analysis components, fault detection mechanisms, profiling techniques \etc~\cite{DBLP:conf/wosp/SumanCNTMI24}, rendering them unusable in large-scale systems. @@ -74,33 +86,47 @@ Downtime, which is the result of failures, disturbs the users and produces unful % DT's merge both simulation and telemetry to develop a holistic virtual representation of the system, bridging both the physical and virtual worlds. However, predicting datacenter behaviour quickly and reliably is a non-trivial problem that remains insufficiently unaddressed in the existing \gls{dcdt} architectures ~\cite{DBLP:conf/wosp/SumanCNTMI24, DBLP:journals/computer/AthavaleBBMMPS24} and deployments~\cite{DBLP:conf/sc/BrewerMKWBHSGGW24}. +\begin{figure}[t] + \centering + \includegraphics[width=0.8\linewidth]{images/simple_dt.pdf} + \caption[Elements of the digital twin ecosystem.]{Elements of the digital twin ecosystem~\cite{DBLP:modsim24/presentation/Iosup2024} include: the insights and decisions coming from the digital twin (\myCircled{A}), the physical infrastructure (\myCircled{B}), the data coming from the physical twin telemetry (\myCircled{C}), and the digital counterpart to the physical twin (\myCircled{D}). + This thesis focuses on components (\myCircled{A}), (\myCircled{C}), and (\myCircled{D}) in this ecosystem, proposing design improvements to (\myCircled{D}, \myCircled{C}), and the feedback loop (\myCircled{A}). + } + \label{fig:simple_dt} +\end{figure} \section{Problem statement}\label{s:problem-statement} -We envision \gls{dcdt}'s as systems indispensable in future datacenters, actively interacting with the real-world facility, lowering operational costs and predicting hardware failure and software faults. In this work, we address the lack of a unified \gls{dcdt} system model and the absence of predictive capabilities in existing \gls{dcdt} system designs. We argue that the current state-of-the-art \gls{dcdt}'s lack sufficient predictive capabilities that are essential to real-time facility management of a modern datacenter. -A \gls{dt} without predictive capabilities cannot maintain the health of the datacenter effectively. +Because the main purpose of many \gls{dt}s is to forecast the short and long-term facility behaviour, \gls{dcdt} without predictive capabilities cannot maintain the health of the datacenter effectively. We posit that including holistic predictive analysis in \gls{dcdt} design can aid in efficient datacenter management and prevent missing \gls{sla}'s. -We propose that digital twinning can be enhanced by integrating predictive analytics through \gls{oda}. +% +For example, preventing compute failures could greatly benefit datacenter operators. + + +%To enable insights from both historical data and immediate telemetry, we propose that digital twinning can be enhanced by integrating predictive analytics through \gls{oda}. +We envision \gls{dcdt}'s as systems indispensable in future datacenters, actively interacting with the real-world facility, lowering operational costs and predicting hardware failure and software faults. +In particular, to enable insights from both historical data and immediate telemetry, we propose that digital twinning can be enhanced by integrating predictive analytics through \gls{oda}. +Our solution to this problem encompasses different levels of \gls{oda} (\eg in-band analytics, out-of-band analytics) for holistic datacenter modelling. +Together with a unified \gls{dcdt} system model and a revolutionary evaluation method of \gls{dcdt}'s, we hope to bring the modern vision of \gls{dt}'s to datacenters. \section{Research Questions}\label{s:research-questions} \emph{Main Research Question:} How to enable predictive analytics in datacenters through digital twinning?\\ -\noindent We divide the problem of designing a predictive \gls{dcdt} into three research questions: +\noindent We scope the problem of designing a predictive \gls{dcdt} by proposing three research questions: \begin{enumerate}[label=\emph{RQ\textsubscript{\arabic*}}, align=left, itemsep=0pt] % First research question stolen from Capelin by Georgios Andreadis and adapted to my work. \item \emph{How to assess the current state-of-the-art of digital twinning for datacenters?}\\ There is currently a lack of a unified system model of what constitutes a \gls{dcdt}, and the differences between existing \gls{dcdt} deployments. - Thus, it is necessary that we establish a common model of a \gls{dcdt} in the research community. - We must develop a holistic \gls{dcdt} model that factors in the necessary components of a \gls{dt}. - This is very challenging, because the \gls{dcdt} system model must address many kinds of operational and technical requirements, compatible with the existing background on \gls{dt}s. + To alleviate this problem, we aim to develop a holistic \gls{dcdt} model that factors in the necessary components of a \gls{dt}. + This is a challenging, because the \gls{dcdt} system model must address many kinds of operational and technical requirements, compatible with the existing background on \gls{dt}s. \item \emph{How to design a \gls{dcdt} reference architecture using discrete-event simulation and predictive data analysis?}\\ % You should start referring to my_system as a framework, rather than a standalone system. Existing \gls{dcdt} frameworks lack the necessary predictive capabilities to prevent unplanned behaviour in datacenters~\cite{DBLP:conf/wosp/SumanCNTMI24, DBLP:conf/sc/BrewerMKWBHSGGW24, DBLP:conf/sc/TaheriBPRHDEWPM24, DBLP:journals/computer/AthavaleBBMMPS24}. In this work, we aim to explore the design space of a predictive \gls{dcdt} and the different design trade-offs. Through discrete-event simulation, we aim provide the foundation for the system to interact with a physical datacenter. - This is a very challenging task, because there are many functional and non-functional requirements of a \gls{dcdt} that need careful consideration. + What makes the task challenging are the many functional and non-functional requirements of a \gls{dcdt} that need careful consideration. The architecture must comply with the generic \gls{dt} model and address the non-trivial challenges in operating a modern datacenter. \item \emph{How to evaluate and validate a \gls{dcdt} reference architecture in relation to system requirements}?\\ To understand the operation of the proposed system and whether it meets its design goals we need to measure it's performance. @@ -111,9 +137,9 @@ We propose that digital twinning can be enhanced by integrating predictive analy \section{Research Methodology}\label{s:research-methodology} To answer \emph{RQ\textsubscript{1}} we conduct a literature review as proposed by \textit{Kitchenham et al.} \cite{DBLP:journals/infsof/KitchenhamPBBTNL10} along with the guidance of the supervisor. Firstly, we determine the right review method. -Secondly, we identify the various works related to \gls{dcdt}'s using different search strings +Secondly, we identify the various works related to \gls{dcdt}'s using different search strings~ (\eg ``Datacenter Digital Twinning'', ``ICT Virtual Twin'') and query combinations (``Datacenter \code{AND} Maintenance''). -To search for the results we use the digital libraries of Google Scholar, DBLP, ACM Digital Library, IEEExplore, Springer \etc +To search for the results we use the digital libraries of Google Scholar, DBLP, ACM Digital Library, IEEExplore, Springer, \etc Thirdly, we select work relevant to our research and organize the details of each article into a table. A potential outcome of this could be a system model for \gls{dcdt}'s. We envision the literature review supplying us with potential use-cases for the predictive \gls{dcdt}. @@ -132,35 +158,41 @@ We first gather a set of questions worth asking about the performance and impact We define the correct experiment setup(s) and perform the experiments on a specified hardware, considering different usage scenarios. \section{Thesis Contributions}\label{s:thesis-contributions} -\begin{enumerate} +\begin{enumerate}[align=left, labelsep=1pt] \item \textbf{Conceptual}: \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}, align=left, labelsep=0pt] \item We conduct a comprehensive literature review and detailed analysis of existing works on digital twinning in the scientific research community. We collect and organize the \gls{dcdt}'s characteristics and based on our findings we propose a unified system model of the design space. - \item We propose the design of \gls{my_system}, a discrete-event \gls{dcdt} for reliable and timely failure prediction in datacenters. - \gls{my_system} includes a set of novel system components which leverage \gls{oda} and discrete-event simulation. + \item We propose the design of \mysystem, a discrete-event \gls{dcdt} for reliable and timely failure prediction in datacenters. + \mysystem includes a set of novel system components which leverage \gls{oda} and discrete-event simulation. - \item We evaluate \gls{my_system} using a novel experimentation technique and datacenter workload traces from the industry. + \item We evaluate \mysystem using a novel experimentation technique and datacenter workload traces from the industry. We design a method to evaluate \gls{dcdt}s without expensive and costly real-world experimentation. We conduct a set of exhaustive experiments and analyse the results. \end{enumerate} \item \textbf{Technical:} \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}, align=left, labelsep=0pt] - \item We prototype \gls{my_system} following the established \gls{dt} design principles using discrete-event simulation and \gls{oda}. + \item We prototype \mysystem following the established \gls{dt} design principles using discrete-event simulation and \gls{oda}. We include the code as an Open Science artifact and ensure the prototype remains accessible to the broader scientific community including detailed project documentation. - \item We provide the experiment setup, validation and evaluation of \gls{my_system} for detecting and predicting datacenter failures in real-time as an Open Science artifact. + \item We provide the experiment setup, validation and evaluation of \mysystem for detecting and predicting datacenter failures in real-time as an Open Science artifact. \end{enumerate} \end{enumerate} + \section{Academic Integrity Declaration}\label{s:academic_integrity_declaration} + +The statements in this section address an ongoing and seemingly growing problem in academia, that of integrity breaches primarily through the generation of material using AI techniques, such as LLM services. + \subsection{Non-Plagiarism Declaration}\label{ss:plagiarism-declaraion} I hereby declare that this thesis is my own independent work and writing. The thesis does not contain any material copied from other sources (person, Internet, or \gls{ai}), and has not been submitted for assessment elsewhere. I acknowledge that the usage of material from other works or paraphrase of such material without proper citations or credit will be treated as plagiarism. I declare that this thesis is free from \gls{ai} generated content and has been written without the help of any \gls{ai} tools. -To order to adhere to the strictest restrictions on AI-usage in higher education, this work follows the Berkley School of Law Artificial Intelligence Policy, as stated in \url{https://www.law.berkeley.edu/wp-content/uploads/2026/05/AI-Final-Policy-26.pdf}. + +To adhere to the current (strictest) restrictions on AI-usage in higher education, this work follows the Berkley School of Law Artificial Intelligence Policy, as stated in \url{https://www.law.berkeley.edu/wp-content/uploads/2026/05/AI-Final-Policy-26.pdf}. \subsection{Preventing Reference Fraud}\label{ss:plagiarism_references} -I hereby declare that all the references in this thesis refer to genuine scientific work published in peer-reviewed journals or other sources of reliable and safe online information (\eg Wikipedia articles) and have been used in accordance to the article authors' wishes. +I hereby declare that all the references in this thesis refer to genuine scientific work published in peer-reviewed journals or other sources of reliable and safe online information~ (\eg Wikipedia articles) and have been used in accordance to the article authors' wishes. + Additionally, under the guidance of the supervisor this work adheres to the strictest rules for referencing and to prove the originality of all references, each \BibTeX citation contains a \texttt{note} field with the following comment: \emph{This BibTeX citation comes from:} followed by the URL leading directly to the citation source. In case of citations not formatted in \BibTeX, the same format follows but with adequate reference-style name (\eg APA, Chicago, MLA). @@ -177,14 +209,18 @@ This work addresses the four grand societal challenges related to this goal: \be \item sustainability \item usability \end{enumerate*}~\cite{DBLP:journals/corr/IosupKLVG22}. -\gls{my_system} addresses (1) directly by making large-scale datacenter management easier. + +\mysystem addresses goal (1) directly by making large-scale datacenter management easier. + We address (2) by ensuring our work adheres to the \gls{fair} principles of Open Science. Moreover, in this thesis we try to make \gls{dcdt} systems more understandable to the broader scientific community by providing a unified system model. Additionally, we contribute to responsible software design by adhering to best software engineering practices in the design of the prototype. -(3) is addressed indirectly, as the consequences of the insights provided by a holistic, \gls{oda} powered \gls{dcdt} can help datacenter managers make decisions that are more sustainable in the future. + +Goal (3) is addressed indirectly, as the consequences of the insights provided by a holistic, \gls{oda} powered \gls{dcdt} can help datacenter managers make decisions that are more sustainable in the future. + We contribute to (4) by helping predict unexpected failures and lowering operational costs, ensuring datacenters can continue to be usable in the future. We believe this work has a strong societal impact due to addressing the four grand societal challenges described by Iosup \etal and we hope through this work we can advance the scientific research community towards a more sustainable future. -Abiding the FAIR data principles, the entire source code of the prototype and related work has been made available at the \url{https://git.denounce.ai/opendc.git} repository. +Abiding the FAIR data principles, the entire source code of the prototype and related work has been made available at the \url{https://github.com/M-J-Kwiatkowski/opendc} and the \url{https://github.com/M-J-Kwiatkowski/sunfish} repositories. The reuse and reproduction of experiments is explained in a detailed guide at the root of the repository, along with the necessary dependencies and experimental setup. % Comment from Alexandru is to merge the above repository into the AtLarge repository, with the proper attribution to you. % I will do this once the project is over. @@ -196,7 +232,6 @@ In Chapter \ref{s:design}, we present the design of \mysystem. In \Cref{s:implementation} we present the technical details of \mysystem prototype. In Chapter \ref{s:evaluation} we evaluate the prototype of the system and validate it against the set of functional and non-functional requirements. In Chapter \ref{s:conclusion} we conclude the thesis with a summary of contributions and potential future work. -\newpage \begin{figure}[t!] \centering \includegraphics[width=\linewidth]{images/thesis_structure.png} |