diff options
Diffstat (limited to 'content')
| -rw-r--r-- | content/background.tex | 89 | ||||
| -rw-r--r-- | content/intro.tex | 6 |
2 files changed, 52 insertions, 43 deletions
diff --git a/content/background.tex b/content/background.tex index ab361f2..2f9cb2a 100644 --- a/content/background.tex +++ b/content/background.tex @@ -1,28 +1,26 @@ \chapter{Background}\label{s:background} -\section{Overview}\label{s:background_overview} - \section{Datacenters}\label{ss:datacenters} +In this section we provide a short background on datacenters, datacenter simulation and hardware failures. + +Datacenters are important building blocks of the digital society~\cite{DBLP:journals/corr/IosupKLVG22}. +Reliable warehouse management is a key priority for datacenter operators, because incorrect decisions can lead to missed \gls{sla}s. +However, reliable and timely management is a difficult challenge, because datacenters are extremely complex facilities. +To help datacenter operators, the scientific community proposes to simulate datacenter operations to make more informed operational decisions. \subsection{Datacenter Simulation}\label{sss:simulation} \input{sources/simulator_comparison.tex} +Simulation is an excellent way to help design, test and manage datacenters. +To simulate is to ``imitate of real-world process or system over time, enabling the study of, and experimentation with the internal interactions of complex systems''~\cite{DBLP:books/daglib/0034857} +Alternatives to simulation include real-world experimentation and mathematical analysis. +However, experimentation \emph{in situ} is unsustainable, and mathematical analysis is not scalable to modern datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +A good datacenter simulator can estimate the results of running a months long workload in a matter of minutes or hours. +The scientific community has developed several datacenter simulators (see \Cref{tab:datacenter_simulator_comparison}). +In our work, we decided to use \code{OpenDC}, because we find it important for a simulator to model hardware failures well. +\emph{Failure models} are a carefully calibrated, advanced feature of \code{OpenDC}. \subsection{Hardware Failures}\label{sss:failures} -Predictive modelling uses statistics to predict outcomes. -When deployed commercially, for example in datacenters, predictive modelling is often referred to as predictive analytics~\cite{Wikipedia:PredictiveModelling}. -Almost any statistical model can be used for prediction purposes, but nowadays predictive analysis is synonymous with machine learning. -A primary example of popular analysis type is linear regression. -A major limitation of predictive analytics is that history cannot always predict the future. -Using historical data to predict outcomes works only under the assumption that there are certain long lasting patterns in the system. -Additionally, no matter how extensive is the training data, there is always the possibility of new variables that have not been considered or even defined, yet are critical to the outcome of the prediction~\cite{Wikipedia:PredictiveModelling}. - -%Here you have to cite Deisenroth, 2024, chapter 8.1.4. -An inference function is a machine learning model which uses probabilistic parameter estimation~\cite{}. -A prime example of using probability to find a good machine learning model is Bayesian inference. -% Stanford Encyclopedia of Philosophy, Douven 2017 -The process of inference from data to provide the best explanation is called abduction. - %What is below here is true, but nonetheless the argumentation should be slightly changed. And a citation is needed. However, there has been little effort made to integrate analytics that enable consistent and reliable prediction of datacenter behaviour into a holistic digital twin of a datacenter. Nor has the fidelity of failure modeling inside a datacenter simulation increased. @@ -34,7 +32,17 @@ Since a digital twin is not a standalone simulator, a change to how we both pred \section{Digital Twinning}\label{ss:digital-twinning} % To fix: remove the \gls commands for ExaDigiT. % This is getting silly. +In this section we explore how the datacenter management can be improved using a novel modelling technique, digital twinning. +We present the generic, field-agnostic \gls{dt} definition and investigate how \emph{datacenter} digital twinning applies the definition in practice. + \subsection{What is Digital Twinning?}\label{sss:what_is_digital_twinning} +\begin{figure}[t] + \centering + \includegraphics[width=0.95\linewidth]{images/five_dimensional_dt.pdf} + \caption{A basic framework for the \gls{dt}. Four core elements of a \gls{dt} are defined: The physical entity (\myCircled{1}) and the simulated virtual twin (\myCircled{2}). A service for out-of-band data analytics (\myCircled{3}) and a persistent storage of historical data (\myCircled{4}) are crucial to the \gls{dt} because they are necessary to gain meaningful monitoring insights. Adapted from Tao \etal ~\cite{DBLP:conf/cirp/TAO2018169}.} + %Fei Tao is a renowned figure with over 62k citations. He is a figure of authority on digital twins.% + \label{fig:five_dimensional_dt} +\end{figure} % Here talk a bit about different types of data analytics that are performed in a digital twin. ``A \emph{digital twin} is a set of virtual information constructs that mimics the structure, context and behaviour of a natural, engineered or social system, is dynamically updated with data from its physical twin, has predictive capability, and informs decisions that realize value''~\cite{DBLP:usdoe/report/AP26894}. A crucial characteristic that differentiates digital twinning from simulation and statistical modelling is the \emph{digital thread}: a bi-directional channel that enables continuous interaction between the virtual and physical entities. @@ -48,13 +56,22 @@ As such, while the concept existed, the hardware did not catch up yet. However, in the last decade, multicore computing paradigms and the advent of GPU computing has finally enabled computation needed to run digital twins. As a result, digital twins have become more relevant today than 10 years ago~\cite{DBLP:conf/cirp/TAO2018169}. -\begin{figure} - \centering - \includegraphics[width=0.95\linewidth]{images/five_dimensional_dt.pdf} - \caption{A basic framework for the \gls{dt}. Four core elements of a \gls{dt} are defined: The physical entity (\myCircled{1}) and the simulated virtual twin (\myCircled{2}). A service for out-of-band data analytics (\myCircled{3}) and a persistent storage of historical data (\myCircled{4}) are crucial to the \gls{dt} because they are necessary to gain meaningful monitoring insights. Adapted from Tao \etal ~\cite{DBLP:conf/cirp/TAO2018169}.} - %Fei Tao is a renowned figure with over 62k citations. He is a figure of authority on digital twins.% - \label{fig:five_dimensional_dt} -\end{figure} +A crucial part any of any \gls{dt} should be \emph{predictive modelling}, which drives actionable insights~\cite{DBLP:usdoe/report/AP26894}. +Predictive modelling uses statistics to predict outcomes. +When deployed commercially, for example in datacenters, predictive modelling is often referred to as predictive analytics~\cite{Wikipedia:PredictiveModelling}. +Almost any statistical model can be used for prediction purposes, but nowadays predictive analysis is synonymous with machine learning. +A primary example of popular analysis type is linear regression. +However, any modelling technique, \eg \emph{discrete-event simulation} can be used to make the predictions. + +A major limitation of predictive analytics is that history cannot always predict the future. +Using historical data to predict outcomes works only under the assumption that there are certain long lasting patterns in the system. +Additionally, no matter how extensive is the training data, there is always the possibility of new variables that have not been considered or even defined, yet are critical to the outcome of the prediction~\cite{Wikipedia:PredictiveModelling}. + +%Here you have to cite Deisenroth, 2024, chapter 8.1.4. +%An inference function is a machine learning model which uses probabilistic parameter estimation~\cite{}. +%A prime example of using probability to find a good machine learning model is Bayesian inference. +% Stanford Encyclopedia of Philosophy, Douven 2017 +%The process of inference from data to provide the best explanation is called abduction. % (3) in the original paper by Fei Tao is referenced to just `Services`. % Nonetheless I name them here as Data Analysis Services, because what Fei Tao lists (e.g., fault detection, fault determination, fault-tolerant management, maintenance) is inherently reliant on good data analytics. %\subsection{Digital Twins across Domains}\label{sss:digital_twins_across_domains} @@ -101,15 +118,6 @@ The digital twin is designed to provide extra datasets for training \gls{ai} mod % At this point it would make sense to create the distinction between _structural_ digital twinning and _behavioural_ digital twinning. % Link to 6SigmaDC: https://www5.cadence.com/trial_datacenter_insights_lp.html -\begin{figure}[t] - \centering - \includegraphics[width=\linewidth]{images/system_model.pdf} - \caption{A generic system model for data center digital twin deployments. - The design of DyTwin~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24} indirectly incorporates in its architecture a ``virtual-to-virtual`` digital thread between different digital twins. - Zhao \etal likewise present key elements to the digital thread in their architecture~\cite{DBLP:conf/AppliedEnergy/Zhao20}. We add the \emph{Digital Thread} to our model explicitly.} - \label{fig:system_model} -\end{figure} - DyTwin~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24} is an adaptive digital twin with visualization and anomaly detection features. The system, developed at \gls{hp} is a precursor to the vision on datacenter digital twinning published by Athavale \etal~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. DyTwin is the only system capable of failure detection in datacenters. @@ -139,12 +147,22 @@ Kalibre~\cite{DBLP:conf/sensys/WangZD0TCWZ20} is a system designed by Wang \etal To lessen the computational overhead, Wang \etal propose to use a knowledge-based neural surrogate to calibrate the different \gls{cfd} models. Kalibre takes the best of both \gls{ml} and \gls{cfd} approaches and achieves sub 1 degree Celsius \gls{mae}, similarly to Reducio~\cite{DBLP:conf/sensys/CaoW0022}. - - % What is more, Microsoft already offers digital twinning as a service https://azure.microsoft.com/en-us/products/digital-twins/ % Documentation: https://learn.microsoft.com/en-us/azure/digital-twins/ % Moreover, NVIDIA is doing too as well https://www.nvidia.com/en-sg/omniverse/ +\subsection{System Model for Datacenter Digital Twinning} +\label{ss:system_model_for_dcdts} + +\begin{figure}[t] + \centering + \includegraphics[width=\linewidth]{images/system_model.pdf} + \caption{A generic system model for data center digital twin deployments. + The design of DyTwin~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24} indirectly incorporates in its architecture a ``virtual-to-virtual`` digital thread between different digital twins. + Zhao \etal likewise present key elements to the digital thread in their architecture~\cite{DBLP:conf/AppliedEnergy/Zhao20}. We add the \emph{Digital Thread} to our model explicitly.} + \label{fig:system_model} +\end{figure} + To summarize, many \gls{dcdt}'s model the cooling systems inside the warehouse, because in a typical datacenter cooling accounts for more than 40\% of total electricity usage~\cite{DBLP:conf/AppliedEnergy/Zhao20}. Since the cooling subsystem is mainly airflow-based, \gls{dt} designers often opt for a \gls{cfd} approach to model the facility. The reason why a digital twin might be needed for a cooling subsystem is primarily because of inefficient operational strategy. @@ -169,6 +187,3 @@ The cooling system parameters are often set constant, regardless of outdoor temp %In your work, consider adding such an endpoint, albeit explain in future work that you envision \emph{implementing} this endpoint in the future. - - - diff --git a/content/intro.tex b/content/intro.tex index 31feecc..36703f9 100644 --- a/content/intro.tex +++ b/content/intro.tex @@ -168,12 +168,6 @@ In order to adhere to the strictest restrictions on AI-usage in higher education I hereby declare that all the references in this thesis refer to genuine scientific work published in peer-reviewed journals or other sources of reliable and safe online information (\eg Wikipedia articles) and have been used in accordance to the article authors' wishes. Additionally, under the guidance of the supervisor this work adheres to the strictest rules for referencing and to prove the originality of all references, each \BibTeX citation contains a \texttt{note} field with the following comment: \emph{This BibTeX citation comes from:} followed by the URL leading directly to the citation source. In case of citations not formatted in \BibTeX, the same format follows but with adequate reference-style name (\eg APA, Chicago, MLA). -In order to verify the originality of a reference we advise: \begin{enumerate*}[label=(\arabic*)] - \item create a \LaTeX~document with the \texttt{is-unstr} bibliography style - \item export the \BibTeX citation from the reference URL - \item inspect the compiled reference (ignoring any differences arising from the contents of the \texttt{note} \BibTeX field). -\end{enumerate*} -The compiled reference should match exactly the reference from this document, subject to the use of different \LaTeX~packages (\eg \texttt{url}, \texttt{natbib}). \section{Societal Impact}\label{s:societal-impact} Any program that is difficult to understand and reason about is sure to accumulate technical debt. |