diff options
Diffstat (limited to 'content/background.tex')
| -rw-r--r-- | content/background.tex | 73 |
1 files changed, 37 insertions, 36 deletions
diff --git a/content/background.tex b/content/background.tex index 97dc6aa..74083e5 100644 --- a/content/background.tex +++ b/content/background.tex @@ -1,12 +1,12 @@ \chapter{Background}\label{s:background} \section{Overview}\label{ss:background_overview} \begin{mynote} - The contribution in this chapter is three-fold: + Addressing RQ$_1$, the contribution in this chapter is three-fold: \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}, itemsep=0.2pt] - \item We provide a brief overview on datacenters (\Cref{ss:datacenters}) datacenter simulation (\Cref{sss:simulation}), compute failures (\Cref{sss:failures}), and digital twinning (\Cref{ss:digital-twinning}). - \item We survey the state-of-the-art concerning datacenter digital twinning (\Cref{sss:advanced_dts}). - \item We construct a system model for existing datacenter digital twins (\Cref{ss:system_model_for_dcdts}) + \item We provide a brief overview on datacenters (\Cref{ss:datacenters}) datacenter simulation (\Cref{sss:simulation}), compute failures (\Cref{sss:failures}), and digital twinning~ (\Cref{ss:digital-twinning}). + \item We survey the state-of-the-art concerning datacenter digital twinning~ (\Cref{sss:advanced_dts}). + \item We propose a system model (architecture) for datacenter digital twins, including a novel Digital Thread component for the operational needs linking the digital with the physical worlds~(\Cref{ss:system_model_for_dcdts}). \end{enumerate} \end{mynote} @@ -28,7 +28,7 @@ An example datacenter in \gls{cern}, is depicted on \Cref{fig:datacenter}. \begin{figure}[t] \centering \includegraphics[width=0.9\linewidth]{images/datacenter.jpg} - \caption[Datacenter in CERN.]{Example of a datacenter in \gls{cern}, Switzerland (2010)~\cite{Wikipedia:article/Datacenter}. In the figure we can see servers within servers racks, and the network cables.} + \caption[Datacenter in CERN.]{Example of a datacenter in \gls{cern}, Switzerland (2010)~\cite{Wikipedia:article/Datacenter}. The figure depicts servers within servers racks, and network cables interconnecting them.} \label{fig:datacenter} \end{figure} @@ -47,31 +47,7 @@ Making sure that all the parts of the datacenter work together is a tough task. What drives datacenter complexity even further is that sophisticated systems are not merely a sum of their parts~\cite{Wikipedia:article/Systems_Thinking}. The combination of the above factors makes datacenter management a difficult, non-trivial challenge. -\subsection{Datacenter Simulation}\label{sss:simulation} -\input{sources/simulator_comparison.tex} - -Efficient and timely datacenter management is a difficult challenge, because datacenters are extremely complex facilities. -They require deep understanding to operate properly. -However, running real-world experiments is costly in both time and resources. -Additionally, experimentation \emph{in situ} is unsustainable and difficult to reproduce. -Alternatives to real-world experiments include simulation and mathematical analysis. -Because mathematical analysis is not scalable to modern datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}, in this project we only consider simulation as a foundation for the \gls{dcdt}. -%To help datacenter operators, the scientific community proposes to simulate datacenters to make more informed decisions. - -Simulation empowers better design, testing and management of datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -A well-designed datacenter simulator can estimate a months-long workload in a few minutes or hours. -To simulate is to ``imitate of real-world process or system over time, enabling the study of, and experimentation with the internal interactions of complex systems''~\cite{DBLP:books/daglib/0034857} -In this project we only consider \emph{discrete-event simulation}. -Discrete-event simulation represents system operations as a sequence of events over time, with an assumption that no changes occur between the events. -Due to the scale and complexity of datacenters, most simulators use discrete-event simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -There exist many datacenter simulation tools, for example DGSim~\cite{DBLP:conf/europar/IosupSE08}, CloudSim~\cite{DBLP:journals/spe/CalheirosRBRB11}, SimGrid~\cite{DBLP:journals/corr/CasanovaGLQS13}, iCanCloud~\cite{DBLP:journals/grid/NunezVCCCL12}, GroudSim~\cite{DBLP:conf/europar/OstermannPPF10} and OpenDC~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -See \Cref{tab:datacenter_simulator_comparison} for a comparison of selected datacenter simulators, combined by Mastenbroek \etal~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -In order to narrow the scope of the project, we only consider {OpenDC} as a simulator for the digital twin design. -We decided to use {OpenDC}, because we find it important for a simulator to model hardware failures well. -\emph{Failure models} are a carefully calibrated, advanced feature of {OpenDC}. -Further details about {OpenDC} can be referred to in the linked literature \cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. - -\subsection{Compute Failures}\label{sss:failures} +\section{Compute Failures}\label{sss:failures} A failure is defined as ``an event that makes a system fail to operate according to its specifications``~\cite{DBLP:journals/jpdc/JavadiKIE13}. A simple example of a failure is when an old hard drive stops working. Data on the disk is lost, and services running on the respective server are disrupted. @@ -99,10 +75,35 @@ A failure trace is defined by an interval, duration, and intensity of several fa In summary OpenDC enables experimentation with failures that enables insights that are not provided by other state-of-the-art software. However, the fidelity of failure modeling inside a datacenter simulation is still insufficient to predict in failures in real-time, as they happen in a physical datacenter. Since a datacenter simulator is quite different from a digital twin, we cannot use the same computation methods from simulation to predict real-time failures. +\section{Datacenter Simulation}\label{sss:simulation} +\input{sources/simulator_comparison.tex} + +Efficient and timely datacenter management is a difficult challenge, because datacenters are extremely complex facilities. +They require deep understanding to operate properly. +However, running real-world experiments is costly in both time and resources. +Additionally, experimentation \emph{in situ} is unsustainable and difficult to reproduce. +Alternatives to real-world experiments include simulation and mathematical analysis. +Because mathematical analysis is not scalable to modern datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}, in this project we only consider simulation as a foundation for the \gls{dcdt}. +%To help datacenter operators, the scientific community proposes to simulate datacenters to make more informed decisions. + +Simulation empowers better design, testing and management of datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +A well-designed datacenter simulator can estimate a months-long workload in a few minutes or hours. +To simulate is to ``imitate of real-world process or system over time, enabling the study of, and experimentation with the internal interactions of complex systems''~\cite{DBLP:books/daglib/0034857} +In this project we only consider \emph{discrete-event simulation}. +Discrete-event simulation represents system operations as a sequence of events over time, with an assumption that no changes occur between the events. +Due to the scale and complexity of datacenters, most simulators use discrete-event simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +There exist many datacenter simulation tools, for example DGSim~\cite{DBLP:conf/europar/IosupSE08}, CloudSim~\cite{DBLP:journals/spe/CalheirosRBRB11}, SimGrid~\cite{DBLP:journals/corr/CasanovaGLQS13}, iCanCloud~\cite{DBLP:journals/grid/NunezVCCCL12}, GroudSim~\cite{DBLP:conf/europar/OstermannPPF10} and OpenDC~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +See \Cref{tab:datacenter_simulator_comparison} for a comparison of selected datacenter simulators, combined by Mastenbroek \etal~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +In order to narrow the scope of the project, we only consider {OpenDC} as a simulator for the digital twin design. +We decided to use {OpenDC}, because we find it important for a simulator to model hardware failures well. +\emph{Failure models} are a carefully calibrated, advanced feature of {OpenDC}. +Further details about {OpenDC} can be referred to in the linked literature \cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. + \begin{figure}[t] \centering \includegraphics[width=0.95\linewidth]{images/five_dimensional_dt.png} - \caption[A basic framework for the \gls{dt}.]{A basic framework for the \gls{dt}. Four core elements of a \gls{dt} are defined: The physical entity (\myCircled{1}) and the simulated virtual twin (\myCircled{2}). A service for out-of-band data analytics (\myCircled{3}) and a persistent storage of historical data (\myCircled{4}) are crucial to the \gls{dt} because they are necessary to gain meaningful monitoring insights. Adapted from Tao \etal ~\cite{DBLP:conf/cirp/TAO2018169}.} + \caption[A basic framework for the Digital Twin.]{A basic framework for the \gls{dt}. Four core elements of a \gls{dt} are defined: The physical entity (\myCircled{1}) and the simulated virtual twin (\myCircled{2}). A service for out-of-band data analytics (\myCircled{3}) and a persistent storage of historical data (\myCircled{4}) are crucial to the \gls{dt} because they are necessary to gain meaningful monitoring insights. Adapted from Tao \etal ~\cite{DBLP:conf/cirp/TAO2018169}.} + %Fei Tao is a renowned figure with over 62k citations. He is a figure of authority on digital twins.% \label{fig:five_dimensional_dt} \end{figure} @@ -119,11 +120,11 @@ The longer the \gls{dt} is working, the more accurate its predictions, because a A generic \gls{dt} architecture is depicted in Figure \ref{fig:five_dimensional_dt} from Tao \etal~\cite{DBLP:conf/cirp/TAO2018169}. % Why has not anyone done this before? -Digital twinning has only recently become feasible because of the developments in \gls{hpc}. -Between 2003 and 2011 the compute needed to run a \gls{dt} was simply not present. +Although digital twinning through simplified models exists since the 1960s, detailed and accurate digital twinning of complex and realistic scenarios has only recently become feasible because of developments in \gls{hpc}. +Between 2003 and 2011, the compute needed to run a \gls{dt} was simply not present or was (largely) unaffordable. As such, while the concept existed, the hardware did not catch up yet. However, in the last decade, multicore computing paradigms and the advent of GPU computing has finally enabled computation needed to run digital twins. -As a result, digital twins have become more relevant today than 10 years ago~\cite{DBLP:conf/cirp/TAO2018169}. +As a result, digital twins have become more relevant today than a decade ago~\cite{DBLP:conf/cirp/TAO2018169}. A crucial part any of any \gls{dt} is \emph{predictive modelling}, which drives actionable insights~\cite{DBLP:usdoe/report/AP26894} (see \Cref{fig:predictive_analytics}). \begin{figure}[t] @@ -269,7 +270,7 @@ Kalibre takes the best of both \gls{ml} and \gls{cfd} approaches and achieves su \end{figure} \subsection{System Model for Datacenter Digital Twinning} \label{ss:system_model_for_dcdts} -In \Cref{fig:system_model} we present a holistic model of \gls{dcdt}s from \Cref{sss:advanced_dts}. +In \Cref{fig:system_model} we propose a new, holistic model of \gls{dcdt}s from \Cref{sss:advanced_dts}. The figure includes the functionality present in the majority of \gls{dcdt}s, combined together into a unified model. We distinguish 3 core elements of every \gls{dcdt}: \begin{enumerate*}[label=(\arabic*)] @@ -298,7 +299,7 @@ We distinguish 3 core elements of every \gls{dcdt}: A digital twin that contains only the infrastructure model, cannot enable insights into the real-time operation of the datacenter. Likewise, a \gls{dcdt} containing just the operations model does not possess a capability to \eg simulate the datacenter. Only both, combined together enable the insights envisioned by the \gls{nasem} \gls{dt} definition~\cite{DBLP:usdoe/report/AP26894}. - \item \textbf{Digital Thread} connects the virtual world to the physical world. + \item \textbf{(novel) Digital Thread} connects the virtual world to the physical world. This is a novel contribution of our thesis. The digital thread is a \emph{conceptual} element that unites the components which do not belong in either of the worlds. All \gls{dcdt} programs from \Cref{sss:advanced_dts} contain elements that are ``in-between'' the physical and virtual twin. |