diff options
Diffstat (limited to 'content')
| -rw-r--r-- | content/background.tex | 6 | ||||
| -rw-r--r-- | content/conclusion.tex | 126 | ||||
| -rw-r--r-- | content/design.tex | 4 | ||||
| -rw-r--r-- | content/evaluation.tex | 18 | ||||
| -rw-r--r-- | content/implementation.tex | 2 | ||||
| -rw-r--r-- | content/intro.tex | 65 | ||||
| -rw-r--r-- | content/preamble/abstract.tex | 8 | ||||
| -rw-r--r-- | content/preamble/acknowledgement.tex | 2 |
8 files changed, 150 insertions, 81 deletions
diff --git a/content/background.tex b/content/background.tex index 76ce5b3..8e63f53 100644 --- a/content/background.tex +++ b/content/background.tex @@ -69,7 +69,7 @@ We present the generic, field-agnostic \gls{dt} definition and investigate how \ \begin{figure}[t] \centering \includegraphics[width=0.95\linewidth]{images/five_dimensional_dt.pdf} - \caption{A basic framework for the \gls{dt}. Four core elements of a \gls{dt} are defined: The physical entity (\myCircled{1}) and the simulated virtual twin (\myCircled{2}). A service for out-of-band data analytics (\myCircled{3}) and a persistent storage of historical data (\myCircled{4}) are crucial to the \gls{dt} because they are necessary to gain meaningful monitoring insights. Adapted from Tao \etal ~\cite{DBLP:conf/cirp/TAO2018169}.} + \caption[A basic framework for the \gls{dt}.]{A basic framework for the \gls{dt}. Four core elements of a \gls{dt} are defined: The physical entity (\myCircled{1}) and the simulated virtual twin (\myCircled{2}). A service for out-of-band data analytics (\myCircled{3}) and a persistent storage of historical data (\myCircled{4}) are crucial to the \gls{dt} because they are necessary to gain meaningful monitoring insights. Adapted from Tao \etal ~\cite{DBLP:conf/cirp/TAO2018169}.} %Fei Tao is a renowned figure with over 62k citations. He is a figure of authority on digital twins.% \label{fig:five_dimensional_dt} \end{figure} @@ -89,7 +89,7 @@ As a result, digital twins have become more relevant today than 10 years ago~\ci A crucial part any of any \gls{dt} is \emph{predictive modelling}, which drives actionable insights~\cite{DBLP:usdoe/report/AP26894} (see \Cref{fig:predictive_analytics}). \begin{figure}[t] \includegraphics[width=\linewidth]{images/predictive_analytics.pdf} - \caption{Datacenter Digital Twin diagram. There are 5 core elements to any Digital Twin: \myCircled{A} The Digital $\rightarrow$ Physical Twin link, \myCircled{B} the Physical Twin (\emph{e.g.,} the datacenter), \myCircled{C} the Physical $\rightarrow$ Digital Twin link, \myCircled{D} the Digital Twin, \myCircled{E} the features necessary to any Digital Twin.} + \caption[Datacenter digital twin diagram.]{Datacenter digital twin diagram. There are 5 core elements to any Digital Twin: \myCircled{A} The Digital $\rightarrow$ Physical Twin link, \myCircled{B} the Physical Twin (\emph{e.g.,} the datacenter), \myCircled{C} the Physical $\rightarrow$ Digital Twin link, \myCircled{D} the Digital Twin, \myCircled{E} the features necessary to any Digital Twin.} \label{fig:predictive_analytics} \end{figure} Predictive modelling uses statistics to predict outcomes. @@ -189,7 +189,7 @@ Kalibre takes the best of both \gls{ml} and \gls{cfd} approaches and achieves su \begin{figure}[t] \centering \includegraphics[width=0.95\linewidth]{images/system_model.pdf} - \caption{A generic system model for datacenter digital twin deployments. + \caption[A system model for datacenter digital twins.]{A generic system model for datacenter digital twin deployments. The design of DyTwin~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24} indirectly incorporates in its architecture a ``virtual-to-virtual`` digital thread between different digital twins. Zhao \etal likewise present key elements to the digital thread in their architecture~\cite{DBLP:conf/AppliedEnergy/Zhao20}. We add the \emph{Digital Thread} to our model explicitly.} \label{fig:system_model} diff --git a/content/conclusion.tex b/content/conclusion.tex index 83a499c..1df3d95 100644 --- a/content/conclusion.tex +++ b/content/conclusion.tex @@ -1,8 +1,7 @@ \chapter{Conclusion}\label{s:conclusion} - Datacenter manageability is a top-priority for the digital society. Over 3 million jobs in the Netherlands directly depend on cloud services, which are hosted in datacenters~\cite{DBLP:journals/corr/IosupKLVG22}. -Datacenter digital twinning, a promising management technique can offer unique insight into complex facility behaviour~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. +Datacenter digital twinning, a promising management technique can offer unique insight into complex warehouse behaviour~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. In this thesis we paved the way to more advanced \gls{dcdt}s. We contribute to the scientific community a set of findings that we hope will prove helpful in enabling predictive analytics in both existing \gls{dcdt}s and future projects. Starting from a thorough investigation into the new, emerging field of datacenter digital twinning, we designed a system capable of incorporating sophisticated data analysis techniques. @@ -10,42 +9,123 @@ We ended our project with a novel evaluation method used in a set of exhaustive We answer the main research question by addressing each sub-research question. \section{Answers to Each Research Question}\label{ss:answers_to_rqs} - -\begin{enumerate}[label=\emph{RQ\textsubscript{\arabic*}}] +\begin{enumerate}[label=\emph{RQ\textsubscript{\arabic*}}, itemsep=1em] \item \emph{How to asses the current state-of-the-art of digital twinning for datacenters?}\\ To answer this research question, we conducted a semi-structured literature review. Our findings indicate that the field of datacenter digital twinning is still under development, and there exist few \gls{dcdt} deployments. The current efforts in modelling datacenters focus on very specialized parts of datacenter management, \ie cooling and heat modelling, network mapping. Many crucial features, inherent to the \gls{dt} definition are still missing from current \gls{dcdt}s. - Present, standalone \gls{dcdt} systems fail to offer the holistic capabilities envisioned by the inventors of \gls{dt}s. - The results of the literature survey are in \Cref{tab:dt_features_comparison}, which contains systems which we found through a semi-structured literature review process. + Standalone \gls{dcdt} systems fail to offer the holistic capabilities envisioned by the inventors of \gls{dt}s. + The results of the literature survey are in \Cref{tab:dt_features_comparison}, which contains systems we found through a semi-structured literature review process. We first used structured queries, followed by a mix of snowballing and manual search. - As a result, the second contribution to answering research question 2 is a holistic system model that encompasses the features of all the systems from \Cref{tab:dt_features_comparison} (see \Cref{fig:system_model}). + As a result of the findings from \Cref{tab:dt_features_comparison}, we additionally provide a holistic system model that encompasses the features of all the systems from \Cref{tab:dt_features_comparison} (see \Cref{fig:system_model}). + This system model organizes the scattered, standalone systems into a single diagram. + We hope it will prove useful to future researchers navigating the field of datacenter digital twinning. \item \emph{How to design a reference architecture for a predictive datacenter digital twin using discrete-event simulation?}\\ To answer this research question, we first brainstormed the potential use-cases for a predictive \gls{dcdt}. - The use-cases are based on the findings of our literature survey. - We list the use-cases we found in \Cref{s:design}. - Based on a set of use-cases we created a set of functional and non-functional requirements to guide our system design. - Then, using the \emph{AtLarge Design Process} we created the reference architecture that enables predictive analysis for datacenter operators through digital twinning. + The use-cases (see \Cref{sss:use_cases}) are based on the findings of our literature survey. + Based on them, we created a set of functional and non-functional requirements (see \Cref{sss:functional_requirements,sss:non_functional_requirements}) to guide our system design. + Using the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTETBFMT19} we created the reference architecture that enables predictive analysis for datacenter operators through digital twinning. + Our system contains 4 major elements: \begin{enumerate*}[label=(\arabic*)] + \item the datacenter (physical twin), + \item the digital thread, + \item the digital twin, and + \item predictive analytics. + \end{enumerate*} + For a detailed discussion of the design, see \Cref{ss:design_of_mysystem}. \item \emph{How to validate and evaluate a datacenter digital twin architecture in relation to system requirements?}\\ - To answer the last research question we crated a prototype to evaluate our system. - Lacking the physical datacenter to experiment with, we came up with a novel digital twin evaluation method, that uses discrete-event simulation to model the physical datacenter. - Our main findings indicate that \gls{my_system} can reliably differentiate between large host failures and insignificant downtime using predictions based on the results from \code{OpenDC}, a state of the art datacenter modelling software. + To answer the last research question we crated a prototype. + During the prototype design, we used state-of-the-practice software, such as \code{Confluent Kafka}, \code{Redis} and \code{PostgreSQL} (see \Cref{ss:implementation_overview}). + However, as it turns out, evaluating \gls{dcdt}s is not a trivial task. + Lacking the physical datacenter to experiment with, we came up with a novel digital twin evaluation method. + Our method, relies solely on discrete-event simulation to model the physical datacenter, overcoming the problems of real-world experimentation (\eg sustainability, costliness, reproducibility). + The findings indicate that \gls{my_system} can reliably differentiate between large host failures and insignificant downtime using predictions from \code{OpenDC}, a state-of-the-art datacenter modelling software. Moreover, we show that \gls{my_system} can be used to incorporate predictive analytics systems and significantly lower the total number of task failures during a workload. \end{enumerate} -\section{Future Work}\label{ss:future_work} +\begin{figure}[ht] + \centering + \includegraphics[width=0.8\textwidth]{images/48_years.pdf} + \caption[48 years of microprocessor trend data.]{48 years of microprocessor trend data. Legend: \textcolor{Orange}{$\blacktriangle$ Transistors (thousands)}, \textcolor{Blue}{$\lgblkcircle$ Single Thread Performance (SpecINT $\times 10^3$)}, \textcolor{Green}{$\lgblksquare$ Frequency (MHz)}, \textcolor{Maroon}{$\blacktriangledown$ Typical Power (Watts)}, $\mdlgblkdiamond$ Number of Logical Cores~\cite{DBLP:image/48Microprocessor/Rupp}.} + \label{fig:rupp_48_years_microprocessor_data} +\end{figure} -\subsection{A Strong, New Principle of \gls{dcdt} Design}\label{sss:future_work_in_analytics} +\section{Future Work}\label{ss:future_work} +The hardware required to run future \gls{ai} models will more heterogeneous and power-hungry than ever before~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. +Due to the end of Dennard's scaling and the fading of Moore's law, we expect datacenter compute to incorporate more sophisticated architectures, (\eg GPUs, NPUs, TPUs) (see \Cref{fig:rupp_48_years_microprocessor_data}). +To tackle datacenter diversification, \gls{dcdt}s are urgently needed. We envision \gls{dcdt}s as systems that encompass features necessary to model the entire datacenter. -It came to our attention that with the growth of \gls{ai} and the diversification of datacenters under way, \gls{dt}s will be indispensable in datacenter management. -To power the predictions, we envision an \gls{ml}-based inference engine as a necessary component of digital twinning. -The need for \gls{ml} arises naturally in scenarios where large volumes of data, requiring little to no preprocessing meet the demand for estimating future facility behaviour. +To take advantage of all benefits of \gls{dt}-ing, a \gls{dcdt} must become a holistic, widely-employed tool. +If successful, \gls{dt}s will be indispensable in datacenter management, given the current growth of \gls{ai} and the diversification of compute under way. +To achieve the \gls{nasem} goals of digital twinning~\cite{DBLP:usdoe/report/AP26894}, we suggest several directions future work should take. + +\subsection{A New, Strong Principle of \gls{dcdt} Design}\label{sss:future_work_in_analytics} -\subsection{}\label{sss:future_work_in_failures} -For future work in failure prediction, we envision an \gls{abc} approach to estimate the real failure distribution within the datacenter. -Additionally, power usage optimization is a critical concern in datacenter management. -We hope future attempts to enhance datacenter digital twinning can enable datacenter operators with actionable insights towards lowering the power consumption. +\begin{enumerate}[label=\textbf{\arabic*.},align=left] + \item \textbf{The Future Goal}\\ + \mysystem enables \gls{dcdt}s to incorporate predictive analysis into facility management. + Naturally, we envision upcoming \gls{dcdt}s will use sophisticated prediction techniques (\eg statistical methods, \gls{ml}). + A \gls{dcdt} must posses predictive capabilities, by definition~\cite{DBLP:usdoe/report/AP26894}. + \item \textbf{What Is Missing?}\\ + To power the predictions, we envision an \gls{ml}-based inference engine as a necessary component of digital twinning. + The need for \gls{ml} arises naturally in scenarios where large volumes of data, requiring little to no preprocessing meet the demand for estimating future facility behaviour~\cite{Wikipedia:PredictiveModelling,CambridgeUniversityPress:book/Deisenroth}. + However, currently there are no \gls{dcdt} deployments that model the warehouse using an \gls{ml} approach to predict events (see \Cref{tab:dt_features_comparison}). + \item \textbf{The Next Steps}\\ + In short, we stipulate \gls{dcdt}s should include \gls{ml} in their \gls{oda} analysis. + The next steps would involve using \mysystem to employ a \gls{ml}-based inference engine. + To achieve this we propose to use battle-tested \gls{ml} algorithms, + \gls{erm}, the \gls{svm}, linear regression, Bayesian liner regression \etc + For future work in failure prediction, we envision an \gls{abc} approach to estimate the real failure distribution within the datacenter. + The clear benefits of this approach stem from the availability and ease of access to large volumes of data. + Moreover, \gls{ml} models are much quicker to train than \gls{cfd}-based models, and have already been employed for other purposes in existing \gls{dcdt}s~\cite{DBLP:conf/noms/ZhangZLZWC22}. + There are few drawbacks to using \gls{ml} in this scenario. + Existing works, envision going a step further, to even employ \gls{ai}-based inference engines~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. + Presently, no other statistical method can approach the accuracy of a good \gls{ml} model. +\end{enumerate} + +\subsection{A Crucial Step to Long-Term Success}\label{sss:future_work_in_education} +\begin{enumerate}[label=\textbf{\arabic*.},align=left] + \item \textbf{The Future Goal}\\ + In \Cref{ss:future_work} we highlighted the paradigm shift in datacenter design. + Computer Systems is a fast-paced, dynamic field, and we need educated engineers to make correct, well-informed decisions using \gls{dcdt}s. + We stipulate \gls{dcdt}s, specifically using simulation-based experimentation, should be included in higher education and academia (alike discrete-event simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}). + New, young students should be made aware of the different Computer Systems trade-offs and encouraged to experiment with \gls{dcdt} tools on their own~\cite{DBLP:conf/icdcs/IosupUVAEHTBT18}. + \item \textbf{What Is Missing?}\\ + Currently, \mysystem is not an education-ready program. + While a pleasant \gls{ui} exists in the form of a \code{Grafana} dashboard, \mysystem does not facilitate an intuitive \gls{ui} to experiment and explore with the \gls{dcdt} capabilities. + A clear, graphical \gls{ui}, since Douglas Engelbrat first demonstrated it in 1968~\cite{Wikipedia:article/UserInterface}, is an indispensable part of many computer programs. + Presently, it is missing from \mysystem, and from many other \gls{dt} deployments~\cite{DBLP:conf/sensys/LiW0Z0T23, DBLP:conf/noms/ZhangZLZWC22, DBLP:conf/sc/TaheriBPRHDEWPM24, DBLP:conf/sc/BrewerMKWBHSGGW24} (excluding the visualization dashboards). + \item \textbf{The Next Steps}\\ + In summary, we propose \gls{dcdt}s be enhanced with a friendly, education-supportive \gls{ui}. + The next steps to achieve this are \gls{dt}-specific. + For example, to include a \gls{ui} with \mysystem we envision exploring the existing Kotlin \gls{ui} libraries (\eg Jetpack Compose~\cite{Wikipedia:article/JetpackCompose}). + We suggested creating either a \gls{gui} or a \gls{tui} for high accessibility. + The benefits of sharing \gls{dcdt}s to higher-education and academia are vast, and there are few to none drawbacks. + Master-level courses on \gls{dt} design already exist (see \url{https://studiegids.vu.nl/en/courses/2026-2027/XMU_0068}). + We believe including \gls{dcdt} experimentation in university courses can only prove beneficial to the Computer Science society. +\end{enumerate} + +\subsection{Real-World Survey of Datacenter Digital Twinning}\label{sss:future_work_in_surveying} +\begin{enumerate}[label=\textbf{\arabic*.},align=left] + \item \textbf{The Future Goal}\\ + In \Cref{s:background} we surveyed the literature on datacenter digital twinning. + We followed the steps outlined by Kitchenham \etal~\cite{DBLP:journals/infsof/KitchenhamPBBTNL10}, however we did not conduct any real-world interviews. + Real-world practice can be much different from state-of-the-art published in scientific-journals~\cite{DBLP:journals/corr/abs-2103-02060}. + We propose a series of interviews with industry practitioners to gain a fuller insight into the potential use-cases of \gls{dcdt}s. + We stipulate a systematic literature survey, alongside qualitative interviews can offer much benefit to the \gls{dcdt} community. + \item \textbf{What Is Missing?}\\ + In this thesis we have conducted a simple, comprehensive literature survey of \gls{dcdt}s. + There already exist systematic literature surveys of generic \gls{dt}s~\cite{DBLP:conf/cirp/TAO2018169}, however what is still missing is a systematic literature survey supported by real-world interviews and careful analysis of the \gls{dcdt} subdomain. + Microsoft already offers digital twinning as a service (see \url{https://azure.microsoft.com/en-us/products/digital-twins/}), and NVIDIA is deploying digital twins as well (see \url{https://www.nvidia.com/en-sg/omniverse/}). + However, there exists little knowledge on state-of-the-practice (\eg datacenters using the 6SigmaDC software, NVIDIA Omniverse, Microsoft Azure Digital Twins). + To develop useful digital twins, engineers must be aware of the practical challenges and any tacit knowledge associated with \gls{dcdt}s. + \item \textbf{The Next Steps}\\ + In short, there is no systematic literature survey of \gls{dcdt}s completed alongside real-world interviews. + The next steps to start the literature survey is to plan, conduct and analyze interviews with practitioners from a wide variety of background and nationalities. + For an excellent example of such survey, in the field of datacenter capacity planning, we advise to read ``Capelin: Fast Data-Driven Capacity Planning for Cloud Datacenters'' by Andreadis \etal~\cite{DBLP:journals/corr/abs-2103-02060}. + We believe the scientific research community can greatly benefit from a more holistic view of \gls{dcdt}s. + Most importantly, the survey could provide valuable insight into practical \gls{dcdt} deployments using proprietary tools (\eg Microsoft Azure Digital Twins). +\end{enumerate} diff --git a/content/design.tex b/content/design.tex index 48d6ffd..2eb195d 100644 --- a/content/design.tex +++ b/content/design.tex @@ -119,7 +119,7 @@ In addition to the functional requirements, we also present non-functional requi \begin{figure}[ht] \centering \includegraphics[width=0.75\linewidth]{images/ref_architecture.png} - \caption{The predictive datacenter digital twin reference architecture. + \caption[The predictive datacenter digital twin architecture.]{The predictive datacenter digital twin reference architecture. We call the system \emph{Sunfish}. The architecture was designed with the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTETBFMT19} over several iterations in the past months.} \label{fig:reference_architecture} @@ -194,7 +194,7 @@ Any discrepancies are communicated to the Analytics Engine (\myCircled{4b}) for \begin{figure}[t] \centering \includegraphics[width=\linewidth]{images/message_broker.png} - \caption{The detailed view of the Message Broker (\myCircled{2b}) from \Cref{fig:reference_architecture}.} + \caption[The detailed view of the Message Broker.]{The detailed view of the Message Broker (\myCircled{2b}) from \Cref{fig:reference_architecture}.} \label{fig:message_broker} \end{figure} \section{The Digital Thread and Predictive Analytics}\label{ss:detailed_design} diff --git a/content/evaluation.tex b/content/evaluation.tex index 8f587d1..b17fd9d 100644 --- a/content/evaluation.tex +++ b/content/evaluation.tex @@ -23,7 +23,7 @@ \begin{figure}[t] \centering \includegraphics[width=0.8\linewidth]{images/novel_eval_method.png} - \caption{A novel evaluation method which solves the issue of real-world experimentation, which is unsustainable and costly~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}.} + \caption[A novel evaluation method proposal.]{A novel evaluation method which solves the issue of real-world experimentation, which is unsustainable and costly~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}.} \label{fig:novel_eval_method} \end{figure} @@ -72,7 +72,7 @@ We believe the deviation in the results of the experiments stemming only from th \begin{figure}[t] \centering \includegraphics[width=0.8\linewidth]{images/red_yellow_alarms.pdf} - \caption{The results of Experiment 1. \textcolor{Orange}{\ding{110} \textbf{\sffamily Red Alarms}} signify 90\% of acceptable failure threshold was reached. \textcolor{Goldenrod}{\ding{110} \textbf{\sffamily Yellow Alarms}} signify 80\% of the threshold was reached.} + \caption[The results of Experiment 1.]{The results of Experiment 1. \textcolor{Orange}{\ding{110} \textbf{\sffamily Red Alarms}} signify 90\% of acceptable failure threshold was reached. \textcolor{Goldenrod}{\ding{110} \textbf{\sffamily Yellow Alarms}} signify 80\% of the threshold was reached.} \label{fig:red_yellow_alarms} \end{figure} @@ -97,7 +97,7 @@ Importantly, the more failure-intense the trace, the more alarms are raised on b \begin{figure}[t] \centering \includegraphics[width=0.8\linewidth]{images/alarms_vs_failures.pdf} - \caption{Comparison between the total number of raised alarms and the ground truth failure distribution during a BitBrains workload in the SURF-SARA cluster. The failure traced used in this experiment models Gmail outage reports~\cite{DBLP:journals/tpds/TalluriNCKCBI26}.} + \caption[Total number of failures versus numbe rof alarms raised.]{Comparison between the total number of raised alarms and the ground truth failure distribution during a BitBrains workload in the SURF-SARA cluster. The failure traced used in this experiment models Gmail outage reports~\cite{DBLP:journals/tpds/TalluriNCKCBI26}.} \label{fig:alarms_vs_failures} \end{figure} @@ -114,7 +114,7 @@ In our experiment, the numbers differ significantly. \begin{figure}[t] \centering \includegraphics[width=0.8\linewidth]{images/failure_detecton_rate.pdf} - \caption{In this figure we show the total failure detection rate (\textcolor{Thistle}{\ding{110} \textbf{\sffamily Red + Yellow Alarms / Total Failures}}). + \caption[Failure detection rate overview.]{In this figure we show the total failure detection rate (\textcolor{Thistle}{\ding{110} \textbf{\sffamily Red + Yellow Alarms / Total Failures}}). Our results are much different from DyTwin's performance~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24}. We believe this is due to the irreconcilable differences between our experimental setups.} \label{fig:failure_detecton_rate} @@ -143,17 +143,11 @@ In this section we try to show \mysystem can additionally work well together wit \centering \includegraphics[width=1.2\linewidth]{images/conceptual_experiment.pdf} \end{minipage} - \caption{Left figure shows the potential failure distribution likelihood to approximate the true failure distribution. + \caption[The results of Experiment 2.]{Left figure shows the potential failure distribution likelihood to approximate the true failure distribution. Right figure shows the results of the conceptual experiment to show the \emph{potential} gains of employing a good predictive analytics engine with \mysystem.} \label{fig:failure_likelihood} \end{figure} -\begin{figure}[ht] - \centering - \includegraphics[width=\linewidth]{images/failure_models_table.png} - \caption{The failure models table, by Javadi \etal~\cite{DBLP:journals/jpdc/JavadiKIE13}.} - \label{tab:failure_models_table} -\end{figure} - +\input{sources/failure_models.tex} \subsection{Context}\label{sss:context_experiment2} In order to predict when a host failure might occur, the most straightforward approach is to use long-established statistical methods. diff --git a/content/implementation.tex b/content/implementation.tex index 0c31453..8db19cf 100644 --- a/content/implementation.tex +++ b/content/implementation.tex @@ -14,7 +14,7 @@ Lastly, \Cref{ss:programming} carefully explains the design decisions behind the \begin{figure}[t] \centering \includegraphics[width=0.85\linewidth]{images/implementation.pdf} - \caption{The prototype and its components based on the architecture. + \caption[The prototype and its components based on the architecture.]{The prototype and its components based on the architecture. The time-series data flows first to the \texttt{Grafana} (\myCircled{2a}) dashboard, \texttt{PostgreSQL} (\myCircled{3a}) database and \texttt{Redis} (\myCircled{3b}) cache as advised in ~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24}.} \label{fig:implementation} \end{figure} diff --git a/content/intro.tex b/content/intro.tex index 85ae411..69322d4 100644 --- a/content/intro.tex +++ b/content/intro.tex @@ -1,5 +1,4 @@ \chapter{Introduction}\label{s:intro} -% Comment from Alexandru, change presently -> currently Currently, computer and network systems play a crucial part in the digital industry. The transport, education and government sectors largely depend on digital services, which are hosted in datacenters~\cite{DBLP:journals/corr/IosupKLVG22}. To address the recent rise in demand for computation, due to the advancements in Artificial Intelligence, managers expand datacenters with new components and more heterogeneous architectures (\eg GPUs, NPUs)~\cite{DBLP:conf/date/MilojicicFDR21}. @@ -30,8 +29,8 @@ To address this new problem a concept of a datacenter \gls{dt} was proposed~\cit \begin{figure} \centering \includegraphics[width=0.8\linewidth]{images/simple_dt.pdf} - \caption{Elements of the digital twin ecosystem~\cite{DBLP:modsim24/presentation/Iosup2024} include: the insights and decisions coming from the digital twin (\myCircled{A}), the physical infrastructure (\myCircled{B}), the data coming from the physical twin telemetry (\myCircled{C}), and the digital counterpart to the physical twin (\myCircled{D}). - This thesis focuses on components (\myCircled{A}), (\myCircled{C}), and (\myCircled{D}) in this ecosystem, proposing design improvements to (\myCircled{D}, \myCircled{C}), experimental improvements to (\myCircled{A}) and a new experimental technique to substitute (\myCircled{B}). + \caption[Elements of the digital twin ecosystem.]{Elements of the digital twin ecosystem~\cite{DBLP:modsim24/presentation/Iosup2024} include: the insights and decisions coming from the digital twin (\myCircled{A}), the physical infrastructure (\myCircled{B}), the data coming from the physical twin telemetry (\myCircled{C}), and the digital counterpart to the physical twin (\myCircled{D}). + This thesis focuses on components (\myCircled{A}), (\myCircled{C}), and (\myCircled{D}) in this ecosystem, proposing design improvements to (\myCircled{D}, \myCircled{C}), and the feedback loop (\myCircled{A}). } \label{fig:simple_dt} \end{figure} @@ -58,7 +57,7 @@ The foundation to any digital twin is good monitoring and sensing capabilities i Datacenters, meet this requirement easily because they already connect hundreds of monitoring sensors. With hundreds of gigabytes of useful information coming from distributed \gls{iot} sensors inside the warehouse, we can gain insight into failure patterns, energy usage, heat dissipation \etc What remains challenging is to connect the physical and virtual spaces with a bi-directional connection -to use the monitoring insights and data analysis results for autonomous decision-making. +and to use the monitoring insights and data analysis results for autonomous decision-making. Crucial to \gls{dcdt} operation are predictive capabilities and the continuous interaction with the real-world datacenter. There already exist \gls{dcdt} deployments. @@ -69,7 +68,7 @@ Nonetheless, existing \gls{dcdt}'s are still very limited in their capabilities After all, only recently did the hardware capabilities needed to continuously simulate a datacenter become available~\cite{DBLP:conf/cirp/TAO2018169}. Many \gls{dcdt} frameworks still lack critical data analysis components, fault detection mechanisms, profiling techniques \etc~\cite{DBLP:conf/wosp/SumanCNTMI24}, rendering them unusable in large-scale systems. Such limitations gravely reduce the applicability of \gls{dcdt}'s in real world scenarios~\cite{DBLP:journals/corr/IosupKLVG22}. -\gls{dcdt}'s are urgently needed, because datacenters exhibit hundreds unexpected events every day,such as \eg service failures or hardware faults. +\gls{dcdt}'s are urgently needed, because datacenters exhibit hundreds unexpected events every day, such as \eg service failures or hardware faults. Downtime, which is the result of failures, disturbs the users and produces unfulfilled \gls{sla}~\cite{DBLP:conf/acsos/TalluriOVTI21}. % On the operational side, two main areas have been instrumental for improving datacenter efficiency: simulations and analysis of system telemetry. Additional improvements necessitate innovative tools that focus on end-to-end improvement, such as digital twins~\cite{DBLP:ExaDigiT}. % DT's merge both simulation and telemetry to develop a holistic virtual representation of the system, bridging both the physical and virtual worlds. @@ -93,42 +92,38 @@ We propose that digital twinning can be enhanced by integrating predictive analy % First research question stolen from Capelin by Georgios Andreadis and adapted to my work. \item \emph{How to assess the current state-of-the-art of digital twinning for datacenters?}\\ There is currently a lack of a unified system model of what constitutes a \gls{dcdt}, and the differences between existing \gls{dcdt} deployments. - It is necessary that we establish a common model of a \gls{dcdt} in the research community. + Thus, it is necessary that we establish a common model of a \gls{dcdt} in the research community. We must develop a holistic \gls{dcdt} model that factors in the necessary components of a \gls{dt}. This is very challenging, because the \gls{dcdt} system model must address many kinds of operational and technical requirements, compatible with the existing background on \gls{dt}s. - \item \emph{How to design a \gls{dcdt} system model using discrete-event simulation and predictive data analysis?}\\ + \item \emph{How to design a \gls{dcdt} reference architecture using discrete-event simulation and predictive data analysis?}\\ % You should start referring to my_system as a framework, rather than a standalone system. - Existing \gls{dcdt} frameworks lack the necessary predictive capabilities to prevent unplanned behaviour in datacenters. + Existing \gls{dcdt} frameworks lack the necessary predictive capabilities to prevent unplanned behaviour in datacenters~\cite{DBLP:conf/wosp/SumanCNTMI24, DBLP:conf/sc/BrewerMKWBHSGGW24, DBLP:conf/sc/TaheriBPRHDEWPM24, DBLP:journals/computer/AthavaleBBMMPS24}. In this work, we aim to explore the design space of a predictive \gls{dcdt} and the different design trade-offs. - Through discrete-event simulation, we aim provide the foundation for the system model to interact with a physical datacenter. + Through discrete-event simulation, we aim provide the foundation for the system to interact with a physical datacenter. This is a very challenging task, because there are many functional and non-functional requirements of a \gls{dcdt} that need careful consideration. The architecture must comply with the generic \gls{dt} model and address the non-trivial challenges in operating a modern datacenter. - \item \emph{How to evaluate and validate a \gls{dcdt} model in relation to system requirements}?\\ + \item \emph{How to evaluate and validate a \gls{dcdt} reference architecture in relation to system requirements}?\\ To understand the operation of the proposed system and whether it meets its design goals we need to measure it's performance. - This is a challenging and non-trivial task that requires a careful design of a set of experiments that realistically show datacenter digital twin workings. - + This is a challenging and non-trivial task that requires a careful design of a set of experiments that faithfully show the system at work. + Additionally, we need to address the novel challenge of overcoming the lack of a physical twin to experiment with. \end{enumerate} \section{Research Methodology}\label{s:research-methodology} -% Alternative formulation in case there is no time to format the results as the literature survey, taken from Mastenbroek et al. -% Toward addressing RQ1 and RQ2 we survey in Chapter 2 the existing state of the art in risk analysis. -%We conduct a review of literature of closely-related fields as well as separate engineering science such as aerospace engineering. -% This will aid in identifying the most important use-cases for digital twins and in return, the crucial functional and non-functional requirements. -% We analyze the found use-cases in the context of datacenters or brainstorm how we can adapt them to datacenters. To answer \emph{RQ\textsubscript{1}} we conduct a literature review as proposed by \textit{Kitchenham et al.} \cite{DBLP:journals/infsof/KitchenhamPBBTNL10} along with the guidance of the supervisor. Firstly, we determine the right review method. -Secondly, we identify the various works related to \gls{dcdt}'s using various search strings -(\eg ``Datacenter Digital Twinning'', ``ICT Virtual Twin''). +Secondly, we identify the various works related to \gls{dcdt}'s using different search strings +(\eg ``Datacenter Digital Twinning'', ``ICT Virtual Twin'') and query combinations (``Datacenter \code{AND} Maintenance''). To search for the results we use the digital libraries of Google Scholar, DBLP, ACM Digital Library, IEEExplore, Springer \etc -Thirdly, we select work relevant to our research and organize the details of each article. +Thirdly, we select work relevant to our research and organize the details of each article into a table. A potential outcome of this could be a system model for \gls{dcdt}'s. -We envision the literature review can supply us with potential use-cases for the predictive \gls{dcdt}. -Based on the found use-cases, we formulate the functional and non-functional requirements for the predictive \gls{dcdt} reference architecture. +We envision the literature review supplying us with potential use-cases for the predictive \gls{dcdt}. +Based on the found use-cases, we brainstorm and formulate the functional and non-functional requirements for the predictive \gls{dcdt} reference architecture. -To answer \emph{RQ\textsubscript{2}} we closely follow the \textit{AtLarge Design Process} \cite{DBLP:conf/icdcs/IosupVTETBFMT19} under the guidance of the supervisor, and propose a simulation-based \gls{dcdt} system that meets the requirements listed as a part of \emph{RQ\textsubscript{1}}. +To answer \emph{RQ\textsubscript{2}} we closely follow the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTETBFMT19} under the guidance of the supervisor. +A potential outcome might be a simulation-based \gls{dcdt} system that meets the requirements listed as a part of \emph{RQ\textsubscript{1}}. Firstly, following the literature review, we list the functional and non-functional requirements of a predictive \gls{dcdt}. We specify the pragmatic and innovative design possibilities to include in the reference architecture. -The designed system builds upon the OpenDC platform for datacenter simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}, extending it with predictive analysis capabilities. +The designed system builds upon the \code{OpenDC} platform for datacenter simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}, extending it with predictive analysis capabilities. Lastly, we ensure that the design is scientific and testable and can be evaluated with comprehensive experiments. To answer \emph{RQ\textsubscript{3}} we implement a prototype of the designed reference architecture. @@ -137,10 +132,8 @@ We first gather a set of questions worth asking about the performance and impact We define the correct experiment setup(s) and perform the experiments on a specified hardware, considering different usage scenarios. \section{Thesis Contributions}\label{s:thesis-contributions} - \begin{enumerate} \item \textbf{Conceptual}: - \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}, align=left, labelsep=0pt] \item We conduct a comprehensive literature review and detailed analysis of existing works on digital twinning in the scientific research community. We collect and organize the \gls{dcdt}'s characteristics and based on our findings we propose a unified system model of the design space. @@ -149,16 +142,14 @@ We define the correct experiment setup(s) and perform the experiments on a speci \item We evaluate \gls{my_system} using a novel experimentation technique and datacenter workload traces from the industry. We design a method to evaluate \gls{dcdt}s without expensive and costly real-world experimentation. - We conduct a set of experiments and analyse the results. + We conduct a set of exhaustive experiments and analyse the results. \end{enumerate} \item \textbf{Technical:} - \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}, align=left, labelsep=0pt] \item We prototype \gls{my_system} following the established \gls{dt} design principles using discrete-event simulation and \gls{oda}. - We include the code as an Open Science artifact and ensure the prototype remains accessible to the broader scientific community including exhaustive project documentation. - \item We provide the experiment setup, validation and evaluation of \gls{my_system} for predicting datacenter failures in real-time as an Open Science artifact. + We include the code as an Open Science artifact and ensure the prototype remains accessible to the broader scientific community including detailed project documentation. + \item We provide the experiment setup, validation and evaluation of \gls{my_system} for detecting and predicting datacenter failures in real-time as an Open Science artifact. \end{enumerate} - \end{enumerate} \section{Academic Integrity Declaration}\label{s:academic_integrity_declaration} \subsection{Non-Plagiarism Declaration}\label{ss:plagiarism-declaraion} @@ -166,7 +157,7 @@ I hereby declare that this thesis is my own independent work and writing. The thesis does not contain any material copied from other sources (person, Internet, or \gls{ai}), and has not been submitted for assessment elsewhere. I acknowledge that the usage of material from other works or paraphrase of such material without proper citations or credit will be treated as plagiarism. I declare that this thesis is free from \gls{ai} generated content and has been written without the help of any \gls{ai} tools. -In order to adhere to the strictest restrictions on AI-usage in higher education, this work follows the Berkley School of Law Artificial Intelligence Policy, as stated in \url{https://www.law.berkeley.edu/wp-content/uploads/2026/05/AI-Final-Policy-26.pdf}. +To order to adhere to the strictest restrictions on AI-usage in higher education, this work follows the Berkley School of Law Artificial Intelligence Policy, as stated in \url{https://www.law.berkeley.edu/wp-content/uploads/2026/05/AI-Final-Policy-26.pdf}. \subsection{Preventing Reference Fraud}\label{ss:plagiarism_references} I hereby declare that all the references in this thesis refer to genuine scientific work published in peer-reviewed journals or other sources of reliable and safe online information (\eg Wikipedia articles) and have been used in accordance to the article authors' wishes. @@ -201,12 +192,14 @@ The reuse and reproduction of experiments is explained in a detailed guide at th \section{Thesis Structure}\label{s:thesis-structure} The remainder of the thesis is structured as depicted in Figure \ref{fig:thesis_structure}. In Chapter \ref{s:background}, we describe the relevant background information. -In Chapter \ref{s:design}, we present the design of \gls{dcdt}. -In Chapter \ref{s:evaluation} we evaluate a prototype of the system and validate it against the set of functional and non-functional requirements. +In Chapter \ref{s:design}, we present the design of \mysystem. +In \Cref{s:implementation} we present the technical details of \mysystem prototype. +In Chapter \ref{s:evaluation} we evaluate the prototype of the system and validate it against the set of functional and non-functional requirements. In Chapter \ref{s:conclusion} we conclude the thesis with a summary of contributions and potential future work. -\begin{figure}[b!] +\newpage +\begin{figure}[t!] \centering - \includegraphics[width=\linewidth]{images/thesis_structure.pdf} + \includegraphics[width=\linewidth]{images/thesis_structure.png} \caption{Structure of this thesis, with suggested reading flows.} \label{fig:thesis_structure} \end{figure} diff --git a/content/preamble/abstract.tex b/content/preamble/abstract.tex index 5f364ec..c8078b4 100644 --- a/content/preamble/abstract.tex +++ b/content/preamble/abstract.tex @@ -3,10 +3,12 @@ \bfseries\Huge Abstract \end{center} In the modern AI economy, the strong computational demand causes datacenters to become complex, diverse facilities. -According to the Jevon's Paradox of Computer Systems, future warehouses are at an even higher risk of becoming unmanageable, due to the sheer volume of CPUs, GPUs, NPUs \etc necessary to satisfy the customers' needs. +The sheer volume of CPUs, GPUs, NPUs \etc necessary to satisfy the customers' needs complicates system administration. +Moreover, future warehouses are at an even higher risk of becoming unmanageable, according to the Jevon's Paradox of Computer Systems. To address this problem, the scientific community has proposed digital twinning as a datacenter management tool. -Digital Twins, which mirror complex objects and processes, provide a novel way to tackle the rising warehouse complexity. -However, datacenter digital twins are still under development, and lack crucial features, such as predictive analytics, mandatory for system administrators to make well-informed operational decisions. +Digital Twins, which mirror complex objects and processes to provide actionable management improvements, are a novel way to tackle the rising warehouse complexity. +However, Datacenter Digital Twins are still under development, and lack crucial features, such as predictive analytics. +Without predictive maintenance and forecasts, system administrators cannot make well-informed operational decisions. In this work, we propose to enable predictive analytics for datacenters using digital twinning. We survey the datacenter digital twinning field, and organize our findings into a system model. diff --git a/content/preamble/acknowledgement.tex b/content/preamble/acknowledgement.tex index bbb7169..df5ea7c 100644 --- a/content/preamble/acknowledgement.tex +++ b/content/preamble/acknowledgement.tex @@ -12,6 +12,6 @@ Jesse Donkervliet guided the thesis timeline and greatly assisted in preparation Daniel Halasz provided many insightful comments which shaped the technical contributions. -Daniele Bonetta, Matthijs Jansen, Tiziano De Matteis, Radu Nicolae, Jure Antunovi\'c, Sacheendra Talluri, and Krijn Doekemeijer of AtLarge Research provided indispensable knowledge on conducting good Computer Systems research over the last 2 years. +Daniele Bonetta, Matthijs Jansen, Tiziano De Matteis, Radu Nicolae, Jure Antunovi\'c, Sacheendra Talluri, and Krijn Doekemeijer of the AtLarge Research Group provided indispensable knowledge on conducting good Computer Systems research over the last 2 years. Most importantly, I would like to thank Waldemar, Anna and Julia for their love and support while I was working on this thesis. |