diff options
Diffstat (limited to 'content/conclusion.tex')
| -rw-r--r-- | content/conclusion.tex | 126 |
1 files changed, 103 insertions, 23 deletions
diff --git a/content/conclusion.tex b/content/conclusion.tex index 83a499c..1df3d95 100644 --- a/content/conclusion.tex +++ b/content/conclusion.tex @@ -1,8 +1,7 @@ \chapter{Conclusion}\label{s:conclusion} - Datacenter manageability is a top-priority for the digital society. Over 3 million jobs in the Netherlands directly depend on cloud services, which are hosted in datacenters~\cite{DBLP:journals/corr/IosupKLVG22}. -Datacenter digital twinning, a promising management technique can offer unique insight into complex facility behaviour~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. +Datacenter digital twinning, a promising management technique can offer unique insight into complex warehouse behaviour~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. In this thesis we paved the way to more advanced \gls{dcdt}s. We contribute to the scientific community a set of findings that we hope will prove helpful in enabling predictive analytics in both existing \gls{dcdt}s and future projects. Starting from a thorough investigation into the new, emerging field of datacenter digital twinning, we designed a system capable of incorporating sophisticated data analysis techniques. @@ -10,42 +9,123 @@ We ended our project with a novel evaluation method used in a set of exhaustive We answer the main research question by addressing each sub-research question. \section{Answers to Each Research Question}\label{ss:answers_to_rqs} - -\begin{enumerate}[label=\emph{RQ\textsubscript{\arabic*}}] +\begin{enumerate}[label=\emph{RQ\textsubscript{\arabic*}}, itemsep=1em] \item \emph{How to asses the current state-of-the-art of digital twinning for datacenters?}\\ To answer this research question, we conducted a semi-structured literature review. Our findings indicate that the field of datacenter digital twinning is still under development, and there exist few \gls{dcdt} deployments. The current efforts in modelling datacenters focus on very specialized parts of datacenter management, \ie cooling and heat modelling, network mapping. Many crucial features, inherent to the \gls{dt} definition are still missing from current \gls{dcdt}s. - Present, standalone \gls{dcdt} systems fail to offer the holistic capabilities envisioned by the inventors of \gls{dt}s. - The results of the literature survey are in \Cref{tab:dt_features_comparison}, which contains systems which we found through a semi-structured literature review process. + Standalone \gls{dcdt} systems fail to offer the holistic capabilities envisioned by the inventors of \gls{dt}s. + The results of the literature survey are in \Cref{tab:dt_features_comparison}, which contains systems we found through a semi-structured literature review process. We first used structured queries, followed by a mix of snowballing and manual search. - As a result, the second contribution to answering research question 2 is a holistic system model that encompasses the features of all the systems from \Cref{tab:dt_features_comparison} (see \Cref{fig:system_model}). + As a result of the findings from \Cref{tab:dt_features_comparison}, we additionally provide a holistic system model that encompasses the features of all the systems from \Cref{tab:dt_features_comparison} (see \Cref{fig:system_model}). + This system model organizes the scattered, standalone systems into a single diagram. + We hope it will prove useful to future researchers navigating the field of datacenter digital twinning. \item \emph{How to design a reference architecture for a predictive datacenter digital twin using discrete-event simulation?}\\ To answer this research question, we first brainstormed the potential use-cases for a predictive \gls{dcdt}. - The use-cases are based on the findings of our literature survey. - We list the use-cases we found in \Cref{s:design}. - Based on a set of use-cases we created a set of functional and non-functional requirements to guide our system design. - Then, using the \emph{AtLarge Design Process} we created the reference architecture that enables predictive analysis for datacenter operators through digital twinning. + The use-cases (see \Cref{sss:use_cases}) are based on the findings of our literature survey. + Based on them, we created a set of functional and non-functional requirements (see \Cref{sss:functional_requirements,sss:non_functional_requirements}) to guide our system design. + Using the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTETBFMT19} we created the reference architecture that enables predictive analysis for datacenter operators through digital twinning. + Our system contains 4 major elements: \begin{enumerate*}[label=(\arabic*)] + \item the datacenter (physical twin), + \item the digital thread, + \item the digital twin, and + \item predictive analytics. + \end{enumerate*} + For a detailed discussion of the design, see \Cref{ss:design_of_mysystem}. \item \emph{How to validate and evaluate a datacenter digital twin architecture in relation to system requirements?}\\ - To answer the last research question we crated a prototype to evaluate our system. - Lacking the physical datacenter to experiment with, we came up with a novel digital twin evaluation method, that uses discrete-event simulation to model the physical datacenter. - Our main findings indicate that \gls{my_system} can reliably differentiate between large host failures and insignificant downtime using predictions based on the results from \code{OpenDC}, a state of the art datacenter modelling software. + To answer the last research question we crated a prototype. + During the prototype design, we used state-of-the-practice software, such as \code{Confluent Kafka}, \code{Redis} and \code{PostgreSQL} (see \Cref{ss:implementation_overview}). + However, as it turns out, evaluating \gls{dcdt}s is not a trivial task. + Lacking the physical datacenter to experiment with, we came up with a novel digital twin evaluation method. + Our method, relies solely on discrete-event simulation to model the physical datacenter, overcoming the problems of real-world experimentation (\eg sustainability, costliness, reproducibility). + The findings indicate that \gls{my_system} can reliably differentiate between large host failures and insignificant downtime using predictions from \code{OpenDC}, a state-of-the-art datacenter modelling software. Moreover, we show that \gls{my_system} can be used to incorporate predictive analytics systems and significantly lower the total number of task failures during a workload. \end{enumerate} -\section{Future Work}\label{ss:future_work} +\begin{figure}[ht] + \centering + \includegraphics[width=0.8\textwidth]{images/48_years.pdf} + \caption[48 years of microprocessor trend data.]{48 years of microprocessor trend data. Legend: \textcolor{Orange}{$\blacktriangle$ Transistors (thousands)}, \textcolor{Blue}{$\lgblkcircle$ Single Thread Performance (SpecINT $\times 10^3$)}, \textcolor{Green}{$\lgblksquare$ Frequency (MHz)}, \textcolor{Maroon}{$\blacktriangledown$ Typical Power (Watts)}, $\mdlgblkdiamond$ Number of Logical Cores~\cite{DBLP:image/48Microprocessor/Rupp}.} + \label{fig:rupp_48_years_microprocessor_data} +\end{figure} -\subsection{A Strong, New Principle of \gls{dcdt} Design}\label{sss:future_work_in_analytics} +\section{Future Work}\label{ss:future_work} +The hardware required to run future \gls{ai} models will more heterogeneous and power-hungry than ever before~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. +Due to the end of Dennard's scaling and the fading of Moore's law, we expect datacenter compute to incorporate more sophisticated architectures, (\eg GPUs, NPUs, TPUs) (see \Cref{fig:rupp_48_years_microprocessor_data}). +To tackle datacenter diversification, \gls{dcdt}s are urgently needed. We envision \gls{dcdt}s as systems that encompass features necessary to model the entire datacenter. -It came to our attention that with the growth of \gls{ai} and the diversification of datacenters under way, \gls{dt}s will be indispensable in datacenter management. -To power the predictions, we envision an \gls{ml}-based inference engine as a necessary component of digital twinning. -The need for \gls{ml} arises naturally in scenarios where large volumes of data, requiring little to no preprocessing meet the demand for estimating future facility behaviour. +To take advantage of all benefits of \gls{dt}-ing, a \gls{dcdt} must become a holistic, widely-employed tool. +If successful, \gls{dt}s will be indispensable in datacenter management, given the current growth of \gls{ai} and the diversification of compute under way. +To achieve the \gls{nasem} goals of digital twinning~\cite{DBLP:usdoe/report/AP26894}, we suggest several directions future work should take. + +\subsection{A New, Strong Principle of \gls{dcdt} Design}\label{sss:future_work_in_analytics} -\subsection{}\label{sss:future_work_in_failures} -For future work in failure prediction, we envision an \gls{abc} approach to estimate the real failure distribution within the datacenter. -Additionally, power usage optimization is a critical concern in datacenter management. -We hope future attempts to enhance datacenter digital twinning can enable datacenter operators with actionable insights towards lowering the power consumption. +\begin{enumerate}[label=\textbf{\arabic*.},align=left] + \item \textbf{The Future Goal}\\ + \mysystem enables \gls{dcdt}s to incorporate predictive analysis into facility management. + Naturally, we envision upcoming \gls{dcdt}s will use sophisticated prediction techniques (\eg statistical methods, \gls{ml}). + A \gls{dcdt} must posses predictive capabilities, by definition~\cite{DBLP:usdoe/report/AP26894}. + \item \textbf{What Is Missing?}\\ + To power the predictions, we envision an \gls{ml}-based inference engine as a necessary component of digital twinning. + The need for \gls{ml} arises naturally in scenarios where large volumes of data, requiring little to no preprocessing meet the demand for estimating future facility behaviour~\cite{Wikipedia:PredictiveModelling,CambridgeUniversityPress:book/Deisenroth}. + However, currently there are no \gls{dcdt} deployments that model the warehouse using an \gls{ml} approach to predict events (see \Cref{tab:dt_features_comparison}). + \item \textbf{The Next Steps}\\ + In short, we stipulate \gls{dcdt}s should include \gls{ml} in their \gls{oda} analysis. + The next steps would involve using \mysystem to employ a \gls{ml}-based inference engine. + To achieve this we propose to use battle-tested \gls{ml} algorithms, + \gls{erm}, the \gls{svm}, linear regression, Bayesian liner regression \etc + For future work in failure prediction, we envision an \gls{abc} approach to estimate the real failure distribution within the datacenter. + The clear benefits of this approach stem from the availability and ease of access to large volumes of data. + Moreover, \gls{ml} models are much quicker to train than \gls{cfd}-based models, and have already been employed for other purposes in existing \gls{dcdt}s~\cite{DBLP:conf/noms/ZhangZLZWC22}. + There are few drawbacks to using \gls{ml} in this scenario. + Existing works, envision going a step further, to even employ \gls{ai}-based inference engines~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. + Presently, no other statistical method can approach the accuracy of a good \gls{ml} model. +\end{enumerate} + +\subsection{A Crucial Step to Long-Term Success}\label{sss:future_work_in_education} +\begin{enumerate}[label=\textbf{\arabic*.},align=left] + \item \textbf{The Future Goal}\\ + In \Cref{ss:future_work} we highlighted the paradigm shift in datacenter design. + Computer Systems is a fast-paced, dynamic field, and we need educated engineers to make correct, well-informed decisions using \gls{dcdt}s. + We stipulate \gls{dcdt}s, specifically using simulation-based experimentation, should be included in higher education and academia (alike discrete-event simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}). + New, young students should be made aware of the different Computer Systems trade-offs and encouraged to experiment with \gls{dcdt} tools on their own~\cite{DBLP:conf/icdcs/IosupUVAEHTBT18}. + \item \textbf{What Is Missing?}\\ + Currently, \mysystem is not an education-ready program. + While a pleasant \gls{ui} exists in the form of a \code{Grafana} dashboard, \mysystem does not facilitate an intuitive \gls{ui} to experiment and explore with the \gls{dcdt} capabilities. + A clear, graphical \gls{ui}, since Douglas Engelbrat first demonstrated it in 1968~\cite{Wikipedia:article/UserInterface}, is an indispensable part of many computer programs. + Presently, it is missing from \mysystem, and from many other \gls{dt} deployments~\cite{DBLP:conf/sensys/LiW0Z0T23, DBLP:conf/noms/ZhangZLZWC22, DBLP:conf/sc/TaheriBPRHDEWPM24, DBLP:conf/sc/BrewerMKWBHSGGW24} (excluding the visualization dashboards). + \item \textbf{The Next Steps}\\ + In summary, we propose \gls{dcdt}s be enhanced with a friendly, education-supportive \gls{ui}. + The next steps to achieve this are \gls{dt}-specific. + For example, to include a \gls{ui} with \mysystem we envision exploring the existing Kotlin \gls{ui} libraries (\eg Jetpack Compose~\cite{Wikipedia:article/JetpackCompose}). + We suggested creating either a \gls{gui} or a \gls{tui} for high accessibility. + The benefits of sharing \gls{dcdt}s to higher-education and academia are vast, and there are few to none drawbacks. + Master-level courses on \gls{dt} design already exist (see \url{https://studiegids.vu.nl/en/courses/2026-2027/XMU_0068}). + We believe including \gls{dcdt} experimentation in university courses can only prove beneficial to the Computer Science society. +\end{enumerate} + +\subsection{Real-World Survey of Datacenter Digital Twinning}\label{sss:future_work_in_surveying} +\begin{enumerate}[label=\textbf{\arabic*.},align=left] + \item \textbf{The Future Goal}\\ + In \Cref{s:background} we surveyed the literature on datacenter digital twinning. + We followed the steps outlined by Kitchenham \etal~\cite{DBLP:journals/infsof/KitchenhamPBBTNL10}, however we did not conduct any real-world interviews. + Real-world practice can be much different from state-of-the-art published in scientific-journals~\cite{DBLP:journals/corr/abs-2103-02060}. + We propose a series of interviews with industry practitioners to gain a fuller insight into the potential use-cases of \gls{dcdt}s. + We stipulate a systematic literature survey, alongside qualitative interviews can offer much benefit to the \gls{dcdt} community. + \item \textbf{What Is Missing?}\\ + In this thesis we have conducted a simple, comprehensive literature survey of \gls{dcdt}s. + There already exist systematic literature surveys of generic \gls{dt}s~\cite{DBLP:conf/cirp/TAO2018169}, however what is still missing is a systematic literature survey supported by real-world interviews and careful analysis of the \gls{dcdt} subdomain. + Microsoft already offers digital twinning as a service (see \url{https://azure.microsoft.com/en-us/products/digital-twins/}), and NVIDIA is deploying digital twins as well (see \url{https://www.nvidia.com/en-sg/omniverse/}). + However, there exists little knowledge on state-of-the-practice (\eg datacenters using the 6SigmaDC software, NVIDIA Omniverse, Microsoft Azure Digital Twins). + To develop useful digital twins, engineers must be aware of the practical challenges and any tacit knowledge associated with \gls{dcdt}s. + \item \textbf{The Next Steps}\\ + In short, there is no systematic literature survey of \gls{dcdt}s completed alongside real-world interviews. + The next steps to start the literature survey is to plan, conduct and analyze interviews with practitioners from a wide variety of background and nationalities. + For an excellent example of such survey, in the field of datacenter capacity planning, we advise to read ``Capelin: Fast Data-Driven Capacity Planning for Cloud Datacenters'' by Andreadis \etal~\cite{DBLP:journals/corr/abs-2103-02060}. + We believe the scientific research community can greatly benefit from a more holistic view of \gls{dcdt}s. + Most importantly, the survey could provide valuable insight into practical \gls{dcdt} deployments using proprietary tools (\eg Microsoft Azure Digital Twins). +\end{enumerate} |