diff options
Diffstat (limited to 'content/intro.tex')
| -rw-r--r-- | content/intro.tex | 65 |
1 files changed, 29 insertions, 36 deletions
diff --git a/content/intro.tex b/content/intro.tex index 85ae411..69322d4 100644 --- a/content/intro.tex +++ b/content/intro.tex @@ -1,5 +1,4 @@ \chapter{Introduction}\label{s:intro} -% Comment from Alexandru, change presently -> currently Currently, computer and network systems play a crucial part in the digital industry. The transport, education and government sectors largely depend on digital services, which are hosted in datacenters~\cite{DBLP:journals/corr/IosupKLVG22}. To address the recent rise in demand for computation, due to the advancements in Artificial Intelligence, managers expand datacenters with new components and more heterogeneous architectures (\eg GPUs, NPUs)~\cite{DBLP:conf/date/MilojicicFDR21}. @@ -30,8 +29,8 @@ To address this new problem a concept of a datacenter \gls{dt} was proposed~\cit \begin{figure} \centering \includegraphics[width=0.8\linewidth]{images/simple_dt.pdf} - \caption{Elements of the digital twin ecosystem~\cite{DBLP:modsim24/presentation/Iosup2024} include: the insights and decisions coming from the digital twin (\myCircled{A}), the physical infrastructure (\myCircled{B}), the data coming from the physical twin telemetry (\myCircled{C}), and the digital counterpart to the physical twin (\myCircled{D}). - This thesis focuses on components (\myCircled{A}), (\myCircled{C}), and (\myCircled{D}) in this ecosystem, proposing design improvements to (\myCircled{D}, \myCircled{C}), experimental improvements to (\myCircled{A}) and a new experimental technique to substitute (\myCircled{B}). + \caption[Elements of the digital twin ecosystem.]{Elements of the digital twin ecosystem~\cite{DBLP:modsim24/presentation/Iosup2024} include: the insights and decisions coming from the digital twin (\myCircled{A}), the physical infrastructure (\myCircled{B}), the data coming from the physical twin telemetry (\myCircled{C}), and the digital counterpart to the physical twin (\myCircled{D}). + This thesis focuses on components (\myCircled{A}), (\myCircled{C}), and (\myCircled{D}) in this ecosystem, proposing design improvements to (\myCircled{D}, \myCircled{C}), and the feedback loop (\myCircled{A}). } \label{fig:simple_dt} \end{figure} @@ -58,7 +57,7 @@ The foundation to any digital twin is good monitoring and sensing capabilities i Datacenters, meet this requirement easily because they already connect hundreds of monitoring sensors. With hundreds of gigabytes of useful information coming from distributed \gls{iot} sensors inside the warehouse, we can gain insight into failure patterns, energy usage, heat dissipation \etc What remains challenging is to connect the physical and virtual spaces with a bi-directional connection -to use the monitoring insights and data analysis results for autonomous decision-making. +and to use the monitoring insights and data analysis results for autonomous decision-making. Crucial to \gls{dcdt} operation are predictive capabilities and the continuous interaction with the real-world datacenter. There already exist \gls{dcdt} deployments. @@ -69,7 +68,7 @@ Nonetheless, existing \gls{dcdt}'s are still very limited in their capabilities After all, only recently did the hardware capabilities needed to continuously simulate a datacenter become available~\cite{DBLP:conf/cirp/TAO2018169}. Many \gls{dcdt} frameworks still lack critical data analysis components, fault detection mechanisms, profiling techniques \etc~\cite{DBLP:conf/wosp/SumanCNTMI24}, rendering them unusable in large-scale systems. Such limitations gravely reduce the applicability of \gls{dcdt}'s in real world scenarios~\cite{DBLP:journals/corr/IosupKLVG22}. -\gls{dcdt}'s are urgently needed, because datacenters exhibit hundreds unexpected events every day,such as \eg service failures or hardware faults. +\gls{dcdt}'s are urgently needed, because datacenters exhibit hundreds unexpected events every day, such as \eg service failures or hardware faults. Downtime, which is the result of failures, disturbs the users and produces unfulfilled \gls{sla}~\cite{DBLP:conf/acsos/TalluriOVTI21}. % On the operational side, two main areas have been instrumental for improving datacenter efficiency: simulations and analysis of system telemetry. Additional improvements necessitate innovative tools that focus on end-to-end improvement, such as digital twins~\cite{DBLP:ExaDigiT}. % DT's merge both simulation and telemetry to develop a holistic virtual representation of the system, bridging both the physical and virtual worlds. @@ -93,42 +92,38 @@ We propose that digital twinning can be enhanced by integrating predictive analy % First research question stolen from Capelin by Georgios Andreadis and adapted to my work. \item \emph{How to assess the current state-of-the-art of digital twinning for datacenters?}\\ There is currently a lack of a unified system model of what constitutes a \gls{dcdt}, and the differences between existing \gls{dcdt} deployments. - It is necessary that we establish a common model of a \gls{dcdt} in the research community. + Thus, it is necessary that we establish a common model of a \gls{dcdt} in the research community. We must develop a holistic \gls{dcdt} model that factors in the necessary components of a \gls{dt}. This is very challenging, because the \gls{dcdt} system model must address many kinds of operational and technical requirements, compatible with the existing background on \gls{dt}s. - \item \emph{How to design a \gls{dcdt} system model using discrete-event simulation and predictive data analysis?}\\ + \item \emph{How to design a \gls{dcdt} reference architecture using discrete-event simulation and predictive data analysis?}\\ % You should start referring to my_system as a framework, rather than a standalone system. - Existing \gls{dcdt} frameworks lack the necessary predictive capabilities to prevent unplanned behaviour in datacenters. + Existing \gls{dcdt} frameworks lack the necessary predictive capabilities to prevent unplanned behaviour in datacenters~\cite{DBLP:conf/wosp/SumanCNTMI24, DBLP:conf/sc/BrewerMKWBHSGGW24, DBLP:conf/sc/TaheriBPRHDEWPM24, DBLP:journals/computer/AthavaleBBMMPS24}. In this work, we aim to explore the design space of a predictive \gls{dcdt} and the different design trade-offs. - Through discrete-event simulation, we aim provide the foundation for the system model to interact with a physical datacenter. + Through discrete-event simulation, we aim provide the foundation for the system to interact with a physical datacenter. This is a very challenging task, because there are many functional and non-functional requirements of a \gls{dcdt} that need careful consideration. The architecture must comply with the generic \gls{dt} model and address the non-trivial challenges in operating a modern datacenter. - \item \emph{How to evaluate and validate a \gls{dcdt} model in relation to system requirements}?\\ + \item \emph{How to evaluate and validate a \gls{dcdt} reference architecture in relation to system requirements}?\\ To understand the operation of the proposed system and whether it meets its design goals we need to measure it's performance. - This is a challenging and non-trivial task that requires a careful design of a set of experiments that realistically show datacenter digital twin workings. - + This is a challenging and non-trivial task that requires a careful design of a set of experiments that faithfully show the system at work. + Additionally, we need to address the novel challenge of overcoming the lack of a physical twin to experiment with. \end{enumerate} \section{Research Methodology}\label{s:research-methodology} -% Alternative formulation in case there is no time to format the results as the literature survey, taken from Mastenbroek et al. -% Toward addressing RQ1 and RQ2 we survey in Chapter 2 the existing state of the art in risk analysis. -%We conduct a review of literature of closely-related fields as well as separate engineering science such as aerospace engineering. -% This will aid in identifying the most important use-cases for digital twins and in return, the crucial functional and non-functional requirements. -% We analyze the found use-cases in the context of datacenters or brainstorm how we can adapt them to datacenters. To answer \emph{RQ\textsubscript{1}} we conduct a literature review as proposed by \textit{Kitchenham et al.} \cite{DBLP:journals/infsof/KitchenhamPBBTNL10} along with the guidance of the supervisor. Firstly, we determine the right review method. -Secondly, we identify the various works related to \gls{dcdt}'s using various search strings -(\eg ``Datacenter Digital Twinning'', ``ICT Virtual Twin''). +Secondly, we identify the various works related to \gls{dcdt}'s using different search strings +(\eg ``Datacenter Digital Twinning'', ``ICT Virtual Twin'') and query combinations (``Datacenter \code{AND} Maintenance''). To search for the results we use the digital libraries of Google Scholar, DBLP, ACM Digital Library, IEEExplore, Springer \etc -Thirdly, we select work relevant to our research and organize the details of each article. +Thirdly, we select work relevant to our research and organize the details of each article into a table. A potential outcome of this could be a system model for \gls{dcdt}'s. -We envision the literature review can supply us with potential use-cases for the predictive \gls{dcdt}. -Based on the found use-cases, we formulate the functional and non-functional requirements for the predictive \gls{dcdt} reference architecture. +We envision the literature review supplying us with potential use-cases for the predictive \gls{dcdt}. +Based on the found use-cases, we brainstorm and formulate the functional and non-functional requirements for the predictive \gls{dcdt} reference architecture. -To answer \emph{RQ\textsubscript{2}} we closely follow the \textit{AtLarge Design Process} \cite{DBLP:conf/icdcs/IosupVTETBFMT19} under the guidance of the supervisor, and propose a simulation-based \gls{dcdt} system that meets the requirements listed as a part of \emph{RQ\textsubscript{1}}. +To answer \emph{RQ\textsubscript{2}} we closely follow the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTETBFMT19} under the guidance of the supervisor. +A potential outcome might be a simulation-based \gls{dcdt} system that meets the requirements listed as a part of \emph{RQ\textsubscript{1}}. Firstly, following the literature review, we list the functional and non-functional requirements of a predictive \gls{dcdt}. We specify the pragmatic and innovative design possibilities to include in the reference architecture. -The designed system builds upon the OpenDC platform for datacenter simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}, extending it with predictive analysis capabilities. +The designed system builds upon the \code{OpenDC} platform for datacenter simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}, extending it with predictive analysis capabilities. Lastly, we ensure that the design is scientific and testable and can be evaluated with comprehensive experiments. To answer \emph{RQ\textsubscript{3}} we implement a prototype of the designed reference architecture. @@ -137,10 +132,8 @@ We first gather a set of questions worth asking about the performance and impact We define the correct experiment setup(s) and perform the experiments on a specified hardware, considering different usage scenarios. \section{Thesis Contributions}\label{s:thesis-contributions} - \begin{enumerate} \item \textbf{Conceptual}: - \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}, align=left, labelsep=0pt] \item We conduct a comprehensive literature review and detailed analysis of existing works on digital twinning in the scientific research community. We collect and organize the \gls{dcdt}'s characteristics and based on our findings we propose a unified system model of the design space. @@ -149,16 +142,14 @@ We define the correct experiment setup(s) and perform the experiments on a speci \item We evaluate \gls{my_system} using a novel experimentation technique and datacenter workload traces from the industry. We design a method to evaluate \gls{dcdt}s without expensive and costly real-world experimentation. - We conduct a set of experiments and analyse the results. + We conduct a set of exhaustive experiments and analyse the results. \end{enumerate} \item \textbf{Technical:} - \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}, align=left, labelsep=0pt] \item We prototype \gls{my_system} following the established \gls{dt} design principles using discrete-event simulation and \gls{oda}. - We include the code as an Open Science artifact and ensure the prototype remains accessible to the broader scientific community including exhaustive project documentation. - \item We provide the experiment setup, validation and evaluation of \gls{my_system} for predicting datacenter failures in real-time as an Open Science artifact. + We include the code as an Open Science artifact and ensure the prototype remains accessible to the broader scientific community including detailed project documentation. + \item We provide the experiment setup, validation and evaluation of \gls{my_system} for detecting and predicting datacenter failures in real-time as an Open Science artifact. \end{enumerate} - \end{enumerate} \section{Academic Integrity Declaration}\label{s:academic_integrity_declaration} \subsection{Non-Plagiarism Declaration}\label{ss:plagiarism-declaraion} @@ -166,7 +157,7 @@ I hereby declare that this thesis is my own independent work and writing. The thesis does not contain any material copied from other sources (person, Internet, or \gls{ai}), and has not been submitted for assessment elsewhere. I acknowledge that the usage of material from other works or paraphrase of such material without proper citations or credit will be treated as plagiarism. I declare that this thesis is free from \gls{ai} generated content and has been written without the help of any \gls{ai} tools. -In order to adhere to the strictest restrictions on AI-usage in higher education, this work follows the Berkley School of Law Artificial Intelligence Policy, as stated in \url{https://www.law.berkeley.edu/wp-content/uploads/2026/05/AI-Final-Policy-26.pdf}. +To order to adhere to the strictest restrictions on AI-usage in higher education, this work follows the Berkley School of Law Artificial Intelligence Policy, as stated in \url{https://www.law.berkeley.edu/wp-content/uploads/2026/05/AI-Final-Policy-26.pdf}. \subsection{Preventing Reference Fraud}\label{ss:plagiarism_references} I hereby declare that all the references in this thesis refer to genuine scientific work published in peer-reviewed journals or other sources of reliable and safe online information (\eg Wikipedia articles) and have been used in accordance to the article authors' wishes. @@ -201,12 +192,14 @@ The reuse and reproduction of experiments is explained in a detailed guide at th \section{Thesis Structure}\label{s:thesis-structure} The remainder of the thesis is structured as depicted in Figure \ref{fig:thesis_structure}. In Chapter \ref{s:background}, we describe the relevant background information. -In Chapter \ref{s:design}, we present the design of \gls{dcdt}. -In Chapter \ref{s:evaluation} we evaluate a prototype of the system and validate it against the set of functional and non-functional requirements. +In Chapter \ref{s:design}, we present the design of \mysystem. +In \Cref{s:implementation} we present the technical details of \mysystem prototype. +In Chapter \ref{s:evaluation} we evaluate the prototype of the system and validate it against the set of functional and non-functional requirements. In Chapter \ref{s:conclusion} we conclude the thesis with a summary of contributions and potential future work. -\begin{figure}[b!] +\newpage +\begin{figure}[t!] \centering - \includegraphics[width=\linewidth]{images/thesis_structure.pdf} + \includegraphics[width=\linewidth]{images/thesis_structure.png} \caption{Structure of this thesis, with suggested reading flows.} \label{fig:thesis_structure} \end{figure} |