diff options
| -rw-r--r-- | appendix/appendix.tex | 2 | ||||
| -rw-r--r-- | citations/TalluriCloudUptimeArchive.pdf | bin | 0 -> 2037561 bytes | |||
| -rw-r--r-- | content/background.tex | 53 | ||||
| -rw-r--r-- | content/conclusion.tex | 2 | ||||
| -rw-r--r-- | content/design.tex | 9 | ||||
| -rw-r--r-- | content/evaluation.tex | 84 | ||||
| -rw-r--r-- | content/implementation.tex | 2 | ||||
| -rw-r--r-- | content/intro.tex | 8 | ||||
| -rw-r--r-- | content/preamble/abstract.tex | 4 | ||||
| -rw-r--r-- | content/preamble/acknowledgement.tex | 2 | ||||
| -rw-r--r-- | images/ai_timeline.pdf | bin | 229777 -> 229781 bytes | |||
| -rw-r--r-- | images/thesis_structure.png | bin | 406365 -> 425132 bytes | |||
| -rw-r--r-- | notes/.~lock.MJKwiatkowski_Dante_Feedback_Applied_Notes.pdf# | 1 | ||||
| -rw-r--r-- | notes/Full_Document_Annot_Dante.pdf | bin | 10455612 -> 11011928 bytes | |||
| -rw-r--r-- | notes/MJKwiatkowski_Dante_Feedback_Applied_Notes.pdf | bin | 0 -> 7968553 bytes | |||
| -rw-r--r-- | sources/failure_characteristics.tex | 15 | ||||
| -rw-r--r-- | sources/failure_model_statistics.tex | 17 | ||||
| -rw-r--r-- | sources/thesis_structure.drawio | 8 |
18 files changed, 127 insertions, 80 deletions
diff --git a/appendix/appendix.tex b/appendix/appendix.tex index 499f919..304de5a 100644 --- a/appendix/appendix.tex +++ b/appendix/appendix.tex @@ -69,5 +69,5 @@ protoc --java_out=/home/matt/git/opendc/opendc-common/src/main/java/ schema.pro Afterwards, start \mysystem using IntellijIDEA. \section*{Configuration files}\label{ss:configuration_files} \input{sources/config_files.tex} - \chapter{Honours Research Project}\label{s:hp} + %\chapter{Honours Research Project}\label{s:hp} \end{appendices} diff --git a/citations/TalluriCloudUptimeArchive.pdf b/citations/TalluriCloudUptimeArchive.pdf Binary files differnew file mode 100644 index 0000000..8d23bf7 --- /dev/null +++ b/citations/TalluriCloudUptimeArchive.pdf diff --git a/content/background.tex b/content/background.tex index 97dc6aa..68ce6a2 100644 --- a/content/background.tex +++ b/content/background.tex @@ -47,31 +47,7 @@ Making sure that all the parts of the datacenter work together is a tough task. What drives datacenter complexity even further is that sophisticated systems are not merely a sum of their parts~\cite{Wikipedia:article/Systems_Thinking}. The combination of the above factors makes datacenter management a difficult, non-trivial challenge. -\subsection{Datacenter Simulation}\label{sss:simulation} -\input{sources/simulator_comparison.tex} - -Efficient and timely datacenter management is a difficult challenge, because datacenters are extremely complex facilities. -They require deep understanding to operate properly. -However, running real-world experiments is costly in both time and resources. -Additionally, experimentation \emph{in situ} is unsustainable and difficult to reproduce. -Alternatives to real-world experiments include simulation and mathematical analysis. -Because mathematical analysis is not scalable to modern datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}, in this project we only consider simulation as a foundation for the \gls{dcdt}. -%To help datacenter operators, the scientific community proposes to simulate datacenters to make more informed decisions. - -Simulation empowers better design, testing and management of datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -A well-designed datacenter simulator can estimate a months-long workload in a few minutes or hours. -To simulate is to ``imitate of real-world process or system over time, enabling the study of, and experimentation with the internal interactions of complex systems''~\cite{DBLP:books/daglib/0034857} -In this project we only consider \emph{discrete-event simulation}. -Discrete-event simulation represents system operations as a sequence of events over time, with an assumption that no changes occur between the events. -Due to the scale and complexity of datacenters, most simulators use discrete-event simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -There exist many datacenter simulation tools, for example DGSim~\cite{DBLP:conf/europar/IosupSE08}, CloudSim~\cite{DBLP:journals/spe/CalheirosRBRB11}, SimGrid~\cite{DBLP:journals/corr/CasanovaGLQS13}, iCanCloud~\cite{DBLP:journals/grid/NunezVCCCL12}, GroudSim~\cite{DBLP:conf/europar/OstermannPPF10} and OpenDC~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -See \Cref{tab:datacenter_simulator_comparison} for a comparison of selected datacenter simulators, combined by Mastenbroek \etal~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -In order to narrow the scope of the project, we only consider {OpenDC} as a simulator for the digital twin design. -We decided to use {OpenDC}, because we find it important for a simulator to model hardware failures well. -\emph{Failure models} are a carefully calibrated, advanced feature of {OpenDC}. -Further details about {OpenDC} can be referred to in the linked literature \cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. - -\subsection{Compute Failures}\label{sss:failures} +\section{Compute Failures}\label{sss:failures} A failure is defined as ``an event that makes a system fail to operate according to its specifications``~\cite{DBLP:journals/jpdc/JavadiKIE13}. A simple example of a failure is when an old hard drive stops working. Data on the disk is lost, and services running on the respective server are disrupted. @@ -99,10 +75,35 @@ A failure trace is defined by an interval, duration, and intensity of several fa In summary OpenDC enables experimentation with failures that enables insights that are not provided by other state-of-the-art software. However, the fidelity of failure modeling inside a datacenter simulation is still insufficient to predict in failures in real-time, as they happen in a physical datacenter. Since a datacenter simulator is quite different from a digital twin, we cannot use the same computation methods from simulation to predict real-time failures. +\section{Datacenter Simulation}\label{sss:simulation} +\input{sources/simulator_comparison.tex} + +Efficient and timely datacenter management is a difficult challenge, because datacenters are extremely complex facilities. +They require deep understanding to operate properly. +However, running real-world experiments is costly in both time and resources. +Additionally, experimentation \emph{in situ} is unsustainable and difficult to reproduce. +Alternatives to real-world experiments include simulation and mathematical analysis. +Because mathematical analysis is not scalable to modern datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}, in this project we only consider simulation as a foundation for the \gls{dcdt}. +%To help datacenter operators, the scientific community proposes to simulate datacenters to make more informed decisions. + +Simulation empowers better design, testing and management of datacenters~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +A well-designed datacenter simulator can estimate a months-long workload in a few minutes or hours. +To simulate is to ``imitate of real-world process or system over time, enabling the study of, and experimentation with the internal interactions of complex systems''~\cite{DBLP:books/daglib/0034857} +In this project we only consider \emph{discrete-event simulation}. +Discrete-event simulation represents system operations as a sequence of events over time, with an assumption that no changes occur between the events. +Due to the scale and complexity of datacenters, most simulators use discrete-event simulation~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +There exist many datacenter simulation tools, for example DGSim~\cite{DBLP:conf/europar/IosupSE08}, CloudSim~\cite{DBLP:journals/spe/CalheirosRBRB11}, SimGrid~\cite{DBLP:journals/corr/CasanovaGLQS13}, iCanCloud~\cite{DBLP:journals/grid/NunezVCCCL12}, GroudSim~\cite{DBLP:conf/europar/OstermannPPF10} and OpenDC~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +See \Cref{tab:datacenter_simulator_comparison} for a comparison of selected datacenter simulators, combined by Mastenbroek \etal~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. +In order to narrow the scope of the project, we only consider {OpenDC} as a simulator for the digital twin design. +We decided to use {OpenDC}, because we find it important for a simulator to model hardware failures well. +\emph{Failure models} are a carefully calibrated, advanced feature of {OpenDC}. +Further details about {OpenDC} can be referred to in the linked literature \cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. + \begin{figure}[t] \centering \includegraphics[width=0.95\linewidth]{images/five_dimensional_dt.png} - \caption[A basic framework for the \gls{dt}.]{A basic framework for the \gls{dt}. Four core elements of a \gls{dt} are defined: The physical entity (\myCircled{1}) and the simulated virtual twin (\myCircled{2}). A service for out-of-band data analytics (\myCircled{3}) and a persistent storage of historical data (\myCircled{4}) are crucial to the \gls{dt} because they are necessary to gain meaningful monitoring insights. Adapted from Tao \etal ~\cite{DBLP:conf/cirp/TAO2018169}.} + \caption[A basic framework for the Digital Twin.]{A basic framework for the \gls{dt}. Four core elements of a \gls{dt} are defined: The physical entity (\myCircled{1}) and the simulated virtual twin (\myCircled{2}). A service for out-of-band data analytics (\myCircled{3}) and a persistent storage of historical data (\myCircled{4}) are crucial to the \gls{dt} because they are necessary to gain meaningful monitoring insights. Adapted from Tao \etal ~\cite{DBLP:conf/cirp/TAO2018169}.} + %Fei Tao is a renowned figure with over 62k citations. He is a figure of authority on digital twins.% \label{fig:five_dimensional_dt} \end{figure} diff --git a/content/conclusion.tex b/content/conclusion.tex index 86d9f68..b7694c8 100644 --- a/content/conclusion.tex +++ b/content/conclusion.tex @@ -70,7 +70,7 @@ To achieve the \gls{nasem} goals of digital twinning~\cite{DBLP:usdoe/report/AP2 A \gls{dcdt} must posses predictive capabilities, by definition~\cite{DBLP:usdoe/report/AP26894}. \item \textbf{What Is Missing?}\\ To power the predictions, we envision an \gls{ml}-based inference engine as a necessary component of digital twinning. - The need for \gls{ml} arises naturally in scenarios where large volumes of data, requiring little to no preprocessing meet the demand for estimating future facility behaviour~\cite{Wikipedia:PredictiveModelling,CambridgeUniversityPress:book/Deisenroth}. + The need for \gls{ml} arises naturally in scenarios where large volumes of data, requiring little to no preprocessing meet the demand for estimating future facility behaviour~\cite{Wikipedia:article/PredictiveModelling,CambridgeUniversityPress:book/Deisenroth}. However, currently there are no \gls{dcdt} deployments that model the warehouse using an \gls{ml} approach to predict events (see \Cref{tab:dt_features_comparison}). \item \textbf{The Next Steps}\\ In short, we stipulate \gls{dcdt}s should include \gls{ml} in their \gls{oda} analysis. diff --git a/content/design.tex b/content/design.tex index 7d4a73a..cb868ab 100644 --- a/content/design.tex +++ b/content/design.tex @@ -107,6 +107,9 @@ In addition to the functional requirements, we also present non-functional requi \item \textbf{Using \mysystem should not introduce any delays in visualizations longer than 1 second.}\\ The system must work in real-time, without significant delay. We impose 1 second delay as acceptable to datacenter engineers. + We have arrived at the 1 second threshold due to the fact that datacenter operators need time to react to sudden changes in the facility behaviour. + To expect a sub-1-second performance from human employees is unjustified. + At the same time, any delay $>1$ second can already negatively impact engineer's ability to react to different events at the same time in the correct order (\ie $>1$ delay hinders the ability of technicians to follow the specific datacenter protocols and policies during an event). The system must support datacenter operators with insights at fine-grained granularity, so that insights derived from data analysis remain accurate upon reception by datacenter operators. Without \textbf{NFR1}, \mysystem's insights will not be timely, and will be useless to datacenter operators. \item \textbf{The system should log the ingestion and processing of metrics.} \\ @@ -227,7 +230,11 @@ The Interactive Dashboard (\myCircled{2a}) supports (\textbf{FR2}), as it allows It enables (\textbf{FR5}) because the predictive insights can be displayed directly through the dashboard to the datacenter operators. Operators can then overwrite the autonomous decisions performed by the \gls{dt}. Dashboards are the current industry practice for clear data visualization (\textbf{FR7}). -For our implementation, we ensure (\textbf{NFR1}) is met by including a state-of-the-art dashboard. +In our design, we ensure (\textbf{NFR1}) is in a two-fold way. +Firstly, we ensure a direct connection between the Interactive Dashboard (\myCircled{2a}) and the Message Broker (\myCircled{2b}). +As the Message Broker (\myCircled{2b}) is capable of accommodating multiple consumers, the Interactive Dashboard (\myCircled{2b}) receives the same telemetry at the same rate as the live prediction engine. +Secondly, through a direct connection to the Database (\myCircled{3a}), the Interactive Dashboard (\myCircled{2a}) can instantly retrieve all historical patterns. +Thus, we meet (\textbf{NFR1}). (\textbf{NFR2}) and (\textbf{NFR3}) are met by including in the system implementation a dashboard that enables logging of metrics. The Message Broker (\myCircled{2b}) indirectly fulfills (\textbf{FR1}). diff --git a/content/evaluation.tex b/content/evaluation.tex index b17fd9d..5e173c7 100644 --- a/content/evaluation.tex +++ b/content/evaluation.tex @@ -18,21 +18,19 @@ \end{enumerate} \end{mynote} -\section{Experimental Setup}\label{ss:experimental_setup} - +\section{Novel Evaluation Technique}\label{ss:experimental_setup} \begin{figure}[t] \centering - \includegraphics[width=0.8\linewidth]{images/novel_eval_method.png} + \includegraphics[width=0.75\linewidth]{images/novel_eval_method.png} \caption[A novel evaluation method proposal.]{A novel evaluation method which solves the issue of real-world experimentation, which is unsustainable and costly~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}.} \label{fig:novel_eval_method} \end{figure} - -In this section we describe the technical setup used to evaluate \mysystem. -However, \Cref{fig:reference_architecture} assumes the system designer is capable of connecting the digital twin directly to the datacenter. -This raises a problem, we cannot just go and test digital twins on large systems, because we do not have large systems at hand. -Moreover, real-world experimentation is costly and unsustainable in the long run~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}. -To overcome this problem, we present a novel datacenter digital twin method capable of evaluating a \gls{dcdt} without the physical datacenter. +In the this section, we evaluate \mysystem as described in the previous sections. +To do this, we run \code{OpenDC} and all the additional software (see \Cref{s:implementation}. +However, as explained earlier in this work, running large experiments on real-world data centers is expensive and time-consuming. +Additionally, we do not have access to a data center. +To alleviate this problem, we again utilize simulation for the benefit of evaluating \gls{dcdt}. \Cref{fig:novel_eval_method} details our approach. In this approach, we replace the real-world datacenter with \emph{another} instance of the event-driven simulator from \Cref{fig:reference_architecture}. @@ -41,32 +39,28 @@ process (see \Cref{fig:implementation}). The ``physical twin'' simulator is capable of fully replacing the real-world facility, and allows for reproducible experimentation. For a detailed overview of the data flow within \Cref{fig:novel_eval_method}, see \Cref{fig:flow_diagram}. -The technical setup used for all experiment adheres to the \code{OpenDC} -documentation (see Mastenbroek \etal~\cite{DBLP:conf/ccgrid/MastenbroekAJLB21}). -The workload trace used for all experiments comes from BitBrains~\cite{DBLP:conf/ccgrid/ShenBI15}. + +\section{Experiment Parameters}\label{ss:experiment_parameters} +The workload trace used for all experiments is based on the BitBrains trace~\cite{DBLP:conf/ccgrid/ShenBI15}. In the experiments we model a Dutch SURF datacenter for scientific computing. The cluster, SURF-SARA, contains 277 hosts, each with 128GB of RAM and 16 processing cores running at maximum 2.1GHz~\cite{DBLP:conf/wosp/NicolaeTKLI26}. -The scheduling policy for all experiments is the \code{FilterScheduler} which considers the RAM and CPU capacity for choosing hosts to run tasks on. -This scheduling policy is also used by \code{SmartScheduler}, as outlined in \Cref{ss:programming}, albeit with modifications to enable the system knobs to take autonomous action. - +The \code{OpenDC} that models the physical datacenter uses the \code{SmartScheduler}. +The event driven simulator within the predictive analytics engine of \mysystem uses the default \code{FilterScheduler}. In all experiments we use either \emph{failure traces} or \emph{failure models}. -For a brief explanation on the differences between the two, consult \Cref{sss:failures}. +For a detailed explanation on the differences between the two, consult \Cref{sss:failures}. +In short, failure traces contain the data necessary to reproduce the outages experienced by the real datacenter during the simulation. +A trace contains 3 elements, failure interval, failure duration and failure intensity (see \Cref{tab:failure_traces_overview}). +A failure model is characterized as as a statistical distribution of failures based on scientific research. +It is akin to the output of an algorithm that determines when failures should happen, rather than a real-world trace~\cite{DBLP:journals/jpdc/JavadiKIE13}. In the experiments we use traces from the archive developed by Talluri \etal~\cite{DBLP:journals/tpds/TalluriNCKCBI26}. -We chose a diverse range of failure models, based on the mean failure intensity in each trace (indicated in parentheses). -As a result, we chose the traces from: \begin{enumerate*}[label=(\Roman*)] - \item Gmail (53.26\%), - \item WhatsApp (57.97\%), - \item YouTube (62.1\%), - \item Twitter (65\%), - \item Facebook (64\%). -\end{enumerate*} +We chose a diverse range of failure traces, based on the mean failure intensity in each trace (indicated in parentheses, see \Cref{tab:failure_traces_overview}). +\input{sources/failure_model_statistics.tex} +\input{sources/failure_models.tex} +\input{sources/failure_characteristics.tex} In \Cref{ss:experiment2} we used a failure trace from Skype. This is the only trace that can be paired with a corresponding failure model (\Cref{tab:failure_models_table}). Additionally, in \Cref{ss:experiment1} we find a need to define a threshold based on a statistical distribution of failures. For this purpose, we use a normal distribution with mean 1.5 and standard deviation 1.5. -Importantly, in our figures we do not report the standard deviation of our experiments. -This is due to the fact that \code{OpenDC} is a fully deterministic simulator, and on each simulation run, given the same random seed will produce exactly the same results. -We believe the deviation in the results of the experiments stemming only from the random number generator is not meaningful, therefore none of the figures contain the standard deviation bars. \section{Experiment 1: Failure Detection}\label{ss:experiment1} \begin{figure}[t] @@ -75,12 +69,21 @@ We believe the deviation in the results of the experiments stemming only from th \caption[The results of Experiment 1.]{The results of Experiment 1. \textcolor{Orange}{\ding{110} \textbf{\sffamily Red Alarms}} signify 90\% of acceptable failure threshold was reached. \textcolor{Goldenrod}{\ding{110} \textbf{\sffamily Yellow Alarms}} signify 80\% of the threshold was reached.} \label{fig:red_yellow_alarms} \end{figure} - -The purpose of this experiment is two fold: \begin{enumerate*} +For the first experiment we copy the idea of Taheri \etal~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24} for different ways a \gls{dcdt} can notify the datacenter. +Imagine a scenario: a datacenter will soon run a workload. +We want to detect and differentiate between failures that are big and unexpected and failures we anticipated would occur. +To achieve this: the \gls{dcdt} runs the workload using the simulator. +We cannot know what kind of failures we can expect, so we use a statistical distribution to approximate what might occur in practice. +In result, we get a overview of what kind of failures the datacenter engineers could expect, based on statistics alone. + +We now use the real-time feedback loop to notify the datacenter operators that what is happening in reality is different from simulation. +If we get within 80\% of the predicted threshold for number of failures we send a yellow alarm. +If we get within 90\% we send a red alarm. + +The purpose of this experiment is two fold: \begin{enumerate*}[label=(\arabic*)] \item to show our system works correctly \item to show our system fulfills the functional and non-functional requirements. \end{enumerate*} -To this end, we replicate an experiment from Taheri \etal~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24}. Inspired by the idea of red and yellow alarms, based on the different confidence threshold, we adapt their experiment to our system. The experimental setup is as defined in \Cref{ss:experimental_setup}. The experiment can be described as follows: \begin{enumerate*}[label=(\arabic*)] @@ -124,13 +127,13 @@ Compared to the DyTwin deployment, the difference is staggering. However, the discrepancy stems from the fact in our setup we differentiate between different types of failures. This capability is not present in the DyTwin digital twin~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24}. As a result, we interpret \Cref{fig:failure_detecton_rate} as showing on average, 12\% of failures in the workload are severe. -Unusually, the WhatsApp failure detection rate is the lowest, contrary to the mean failure intensity, which places WhatsApp trace as the 2nd least failure-intensive trace. +Surprisingly, the WhatsApp failure detection rate is the lowest, contrary to the mean failure intensity, which places WhatsApp trace as the 2nd least failure-intensive trace. \section{Experiment 2: Failure Prediction}\label{ss:experiment2} In \Cref{ss:experiment1} we show \mysystem is capable of incorporating descriptive analytics. Through experiment-based evaluation, we concluded \mysystem can detect and differentiate between severe and one-off host failures. -In this section we try to show \mysystem can additionally work well together with a predictive analytics engine, enabling actionable insights into the future behaviour of the datacenter. +In this section we try to show \mysystem can additionally work together with a predictive analytics engine, enabling actionable insights into the future behaviour of the datacenter. \begin{figure}[ht] \hspace{-0.8cm} @@ -147,35 +150,30 @@ In this section we try to show \mysystem can additionally work well together wit Right figure shows the results of the conceptual experiment to show the \emph{potential} gains of employing a good predictive analytics engine with \mysystem.} \label{fig:failure_likelihood} \end{figure} -\input{sources/failure_models.tex} \subsection{Context}\label{sss:context_experiment2} In order to predict when a host failure might occur, the most straightforward approach is to use long-established statistical methods. Our goal was to approximate the real failure distribution of a workload, using past data, and relevant statistical distributions. -For the task at hand, we chose the Skype trace, because it is supported by 4 different failure models, based on past Skype workload data. +For the task at hand, we chose the Skype Failure Trace, because it is supported by 4 different failure models, based on past Skype workload data. These 4 statistical distribution, published in a peer-reviewed journal are in \Cref{tab:failure_models_table}~\cite{DBLP:journals/jpdc/JavadiKIE13}. -The Skype trace model was taken from the Cloud Uptime Archive~\cite{DBLP:journals/tpds/TalluriNCKCBI26}. -The goal was to use the failure distribution to predict when a host will fail, and then in advance re-schedule all the tasks from the hosts onto different machines before it crashes. - +The Skype Failure Trace model was taken from the Cloud Uptime Archive~\cite{DBLP:journals/tpds/TalluriNCKCBI26}. +The goal is to use the failure distribution to predict when a host will fail, and then in advance re-schedule all the tasks from the hosts onto different machines before it crashes. Initial experiment results were unpromising. Using the insights from the failure models we were not able to do better than the baseline (switching hosts on and off randomly). To investigate why this might be the case, we run an experiment to identify which failure distribution at any given moment is most likely to resemble the actual, ground truth failure distribution. Using a similarity score $\mathcal{S}$, which is a weighted average of the exported metrics, we tried to determine the most similar distribution at any given time. -The results are in \Cref{fig:failure_likelihood}. +The results are shown in \Cref{fig:failure_likelihood}. In \Cref{fig:failure_likelihood} we can notice an almost random fluctuation of the similarity score $\mathcal{S}$. Any given failure model, at any time interval is almost as likely to model the actual failures as the other models. Moreover, the similarity score $\mathcal{S}$ of each failure model is never higher than 32\%. This shows, the difficulty of good predictive analytics, and the correct design of a predictive analytics engine, which is not within the scope of this thesis. -Undeterred, we set out for a different solution to show \mysystem is capable of incorporating a predictive analytics engine. +Despite the aforementioned results, we set out for a different solution to show \mysystem is capable of incorporating a predictive analytics engine. Instead, we designed a \emph{conceptual experiment}. In this setup, we \emph{assume} the predictive analytics engine is capable of fully predicting when each failure is going to happen with 100\% accuracy. Equipped with this assumption, which only serves to show \mysystem meets the functional and non-functional requirements, we conducted the second experiment. -The results are in \Cref{fig:failure_likelihood} on the right side. +The results are shown in \Cref{fig:failure_likelihood} on the right side. \Cref{fig:failure_likelihood} shows that using a perfect predictive analytics engine, \mysystem is capable of lowering the total number of failures significantly. -\section{Experiment 3: Additional Experiment}\label{ss:additional_experiment} - -\section{Discussion}\label{ss:discussion_evaluation} diff --git a/content/implementation.tex b/content/implementation.tex index 571afd3..3c77011 100644 --- a/content/implementation.tex +++ b/content/implementation.tex @@ -77,7 +77,7 @@ For a specific overview of advantages of \code{OpenDC} (\myCircled{4a}) and a t \begin{figure}[t] \centering \includegraphics[width=\linewidth]{images/flow_diagram.png} - \caption{The data flow within \gls{my_system}.} + \caption{The data flow within \mysystem.} \label{fig:flow_diagram} \end{figure} diff --git a/content/intro.tex b/content/intro.tex index 3f23487..d3fea28 100644 --- a/content/intro.tex +++ b/content/intro.tex @@ -28,7 +28,13 @@ In short, the high computational demand of \gls{ai} and the end of Dennard's sca Both events create a need for more careful datacenter management to tackle the unprecedented complexity and ensure availability of all cloud services. To address this new problem a concept of a datacenter \gls{dt} was proposed~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. +\begin{figure}[t] + \hspace{-1.75cm} + \centering + \includegraphics[width=0.8\linewidth]{images/ai_timeline.pdf} + \caption[Explosive growth in AI computational requirements.]{Explosive growth in AI computational requirements drives datacenter upgrades (source: NVIDIA Analysis: reproduction with NVIDIA permission by~\cite{DBLP:journals/computer/AthavaleBBMMPS24}). \emph{X}-axis presents the year, and the \emph{Y}-axis presents the training compute requirement (in petaFLOPs). The plot shows the one of the reasons behind why the complexity of datacenters is rising.} +\end{figure} \section{Context}\label{s:context} % A digital twin is often called a virtual twin. @@ -204,7 +210,7 @@ In Chapter \ref{s:evaluation} we evaluate the prototype of the system and valida In Chapter \ref{s:conclusion} we conclude the thesis with a summary of contributions and potential future work. \begin{figure}[t!] \centering - \includegraphics[width=\linewidth]{images/thesis_structure.png} + \includegraphics[width=0.95\linewidth]{images/thesis_structure.png} \caption{Structure of this thesis, with suggested reading flows.} \label{fig:thesis_structure} \end{figure} diff --git a/content/preamble/abstract.tex b/content/preamble/abstract.tex index c8078b4..9c2247a 100644 --- a/content/preamble/abstract.tex +++ b/content/preamble/abstract.tex @@ -12,6 +12,8 @@ Without predictive maintenance and forecasts, system administrators cannot make In this work, we propose to enable predictive analytics for datacenters using digital twinning. We survey the datacenter digital twinning field, and organize our findings into a system model. -Additionally, we design \mysystem~-- a novel reference architecture for predictive datacenter digital twins, and evaluate it through prototype-based experiments. +Based on real-world use-cases we design \mysystem~-- a novel reference architecture for predictive datacenter digital twins, and evaluate it through prototype-based experiments. Our results indicate \mysystem is capable of reliably differentiating between mild and severe compute failures, and can successfully incorporate a predictive analytics engine to the benefit of datacenter managers. +The main findings are backed-up by experiments which replicate a peer-reviewed publication evaluation approach to show that \mysystem can notify datacenter operators of compute failures. +Moreover, to show that the system can incorporate a predictive analytics-engine, we have conducted a novel, conceptual experiment to exhibit the potential future gains of a holistic Datacenter Digital Twin. diff --git a/content/preamble/acknowledgement.tex b/content/preamble/acknowledgement.tex index 180ffad..81660b5 100644 --- a/content/preamble/acknowledgement.tex +++ b/content/preamble/acknowledgement.tex @@ -4,7 +4,7 @@ \end{center} I wish to express my thanks to the many people who have helped me in preparation of this thesis. -Dante Niewehuis, my daily supervisor, with patience and understanding mentored me during our collaboration, which led to the timely and successful completion of this thesis. +Dante Niewenhuis, my daily supervisor, with patience and understanding mentored me during our collaboration, which led to the timely and successful completion of this thesis. Alexandru Iosup, my 1\textsuperscript{st} supervisor, provided invaluable feedback and suggestions that helped immensely during the design and writing. diff --git a/images/ai_timeline.pdf b/images/ai_timeline.pdf Binary files differindex ff61876..6a8bd9c 100644 --- a/images/ai_timeline.pdf +++ b/images/ai_timeline.pdf diff --git a/images/thesis_structure.png b/images/thesis_structure.png Binary files differindex 39c4c6c..e95ecf4 100644 --- a/images/thesis_structure.png +++ b/images/thesis_structure.png diff --git a/notes/.~lock.MJKwiatkowski_Dante_Feedback_Applied_Notes.pdf# b/notes/.~lock.MJKwiatkowski_Dante_Feedback_Applied_Notes.pdf# new file mode 100644 index 0000000..802385d --- /dev/null +++ b/notes/.~lock.MJKwiatkowski_Dante_Feedback_Applied_Notes.pdf# @@ -0,0 +1 @@ +,matt,frame,20.07.2026 10:14,/home/matt/.local/share/onlyoffice;
\ No newline at end of file diff --git a/notes/Full_Document_Annot_Dante.pdf b/notes/Full_Document_Annot_Dante.pdf Binary files differindex 2b7e9fc..726f301 100644 --- a/notes/Full_Document_Annot_Dante.pdf +++ b/notes/Full_Document_Annot_Dante.pdf diff --git a/notes/MJKwiatkowski_Dante_Feedback_Applied_Notes.pdf b/notes/MJKwiatkowski_Dante_Feedback_Applied_Notes.pdf Binary files differnew file mode 100644 index 0000000..bc7ca98 --- /dev/null +++ b/notes/MJKwiatkowski_Dante_Feedback_Applied_Notes.pdf diff --git a/sources/failure_characteristics.tex b/sources/failure_characteristics.tex new file mode 100644 index 0000000..3fbcdaf --- /dev/null +++ b/sources/failure_characteristics.tex @@ -0,0 +1,15 @@ +\begin{table}[t] + \centering + \renewcommand{\arraystretch}{1.4} + \begin{tabular}{lccl} + \toprule + \textbf{Metric} & \textbf{Datatype} & \textbf{Unit} & \textbf{Summary} \\ + \midrule + Failure Interval & int64 & miliseconds & The duration since the last failure. \\ + Failure Duration & int64 & miliseconds & The duration of the failure. \\ + Failure Intensity & float64 & ratio & The ratio of hosts affected by the failure. \\ + \bottomrule + \end{tabular} + \caption[Generic format of all failure traces.]{All traces used in our work follow the above format.} +\end{table} + diff --git a/sources/failure_model_statistics.tex b/sources/failure_model_statistics.tex new file mode 100644 index 0000000..03c08dd --- /dev/null +++ b/sources/failure_model_statistics.tex @@ -0,0 +1,17 @@ +\begin{table}[t] + \centering + \renewcommand{\arraystretch}{1.4} + \begin{tabular}{lr} + \toprule + \textbf{Trace} & \textbf{Mean Failure Intensity} \\ + \midrule + Gmail & 53.26\% \\ + WhatsApp & 57.97\% \\ + YouTube & 62.1\% \\ + Twitter & 65\% \\ + Facebook & 64\% \\ + \bottomrule + \end{tabular} + \caption[Overview of the failure traces]{This table shows the traces we used for scientific evaluation, and the right-hand-side shows the format of failure traces. Mean Failure Intensity can be characterized as the average of the ratio of hosts that are affected by a failure~\cite{DBLP:journals/tpds/TalluriNCKCBI26}} + \label{tab:failure_traces_overview} +\end{table} diff --git a/sources/thesis_structure.drawio b/sources/thesis_structure.drawio index 103a7b3..bc38a42 100644 --- a/sources/thesis_structure.drawio +++ b/sources/thesis_structure.drawio @@ -108,22 +108,22 @@ <mxCell id="MXKTANBymBVMColYor_E-8" edge="1" parent="1" source="MXKTANBymBVMColYor_E-9" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" target="MXKTANBymBVMColYor_E-11"> <mxGeometry relative="1" as="geometry" /> </mxCell> - <mxCell id="MXKTANBymBVMColYor_E-9" parent="1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#000000;" value="Section 5.2<br><span style="font-size: 11px;"><b>Experimental Setup</b></span>" vertex="1"> + <mxCell id="MXKTANBymBVMColYor_E-9" parent="1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#000000;" value="Section 5.2<br><span style="font-size: 11px;"><b>Novel Evaluation&nbsp;<br>Technique</b></span>" vertex="1"> <mxGeometry height="50" width="120" x="117.5" y="580" as="geometry" /> </mxCell> <mxCell id="MXKTANBymBVMColYor_E-10" edge="1" parent="1" source="MXKTANBymBVMColYor_E-11" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" target="MXKTANBymBVMColYor_E-12"> <mxGeometry relative="1" as="geometry" /> </mxCell> - <mxCell id="MXKTANBymBVMColYor_E-11" parent="1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#000000;" value="Section 5.3<br><span style="font-size: 11px;"><b>Experiment 1</b></span>" vertex="1"> + <mxCell id="MXKTANBymBVMColYor_E-11" parent="1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#000000;" value="Section 5.3<br><span style="font-size: 11px;"><b>Experimental Parameters</b></span>" vertex="1"> <mxGeometry height="50" width="120" x="252.5" y="580" as="geometry" /> </mxCell> <mxCell id="MXKTANBymBVMColYor_E-15" edge="1" parent="1" source="MXKTANBymBVMColYor_E-12" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" target="MXKTANBymBVMColYor_E-14"> <mxGeometry relative="1" as="geometry" /> </mxCell> - <mxCell id="MXKTANBymBVMColYor_E-12" parent="1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#000000;" value="Section 5.3<br><span style="font-size: 11px;"><b>Experiment 2</b></span>" vertex="1"> + <mxCell id="MXKTANBymBVMColYor_E-12" parent="1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#000000;" value="Section 5.3<br><span style="font-size: 11px;"><b>Experiment 1</b></span>" vertex="1"> <mxGeometry height="50" width="120" x="387.5" y="580" as="geometry" /> </mxCell> - <mxCell id="MXKTANBymBVMColYor_E-14" parent="1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#000000;" value="Section 5.4<br><span style="font-size: 11px;"><b>Experiment 3</b></span>" vertex="1"> + <mxCell id="MXKTANBymBVMColYor_E-14" parent="1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#000000;" value="Section 5.4<br><span style="font-size: 11px;"><b>Experiment 2</b></span>" vertex="1"> <mxGeometry height="50" width="120" x="524.5" y="580" as="geometry" /> </mxCell> <mxCell id="1ibH81aFUOVbOoGZI-IO-2" edge="1" parent="1" source="1ibH81aFUOVbOoGZI-IO-1" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" target="TxTclPCki953P5ZPPJ2z-20"> |
