diff options
| -rw-r--r-- | content/evaluation.tex | 21 | ||||
| -rw-r--r-- | images/failure_detecton_rate2.pdf | bin | 0 -> 21231 bytes |
2 files changed, 11 insertions, 10 deletions
diff --git a/content/evaluation.tex b/content/evaluation.tex index 87a9306..4978206 100644 --- a/content/evaluation.tex +++ b/content/evaluation.tex @@ -6,9 +6,9 @@ \vspace{-0.2cm} \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}] \item We provide a novel method for evaluating datacenter digital twins in \Cref{ss:experimental_setup}. - + \item We provide a comprehensive set of experiments to evaluate \mysystem~(\Cref{ss:experiment1,ss:experiment2}). - + \end{enumerate} Our findings indicate: \vspace{-0.2cm} @@ -104,13 +104,13 @@ For example, imagine the following sequence of events: \item To predict what kind of failures might occur during the workload, we ask the digital twin to run the workload first. \item We have no \emph{a priori} knowledge of the workload type, so we can only estimate the distribution of failures. To do this, the \gls{dcdt} will run the workload, assuming failures follow \eg a normal distribution. (Unlike traditional analytical tools, the distribution can be selected arbitrarily, even changed during the run, and the predictive results do not rely on underlying statistical assumptions.) \item After running the simulation, the \gls{dcdt} can analyze the results and compare them with the running workload. - \item If the running workload has more failures than what the \gls{dcdt} predicted, with the latter representing the known errors inherent in typical datacenter operations, the discrepancy is used to notify the datacenter operators -- - something is wrong. + \item If the running workload has more failures than what the \gls{dcdt} predicted, with the latter representing the known errors inherent in typical datacenter operations, the discrepancy is used to notify the datacenter operators -- + something is wrong. Through the real-time feedback loop, the \gls{dcdt} not only notifies the datacenter operators, but can also help understand where and how what is happening in reality is different from the simulation. \end{enumerate} -In our experiment, we notify the datacenter operators in two cases: -(1) we send a yellow alarm if the 80\% threshold is reached, that is, the simulated results get within 80\% of the predicted threshold for the number of failures; +In our experiment, we notify the datacenter operators in two cases: +(1) we send a yellow alarm if the 80\% threshold is reached, that is, the simulated results get within 80\% of the predicted threshold for the number of failures; (2) we send a red alarm if the 90\% threshold is reached. The purpose of this experiment is two-fold: \begin{enumerate*}[label=(\arabic*)] @@ -128,7 +128,7 @@ The results are in \Cref{fig:red_yellow_alarms,fig:alarms_vs_failures,fig:failur \begin{figure}[t] \centering \includegraphics[width=0.8\linewidth]{images/alarms_vs_failures.pdf} - \caption[Total number of failures versus numbe rof alarms raised.]{Comparison between the total number of raised alarms and the ground truth failure distribution during a SURF month workload in the SURF-SARA cluster. The failure trace used in this plot models Gmail outage reports~\cite{DBLP:journals/tpds/TalluriNCKCBI26}.} + \caption[Total number of failures versus number of alarms raised.]{Comparison between the total number of raised alarms and the ground truth failure distribution during a SURF month workload in the SURF-SARA cluster. The failure trace used in this plot models Gmail outage reports~\cite{DBLP:journals/tpds/TalluriNCKCBI26}.} \label{fig:alarms_vs_failures} \end{figure} @@ -150,9 +150,10 @@ In short, \Cref{fig:alarms_vs_failures} backs our claims, and verifies the resul \begin{figure}[!t] \centering - \includegraphics[width=0.8\linewidth]{images/failure_detecton_rate.pdf} - \caption[Failure detection ratio.]{Total failure detection ratio (\textcolor{Thistle}{\ding{110} \textbf{\sffamily Red + Yellow Alarms / Total Failures}}). - } + \includegraphics[width=0.8\linewidth]{images/failure_detecton_rate2.pdf} + \caption[Failure detection rate overview.]{In this figure we show the total failure detection rate (\textcolor{Thistle}{\ding{110} \textbf{\sffamily Red + Yellow Alarms / Total Failures}}). + Our results are much different from DyTwin's performance~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24}. + We believe this is due to the irreconcilable differences between our experimental setups.} \label{fig:failure_detecton_rate} \end{figure} diff --git a/images/failure_detecton_rate2.pdf b/images/failure_detecton_rate2.pdf Binary files differnew file mode 100644 index 0000000..ca66dae --- /dev/null +++ b/images/failure_detecton_rate2.pdf |