diff options
Diffstat (limited to 'content/design.tex')
| -rw-r--r-- | content/design.tex | 144 |
1 files changed, 111 insertions, 33 deletions
diff --git a/content/design.tex b/content/design.tex index 2eb195d..72414a3 100644 --- a/content/design.tex +++ b/content/design.tex @@ -2,17 +2,19 @@ \chapter{Design of \mysystem, a Digital Twin For Predictive Analysis of Datacenters}\label{s:design} \section{Overview}\label{ss:design_overview} \begin{mynote} - Our contribution in this chapter is three-fold: + Addressing RQ$_2$, our contribution in this chapter is three-fold: \vspace{-0.2cm} \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}, itemsep=0.2pt] - \item We analyze the requirements for \mysystem (\Cref{ss:requirements_analysis}). - \item We propose a conceptual design for \mysystem's architecture (\Cref{ss:design_of_mysystem}) - \item We describe how \mysystem enables predictive analytics through digital twinning (\Cref{ss:design_discussion}) + \item We analyze the requirements for digital twins enabling predictive analysis of datacenters~(\Cref{ss:requirements_analysis}). + + \item We design the \mysystem architecture as a digital twin enabling predictive analysis of datacenters through discrete-event simulation and ODA~(\Cref{ss:design_of_mysystem}). + + \item We analyze how \mysystem fulfills the functional and non-functional requirements~(\Cref{ss:requirement_validation}). \end{enumerate} \end{mynote} \section{Requirements Analysis}\label{ss:requirements_analysis} In this section we determine the requirements that should be fulfilled by \mysystem. -We present here the stakeholders identified by our literature survey (see \Cref{sss:digital_twins_for_datacenters}) and the relevant use-cases. +We present here the stakeholders identified by our literature survey (see \Cref{ss:digital_twins_for_datacenters}) and the relevant use-cases. Afterwards, we list the functional and non-functional requirements for \mysystem. \subsection{Stakeholders}\label{sss:stakeholders} @@ -21,10 +23,11 @@ We identify four main stakeholders of a predictive datacenter digital twin: \item \textbf{Datacenter Managers}\\ Responsible for maintenance and operation of the warehouse, operators manage the datacenter daily. They interact with the servers, bring downed hosts up and ensure customers' services run smoothly at all times. + Importantly, datacenter operators follow a strict set of policies for taking different action. Datacenter operators need to ensure different \gls{sla}s are met, energy costs are balanced and carbon emission quota is maintained. - \item \textbf{Datacenter Technicians}\\ + \item \textbf{Datacenter Engineers}\\ The term datacenter engineers encompasses datacenter architects and technicians alike. - From the moment the datacenter layout is determined, to the physical process of booting the server racks for the first time, datacenter engineers help build and maintain the datacenter. + From the moment datacenter architects determine the warehouse layout, to the physical process of booting the server racks for the first time by the technicians, datacenter engineers help build and maintain the datacenter. They must continuously adapt to changing requirements and ensure everything goes smoothly~\cite{DBLP:journals/computer/AthavaleBBMMPS24}. \item \textbf{Scientists and Academia}\\ Digital twinning generates unprecedented amount of data. @@ -64,9 +67,10 @@ Based on the identified stakeholders we list 6 potential use-cases for a predict \subsection{Functional Requirements}\label{sss:functional_requirements} Based on a subset of the above use-cases, we formulate the functional and non-functional requirements for \mysystem: \begin{enumerate}[label=\textbf{FR\arabic* --},align=left] - \item \textbf{The system should be able to handle workloads of arbitrary size.} \\ - Existing systems range from Cloud through the Edge to HPC digital twins. - Therefore, \mysystem must support workloads similar in length and type to the commercial setting. + \item \textbf{The system should be able to handle workload size representative of its modelling technique.} \\ + Existing \gls{dt}s range from Cloud through the Edge to HPC. + Therefore, \mysystem must not impose any significant limitations on the size or type of the workload it runs. + In particular, \mysystem must be able to handle workloads as large as the underlying modelling technique permits. Without \textbf{FR1}, \mysystem will be incomplete, and like the majority of the \Cref{tab:dt_features_comparison} systems, its use-case will be niche. \textbf{FR1} is necessary to avoid overly-specializing the \gls{dcdt}. \item \textbf{The system should support failure detection.}\\ @@ -102,8 +106,12 @@ Based on a subset of the above use-cases, we formulate the functional and non-fu \subsection{Non-functional Requirements}\label{sss:non_functional_requirements} In addition to the functional requirements, we also present non-functional requirements for \mysystem: \begin{enumerate}[label=\textbf{NFR\arabic* --},align=left] - \item \textbf{The system should enable real-time insights and visualizations.} \\ + \item \textbf{Using \mysystem should not introduce any delays in visualizations longer than 1 second.}\\ The system must work in real-time, without significant delay. + We impose 1 second delay as acceptable to datacenter engineers. + We have arrived at the 1 second threshold due to the fact that datacenter operators need time to react to sudden changes in the facility behaviour. + To expect a sub-1-second performance from human employees is unjustified. + At the same time, any delay $>1$ second can already negatively impact engineer's ability to react to different events at the same time in the correct order (\ie $>1$ delay hinders the ability of technicians to follow the specific datacenter protocols and policies during an event). The system must support datacenter operators with insights at fine-grained granularity, so that insights derived from data analysis remain accurate upon reception by datacenter operators. Without \textbf{NFR1}, \mysystem's insights will not be timely, and will be useless to datacenter operators. \item \textbf{The system should log the ingestion and processing of metrics.} \\ @@ -116,14 +124,6 @@ In addition to the functional requirements, we also present non-functional requi Without \textbf{NFR4}, the system will present overwhelming amount of information to its users, rendering it unusable. \end{enumerate} -\begin{figure}[ht] - \centering - \includegraphics[width=0.75\linewidth]{images/ref_architecture.png} - \caption[The predictive datacenter digital twin architecture.]{The predictive datacenter digital twin reference architecture. - We call the system \emph{Sunfish}. - The architecture was designed with the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTETBFMT19} over several iterations in the past months.} - \label{fig:reference_architecture} -\end{figure} \section{Overview of \mysystem Architecture}\label{ss:design_of_mysystem} As a result of the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTETBFMT19} designed a reference architecture for a predictive datacenter digital twin. @@ -135,15 +135,28 @@ As a result of the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTET \item predictive analytics. \end{enumerate*} -The physical datacenter (I) encompasses 3 core elements important to digital twinning. +\subsection{The Physical Datacenter}\label{sss:physical_datacenter} +The Physical Datacenter (I) encompasses 3 core elements important to digital twinning. Workloads (\myCircled{1a}) include the hardware requirements of each datacenter job and the submission time. They are executed on the datacenter compute (\myCircled{1b}), which is controlled partly by the Datacenter Operators (\myCircled{1c}). Component (\myCircled{1c}), while seemingly unimportant, is crucial to the digital twin design. We envision \gls{dcdt}s as systems that contain a human-in-the-loop, which can control and overwrite the system's autonomous decisions. Datacenter Operators (\myCircled{1c}) interact with both the Servers (\myCircled{1b}), and have the ability to overwrite the potential autonomous digital twin decisions, stemming from component (\myCircled{2c}), the System Knobs. +\begin{figure}[t!] + \centering + \includegraphics[width=0.75\linewidth]{images/ref_architecture.png} + \caption[The predictive datacenter digital twin architecture.]{The predictive datacenter digital twin reference architecture. + We call the system \emph{Sunfish}. + The architecture was designed with the \emph{AtLarge Design Process}~\cite{DBLP:conf/icdcs/IosupVTETBFMT19} over several iterations in the past months.} + \label{fig:reference_architecture} +\end{figure} +\subsection{The Digital Thread}\label{sss:digital_thread} The Digital Thread (II) is a novel contribution from \Cref{s:background}. +It is a crucial element of our architecture. It separates the physical world from the virtual world, and contains components that do not belong to either twin, or belong to both twins. +Otherwise, it would be unclear how to model the data flow between the physical and the digital twin (\ie which elements take part in the information exchange). +Conceptually separating the physical and virtual twin makes it easier to comprehend and compartmentalize the design. It contains the Interactive Dashboard (\myCircled{2a}), the Message Broker (\myCircled{2b}), and System Knobs (\myCircled{2c}). To fulfill the functional requirements of our system, we incorporate element (\myCircled{2a}), the Interactive Dashboard to our system. @@ -152,12 +165,21 @@ The Interactive Dashboard (\myCircled{2a}) allows datacenter operators to see th The Message Broker (\myCircled{2b}) is a crucial component to the reference architecture, because it facilities the physical twin $\rightarrow$ virtual twin connection. A low-latency, high-throughput message broker partly meets our functional requirements to enable arbitrary amounts of telemetry data transfer. -We elaborate on the specific components that make up the message broker (\myCircled{2b}) in \Cref{sss:message_broker}. +The Message Broker (\myCircled{2b}) is a component is slightly more complex, and necessities a separate diagram. +In \Cref{fig:message_broker} we present the composite elements that make up the Message Broker. +In particular, the \emph{schema registry} and the \emph{connector manager} play a crucial part in fulfilling the functional requirements of our system. +The \emph{schema registry} allows the telemetry producer to submit \emph{any} data format for sending (and storing) the telemetry data.The registry is responsible for detecting what kind of format is the data sent in, and automatically adjusting the schema within the \emph{data pipeline}. +The connector manager is responsible for joining multiple distinct services to a single data pipeline. +In the reference architecture, the consumers would constitute the Database (\myCircled{3a}), the Cache (\myCircled{3b}), and the Interactive Dashboard (\myCircled{2a}). +What is remarkable about the connector manager is the ability to swiftly connect more consumers to the system. +This way, the predictive digital twin can facilitate multiple different types of analytics engines or techniques. +Additionally, the setup in \Cref{fig:message_broker} is currently standard industry practice for large software deployments. The System Knobs (\myCircled{2c}) represent the different cogs within the datacenter software and hardware (\myCircled{1b}) that can be adjusted during runtime (\eg to optimize \gls{pue}, change cooling strategy, allocate compute resources). For example, System Knobs (\myCircled{2c}) within the datacenter scheduler can be tuned to schedule jobs on Servers (\myCircled{1b}) that are least likely to experience future downtime. The autonomous actions of the digital twin (the tuning of the System Knobs (\myCircled{2c})) can be further adjusted by Datacenter Operators (\myCircled{1c}). +\subsection{The Digital Twin}\label{sss:the_digital_twin_archi} In our design, we explicitly differentiate between the physical and virtual space by including the Digital Twin (III) in a separate box. The Digital Twin (III) constitutes of the long-term storage (\myCircled{3a}), short-term storage (\myCircled{3b}), the API Server (\myCircled{3c}) and the Predictive Analytics (IV) module. @@ -176,6 +198,7 @@ The API Server (\myCircled{3c}) communicates directly with the System Knobs (\my Additionally, the Physical Twin (III) can query the API Server (\myCircled{3c}) for one-shot requests (\eg to create a new datacenter prototype configuration, to request special data analysis). Moreover, the Datacenter Operators (\myCircled{1c}) can query the API Server (\myCircled{3c}) for extra insights, when necessary. +\subsection{The Predictive Analytics Engine}\label{sss:predictive_analytics_engine} The Predictive Analytics (IV) module is an extensible part of the reference architecture, enabling different kinds of predictive analysis. In our design, to facilitate meaningful predictions we incorporate an Event-driven Simulator (\myCircled{4a}), Analytics Engine (\myCircled{4b}), and a Monitoring Service (\myCircled{4c}). @@ -197,15 +220,70 @@ Any discrepancies are communicated to the Analytics Engine (\myCircled{4b}) for \caption[The detailed view of the Message Broker.]{The detailed view of the Message Broker (\myCircled{2b}) from \Cref{fig:reference_architecture}.} \label{fig:message_broker} \end{figure} -\section{The Digital Thread and Predictive Analytics}\label{ss:detailed_design} -\subsection{Message Broker}\label{sss:message_broker} -The Message Broker (\myCircled{2b}) is a component is slightly more complex, and necessities a separate diagram. -In \Cref{fig:message_broker} we present the composite elements that make up the Message Broker. -In particular, the \emph{schema registry} and the \emph{connector manager} play a crucial part in fulfilling the functional requirements of our system. -The \emph{schema registry} allows the telemetry producer to submit \emph{any} data format for sending (and storing) the telemetry data.The registry is responsible for detecting what kind of format is the data sent in, and automatically adjusting the schema within the \emph{data pipeline}. -The connector manager is responsible for joining multiple distinct services to a single data pipeline. -In the reference architecture, the consumers would constitute the Database (\myCircled{3a}), the Cache (\myCircled{3b}), and the Interactive Dashboard (\myCircled{2a}). -What is remarkable about the connector manager is the ability to swiftly connect more consumers to the system. -This way, the predictive digital twin can facilitate multiple different types of analytics engines or techniques. -Additionally, the setup is \Cref{fig:message_broker} is currently standard industry practice for large software deployments. -\section{Discussion}\label{ss:design_discussion} + +\section{Requirement Validation}\label{ss:requirement_validation} +%How does this model satisfy the requirements? +%How did you arrive at this specific model? +In this section we describe the rationale behind the different design decisions of \mysystem. +To show that our model satisfies the functional and non-functional requirements, we map each component of \mysystem onto the requirements it fulfills. +\input{sources/requirements_validation.tex} + +The Interactive Dashboard (\myCircled{2a}) supports (\textbf{FR2}), as it allows to display the failures detected to datacenter operators; it is the core of (\textbf{FR4}) because descriptive analytics are synonymous to visualizations. +It enables (\textbf{FR5}) because the predictive insights can be displayed directly through the dashboard to the datacenter operators. +Operators can then overwrite the autonomous decisions performed by the \gls{dt}. +Dashboards are the current industry practice for clear data visualization (\textbf{FR7}). +In our design, we ensure (\textbf{NFR1}) is in a two-fold way. +Firstly, we ensure a direct connection between the Interactive Dashboard (\myCircled{2a}) and the Message Broker (\myCircled{2b}). +As the Message Broker (\myCircled{2b}) is capable of accommodating multiple consumers, the Interactive Dashboard (\myCircled{2b}) receives the same telemetry at the same rate as the live prediction engine. +Secondly, through a direct connection to the Database (\myCircled{3a}), the Interactive Dashboard (\myCircled{2a}) can instantly retrieve all historical patterns. +Thus, we meet (\textbf{NFR1}). +(\textbf{NFR2}) and (\textbf{NFR3}) are met by including in the system implementation a dashboard that enables logging of metrics. + +The Message Broker (\myCircled{2b}) indirectly fulfills (\textbf{FR1}). +A good and robust message broker alleviates potential bottlenecks that might stem from large data transfer. +My including a dedicated message broker, we ensure the bottle neck when twinning is not \mysystem. +As the Message Broker (\myCircled{2b}) is a vital component in the communication between the datacenter operators and the physical twin, it indirectly enables (\textbf{FR2}), (\textbf{FR3}), (\textbf{FR4}), (\textbf{FR5}). +In the other direction, to ensure that the \gls{dcdt} can ingest any amount of data, the Message Broker (\myCircled{2b}) is needed for (\textbf{FR6}). +The non-functional requirements are met by including a state-of-the-art message broker from either academia or the industry in the implementation. + +The inclusion of System Knobs (\myCircled{2c}) in the reference architecture fulfills (\textbf{FR2}). +With autonomous actions, the \gls{dt} can alter the operation of the datacenter based on the detected failures. +Failure detection must support datacenter operators in meeting \gls{sla}s. +Autonomous actions, enabled by System Knobs (\myCircled{2c}) help achieve that. +(\textbf{FR4}) and (\textbf{FR5}) are met by ensuring the insights that come from the \gls{dt} can be automated with the system knobs. +Supporting both predictive and descriptive analytics is core to the system knobs. +(\textbf{NFR4}) is met as the existing digital twin deployments often contain (indirectly) the System Knobs (\myCircled{2c}) element. +Including it in our design adheres to best design practice. +Importantly, (\textbf{NFR5}) is fulfilled together by creating a predictive analytics engine that is capable of intelligent insights and system knobs. +In order for the predictive insights to be realized, systems knobs are needed. + +The Database (\myCircled{2c}) meets (\textbf{FR3}), (\textbf{FR4}), (\textbf{FR5}), (\textbf{FR6}) and (\textbf{NFR2}). +Trivially, including a simple \gls{dbms} in the architecture fulfills (\textbf{FR3}). +For the different types of analytics (\textbf{FR4}), (\textbf{FR5}), the Database (\myCircled{2c}) enables insights from historical patterns. +Database are the current state-of-the-art in storing telemetry (\textbf{FR6}). +They are designed to hold arbitrary volume of data. +Lastly, (\textbf{NFR2}) is met by storing in the database the logs from both the \gls{dcdt} and the physical datacenter. + +The API Server (\myCircled{3c}) ensures (\textbf{FR4}), (\textbf{FR5}), (\textbf{FR2}) and (\textbf{NFR3}). +The server, which facilitates the insights that come into the System Knobs (\myCircled{2c}), by proxy enables (\textbf{FR4}) and (\textbf{FR5}). +To detect failures, some sort of notification daemon is needed for the datacenter operators, not just the dashboard. +The API Server (\myCircled{3c}) fulfills that role, ensuring (\textbf{FR2}). +The API server (\myCircled{3c}), specifically a server using the \gls{http} protocol is a state-of-the-art approach to existing digital twin communication (\textbf{NFR3}). + +The Cache (\myCircled{3b}) ensures the same requirements as the Database (\myCircled{3a}), with a couple of exceptions. +Namely, it also ensures (\textbf{NFR1}) and (\textbf{NFR3}). +Caching is needed for sub 1-second visualizations, and the inclusion of a cache for in-band data analytics is the current community practice. + +In our design, we include an Event-driven Simulator (\myCircled{4a}). +This ensures (\textbf{FR1}), (\textbf{FR2}), (\textbf{FR4}), (\textbf{FR5}) and all non-functional requirements except (\textbf{NFR2}). +Using a discrete-event simulator ensures that our system can handle workloads that are representative of the simulator at use. +Simulation, a method that is widely accepted by the scientific community, is one way \mysystem can ensure (\textbf{FR1}). +Detecting failures, enabling both descriptive and predictive analytics depends on how good of a simulator (\myCircled{4a}) is. +In principle, a robust, holistic simulator incorporated into the predictive engine is capable of meeting (\textbf{FR4}), (\textbf{FR2}) and (\textbf{FR5}). +Simulation also allows to model large workloads and to interact with the simulation in real-time (\textbf{NFR1}), it is considered a state-of-the-art approach to datacenter modelling (\textbf{NFR3}) and, with good data analytics provides enough data for insights at varying levels of confidence (\textbf{NFR4}). + +The Analytics Engine component (\myCircled{4b}) and the Monitoring Service (\myCircled{4c}) fulfill the same set of requirements, namely (\textbf{FR1}), (\textbf{FR2}), (\textbf{FR4}) and (\textbf{FR5}). +Additionally the Analytics Engine (\myCircled{4b}) also fulfills (\textbf{NFR4}), as it provides the capability to differentiate between different levels of confidence. +Both components are working together in unison, closely watching and reacting to the metrics from the real datacenter. +Hence, they fulfill (\textbf{FR2}), (\textbf{FR4}) and (\textbf{FR5}) and are crucial to the \gls{dcdt} operation. +(\textbf{FR1}) is fulfilled by ensuring both the Analytics Engine and the Monitoring Service are not the source of any bottlenecks during the system implementation. |