diff options
Diffstat (limited to 'content/implementation.tex')
| -rw-r--r-- | content/implementation.tex | 68 |
1 files changed, 56 insertions, 12 deletions
diff --git a/content/implementation.tex b/content/implementation.tex index b289aa8..79c5191 100644 --- a/content/implementation.tex +++ b/content/implementation.tex @@ -3,17 +3,19 @@ \section{Overview}\label{ss:overview_implementation} \begin{mynote} - The contribution of this chapter is two-fold: + Addressing RQ$_3$ together with the next chapter, the contribution of this chapter is two-fold: \vspace{-0.2cm} \begin{enumerate}[label=\emph{C\textsubscript{\arabic*}}] \item We implement the real-world prototype of \mysystem (see \Cref{ss:detailed_implementation_overview}) realizing key features of the design. - \item We engineer \mysystem to enable predictive analytics engines (see \Cref{ss:programming}) + + \item We engineer \mysystem to enable predictive analytics engines (see \Cref{ss:programming}). + \end{enumerate} \end{mynote} \section{Detailed Implementation of \mysystem}\label{ss:detailed_implementation_overview} -In this section we describe the detailed implementation of \gls{my_system}. -After reading one should understand the technical decisions, choice of tools and modifications to existing software necessary for evaluation of \gls{my_system} in \Cref{s:evaluation}. +In this section, we describe the detailed implementation of \gls{my_system}. +After completing it, the reader should understand the technical decisions, choice of tools, and modifications to existing software necessary for evaluation of \gls{my_system} in \Cref{s:evaluation}. Any complex system is more than the sum of its parts~\cite{Wikipedia:article/Systems_Thinking}. To understand \gls{my_system} it is crucial to provide a holistic view on the prototype. @@ -30,7 +32,7 @@ Lastly, \Cref{ss:programming} carefully explains the design decisions behind the At the onset of the project, we decided \gls{my_system} will use only state-of-the-art software, deployed in the industry or evaluated in peer-reviewed scientific publications. The mapping of software packages used onto the reference architecture can be seen in \Cref{fig:implementation}. -In order to facilitate visualizations and interactive dashboards, we decided to use \code{Grafana} (\myCircled{2a})~\cite{Wikipedia:article/Grafana}. +To facilitate visualizations and interactive dashboards, we decided to use \code{Grafana} (\myCircled{2a})~\cite{Wikipedia:article/Grafana}. To enable the flow of data into the \gls{dt}, we use \code{Kafka} (\myCircled{2b})~\cite{Wikipedia:article/Confluent}. To store the in-band data we use a \code{Redis} (\myCircled{3b})~\cite{Wikipedia:article/Redis} cache, and for out-of-band data we use a \code{PostgreSQL}(\myCircled{3a})~\cite{Wikipedia:article/Postgresql}. To enable predictive analytics, we chose a discrete-event simulator, \code{OpenDC}(\myCircled{4a})~\cite{GitHub:software/OpenDC}. @@ -77,13 +79,13 @@ For a specific overview of advantages of \code{OpenDC} (\myCircled{4a}) and a t \begin{figure}[t] \centering \includegraphics[width=\linewidth]{images/flow_diagram.png} - \caption{The data flow within \gls{my_system}.} + \caption{The data flow within \mysystem.} \label{fig:flow_diagram} \end{figure} -\section{Data Flow}\label{ss:data_flow} +\section{Main Data Flow}\label{ss:data_flow} -In this section we describe the data flow within \Cref{fig:implementation} using a separate diagram. +In this section, we describe the data flow within \Cref{fig:implementation} using a separate diagram. Efficient data flow is of utmost importance to \gls{dt}s. In \Cref{fig:flow_diagram} we present the moving of data within \gls{my_system}. In the diagram whenever we refer to \emph{control}, we mean small, one-in-a-while data packets that contain either instructions, insights or small amount of data. @@ -120,20 +122,44 @@ Due to the massive volume of data incoming from the physical datacenter, the \co \code{Kafka} comes with excellent capability to efficiently compare data packets against a condition and filter our packets that are of no use to the \code{Analytics Engine} (see \Cref{lst:redis-sink}). On the contrary, the \code{PostgreSQL} sink (\myCircled{3}) contains all metrics collected by the datacenter sensors (see \Cref{lst:postgresql-sink}). This setup achieves excellent abstraction level, because only the most important metrics are forwarded to the \code{Analytics Engine}, with the majority of packets being filtered out. - \begin{figure}[t] \input{sources/listing_sinks.tex} \end{figure} + +\section{The \code{OpenDC} Scheduling Paradigm}\label{ss:opendc_scheduling} +In this section we introduce the scheduling paradigm of \code{OpenDC}. +\code{OpenDC}, a robust datacenter simulator uses discrete-event simulation. +``Discrete-event simulation models the operation of a system as a (discrete) sequence of events in time''~\cite{Wikipedia:article/DiscreteEventSimulation}. +Colloquially, it is akin to calling an \code{Update()} method on a set of objects to model changes in the simulator. +Scheduling in \code{OpenDC} also works using this method. +\Cref{fig:scheduling_in_opendc} represents how a task is assigned to a host in the simulation. + +A task (\myCircled{1}) is represented by its submission time, duration and computational requirements. +To be assigned a server to run on, it is deserialized into a \code{ServiceTask} (\myCircled{2}). +The \code{FilterScheduler} (\myCircled{4}) class takes care of scheduling the service task once the simulation reaches its submission time. +\code{OpenDC} maps tasks to available hosts via an \emph{allocation policy}~\cite{VUAmsterdam:thesis/AnaMariaMusca2025}. +In our work, this is always (also in \code{SmartScheduler}) the \code{FilterPolicy}. +In the \code{FilterPolicy}, a series of \code{HostFilter}s (\myCircled{3}) and \code{Host Weighter}s (\myCircled{5}) are used to find the matching host for the task. +In this project, the different filters and weighters take into account the default metrics (\ie \gls{cpu} capacity, \gls{ram} capacity, number of \gls{cpu} cores). +The host itself is represented as a \code{HostView} class (\myCircled{6}). +Importantly, the \code{HostView} class does not serve to simulate the behaviour of the host, but to provide a interface for the \emph{current state} of the host. +For actual computation, the \code{SimHost} (\myCircled{8}) class is used. +The \code{SimHost} object is created via the \code{HostProvisioningStep} class (\myCircled{7})~\cite{VUAmsterdam:thesis/AnaMariaMusca2025}. + + \section{Extensions to \code{OpenDC}}\label{ss:extensions} \code{OpenDC} is a state-of-the-art datacenter simulator. -In order to turn it into a \gls{dt}, we have made several design decisions and extensions. +To turn it into a \gls{dt}, we have made several design decisions and extensions. \begin{enumerate}[label=\textbf{\arabic*.}] \item \textbf{\code{SmartScheduler}}\\ The new \code{SmartScheduler} is a scheduling mechanism capable of incorporating the insights from the \gls{dt} into its scheduling decisions. It relies on the functionality of the \code{HTTPClient} to poll the \gls{dt} at each scheduling step for potential insights. For example, if \gls{dt} sends to the datacenter a list of hosts likely to fail in the future, the \code{SmartScheduler} acts as \emph{system knobs} to enforce the \gls{dt} insights (\ie it can be mapped to \myCircled{2c} from \Cref{fig:implementation}). + It \emph{replaces} the \code{FilterScheduler} (\myCircled{4}) in \Cref{fig:scheduling_in_opendc}. + Importantly, it is functionally almost exactly the same as the \code{FilterScheduler}, with the exception that it can change the scheduling outcome based on the information from the \gls{dt}. + It is a \code{FilterScheduler} with an attached network socket. \item \textbf{\code{KafkaMonitor}}\\ The datacenter acts as the \emph{producer} of metrics, ingested by the \code{Kafka} topic (see \Cref{fig:flow_diagram}). We equip \code{OpenDC} with a new \code{ComputeMonitor} capable of exporting data directly into a \code{Kafka} topic. @@ -141,7 +167,24 @@ In order to turn it into a \gls{dt}, we have made several design decisions and e \item \textbf{\code{HTTPClient}}\\ The \code{HTTPClient} offers the necessary functionality to communicate between the \gls{dt} and the datacenter. We decided to use the \gls{http} protocol for short, one-off communications between the \gls{dt} and the datacenter, as is common industry practice. + \item \textbf{\code{CpuUtilVictimSelector}}\\ + To ensure \code{OpenDC} can model different failure injection algorithms, we added new methods for selecting the hosts to be stopped. + The \code{CpuUtilVictimSelector} selects hosts based on their current \gls{cpu} utilization (\ie the hosts with the highest utilization are stopped first). + \item \textbf{\code{RoundRobinVictimSelector}}\\ + The \code{RoundRobinVictimSelector} is another algorithm to used to inject failures into hosts. + This method works in a round robin fashion, using the alphabetical order of the hosts (\ie by name). + For example, in a scenario hosts \code{H-01} to \code{H-20} are stopped first, and then \code{H-20} to \code{H-40} are stopped next, \etc + \item \textbf{\code{RandomVictimSelector}}\\ + This is the default \code{OpenDC} victim selection algorithm. + It randomly chooses which host to stop when injecting failures. + This component is left unmodified, but we include it here for the purposes of listing all the failure injection algorithms we use together. \end{enumerate} +\begin{figure}[t] + \centering + \includegraphics[width=\linewidth]{images/scheduling_opendc.png} + \caption[OpenDC scheduling paradigm.]{The scheduling paradigm in \code{OpenDC}. Adapted from Musc{\u a} \etal~\cite{VUAmsterdam:thesis/AnaMariaMusca2025}. The highlighted \textcolor{Orchid}{\ding{110} \textbf{Filter Scheduler}} is the component that is substituted by the \code{SmartScheduler}.} + \label{fig:scheduling_in_opendc} +\end{figure} \section{Python Modules}\label{ss:programming} @@ -152,12 +195,12 @@ For future work, we envision a system that implements the reference architecture \begin{enumerate}[label=\textbf{\arabic*.}] \item \textbf{\code{AnalyticsEngine}}\\ - The \code{AnalyticsEngine} module is necessary in order to encapsulate the logic of data preprocessing and analysis from monitoring. + The \code{AnalyticsEngine} module is necessary to encapsulate the logic of data preprocessing and analysis from monitoring. This component can contain capabilities for different statistical metrics, subject to \gls{dt}s focus. In \gls{my_system} \code{AnalyticsEngine} continuously checks whether the incoming datacenter sensor readings exceed different thresholds. For example, the \code{AnalyticsEngine} is capable of calculating a similarity score $S$ between potential failure distributions and the true failure distribution. \item \textbf{\code{HTTPServer}}\\ - The \code{HTTPServer} is crucial for interrupting the operation of the datacenter in order to adjust its operation or offer insights. + The \code{HTTPServer} is crucial for interrupting the operation of the datacenter to adjust its operation or offer insights. It maintains a python \code{Queue} structure. The \code{Queue} \emph{producer} is the \code{AnalyticsEngine} (\grayCircled{4}). The \emph{consumer} is the \code{HTTPClient} within \code{OpenDC} (\ie the real datacenter, (\myCircled{2}), (\myCircled{1})). @@ -167,3 +210,4 @@ For future work, we envision a system that implements the reference architecture It contains a \code{while True} Python loop which contains the function call to fetch the latest changes to the \code{Redis} stream. Upon update, the \code{MonitoringService} informs the \code{AnalyticsEngine} that new data is awaiting \code{AnalyticsEngine} (\grayCircled{6}). \end{enumerate} + |