diff options
| author | mjkwiatkowski <mati.rewa@gmail.com> | 2026-07-15 17:50:43 +0200 |
|---|---|---|
| committer | mjkwiatkowski <mati.rewa@gmail.com> | 2026-07-15 17:50:43 +0200 |
| commit | 9a51d284a15393bda1fd0422078cdb15793acf32 (patch) | |
| tree | 894e3e32d6472c7f5fd9f9d6fc28ff94856cd033 /content | |
| parent | d4c3a916398e4eee9c1eaaa840968d4b19f40c91 (diff) | |
feat: applied Dante's comments to the implementation
Diffstat (limited to 'content')
| -rw-r--r-- | content/background.tex | 24 | ||||
| -rw-r--r-- | content/implementation.tex | 33 |
2 files changed, 43 insertions, 14 deletions
diff --git a/content/background.tex b/content/background.tex index 6e39e6d..3e0a5cf 100644 --- a/content/background.tex +++ b/content/background.tex @@ -160,11 +160,11 @@ In this subsection, we describe the methods for collecting relevant scientific a \begin{enumerate}[label=\textbf{\arabic*.}] \item \textbf{Review Strategy}\\ - The most common methods for conducting literature surveys are \begin{enumerate*}[label=(\arabic*)] + According to Suman \etal~\cite{DBLP:conf/wosp/SumanCNTMI24}, the most common methods for conducting literature surveys are \begin{enumerate*}[label=(\arabic*)] \item random traversal of the related literature, \item snowballing~\cite{ACM:article/Webster2002}, \item systematic literature survey as proposed by Kitchenham \etal~\cite{DBLP:journals/infsof/KitchenhamPBBTNL10} - \end{enumerate*}~\cite{DBLP:conf/wosp/SumanCNTMI24} + \end{enumerate*}. Random traversal encompasses surveying the field by following suggestions from portals like Google Scholar and randomly querying the different databases. It is an unstructured way to conduct the literature review, and requires little effort. Snowballing is similar to random traversal, but it is more structured. @@ -259,7 +259,14 @@ Kalibre takes the best of both \gls{ml} and \gls{cfd} approaches and achieves su % What is more, Microsoft already offers digital twinning as a service https://azure.microsoft.com/en-us/products/digital-twins/ % Documentation: https://learn.microsoft.com/en-us/azure/digital-twins/ % Moreover, NVIDIA is doing too as well https://www.nvidia.com/en-sg/omniverse/ - +\begin{figure}[t] + \centering + \includegraphics[width=0.95\linewidth]{images/system_model.png} + \caption[A system model for datacenter digital twins.]{A generic system model for datacenter digital twin deployments. + The design of DyTwin~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24} indirectly incorporates in its architecture a ``virtual-to-virtual`` digital thread between different digital twins. + Zhao \etal likewise present key elements to the digital thread in their architecture~\cite{DBLP:conf/AppliedEnergy/Zhao20}. We add the \emph{Digital Thread} to our model explicitly.} + \label{fig:system_model} +\end{figure} \subsection{System Model for Datacenter Digital Twinning} \label{ss:system_model_for_dcdts} In \Cref{fig:system_model} we present a holistic model of \gls{dcdt}s from \Cref{sss:advanced_dts}. @@ -316,15 +323,6 @@ We distinguish 3 core elements of every \gls{dcdt}: In order to adhere to the holistic view of \gls{dcdt}s, and to fulfill the \gls{nasem}'s definition, the system must contain all 3 of these elements. \end{enumerate} -\begin{figure}[t] - \centering - \includegraphics[width=0.95\linewidth]{images/system_model.png} - \caption[A system model for datacenter digital twins.]{A generic system model for datacenter digital twin deployments. - The design of DyTwin~\cite{DBLP:conf/sc/TaheriBPRHDEWPM24} indirectly incorporates in its architecture a ``virtual-to-virtual`` digital thread between different digital twins. - Zhao \etal likewise present key elements to the digital thread in their architecture~\cite{DBLP:conf/AppliedEnergy/Zhao20}. We add the \emph{Digital Thread} to our model explicitly.} - \label{fig:system_model} -\end{figure} - %To summarize, many \gls{dcdt}'s model the cooling systems inside the warehouse, because in a typical datacenter cooling accounts for more than 40\% of total electricity usage~\cite{DBLP:conf/AppliedEnergy/Zhao20}. %Since the cooling subsystem is mainly airflow-based, \gls{dt} designers often opt for a \gls{cfd} approach to model the facility. %The reason why a digital twin might be needed for a cooling subsystem is primarily because of inefficient operational strategy. @@ -347,4 +345,4 @@ We distinguish 3 core elements of every \gls{dcdt}: %In your work, consider adding such an endpoint, albeit explain in future work that you envision \emph{implementing} this endpoint in the future. -\section{Discussion}\label{ss:background_discussion} +%\section{Discussion}\label{ss:background_discussion} diff --git a/content/implementation.tex b/content/implementation.tex index b289aa8..571afd3 100644 --- a/content/implementation.tex +++ b/content/implementation.tex @@ -120,10 +120,31 @@ Due to the massive volume of data incoming from the physical datacenter, the \co \code{Kafka} comes with excellent capability to efficiently compare data packets against a condition and filter our packets that are of no use to the \code{Analytics Engine} (see \Cref{lst:redis-sink}). On the contrary, the \code{PostgreSQL} sink (\myCircled{3}) contains all metrics collected by the datacenter sensors (see \Cref{lst:postgresql-sink}). This setup achieves excellent abstraction level, because only the most important metrics are forwarded to the \code{Analytics Engine}, with the majority of packets being filtered out. - \begin{figure}[t] \input{sources/listing_sinks.tex} \end{figure} + +\section{The \code{OpenDC} Scheduling Paradigm}\label{ss:opendc_scheduling} +In this section we introduce the scheduling paradigm of \code{OpenDC}. +\code{OpenDC}, a robust datacenter simulator uses discrete-event simulation. +``Discrete-event simulation models the operation of a system as a (discrete) sequence of events in time''~\cite{Wikipedia:article/DiscreteEventSimulation}. +Colloquially, it is akin to calling an \code{Update()} method on a set of objects to model changes in the simulator. +Scheduling in \code{OpenDC} also works using this method. +\Cref{fig:scheduling_in_opendc} represents how a task is assigned to a host in the simulation. + +A task (\myCircled{1}) is represented by its submission time, duration and computational requirements. +In order to be assigned a server to run on, it is deserialized into a \code{ServiceTask} (\myCircled{2}). +The \code{FilterScheduler} (\myCircled{4}) class takes care of scheduling the service task once the simulation reaches its submission time. +\code{OpenDC} maps tasks to available hosts via an \emph{allocation policy}~\cite{VUAmsterdam:thesis/AnaMariaMusca2025}. +In our work, this is always (also in \code{SmartScheduler}) the \code{FilterPolicy}. +In the \code{FilterPolicy}, a series of \code{HostFilter}s (\myCircled{3}) and \code{Host Weighter}s (\myCircled{5}) are used to find the matching host for the task. +In this project, the different filters and weighters take into account the default metrics (\ie \gls{cpu} capacity, \gls{ram} capacity, number of \gls{cpu} cores). +The host itself is represented as a \code{HostView} class (\myCircled{6}). +Importantly, the \code{HostView} class does not serve to simulate the behaviour of the host, but to provide a interface for the \emph{current state} of the host. +For actual computation, the \code{SimHost} (\myCircled{8}) class is used. +The \code{SimHost} object is created via the \code{HostProvisioningStep} class (\myCircled{7})~\cite{VUAmsterdam:thesis/AnaMariaMusca2025}. + + \section{Extensions to \code{OpenDC}}\label{ss:extensions} \code{OpenDC} is a state-of-the-art datacenter simulator. @@ -134,6 +155,9 @@ In order to turn it into a \gls{dt}, we have made several design decisions and e The new \code{SmartScheduler} is a scheduling mechanism capable of incorporating the insights from the \gls{dt} into its scheduling decisions. It relies on the functionality of the \code{HTTPClient} to poll the \gls{dt} at each scheduling step for potential insights. For example, if \gls{dt} sends to the datacenter a list of hosts likely to fail in the future, the \code{SmartScheduler} acts as \emph{system knobs} to enforce the \gls{dt} insights (\ie it can be mapped to \myCircled{2c} from \Cref{fig:implementation}). + It \emph{replaces} the \code{FilterScheduler} (\myCircled{4}) in \Cref{fig:scheduling_in_opendc}. + Importantly, it is functionally almost exactly the same as the \code{FilterScheduler}, with the exception that it can change the scheduling outcome based on the information from the \gls{dt}. + It is a \code{FilterScheduler} with an attached network socket. \item \textbf{\code{KafkaMonitor}}\\ The datacenter acts as the \emph{producer} of metrics, ingested by the \code{Kafka} topic (see \Cref{fig:flow_diagram}). We equip \code{OpenDC} with a new \code{ComputeMonitor} capable of exporting data directly into a \code{Kafka} topic. @@ -142,6 +166,12 @@ In order to turn it into a \gls{dt}, we have made several design decisions and e The \code{HTTPClient} offers the necessary functionality to communicate between the \gls{dt} and the datacenter. We decided to use the \gls{http} protocol for short, one-off communications between the \gls{dt} and the datacenter, as is common industry practice. \end{enumerate} +\begin{figure}[t] + \centering + \includegraphics[width=\linewidth]{images/scheduling_opendc.png} + \caption[OpenDC scheduling paradigm.]{The scheduling paradigm in \code{OpenDC}. Adapted from Musc{\u a} \etal~\cite{VUAmsterdam:thesis/AnaMariaMusca2025}. The highlighted \textcolor{Orchid}{\ding{110} \textbf{Filter Scheduler}} is the component that is substituted by the \code{SmartScheduler}.} + \label{fig:scheduling_in_opendc} +\end{figure} \section{Python Modules}\label{ss:programming} @@ -167,3 +197,4 @@ For future work, we envision a system that implements the reference architecture It contains a \code{while True} Python loop which contains the function call to fetch the latest changes to the \code{Redis} stream. Upon update, the \code{MonitoringService} informs the \code{AnalyticsEngine} that new data is awaiting \code{AnalyticsEngine} (\grayCircled{6}). \end{enumerate} + |
