1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
|
\begin{appendices}
\chapter{Reproducibility}\label{s:reproducibility}
\section{Artifact Checklist}\label{ss:artifact_checklist}
\begin{enumerate}[label=\textcolor{Green}{\textbf{\ding{51}}}]
\item Program(s): \code{OpenDC}, \code{Confluent Kafka}, \code{PostgreSQL}, \code{Redis}, \code{Grafana}.
\item Data set(s): Failure traces from Talluri \etal and failure models from Javadi \etal (see also \url{https://opendc.org/learn/documentation/Input/FailureModel}).
\item Experiments: in \Cref{s:evaluation}.
\item Time is needed to reproduce the experiments: around 4 hours.
\item Public Experiment Archive: \url{https://git.denounce.ai}.
\end{enumerate}
\section{Experimental Setup}\label{ss:experimental_setup_appendix}
Confluent Kafka, (see \url{https://www.confluent.io/}) must be present on the system.
This includes the two different connectors from \Cref{s:implementation}.
The scripts to start the Confluent Kafka (assuming installation in \code{/opt/confluent} instance are listed below. First, start Kafka Connect:
\begin{verbatim}
/opt/confluent/bin/connect-standalone \
/opt/confluent/etc/kafka/connect-standalone.properties \
/opt/confluent/share/confluent-common/connectors/sink-jdbc.properties \
/opt/confluent/share/confluent-common/connectors/sink-redis.properties
\end{verbatim}
Then, format Kafka Storage:
\begin{verbatim}
/opt/confluent/bin/kafka-storage format -t \
2vi2WtHxQAOPyXb1Bj1Jvw -c /opt/confluent/etc/kafka/server.properties \
--standalone > /dev/null 2>&1; echo 0
\end{verbatim}
Afterwards, start Kafka Broker:
\begin{verbatim}
/opt/confluent/bin/kafka-server-start /opt/confluent/etc/kafka/server.properties
\end{verbatim}
Then create a new Kafka Topic (\eg \code{postgres\_topic}):
\begin{verbatim}
kafka-topics --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1
\end{verbatim}
Lastly, run the Schema Registry:
\begin{verbatim}
/opt/confluent/bin/schema-registry-start \
/opt/confluent/etc/schema-registry/schema-registry.properties
\end{verbatim}
At this point, ensure \code{PostgreSQL}, \code{Redi} and \code{Grafana} are up and running on the system, on their default port configurations.
To see metrics flow seamlessly into \code{PostgreSQL}, once can run the following command (\ie assuming the database name is \code{opendc}):
\begin{verbatim}
sudo su postgres; psql -d opendc
\end{verbatim}
Additionally, to interact with the \code{Redis} cache, we suggest to use the excellent \code{redis-cli} tool:
\begin{verbatim}
redis-cli -h localhost -p 6379
\end{verbatim}
The following commands can be used to list the contents of the cache (\ie assuming stream name \code{postgres\_topic}), and to clear the cache respectively:
\begin{verbatim}
XRANGE postgres_topic - +
XTRIM postgres_topic MAXLEN 0
\end{verbatim}
For the predictive analytics in our implementation it is useful to create a separate consumer group:
\begin{verbatim}
XGROUP CREATE mystream mygroup 0
\end{verbatim}
Lastly, each time you change the database schema, you must run (\ie assuming \code{schema.proto} lives in \code{resources/experiments}):
\begin{verbatim}
cd resources/experiments/;
protoc --java_out=/home/matt/git/opendc/opendc-common/src/main/java/ schema.proto
\end{verbatim}
Lastly, one has to checkout the \url{https://git.denounce.ai/opendc.git} and \url{https://git.denounce.ai/sunfish.git} repositories on their local machine.
\code{sunfish.git} acts as the digital twin and \code{opendc.git} acts as the physical datacenter.
Both contain modifications.
Change the directory to \code{sunfish.git}.
Create a \code{.venv} in \code{python\_scripts} and \code{http\_server} and install all the dependencies.
Then, run respectively the HTTP Server and the Python Scripts.
Afterwards, start \mysystem using IntellijIDEA.
\section{Configuration files}\label{ss:configuration_files}
\input{sources/config_files.tex}
\end{appendices}
|
