summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorFabian Mastenbroek <mail.fabianm@gmail.com>2021-05-03 21:12:00 +0200
committerGitHub <noreply@github.com>2021-05-03 21:12:00 +0200
commit17ffe995ee06d5755cd3943a5ea14f982884009e (patch)
tree2f360f1974bdbfd5f7fcd3d8e9c142b28650d7ed /docs
parentc3ee3cb899d5259c619e79ff9f29984756c3813e (diff)
parentf081b9cab58a31a55c4f9dc6ccedd8cb92fc9242 (diff)
Prepare for OpenDC 2.0 release (v1)
This pull request performs several preparations for the official release of OpenDC 2.0. This pull request mostly focuses on documentation changes. ## Details 1. Restructure project documentation 2. Migrate to Dokka 1.4.32 3. Merge CI builds into single workflow
Diffstat (limited to 'docs')
-rw-r--r--docs/architecture.md25
-rw-r--r--docs/deploy.md65
-rw-r--r--docs/images/logo.pngbin0 -> 2825 bytes
-rw-r--r--docs/images/opendc-component-diagram.pngbin0 -> 39965 bytes
-rw-r--r--docs/images/opendc-frontend-construction.pngbin0 -> 76461 bytes
-rw-r--r--docs/images/opendc-frontend-simulation-zoom.pngbin0 -> 100583 bytes
-rw-r--r--docs/images/opendc-frontend-simulation.pngbin0 -> 96351 bytes
-rw-r--r--docs/research.md35
-rw-r--r--docs/toolchain.md70
9 files changed, 195 insertions, 0 deletions
diff --git a/docs/architecture.md b/docs/architecture.md
new file mode 100644
index 00000000..032e8f4b
--- /dev/null
+++ b/docs/architecture.md
@@ -0,0 +1,25 @@
+# Architecture
+
+OpenDC consists of four components: a Kotlin simulator, a MongoDB database, a Python
+Flask [API](/opendc-web/opendc-web-api), and a React.js [frontend](/opendc-web/opendc-web-ui).
+
+<p align="center">
+ <img src="https://raw.githubusercontent.com/atlarge-research/opendc/master/misc/artwork/opendc-component-diagram.png" alt="OpenDC Component Diagram">
+</p>
+
+On the frontend, users can construct a topology by specifying a datacenter's rooms, racks and machines, and create
+scenarios to see how a workload trace runs on that topology. The frontend communicates with the web server over
+SocketIO, through a custom REST request/response layer. For example, the frontend might make a `GET` request
+to `/api/v1/users/{userId}`, but this request is completed via SocketIO, not plain HTTP requests. Note that the API
+itself can also be accessed by HTTP.
+
+The (Swagger/OpenAPI compliant) API spec specifies what requests the frontend can make to the web server. To view this
+specification, go to the [Swagger Editor](https://editor.swagger.io/) and paste in
+our [opendc-api-spec.yml](../opendc-api-spec.yml).
+
+The web server receives API requests and processes them in the database. When the frontend requests to run a new
+scenario, the web server adds it to the `scenarios` collection in the database and sets its `state` as `QUEUED`.
+
+The simulator monitors the database for `QUEUED` scenarios, and simulates them as they are submitted. The results of the
+simulations are processed and aggregated in memory. Afterwards, the aggregated summary is written to the database, which
+the frontend can then again retrieve via the web server.
diff --git a/docs/deploy.md b/docs/deploy.md
new file mode 100644
index 00000000..b9d65d51
--- /dev/null
+++ b/docs/deploy.md
@@ -0,0 +1,65 @@
+# Deploying OpenDC
+
+### Preamble
+
+The official way to run OpenDC is using Docker. Other options include building and running locally, and building and
+running to deploy on a server.
+
+For all of these options, you have to create a Google API Console project and client ID, which the OpenDC frontend and
+web server will use to authenticate users and requests.
+Follow [these steps](https://developers.google.com/identity/sign-in/web/sign-in) to make such a project. In the '
+Authorized JavaScript origins' and 'Authorized redirect URI' fields, be sure to add `http://localhost:8080` (frontend)
+, `http://localhost:8081` (api) and `https://localhost:3000` (frontend dev). Download the JSON of the OAuth 2.0 client
+ID you created from the Credentials tab, and specifically note the `client_id`, which you'll need to build OpenDC.
+
+### Installing Docker
+
+OpenDC uses [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) to orchestrate the
+deployment of the software stack. Please refer to [Docker Desktop](https://www.docker.com/products/docker-desktop) for
+instructions on how install Docker on your machine.
+
+### Running OpenDC
+
+To build and run the full OpenDC stack locally on Linux or Mac, you first need to clone the project:
+
+```bash
+git clone https://github.com/atlarge-research/opendc.git
+
+# Enter the directory
+cd opendc/
+```
+
+In the directory you just entered, you need to set up a set of environment variables. To do this, create a file
+called `.env` in the `opendc` folder. In this file, replace `your-google-oauth-client-id` with your `client_id` from the
+OAuth client ID you created. For a standard setup, you can leave the other settings as-is.
+
+```.env
+MONGO_INITDB_ROOT_USERNAME=root
+MONGO_INITDB_ROOT_PASSWORD=rootpassword
+MONGO_INITDB_DATABASE=admin
+OPENDC_DB=opendc
+OPENDC_DB_USERNAME=opendc
+OPENDC_DB_PASSWORD=opendcpassword
+OPENDC_FLASK_SECRET="This is a secret flask key, please change"
+OPENDC_OAUTH_CLIENT_ID=your-google-oauth-client-id
+OPENDC_API_BASE_URL=http://localhost:8081
+```
+
+We provide a list of default traces for you to experiment with. If you want to add others, place them in the `traces`
+directory and add entries to the database (see also [the database folder](database/mongo-init-opendc-db.sh))
+
+If you plan to publicly deploy, please also tweak the other settings. In that case, also check the `docker-compose.yml`
+and `docker-compose.pod.yml` for further instructions.
+
+Now, start the server:
+
+```bash
+# Build the Docker image
+docker-compose build
+
+# Start the containers
+docker-compose up
+```
+
+Wait a few seconds and open `http://localhost:8080` in your browser to use OpenDC. We recommend Google Chrome for the
+best development experience.
diff --git a/docs/images/logo.png b/docs/images/logo.png
new file mode 100644
index 00000000..d743038b
--- /dev/null
+++ b/docs/images/logo.png
Binary files differ
diff --git a/docs/images/opendc-component-diagram.png b/docs/images/opendc-component-diagram.png
new file mode 100644
index 00000000..312ca72a
--- /dev/null
+++ b/docs/images/opendc-component-diagram.png
Binary files differ
diff --git a/docs/images/opendc-frontend-construction.png b/docs/images/opendc-frontend-construction.png
new file mode 100644
index 00000000..223e8d48
--- /dev/null
+++ b/docs/images/opendc-frontend-construction.png
Binary files differ
diff --git a/docs/images/opendc-frontend-simulation-zoom.png b/docs/images/opendc-frontend-simulation-zoom.png
new file mode 100644
index 00000000..d7744926
--- /dev/null
+++ b/docs/images/opendc-frontend-simulation-zoom.png
Binary files differ
diff --git a/docs/images/opendc-frontend-simulation.png b/docs/images/opendc-frontend-simulation.png
new file mode 100644
index 00000000..bbf4cbd6
--- /dev/null
+++ b/docs/images/opendc-frontend-simulation.png
Binary files differ
diff --git a/docs/research.md b/docs/research.md
new file mode 100644
index 00000000..1eeefecf
--- /dev/null
+++ b/docs/research.md
@@ -0,0 +1,35 @@
+## Research with OpenDC
+
+This document contains publications related to OpenDC or publications that have used OpenDC in their work.
+
+## Publications about OpenDC
+
+1. [OpenDC 2.0: Convenient Modeling and Simulation of Emerging Technologies in Cloud Datacenters](https://atlarge-research.com/pdfs/ccgrid21-opendc-paper.pdf)
+ **CCGrid 2021**
+ Fabian Mastenbroek, Georgios Andreadis, Soufiane Jounaid, Wenchen Lai, Jacob Burley, Jaro Bosch, Erwin van Eyk,
+ Laurens Versluis, Vincent van Beek, Alexandru Iosup
+1. [The OpenDC vision: Towards collaborative datacenter simulation and exploration for everybody](https://atlarge-research.com/pdfs/opendc-vision17ispdc_cr.pdf)
+ **ISPDC 2017 (invited paper)**
+ Alexandru Iosup, Georgios Andreadis, Vincent van Beek, Matthijs Bijman, Erwin van Eyk, Mihai Neacsu, Leon Overweel,
+ Sacheendra Talluri, Laurens Versluis, Maaike Visser.
+
+## Publications using OpenDC
+
+1. [A Reference Architecture for Datacenter Scheduling](https://arxiv.org/pdf/1808.04224)
+ **International Conference for High Performance Computing, Networking, Storage and Analysis 2018 (SC18)**
+ Georgios Andreadis, Laurens Versluis, Fabian S. Mastenbroek, Alexandru Iosup
+
+## Students using OpenDC
+
+1. [Capelin: Fast Data-Driven Capacity Planning for Cloud Datacenters](https://repository.tudelft.nl/islandora/object/uuid:d6d50861-86a3-4dd3-a13f-42d84db7af66?collection=education)
+ **Master Thesis, 2020**
+ Georgios Andreadis
+1. [OpenDC Serverless: Design, Implementation and Evaluation of a FaaS Platform Simulator](https://zenodo.org/record/4046675)
+ **Bachelor Thesis, 2020**
+ Soufiane Jounaid
+1. LEGO, but with Servers: Creating the Building Blocks to Design and Simulate Datacenters
+ **Bachelor Thesis, 2020**
+ Jacob Burley
+1. [A Systematic Design Space Exploration of Datacenter Schedulers](https://repository.tudelft.nl/islandora/object/uuid%3A20478016-cc7d-4c87-aa12-25b46f511277?collection=education)
+ **Bachelor Thesis, 2019**
+ Fabian S. Mastenbroek
diff --git a/docs/toolchain.md b/docs/toolchain.md
new file mode 100644
index 00000000..4b029ebc
--- /dev/null
+++ b/docs/toolchain.md
@@ -0,0 +1,70 @@
+# Toolchain Setup
+
+The OpenDC simulator is built using the [Kotlin](https://kotlinlang.org/) language. This is a JVM-based language that
+should appear familiar to programmers knowledgeable in Java or Scala. For a short interactive introduction to Kotlin,
+the [Learn Kotlin By Example](https://play.kotlinlang.org/byExample/overview) docs are a great place to start.
+
+For the build and dependency toolchain, we use [Gradle](https://gradle.org/). You will likely not need to change the
+Gradle build configurations of components, but you will use Gradle to execute builds and tests on the codebase.
+
+Follow the steps below to get it all set up!
+
+## Contents
+
+1. [Installing Java](#1-installing-java)
+1. [Building and Developing](#2-building-and-developing)
+ 1. [Setup with IntelliJ IDEA](#21-setup-with-intellij-idea)
+ 1. [Setup with Command Line](#22-setup-with-command-line)
+
+## 1. Installing Java
+
+Kotlin requires a Java installation of version 8 or higher. Make sure to install
+the [JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html), not only the JRE (the JDK also includes
+a JRE).
+
+## 2. Building and Developing
+
+With Java installed, we're ready to set up the development environment on your machine. You can either use a visual IDE
+or work from a command line shell. We outline both approaches below, feel free to choose which you are most comfortable
+with. If in doubt which one to choose, we suggest going with the first one.
+
+## 2.1 Setup with IntelliJ IDEA
+
+We suggest using [IntelliJ IDEA](https://www.jetbrains.com/idea/) as development environment. Once you have installed
+any version of this IDE on your machine, choose "Get from Version Control" in the new project dialogue.
+Enter `https://github.com/atlarge-research/opendc` as URL and submit your credentials when asked.
+Open the project once it's ready fetching the codebase, and let it set up with the defaults (IntelliJ will recognize
+that this is a Gradle codebase).
+
+You will now be prompted in a dialogue to enable auto-import for Gradle, which we suggest you do. Wait for any progress
+bars in the lower bar to disappear and then look for the Gradle context menu on the right-hand side. In it, go
+to `opendc > Tasks > verification > test`. This will build the codebase and run checks to verify that tests
+pass. If you get a `BUILD SUCCESSFUL` message, you're ready to go to the [next section](architecture.md)!
+
+## 2.2 Setup with Command Line
+
+First, clone the repository with the following command:
+
+```shell script
+git clone https://github.com/atlarge-research/opendc
+```
+
+And enter the directory:
+
+```shell script
+cd opendc
+```
+
+If on Windows, run the batch file included in the root, as follows:
+
+```commandline
+gradlew.bat test
+```
+
+If on Linux/macOS, run the shell script included in the root, as follows:
+
+```shell script
+./gradlew test
+```
+
+If the build is successful, you are ready to go to the next section!