diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/architecture.md | 25 | ||||
| -rw-r--r-- | docs/deploy.md | 65 | ||||
| -rw-r--r-- | docs/images/logo.png | bin | 0 -> 2825 bytes | |||
| -rw-r--r-- | docs/images/opendc-component-diagram.png | bin | 0 -> 39965 bytes | |||
| -rw-r--r-- | docs/images/opendc-frontend-construction.png | bin | 0 -> 76461 bytes | |||
| -rw-r--r-- | docs/images/opendc-frontend-simulation-zoom.png | bin | 0 -> 100583 bytes | |||
| -rw-r--r-- | docs/images/opendc-frontend-simulation.png | bin | 0 -> 96351 bytes | |||
| -rw-r--r-- | docs/research.md | 35 | ||||
| -rw-r--r-- | docs/toolchain.md | 70 |
9 files changed, 195 insertions, 0 deletions
diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..032e8f4b --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,25 @@ +# Architecture + +OpenDC consists of four components: a Kotlin simulator, a MongoDB database, a Python +Flask [API](/opendc-web/opendc-web-api), and a React.js [frontend](/opendc-web/opendc-web-ui). + +<p align="center"> + <img src="https://raw.githubusercontent.com/atlarge-research/opendc/master/misc/artwork/opendc-component-diagram.png" alt="OpenDC Component Diagram"> +</p> + +On the frontend, users can construct a topology by specifying a datacenter's rooms, racks and machines, and create +scenarios to see how a workload trace runs on that topology. The frontend communicates with the web server over +SocketIO, through a custom REST request/response layer. For example, the frontend might make a `GET` request +to `/api/v1/users/{userId}`, but this request is completed via SocketIO, not plain HTTP requests. Note that the API +itself can also be accessed by HTTP. + +The (Swagger/OpenAPI compliant) API spec specifies what requests the frontend can make to the web server. To view this +specification, go to the [Swagger Editor](https://editor.swagger.io/) and paste in +our [opendc-api-spec.yml](../opendc-api-spec.yml). + +The web server receives API requests and processes them in the database. When the frontend requests to run a new +scenario, the web server adds it to the `scenarios` collection in the database and sets its `state` as `QUEUED`. + +The simulator monitors the database for `QUEUED` scenarios, and simulates them as they are submitted. The results of the +simulations are processed and aggregated in memory. Afterwards, the aggregated summary is written to the database, which +the frontend can then again retrieve via the web server. diff --git a/docs/deploy.md b/docs/deploy.md new file mode 100644 index 00000000..b9d65d51 --- /dev/null +++ b/docs/deploy.md @@ -0,0 +1,65 @@ +# Deploying OpenDC + +### Preamble + +The official way to run OpenDC is using Docker. Other options include building and running locally, and building and +running to deploy on a server. + +For all of these options, you have to create a Google API Console project and client ID, which the OpenDC frontend and +web server will use to authenticate users and requests. +Follow [these steps](https://developers.google.com/identity/sign-in/web/sign-in) to make such a project. In the ' +Authorized JavaScript origins' and 'Authorized redirect URI' fields, be sure to add `http://localhost:8080` (frontend) +, `http://localhost:8081` (api) and `https://localhost:3000` (frontend dev). Download the JSON of the OAuth 2.0 client +ID you created from the Credentials tab, and specifically note the `client_id`, which you'll need to build OpenDC. + +### Installing Docker + +OpenDC uses [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) to orchestrate the +deployment of the software stack. Please refer to [Docker Desktop](https://www.docker.com/products/docker-desktop) for +instructions on how install Docker on your machine. + +### Running OpenDC + +To build and run the full OpenDC stack locally on Linux or Mac, you first need to clone the project: + +```bash +git clone https://github.com/atlarge-research/opendc.git + +# Enter the directory +cd opendc/ +``` + +In the directory you just entered, you need to set up a set of environment variables. To do this, create a file +called `.env` in the `opendc` folder. In this file, replace `your-google-oauth-client-id` with your `client_id` from the +OAuth client ID you created. For a standard setup, you can leave the other settings as-is. + +```.env +MONGO_INITDB_ROOT_USERNAME=root +MONGO_INITDB_ROOT_PASSWORD=rootpassword +MONGO_INITDB_DATABASE=admin +OPENDC_DB=opendc +OPENDC_DB_USERNAME=opendc +OPENDC_DB_PASSWORD=opendcpassword +OPENDC_FLASK_SECRET="This is a secret flask key, please change" +OPENDC_OAUTH_CLIENT_ID=your-google-oauth-client-id +OPENDC_API_BASE_URL=http://localhost:8081 +``` + +We provide a list of default traces for you to experiment with. If you want to add others, place them in the `traces` +directory and add entries to the database (see also [the database folder](database/mongo-init-opendc-db.sh)) + +If you plan to publicly deploy, please also tweak the other settings. In that case, also check the `docker-compose.yml` +and `docker-compose.pod.yml` for further instructions. + +Now, start the server: + +```bash +# Build the Docker image +docker-compose build + +# Start the containers +docker-compose up +``` + +Wait a few seconds and open `http://localhost:8080` in your browser to use OpenDC. We recommend Google Chrome for the +best development experience. diff --git a/docs/images/logo.png b/docs/images/logo.png Binary files differnew file mode 100644 index 00000000..d743038b --- /dev/null +++ b/docs/images/logo.png diff --git a/docs/images/opendc-component-diagram.png b/docs/images/opendc-component-diagram.png Binary files differnew file mode 100644 index 00000000..312ca72a --- /dev/null +++ b/docs/images/opendc-component-diagram.png diff --git a/docs/images/opendc-frontend-construction.png b/docs/images/opendc-frontend-construction.png Binary files differnew file mode 100644 index 00000000..223e8d48 --- /dev/null +++ b/docs/images/opendc-frontend-construction.png diff --git a/docs/images/opendc-frontend-simulation-zoom.png b/docs/images/opendc-frontend-simulation-zoom.png Binary files differnew file mode 100644 index 00000000..d7744926 --- /dev/null +++ b/docs/images/opendc-frontend-simulation-zoom.png diff --git a/docs/images/opendc-frontend-simulation.png b/docs/images/opendc-frontend-simulation.png Binary files differnew file mode 100644 index 00000000..bbf4cbd6 --- /dev/null +++ b/docs/images/opendc-frontend-simulation.png diff --git a/docs/research.md b/docs/research.md new file mode 100644 index 00000000..1eeefecf --- /dev/null +++ b/docs/research.md @@ -0,0 +1,35 @@ +## Research with OpenDC + +This document contains publications related to OpenDC or publications that have used OpenDC in their work. + +## Publications about OpenDC + +1. [OpenDC 2.0: Convenient Modeling and Simulation of Emerging Technologies in Cloud Datacenters](https://atlarge-research.com/pdfs/ccgrid21-opendc-paper.pdf) + **CCGrid 2021** + Fabian Mastenbroek, Georgios Andreadis, Soufiane Jounaid, Wenchen Lai, Jacob Burley, Jaro Bosch, Erwin van Eyk, + Laurens Versluis, Vincent van Beek, Alexandru Iosup +1. [The OpenDC vision: Towards collaborative datacenter simulation and exploration for everybody](https://atlarge-research.com/pdfs/opendc-vision17ispdc_cr.pdf) + **ISPDC 2017 (invited paper)** + Alexandru Iosup, Georgios Andreadis, Vincent van Beek, Matthijs Bijman, Erwin van Eyk, Mihai Neacsu, Leon Overweel, + Sacheendra Talluri, Laurens Versluis, Maaike Visser. + +## Publications using OpenDC + +1. [A Reference Architecture for Datacenter Scheduling](https://arxiv.org/pdf/1808.04224) + **International Conference for High Performance Computing, Networking, Storage and Analysis 2018 (SC18)** + Georgios Andreadis, Laurens Versluis, Fabian S. Mastenbroek, Alexandru Iosup + +## Students using OpenDC + +1. [Capelin: Fast Data-Driven Capacity Planning for Cloud Datacenters](https://repository.tudelft.nl/islandora/object/uuid:d6d50861-86a3-4dd3-a13f-42d84db7af66?collection=education) + **Master Thesis, 2020** + Georgios Andreadis +1. [OpenDC Serverless: Design, Implementation and Evaluation of a FaaS Platform Simulator](https://zenodo.org/record/4046675) + **Bachelor Thesis, 2020** + Soufiane Jounaid +1. LEGO, but with Servers: Creating the Building Blocks to Design and Simulate Datacenters + **Bachelor Thesis, 2020** + Jacob Burley +1. [A Systematic Design Space Exploration of Datacenter Schedulers](https://repository.tudelft.nl/islandora/object/uuid%3A20478016-cc7d-4c87-aa12-25b46f511277?collection=education) + **Bachelor Thesis, 2019** + Fabian S. Mastenbroek diff --git a/docs/toolchain.md b/docs/toolchain.md new file mode 100644 index 00000000..4b029ebc --- /dev/null +++ b/docs/toolchain.md @@ -0,0 +1,70 @@ +# Toolchain Setup + +The OpenDC simulator is built using the [Kotlin](https://kotlinlang.org/) language. This is a JVM-based language that +should appear familiar to programmers knowledgeable in Java or Scala. For a short interactive introduction to Kotlin, +the [Learn Kotlin By Example](https://play.kotlinlang.org/byExample/overview) docs are a great place to start. + +For the build and dependency toolchain, we use [Gradle](https://gradle.org/). You will likely not need to change the +Gradle build configurations of components, but you will use Gradle to execute builds and tests on the codebase. + +Follow the steps below to get it all set up! + +## Contents + +1. [Installing Java](#1-installing-java) +1. [Building and Developing](#2-building-and-developing) + 1. [Setup with IntelliJ IDEA](#21-setup-with-intellij-idea) + 1. [Setup with Command Line](#22-setup-with-command-line) + +## 1. Installing Java + +Kotlin requires a Java installation of version 8 or higher. Make sure to install +the [JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html), not only the JRE (the JDK also includes +a JRE). + +## 2. Building and Developing + +With Java installed, we're ready to set up the development environment on your machine. You can either use a visual IDE +or work from a command line shell. We outline both approaches below, feel free to choose which you are most comfortable +with. If in doubt which one to choose, we suggest going with the first one. + +## 2.1 Setup with IntelliJ IDEA + +We suggest using [IntelliJ IDEA](https://www.jetbrains.com/idea/) as development environment. Once you have installed +any version of this IDE on your machine, choose "Get from Version Control" in the new project dialogue. +Enter `https://github.com/atlarge-research/opendc` as URL and submit your credentials when asked. +Open the project once it's ready fetching the codebase, and let it set up with the defaults (IntelliJ will recognize +that this is a Gradle codebase). + +You will now be prompted in a dialogue to enable auto-import for Gradle, which we suggest you do. Wait for any progress +bars in the lower bar to disappear and then look for the Gradle context menu on the right-hand side. In it, go +to `opendc > Tasks > verification > test`. This will build the codebase and run checks to verify that tests +pass. If you get a `BUILD SUCCESSFUL` message, you're ready to go to the [next section](architecture.md)! + +## 2.2 Setup with Command Line + +First, clone the repository with the following command: + +```shell script +git clone https://github.com/atlarge-research/opendc +``` + +And enter the directory: + +```shell script +cd opendc +``` + +If on Windows, run the batch file included in the root, as follows: + +```commandline +gradlew.bat test +``` + +If on Linux/macOS, run the shell script included in the root, as follows: + +```shell script +./gradlew test +``` + +If the build is successful, you are ready to go to the next section! |
