From 8f75336cb802b770d1ac53efb155632dd2dd94d2 Mon Sep 17 00:00:00 2001 From: Fabian Mastenbroek Date: Thu, 8 Apr 2021 20:59:35 +0200 Subject: docs: Revamp project documentation This change updates the project documentation by moving most of the documentation to the docs directory. --- CONTRIBUTING.md | 20 ++-- README.md | 116 ++++++----------------- docs/architecture.md | 25 +++++ docs/deploy.md | 65 +++++++++++++ docs/images/logo.png | Bin 0 -> 2825 bytes docs/images/opendc-component-diagram.png | Bin 0 -> 39965 bytes docs/images/opendc-frontend-construction.png | Bin 0 -> 76461 bytes docs/images/opendc-frontend-simulation-zoom.png | Bin 0 -> 100583 bytes docs/images/opendc-frontend-simulation.png | Bin 0 -> 96351 bytes misc/artwork/logo.png | Bin 2825 -> 0 bytes misc/artwork/opendc-component-diagram.png | Bin 39965 -> 0 bytes misc/artwork/opendc-frontend-construction.png | Bin 76461 -> 0 bytes misc/artwork/opendc-frontend-simulation-zoom.png | Bin 100583 -> 0 bytes misc/artwork/opendc-frontend-simulation.png | Bin 96351 -> 0 bytes opendc-web/opendc-web-api/README.md | 4 +- opendc-web/opendc-web-ui/README.md | 2 +- 16 files changed, 136 insertions(+), 96 deletions(-) create mode 100644 docs/architecture.md create mode 100644 docs/deploy.md create mode 100644 docs/images/logo.png create mode 100644 docs/images/opendc-component-diagram.png create mode 100644 docs/images/opendc-frontend-construction.png create mode 100644 docs/images/opendc-frontend-simulation-zoom.png create mode 100644 docs/images/opendc-frontend-simulation.png delete mode 100644 misc/artwork/logo.png delete mode 100644 misc/artwork/opendc-component-diagram.png delete mode 100644 misc/artwork/opendc-frontend-construction.png delete mode 100644 misc/artwork/opendc-frontend-simulation-zoom.png delete mode 100644 misc/artwork/opendc-frontend-simulation.png diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7ed01f8a..e4109272 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,22 +1,26 @@ # Contributing to OpenDC -First of all, thanks for wanting to contribute! 🎉 - +First of all, thank you for wanting to contribute to OpenDC! 🎉 ## 💬 Have a question or general feedback relating to OpenDC? Contact us at 📧[opendc@atlarge-research.com](mailto:opendc@atlarge-research.com)! - ## 🐞 Want to report a bug or suggest a feature? -Please file an issue! First, have a look if someone has already filed an issue addressing your concern. If there already is such an issue, feel free to comment on the issue to show your support for it, or to add additional information that might be helpful. You can also just react with a thumbs-up 👍 to the issue, to indicate that you'd be interested in its resolution. This can help us prioritize what we spend our development time on. - -If you can't find an issue that fits your problem or feature request, open a new one. Describe actual and expected behavior, and be as detailed as you can. We'll get back to you asap! +Please file an issue! First, have a look if someone has already filed an issue addressing your concern. If there already +is such an issue, feel free to comment on the issue to show your support for it, or to add additional information that +might be helpful. You can also just react with a thumbs-up 👍 to the issue, to indicate that you'd be interested in its +resolution. This can help us prioritize what we spend our development time on. +If you can't find an issue that fits your problem or feature request, open a new one. Describe actual and expected +behavior, and be as detailed as you can. We'll get back to you asap! ## 💻 Want to contribute code? -That's great! If you want to contribute to this repository, [fork it](https://github.com/atlarge-research/opendc/new/master) and submit a PR here when you're ready! Be sure to describe *what* you changed and *why* you changed it, to help us understand what your contribution is about. +That's great! If you want to contribute to this +repository, [fork it](https://github.com/atlarge-research/opendc/new/master) and submit a PR here when you're ready! Be +sure to describe *what* you changed and *why* you changed it, to help us understand what your contribution is about. -A quick note on commit messages: Please follow common Git standards when writing commit messages, see [this post](https://chris.beams.io/posts/git-commit/) for details. +A quick note on commit messages: Please follow common Git standards when writing commit messages, +see [this post](https://chris.beams.io/posts/git-commit/) for details. diff --git a/README.md b/README.md index 3be20134..1d822e7e 100644 --- a/README.md +++ b/README.md @@ -1,104 +1,50 @@ -

- OpenDC -
- OpenDC -

-

- Collaborative Datacenter Simulation and Exploration for Everybody -

+ + OpenDC logo + -
+# OpenDC -OpenDC is an open-source simulator for datacenters aimed at both research and education. +Collaborative Datacenter Simulation and Exploration for Everybody -![opendc-frontend-construction](misc/artwork/opendc-frontend-construction.png) +[![MIT License](https://img.shields.io/badge/License-MIT-green.svg)](/LICENSE.txt) +[![Documentation](https://img.shields.io/badge/docs-master-green.svg)](./docs) +[![GitHub release](https://img.shields.io/github/release/atlarge-research/opendc)](https://github.com/atlarge-research/opendc/releases) +[![Build](https://github.com/atlarge-research/opendc/actions/workflows/build-simulator.yml/badge.svg)](https://github.com/atlarge-research/opendc/actions/workflows/build-simulator.yml) -Users can construct datacenters (see above) and define portfolios of scenarios (experiments) to see how these datacenters perform under different workloads and schedulers (see below). +----- -![opendc-frontend-simulation](misc/artwork/opendc-frontend-simulation.png) +OpenDC is an open-source simulation platform for datacenters aimed at both research and education. -The simulator is accessible both as a ready-to-use website hosted by us at [opendc.org](https://opendc.org), and as source code that users can run locally on their own machine, through Docker. +![Datacenter construction in OpenDC](docs/images/opendc-frontend-construction.png) -🛠 OpenDC is a project by the [@Large Research Group](https://atlarge-research.com). - -🐟 OpenDC comes bundled with [Capelin](https://repository.tudelft.nl/islandora/object/uuid:d6d50861-86a3-4dd3-a13f-42d84db7af66?collection=education), the capacity planning tool for cloud datacenters based on portfolios of what-if scenarios. More information on how to use and extend Capelin coming soon! - -## Architecture - -OpenDC consists of four components: a Kotlin simulator, a MongoDB database, a Python Flask [API](/opendc-web/opendc-web-api), and a React.js [frontend](/opendc-web/opendc-web-ui). - -

- OpenDC Component Diagram -

- -On the frontend, users can construct a topology by specifying a datacenter's rooms, racks and machines, and create scenarios to see how a workload trace runs on that topology. The frontend communicates with the web server over SocketIO, through a custom REST request/response layer. For example, the frontend might make a `GET` request to `/api/v1/users/{userId}`, but this request is completed via SocketIO, not plain HTTP requests. Note that the API itself can also be accessed by HTTP. - -The (Swagger/OpenAPI compliant) API spec specifies what requests the frontend can make to the web server. To view this specification, go to the [Swagger Editor](https://editor.swagger.io/) and paste in our [opendc-api-spec.yml](opendc-api-spec.yml). - -The web server receives API requests and processes them in the database. When the frontend requests to run a new scenario, the web server adds it to the `scenarios` collection in the database and sets its `state` as `QUEUED`. - -The simulator monitors the database for `QUEUED` scenarios, and simulates them as they are submitted. The results -of the simulations are processed and aggregated in memory. Afterwards, the aggregated summary is written to the database, -which the frontend can then again retrieve via the web server. - -## Setup +Users can construct datacenters (see above) and define portfolios of scenarios (experiments) to see how these +datacenters perform under different workloads and schedulers (see below). -### Preamble +![Datacenter simulation in OpenDC](docs/images/opendc-frontend-simulation.png) -The official way to run OpenDC is using Docker. Other options include building and running locally, and building and running to deploy on a server. +The simulator is accessible both as a ready-to-use website hosted by us at [opendc.org](https://opendc.org), and as +source code that users can run locally on their own machine, through Docker. -For all of these options, you have to create a Google API Console project and client ID, which the OpenDC frontend and web server will use to authenticate users and requests. Follow [these steps](https://developers.google.com/identity/sign-in/web/sign-in) to make such a project. In the 'Authorized JavaScript origins' and 'Authorized redirect URI' fields, be sure to add `http://localhost:8080` (frontend), `http://localhost:8081` (api) and `https://localhost:3000` (frontend dev). Download the JSON of the OAuth 2.0 client ID you created from the Credentials tab, and specifically note the `client_id`, which you'll need to build OpenDC. - -### Installing Docker -OpenDC uses [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) to orchestrate the -deployment of the software stack. Please refer to [Docker Desktop](https://www.docker.com/products/docker-desktop) for -instructions on how install Docker on your machine. - - -### Running OpenDC - -To build and run the full OpenDC stack locally on Linux or Mac, you first need to clone the project: - -```bash -git clone https://github.com/atlarge-research/opendc.git - -# Enter the directory -cd opendc/ -``` - -In the directory you just entered, you need to set up a set of environment variables. -To do this, create a file called `.env` in the `opendc` folder. In this file, -replace `your-google-oauth-client-id` with your `client_id` from the OAuth client ID you created. -For a standard setup, you can leave the other settings as-is. - -```.env -MONGO_INITDB_ROOT_USERNAME=root -MONGO_INITDB_ROOT_PASSWORD=rootpassword -MONGO_INITDB_DATABASE=admin -OPENDC_DB=opendc -OPENDC_DB_USERNAME=opendc -OPENDC_DB_PASSWORD=opendcpassword -OPENDC_FLASK_SECRET="This is a secret flask key, please change" -OPENDC_OAUTH_CLIENT_ID=your-google-oauth-client-id -OPENDC_API_BASE_URL=http://localhost:8081 -``` +🛠 OpenDC is a project by the [@Large Research Group](https://atlarge-research.com). -We provide a list of default traces for you to experiment with. If you want to add others, place them in the `traces` directory and add entries to the database (see also [the database folder](database/mongo-init-opendc-db.sh)) +🐟 OpenDC comes bundled +with [Capelin](https://repository.tudelft.nl/islandora/object/uuid:d6d50861-86a3-4dd3-a13f-42d84db7af66?collection=education) +, the capacity planning tool for cloud datacenters based on portfolios of what-if scenarios. More information on how to +use and extend Capelin coming soon! -If you plan to publicly deploy, please also tweak the other settings. -In that case, also check the `docker-compose.yml` and `docker-compose.pod.yml` for further instructions. +## Documentation -Now, start the server: +The documentation is located in the [docs/](docs) directory and is divided as follows: -```bash -# Build the Docker image -docker-compose build +1. [Deployment Guide](docs/deploy.md) +1. [Architectural Overview](docs/architecture.md) +1. [Contributing Guide](CONTRIBUTING.md) -# Start the containers -docker-compose up -``` +## Contributing -Wait a few seconds and open `http://localhost:8080` in your browser to use OpenDC. We recommend Google Chrome for the best development experience. +Questions, suggestions and contributions are welcome and appreciated! +Please refer to the [contributing guidelines](CONTRIBUTING.md) for more details. ## License + OpenDC is distributed under the MIT license. See [LICENSE.txt](/LICENSE.txt). diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..032e8f4b --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,25 @@ +# Architecture + +OpenDC consists of four components: a Kotlin simulator, a MongoDB database, a Python +Flask [API](/opendc-web/opendc-web-api), and a React.js [frontend](/opendc-web/opendc-web-ui). + +

+ OpenDC Component Diagram +

+ +On the frontend, users can construct a topology by specifying a datacenter's rooms, racks and machines, and create +scenarios to see how a workload trace runs on that topology. The frontend communicates with the web server over +SocketIO, through a custom REST request/response layer. For example, the frontend might make a `GET` request +to `/api/v1/users/{userId}`, but this request is completed via SocketIO, not plain HTTP requests. Note that the API +itself can also be accessed by HTTP. + +The (Swagger/OpenAPI compliant) API spec specifies what requests the frontend can make to the web server. To view this +specification, go to the [Swagger Editor](https://editor.swagger.io/) and paste in +our [opendc-api-spec.yml](../opendc-api-spec.yml). + +The web server receives API requests and processes them in the database. When the frontend requests to run a new +scenario, the web server adds it to the `scenarios` collection in the database and sets its `state` as `QUEUED`. + +The simulator monitors the database for `QUEUED` scenarios, and simulates them as they are submitted. The results of the +simulations are processed and aggregated in memory. Afterwards, the aggregated summary is written to the database, which +the frontend can then again retrieve via the web server. diff --git a/docs/deploy.md b/docs/deploy.md new file mode 100644 index 00000000..b9d65d51 --- /dev/null +++ b/docs/deploy.md @@ -0,0 +1,65 @@ +# Deploying OpenDC + +### Preamble + +The official way to run OpenDC is using Docker. Other options include building and running locally, and building and +running to deploy on a server. + +For all of these options, you have to create a Google API Console project and client ID, which the OpenDC frontend and +web server will use to authenticate users and requests. +Follow [these steps](https://developers.google.com/identity/sign-in/web/sign-in) to make such a project. In the ' +Authorized JavaScript origins' and 'Authorized redirect URI' fields, be sure to add `http://localhost:8080` (frontend) +, `http://localhost:8081` (api) and `https://localhost:3000` (frontend dev). Download the JSON of the OAuth 2.0 client +ID you created from the Credentials tab, and specifically note the `client_id`, which you'll need to build OpenDC. + +### Installing Docker + +OpenDC uses [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) to orchestrate the +deployment of the software stack. Please refer to [Docker Desktop](https://www.docker.com/products/docker-desktop) for +instructions on how install Docker on your machine. + +### Running OpenDC + +To build and run the full OpenDC stack locally on Linux or Mac, you first need to clone the project: + +```bash +git clone https://github.com/atlarge-research/opendc.git + +# Enter the directory +cd opendc/ +``` + +In the directory you just entered, you need to set up a set of environment variables. To do this, create a file +called `.env` in the `opendc` folder. In this file, replace `your-google-oauth-client-id` with your `client_id` from the +OAuth client ID you created. For a standard setup, you can leave the other settings as-is. + +```.env +MONGO_INITDB_ROOT_USERNAME=root +MONGO_INITDB_ROOT_PASSWORD=rootpassword +MONGO_INITDB_DATABASE=admin +OPENDC_DB=opendc +OPENDC_DB_USERNAME=opendc +OPENDC_DB_PASSWORD=opendcpassword +OPENDC_FLASK_SECRET="This is a secret flask key, please change" +OPENDC_OAUTH_CLIENT_ID=your-google-oauth-client-id +OPENDC_API_BASE_URL=http://localhost:8081 +``` + +We provide a list of default traces for you to experiment with. If you want to add others, place them in the `traces` +directory and add entries to the database (see also [the database folder](database/mongo-init-opendc-db.sh)) + +If you plan to publicly deploy, please also tweak the other settings. In that case, also check the `docker-compose.yml` +and `docker-compose.pod.yml` for further instructions. + +Now, start the server: + +```bash +# Build the Docker image +docker-compose build + +# Start the containers +docker-compose up +``` + +Wait a few seconds and open `http://localhost:8080` in your browser to use OpenDC. We recommend Google Chrome for the +best development experience. diff --git a/docs/images/logo.png b/docs/images/logo.png new file mode 100644 index 00000000..d743038b Binary files /dev/null and b/docs/images/logo.png differ diff --git a/docs/images/opendc-component-diagram.png b/docs/images/opendc-component-diagram.png new file mode 100644 index 00000000..312ca72a Binary files /dev/null and b/docs/images/opendc-component-diagram.png differ diff --git a/docs/images/opendc-frontend-construction.png b/docs/images/opendc-frontend-construction.png new file mode 100644 index 00000000..223e8d48 Binary files /dev/null and b/docs/images/opendc-frontend-construction.png differ diff --git a/docs/images/opendc-frontend-simulation-zoom.png b/docs/images/opendc-frontend-simulation-zoom.png new file mode 100644 index 00000000..d7744926 Binary files /dev/null and b/docs/images/opendc-frontend-simulation-zoom.png differ diff --git a/docs/images/opendc-frontend-simulation.png b/docs/images/opendc-frontend-simulation.png new file mode 100644 index 00000000..bbf4cbd6 Binary files /dev/null and b/docs/images/opendc-frontend-simulation.png differ diff --git a/misc/artwork/logo.png b/misc/artwork/logo.png deleted file mode 100644 index d743038b..00000000 Binary files a/misc/artwork/logo.png and /dev/null differ diff --git a/misc/artwork/opendc-component-diagram.png b/misc/artwork/opendc-component-diagram.png deleted file mode 100644 index 312ca72a..00000000 Binary files a/misc/artwork/opendc-component-diagram.png and /dev/null differ diff --git a/misc/artwork/opendc-frontend-construction.png b/misc/artwork/opendc-frontend-construction.png deleted file mode 100644 index 223e8d48..00000000 Binary files a/misc/artwork/opendc-frontend-construction.png and /dev/null differ diff --git a/misc/artwork/opendc-frontend-simulation-zoom.png b/misc/artwork/opendc-frontend-simulation-zoom.png deleted file mode 100644 index d7744926..00000000 Binary files a/misc/artwork/opendc-frontend-simulation-zoom.png and /dev/null differ diff --git a/misc/artwork/opendc-frontend-simulation.png b/misc/artwork/opendc-frontend-simulation.png deleted file mode 100644 index bbf4cbd6..00000000 Binary files a/misc/artwork/opendc-frontend-simulation.png and /dev/null differ diff --git a/opendc-web/opendc-web-api/README.md b/opendc-web/opendc-web-api/README.md index 182cd803..4932f823 100644 --- a/opendc-web/opendc-web-api/README.md +++ b/opendc-web/opendc-web-api/README.md @@ -1,5 +1,5 @@

- OpenDC + OpenDC
OpenDC Web Server

@@ -9,7 +9,7 @@
-The OpenDC web server is the bridge between OpenDC's frontend and database. It is built with Flask/SocketIO in Python and implements the OpenAPI-compliant [OpenDC API specification](../opendc-api-spec.yml). +The OpenDC web server is the bridge between OpenDC's frontend and database. It is built with Flask/SocketIO in Python and implements the OpenAPI-compliant [OpenDC API specification](../../opendc-api-spec.yml). This document explains a high-level view of the web server architecture ([jump](#architecture)), and describes how to set up the web server for local development ([jump](#setup-for-local-development)). diff --git a/opendc-web/opendc-web-ui/README.md b/opendc-web/opendc-web-ui/README.md index ae556b09..f3a58e7a 100644 --- a/opendc-web/opendc-web-ui/README.md +++ b/opendc-web/opendc-web-ui/README.md @@ -1,5 +1,5 @@

- OpenDC + OpenDC
OpenDC Frontend

-- cgit v1.2.3