From b593268b50784964c672f8b5aaa857e9b9243634 Mon Sep 17 00:00:00 2001 From: Georgios Andreadis Date: Fri, 20 Dec 2019 11:26:33 +0100 Subject: Adapt documentation outline and README --- README.md | 11 +++++++++++ docs/architecture.md | 4 ++++ docs/models.md | 4 ++++ docs/run.md | 4 ++++ docs/setup.md | 18 ++++++++++++++++++ odcsim/docs/build.md | 5 ++++- 6 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 docs/architecture.md create mode 100644 docs/models.md create mode 100644 docs/run.md create mode 100644 docs/setup.md diff --git a/README.md b/README.md index d3cae15f..68feccc9 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,17 @@ This repository hosts the source code and development of the simulation componen - **[opendc](/opendc)** A collection of models for simulating clouds, datacenters and their components using the **odcsim** framework. +## Documentation +Check out the [Getting Started](#getting-started) section for a quick +overview. +The documentation is located in the [docs/](docs/) directory and is divided as follows: +1. [Toolchain Setup](docs/setup.md) +2. [Architecture Overview](docs/architecture.md) +3. [Pre-built Models](docs/models.md) +4. [Running a Model](docs/run.md) +5. [Building Your Own Model](docs/build.md) +6. [API Reference](https://atlarge-research.github.io/opendc-simulator) **(TODO maybe remove this until we have created this - see #36)** + ## Contributing ### Contributing Guide Read our [contributing guide](CONTRIBUTING.md) to learn about our diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..d26a8749 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,4 @@ +# 2. Architecture Overview + +--- +[< Previous](setup.md) | [Next >](models.md) diff --git a/docs/models.md b/docs/models.md new file mode 100644 index 00000000..447f50b7 --- /dev/null +++ b/docs/models.md @@ -0,0 +1,4 @@ +# 2. Architecture Overview + +--- +[< Previous](architecture.md) | [Next >](models.md) diff --git a/docs/run.md b/docs/run.md new file mode 100644 index 00000000..c6e26bc7 --- /dev/null +++ b/docs/run.md @@ -0,0 +1,4 @@ +# 4. Running a Model + +--- +[< Previous](models.md) | [Next >](build.md) diff --git a/docs/setup.md b/docs/setup.md new file mode 100644 index 00000000..f7ab9f12 --- /dev/null +++ b/docs/setup.md @@ -0,0 +1,18 @@ +# 1. Toolchain Setup + +**TODO: Introduction: What basic libraries/language(s) do we use, with links to their respective sites. What tool do we recommend (IDEA probably). Explain that we will now give two alternative ways (IDEA and shell).** + +## Installing Java + +**TODO: Explain which version they need and where they can get it.** + +## Setup With IntelliJ IDEA + +**TODO: Take the user from not having IDEA to having the opendc-simulator codebase open in the environment. Finish with running a test to see if it works.** + +## Setup With Shell + +**TODO: Take the user from not having the codebase to having it and having all dependencies fetched with Gradle. This will likely need separate instructions for Windows and for Linux/MacOS. Advise maybe VS Code as editor to view things after installation. Finish with running a test to see if it works.** + +--- +[Next >](architecture.md) diff --git a/odcsim/docs/build.md b/odcsim/docs/build.md index f442e70b..761fb71a 100644 --- a/odcsim/docs/build.md +++ b/odcsim/docs/build.md @@ -1 +1,4 @@ -# Building a Simulation Model +# Building Your Own Model + +--- +[< Previous](run.md) -- cgit v1.2.3 From 773eaf770313fa245250e8e1661ee50444223c98 Mon Sep 17 00:00:00 2001 From: Georgios Andreadis Date: Fri, 20 Dec 2019 11:37:25 +0100 Subject: Create guidelines for documentation sections --- docs/architecture.md | 4 ++++ docs/models.md | 6 +++++- docs/run.md | 2 ++ odcsim/docs/build.md | 4 +++- 4 files changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index d26a8749..a854bf6d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,4 +1,8 @@ # 2. Architecture Overview +**TODO: What are the main components (odcsim-core, odcsim-engine, opendc-core, opendc-format, opendc-testkit, opendc-workflows, I assume)? Short description of each in a bullet list (or even a schema drawing, if you have the time). Why this division?** + +**TODO: One section per component, explaining what their responsibility is and what other modules they talk to. In what kind of scenarios do you as OpenDC developer need to touch/change them?** + --- [< Previous](setup.md) | [Next >](models.md) diff --git a/docs/models.md b/docs/models.md index 447f50b7..fab8cdcf 100644 --- a/docs/models.md +++ b/docs/models.md @@ -1,4 +1,8 @@ -# 2. Architecture Overview +# 3. Pre-built Models + +**TODO: Introduction: we have a library of pre-built models to make things easier for you, here we present them etc.** + +**Per model in the architecture (corresponding with the architecture document), give the list of models we include out of the box. Explain what it does (and if applicable, what it doesn't do).** --- [< Previous](architecture.md) | [Next >](models.md) diff --git a/docs/run.md b/docs/run.md index c6e26bc7..d3143d96 100644 --- a/docs/run.md +++ b/docs/run.md @@ -1,4 +1,6 @@ # 4. Running a Model +**TODO: Not sure yet what's useful to include here. Maybe give an example of how to run one of the pre-builts?** + --- [< Previous](models.md) | [Next >](build.md) diff --git a/odcsim/docs/build.md b/odcsim/docs/build.md index 761fb71a..21983355 100644 --- a/odcsim/docs/build.md +++ b/odcsim/docs/build.md @@ -1,4 +1,6 @@ -# Building Your Own Model +# 5. Building Your Own Model + +**TODO: What goes into building a model? Where do you start? Can be quite technical (what gradle scripts do I need to make, what directory structure, ...), maybe center around a simple example scenario?** --- [< Previous](run.md) -- cgit v1.2.3 From f33cfe79358c31cc786fdb1594cb1c40ab8e0a7b Mon Sep 17 00:00:00 2001 From: Georgios Andreadis Date: Fri, 14 Feb 2020 11:37:32 +0100 Subject: Get first two parts of docs up to date and remove odcsim doc --- README.md | 13 +++++++------ docs/architecture.md | 13 +++++++++++-- docs/introduction.md | 16 +++++++++++++++ docs/models.md | 8 -------- docs/run.md | 6 +++--- docs/setup.md | 52 ++++++++++++++++++++++++++++++++++++++++--------- odcsim/README.md | 12 ++---------- odcsim/docs/build.md | 6 ------ odcsim/docs/concepts.md | 0 odcsim/docs/models.md | 0 odcsim/docs/run.md | 0 11 files changed, 82 insertions(+), 44 deletions(-) create mode 100644 docs/introduction.md delete mode 100644 docs/models.md delete mode 100644 odcsim/docs/build.md delete mode 100644 odcsim/docs/concepts.md delete mode 100644 odcsim/docs/models.md delete mode 100644 odcsim/docs/run.md diff --git a/README.md b/README.md index 68feccc9..1bc866cf 100644 --- a/README.md +++ b/README.md @@ -27,18 +27,19 @@ This repository hosts the source code and development of the simulation componen ## Documentation Check out the [Getting Started](#getting-started) section for a quick overview. -The documentation is located in the [docs/](docs/) directory and is divided as follows: +The documentation is located in the [docs/](docs) directory and is divided as follows: +1. [Introduction](docs/introduction.md) 1. [Toolchain Setup](docs/setup.md) 2. [Architecture Overview](docs/architecture.md) -3. [Pre-built Models](docs/models.md) -4. [Running a Model](docs/run.md) -5. [Building Your Own Model](docs/build.md) -6. [API Reference](https://atlarge-research.github.io/opendc-simulator) **(TODO maybe remove this until we have created this - see #36)** +3. [Running an Experiment](docs/run.md) + +## Getting Started +TODO ## Contributing ### Contributing Guide Read our [contributing guide](CONTRIBUTING.md) to learn about our -development process, how to propose bug fixes and improvements, and how +development process, how to propose bug fixes and improvements, and how to build and test your changes to the project. ### License diff --git a/docs/architecture.md b/docs/architecture.md index a854bf6d..60ac3e0d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,8 +1,17 @@ -# 2. Architecture Overview +# 3. Architecture Overview + +The OpenDC simulator has two main components: `odcsim`, a general framework for discrete event-based simulation, and `opendc` a collection of components using `odcsim` to simulate datacenters. We discuss both in turn! + +`TODO: Add a schematic general overview here.` + +## 3.1 `odcsim` + + +## 3.2 `opendc` **TODO: What are the main components (odcsim-core, odcsim-engine, opendc-core, opendc-format, opendc-testkit, opendc-workflows, I assume)? Short description of each in a bullet list (or even a schema drawing, if you have the time). Why this division?** **TODO: One section per component, explaining what their responsibility is and what other modules they talk to. In what kind of scenarios do you as OpenDC developer need to touch/change them?** --- -[< Previous](setup.md) | [Next >](models.md) +[< Previous](setup.md) | [Next >](run.md) diff --git a/docs/introduction.md b/docs/introduction.md new file mode 100644 index 00000000..e6e3289d --- /dev/null +++ b/docs/introduction.md @@ -0,0 +1,16 @@ +# 1. Introduction + +This repository hosts the source code and development of the simulation component of the [OpenDC](https://opendc.org) project. OpenDC is a simulation platform for cloud infrastructure, with the aim of making datacenter concepts and technology accessible to everyone. To learn more about OpenDC, have a look through our [vision paper](https://ieeexplore.ieee.org/document/8121623)! + +The simulator is one of the components of the OpenDC stack. It is responsible for modelling and simulation of datacenters and their components. This entails receiving environments and experiment configurations from up the stack, simulating these configurations, and reporting back results. + +The simulator is composed of two main components: +- **[odcsim](/odcsim)** + A framework for discrete event simulation. +- **[opendc](/opendc)** + A collection of models for simulating clouds, datacenters and their components, using the **odcsim** framework. + +In the remainder of this documentation, we walk you through a typical toolchain setup for amending OpenDC in your own way, explain the architecture of the system, and show you how to run your own experiments with the simulator. + +--- +[Next >](setup.md) diff --git a/docs/models.md b/docs/models.md deleted file mode 100644 index fab8cdcf..00000000 --- a/docs/models.md +++ /dev/null @@ -1,8 +0,0 @@ -# 3. Pre-built Models - -**TODO: Introduction: we have a library of pre-built models to make things easier for you, here we present them etc.** - -**Per model in the architecture (corresponding with the architecture document), give the list of models we include out of the box. Explain what it does (and if applicable, what it doesn't do).** - ---- -[< Previous](architecture.md) | [Next >](models.md) diff --git a/docs/run.md b/docs/run.md index d3143d96..1eadfedf 100644 --- a/docs/run.md +++ b/docs/run.md @@ -1,6 +1,6 @@ -# 4. Running a Model +# 4. Running an Experiment -**TODO: Not sure yet what's useful to include here. Maybe give an example of how to run one of the pre-builts?** +**TODO: What did we do for SC18?** --- -[< Previous](models.md) | [Next >](build.md) +[< Previous](architecture.md) diff --git a/docs/setup.md b/docs/setup.md index f7ab9f12..0c5d412a 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -1,18 +1,52 @@ -# 1. Toolchain Setup +# 2. Toolchain Setup -**TODO: Introduction: What basic libraries/language(s) do we use, with links to their respective sites. What tool do we recommend (IDEA probably). Explain that we will now give two alternative ways (IDEA and shell).** +The OpenDC simulator is built using the [Kotlin](https://kotlinlang.org/) language. This is a JVM-based language that should appear familiar to programmers knowledgeable in Java or Scala. For a short interactive introduction to Kotlin, the [Learn Kotlin By Example](https://play.kotlinlang.org/byExample/overview) docs are a great place to start. -## Installing Java +For the build and dependency toolchain, we use [Gradle](https://gradle.org/). You will likely not need to change the Gradle build configurations of components, but you will use Gradle to execute builds and tests on the codebase. -**TODO: Explain which version they need and where they can get it.** +Follow the steps below to get it all set up! -## Setup With IntelliJ IDEA +## 2.1 Installing Java -**TODO: Take the user from not having IDEA to having the opendc-simulator codebase open in the environment. Finish with running a test to see if it works.** +Kotlin requires a Java installation of version 8 or higher. Make sure to install the [JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html), not only the JRE (the JDK also includes a JRE). -## Setup With Shell +## 2.2 Building and Developing -**TODO: Take the user from not having the codebase to having it and having all dependencies fetched with Gradle. This will likely need separate instructions for Windows and for Linux/MacOS. Advise maybe VS Code as editor to view things after installation. Finish with running a test to see if it works.** +With Java installed, we're ready to set up the development environment on your machine. You can either use a visual IDE or work from a command line shell. We outline both approaches below, feel free to choose which you are most comfortable with. If in doubt which one to choose, we suggest going with the first one. + +## 2.2.1 Setup With IntelliJ IDEA + +We suggest using [IntelliJ IDEA](https://www.jetbrains.com/idea/) as development environment. Once you have installed any version of this IDE on your machine, choose "Get from Version Control" in the new project dialogue. Enter `https://atlarge.ewi.tudelft.nl/gitlab/opendc/opendc-simulator` as URL and submit your credentials when asked. Open the project once it's ready fetching the codebase, and let it set up with the defaults (IntelliJ will recognize that this is a Gradle codebase). + +You will now be prompted in a dialogue to enable auto-import for Gradle, which we suggest you do. Wait for any progress bars in the lower bar to disappear and then look for the Gradle context menu on the right hand side. In it, go to `opendc-simulator > Tasks > verification > test`. This will build the codebase and run checks to verify that tests pass. If you get a `BUILD SUCCESSFUL` message, you're ready to go to the [next section](architecture.md)! + +## 2.2.2 Setup With Command Line + +First, clone the repository with the following command: + +```shell script +git clone https://atlarge.ewi.tudelft.nl/gitlab/opendc/opendc-simulator +``` + +And enter the directory: + +```shell script +cd opendc-simulator +``` + +If on Windows, run the batch file included in the root, as follows: + +```commandline +gradlew.bat test +``` + +If on Linux/macOS, run the shell script included in the root, as follows: + +```shell script +./gradlew test +``` + +If the build is successful, you are ready to go to the next section! --- -[Next >](architecture.md) +[< Previous](introduction.md) | [Next >](architecture.md) diff --git a/odcsim/README.md b/odcsim/README.md index e5dcc319..b0040cab 100644 --- a/odcsim/README.md +++ b/odcsim/README.md @@ -14,18 +14,10 @@ and the interactions between these actors ([Actor model](https://en.wikipedia.org/wiki/Actor_model)), using an API similar to [Akka](https://doc.akka.io/docs/akka/current/index.html). -## Documentation -Check out the [Getting Started](#getting-started) section for a quick -overview. -The documentation is located in the [docs/](docs/) directory and is divided as follows: -* [Main Concepts](docs/concepts.md) -* [Building a Model](docs/build.md) -* [Running a Model](docs/run.md) -* [API Reference](https://atlarge-research.github.io/opendc-simulator) -* [Contributing Guide](../CONTRIBUTING.md) - ## Getting Started +The instructions below are meant for when you would like to use `odcsim` separately from `opendc`. If you simply want to use the simulator, you do not need to follow them, please refer to the [main README](../README.md) for instructions. + ### Installation Please add the required packages as dependency in your project. Releases are available in the [Maven Central Repository](https://search.maven.org/). diff --git a/odcsim/docs/build.md b/odcsim/docs/build.md deleted file mode 100644 index 21983355..00000000 --- a/odcsim/docs/build.md +++ /dev/null @@ -1,6 +0,0 @@ -# 5. Building Your Own Model - -**TODO: What goes into building a model? Where do you start? Can be quite technical (what gradle scripts do I need to make, what directory structure, ...), maybe center around a simple example scenario?** - ---- -[< Previous](run.md) diff --git a/odcsim/docs/concepts.md b/odcsim/docs/concepts.md deleted file mode 100644 index e69de29b..00000000 diff --git a/odcsim/docs/models.md b/odcsim/docs/models.md deleted file mode 100644 index e69de29b..00000000 diff --git a/odcsim/docs/run.md b/odcsim/docs/run.md deleted file mode 100644 index e69de29b..00000000 -- cgit v1.2.3 From 4ad068f741321973a70f10755074505a3f813ebd Mon Sep 17 00:00:00 2001 From: Georgios Andreadis Date: Fri, 14 Feb 2020 12:28:20 +0100 Subject: Add documentation for the odcsim component --- docs/architecture.md | 14 ++++++++++++-- docs/introduction.md | 1 - docs/run.md | 1 - docs/setup.md | 5 ----- odcsim/README.md | 3 +-- 5 files changed, 13 insertions(+), 11 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 60ac3e0d..834de69c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,13 +1,23 @@ # 3. Architecture Overview - -The OpenDC simulator has two main components: `odcsim`, a general framework for discrete event-based simulation, and `opendc` a collection of components using `odcsim` to simulate datacenters. We discuss both in turn! +The OpenDC simulator has two main components: `odcsim`, a general framework for discrete event-based simulation, and `opendc` a collection of components using `odcsim` to simulate datacenters. While `opendc` is the focus of this project and thus these docs, we also shortly explain what `odcsim` looks like, to give you a better understanding of the other concepts. `TODO: Add a schematic general overview here.` ## 3.1 `odcsim` +The `odcsim` framework has an API module (`odcsim-api`) and an module implementing this API (`odcsim-engine-omega`). + +### 3.1.1 `odcsim-api` +The API defines the behavior of any implementation adhering to the `odcsim` framework. It centers around a `SimulationEngine`, which can be created through a `SimulationEngineProvider`. + +A simulation has a root process with certain `Behavior` (a coroutine). Processes have a `ProcessContext` which allow them to spawn other processes and open communication `Channel`s with other processes. Each of these `Channel`s has a `SendPort` and a `ReceivePort`. + +### 3.1.2 `odcsim-engine-omega` +The implementation is an executable interpretation of this API. The main class is `OmegaSimulationEngine` and takes care of transmitting timestamped events between processes. It ensures that message delivery order is the same as sent order. +You might note that the simulation framework and the simulator itself makes extensive use of Kotlin's coroutine feature. This is a paradigm for asynchronous programming. If you are not familiar with coroutines, we suggest having a look at the [Kotlin documentation on this](https://kotlinlang.org/docs/reference/coroutines-overview.html), especially their [quick start guide](https://kotlinlang.org/docs/tutorials/coroutines/coroutines-basic-jvm.html) and their [hands-on guide](https://play.kotlinlang.org/hands-on/Introduction%20to%20Coroutines%20and%20Channels/01_Introduction). ## 3.2 `opendc` +The `opendc` package consists of **TODO: What are the main components (odcsim-core, odcsim-engine, opendc-core, opendc-format, opendc-testkit, opendc-workflows, I assume)? Short description of each in a bullet list (or even a schema drawing, if you have the time). Why this division?** diff --git a/docs/introduction.md b/docs/introduction.md index e6e3289d..711a829b 100644 --- a/docs/introduction.md +++ b/docs/introduction.md @@ -1,5 +1,4 @@ # 1. Introduction - This repository hosts the source code and development of the simulation component of the [OpenDC](https://opendc.org) project. OpenDC is a simulation platform for cloud infrastructure, with the aim of making datacenter concepts and technology accessible to everyone. To learn more about OpenDC, have a look through our [vision paper](https://ieeexplore.ieee.org/document/8121623)! The simulator is one of the components of the OpenDC stack. It is responsible for modelling and simulation of datacenters and their components. This entails receiving environments and experiment configurations from up the stack, simulating these configurations, and reporting back results. diff --git a/docs/run.md b/docs/run.md index 1eadfedf..76be70ba 100644 --- a/docs/run.md +++ b/docs/run.md @@ -1,5 +1,4 @@ # 4. Running an Experiment - **TODO: What did we do for SC18?** --- diff --git a/docs/setup.md b/docs/setup.md index 0c5d412a..801dfe58 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -1,5 +1,4 @@ # 2. Toolchain Setup - The OpenDC simulator is built using the [Kotlin](https://kotlinlang.org/) language. This is a JVM-based language that should appear familiar to programmers knowledgeable in Java or Scala. For a short interactive introduction to Kotlin, the [Learn Kotlin By Example](https://play.kotlinlang.org/byExample/overview) docs are a great place to start. For the build and dependency toolchain, we use [Gradle](https://gradle.org/). You will likely not need to change the Gradle build configurations of components, but you will use Gradle to execute builds and tests on the codebase. @@ -7,21 +6,17 @@ For the build and dependency toolchain, we use [Gradle](https://gradle.org/). Yo Follow the steps below to get it all set up! ## 2.1 Installing Java - Kotlin requires a Java installation of version 8 or higher. Make sure to install the [JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html), not only the JRE (the JDK also includes a JRE). ## 2.2 Building and Developing - With Java installed, we're ready to set up the development environment on your machine. You can either use a visual IDE or work from a command line shell. We outline both approaches below, feel free to choose which you are most comfortable with. If in doubt which one to choose, we suggest going with the first one. ## 2.2.1 Setup With IntelliJ IDEA - We suggest using [IntelliJ IDEA](https://www.jetbrains.com/idea/) as development environment. Once you have installed any version of this IDE on your machine, choose "Get from Version Control" in the new project dialogue. Enter `https://atlarge.ewi.tudelft.nl/gitlab/opendc/opendc-simulator` as URL and submit your credentials when asked. Open the project once it's ready fetching the codebase, and let it set up with the defaults (IntelliJ will recognize that this is a Gradle codebase). You will now be prompted in a dialogue to enable auto-import for Gradle, which we suggest you do. Wait for any progress bars in the lower bar to disappear and then look for the Gradle context menu on the right hand side. In it, go to `opendc-simulator > Tasks > verification > test`. This will build the codebase and run checks to verify that tests pass. If you get a `BUILD SUCCESSFUL` message, you're ready to go to the [next section](architecture.md)! ## 2.2.2 Setup With Command Line - First, clone the repository with the following command: ```shell script diff --git a/odcsim/README.md b/odcsim/README.md index b0040cab..b930315a 100644 --- a/odcsim/README.md +++ b/odcsim/README.md @@ -11,8 +11,7 @@ by the [OpenDC](https://opendc.org) project. Simulations are defined in terms of a hierarchical grouping of actors and the interactions between these actors -([Actor model](https://en.wikipedia.org/wiki/Actor_model)), using -an API similar to [Akka](https://doc.akka.io/docs/akka/current/index.html). +([Actor model](https://en.wikipedia.org/wiki/Actor_model)). ## Getting Started -- cgit v1.2.3 From 19997aaef104e63f6cf7dc3f831339e5fa5b933d Mon Sep 17 00:00:00 2001 From: Georgios Andreadis Date: Fri, 14 Feb 2020 19:13:54 +0100 Subject: Add texts around non-compute modules --- docs/architecture.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 834de69c..dcb24c88 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,7 +1,7 @@ # 3. Architecture Overview The OpenDC simulator has two main components: `odcsim`, a general framework for discrete event-based simulation, and `opendc` a collection of components using `odcsim` to simulate datacenters. While `opendc` is the focus of this project and thus these docs, we also shortly explain what `odcsim` looks like, to give you a better understanding of the other concepts. -`TODO: Add a schematic general overview here.` +**TODO: Add a schematic general overview here.** ## 3.1 `odcsim` The `odcsim` framework has an API module (`odcsim-api`) and an module implementing this API (`odcsim-engine-omega`). @@ -17,11 +17,21 @@ The implementation is an executable interpretation of this API. The main class i You might note that the simulation framework and the simulator itself makes extensive use of Kotlin's coroutine feature. This is a paradigm for asynchronous programming. If you are not familiar with coroutines, we suggest having a look at the [Kotlin documentation on this](https://kotlinlang.org/docs/reference/coroutines-overview.html), especially their [quick start guide](https://kotlinlang.org/docs/tutorials/coroutines/coroutines-basic-jvm.html) and their [hands-on guide](https://play.kotlinlang.org/hands-on/Introduction%20to%20Coroutines%20and%20Channels/01_Introduction). ## 3.2 `opendc` -The `opendc` package consists of +The `opendc` package consists of a number of submodules, the most important being `opendc-compute`. Below, we will explain each in turn. -**TODO: What are the main components (odcsim-core, odcsim-engine, opendc-core, opendc-format, opendc-testkit, opendc-workflows, I assume)? Short description of each in a bullet list (or even a schema drawing, if you have the time). Why this division?** +### 3.2.1 `opendc-compute` +**TODO** -**TODO: One section per component, explaining what their responsibility is and what other modules they talk to. In what kind of scenarios do you as OpenDC developer need to touch/change them?** +### 3.2.2 `opendc-workflows` +This module contains all workflow-related models and logic of the simulator. The models for workflows can be found in the `workload` package: A `Job` and a `Task`. The logic concerning the scheduling of a workflow is contained in the `service` package. It follows the Reference Architecture for Datacenter Schedulers by [Andreadis et al.](https://dl.acm.org/doi/10.5555/3291656.3291706). For a good introduction into datacenter schedulers and to fully grasp the modeling approach taken, we highly recommend reading this publication (or its more extensive [technical report](https://arxiv.org/pdf/1808.04224.pdf)). + +The `service` package merits its own explanation. A scheduler is defined by the `StageWorkflowScheduler` class, according to the architecture. The main component, however, is the `StageWorkflowSchedulerLogic`, responsible for pulling together the different scheduling stage implementations from the `stage` package. The scheduler is managed by the `WorkflowService`, which also orchestrates the lifecycle of a workflow. + +### 3.2.3 `opendc-format` +Running scientific experiments does not require running the full OpenDC stack. We also support directly reading out environment and workload trace files. Example implementations of these can be found in the `opendc-format` module. To parse a different format, you can take one of the existing parsers and adapt it to your needs. + +### 3.2.4 `opendc-experiments-sc18` +This is a module created for the experiments of our [SC18 publication](https://dl.acm.org/doi/10.5555/3291656.3291706). We aim to separate these kinds of custom experiment setups from the rest of the codebase by placing them in separate modules. --- [< Previous](setup.md) | [Next >](run.md) -- cgit v1.2.3 From 97afd8f9c77a5800bed11ddab1f51e7dbfc3128f Mon Sep 17 00:00:00 2001 From: Georgios Andreadis Date: Fri, 14 Feb 2020 19:19:18 +0100 Subject: Add core module to list --- docs/architecture.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index dcb24c88..95d72126 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -17,20 +17,23 @@ The implementation is an executable interpretation of this API. The main class i You might note that the simulation framework and the simulator itself makes extensive use of Kotlin's coroutine feature. This is a paradigm for asynchronous programming. If you are not familiar with coroutines, we suggest having a look at the [Kotlin documentation on this](https://kotlinlang.org/docs/reference/coroutines-overview.html), especially their [quick start guide](https://kotlinlang.org/docs/tutorials/coroutines/coroutines-basic-jvm.html) and their [hands-on guide](https://play.kotlinlang.org/hands-on/Introduction%20to%20Coroutines%20and%20Channels/01_Introduction). ## 3.2 `opendc` -The `opendc` package consists of a number of submodules, the most important being `opendc-compute`. Below, we will explain each in turn. +The `opendc` package consists of a number of submodules, the most feature-rich being `opendc-compute`. Below, we will explain each in turn. -### 3.2.1 `opendc-compute` +### 3.2.1 `opendc-core` **TODO** -### 3.2.2 `opendc-workflows` +### 3.2.2 `opendc-compute` +**TODO** + +### 3.2.3 `opendc-workflows` This module contains all workflow-related models and logic of the simulator. The models for workflows can be found in the `workload` package: A `Job` and a `Task`. The logic concerning the scheduling of a workflow is contained in the `service` package. It follows the Reference Architecture for Datacenter Schedulers by [Andreadis et al.](https://dl.acm.org/doi/10.5555/3291656.3291706). For a good introduction into datacenter schedulers and to fully grasp the modeling approach taken, we highly recommend reading this publication (or its more extensive [technical report](https://arxiv.org/pdf/1808.04224.pdf)). The `service` package merits its own explanation. A scheduler is defined by the `StageWorkflowScheduler` class, according to the architecture. The main component, however, is the `StageWorkflowSchedulerLogic`, responsible for pulling together the different scheduling stage implementations from the `stage` package. The scheduler is managed by the `WorkflowService`, which also orchestrates the lifecycle of a workflow. -### 3.2.3 `opendc-format` +### 3.2.4 `opendc-format` Running scientific experiments does not require running the full OpenDC stack. We also support directly reading out environment and workload trace files. Example implementations of these can be found in the `opendc-format` module. To parse a different format, you can take one of the existing parsers and adapt it to your needs. -### 3.2.4 `opendc-experiments-sc18` +### 3.2.5 `opendc-experiments-sc18` This is a module created for the experiments of our [SC18 publication](https://dl.acm.org/doi/10.5555/3291656.3291706). We aim to separate these kinds of custom experiment setups from the rest of the codebase by placing them in separate modules. --- -- cgit v1.2.3 From 24d42cabd6c4a40313c0de0a53d2cec6aa5b0a5e Mon Sep 17 00:00:00 2001 From: Fabian Mastenbroek Date: Fri, 14 Feb 2020 22:54:10 +0100 Subject: docs: Add initial description of opendc-core and opendc-compute modules --- docs/architecture.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 95d72126..4f271360 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -20,10 +20,13 @@ You might note that the simulation framework and the simulator itself makes exte The `opendc` package consists of a number of submodules, the most feature-rich being `opendc-compute`. Below, we will explain each in turn. ### 3.2.1 `opendc-core` -**TODO** +This module defines a base model for datacenter simulation, establishing core concepts and terminology of datacenters. +The other `opendc` modules build on this model and extend it in various directions (e.g. virtual machines or workflows). ### 3.2.2 `opendc-compute` -**TODO** +This module models management and provisioning of compute instances such as virtual machines and bare-metal servers. We +represent such compute instances as a `Server`. The hardware configuration of the server is represented as a `Flavor`. +Servers run bootable disk images called `Image`s, which characterizes the runtime behavior of a server. ### 3.2.3 `opendc-workflows` This module contains all workflow-related models and logic of the simulator. The models for workflows can be found in the `workload` package: A `Job` and a `Task`. The logic concerning the scheduling of a workflow is contained in the `service` package. It follows the Reference Architecture for Datacenter Schedulers by [Andreadis et al.](https://dl.acm.org/doi/10.5555/3291656.3291706). For a good introduction into datacenter schedulers and to fully grasp the modeling approach taken, we highly recommend reading this publication (or its more extensive [technical report](https://arxiv.org/pdf/1808.04224.pdf)). -- cgit v1.2.3 From 976ea9e7a6071aa75e32a61bf2b150a42f16224b Mon Sep 17 00:00:00 2001 From: Fabian Mastenbroek Date: Tue, 18 Feb 2020 21:11:22 +0100 Subject: docs: Add extended description of compute module --- docs/architecture.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 4f271360..33fb90e5 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -20,13 +20,22 @@ You might note that the simulation framework and the simulator itself makes exte The `opendc` package consists of a number of submodules, the most feature-rich being `opendc-compute`. Below, we will explain each in turn. ### 3.2.1 `opendc-core` -This module defines a base model for datacenter simulation, establishing core concepts and terminology of datacenters. -The other `opendc` modules build on this model and extend it in various directions (e.g. virtual machines or workflows). +This module defines a base model for datacenter simulation, establishing core concepts and terminology of datacenters +that we share across the various modules. Other `opendc` modules build on this model and extend it in various directions (e.g. virtual machines or workflows). ### 3.2.2 `opendc-compute` -This module models management and provisioning of compute instances such as virtual machines and bare-metal servers. We -represent such compute instances as a `Server`. The hardware configuration of the server is represented as a `Flavor`. -Servers run bootable disk images called `Image`s, which characterizes the runtime behavior of a server. +This module is concerned with modeling cloud computing services (such as [Amazon EC2](https://aws.amazon.com/ec2/) and [Google Compute Engine](https://cloud.google.com/compute)) and provides the following features: + +1. **Model for simulated workloads** + We represent workloads as bootable disk images (called `Image`) which characterize the runtime behavior + of a running server in terms of the cpu time they require over time. +2. **Bare-metal management & provisioning** + We provide models for simulating management of bare-metal machines (`Node`) and deploying workloads on it (using `BareMetalDriver`). + Furthermore, we also include functionality for simulating and experimenting with (custom) provisioning policies on a pool of bare-metal machies (using `ProvisioningService`) +3. **Virtual machine management, scheduling and provisioning** + We provide functionality for simulating deployment of multiple `Image`s on a single machine using a hypervisor, which + is concerned with scheduling/distributing the load of the running guest machines on the host machine. This may be used to experiment with overcommitting of virtual resources. + Moreover, we also model provisioning policies for virtual machine provisioning with a pool of host machines. ### 3.2.3 `opendc-workflows` This module contains all workflow-related models and logic of the simulator. The models for workflows can be found in the `workload` package: A `Job` and a `Task`. The logic concerning the scheduling of a workflow is contained in the `service` package. It follows the Reference Architecture for Datacenter Schedulers by [Andreadis et al.](https://dl.acm.org/doi/10.5555/3291656.3291706). For a good introduction into datacenter schedulers and to fully grasp the modeling approach taken, we highly recommend reading this publication (or its more extensive [technical report](https://arxiv.org/pdf/1808.04224.pdf)). -- cgit v1.2.3 From f334239b072855561ee522d70d78d8b61e1b6474 Mon Sep 17 00:00:00 2001 From: Georgios Andreadis Date: Tue, 18 Feb 2020 21:14:56 +0100 Subject: Update architecture.md Adds missing fullstop --- docs/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/architecture.md b/docs/architecture.md index 33fb90e5..940ec09b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -31,7 +31,7 @@ This module is concerned with modeling cloud computing services (such as [Amazon of a running server in terms of the cpu time they require over time. 2. **Bare-metal management & provisioning** We provide models for simulating management of bare-metal machines (`Node`) and deploying workloads on it (using `BareMetalDriver`). - Furthermore, we also include functionality for simulating and experimenting with (custom) provisioning policies on a pool of bare-metal machies (using `ProvisioningService`) + Furthermore, we also include functionality for simulating and experimenting with (custom) provisioning policies on a pool of bare-metal machies (using `ProvisioningService`). 3. **Virtual machine management, scheduling and provisioning** We provide functionality for simulating deployment of multiple `Image`s on a single machine using a hypervisor, which is concerned with scheduling/distributing the load of the running guest machines on the host machine. This may be used to experiment with overcommitting of virtual resources. -- cgit v1.2.3