summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2021-03-25Add advanced energy model to OpenDC (v1)Fabian Mastenbroek
This pull request is a preparation for the adding an advanced energy model to OpenDC (#90): * Add benchmarks for the `opendc-simulator-compute` module, which enables us to quantify the effect on future changes w.r.t. to the energy model on the performance. * Add `SimResourceTransformer` which can transform resource consumptions. This is useful when transforming from CPU frequency to energy usage for instance. * Move power models to `opendc-simulator-compute` to be able to be used more generically. **Breaking API Changes** * Power models have moved to `opendc-simulator-compute` and now implement the `MachinePowerModel` interface.
2021-03-24simulator: Move power models to simulator moduleFabian Mastenbroek
This change moves the power models from the `opendc-compute-simulator` to the `opendc-simulator-compute` module, since it better fits the scope of the models and allows them to be re-used for other purposes.
2021-03-23simulator: Add support for transforming resource consumersFabian Mastenbroek
This change adds support for transforming the resource commands emitted by the resource consumers. The SimResourceForwarder is modified to also support transforming the resource commands.
2021-03-23simulator: Add benchmarks for opendc-simulator-computeFabian Mastenbroek
This change adds benchmarks to the opendc-simulator-compute module in order to quantify effect of changes on the performance of this module.
2021-03-23simulator: Add uniform resource consumption model (v2)Fabian Mastenbroek
This is the second pull request in the series of pull requests to add a uniform resource consumption model to OpenDC. This pull request focusses on adding dynamic capacity negotiation and propagation between resource consumer and resource provider: * The generic resource constraint is removed from the interfaces of `opendc-simulator-resources`. Users of the API are expected to use the untyped variants where only the capacity needs to be specified. Users are expected to build higher-level abstractions on top of these interface to represent actual resources (e.g., CPU, disk or network). * Added benchmarks for the most important implementations of `opendc-simulator-resources`. This allows us to quantify the effects of changes on the runtime. * The `SimResourceSwitchMaxMin` has been split into a `SimResourceAggregatorMaxMin` and `SimResourceDistributorMaxMin` which respectively aggregate input resources and distribute output resources using max-min fair sharing. * The `SimResourceConsumer` interface has a new method for receiving capacity change events: `onCapacityChanged(ctx, isThrottled)` **Breaking API Changes** * All interfaces in `opendc-simulator-resources`.
2021-03-23simulator: Add support for emitting VM usage metricsFabian Mastenbroek
This change re-enables support for VM usage metrics by adding an adapter for SimResourceConsumer instances that can export the consumer speed.
2021-03-23simulator: Add support for signaling dynamic capacity changesFabian Mastenbroek
This change adds support for dynamically changing the capacity of resources and propagating this change to consumers.
2021-03-22simulator: Remove generic resource constraint from resource modelFabian Mastenbroek
This change removes the generic resource constraint (e.g., SimResource) and replaces it by a simple capacity property. In the future, users should handle the resource properties on a higher level. This change simplifies compositions of consumers and providers by not requiring a translation from resource to capacity.
2021-03-22simulator: Expose capacity and remaining work outside consumer callbackFabian Mastenbroek
This change changes the consumer and context interfaces to expose the provider capacity and remaining work via the context instance as opposed to only via the callback. This simplifies aggregation of resources.
2021-03-22simulator: Add benchmarks for resource consumption frameworkFabian Mastenbroek
This change adds an initial set of benchmarks for the resource consumption framework in order to measure the effect of changes on the performance of the simulator.
2021-03-22simulator: Re-design consumer interface to support capacity negotiationFabian Mastenbroek
This change re-designs the SimResourceConsumer interface to support in the future capacity negotiation. This basically means that the consumer will be informed directly when not enough capacity is available, instead of after the deadline specified by the consumer.
2021-03-20serverless: Add initial implementation of OpenDC Serverless (#100)Fabian Mastenbroek
This change adds the initial implementation of OpenDC Serverless to the main repository, which is based on Soufiane Jounaid's work. This commit introduces the main interfaces into the repository. Later commits will introduce the other features that Soufiane has implemented previously in OpenDC.
2021-03-20Add support for code coverage tracking via CodecovFabian Mastenbroek
This pull request enables aggregate code coverage reports via Jacoco and adds support for coverage tracking via Codecov.
2021-03-20ci: Add reports and coverage tracking for APIFabian Mastenbroek
2021-03-20ci: Generate test reports for CI buildsFabian Mastenbroek
This change updates the Github Actions workflow configuration to generate test reports for the simulator CI builds.
2021-03-20ci: Add Codecov support to repositoryFabian Mastenbroek
This change enables Codecov coverage reports when building changes.
2021-03-20ci: Enable stricter CI triggersFabian Mastenbroek
This change will cause the CI to only trigger when opening a pull request.
2021-03-20build: Update dependencies for simulatorFabian Mastenbroek
This change updates the dependencies for the OpenDC simulator.
2021-03-20build: Add support for aggregate code coverage resultsFabian Mastenbroek
This change adds support for aggregating code coverage results from the different modules.
2021-03-20api: Bump jinja2 from 2.11.2 to 2.11.3 in /api (#99)dependabot[bot]
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.2 to 2.11.3. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst) - [Commits](https://github.com/pallets/jinja/compare/2.11.2...2.11.3) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-20api: Bump urllib3 from 1.26.0 to 1.26.3 in /api (#98)dependabot[bot]
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.0 to 1.26.3. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](https://github.com/urllib3/urllib3/compare/1.26.0...1.26.3) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-18simulator: Add the CPU power model from iCanCloud/E-mc2Hongyu
This change implements the CPU energy model with p-states from iCanCloud/E-mc2: - Only pushed a portion of the code for discussion as not sure if the idea is on track. - Inline comments have been added, and formal documents will follow once the model is finalized. - The p-state power consumptions are currently hard-coded in a companion object, which should be improved in the next PR(s). **Breaking Changes** - CpuPowerModel: directly interact with the machine it is measuring. - SimBareMetalMachine: expose the speeds of its CPU cores and its clock instant.
2021-03-17Add uniform resource consumption model (v1)Fabian Mastenbroek
This is the first in the series of pull requests to add a uniform resource consumption model to OpenDC. This pull request introduces the `opendc-simulator-resources` module which introduces the primitives with which we can model resource consumption of CPUs, disks and network: * `SimResourceProvider` represents a provider of some generic resource `R`, which may be consumed via `consume(SimResourceConsumer<R>)` * `SimResourceConsumer` represents a resource consumers and characterizes how the resource is being consumed. * `SimResourceSwitch` is a generic scheduler for sharing the capacity of multiple resources across multiple consumers. - `SimResourceSwitchExclusive`: A space-shared switch - each consumer is allocated a single resource exclusively. - `SimResourceSwitchMinMax`: A time-shared switch - each consumer gets a fair share of the resource capacity. * `SimResourceForwarder` converts a consumer in a provider. **Breaking Changes** * `ProcessingUnit` and `MemoryUnit` renamed to `SimProcessingUnit` and `SimMemoryUnit` respectively. * `TimerScheduler` accepts a `CoroutineContext` as opposed to a `CoroutineScope`.
2021-03-17simulator: Make hypervisors generic for the resource typeFabian Mastenbroek
This change moves the hypervisor implementations to the opendc-simulator-resources module and makes them generic to the resource type that is being used (e.g., CPU, disk or networking).
2021-03-16utils: Prevent memory leakage in TimerSchedulerFabian Mastenbroek
This change fixes a possible memory leakage issue in TimerScheduler when a large number of timers was scheduled for the same timestamp.
2021-03-16simulator: Add generic framework for resource consumption modelingFabian Mastenbroek
This change adds a generic framework for modeling resource consumptions and adapts opendc-simulator-compute to model machines and VMs on top of this framework. This framework anticipates the addition of additional resource types such as memory, disk and network to the OpenDC codebase.
2021-03-15ci: Update deployment workflowFabian Mastenbroek
This change updates the deployment workflow for opendc.org due to changes on the server.
2021-03-09compute: Improvements to cloud compute model (v2)Fabian Mastenbroek
This is the second in the series of pull requests to improve the existing cloud compute model (see #86). This pull request removes the dependency on the bare-metal provisioning code which simplifies experiment setup tremendously: - Remove bare-metal provisioning code (opendc-metal) - Remove opendc-core which was a relic of the previous codebase and was only used sparingly. - Move ownership of Server, Image and Flavor to the compute service. Users are expected to create instances via the compute service.
2021-03-09workflow: Split workflow module in API and service moduleFabian Mastenbroek
2021-03-09compute: Move implementation of Flavor into service moduleFabian Mastenbroek
2021-03-09compute: Model storage of VM imagesFabian Mastenbroek
2021-03-09compute: Extend capabilities of compute APIFabian Mastenbroek
2021-03-09utils: Remove unused StateFlowFabian Mastenbroek
This change removes the unused StateFlow from the utils module. It has been replaced by the proper implementation in the kotlinx-coroutines library.
2021-03-09core: Remove OpenDC core moduleFabian Mastenbroek
This change removes the opendc-core module. This module was an artifact of the old codebase and remained mostly unused. This change removes all usages of the module and if necessary introduces replacement classes.
2021-03-09compute: Add support for custom workload mappingsFabian Mastenbroek
2021-03-09compute: Introduce labels and meta-data for resourcesFabian Mastenbroek
This change adds the ability to define labels and meta-data for resources. This can be used in the future to identify servers and pass data between client and server.
2021-03-09compute: Improvements to cloud compute model (v1)Fabian Mastenbroek
This is the first of the pull requests in an attempt to improve the existing cloud compute model (see #86). This pull request restructures the compute API and splits the consumer and service interfaces into different modules: - opendc-compute-api now defines the API interface for the OpenDC Compute module, which can be used by consumers of the OpenDC Compute service. - opendc-compute-service hosts the service implementation for OpenDC Compute and contains all business logic regarding the IaaS platform (such as scheduling). - opendc-compute-simulator implements a "compute driver" for the OpenDC Compute platform that simulates submitted workloads. - Image is now a data-class and does not specify itself the workload to simulate. Instead, the workload should be passed via its tags currently (with key "workload"). In the future, the simulation backend will accept a mapper interface that maps Images to Workloads.
2021-03-09compute: Add lifecycle methods for Server instancesFabian Mastenbroek
This change adds more methods for controlling the lifecycle of Server instances.
2021-03-08compute: Remove use of bare-metal provisioning from compute moduleFabian Mastenbroek
This change removes the usage of bare-metal provisioning from the OpenDC Compute module. This significantly simplifies the experiment setup.
2021-03-07compute: Move bare-metal provisioning in separate moduleFabian Mastenbroek
This change moves the bare-metal provisioning packages outside the compute module since these modules represent different layers in the ecosystem and should not be mixed.
2021-03-07compute: Move ComputeService implementation in service moduleFabian Mastenbroek
This change introduces the ComputeService interface (previously VirtProvisioningService) and provides a central implementation in opendc-compute-service. Previously, the implementation of this interface was bound to the simulator package, which meant that independent business logic could not be re-used without importing the simulator code.
2021-03-07compute: Extract API from compute serviceFabian Mastenbroek
This change extracts the API for the OpenDC Compute service into a separate module to establish a clearer boundary between the interface meant for consumers and interfaces meant for the the serve implementation.
2021-03-07compute: Convert Server to stateful interfaceFabian Mastenbroek
This change converts the Server data class which can be used as a stateful object to control an instance running in the cloud.
2021-03-07workflows: Reduce output from testsFabian Mastenbroek
This change increases the logging level from INFO to WARN in order to reduce the amount of messages printed to the console during tests.
2021-03-07compute: Make VirtProvisoningService responsible for Server lifecycleFabian Mastenbroek
This change refactors the OpenDC Compute module so that the VirtProvisioningService is now responsible for managing the lifecycle of Server objects as opposed to the VirtDriver and BareMetalDriver previously.
2021-03-07compute: Remove ServiceRegistry fieldsFabian Mastenbroek
This change removes the use of ServiceRegistry in the OpenDC compute module. It was not actually being used by any of the code and we are moving to another interface in the future.
2021-03-07compute: Separate cloud compute layer from bare-metal layerFabian Mastenbroek
This change separates the cloud compute layer in OpenDC (e.g., Server) from the bare-metal layer (e.g., Node), such that Node and BareMetalDriver are unaware of the existence of Server and co.
2021-03-07compute: Pass simulation workload via image metadataFabian Mastenbroek
This change removes the SimWorkloadImage implementation and changes Image to a data class without workload. Simulation workloads should now be pased via image metadata as the image storage should be unaware of any simulation details.
2021-03-06sim: Implement energy models from CloudSim (#79)Hongyu
This commit implements the energy models that are present in CloudSim: 1. Constant 2. Linear 3. Cubic 4. Square root 5. Interpolation based on data.
2021-03-02Add a simple PR template (#78)Hongyu
This change adds a simple template for pull request which we can use as basis for future pull requests.