summaryrefslogtreecommitdiff
path: root/opendc-experiments
AgeCommit message (Collapse)Author
2021-09-28refactor(telemetry): Do not require clock for ComputeMetricExporterFabian Mastenbroek
This change drops the requirement for a clock parameter when constructing a ComputeMetricExporter, since it will now derive the timestamp from the recorded metrics.
2021-09-21feat(trace): Add support for writing tracesFabian Mastenbroek
This change adds a new API for writing traces in a trace format. Currently, writing is only supported by the OpenDC VM format, but over time the other formats will also have support for writing added.
2021-09-19feat(trace): Update OpenDC VM trace formatFabian Mastenbroek
This change optimizes the OpenDC VM trace format by removing unnecessary columns as well as optimizing the writer settings. The new implementation still supports reading the old trace format in case users run OpenDC with older workload traces.
2021-09-19refactor(capelin): Make workload sampling model extensibleFabian Mastenbroek
This change updates the workload sampling implementation to be more flexible in the way the workload is constructed. Users can now sample multiple workloads at the same time using multiple samplers and use them as a single workload to simulate.
2021-09-19refactor(capelin): Support flexible topology creationFabian Mastenbroek
This change adds support for creating flexible topologies by creating a TopologyFactory interface that is responsible for configuring the hosts of a compute service.
2021-09-19refactor(capelin): Extract common code out of Capelin experimentsFabian Mastenbroek
This change creates a new module for doing simulations with virtual machine workloads. We have found that a lot of code in the Capelin experiments code is being re-used by non-experiment modules.
2021-09-17feat(capelin): Use logical types for Parquet output columnsFabian Mastenbroek
This change updates the output schema for the experiment data to use logical types where possible. This adds additional context for the writer and the reader on how to process the column (efficiently).
2021-09-17refactor(telemetry): Standardize SimHost metricsFabian Mastenbroek
This change standardizes the metrics emitted by SimHost instances and their guests based on the OpenTelemetry semantic conventions. We now also report CPU time as opposed to CPU work as this metric is more commonly used.
2021-09-17refactor(telemetry): Standardize compute scheduler metricsFabian Mastenbroek
This change updates the OpenDC compute service implementation with multiple meters that follow the OpenTelemetry conventions.
2021-09-17refactor(telemetry): Create separate MeterProvider per service/hostFabian Mastenbroek
This change refactors the telemetry implementation by creating a separate MeterProvider per service or host. This means we have to keep track of multiple metric producers, but that we can attach resource information to each of the MeterProviders like we would in a real world scenario.
2021-09-17refactor(capelin): Improve ParquetDataWriter implementationFabian Mastenbroek
This change improves the ParquetDataWriter class to support more complex use-cases. It now allows subclasses to modify the writer options. In addition to this, a subclass for writing server data is added.
2021-09-17refactor(experiments): Remove energy experiments shellFabian Mastenbroek
This change removes the energy experiments. The experiments only provided a setup for the original experiments and is not able to reproduce the results without further worker.
2021-09-17build(telemetry): Update to OpenTelemetry 1.6.0Fabian Mastenbroek
This change updates the opentelemetry-java library to version 1.6.0.
2021-09-12refactor(trace): Add API for accessing available table columnsFabian Mastenbroek
This change adds a new API to the Table interface for accessing the table columns that the table supports. This does not necessarily mean that the column will have a value for every row, but that the table format has defined this particular column.
2021-09-11feat(capelin): Implement trace API for Azure VM trace formatFabian Mastenbroek
This change adds a trace API implementation for the Azure VM traces.
2021-09-11fix(capelin): Parse last column in Solvinity trace formatFabian Mastenbroek
This change fixes an issue where the last column in the Solvinity traces is not parsed correctly, due to the last column having no whitespace at the end to seek to.
2021-09-11perf(trace): Keep reader state in own classFabian Mastenbroek
This change removes the external class that holds the state of the reader and instead puts the state in the reader implementation. Maintaining a separate class for the state increases the complexity and has worse performance characteristics due to the bytecode produced by Kotlin for property accesses.
2021-09-10refactor(compute): Integrate fault injection into compute simulatorFabian Mastenbroek
This change moves the fault injection logic directly into the opendc-compute-simulator module, so that it can operate at a higher abstraction. In the future, we might again split the module if we can re-use some of its logic.
2021-09-10refactor(capelin): Terminate servers after reaching deadlineFabian Mastenbroek
This change updates the Capelin experiment helpers to terminate a server when it has reached its end-date.
2021-09-10feat(simulator): Support generic distribution in fault injectorFabian Mastenbroek
This change adds support for specifying the distribution of the failures, group size and duration for the fault injector.
2021-09-07refactor(capelin): Restructure input reading classesFabian Mastenbroek
2021-09-07refactor(capelin): Move metric collection outside Capelin codeFabian Mastenbroek
This change moves the metric collection outside the Capelin codebase in a separate module so other modules can also benefit from the compute metric collection code.
2021-09-07feat(capelin): Report up/downtime metrics in experiment monitorFabian Mastenbroek
2021-09-07refactor(capelin): Make ExperimentMonitor optional for trace processingFabian Mastenbroek
2021-09-02refactor(capelin): Migrate trace reader to new trace APIFabian Mastenbroek
This change updates the trace reading classes in the Capelin experiment to use the new trace API in order to re-use many of the trace reading parts.
2021-09-02refactor(trace): Implement trace API for WTF readerFabian Mastenbroek
This change updates the WTF trace reader to support the new streaming trace API.
2021-09-02refactor(trace): Move Bitbrains format into separate moduleFabian Mastenbroek
This change moves Bitbrains trace support into a separate module and adds support for the new trace api.
2021-09-02refactor(format): Remove environment reader from format libraryFabian Mastenbroek
This change removes the environment reader from the format library since they are highly specific for the particular experiment. In the future, we hope to have a single format to setup the entire datacenter (perhaps similar to the format used by the web runner).
2021-09-02refactor(trace): Extract Parquet helpers into separate moduleFabian Mastenbroek
This change extracts the Parquet helpers outside format module into a new module, in order to improve re-usability of these helpers.
2021-08-25refactor(compute): Measure power draw without PSU overheadFabian Mastenbroek
This change updates the SimHost implementation to measure the power draw of the machine without PSU overhead to make the results more realistic.
2021-08-25refactor(capelin): Simplify metric extraction for monitorFabian Mastenbroek
This change updates the metric exporter logic in the Capelin module and reduces the number of allocations by looping directly over the collection of metrics.
2021-08-25fix(capelin): Eliminate unnecessary double to long conversionsFabian Mastenbroek
This change eliminates the unnecessary conversions from double to long in the Capelin metric processing code.
2021-08-25build: Upgrade to OpenTelemetry 1.5Fabian Mastenbroek
This change upgrades the OpenTelemetry dependency to version 1.5, which contains various breaking changes in the metrics API.
2021-08-24test(capelin): Add tests for interference and failuresFabian Mastenbroek
This change adds tests to the Capelin integration suite for performance interference as well as failures. These test more accurately the experiment setup.
2021-08-24fix(capelin): Update Bitbrains trace testsFabian Mastenbroek
This change updates the Bitbrains trace tests with the updated trace that does not hardcode the duration of the trace fragments.
2021-08-24fix(capelin): Keep trace order after samplingFabian Mastenbroek
This change fixes an issue with the workload sampler where the resulting workload entries would not be ordered properly according to their submission time.
2021-08-24refactor(format): Clean up Bitbrains trace reader to enable re-useFabian Mastenbroek
This change updates the code for the Bitbrains trace reader and upgrades the TraceConverter to re-use existing code of the Bitbrains trace reader.
2021-08-24fix(simulator): Support unaligned trace fragmentsFabian Mastenbroek
2021-08-24fix(simulator): Record overcommit only after deadlineFabian Mastenbroek
This change fixes an issue with the simulator where it would record overcomitted work if the output was updated before the deadline was reached.
2021-08-24refactor(simulator): Execute traces based on timestampsFabian Mastenbroek
This change refactors the trace workload in the OpenDC simulator to track execute a fragment based on the fragment's timestamp. This makes sure that the trace is replayed identically to the original execution.
2021-08-22refactor(compute): Update FilterScheduler to follow OpenStack's NovaFabian Mastenbroek
This change updates the FilterScheduler implementation to follow more closely the scheduler implementation in OpenStack's Nova. We now normalize the weights, support many of the filters and weights in OpenStack and support overcommitting resources.
2021-08-13build: Update Kotlin dependenciesFabian Mastenbroek
This change updates the Kotlin dependencies used by OpenDC to their latest version.
2021-06-24simulator: Re-implement performance interference modelFabian Mastenbroek
This change updates reimplements the performance interference model to work on top of the universal resource model in `opendc-simulator-resources`. This enables us to model interference and performance variability of other resources such as disk or network in the future.
2021-06-24format: Remove performance interference from trace readersFabian Mastenbroek
This change updates the trace reader implementation to remove their dependency on the performance interference model. In a future commit, we will instead pass the performance interference model via the host/hypervisor.
2021-06-21simulator: Re-organize compute simulator moduleFabian Mastenbroek
This change re-organizes the classes of the compute simulator module to make a clearer distinction between the hardware, firmware and software interfaces in this module.
2021-06-21exp: Enable interpreter sharing across hostsFabian Mastenbroek
This change enables the experiments to share the SimResourceInterpreter across multiple hosts, which allows updates to be scheduled efficiently for all machines at the same time. This is especially beneficial if the machines operate on the same time slices.
2021-06-17faas: Rename opendc-serverless to opendc-faasFabian Mastenbroek
This change renames the opendc-serverless module to opendc-faas to better distinguish between the two terms (Serverless and FaaS) and be clearer about the intent of the module. The opendc-faas module holds the code for the FaaS platform on top of OpenDC. Although this is one approach of doing serverless, serverless can also entail other services that will not be covered by this module.
2021-06-16exp: Fix power tracking for energy experimentsFabian Mastenbroek
This change fixes an issue where the power in the energy experiments is always reported as zero due to the changes in commit 652b869.
2021-06-15exp: Fix execution of energy experimentsFabian Mastenbroek
This change fixes the execution of the Energy21 experiments which failed due to various changes in the OpenDC codebase. First, the directory structure is now required to be pre-generated before the writer starts writing the experiment output. Second, we must include the configuration of the Capelin experiment in this experiment in order to workaround an issue with harness filtering.
2021-06-11simulator: Integrate power subsystem with compute subsystemFabian Mastenbroek
This change integrates the power subsystem of the simulator with the compute subsystem by exposing a new field on a SimBareMetalMachine, psu, which provides access to the machine's PSU, which in turn can be connected to a SimPowerOutlet.
generated by cgit v1.2.3 (git 2.25.1) at 2026-02-04 13:22:29 +0000