| Age | Commit message (Collapse) | Author |
|---|
|
This change updates the OpenDC codebase to use OpenTelemetry v1.11,
which stabilizes the metrics API. This stabilization brings quite a few
breaking changes, so significant changes are necessary inside the OpenDC
codebase.
|
|
This change adds a new module, opendc-workflow-workload that contains
helper code for constructing workflow simulations using OpenDC.
|
|
This change redesigns the virtual machine interference algorithm to have
a fixed memory usage per `VmInterferenceModel` instance. Previously, for
every interference domain, a copy of the model would be created, leading
to OutOfMemory errors when running multiple experiments at the same
time.
|
|
This change renames the `opendc-simulator-resources` module into the
`opendc-simulator-flow` module to indicate that the core simulation
model of OpenDC is based around modelling and simulating flows.
Previously, the distinction between resource consumer and provider, and
input and output caused some confusion. By switching to a flow-based
model, this distinction is now clear (as in, the water flows from source
to consumer/sink).
|
|
This change drops the requirement for a clock parameter when
constructing a ComputeMetricExporter, since it will now derive the
timestamp from the recorded metrics.
|
|
This change adds a new API for writing traces in a trace format.
Currently, writing is only supported by the OpenDC VM format, but over
time the other formats will also have support for writing added.
|
|
This change updates the workload sampling implementation to be more
flexible in the way the workload is constructed. Users can now sample
multiple workloads at the same time using multiple samplers and use them
as a single workload to simulate.
|
|
This change adds support for creating flexible topologies by creating a
TopologyFactory interface that is responsible for configuring the hosts
of a compute service.
|
|
This change creates a new module for doing simulations with virtual
machine workloads. We have found that a lot of code in the Capelin
experiments code is being re-used by non-experiment modules.
|
|
This change updates the output schema for the experiment data to use
logical types where possible. This adds additional context for the
writer and the reader on how to process the column (efficiently).
|
|
This change standardizes the metrics emitted by SimHost instances and
their guests based on the OpenTelemetry semantic conventions. We now
also report CPU time as opposed to CPU work as this metric is more
commonly used.
|
|
This change updates the OpenDC compute service implementation with
multiple meters that follow the OpenTelemetry conventions.
|
|
This change refactors the telemetry implementation by creating a
separate MeterProvider per service or host. This means we have to keep
track of multiple metric producers, but that we can attach resource
information to each of the MeterProviders like we would in a real world
scenario.
|
|
This change improves the ParquetDataWriter class to support more complex
use-cases. It now allows subclasses to modify the writer options.
In addition to this, a subclass for writing server data is added.
|
|
This change adds a new API to the Table interface for accessing the
table columns that the table supports. This does not necessarily mean
that the column will have a value for every row, but that the table
format has defined this particular column.
|
|
This change adds a trace API implementation for the Azure VM traces.
|
|
This change fixes an issue where the last column in the Solvinity traces
is not parsed correctly, due to the last column having no whitespace at
the end to seek to.
|
|
This change removes the external class that holds the state of the
reader and instead puts the state in the reader implementation.
Maintaining a separate class for the state increases the complexity and
has worse performance characteristics due to the bytecode produced by
Kotlin for property accesses.
|
|
This change moves the fault injection logic directly into the
opendc-compute-simulator module, so that it can operate at a higher
abstraction. In the future, we might again split the module if we can
re-use some of its logic.
|
|
This change updates the Capelin experiment helpers to terminate a server
when it has reached its end-date.
|
|
This change adds support for specifying the distribution of the
failures, group size and duration for the fault injector.
|
|
|
|
This change moves the metric collection outside the Capelin codebase in
a separate module so other modules can also benefit from the compute
metric collection code.
|
|
|
|
|
|
This change updates the trace reading classes in the Capelin experiment
to use the new trace API in order to re-use many of the trace reading
parts.
|
|
This change updates the WTF trace reader to support the new streaming
trace API.
|
|
This change moves Bitbrains trace support into a separate module and
adds support for the new trace api.
|
|
This change removes the environment reader from the format library since
they are highly specific for the particular experiment. In the future,
we hope to have a single format to setup the entire datacenter (perhaps
similar to the format used by the web runner).
|
|
This change extracts the Parquet helpers outside format module into a
new module, in order to improve re-usability of these helpers.
|
|
This change updates the SimHost implementation to measure the power draw
of the machine without PSU overhead to make the results more realistic.
|
|
This change updates the metric exporter logic in the Capelin module and
reduces the number of allocations by looping directly over the
collection of metrics.
|
|
This change eliminates the unnecessary conversions from double to long
in the Capelin metric processing code.
|
|
This change upgrades the OpenTelemetry dependency to version 1.5, which
contains various breaking changes in the metrics API.
|
|
This change adds tests to the Capelin integration suite for performance
interference as well as failures. These test more accurately the
experiment setup.
|
|
This change fixes an issue with the workload sampler where the resulting
workload entries would not be ordered properly according to their
submission time.
|
|
This change updates the code for the Bitbrains trace reader and upgrades
the TraceConverter to re-use existing code of the Bitbrains trace
reader.
|
|
This change refactors the trace workload in the OpenDC simulator to
track execute a fragment based on the fragment's timestamp. This makes
sure that the trace is replayed identically to the original execution.
|
|
This change updates the FilterScheduler implementation to follow more
closely the scheduler implementation in OpenStack's Nova. We now
normalize the weights, support many of the filters and weights in
OpenStack and support overcommitting resources.
|
|
This change updates the Kotlin dependencies used by OpenDC to their
latest version.
|
|
This change updates reimplements the performance interference model to
work on top of the universal resource model in
`opendc-simulator-resources`. This enables us to model interference and
performance variability of other resources such as disk or network in
the future.
|
|
This change updates the trace reader implementation to remove their
dependency on the performance interference model. In a future commit, we
will instead pass the performance interference model via the
host/hypervisor.
|
|
This change re-organizes the classes of the compute simulator module to
make a clearer distinction between the hardware, firmware and software
interfaces in this module.
|
|
This change enables the experiments to share the SimResourceInterpreter
across multiple hosts, which allows updates to be scheduled efficiently
for all machines at the same time. This is especially beneficial if the
machines operate on the same time slices.
|
|
This change fixes the execution of the Energy21 experiments which failed
due to various changes in the OpenDC codebase. First, the directory
structure is now required to be pre-generated before the writer starts
writing the experiment output. Second, we must include the configuration
of the Capelin experiment in this experiment in order to workaround an
issue with harness filtering.
|
|
This change updates the Parquet writers used in the Capelin experiments
to use our OutputFile implementation for local files, to reduce our
dependency on Apache Hadoop.
|
|
This change updates the Parquet readers used in the Capelin experiments
to use our InputFile implementation for local files, to reduce our
dependency on Apache Hadoop.
|
|
This change addresses the deprecations that were caused by the migration
to Kotlin 1.5.
|
|
This change adds support for configuring the experiments via
configuration files using the TypeSafe config library. In the future, we
will also integrate support for configuration into the harness.
|
|
This change fixes the aggregation of the power draw metric. Previously,
if the power draw did not change between collection cycles, the power
draw would be reported as zero. This change uses OpenTelemetry Views to
collect the latest value of the power draw each cycle.
|
