| Age | Commit message (Collapse) | Author |
|---|
|
This change fixes an issue in the `SimBareMetalMachine` implementation
where the power usage was only updated after a non-zero duration. However,
this would mean that OpenDC would possibly report incorrect power usage
values when multiple convergence calls occured at the same timestamp.
|
|
This change removes the OpenTelemetry integration from the OpenDC
Compute modules. Previously, we chose to integrate OpenTelemetry to
provide a unified way to report metrics to the users.
Although this worked as expected, the overhead of the OpenTelemetry when
collecting metrics during simulation was considerable and lacked more
optimization opportunities (other than providing a separate API
implementation). Furthermore, since we were tied to OpenTelemetry's SDK
implementation, we experienced issues with throttling and registering
multiple instruments.
We will instead use another approach, where we expose the core metrics
in OpenDC via specialized interfaces (see the commits before) such that
access is fast and can be done without having to interface with
OpenTelemetry. In addition, we will provide an adapter to that is able
to forward these metrics to OpenTelemetry implementations, so we can
still integrate with the wider ecosystem.
|
|
This change introduces a `ComputeMetricReader` class that can be used as
a replacement for the `CoroutineMetricReader` class when reading metrics
from the Compute service. This implementation operates directly on a
`ComputeService` instance, providing better performance.
|
|
This change updates the compute support library to load the VM
interference model via the OpenDC trace library, which provides a
generic interface for reading interference models associated with
workload traces.
|
|
This change updates the simulator implementation to flush the active
progress when accessing the hypervisor counters. Previously, if the
counters were accessed, while the mux or consumer was in progress, its
counter values were not accurate.
|
|
This change updates the OpenDC codebase to use OpenTelemetry v1.11,
which stabilizes the metrics API. This stabilization brings quite a few
breaking changes, so significant changes are necessary inside the OpenDC
codebase.
|
|
This change adds a new module, opendc-workflow-workload that contains
helper code for constructing workflow simulations using OpenDC.
|
|
This change updates the implementation of the trace converter and
SimTrace implementation to support cases where there is a gap between
samples in the trace data.
This change allows users to specify what to do in case samples are
missing in the trace. The available options are specified in
`SimTrace.FillMode`. Currently, we support either carrying the previous
value forward or set the usage to zero.
|
|
This change redesigns the ComputeMonitor interface to reduce the number
of memory allocations necessary during a collection cycle.
|
|
This change redesigns the virtual machine interference algorithm to have
a fixed memory usage per `VmInterferenceModel` instance. Previously, for
every interference domain, a copy of the model would be created, leading
to OutOfMemory errors when running multiple experiments at the same
time.
|
|
This change addresses an issue where energy usage was not computed
correctly if the machine performed no work in between collection cycles.
|
|
This change updates the MaxMinFlowMultiplexer implementation to skip the
fair-share algorithm in case the total demand is lower than the
available capacity. In this case, no re-division of capacity is
necessary.
|
|
This change updates the MaxMinFlowMultiplexer implementation to
centrally manage the deadlines of the `FlowSource`s as opposed to each
source using its own timers. For large amounts of inputs, this is much
faster as the multiplexer already needs to traverse each input on a
timer expiration of an input.
|
|
This change adds a new interface to the SimHypervisor interface that
exposes the CPU time counters directly. These are derived from the flow
counters and will be used by SimHost to expose them via telemetry.
|
|
This change separates the push and pull flags in
FlowConsumerContextImpl, meaning that sources can now push directly
without pulling and vice versa.
|
|
This change removes the distributor and aggregator interfaces in favour
of a single switch interface. Since the switch interface is as powerful
as both the distributor and aggregator, we don't need the latter two.
|
|
This change removes the `onUpdate` callback from the
`SimResourceProviderLogic` interface. Instead, users should now update
counters using either `onConsume` or `onConverge`.
|
|
This change updates the simulator implementation to always invoke the
`SimResourceConsumer.onNext` callback when the resource context is
invalidated. This allows users to update the resource counter or do some
other work if the context has changed.
|
|
This change removes unnecessary allocations in the SimResourceInterpreter
caused by the way timers were allocated for the resource context.
|
|
This change removes the work and deadline properties from the
SimResourceCommand.Consume class and introduces a new property duration.
This property is now used in conjunction with the limit to compute the amount
of work processed by a resource provider.
Previously, we used both work and deadline to compute the duration and
the amount of remaining work at the end of a consumption. However, with
this change, we ensure that a resource consumption always runs at the
same speed once establishing, drastically simplifying the computation
for the amount of work processed during the consumption.
|
|
This change drops the requirement for a clock parameter when
constructing a ComputeMetricExporter, since it will now derive the
timestamp from the recorded metrics.
|
|
This change optimizes the OpenDC VM trace format by removing
unnecessary columns as well as optimizing the writer settings.
The new implementation still supports reading the old trace format in
case users run OpenDC with older workload traces.
|
|
This change updates the workload sampling implementation to be more
flexible in the way the workload is constructed. Users can now sample
multiple workloads at the same time using multiple samplers and use them
as a single workload to simulate.
|
|
This change adds support for creating flexible topologies by creating a
TopologyFactory interface that is responsible for configuring the hosts
of a compute service.
|
|
This change creates a new module for doing simulations with virtual
machine workloads. We have found that a lot of code in the Capelin
experiments code is being re-used by non-experiment modules.
|
|
This change standardizes the metrics emitted by SimHost instances and
their guests based on the OpenTelemetry semantic conventions. We now
also report CPU time as opposed to CPU work as this metric is more
commonly used.
|
|
This change updates the OpenDC compute service implementation with
multiple meters that follow the OpenTelemetry conventions.
|
|
This change refactors the telemetry implementation by creating a
separate MeterProvider per service or host. This means we have to keep
track of multiple metric producers, but that we can attach resource
information to each of the MeterProviders like we would in a real world
scenario.
|
|
This change updates the opentelemetry-java library to version 1.6.0.
|
|
This change moves the fault injection logic directly into the
opendc-compute-simulator module, so that it can operate at a higher
abstraction. In the future, we might again split the module if we can
re-use some of its logic.
|
|
This change updates the Capelin experiment helpers to terminate a server
when it has reached its end-date.
|
|
|
|
This change moves the metric collection outside the Capelin codebase in
a separate module so other modules can also benefit from the compute
metric collection code.
|
|
|
|
This change updates the WTF trace reader to support the new streaming
trace API.
|
|
This change removes the environment reader from the format library since
they are highly specific for the particular experiment. In the future,
we hope to have a single format to setup the entire datacenter (perhaps
similar to the format used by the web runner).
|
|
This change updates the SimHost implementation to measure the power draw
of the machine without PSU overhead to make the results more realistic.
|
|
This change eliminates the unnecessary conversions from double to long
in the Capelin metric processing code.
|
|
This change upgrades the OpenTelemetry dependency to version 1.5, which
contains various breaking changes in the metrics API.
|
|
This change adds tests to the Capelin integration suite for performance
interference as well as failures. These test more accurately the
experiment setup.
|
|
This change updates the Bitbrains trace tests with the updated trace
that does not hardcode the duration of the trace fragments.
|
|
|
|
This change fixes an issue with the simulator where it would record
overcomitted work if the output was updated before the deadline was
reached.
|
|
This change refactors the trace workload in the OpenDC simulator to
track execute a fragment based on the fragment's timestamp. This makes
sure that the trace is replayed identically to the original execution.
|
|
This change updates the FilterScheduler implementation to follow more
closely the scheduler implementation in OpenStack's Nova. We now
normalize the weights, support many of the filters and weights in
OpenStack and support overcommitting resources.
|
|
This change updates reimplements the performance interference model to
work on top of the universal resource model in
`opendc-simulator-resources`. This enables us to model interference and
performance variability of other resources such as disk or network in
the future.
|
|
This change updates the trace reader implementation to remove their
dependency on the performance interference model. In a future commit, we
will instead pass the performance interference model via the
host/hypervisor.
|
|
This change adds a new interface to the resources library for accessing
metrics of resources such as work, demand and overcommitted work. With
this change, we do not need an implementation specific listener
interface in SimResourceSwitchMaxMin anymore.
Another benefit of this approach is that updates will be scheduled more
efficiently and progress will only be reported once the system has
reached a steady-state for that timestamp.
|
|
This change introduces the SimResourceInterpreter which centralizes the
logic for scheduling and interpreting the communication between resource
consumer and provider.
This approach offers better performance due to avoiding invalidating the
state of the resource context when not necessary. Benchmarks show in the
best case a 5x performance improvement and at worst a 2x improvement.
|
|
This change fixes the aggregation of the power draw metric. Previously,
if the power draw did not change between collection cycles, the power
draw would be reported as zero. This change uses OpenTelemetry Views to
collect the latest value of the power draw each cycle.
|
