<feed xmlns='http://www.w3.org/2005/Atom'>
<title>opendc.git/opendc-trace/opendc-trace-parquet/src/main/kotlin/org/opendc/trace/util, branch master</title>
<subtitle>The OpenDC repository.
</subtitle>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/'/>
<entry>
<title>Unit System v2 (#243)</title>
<updated>2025-03-18T09:31:21+00:00</updated>
<author>
<name>Alessio Leonardo Tomei</name>
<email>122273875+T0mexX@users.noreply.github.com</email>
</author>
<published>2025-03-18T09:31:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=46ba81a45f7cb10c7f870bbf6946a46207ee353c'/>
<id>46ba81a45f7cb10c7f870bbf6946a46207ee353c</id>
<content type='text'>
* Separated `Time` unit into `TimeDelta` and `TimeStamp` + small fixes

Addition and subtruction between `Timestamp`s is not allowed, but any
other `Unit` operation/comparison is. `TimeDelta`s can be
added/subtructed to/form `Timestamp`s.

Deserialization of `Timestamp`:
- `Number` -&gt; interpreted as millis from Epoch
- `Instant` (string representation) -&gt; converted to Timestamp
- `Duration` (string representation) -&gt; interpreted as duration since
  Epoch (warn msg is logged)

Deserialization of `TimeDelta` is the same as `Time` was before, with the
diference that when an `Instant` is converted to an timedelta since Epoch
a warning message is logged.

* Unit System v2

- Merged `BoundedPercentage` and `UnboundedPercentage`
- Overrided all operation defined in `Unit` in all subclasses to avoid
  as much as possible value classes being boxed in bytecode. If units are used as generics
  (hence also functions defined in Unit&lt;T&gt;) they are boxed (as double would if used as generic).
- All units companions now subclass `UnitId`, and can be used as keys
  (e.g `Map&lt;UnitId, idk&gt;`), while offering `max` `min` and `zero`
  methods.
- Division between the same unit now returns `Percentage`
- Added `Iterable&lt;T&gt;.averageOfUnitOrNull(selector (T) -&gt; &lt;specific unit&gt;)`
- `ifNeg0ThenPos0()` now is optional and not invoked on every
  constructor
- Now methods in `Unit&lt;T&gt;` are all abstract, forcing override and avoid
  boxing in some cases
- Added `@UnintendedOperation` and `UnitOperationException` for methods
  that must be defined but are not intended for use (e.g. `Timestamp` +
  `Timestamp`)</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Separated `Time` unit into `TimeDelta` and `TimeStamp` + small fixes

Addition and subtruction between `Timestamp`s is not allowed, but any
other `Unit` operation/comparison is. `TimeDelta`s can be
added/subtructed to/form `Timestamp`s.

Deserialization of `Timestamp`:
- `Number` -&gt; interpreted as millis from Epoch
- `Instant` (string representation) -&gt; converted to Timestamp
- `Duration` (string representation) -&gt; interpreted as duration since
  Epoch (warn msg is logged)

Deserialization of `TimeDelta` is the same as `Time` was before, with the
diference that when an `Instant` is converted to an timedelta since Epoch
a warning message is logged.

* Unit System v2

- Merged `BoundedPercentage` and `UnboundedPercentage`
- Overrided all operation defined in `Unit` in all subclasses to avoid
  as much as possible value classes being boxed in bytecode. If units are used as generics
  (hence also functions defined in Unit&lt;T&gt;) they are boxed (as double would if used as generic).
- All units companions now subclass `UnitId`, and can be used as keys
  (e.g `Map&lt;UnitId, idk&gt;`), while offering `max` `min` and `zero`
  methods.
- Division between the same unit now returns `Percentage`
- Added `Iterable&lt;T&gt;.averageOfUnitOrNull(selector (T) -&gt; &lt;specific unit&gt;)`
- `ifNeg0ThenPos0()` now is optional and not invoked on every
  constructor
- Now methods in `Unit&lt;T&gt;` are all abstract, forcing override and avoid
  boxing in some cases
- Added `@UnintendedOperation` and `UnitOperationException` for methods
  that must be defined but are not intended for use (e.g. `Timestamp` +
  `Timestamp`)</pre>
</div>
</content>
</entry>
<entry>
<title>Refactored exporters. Allows output column selection in scenario (#241) (#241)</title>
<updated>2024-08-22T14:45:22+00:00</updated>
<author>
<name>Alessio Leonardo Tomei</name>
<email>122273875+T0mexX@users.noreply.github.com</email>
</author>
<published>2024-08-22T14:45:22+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=f9ffdfb29a3f08ac11e739494e754c81ef4f5157'/>
<id>f9ffdfb29a3f08ac11e739494e754c81ef4f5157</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Updated package versions, updated web server tests. (#207)</title>
<updated>2024-03-05T12:23:57+00:00</updated>
<author>
<name>Dante Niewenhuis</name>
<email>d.niewenhuis@hotmail.com</email>
</author>
<published>2024-03-05T12:23:57+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=5864cbcbfe2eb8c36ca05c3a39c7e5916aeecaec'/>
<id>5864cbcbfe2eb8c36ca05c3a39c7e5916aeecaec</id>
<content type='text'>
* Updated all package versions including kotlin. Updated all web-server tests to run.

* Changed the java version of the tests. OpenDC now only supports java 19.

* small update

* test update

* new update

* updated docker version to 19

* updated docker version to 19</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Updated all package versions including kotlin. Updated all web-server tests to run.

* Changed the java version of the tests. OpenDC now only supports java 19.

* small update

* test update

* new update

* updated docker version to 19

* updated docker version to 19</pre>
</div>
</content>
</entry>
<entry>
<title>fix(trace/wtf): Disable Parquet strict typing</title>
<updated>2022-12-14T00:24:24+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-12-13T23:59:47+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=3542350909b1213240e5097a1793a7c0733f6196'/>
<id>3542350909b1213240e5097a1793a7c0733f6196</id>
<content type='text'>
This change fixes an issue where some of the traces from the Workflow
Trace Archive would fail to load with the trace format in OpenDC. This
was caused by one of the fields being stored as a double, while the
formats expects it to be a long.

Parquet does not support unioning primitive types. Therefore, we have to
disable strict type checking when reading the file. Furthermore, we need
to support double entries for storing the workflow ids.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change fixes an issue where some of the traces from the Workflow
Trace Archive would fail to load with the trace format in OpenDC. This
was caused by one of the fields being stored as a double, while the
formats expects it to be a long.

Parquet does not support unioning primitive types. Therefore, we have to
disable strict type checking when reading the file. Furthermore, we need
to support double entries for storing the workflow ids.
</pre>
</div>
</content>
</entry>
<entry>
<title>build: Switch to Spotless for formatting</title>
<updated>2022-10-06T12:46:56+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-10-06T11:13:10+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=47357afd16f928260db34d4dd3e686fb9ee7c5ff'/>
<id>47357afd16f928260db34d4dd3e686fb9ee7c5ff</id>
<content type='text'>
This change updates the build configuration to use Spotless for code
formating of both Kotlin and Java.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the build configuration to use Spotless for code
formating of both Kotlin and Java.
</pre>
</div>
</content>
</entry>
<entry>
<title>refactor(trace/parquet): Drop dependency on Avro</title>
<updated>2022-05-02T13:37:03+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-05-02T09:44:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=9411845b3f26536a1e6ea40504e396f19d25a09a'/>
<id>9411845b3f26536a1e6ea40504e396f19d25a09a</id>
<content type='text'>
This change updates the Parquet support library in OpenDC to not rely on
Avro, but instead interface directly with Parquet's reading and writing
functionality, providing less overhead.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the Parquet support library in OpenDC to not rely on
Avro, but instead interface directly with Parquet's reading and writing
functionality, providing less overhead.
</pre>
</div>
</content>
</entry>
<entry>
<title>perf(trace/opendc): Read records using low-level API</title>
<updated>2022-05-02T13:37:03+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-05-01T20:54:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=ea5e79fc77072e6151ee7952581b97e35a2027fb'/>
<id>ea5e79fc77072e6151ee7952581b97e35a2027fb</id>
<content type='text'>
This change updates the OpenDC VM format reader implementation to use
the low-level record reading APIs provided by the `parquet-mr` library
for improved performance. Previously, we used the `parquet-avro` library
to read/write Avro records in Parquet format, but that library carries
considerable overhead.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the OpenDC VM format reader implementation to use
the low-level record reading APIs provided by the `parquet-mr` library
for improved performance. Previously, we used the `parquet-avro` library
to read/write Avro records in Parquet format, but that library carries
considerable overhead.
</pre>
</div>
</content>
</entry>
<entry>
<title>refactor(trace/parquet): Support custom ReadSupport implementations</title>
<updated>2022-05-01T19:19:54+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-05-01T19:16:43+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=ee057033b4c534fdd3e8a9d2320d75035d30f27a'/>
<id>ee057033b4c534fdd3e8a9d2320d75035d30f27a</id>
<content type='text'>
This change updates the `LocalParquetReader` implementation to support
custom `ReadSupport` implementations, so we do not have to rely on the
Avro implementation necessarily.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the `LocalParquetReader` implementation to support
custom `ReadSupport` implementations, so we do not have to rely on the
Avro implementation necessarily.
</pre>
</div>
</content>
</entry>
<entry>
<title>refactor(capelin): Extract common code out of Capelin experiments</title>
<updated>2021-09-19T12:02:26+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2021-09-17T15:48:02+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=859ce303f0b9110c7110b918e5957c2156fa8b26'/>
<id>859ce303f0b9110c7110b918e5957c2156fa8b26</id>
<content type='text'>
This change creates a new module for doing simulations with virtual
machine workloads. We have found that a lot of code in the Capelin
experiments code is being re-used by non-experiment modules.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change creates a new module for doing simulations with virtual
machine workloads. We have found that a lot of code in the Capelin
experiments code is being re-used by non-experiment modules.
</pre>
</div>
</content>
</entry>
<entry>
<title>refactor(trace): Extract Parquet helpers into separate module</title>
<updated>2021-09-02T07:26:42+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2021-08-31T13:14:46+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/opendc.git/commit/?id=e8cdfbcec3f75b3f303ce52bac5f5595a94555e4'/>
<id>e8cdfbcec3f75b3f303ce52bac5f5595a94555e4</id>
<content type='text'>
This change extracts the Parquet helpers outside format module into a
new module, in order to improve re-usability of these helpers.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change extracts the Parquet helpers outside format module into a
new module, in order to improve re-usability of these helpers.
</pre>
</div>
</content>
</entry>
</feed>
