<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sunfish.git/opendc-trace/opendc-trace-parquet/src, branch master</title>
<subtitle>The OpenDC BSc thesis repository.
</subtitle>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/'/>
<entry>
<title>Unit System v2 (#243)</title>
<updated>2025-03-18T09:31:21+00:00</updated>
<author>
<name>Alessio Leonardo Tomei</name>
<email>122273875+T0mexX@users.noreply.github.com</email>
</author>
<published>2025-03-18T09:31:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=46ba81a45f7cb10c7f870bbf6946a46207ee353c'/>
<id>46ba81a45f7cb10c7f870bbf6946a46207ee353c</id>
<content type='text'>
* Separated `Time` unit into `TimeDelta` and `TimeStamp` + small fixes

Addition and subtruction between `Timestamp`s is not allowed, but any
other `Unit` operation/comparison is. `TimeDelta`s can be
added/subtructed to/form `Timestamp`s.

Deserialization of `Timestamp`:
- `Number` -&gt; interpreted as millis from Epoch
- `Instant` (string representation) -&gt; converted to Timestamp
- `Duration` (string representation) -&gt; interpreted as duration since
  Epoch (warn msg is logged)

Deserialization of `TimeDelta` is the same as `Time` was before, with the
diference that when an `Instant` is converted to an timedelta since Epoch
a warning message is logged.

* Unit System v2

- Merged `BoundedPercentage` and `UnboundedPercentage`
- Overrided all operation defined in `Unit` in all subclasses to avoid
  as much as possible value classes being boxed in bytecode. If units are used as generics
  (hence also functions defined in Unit&lt;T&gt;) they are boxed (as double would if used as generic).
- All units companions now subclass `UnitId`, and can be used as keys
  (e.g `Map&lt;UnitId, idk&gt;`), while offering `max` `min` and `zero`
  methods.
- Division between the same unit now returns `Percentage`
- Added `Iterable&lt;T&gt;.averageOfUnitOrNull(selector (T) -&gt; &lt;specific unit&gt;)`
- `ifNeg0ThenPos0()` now is optional and not invoked on every
  constructor
- Now methods in `Unit&lt;T&gt;` are all abstract, forcing override and avoid
  boxing in some cases
- Added `@UnintendedOperation` and `UnitOperationException` for methods
  that must be defined but are not intended for use (e.g. `Timestamp` +
  `Timestamp`)</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Separated `Time` unit into `TimeDelta` and `TimeStamp` + small fixes

Addition and subtruction between `Timestamp`s is not allowed, but any
other `Unit` operation/comparison is. `TimeDelta`s can be
added/subtructed to/form `Timestamp`s.

Deserialization of `Timestamp`:
- `Number` -&gt; interpreted as millis from Epoch
- `Instant` (string representation) -&gt; converted to Timestamp
- `Duration` (string representation) -&gt; interpreted as duration since
  Epoch (warn msg is logged)

Deserialization of `TimeDelta` is the same as `Time` was before, with the
diference that when an `Instant` is converted to an timedelta since Epoch
a warning message is logged.

* Unit System v2

- Merged `BoundedPercentage` and `UnboundedPercentage`
- Overrided all operation defined in `Unit` in all subclasses to avoid
  as much as possible value classes being boxed in bytecode. If units are used as generics
  (hence also functions defined in Unit&lt;T&gt;) they are boxed (as double would if used as generic).
- All units companions now subclass `UnitId`, and can be used as keys
  (e.g `Map&lt;UnitId, idk&gt;`), while offering `max` `min` and `zero`
  methods.
- Division between the same unit now returns `Percentage`
- Added `Iterable&lt;T&gt;.averageOfUnitOrNull(selector (T) -&gt; &lt;specific unit&gt;)`
- `ifNeg0ThenPos0()` now is optional and not invoked on every
  constructor
- Now methods in `Unit&lt;T&gt;` are all abstract, forcing override and avoid
  boxing in some cases
- Added `@UnintendedOperation` and `UnitOperationException` for methods
  that must be defined but are not intended for use (e.g. `Timestamp` +
  `Timestamp`)</pre>
</div>
</content>
</entry>
<entry>
<title>Refactored exporters. Allows output column selection in scenario (#241) (#241)</title>
<updated>2024-08-22T14:45:22+00:00</updated>
<author>
<name>Alessio Leonardo Tomei</name>
<email>122273875+T0mexX@users.noreply.github.com</email>
</author>
<published>2024-08-22T14:45:22+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=f9ffdfb29a3f08ac11e739494e754c81ef4f5157'/>
<id>f9ffdfb29a3f08ac11e739494e754c81ef4f5157</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Updated package versions, updated web server tests. (#207)</title>
<updated>2024-03-05T12:23:57+00:00</updated>
<author>
<name>Dante Niewenhuis</name>
<email>d.niewenhuis@hotmail.com</email>
</author>
<published>2024-03-05T12:23:57+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=5864cbcbfe2eb8c36ca05c3a39c7e5916aeecaec'/>
<id>5864cbcbfe2eb8c36ca05c3a39c7e5916aeecaec</id>
<content type='text'>
* Updated all package versions including kotlin. Updated all web-server tests to run.

* Changed the java version of the tests. OpenDC now only supports java 19.

* small update

* test update

* new update

* updated docker version to 19

* updated docker version to 19</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Updated all package versions including kotlin. Updated all web-server tests to run.

* Changed the java version of the tests. OpenDC now only supports java 19.

* small update

* test update

* new update

* updated docker version to 19

* updated docker version to 19</pre>
</div>
</content>
</entry>
<entry>
<title>fix(trace/wtf): Disable Parquet strict typing</title>
<updated>2022-12-14T00:24:24+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-12-13T23:59:47+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=3542350909b1213240e5097a1793a7c0733f6196'/>
<id>3542350909b1213240e5097a1793a7c0733f6196</id>
<content type='text'>
This change fixes an issue where some of the traces from the Workflow
Trace Archive would fail to load with the trace format in OpenDC. This
was caused by one of the fields being stored as a double, while the
formats expects it to be a long.

Parquet does not support unioning primitive types. Therefore, we have to
disable strict type checking when reading the file. Furthermore, we need
to support double entries for storing the workflow ids.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change fixes an issue where some of the traces from the Workflow
Trace Archive would fail to load with the trace format in OpenDC. This
was caused by one of the fields being stored as a double, while the
formats expects it to be a long.

Parquet does not support unioning primitive types. Therefore, we have to
disable strict type checking when reading the file. Furthermore, we need
to support double entries for storing the workflow ids.
</pre>
</div>
</content>
</entry>
<entry>
<title>build: Switch to Spotless for formatting</title>
<updated>2022-10-06T12:46:56+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-10-06T11:13:10+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=47357afd16f928260db34d4dd3e686fb9ee7c5ff'/>
<id>47357afd16f928260db34d4dd3e686fb9ee7c5ff</id>
<content type='text'>
This change updates the build configuration to use Spotless for code
formating of both Kotlin and Java.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the build configuration to use Spotless for code
formating of both Kotlin and Java.
</pre>
</div>
</content>
</entry>
<entry>
<title>style: Eliminate use of wildcard imports</title>
<updated>2022-10-06T10:51:27+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-10-06T10:51:27+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=402a8f55342c4565431c2a2e7287a70592f3fe33'/>
<id>402a8f55342c4565431c2a2e7287a70592f3fe33</id>
<content type='text'>
This change updates the repository to remove the use of wildcard imports
everywhere. Wildcard imports are not allowed by default by Ktlint as
well as Google's Java style guide.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the repository to remove the use of wildcard imports
everywhere. Wildcard imports are not allowed by default by Ktlint as
well as Google's Java style guide.
</pre>
</div>
</content>
</entry>
<entry>
<title>test(trace): Add conformance suite for OpenDC trace API</title>
<updated>2022-06-08T13:06:14+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-06-08T13:06:14+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=9d759c9bc987965fae8b0c16c000772c546bf3a2'/>
<id>9d759c9bc987965fae8b0c16c000772c546bf3a2</id>
<content type='text'>
This change adds a re-usable test suite for the interface of the OpenDC
trace API, so implementors can verify whether they match the
specification of the interfaces.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change adds a re-usable test suite for the interface of the OpenDC
trace API, so implementors can verify whether they match the
specification of the interfaces.
</pre>
</div>
</content>
</entry>
<entry>
<title>refactor(trace/parquet): Drop dependency on Avro</title>
<updated>2022-05-02T13:37:03+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-05-02T09:44:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=9411845b3f26536a1e6ea40504e396f19d25a09a'/>
<id>9411845b3f26536a1e6ea40504e396f19d25a09a</id>
<content type='text'>
This change updates the Parquet support library in OpenDC to not rely on
Avro, but instead interface directly with Parquet's reading and writing
functionality, providing less overhead.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the Parquet support library in OpenDC to not rely on
Avro, but instead interface directly with Parquet's reading and writing
functionality, providing less overhead.
</pre>
</div>
</content>
</entry>
<entry>
<title>perf(trace/opendc): Read records using low-level API</title>
<updated>2022-05-02T13:37:03+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-05-01T20:54:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=ea5e79fc77072e6151ee7952581b97e35a2027fb'/>
<id>ea5e79fc77072e6151ee7952581b97e35a2027fb</id>
<content type='text'>
This change updates the OpenDC VM format reader implementation to use
the low-level record reading APIs provided by the `parquet-mr` library
for improved performance. Previously, we used the `parquet-avro` library
to read/write Avro records in Parquet format, but that library carries
considerable overhead.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the OpenDC VM format reader implementation to use
the low-level record reading APIs provided by the `parquet-mr` library
for improved performance. Previously, we used the `parquet-avro` library
to read/write Avro records in Parquet format, but that library carries
considerable overhead.
</pre>
</div>
</content>
</entry>
<entry>
<title>refactor(trace/parquet): Support custom ReadSupport implementations</title>
<updated>2022-05-01T19:19:54+00:00</updated>
<author>
<name>Fabian Mastenbroek</name>
<email>mail.fabianm@gmail.com</email>
</author>
<published>2022-05-01T19:16:43+00:00</published>
<link rel='alternate' type='text/html' href='https://git.denounce.ai/sunfish.git/commit/?id=ee057033b4c534fdd3e8a9d2320d75035d30f27a'/>
<id>ee057033b4c534fdd3e8a9d2320d75035d30f27a</id>
<content type='text'>
This change updates the `LocalParquetReader` implementation to support
custom `ReadSupport` implementations, so we do not have to rely on the
Avro implementation necessarily.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change updates the `LocalParquetReader` implementation to support
custom `ReadSupport` implementations, so we do not have to rely on the
Avro implementation necessarily.
</pre>
</div>
</content>
</entry>
</feed>
