## _Sunfish_: Enabling Predictive Analytics for Datacenters Through Digital Twinning

_Sunfish_ is a system that creates the pipeline between the Datacenter and its Digital Twin.

## Dependencies

_Sunfish_ is not a monolithic program. The entire ecosystem consists of:

1. [OpenDC](https://opendc.org) version 2.4
2. [Confluent Kafka](https://docs.confluent.io/kafka/overview.html) confluent version v4.39.0 
3. [PostgreSQL](https://www.postgresql.org/) version 18.4
4. [Redis](https://redis.io/) version 9.1.1
5. [Grafana](https://grafana.com/) version 13.1.0

Be aware the project will only work with this OpenDC (_i.e.,_ in this repository) as there are hidden Maven dependencies.

## Schema Registry
The Confluent ecosystem contains the Schema Registry, which enables seamless schema of the data interchange formats.
_Sunfish_ uses ProtoBuf (see `schema.proto`) file.
Each change to the `schema.proto` file must be updated with:
```bash
cd resources/experiments/
protoc --java_out=/home/matt/git/opendc/opendc-common/src/main/java/  schema.proto
curl -X DELETE http://localhost:8081/subjects/postgres-topic-value
```

## Confluent Configuration
First, see https://www.confluent.io/installation/.
Confluent JDBC sink and source (includes Postgres connector) files: 
https://www.confluent.io/hub/confluentinc/kafka-connect-jdbc

By default, the Kafka topic is named `postgres_topic`.

## Useful commands

To access a certain PostgreSQL database _e.g.,_ called "opendc":
`psql -d opendc`

This command selects everything from the topic which Kafka writes to.
`SELECT * FROM postgres_topic;`

Connect to Redis server via an excellent command-line tool:
`redis-cli -h localhost -p 6379`
Very useful tutorial https://redis.io/docs/latest/develop/data-types/streams/
See also https://redis.io/docs/latest/develop/tools/cli/

The below command returns all items in the Redis stream
`XRANGE postgres_topic - +`
`XTRIM postgres_topic MAXLEN 0`

To create a group for consumers to read from stream:
`XGROUP CREATE mystream mygroup 0`

## Contributing

Questions, suggestions and contributions are welcome and appreciated!
Please refer to the [contributing guidelines](CONTRIBUTING.md) for more details.

## License

OpenDC is distributed under the MIT license. See [LICENSE.txt](/LICENSE.txt).