---
sidebar_position: 2
title: M3SA Integration
hide_title: true
sidebar_label: M3SA Integration
description: M3SA Integration
---

# M3SA integration tutorial

M3SA is a tool able to perform "Multi-Meta-Model Simulation Analysis". The tool is designed to analyze the output of
simulations, by leveraging predictions, generate Multi-Model graphs, novel models, and more. M3SA can integrate with any
simulation infrastructure, as long as integration steps are followed.

We build our tool towards performance, scalability, and **universality**. In this document, we present the steps to
integrate our tool into your simulation infrastructure.

If you are using OpenDC, none of adaptation steps are necessary, yet they can be useful to understand the structure
of the tool. Step 3 is still necessary.

## Step 1: Adapt the simulator output folder structure

The first step is to adapt the I/O of your simulation to the format of our tool. The output folder structure should have
the
following format:

```
[1] ── {simulation-folder-name} 📁 🔧
[2]    ├── inputs 📁 🔒
[3]    │   └── {m3sa-config-file}.json 📄 🔧
[4]    │   └── {other input files / folders} 🔧
[5]    ├── outputs 📁 🔒
[6]    │   ├── raw-output 📁 🔒
[7]    │   │   ├── 0 📁 🔒
[8]    │   │   │   └── seed={your_seed}🔒
[9]    │   │   │       └── {simulation_data_file}.parquet 📄 🔧
[10]   │   │   │       └── {any other files / folders} ⚪
[11]   │   │   ├── 1 📁 ⚪ 🔒
[12]   │   │   │   └── seed={your_seed} 📁 ⚪ 🔒
[13]   │   │   │       └── {simulation_data_file}.parquet 📄 ⚪ 🔧
[14]   │   │   │       └── {any other files / folders} ⚪󠁪
[15]   │   │   ├── metamodel 📁 ⚪
[16]   │   │      └── seed={your_seed} 📁 ⚪
[17]   │   │           └── {your_metric_name}.parquet 📄 ⚪
[18]   │   │           └── {any other files / folders} ⚪
[19]   │   └── {any other files / folders} 📁 ⚪
[20]|  └──{any other files / folders} 📁 ⚪
```

📄 = file <br />
📁 = folder <br />
🔒 = fixed, the name of the folder/file must be the same.<br />
🔧 = flexible, the name of the folder/file can differ. However, the item must be present.<br />
⚪ = optional and flexible. The item can be absent. <br />

- [1] = the name of the analyzed folder.
- [2] = the _inputs_ folder, containing various inputs / configuration files.
- [3] = the configuration file for M3SA, flexible naming, but needs to be a JSON file
- [4],[10],[14],[18],[19],[20] = any other input files or folders.
- [5] = the _outputs_ folder, containing the raw-output. can contain any other files or folders, besides the raw-output
folder.
After running a simulation, also a "simulation-analysis" folder will be generated in this folder.
- [6] = raw-output folder, containing the raw output of the simulation.
- [7],[11] = the IDs of the models. Must always start from zero. Possible values are 0, 1, 2 ... n, and "metamodel". The
id
of "metamodel" is reserved for the Meta-Model. Any simulation data in the respective folder will be treated as
Meta-Model data.
- [8],[12] = the seed of the simulation. the seed must be the same for both [8], [12], and other equivalent, further
files.
- [9],[13] = the file in which the simulation data is stored. The name of the file can differ, but it must be a parquet
file.
- [15] = the Meta-Model folder, optional. If the folder is present, its data will be treated as Meta-Model data.
- [16] = the Meta-Model seed folder. The seed must be the same as the seed of the simulation.
- [17] = the Meta-Model output. The name of the file is of the type ```{your_metric_name}.parquet```. For example, if
you analyze CO2 emissions, the file will be named ```co2_emissions.parquet```.

---

## Step 2: Adapt the simulation file format

The simulator data file must be a 🪵 _parquet_ 🪵 file.

The file must contain (at least) the columns:

- timestamp: the timestamp, in miliseconds, of the data point (e.g., 30000, 60000, 90000) - the time unit is flexible.
- {metric_name}: the value of the metric at the given timestamp. This is the metric analyzed (e.g., CO2_emissions,
energy_usage).

e.g., if you are analyzing the CO2 emissions of a datacenter, for a timeperiod of 5 minutes, and the data is sampled
every 30 seconds, the file will look like this:

| timestamp | co2_emissions |
|-----------|---------------|
| 30000     | 31.2          |
| 60000     | 31.4          |
| 90000     | 28.5          |
| 120000    | 31.8          |
| 150000    | 51.5          |
| 180000    | 51.2          |
| 210000    | 51.4          |
| 240000    | 21.5          |
| 270000    | 21.8          |
| 300000    | 21.2          |

---

## Step 3: Running M3SA

### 3.1 Setup the Simulator Specifics

Update the simulation folder name ([9], [13], [17] from Step 1), in the
file ```simulator_specifics.py```, from ```opendc/src/python/simulator_specifics.py```.

### 3.2 Setup the python program arguments

### Arguments for Main.py Setup
Main.py takes two arguments:

1. Argument 1 is the path to the output directory where M3SA output files will be stored.
2. Argument 2 is the path to the input file that contains the configuration of M3SA.

e.g.,

```json
"simulation-123/outputs/" "simulation-123/inputs/m3sa-configurator.json"
```

### 3.3 Working directory Main.py Setup

Make sure to set the working directory to the directory where the main.py file is located.

e.g.,

```
/your/path/to-analyzer/src/main/python
```

If you are using OpenDC, you can set the working directory to the following path:

```
/your/path/opendc/opendc-analyze/src/main/python
```

---

## Optional: Step 4: Simulate and analyze, with one click

The simulation and analysis can be executed as a single command; if no errors are encountered, from the user
perspective,
this operation is atomic. We integrated M3SA into OpenDC to facilitate this process.

To further integrate M3SA into any simulation infrastructure, M3SA needs to called from
the simulation infrastructure, and provided the following running setup:

1. script language: Python
2. argument 1: the path of the output directory, in which M3SA output files will be stored
3. argument 2: the path of the input file, containing the configuration of M3SA
4. other language-specific setup

For example, the integration of the M3SA into OpenDC can be found
in ```Analyzr.kt``` from ```opendc-analyze/src/main/kotlin/Analyzr.kt```.
Below, we provide a snippet of the code:

```kotlin
val ANALYSIS_SCRIPTS_DIRECTORY: String = "./opendc-analyze/src/main/python"
val ABSOLUTE_SCRIPT_PATH: String =
    Path("$ANALYSIS_SCRIPTS_DIRECTORY/main.py").toAbsolutePath().normalize().toString()
val SCRIPT_LANGUAGE: String = "python3"

fun analyzeResults(outputFolderPath: String, analyzerSetupPath: String) {
    val process = ProcessBuilder(
        SCRIPT_LANGUAGE,
        ABSOLUTE_SCRIPT_PATH,
        outputFolderPath, // argument 1
        analyzerSetupPath // argument 2
    )
        .directory(Path(ANALYSIS_SCRIPTS_DIRECTORY).toFile())
        .start()

    val exitCode = process.waitFor()
    if (exitCode == 0) {
        println("[Analyzr.kt says] Analysis completed successfully.")
    } else {
        val errors = process.errorStream.bufferedReader().readText()
        println("[Analyzr.kt says] Exit code ${exitCode}; Error(s): $errors")
    }
}
```
