summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorFabian Mastenbroek <mail.fabianm@gmail.com>2021-05-14 12:43:47 +0200
committerFabian Mastenbroek <mail.fabianm@gmail.com>2021-05-17 17:06:50 +0200
commit17327a642738e0500f9a007b32a46bb4f426f881 (patch)
tree733f897b96de27a105a127cbcafe788bea9de07e
parentd9e65dceb38cdb8dc4e464d388755f9456620566 (diff)
api: Remove Socket.IO endpoint from public API
This change removes the Socket.IO endpoint from the public API now that we have switched to the REST API instead. This decreases the possible exposure to vulnerabilities as well as the maintenance burden.
-rw-r--r--opendc-web/opendc-web-api/README.md65
-rwxr-xr-xopendc-web/opendc-web-api/app.py (renamed from opendc-web/opendc-web-api/main.py)33
-rw-r--r--opendc-web/opendc-web-api/conftest.py6
-rw-r--r--opendc-web/opendc-web-api/docs/component-diagram.png (renamed from opendc-web/opendc-web-api/misc/artwork/opendc-web-server-component-diagram.png)bin90161 -> 90161 bytes
-rw-r--r--opendc-web/opendc-web-api/requirements.txt2
-rw-r--r--opendc-web/opendc-web-api/static/index.html22
6 files changed, 55 insertions, 73 deletions
diff --git a/opendc-web/opendc-web-api/README.md b/opendc-web/opendc-web-api/README.md
index 4932f823..e1d83daf 100644
--- a/opendc-web/opendc-web-api/README.md
+++ b/opendc-web/opendc-web-api/README.md
@@ -9,15 +9,19 @@
<br>
-The OpenDC web server is the bridge between OpenDC's frontend and database. It is built with Flask/SocketIO in Python and implements the OpenAPI-compliant [OpenDC API specification](../../opendc-api-spec.yml).
+The OpenDC web server is the bridge between OpenDC's frontend and database. It is built with Flask/SocketIO in Python
+and implements the OpenAPI-compliant [OpenDC API specification](../../opendc-api-spec.yml).
-This document explains a high-level view of the web server architecture ([jump](#architecture)), and describes how to set up the web server for local development ([jump](#setup-for-local-development)).
+This document explains a high-level view of the web server architecture ([jump](#architecture)), and describes how to
+set up the web server for local development ([jump](#setup-for-local-development)).
## Architecture
-The following diagram shows a high-level view of the architecture of the OpenDC web server. Squared-off colored boxes indicate packages (colors become more saturated as packages are nested); rounded-off boxes indicate individual components; dotted lines indicate control flow; and solid lines indicate data flow.
+The following diagram shows a high-level view of the architecture of the OpenDC web server. Squared-off colored boxes
+indicate packages (colors become more saturated as packages are nested); rounded-off boxes indicate individual
+components; dotted lines indicate control flow; and solid lines indicate data flow.
-![OpenDC Web Server Component Diagram](misc/artwork/opendc-web-server-component-diagram.png)
+![OpenDC Web Server Component Diagram](docs/component-diagram.png)
The OpenDC API is implemented by the `Main Server Loop`, which is the only component in the base package.
@@ -25,74 +29,91 @@ The OpenDC API is implemented by the `Main Server Loop`, which is the only compo
The `Util` package handles several miscellaneous tasks:
-* `Database API`: Wraps database access functionality used by `Models` to read themselves from/write themselves into the database.
+* `Database API`: Wraps database access functionality used by `Models` to read themselves from/write themselves into the
+ database.
* `Exceptions`: Holds definitions for exceptions used throughout the web server.
* `Parameter Checker`: Recursively checks whether required `Request` parameters are present and correctly typed.
-* `REST`: Parses SocketIO and HTTP messages into `Request` objects, and calls the appropriate `API` endpoint to get a `Response` object to return to the `Main Server Loop`.
+* `REST`: Parses HTTP messages into `Request` objects, and calls the appropriate `API` endpoint to get
+ a `Response` object to return to the `Main Server Loop`.
### API Package
-The `API` package contains the logic for the HTTP methods in each API endpoint. Packages are structured to mirror the API: the code for the endpoint `GET api/projects`, for example, would be located at the `endpoint.py` inside the `projects` package (so at `api/projects/endpoint.py`).
+The `API` package contains the logic for the HTTP methods in each API endpoint. Packages are structured to mirror the
+API: the code for the endpoint `GET api/projects`, for example, would be located at the `endpoint.py` inside
+the `projects` package (so at `api/projects/endpoint.py`).
-An `endpoint.py` file contains methods for each HTTP method it supports, which takes a request as input (such as `def GET(request):`). Typically, such a method checks whether the parameters were passed correctly (using the `Parameter Checker`); fetches some model from the database; checks whether the data exists and is accessible by the user who made the request; possibly modifies this data and writes it back to the database; and returns a JSON representation of the model.
+An `endpoint.py` file contains methods for each HTTP method it supports, which takes a request as input (such
+as `def GET(request):`). Typically, such a method checks whether the parameters were passed correctly (using
+the `Parameter Checker`); fetches some model from the database; checks whether the data exists and is accessible by the
+user who made the request; possibly modifies this data and writes it back to the database; and returns a JSON
+representation of the model.
-The `REST` component dynamically imports the appropriate method from the appropriate `endpoint`, according to request it receives, and executes it.
+The `REST` component dynamically imports the appropriate method from the appropriate `endpoint`, according to request it
+receives, and executes it.
### Models Package
-The `models` package contains the logic for mapping Python objects to their database representations. This involves an abstract `model` which has generic CRUD operations. Extensions of `model`, such as a `User` or `Project`, specify some more specific operations and their collection metadata.
+The `models` package contains the logic for mapping Python objects to their database representations. This involves an
+abstract `model` which has generic CRUD operations. Extensions of `model`, such as a `User` or `Project`, specify some
+more specific operations and their collection metadata.
`Endpoint`s import these `models` and use them to execute requests.
## Setup for Local Development
-The following steps will guide you through setting up the OpenDC web server locally for development. To test individual endpoints, edit `static/index.html`.
+The following steps will guide you through setting up the OpenDC web server locally for development.
### Local Setup
#### Install requirements
-Make sure you have Python 3.7+ installed (if not, get it [here](https://www.python.org/)), as well as pip (if not, get it [here](https://pip.pypa.io/en/stable/installing/)). Then run the following to install the requirements.
+Make sure you have Python 3.7+ installed (if not, get it [here](https://www.python.org/)), as well as pip (if not, get
+it [here](https://pip.pypa.io/en/stable/installing/)). Then run the following to install the requirements.
```bash
pip install -r requirements.txt
```
-The web server also requires a running MongoDB instance. We recommend setting this up through docker, by running `docker-compose build` and `docker-compose up` in the [`mongodb` directory](../database) of the main OpenDC repository.
+The web server also requires a running MongoDB instance. We recommend setting this up through docker, by
+running `docker-compose build` and `docker-compose up` in the [`mongodb` directory](../../database) of the main OpenDC
+repository.
#### Get and configure the code
-Clone OpenDC and follow the [instructions in the main repository](../) to set up a Google OAuth ID and environment variables.
+Clone OpenDC and follow the [instructions in the main repository](../../) to set up a Google OAuth ID and environment
+variables.
**Important:** Be sure to set up environment variables according to those instructions, in a `.env` file.
-If you want to test REST calls manually, add your own `OAUTH_CLIENT_ID` in `content=` on line `2` in `api/static/index.html`.
-
#### Set up the database
-You can selectively run only the database services from the standard OpenDC `docker-compose` setup (in the root directory):
+You can selectively run only the database services from the standard OpenDC `docker-compose` setup (in the root
+directory):
```bash
docker-compose build mongo mongo-express
docker-compose up mongo mongo-express
```
-This will set you up with a running MongoDB instance and a visual inspection tool running on [localhost:8082](http://localhost:8082), with which you can view and manipulate the database. Add the simulator images to the command lists above if you want to test simulation capabilities, as well.
+This will set you up with a running MongoDB instance and a visual inspection tool running
+on [localhost:8082](http://localhost:8082), with which you can view and manipulate the database. Add the simulator
+images to the command lists above if you want to test simulation capabilities, as well.
### Local Development
Run the server.
```bash
-cd api
-python main.py
+python3 -m flask run --port 8081
```
-When editing the web server code, restart the server (`CTRL` + `c` followed by `python main.py` in the console running the server) to see the result of your changes.
+When editing the web server code, restart the server (`CTRL` + `c` followed by `python app.py` in the console running
+the server) to see the result of your changes.
#### Code Style
-To format all files, run `format.sh` in this directory. The script uses `yapf` internally to format everything automatically.
+To format all files, run `format.sh` in this directory. The script uses `yapf` internally to format everything
+automatically.
To check if code style is up to modern standards, run `check.sh` in this directory. The script uses `pylint` internally.
diff --git a/opendc-web/opendc-web-api/main.py b/opendc-web/opendc-web-api/app.py
index 5c6dac31..7a687678 100755
--- a/opendc-web/opendc-web-api/main.py
+++ b/opendc-web/opendc-web-api/app.py
@@ -5,7 +5,6 @@ import sys
import traceback
import urllib.request
-import flask_socketio
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from flask_compress import Compress
@@ -40,23 +39,21 @@ if not TEST_MODE:
host=os.environ.get('OPENDC_DB_HOST', 'localhost'))
# Set up the core app
-FLASK_CORE_APP = Flask(__name__)
-FLASK_CORE_APP.testing = TEST_MODE
-FLASK_CORE_APP.config['SECRET_KEY'] = os.environ['OPENDC_FLASK_SECRET']
-FLASK_CORE_APP.json_encoder = JSONEncoder
+app = Flask("opendc")
+app.testing = TEST_MODE
+app.config['SECRET_KEY'] = os.environ['OPENDC_FLASK_SECRET']
+app.json_encoder = JSONEncoder
# Set up CORS support
-CORS(FLASK_CORE_APP)
+CORS(app)
compress = Compress()
-compress.init_app(FLASK_CORE_APP)
-
-SOCKET_IO_CORE = flask_socketio.SocketIO(FLASK_CORE_APP, cors_allowed_origins="*")
+compress.init_app(app)
API_VERSIONS = {'v2'}
-@FLASK_CORE_APP.route('/tokensignin', methods=['POST'])
+@app.route('/tokensignin', methods=['POST'])
def sign_in():
"""Authenticate a user with Google sign in"""
@@ -92,7 +89,7 @@ def sign_in():
return jsonify(**data)
-@FLASK_CORE_APP.route('/<string:version>/<path:endpoint_path>', methods=['GET', 'POST', 'PUT', 'DELETE'])
+@app.route('/<string:version>/<path:endpoint_path>', methods=['GET', 'POST', 'PUT', 'DELETE'])
def api_call(version, endpoint_path):
"""Call an API endpoint directly over HTTP."""
@@ -137,17 +134,6 @@ def api_call(version, endpoint_path):
return flask_response
-@SOCKET_IO_CORE.on('request')
-def receive_message(message):
- """"Receive a SocketIO request"""
- (req, res) = _process_message(message)
-
- print(f'Socket: {req.method} to `/{req.path}` resulted in {res.status["code"]}: {res.status["description"]}')
- sys.stdout.flush()
-
- flask_socketio.emit('response', res.to_JSON(), json=True)
-
-
def _process_message(message):
"""Process a request message and return the response."""
@@ -184,5 +170,4 @@ def _process_message(message):
if __name__ == '__main__':
- print("Web server started on 8081")
- SOCKET_IO_CORE.run(FLASK_CORE_APP, host='0.0.0.0', port=8081, use_reloader=False)
+ app.run()
diff --git a/opendc-web/opendc-web-api/conftest.py b/opendc-web/opendc-web-api/conftest.py
index 1f4831b8..8bb55ccc 100644
--- a/opendc-web/opendc-web-api/conftest.py
+++ b/opendc-web/opendc-web-api/conftest.py
@@ -3,13 +3,13 @@ Configuration file for all unit tests.
"""
import pytest
-from main import FLASK_CORE_APP
+from app import app
@pytest.fixture
def client():
"""Returns a Flask API client to interact with."""
- FLASK_CORE_APP.config['TESTING'] = True
+ app.config['TESTING'] = True
- with FLASK_CORE_APP.test_client() as client:
+ with app.test_client() as client:
yield client
diff --git a/opendc-web/opendc-web-api/misc/artwork/opendc-web-server-component-diagram.png b/opendc-web/opendc-web-api/docs/component-diagram.png
index 91b26006..91b26006 100644
--- a/opendc-web/opendc-web-api/misc/artwork/opendc-web-server-component-diagram.png
+++ b/opendc-web/opendc-web-api/docs/component-diagram.png
Binary files differ
diff --git a/opendc-web/opendc-web-api/requirements.txt b/opendc-web/opendc-web-api/requirements.txt
index 146f1717..555ba751 100644
--- a/opendc-web/opendc-web-api/requirements.txt
+++ b/opendc-web/opendc-web-api/requirements.txt
@@ -33,8 +33,6 @@ pytest-cov==2.11.1
pytest-env==0.6.2
pytest-mock==3.2.0
python-dotenv==0.14.0
-python-engineio==3.13.2
-python-socketio==4.6.0
rsa==4.7
sentry-sdk==0.19.2
six==1.15.0
diff --git a/opendc-web/opendc-web-api/static/index.html b/opendc-web/opendc-web-api/static/index.html
deleted file mode 100644
index ac78cbfb..00000000
--- a/opendc-web/opendc-web-api/static/index.html
+++ /dev/null
@@ -1,22 +0,0 @@
-<script src="https://apis.google.com/js/platform.js" async defer></script>
-<meta name="google-signin-client_id" content="561588943542-fq7065hk47qdf3lfsc50ebll4spi6u76.apps.googleusercontent.com">
-<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/socket.io/1.3.6/socket.io.min.js"></script>
-<script>
- function onSignIn(googleUser) {
- document.getElementById('token').innerText = googleUser.getAuthResponse().id_token;
- }
-</script>
-<script>
- function signOut() {
- var auth2 = gapi.auth2.getAuthInstance();
- auth2.signOut().then(function () {
- console.log('User signed out.');
- });
- }
-</script>
-
-<div class="g-signin2" data-onsuccess="onSignIn"></div>
-<a href="#" onclick="signOut();">Sign out</a>
-
-<p>Your auth token:</p>
-<p id="token" style="word-wrap:break-word;">Loading...</p> \ No newline at end of file