From 17327a642738e0500f9a007b32a46bb4f426f881 Mon Sep 17 00:00:00 2001 From: Fabian Mastenbroek Date: Fri, 14 May 2021 12:43:47 +0200 Subject: api: Remove Socket.IO endpoint from public API This change removes the Socket.IO endpoint from the public API now that we have switched to the REST API instead. This decreases the possible exposure to vulnerabilities as well as the maintenance burden. --- opendc-web/opendc-web-api/README.md | 65 ++++++++++++++++++++++++------------- 1 file changed, 43 insertions(+), 22 deletions(-) (limited to 'opendc-web/opendc-web-api/README.md') diff --git a/opendc-web/opendc-web-api/README.md b/opendc-web/opendc-web-api/README.md index 4932f823..e1d83daf 100644 --- a/opendc-web/opendc-web-api/README.md +++ b/opendc-web/opendc-web-api/README.md @@ -9,15 +9,19 @@
-The OpenDC web server is the bridge between OpenDC's frontend and database. It is built with Flask/SocketIO in Python and implements the OpenAPI-compliant [OpenDC API specification](../../opendc-api-spec.yml). +The OpenDC web server is the bridge between OpenDC's frontend and database. It is built with Flask/SocketIO in Python +and implements the OpenAPI-compliant [OpenDC API specification](../../opendc-api-spec.yml). -This document explains a high-level view of the web server architecture ([jump](#architecture)), and describes how to set up the web server for local development ([jump](#setup-for-local-development)). +This document explains a high-level view of the web server architecture ([jump](#architecture)), and describes how to +set up the web server for local development ([jump](#setup-for-local-development)). ## Architecture -The following diagram shows a high-level view of the architecture of the OpenDC web server. Squared-off colored boxes indicate packages (colors become more saturated as packages are nested); rounded-off boxes indicate individual components; dotted lines indicate control flow; and solid lines indicate data flow. +The following diagram shows a high-level view of the architecture of the OpenDC web server. Squared-off colored boxes +indicate packages (colors become more saturated as packages are nested); rounded-off boxes indicate individual +components; dotted lines indicate control flow; and solid lines indicate data flow. -![OpenDC Web Server Component Diagram](misc/artwork/opendc-web-server-component-diagram.png) +![OpenDC Web Server Component Diagram](docs/component-diagram.png) The OpenDC API is implemented by the `Main Server Loop`, which is the only component in the base package. @@ -25,74 +29,91 @@ The OpenDC API is implemented by the `Main Server Loop`, which is the only compo The `Util` package handles several miscellaneous tasks: -* `Database API`: Wraps database access functionality used by `Models` to read themselves from/write themselves into the database. +* `Database API`: Wraps database access functionality used by `Models` to read themselves from/write themselves into the + database. * `Exceptions`: Holds definitions for exceptions used throughout the web server. * `Parameter Checker`: Recursively checks whether required `Request` parameters are present and correctly typed. -* `REST`: Parses SocketIO and HTTP messages into `Request` objects, and calls the appropriate `API` endpoint to get a `Response` object to return to the `Main Server Loop`. +* `REST`: Parses HTTP messages into `Request` objects, and calls the appropriate `API` endpoint to get + a `Response` object to return to the `Main Server Loop`. ### API Package -The `API` package contains the logic for the HTTP methods in each API endpoint. Packages are structured to mirror the API: the code for the endpoint `GET api/projects`, for example, would be located at the `endpoint.py` inside the `projects` package (so at `api/projects/endpoint.py`). +The `API` package contains the logic for the HTTP methods in each API endpoint. Packages are structured to mirror the +API: the code for the endpoint `GET api/projects`, for example, would be located at the `endpoint.py` inside +the `projects` package (so at `api/projects/endpoint.py`). -An `endpoint.py` file contains methods for each HTTP method it supports, which takes a request as input (such as `def GET(request):`). Typically, such a method checks whether the parameters were passed correctly (using the `Parameter Checker`); fetches some model from the database; checks whether the data exists and is accessible by the user who made the request; possibly modifies this data and writes it back to the database; and returns a JSON representation of the model. +An `endpoint.py` file contains methods for each HTTP method it supports, which takes a request as input (such +as `def GET(request):`). Typically, such a method checks whether the parameters were passed correctly (using +the `Parameter Checker`); fetches some model from the database; checks whether the data exists and is accessible by the +user who made the request; possibly modifies this data and writes it back to the database; and returns a JSON +representation of the model. -The `REST` component dynamically imports the appropriate method from the appropriate `endpoint`, according to request it receives, and executes it. +The `REST` component dynamically imports the appropriate method from the appropriate `endpoint`, according to request it +receives, and executes it. ### Models Package -The `models` package contains the logic for mapping Python objects to their database representations. This involves an abstract `model` which has generic CRUD operations. Extensions of `model`, such as a `User` or `Project`, specify some more specific operations and their collection metadata. +The `models` package contains the logic for mapping Python objects to their database representations. This involves an +abstract `model` which has generic CRUD operations. Extensions of `model`, such as a `User` or `Project`, specify some +more specific operations and their collection metadata. `Endpoint`s import these `models` and use them to execute requests. ## Setup for Local Development -The following steps will guide you through setting up the OpenDC web server locally for development. To test individual endpoints, edit `static/index.html`. +The following steps will guide you through setting up the OpenDC web server locally for development. ### Local Setup #### Install requirements -Make sure you have Python 3.7+ installed (if not, get it [here](https://www.python.org/)), as well as pip (if not, get it [here](https://pip.pypa.io/en/stable/installing/)). Then run the following to install the requirements. +Make sure you have Python 3.7+ installed (if not, get it [here](https://www.python.org/)), as well as pip (if not, get +it [here](https://pip.pypa.io/en/stable/installing/)). Then run the following to install the requirements. ```bash pip install -r requirements.txt ``` -The web server also requires a running MongoDB instance. We recommend setting this up through docker, by running `docker-compose build` and `docker-compose up` in the [`mongodb` directory](../database) of the main OpenDC repository. +The web server also requires a running MongoDB instance. We recommend setting this up through docker, by +running `docker-compose build` and `docker-compose up` in the [`mongodb` directory](../../database) of the main OpenDC +repository. #### Get and configure the code -Clone OpenDC and follow the [instructions in the main repository](../) to set up a Google OAuth ID and environment variables. +Clone OpenDC and follow the [instructions in the main repository](../../) to set up a Google OAuth ID and environment +variables. **Important:** Be sure to set up environment variables according to those instructions, in a `.env` file. -If you want to test REST calls manually, add your own `OAUTH_CLIENT_ID` in `content=` on line `2` in `api/static/index.html`. - #### Set up the database -You can selectively run only the database services from the standard OpenDC `docker-compose` setup (in the root directory): +You can selectively run only the database services from the standard OpenDC `docker-compose` setup (in the root +directory): ```bash docker-compose build mongo mongo-express docker-compose up mongo mongo-express ``` -This will set you up with a running MongoDB instance and a visual inspection tool running on [localhost:8082](http://localhost:8082), with which you can view and manipulate the database. Add the simulator images to the command lists above if you want to test simulation capabilities, as well. +This will set you up with a running MongoDB instance and a visual inspection tool running +on [localhost:8082](http://localhost:8082), with which you can view and manipulate the database. Add the simulator +images to the command lists above if you want to test simulation capabilities, as well. ### Local Development Run the server. ```bash -cd api -python main.py +python3 -m flask run --port 8081 ``` -When editing the web server code, restart the server (`CTRL` + `c` followed by `python main.py` in the console running the server) to see the result of your changes. +When editing the web server code, restart the server (`CTRL` + `c` followed by `python app.py` in the console running +the server) to see the result of your changes. #### Code Style -To format all files, run `format.sh` in this directory. The script uses `yapf` internally to format everything automatically. +To format all files, run `format.sh` in this directory. The script uses `yapf` internally to format everything +automatically. To check if code style is up to modern standards, run `check.sh` in this directory. The script uses `pylint` internally. -- cgit v1.2.3 From 0c6ccca5fac44ab40671627fd3181e9b138672fa Mon Sep 17 00:00:00 2001 From: Fabian Mastenbroek Date: Fri, 14 May 2021 15:17:49 +0200 Subject: api: Migrate to Auth0 for API authorization This change updates the OpenDC API to use Auth0 for API authorization. This removes the hard dependency on Google for logging into OpenDC and simplifies implementation as we do not have to store user information anymore, other than the user identifier. --- opendc-web/opendc-web-api/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'opendc-web/opendc-web-api/README.md') diff --git a/opendc-web/opendc-web-api/README.md b/opendc-web/opendc-web-api/README.md index e1d83daf..af3cf927 100644 --- a/opendc-web/opendc-web-api/README.md +++ b/opendc-web/opendc-web-api/README.md @@ -33,8 +33,8 @@ The `Util` package handles several miscellaneous tasks: database. * `Exceptions`: Holds definitions for exceptions used throughout the web server. * `Parameter Checker`: Recursively checks whether required `Request` parameters are present and correctly typed. -* `REST`: Parses HTTP messages into `Request` objects, and calls the appropriate `API` endpoint to get - a `Response` object to return to the `Main Server Loop`. +* `REST`: Parses HTTP messages into `Request` objects, and calls the appropriate `API` endpoint to get a `Response` + object to return to the `Main Server Loop`. ### API Package @@ -80,8 +80,8 @@ repository. #### Get and configure the code -Clone OpenDC and follow the [instructions in the main repository](../../) to set up a Google OAuth ID and environment -variables. +Clone OpenDC and follow the [instructions in the main repository](../../) to set up an [Auth0](https://auth0.com) +application and environment variables. **Important:** Be sure to set up environment variables according to those instructions, in a `.env` file. -- cgit v1.2.3 From 2bdde75f0a95e070459004c9b7cf7c0d6618bc9d Mon Sep 17 00:00:00 2001 From: Fabian Mastenbroek Date: Mon, 17 May 2021 12:53:56 +0200 Subject: docs: Add instructions on configuring Auth0 for OpenDC This change updates the deployment guide to include instructions on configuring Auth0 for OpenDC. This process should not be much more difficult than creating the Google Application. --- opendc-web/opendc-web-api/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'opendc-web/opendc-web-api/README.md') diff --git a/opendc-web/opendc-web-api/README.md b/opendc-web/opendc-web-api/README.md index af3cf927..d1c469c1 100644 --- a/opendc-web/opendc-web-api/README.md +++ b/opendc-web/opendc-web-api/README.md @@ -80,7 +80,7 @@ repository. #### Get and configure the code -Clone OpenDC and follow the [instructions in the main repository](../../) to set up an [Auth0](https://auth0.com) +Clone OpenDC and follow the [instructions from the deployment guide](../../docs/deploy.md) to set up an [Auth0](https://auth0.com) application and environment variables. **Important:** Be sure to set up environment variables according to those instructions, in a `.env` file. -- cgit v1.2.3