diff options
80 files changed, 2560 insertions, 386 deletions
@@ -129,3 +129,8 @@ opendc-experiments/opendc-experiments-m3sa/src/test/resources/outputs/* opendc-experiments/opendc-experiments-m3sa/src/test/resources/outputs opendc-experiments/opendc-experiments-m3sa/analysis.txt opendc-experiments/opendc-experiments-m3sa/src/test/resources/outputs/analysis.txt + +# Results and output files +results/* +output/* + @@ -1,39 +1,129 @@ -<a href="https://opendc.org/"> - <img src="https://opendc.org/img/logo.svg" alt="OpenDC logo" title="OpenDC" align="right" height="100" /> -</a> +## _Sunfish_: Enabling Predictive Analytics for Datacenters Through Digital Twinning -# OpenDC +_Sunfish_ is a system that creates the pipeline between the Datacenter and its Digital Twin. -Collaborative Datacenter Simulation and Exploration for Everybody +## Dependencies -[](/LICENSE.txt) -[](https://opendc.org/learn/category/documentation) -[](https://github.com/atlarge-research/opendc/releases) -[](https://github.com/atlarge-research/opendc/actions/workflows/build.yml) +_Sunfish_ is not a monolithic program. The entire ecosystem consists of: ------ +1. [OpenDC](https://opendc.org) version 2.4 +2. [Confluent Kafka](https://docs.confluent.io/kafka/overview.html) confluent version v4.39.0 +3. [PostgreSQL](https://www.postgresql.org/) version 18.4 +4. [Redis](https://redis.io/) version 9.1.1 +5. [Grafana](https://grafana.com/) version 13.1.0 -This repository is the home of the OpenDC project, a free and open-source platform for cloud datacenter simulation. +Be aware the project will only work with this OpenDC (_i.e.,_ in this repository) as there are hidden Maven dependencies. -## Latest Release +## Schema Registry +The Confluent ecosystem contains the Schema Registry, which enables seamless schema of the data interchange formats. +_Sunfish_ uses ProtoBuf (see `schema.proto`) file. +Each change to the `schema.proto` file must be updated with: +```bash +cd resources/experiments/ +protoc --java_out=/home/matt/git/opendc/opendc-common/src/main/java/ schema.proto +curl -X DELETE http://localhost:8081/subjects/postgres-topic-value +``` -All releases of OpenDC can be found [here](https://github.com/atlarge-research/opendc/releases). -The latest release will be marked _latest_. +## Confluent Configuration +First, see https://www.confluent.io/installation/. +Confluent JDBC sink and source (includes Postgres connector) files: +https://www.confluent.io/hub/confluentinc/kafka-connect-jdbc -## Documentation +By default, the Kafka topic is named `postgres_topic`. -You can find the OpenDC documentation [on the website](https://opendc.org). -The documentation is divided into several sections: +The scripts to start the Confluent Kafka (assuming installation in `opt/confluent`) instance are listed below. First, start Kafka Connect: -* [Getting Started](https://opendc.org/learn/category/getting-started) -* [Tutorials](https://opendc.org/learn/category/tutorials) -* [Where to Get Support](https://opendc.org/community/support) -* [Contributing Guide](https://opendc.org/community/contributing) +``` +/opt/confluent/bin/connect-standalone \ +/opt/confluent/etc/kafka/connect-standalone.properties \ +/opt/confluent/share/confluent-common/connectors/sink-jdbc.properties \ +/opt/confluent/share/confluent-common/connectors/sink-redis.properties +``` -You find information about the research projects and publications related to OpenDC in the [research section](https://opendc.org/research/publications). +Then, format Kafka Storage: -The source code for the website is located in the [gh-pages branch](https://github.com/atlarge-research/opendc/tree/gh-pages). -Push to that branch to update the website. +``` +/opt/confluent/bin/kafka-storage format -t \ +2vi2WtHxQAOPyXb1Bj1Jvw -c /opt/confluent/etc/kafka/server.properties \ +--standalone > /dev/null 2>&1; echo 0 +``` + +Afterwards, start Kafka Broker: + +``` +/opt/confluent/bin/kafka-server-start /opt/confluent/etc/kafka/server.properties +``` + +Then create a new Kafka Topic (_e.g.,_ "posgres-topic"): + +``` +kafka-topics --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1 +``` + +Lastly, run the Schema Registry: +``` +/opt/confluent/bin/schema-registry-start \ +/opt/confluent/etc/schema-registry/schema-registry.properties +``` + +At this point, ensure PostgreSQL, Redis and Grafana are up and running on the system, on their default port configurations. +To see metrics flow seamlessly into PostgreSQL, once can run the following command (_i.e.,_ assuming the database name is "opendc"): + +``` +sudo su postgres; psql -d opendc +``` + +Additionally, to interact with the Redis cache, we suggest to use the excellent redis-cli tool: + +``` +redis-cli -h localhost -p 6379 +``` + +The following commands can be used to list the contents of the cache (_i.e.,_ assuming stream name "postgres_topic"), and to clear the cache respectively: + +``` +XRANGE postgres_topic - + +XTRIM postgres_topic MAXLEN 0 +``` + +For the predictive analytics in our implementation it is useful to create a separate consumer group: + +``` +XGROUP CREATE mystream mygroup 0 +``` + +Lastly, each time you change the database schema, you must run (_i.e.,_ assuming `schema.proto` lives in `resources/experiments`): + +``` +cd resources/experiments/; +protoc --java_out=/home/matt/git/opendc/opendc-common/src/main/java/ schema.proto +``` + +_Sunfish_ acts as the digital twin and the modified OpenDC (this repository) acts as the physical datacenter. +Both contain modifications. +Additionally: create a `.venv` in `python_scripts` and `http_server` and install all the dependencies. +Then, run respectively the HTTP Server and the Python Scripts. +Afterwards, start IntellijIDEA in this repository and run some experiments. + +## Useful commands + +To access a certain PostgreSQL database _e.g.,_ called "opendc": +`psql -d opendc` + +This command selects everything from the topic which Kafka writes to. +`SELECT * FROM postgres_topic;` + +Connect to Redis server via an excellent command-line tool: +`redis-cli -h localhost -p 6379` +Very useful tutorial https://redis.io/docs/latest/develop/data-types/streams/ +See also https://redis.io/docs/latest/develop/tools/cli/ + +The below command returns all items in the Redis stream +`XRANGE postgres_topic - +` +`XTRIM postgres_topic MAXLEN 0` + +To create a group for consumers to read from stream: +`XGROUP CREATE mystream mygroup 0` ## Contributing diff --git a/http_server/LICENSE b/http_server/LICENSE new file mode 100644 index 00000000..f288702d --- /dev/null +++ b/http_server/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see <https://www.gnu.org/licenses/>. + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + <program> Copyright (C) <year> <name of author> + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +<https://www.gnu.org/licenses/>. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +<https://www.gnu.org/licenses/why-not-lgpl.html>. diff --git a/http_server/README.md b/http_server/README.md new file mode 100644 index 00000000..e69de29b --- /dev/null +++ b/http_server/README.md diff --git a/http_server/pyproject.toml b/http_server/pyproject.toml new file mode 100644 index 00000000..22dbb379 --- /dev/null +++ b/http_server/pyproject.toml @@ -0,0 +1,7 @@ +[project] +name = "http_server" +version = "0.1" +dynamic = ["version"] +description = "My Python HTTP server" +readme = "README.md" +license = "GPL-3.0-or-later" diff --git a/http_server/requirements.txt b/http_server/requirements.txt new file mode 100644 index 00000000..812865e8 --- /dev/null +++ b/http_server/requirements.txt @@ -0,0 +1,25 @@ +annotated-doc==0.0.4 +annotated-types==0.7.0 +anyio==4.14.0 +certifi==2026.6.17 +charset-normalizer==3.4.7 +click==8.4.1 +fastapi==0.137.2 +h11==0.16.0 +idna==3.18 +numpy==2.5.0 +pandas==3.0.3 +pyarrow==24.0.0 +pydantic==2.13.4 +pydantic_core==2.46.4 +python-dateutil==2.9.0.post0 +redis==8.0.0 +redis-streams==0.3.0 +requests==2.34.2 +six==1.17.0 +starlette==1.3.1 +tabulate==0.10.0 +typing-inspection==0.4.2 +typing_extensions==4.15.0 +urllib3==2.7.0 +uvicorn==0.49.0 diff --git a/http_server/src/http_server/__init__.py b/http_server/src/http_server/__init__.py new file mode 100644 index 00000000..e69de29b --- /dev/null +++ b/http_server/src/http_server/__init__.py diff --git a/http_server/src/http_server/__main__.py b/http_server/src/http_server/__main__.py new file mode 100644 index 00000000..fe32b0f2 --- /dev/null +++ b/http_server/src/http_server/__main__.py @@ -0,0 +1,33 @@ +import uvicorn +import queue +from fastapi import FastAPI +from pydantic import BaseModel + +PORT = 1234 +HOST = "localhost" +APP = FastAPI() + +q = queue.Queue() + + +class Item(BaseModel): + Name: str + Type: str + Timestamp: str + + +@APP.get("/insight") +def read_assets(): + if q.empty(): + return + return q.get_nowait() + + +@APP.post("/insight") +def post_assets(item: Item): + q.put(item) + return item + + +if __name__ == "__main__": + uvicorn.run(APP, host=HOST, port=PORT) diff --git a/http_server/src/http_server/__pycache__/__main__.cpython-314.pyc b/http_server/src/http_server/__pycache__/__main__.cpython-314.pyc Binary files differnew file mode 100644 index 00000000..55032434 --- /dev/null +++ b/http_server/src/http_server/__pycache__/__main__.cpython-314.pyc diff --git a/opendc-common/src/main/java/org/opendc/common/ProtobufMetrics.java b/opendc-common/src/main/java/org/opendc/common/ProtobufMetrics.java index c0f15f63..b65155ca 100644 --- a/opendc-common/src/main/java/org/opendc/common/ProtobufMetrics.java +++ b/opendc-common/src/main/java/org/opendc/common/ProtobufMetrics.java @@ -71,6 +71,18 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { * @return The energyusage. */ double getEnergyusage(); + + /** + * <code>double uptime = 6;</code> + * @return The uptime. + */ + double getUptime(); + + /** + * <code>double downtime = 7;</code> + * @return The downtime. + */ + double getDowntime(); } /** * Protobuf type {@code proto.ProtoExport} @@ -227,6 +239,28 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { return energyusage_; } + public static final int UPTIME_FIELD_NUMBER = 6; + private double uptime_ = 0D; + /** + * <code>double uptime = 6;</code> + * @return The uptime. + */ + @java.lang.Override + public double getUptime() { + return uptime_; + } + + public static final int DOWNTIME_FIELD_NUMBER = 7; + private double downtime_ = 0D; + /** + * <code>double downtime = 7;</code> + * @return The downtime. + */ + @java.lang.Override + public double getDowntime() { + return downtime_; + } + private byte memoizedIsInitialized = -1; @java.lang.Override public final boolean isInitialized() { @@ -256,6 +290,12 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { if (java.lang.Double.doubleToRawLongBits(energyusage_) != 0) { output.writeDouble(5, energyusage_); } + if (java.lang.Double.doubleToRawLongBits(uptime_) != 0) { + output.writeDouble(6, uptime_); + } + if (java.lang.Double.doubleToRawLongBits(downtime_) != 0) { + output.writeDouble(7, downtime_); + } getUnknownFields().writeTo(output); } private int computeSerializedSize_0() { @@ -278,6 +318,14 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { size += com.google.protobuf.CodedOutputStream .computeDoubleSize(5, energyusage_); } + if (java.lang.Double.doubleToRawLongBits(uptime_) != 0) { + size += com.google.protobuf.CodedOutputStream + .computeDoubleSize(6, uptime_); + } + if (java.lang.Double.doubleToRawLongBits(downtime_) != 0) { + size += com.google.protobuf.CodedOutputStream + .computeDoubleSize(7, downtime_); + } return size; } @java.lang.Override @@ -314,6 +362,12 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { if (java.lang.Double.doubleToLongBits(getEnergyusage()) != java.lang.Double.doubleToLongBits( other.getEnergyusage())) return false; + if (java.lang.Double.doubleToLongBits(getUptime()) + != java.lang.Double.doubleToLongBits( + other.getUptime())) return false; + if (java.lang.Double.doubleToLongBits(getDowntime()) + != java.lang.Double.doubleToLongBits( + other.getDowntime())) return false; if (!getUnknownFields().equals(other.getUnknownFields())) return false; return true; } @@ -337,6 +391,12 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { hash = (37 * hash) + ENERGYUSAGE_FIELD_NUMBER; hash = (53 * hash) + com.google.protobuf.Internal.hashLong( java.lang.Double.doubleToLongBits(getEnergyusage())); + hash = (37 * hash) + UPTIME_FIELD_NUMBER; + hash = (53 * hash) + com.google.protobuf.Internal.hashLong( + java.lang.Double.doubleToLongBits(getUptime())); + hash = (37 * hash) + DOWNTIME_FIELD_NUMBER; + hash = (53 * hash) + com.google.protobuf.Internal.hashLong( + java.lang.Double.doubleToLongBits(getDowntime())); hash = (29 * hash) + getUnknownFields().hashCode(); memoizedHashCode = hash; return hash; @@ -473,6 +533,8 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { tasksactive_ = 0; cpuutilization_ = 0D; energyusage_ = 0D; + uptime_ = 0D; + downtime_ = 0D; return this; } @@ -521,6 +583,12 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { if (((from_bitField0_ & 0x00000010) != 0)) { result.energyusage_ = energyusage_; } + if (((from_bitField0_ & 0x00000020) != 0)) { + result.uptime_ = uptime_; + } + if (((from_bitField0_ & 0x00000040) != 0)) { + result.downtime_ = downtime_; + } } @java.lang.Override @@ -554,6 +622,12 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { if (java.lang.Double.doubleToRawLongBits(other.getEnergyusage()) != 0) { setEnergyusage(other.getEnergyusage()); } + if (java.lang.Double.doubleToRawLongBits(other.getUptime()) != 0) { + setUptime(other.getUptime()); + } + if (java.lang.Double.doubleToRawLongBits(other.getDowntime()) != 0) { + setDowntime(other.getDowntime()); + } this.mergeUnknownFields(other.getUnknownFields()); onChanged(); return this; @@ -605,6 +679,16 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { bitField0_ |= 0x00000010; break; } // case 41 + case 49: { + uptime_ = input.readDouble(); + bitField0_ |= 0x00000020; + break; + } // case 49 + case 57: { + downtime_ = input.readDouble(); + bitField0_ |= 0x00000040; + break; + } // case 57 default: { if (!super.parseUnknownField(input, extensionRegistry, tag)) { done = true; // was an endgroup tag @@ -862,6 +946,70 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { return this; } + private double uptime_ ; + /** + * <code>double uptime = 6;</code> + * @return The uptime. + */ + @java.lang.Override + public double getUptime() { + return uptime_; + } + /** + * <code>double uptime = 6;</code> + * @param value The uptime to set. + * @return This builder for chaining. + */ + public Builder setUptime(double value) { + + uptime_ = value; + bitField0_ |= 0x00000020; + onChanged(); + return this; + } + /** + * <code>double uptime = 6;</code> + * @return This builder for chaining. + */ + public Builder clearUptime() { + bitField0_ = (bitField0_ & ~0x00000020); + uptime_ = 0D; + onChanged(); + return this; + } + + private double downtime_ ; + /** + * <code>double downtime = 7;</code> + * @return The downtime. + */ + @java.lang.Override + public double getDowntime() { + return downtime_; + } + /** + * <code>double downtime = 7;</code> + * @param value The downtime to set. + * @return This builder for chaining. + */ + public Builder setDowntime(double value) { + + downtime_ = value; + bitField0_ |= 0x00000040; + onChanged(); + return this; + } + /** + * <code>double downtime = 7;</code> + * @return This builder for chaining. + */ + public Builder clearDowntime() { + bitField0_ = (bitField0_ & ~0x00000040); + downtime_ = 0D; + onChanged(); + return this; + } + // @@protoc_insertion_point(builder_scope:proto.ProtoExport) } @@ -927,11 +1075,12 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { descriptor; static { java.lang.String[] descriptorData = { - "\n\014schema.proto\022\005proto\"s\n\013ProtoExport\022\021\n\t" + - "timestamp\030\001 \001(\t\022\017\n\007host_id\030\002 \001(\t\022\023\n\013task" + - "sactive\030\003 \001(\005\022\026\n\016cpuutilization\030\004 \001(\001\022\023\n" + - "\013energyusage\030\005 \001(\001B$\n\021org.opendc.commonB" + - "\017ProtobufMetricsb\006proto3" + "\n\014schema.proto\022\005proto\"\225\001\n\013ProtoExport\022\021\n" + + "\ttimestamp\030\001 \001(\t\022\017\n\007host_id\030\002 \001(\t\022\023\n\013tas" + + "ksactive\030\003 \001(\005\022\026\n\016cpuutilization\030\004 \001(\001\022\023" + + "\n\013energyusage\030\005 \001(\001\022\016\n\006uptime\030\006 \001(\001\022\020\n\010d" + + "owntime\030\007 \001(\001B$\n\021org.opendc.commonB\017Prot" + + "obufMetricsb\006proto3" }; descriptor = com.google.protobuf.Descriptors.FileDescriptor .internalBuildGeneratedFileFrom(descriptorData, @@ -942,7 +1091,7 @@ public final class ProtobufMetrics extends com.google.protobuf.GeneratedFile { internal_static_proto_ProtoExport_fieldAccessorTable = new com.google.protobuf.GeneratedMessage.FieldAccessorTable( internal_static_proto_ProtoExport_descriptor, - new java.lang.String[] { "Timestamp", "HostId", "Tasksactive", "Cpuutilization", "Energyusage", }); + new java.lang.String[] { "Timestamp", "HostId", "Tasksactive", "Cpuutilization", "Energyusage", "Uptime", "Downtime", }); descriptor.resolveAllFeaturesImmutable(); } diff --git a/opendc-common/src/main/kotlin/org/opendc/common/annotations/Endpoint.kt b/opendc-common/src/main/kotlin/org/opendc/common/annotations/Endpoint.kt deleted file mode 100644 index 4ef08a71..00000000 --- a/opendc-common/src/main/kotlin/org/opendc/common/annotations/Endpoint.kt +++ /dev/null @@ -1,6 +0,0 @@ -package org.opendc.common.annotations - -@RequiresOptIn(message = "This is a registered API endpoint.") -@Retention(AnnotationRetention.BINARY) -@Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION, AnnotationTarget.PROPERTY, AnnotationTarget.CONSTRUCTOR) -public annotation class Endpoint(val method: String, val name : String)
\ No newline at end of file diff --git a/opendc-common/src/main/kotlin/org/opendc/common/api/AssetsController.kt b/opendc-common/src/main/kotlin/org/opendc/common/api/AssetsController.kt deleted file mode 100644 index c6f34d19..00000000 --- a/opendc-common/src/main/kotlin/org/opendc/common/api/AssetsController.kt +++ /dev/null @@ -1,72 +0,0 @@ -package org.opendc.common.api - -import org.opendc.common.annotations.Endpoint -import io.javalin.http.Handler - -/** - * This class represents the `/assets` endpoint. - * - * @author Mateusz Kwiatkowski - * - * */ - - -//TODO: fix -> this is all wrong. -// Sending the experiment file is completely useless. -// You need to send tasks.parquet -public class AssetsController { - /** - * Returns a concatenated JSON string of all assets. - */ - @Endpoint("GET","/assets") - public fun getAssets() : Handler { - return Handler { ctx -> ctx.status(200) - println(ctx.body()) - } - } - - /** - * Returns an asset with `id` as a JSON string. - */ - @Endpoint("GET", "/assets/{id}") - public fun getAssetsId(): Handler { - return Handler { ctx -> ctx.status(200) } - } - - /** - * Saves the asset specified in the HTTP body. - * Returns the asset `id`. - */ - @Endpoint("POST", "/assets") - public fun postAsset() : Handler { - return Handler { ctx -> ctx.status(200) - println(ctx.body()) - } - } - - /** - * Modifies the specified asset. - * Deletes all results from experiments with this asset. - */ - @Endpoint("PUT", "/assets/{id}") - public fun putAssetId() : Handler { - return Handler { ctx -> ctx.status(200) } - } - - /** - * Deletes an asset with `id`. - * Deletes all results from experiments with this asset. - */ - @Endpoint("DELETE", "/assets/{id}") - public fun deleteAssetId() : Handler { - return Handler { ctx -> ctx.status(200) } - } - - /** - * Deletes all assets - */ - @Endpoint("DELETE", "/assets") - public fun deleteAsset() : Handler { - return Handler { ctx -> ctx.status(200) } - } -}
\ No newline at end of file diff --git a/opendc-common/src/main/kotlin/org/opendc/common/api/ResourceController.kt b/opendc-common/src/main/kotlin/org/opendc/common/api/ResourceController.kt deleted file mode 100644 index cf1d3cac..00000000 --- a/opendc-common/src/main/kotlin/org/opendc/common/api/ResourceController.kt +++ /dev/null @@ -1,30 +0,0 @@ -package org.opendc.common.api - -import org.opendc.common.annotations.Endpoint - -/** - * This class represents the `/resources` endpoint. - * - * @author Mateusz Kwiatkowski - * - * */ - -public class ResourceController { - - /** - * Returns all data analytics for all experiments. - */ - @Endpoint("GET", "/resources") - public fun getResources() { - return - } - - /** - * Returns data analytics for experiment with `id`. - * - * */ - @Endpoint("GET", "/resources") - public fun getResourcesId() { - return - } -}
\ No newline at end of file diff --git a/opendc-common/src/main/kotlin/org/opendc/common/utils/HTTPClient.kt b/opendc-common/src/main/kotlin/org/opendc/common/utils/HTTPClient.kt index fc5bc57b..5faf50b0 100644 --- a/opendc-common/src/main/kotlin/org/opendc/common/utils/HTTPClient.kt +++ b/opendc-common/src/main/kotlin/org/opendc/common/utils/HTTPClient.kt @@ -26,7 +26,18 @@ public class HTTPClient private constructor() { } } - // TODO: this class must send the experiment JSON file to the digital twin + public fun checkForInsights() { + val request = HttpRequest.newBuilder() + .uri(URI.create("http://localhost:1234/insight")) + .header("Content-type", "text/plain") + .GET() + .build() + val response = client?.send(request, ofString()) +// if (response?.body() != "null") { + // println(response?.body()) + //} + } + public fun sendExperiment(experiment: File) { val input = experiment.inputStream() val charArray = CharArray(experiment.length().toInt()) @@ -35,12 +46,12 @@ public class HTTPClient private constructor() { isr.read(charArray) val request = HttpRequest.newBuilder() - .uri(URI.create("http://localhost:8080/assets")) + .uri(URI.create("http://localhost:1234/assets")) .header("Content-type", "application/json") - // TODO: this is obviously wrong, find an efficient way to send JSON over network + // TODO(this is obviously wrong, find an efficient way to send JSON over network) .POST(HttpRequest.BodyPublishers.ofString(String(charArray))) .build() val response = client?.send(request, ofString()) check(response?.statusCode() == 200) } -}
\ No newline at end of file +} diff --git a/opendc-common/src/main/kotlin/org/opendc/common/utils/JavalinRunner.kt b/opendc-common/src/main/kotlin/org/opendc/common/utils/JavalinRunner.kt deleted file mode 100644 index 23baac27..00000000 --- a/opendc-common/src/main/kotlin/org/opendc/common/utils/JavalinRunner.kt +++ /dev/null @@ -1,33 +0,0 @@ -package org.opendc.common.utils - -import io.javalin.Javalin -import org.opendc.common.annotations.Endpoint -import org.opendc.common.api.AssetsController - -/** - * Represents the digital twin monitoring server. - * For endpoint documentation see `AssetsController`. - * @author Mateusz Kwiatkowski - * @see <a href=https://javalin.io/documentation>https://javalin.io/documentation</a> - * @see org.opendc.common.api.AssetsController - */ - -@OptIn(Endpoint::class) -public class JavalinRunner { - private val assetsController : AssetsController = AssetsController() - - init { - val app = Javalin.create().start() - app.get("/assets", assetsController.getAssets()) - - app.get("/assets/{id}", assetsController.getAssetsId()) - - app.post("/assets", assetsController.postAsset()) - - app.put("/assets/{id}", assetsController.putAssetId()) - - app.delete("/assets/{id}", assetsController.deleteAssetId()) - - app.delete("/assets", assetsController.deleteAsset()) - } -}
\ No newline at end of file diff --git a/opendc-common/src/main/kotlin/org/opendc/common/utils/PostgresqlDB.kt b/opendc-common/src/main/kotlin/org/opendc/common/utils/PostgresqlDB.kt deleted file mode 100644 index 35d03feb..00000000 --- a/opendc-common/src/main/kotlin/org/opendc/common/utils/PostgresqlDB.kt +++ /dev/null @@ -1,52 +0,0 @@ -package org.opendc.common.utils - -import com.fasterxml.jackson.dataformat.toml.TomlMapper -import java.sql.Connection -import java.sql.DriverManager -import java.sql.SQLException -import java.util.Properties -/** - * Represents the Postgresql database. - * On setup cleans the entire database. - * - * @author Mateusz Kwiatkowski - * - * @see <a href=https://docs.oracle.com/en/java/javase/21/docs/api/java.sql/java/sql/DriverManager.html> - * https://docs.oracle.com/en/java/javase/21/docs/api/java.sql/java/sql/DriverManager.html</a> - */ -@Suppress("DEPRECATION") -public class PostgresqlDB { - private var properties = Properties() - private var connection : Connection? = null - - init { - try { - properties = TomlMapper().readerFor(Properties().javaClass) - .readValue(PostgresqlDB::class.java.getResource("/database.toml")) - connection = DriverManager.getConnection( - properties.getProperty("address").asJdbc(properties.getProperty("database")), - properties.getProperty("user"), - properties.getProperty("password")) - clear() - } catch (e: SQLException) { - print("${e.message}") - } - } - public fun clear(){ - val DELETE_ALL_TABLES = """ - DROP SCHEMA public CASCADE; - CREATE SCHEMA public; - """.trimIndent() - try { - val st = connection?.createStatement() - st?.executeQuery(DELETE_ALL_TABLES) - } catch (e: SQLException){ - println("${e.message}") - } - } - - private fun String.asJdbc(database : String) : String { - return "jdbc:postgresql://$this/$database" - } - -}
\ No newline at end of file diff --git a/opendc-common/src/main/kotlin/org/opendc/common/utils/Redis.kt b/opendc-common/src/main/kotlin/org/opendc/common/utils/Redis.kt deleted file mode 100644 index b659d40a..00000000 --- a/opendc-common/src/main/kotlin/org/opendc/common/utils/Redis.kt +++ /dev/null @@ -1,43 +0,0 @@ -package org.opendc.common.utils - -import com.fasterxml.jackson.dataformat.toml.TomlMapper -import redis.clients.jedis.RedisClient -import redis.clients.jedis.StreamEntryID -import redis.clients.jedis.params.XReadParams -import java.util.Properties - -/** - * This class represents the Redis server instance. - * @author Mateusz Kwiatkowski - * @see <a href=https://redis.io/docs/latest/>https://redis.io/docs/latest/</a> - * - * @see <a href=https://redis.io/docs/latest/develop/data-types/streams/>https://redis.io/docs/latest/develop/data-types/streams/</a> - */ - -@Suppress("DEPRECATION") -public class Redis { - private var properties : Properties - - init { - properties = TomlMapper().readerFor(Properties().javaClass) - .readValue(Kafka::class.java.getResource("/producer.toml")) - } - - public fun run() { - val jedis : RedisClient = RedisClient.create("redis://localhost:6379") - - val res5 = jedis.xread( - XReadParams.xReadParams().block(300).count(100), - object : HashMap<String?, StreamEntryID?>() { - init { - put("${properties.getProperty("table")}", StreamEntryID()) - } - }) - - // in Redis you can subscribe to updates to a stream. - // you should base your application off this. - // you can listen for new items with XREAD - println(res5) - jedis.close() - } -}
\ No newline at end of file diff --git a/opendc-common/src/main/resources/schema.proto b/opendc-common/src/main/resources/schema.proto index d0aa18d5..ea515a39 100644 --- a/opendc-common/src/main/resources/schema.proto +++ b/opendc-common/src/main/resources/schema.proto @@ -11,4 +11,6 @@ message ProtoExport { int32 tasksactive = 3; double cpuutilization = 4; double energyusage = 5; -}
\ No newline at end of file + double uptime = 6; + double downtime = 7; +} diff --git a/opendc-common/src/main/resources/subscriber.toml b/opendc-common/src/main/resources/subscriber.toml new file mode 100644 index 00000000..75dece02 --- /dev/null +++ b/opendc-common/src/main/resources/subscriber.toml @@ -0,0 +1,3 @@ +"table" = "postgres_topic" +"server" = "localhost" +"port" = "6379" diff --git a/opendc-compute/opendc-compute-failure/src/main/kotlin/org/opendc/compute/failure/models/TraceBasedFailureModel.kt b/opendc-compute/opendc-compute-failure/src/main/kotlin/org/opendc/compute/failure/models/TraceBasedFailureModel.kt index 3bd253da..6cb02bb8 100644 --- a/opendc-compute/opendc-compute-failure/src/main/kotlin/org/opendc/compute/failure/models/TraceBasedFailureModel.kt +++ b/opendc-compute/opendc-compute-failure/src/main/kotlin/org/opendc/compute/failure/models/TraceBasedFailureModel.kt @@ -98,6 +98,10 @@ public class TraceBasedFailureModel( pathToFile: String, startPoint: Double, ): List<Failure> { + /* + I assume this returns a file descriptor. + @Mateusz Kwiatkowski + */ val trace = Trace.open(File(pathToFile), "failure") val reader = checkNotNull(trace.getTable(TABLE_FAILURES)).newReader() diff --git a/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/scheduler/ComputeSchedulers.kt b/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/scheduler/ComputeSchedulers.kt index fc40fac0..240d21d5 100644 --- a/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/scheduler/ComputeSchedulers.kt +++ b/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/scheduler/ComputeSchedulers.kt @@ -39,6 +39,7 @@ import java.util.SplittableRandom import java.util.random.RandomGenerator public enum class ComputeSchedulerEnum { + Smart, Mem, MemInv, CoreMem, @@ -81,6 +82,9 @@ public fun createPrefabComputeScheduler( /** * Create a [ComputeScheduler] for the experiment. + * I have changed this to use the new SmartScheduler. + * + * @author Mateusz Kwiatkowski */ public fun createPrefabComputeScheduler( name: ComputeSchedulerEnum, @@ -92,6 +96,8 @@ public fun createPrefabComputeScheduler( val ramAllocationRatio = 1.0 val gpuAllocationRatio = 1.0 return when (name) { + ComputeSchedulerEnum.Smart -> + SmartScheduler() ComputeSchedulerEnum.Mem -> FilterScheduler( filters = listOf(VCpuFilter(cpuAllocationRatio), RamFilter(ramAllocationRatio)), diff --git a/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/scheduler/SmartScheduler.kt b/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/scheduler/SmartScheduler.kt index baadd806..0d1cc408 100644 --- a/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/scheduler/SmartScheduler.kt +++ b/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/scheduler/SmartScheduler.kt @@ -1,72 +1,165 @@ -/* - * Copyright (c) 2021 AtLarge Research - * - * Permission is hereby granted, free of charge, to any person obtaining a copy - * of this software and associated documentation files (the "Software"), to deal - * in the Software without restriction, including without limitation the rights - * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell - * copies of the Software, and to permit persons to whom the Software is - * furnished to do so, subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE - * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER - * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, - * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE - * SOFTWARE. - */ - package org.opendc.compute.simulator.scheduler + import org.opendc.common.utils.HTTPClient +import org.opendc.compute.simulator.scheduler.filters.HostFilter import org.opendc.compute.simulator.service.HostView import org.opendc.compute.simulator.service.ServiceTask +import java.util.SplittableRandom +import java.util.random.RandomGenerator +/** + * A scheduler that is able to cooperate with a digital twin. + * Currently, this is hardcoded. + * Important: the random seed can aid you in finding good standard deviation measure. + * The scheduling significantly affects downtime. + * + * @author Mateusz Kwiatkowski + */ public class SmartScheduler : ComputeScheduler { + private val maxTimesSkipped: Int = 7 + private val random: RandomGenerator = SplittableRandom(128) + // We will use a priority queue. + private val hostsQueue = List(100, { mutableListOf<HostView>() }) + private var minAvailableHost = 0 + private var numHosts = 0 private val client = HTTPClient.getInstance() - // the point is that a smart scheduler listens for suggestions from the - // digital twin - // and here is where you change your actions based on the result from the DT - // predictive analytics is going to be much easier to do. - // you will completely overcome the overhead of having to tap-into - // the digital twin mid-through the simulation/in between two hosts being scheduled - // i.e., the normal simulation will NOT have to wait. - // predictive analytics will overcome the problem of the scheduling time overhead override fun addHost(host: HostView) { - TODO("Not yet implemented") + val zeroQueue = hostsQueue[0] + zeroQueue.add(host) + host.priorityIndex = 0; + host.listIndex = zeroQueue.size - 1 + numHosts++ + minAvailableHost = 0 } override fun removeHost(host: HostView) { - TODO("Not yet implemented") + val priorityIdx = host.priorityIndex + val listIdx = host.listIndex + val chosenList = hostsQueue[priorityIdx] + + if (chosenList.size == 1) { + chosenList.removeLast() + if (listIdx == minAvailableHost) { + for (i in minAvailableHost + 1..hostsQueue.lastIndex) { + if (hostsQueue[i].size > 0) { + minAvailableHost = i + break + } + } + } + } else { + val lastItem = chosenList.removeLast() + chosenList[listIdx] = lastItem + lastItem.listIndex = listIdx + } + numHosts-- } override fun updateHost(host: HostView) { - TODO("Not yet implemented") + // No-op + } override fun setHostEmpty(hostView: HostView) { - TODO("Not yet implemented") + // No-op } override fun select(iter: MutableIterator<SchedulingRequest>): SchedulingResult { - TODO("Not yet implemented") - // Here be the API calls using HTTP between the other OpenDC - // You need to specify how much time do you have to make the prediction between receiving a time and putting onto a host - return SchedulingResult(SchedulingResultType.EMPTY) + client?.checkForInsights() + + var unluckyHost = listOf("") + + if (numHosts == 0) { + return SchedulingResult(SchedulingResultType.FAILURE) + } + + val maxIters = 10000 + var numIters = 0 + + var chosenList: MutableList<HostView>? = null + var chosenHost: HostView? = null + + var result: SchedulingResult? = null + taskloop@ for (req in iter) { + if (req.isCancelled) { + iter.remove() + continue + } + + numIters++ + if (numIters > maxIters) { + return SchedulingResult(SchedulingResultType.EMPTY) + } + + for (chosenListIndex in minAvailableHost until hostsQueue.size) { + chosenList = hostsQueue[chosenListIndex] + for (host in chosenList) { + if(unluckyHost.isNotEmpty()){ + if(host.host.getName() in unluckyHost){ + continue + } + } + // Here be filtering, but for now we remove it completely. + iter.remove() + chosenHost = host + result = SchedulingResult(SchedulingResultType.SUCCESS, host, req) + break@taskloop + } + } + req.timesSkipped++ + } + + if (result == null) return SchedulingResult(SchedulingResultType.EMPTY) // No tasks to schedule that fit + + // Bookkeeping to maintain the calendar priority queue + if (chosenList!!.size == 1) { + chosenList.removeLast() + minAvailableHost++ + } else { + val listIdx = chosenHost!!.listIndex + // Not using removeLast here as it would cause problems during swapping + // if chosenHost is lastItem + val lastItem = chosenList.last() + chosenList[listIdx] = lastItem + lastItem.listIndex = listIdx + chosenList.removeLast() + } + + val nextList = hostsQueue[chosenHost!!.priorityIndex + 1] + nextList.add(chosenHost) + chosenHost.priorityIndex++ + chosenHost.listIndex = nextList.size - 1 + + return result } - // Benefits of a digital twin: during operations you make sure what is happening in the real world. - // The use-case is making split-second automated decisions before operators can make them. - // Make a strong case for making a Digital Twin. override fun removeTask( task: ServiceTask, - host: HostView?, + host: HostView? ) { - TODO("Not yet implemented") + if (host == null) return + + val priorityIdx = host.priorityIndex + val listIdx = host.listIndex + val chosenList = hostsQueue[priorityIdx] + val nextList = hostsQueue[priorityIdx - 1] + + if (chosenList.size == 1) { + chosenList.removeLast() + } else { + val lastItem = chosenList.last() + chosenList[listIdx] = lastItem + lastItem.listIndex = listIdx + chosenList.removeLast() + } + + nextList.add(host) + host.priorityIndex-- + host.listIndex = nextList.size - 1 + if (priorityIdx == minAvailableHost) { + minAvailableHost-- + } } } diff --git a/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/telemetry/KafkaComputeMonitor.kt b/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/telemetry/KafkaComputeMonitor.kt index c8368af2..d158f782 100644 --- a/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/telemetry/KafkaComputeMonitor.kt +++ b/opendc-compute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/telemetry/KafkaComputeMonitor.kt @@ -47,9 +47,10 @@ public class KafkaComputeMonitor : ComputeMonitor { .setTasksactive(reader.tasksActive) .setCpuutilization(reader.cpuUtilization) .setEnergyusage(reader.energyUsage) + .setUptime(reader.uptime.toDouble()) + .setDowntime(reader.downtime.toDouble()) .build() this.send(packet) - } catch (e: Exception) { println("${e.message}") } diff --git a/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/experiment/specs/ExperimentSpec.kt b/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/experiment/specs/ExperimentSpec.kt index 71ab4002..e3497cc4 100644 --- a/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/experiment/specs/ExperimentSpec.kt +++ b/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/experiment/specs/ExperimentSpec.kt @@ -40,6 +40,11 @@ import java.util.UUID * @property initialSeed * @property runs * be included in the output files. + * + * The allocation policy is hardcoded here regardless of the input in experiemnt.json. + * I did not design this, it was like that before. + * + * @author Mateusz Kwiatkowski */ @Serializable @@ -50,7 +55,7 @@ public data class ExperimentSpec( val runs: Int = 1, val topologies: Set<ScenarioTopologySpec>, val workloads: Set<WorkloadSpec>, - val allocationPolicies: Set<AllocationPolicySpec> = setOf(PrefabAllocationPolicySpec(ComputeSchedulerEnum.Mem)), + val allocationPolicies: Set<AllocationPolicySpec> = setOf(PrefabAllocationPolicySpec(ComputeSchedulerEnum.Smart)), val failureModels: Set<FailureModelSpec?> = setOf(null), val maxNumFailures: Set<Int> = setOf(10), val checkpointModels: Set<CheckpointModelSpec?> = setOf(null), diff --git a/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/experiment/specs/allocation/AllocationPolicySpec.kt b/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/experiment/specs/allocation/AllocationPolicySpec.kt index afe15032..3972b53f 100644 --- a/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/experiment/specs/allocation/AllocationPolicySpec.kt +++ b/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/experiment/specs/allocation/AllocationPolicySpec.kt @@ -41,10 +41,14 @@ import kotlin.coroutines.CoroutineContext @Serializable public sealed interface AllocationPolicySpec +/* This has to be changed in order to enforce the scheduler policy to Smart. + * + * @author: Mateusz Kwiatkowski + */ @Serializable @SerialName("prefab") public data class PrefabAllocationPolicySpec( - val policyName: ComputeSchedulerEnum = ComputeSchedulerEnum.Mem, + val policyName: ComputeSchedulerEnum = ComputeSchedulerEnum.Smart, ) : AllocationPolicySpec { public val name: String = policyName.toString() } diff --git a/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/runner/ExperimentCli.kt b/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/runner/ExperimentCli.kt index 1fe597ea..0e190df0 100644 --- a/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/runner/ExperimentCli.kt +++ b/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/runner/ExperimentCli.kt @@ -37,8 +37,7 @@ import java.io.IOException * Main entrypoint of the application. */ public fun main(args: Array<String>) { - if(args.size == 2) ExperimentCommand().main(args) - else ExperimentListener().main(args) + ExperimentCommand().main(args) } internal class ExperimentCommand : CliktCommand(name = "experiment") { @@ -49,24 +48,10 @@ internal class ExperimentCommand : CliktCommand(name = "experiment") { override fun run() { try { val experiment = getExperiment(experimentPath) - HTTPClient.getInstance()?.sendExperiment(experimentPath) + //HTTPClient.getInstance()?.sendExperiment(experimentPath) runExperiment(experiment) } catch (e: IOException) { println("${e.message}") } } } -/** - * Entry point to the digital twin. - * - * @author Mateusz Kwiatkowski - */ -internal class ExperimentListener: CliktCommand(name = "listener") { - override fun run() { - try { - runListener() - } catch (e: IOException) { - println("${e.message}") - } - } -} diff --git a/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/runner/ExperimentListener.kt b/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/runner/ExperimentListener.kt deleted file mode 100644 index de759c0f..00000000 --- a/opendc-experiments/opendc-experiments-base/src/main/kotlin/org/opendc/experiments/base/runner/ExperimentListener.kt +++ /dev/null @@ -1,19 +0,0 @@ -package org.opendc.experiments.base.runner - -import org.opendc.common.utils.JavalinRunner -import org.opendc.common.utils.PostgresqlDB -import org.opendc.common.utils.Redis - -/** - * Established a connection with PostgreSQL. - * Creates a Javalin HTTP server and listens for requests. - * - * @author Mateusz Kwiatkowski - * - * @see <a href=https://javalin.io/documentation>https://javalin.io/documentation</a> - */ -public fun runListener() { - PostgresqlDB() - Redis().run() - JavalinRunner() -}
\ No newline at end of file diff --git a/output/greenifier-demo-scaling/raw-output/0/seed=0/host.parquet b/output/greenifier-demo-scaling/raw-output/0/seed=0/host.parquet Binary files differdeleted file mode 100644 index 3afc94b3..00000000 --- a/output/greenifier-demo-scaling/raw-output/0/seed=0/host.parquet +++ /dev/null diff --git a/output/greenifier-demo-scaling/raw-output/0/seed=0/powerSource.parquet b/output/greenifier-demo-scaling/raw-output/0/seed=0/powerSource.parquet Binary files differdeleted file mode 100644 index f7d137b2..00000000 --- a/output/greenifier-demo-scaling/raw-output/0/seed=0/powerSource.parquet +++ /dev/null diff --git a/output/greenifier-demo-scaling/raw-output/0/seed=0/service.parquet b/output/greenifier-demo-scaling/raw-output/0/seed=0/service.parquet Binary files differdeleted file mode 100644 index 75c94c5d..00000000 --- a/output/greenifier-demo-scaling/raw-output/0/seed=0/service.parquet +++ /dev/null diff --git a/output/greenifier-demo-scaling/raw-output/0/seed=0/task.parquet b/output/greenifier-demo-scaling/raw-output/0/seed=0/task.parquet Binary files differdeleted file mode 100644 index b39c11ad..00000000 --- a/output/greenifier-demo-scaling/raw-output/0/seed=0/task.parquet +++ /dev/null diff --git a/output/greenifier-demo-scaling/trackr.json b/output/greenifier-demo-scaling/trackr.json deleted file mode 100644 index a595beff..00000000 --- a/output/greenifier-demo-scaling/trackr.json +++ /dev/null @@ -1,20 +0,0 @@ -[{ - "name": "greenifier-demo-scaling", - "topology": { - "pathToFile": "resources/topologies/surf.json" - }, - "workload": { - "pathToFile": "resources/workloads/surf_month", - "type": "ComputeWorkload" - }, - "exportModel": { - "exportInterval": 3600, - "filesToExportDict": { - "host": true, - "task": true, - "powerSource": true, - "battery": true, - "service": true - } - } -}]
\ No newline at end of file diff --git a/python_scripts/LICENSE b/python_scripts/LICENSE new file mode 100644 index 00000000..f288702d --- /dev/null +++ b/python_scripts/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see <https://www.gnu.org/licenses/>. + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + <program> Copyright (C) <year> <name of author> + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +<https://www.gnu.org/licenses/>. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +<https://www.gnu.org/licenses/why-not-lgpl.html>. diff --git a/python_scripts/README.md b/python_scripts/README.md new file mode 100644 index 00000000..e69de29b --- /dev/null +++ b/python_scripts/README.md diff --git a/python_scripts/output.txt b/python_scripts/output.txt new file mode 100644 index 00000000..8f72c822 --- /dev/null +++ b/python_scripts/output.txt @@ -0,0 +1,386 @@ +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +3600000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.20000002685183743, 0.20000002685183743, 0.20000002685183743, 0.3999999194444877 +61200000, 0.24999996643522573, 0.24999996643522573, 0.24999996643522573, 0.25000010069432277 +61200000, 0.24999999999997746, 0.24999999999997746, 0.24999999999997746, 0.2500000000000676 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.20000002685183743, 0.20000002685183743, 0.20000002685183743, 0.3999999194444877 +61200000, 0.2481001725847513, 0.2481001725847513, 0.2556993490069075, 0.2481003058235898 +61200000, 0.22328502435637435, 0.22328502435637435, 0.32748490629710375, 0.22594504499014756 +61200000, 0.20000001250491378, 0.20000001250491378, 0.20000006846407958, 0.3999999065260929 +61200000, 0.24999996643521613, 0.24999996643521613, 0.24999996643525368, 0.2500001006943141 +61200000, 0.24999999999997746, 0.24999999999997746, 0.24999999999997746, 0.2500000000000676 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.2613811830068312, 0.21585645097950631, 0.2613811830068312, 0.2613811830068312 +61200000, 0.22703899281086978, 0.3188830215673906, 0.22703899281086978, 0.22703899281086978 +61200000, 0.24999998712191063, 0.2500000386342681, 0.24999998712191063, 0.24999998712191063 +61200000, 0.24999999999999137, 0.2500000000000259, 0.24999999999999137, 0.24999999999999137 +61200000, 0.25, 0.25, 0.25, 0.25 +61200000, 0.2534023077724438, 0.23979307668266847, 0.2534023077724438, 0.2534023077724438 +61200000, 0.2500000022839551, 0.24999999314813479, 0.2500000022839551, 0.2500000022839551 +925200000, 0.248295227273145, 0.25511431818056485, 0.248295227273145, 0.248295227273145 +925200000, 0.2652861490475654, 0.2652861525322103, 0.20414154937265877, 0.2652861490475654 +925200000, 0.23851770261385766, 0.23851770261385968, 0.2279750806061642, 0.29498951416611846 +925200000, 0.23526092225334203, 0.22432706208201644, 0.3160849195663033, 0.2243270960983383 +925200000, 0.23516372305784417, 0.254796073760443, 0.25498286003864346, 0.25505734314306944 +925200000, 0.22390193761243352, 0.3282941632908147, 0.2239019495259895, 0.2239019495707623 +925200000, 0.24345441405726143, 0.2368748994780613, 0.23736731206266468, 0.28230337440201264 +925200000, 0.2753233668099399, 0.27550644720981293, 0.19291811469299797, 0.2562520712872492 +925200000, 0.23087781318798842, 0.2724950358296014, 0.22413212570353935, 0.2724950252788707 +925200000, 0.2208090941031873, 0.22086223066633617, 0.32359275573762925, 0.23473591949284728 +925200000, 0.25143330031103295, 0.2511003496751606, 0.2512661797677872, 0.2462001702460194 +925200000, 0.2741447358424723, 0.24165160823968082, 0.24201697750129028, 0.24218667841655656 +925200000, 0.2400214383697987, 0.2602326016851776, 0.23951335800403126, 0.26023260194099246 +925200000, 0.24990655029452266, 0.25003115306329543, 0.25003114357888634, 0.25003115306329554 +925200000, 0.2966703805581005, 0.20246461663215998, 0.20419462218390777, 0.29667038062583184 +925200000, 0.25082897888538336, 0.25098745202112643, 0.25055169922886794, 0.24763186986462227 +925200000, 0.2262221866547984, 0.22328830739780292, 0.32731610200340266, 0.2231734039439959 +925200000, 0.2479059042268697, 0.25605917736816214, 0.24797796444557635, 0.24805695395939187 +925200000, 0.2776865814029007, 0.19149081742882548, 0.25367183504450114, 0.2771507661237727 +925200000, 0.2676446771207583, 0.2676113667467238, 0.26759841019704766, 0.19714554593547026 +925200000, 0.3274401475692214, 0.22522437619457183, 0.22368948277342993, 0.22364599346277672 +925200000, 0.24699818261415657, 0.25119056469250456, 0.2508426397018037, 0.25096861299153517 +925200000, 0.2502820883367412, 0.2497179104500543, 0.24971791021700884, 0.2502820909961957 +925200000, 0.20918062818627947, 0.2966028286289835, 0.28483954673097867, 0.2093769964537583 +925200000, 0.22357058392664114, 0.2233767589347581, 0.22543327716258987, 0.32761937997601087 +925200000, 0.253542522944931, 0.2541331208882812, 0.24600141376949777, 0.24632294239729008 +925200000, 0.2943635797309663, 0.29447886217097546, 0.20078970097695625, 0.21036785712110206 +925200000, 0.27358404036446904, 0.2735840404222116, 0.18833974181964186, 0.2644921773936775 +925200000, 0.2153769342333478, 0.21542941113381345, 0.25330730426859505, 0.3158863503642438 +925200000, 0.2715029045503615, 0.2715029045769939, 0.1947033497001468, 0.2622908411724977 +925200000, 0.23052487942884817, 0.30778158203175227, 0.2308467486722446, 0.2308467898671549 +925200000, 0.2248569475605296, 0.23145539642840113, 0.31883070825617593, 0.22485694775489337 +925200000, 0.22617525003274397, 0.3061872012656058, 0.2415013943515515, 0.2261361543500987 +925200000, 0.2081051724931678, 0.28004824290013974, 0.23210620261232567, 0.27974038199436674 +925200000, 0.27159589013242746, 0.2714685511866513, 0.2715959020666841, 0.1853396566142372 +925200000, 0.22378738738946333, 0.22891879728194123, 0.317314139640781, 0.22997967568781463 +925200000, 0.2022089999614515, 0.253666840268954, 0.27215668864514564, 0.27196747112444886 +925200000, 0.25235959775306477, 0.23402810938261073, 0.25527716684789714, 0.25833512601642733 +925200000, 0.2818125443966189, 0.3029875101785277, 0.20658227685295275, 0.20861766857190073 +925200000, 0.25820775361747295, 0.19402936389472486, 0.2786099928344677, 0.26915288965333456 +925200000, 0.24833296736727004, 0.2480497461251616, 0.24804980246086392, 0.2555674840467043 +925200000, 0.3041331984487847, 0.20828944869829386, 0.28017011792302254, 0.20740723492989874 +925200000, 0.2749466864726847, 0.27494663716820905, 0.25267861247168527, 0.19742806388742093 +925200000, 0.2902291388914889, 0.20478455089214487, 0.300207640430469, 0.20477866978589723 +925200000, 0.24999257471187958, 0.2502372037522278, 0.25023724746895615, 0.24953297406693647 +925200000, 0.25008186812052413, 0.2500319073825283, 0.25002826372428666, 0.249857960772661 +925200000, 0.24918337393767342, 0.22635094892806001, 0.3000307740107135, 0.2244349031235532 +925200000, 0.2274046450538449, 0.23550209244600956, 0.2275187755897485, 0.3095744869103971 +925200000, 0.24999998484645422, 0.2499999902769969, 0.24999998492299572, 0.2500000399535532 +925200000, 0.22464158724479702, 0.32618248530191185, 0.2234553438025491, 0.22572058365074202 +925200000, 0.2874174869865661, 0.21258254067665797, 0.2125824847629187, 0.28741748757385716 +925200000, 0.22395586531607453, 0.2240758462417732, 0.22402088435931367, 0.3279474040828385 +1634400000, 0.250907851168518, 0.25358159923857465, 0.24176512962053676, 0.25374541997237055 +1634400000, 0.2504201504478932, 0.24966861543007224, 0.24977345094850822, 0.2501377831735264 +1634400000, 0.23349646836066382, 0.22477048596650778, 0.31749467429594935, 0.224238371376879 +1634400000, 0.26074225440550314, 0.2605355033997384, 0.19723414648990573, 0.2814880957048527 +1634400000, 0.2504604252252252, 0.2504280963125586, 0.24963613750538968, 0.24947534095682636 +1634400000, 0.23320484953468157, 0.23119244433107028, 0.3065705584021287, 0.2290321477321194 +1634400000, 0.2480864629616976, 0.24756850982913825, 0.24751974316297057, 0.25682528404619354 +1634400000, 0.23100017938714854, 0.23100061054635, 0.22508721373357418, 0.3129119963329273 +1634400000, 0.25246939778201083, 0.2524593985703774, 0.24714539347015632, 0.2479258101774554 +1634400000, 0.22898534709439422, 0.25397010178124824, 0.2881115087726885, 0.22893304235166906 +1634400000, 0.24201126289289387, 0.28404029123477353, 0.23802012172628761, 0.235928324146045 +1634400000, 0.23607512810792494, 0.243270960589367, 0.24094475770347085, 0.2797091535992372 +1634400000, 0.27253719455853176, 0.27274574897566417, 0.24753601344090612, 0.207181043024898 +1634400000, 0.23134097635720857, 0.23137030361772856, 0.3064080472192632, 0.2308806728057996 +1634400000, 0.22879300690386636, 0.26728122985982744, 0.22989474925100137, 0.27403101398530494 +1634400000, 0.3023156513943947, 0.2325049114363377, 0.23250488856167703, 0.2326745486075907 +1634400000, 0.22450376237480793, 0.22375523378174214, 0.3274058792965274, 0.2243351245469225 +1634400000, 0.22790391828795897, 0.22869403312277614, 0.3089465326002865, 0.23445551598897835 +1634400000, 0.23727488754531895, 0.23216090903648356, 0.23729004170733645, 0.2932741617108611 +1634400000, 0.2507594223910994, 0.2510601480271692, 0.25100984146230826, 0.24717058811942313 +1634400000, 0.24510753104537875, 0.2402440583497396, 0.2476968519263108, 0.26695155867857095 +1634400000, 0.24784042540165943, 0.24782637119174958, 0.24881596620294785, 0.2555172372036433 +1634400000, 0.2574672203269544, 0.257467220319176, 0.226865403327805, 0.2582001560260646 +1634400000, 0.2421936229275743, 0.24342913761114238, 0.2785769449911936, 0.23580029447008963 +1634400000, 0.25170017597572564, 0.24720056773500432, 0.24625319987755978, 0.2548460564117102 +1634400000, 0.3011745182905593, 0.23278094620018278, 0.23288560566010733, 0.23315892984915051 +1634400000, 0.25948928663811527, 0.2851799636167783, 0.22358394985091531, 0.23174679989419114 +1634400000, 0.22054798038338438, 0.2618980876055311, 0.2611157358890904, 0.25643819612199414 +1634400000, 0.268415555889427, 0.264129989975962, 0.27083330071060935, 0.1966211534240017 +1634400000, 0.2174332087130431, 0.2556070827277623, 0.3095280795469605, 0.2174316290122343 +1634400000, 0.2320301256339194, 0.2408117628252029, 0.29240936746174606, 0.2347487440791317 +1634400000, 0.24748313835475017, 0.2511003413242548, 0.25008340075308755, 0.2513331195679074 +1634400000, 0.30988740329676673, 0.2305372299337428, 0.2232709286045099, 0.23630443816498062 +1634400000, 0.2478611344376212, 0.248006615965382, 0.2511046702894231, 0.25302757930757375 +1634400000, 0.2226354515015137, 0.26061253488188835, 0.29675054323374916, 0.22000147038284876 +1634400000, 0.2957008045890112, 0.206784589154521, 0.29570084471275465, 0.20181376154371308 +1634400000, 0.24676620204496896, 0.2512897014061452, 0.24878290854543894, 0.253161188003447 +1634400000, 0.27229987268444183, 0.1949386444752037, 0.277301075654122, 0.2554604071862324 +1634400000, 0.26002512731315947, 0.2560498036885042, 0.24204585563947112, 0.2418792133588653 +1634400000, 0.24647548584079176, 0.2536569327628858, 0.2463463851288097, 0.2535211962675127 +1634400000, 0.2366061726461301, 0.2366746090741287, 0.29276758670375314, 0.2339516315759881 +1634400000, 0.2410835334355473, 0.24974942719972498, 0.26803182189496244, 0.24113521746976524 +1634400000, 0.2462180384750517, 0.24714899966333273, 0.26041492335310007, 0.24621803850851556 +1634400000, 0.22777470441459044, 0.3089532608179328, 0.23110013849101063, 0.23217189627646612 +1634400000, 0.20288965819534993, 0.29715282653537983, 0.20280473050518272, 0.29715278476408763 +1634400000, 0.23475715300530048, 0.228866481723737, 0.30756009952907953, 0.22881626574188302 +1634400000, 0.24559174646698542, 0.25172633306595327, 0.25187515290453616, 0.25080676756252523 +1634400000, 0.28686680358892164, 0.2854135280219375, 0.21451972828248217, 0.2131999401066588 +1634400000, 0.2755625599785009, 0.23871596818544857, 0.2511893726769351, 0.23453209915911546 +1634400000, 0.2538422872135996, 0.25600733475537785, 0.2346571065329338, 0.2554932714980888 +1634400000, 0.2521256015711214, 0.2519022375576867, 0.24766259379973307, 0.2483095670714588 +1634400000, 0.2444459905119514, 0.24932527819720934, 0.2484766186922241, 0.25775211259861514 +1634400000, 0.2489560413777486, 0.25402192062191853, 0.248839284723309, 0.24818275327702394 +1634400000, 0.2685527556238259, 0.2384882981538453, 0.26867839241123553, 0.22428055381109324 +1634400000, 0.25171759222593515, 0.25429218175704343, 0.24662823775294582, 0.24736198826407568 +1634400000, 0.30345095703372604, 0.20984846977615557, 0.20759698299330379, 0.27910359019681463 +1634400000, 0.2511239059780179, 0.25142815026694354, 0.246648798082867, 0.2507991456721716 +1634400000, 0.24250981674972122, 0.23632750684313197, 0.23981311355001678, 0.28134956285713014 +1634400000, 0.22118177123289126, 0.2910822851673943, 0.22611487511802858, 0.2616210684816859 +1634400000, 0.27012638973228137, 0.24282422534725998, 0.24358153488653242, 0.24346785003392626 +1634400000, 0.2667524594962123, 0.23887911413413085, 0.25746610545162324, 0.23690232091803354 +1634400000, 0.2893676976553488, 0.21393516911464042, 0.2201678526706805, 0.2765292805593303 +1634400000, 0.2970247153512438, 0.20259380990490589, 0.20335676637406727, 0.29702470836978306 +1634400000, 0.1901261499516298, 0.2693070198688925, 0.26928731261591315, 0.27127951756356455 +1634400000, 0.3087080735645056, 0.23023231706921335, 0.22994936649258704, 0.2311102428736939 +1634400000, 0.2505678206505557, 0.24475389757289967, 0.25835837381415416, 0.2463199079623906 +1634400000, 0.24638894700032116, 0.25431943081595976, 0.24633769560087002, 0.25295392658284915 +1634400000, 0.25076017692435193, 0.26861256200873784, 0.27030373446538786, 0.21032352660152237 +1634400000, 0.3245489403396347, 0.22798487366750916, 0.22151752929229443, 0.22594865670056172 +1634400000, 0.2653176045530706, 0.2671219899901153, 0.2257376236245442, 0.24182278183226988 +1634400000, 0.24849258230021481, 0.2485199317529215, 0.26004529639674734, 0.24294218955011632 +1634400000, 0.257580405562529, 0.2578044423772163, 0.2519408758194676, 0.2326742762407872 +1634400000, 0.29412273646881365, 0.23505134974955197, 0.2351535368621561, 0.2356723769194781 +1634400000, 0.23875673671761452, 0.2822484207009355, 0.2425572679701363, 0.23643757461131368 +1634400000, 0.2597454061043378, 0.21520136119242678, 0.21565220175023875, 0.30940103095299676 +1634400000, 0.2494304953339776, 0.2498773873303872, 0.2491780376434437, 0.2515140796921916 +1634400000, 0.2774960219924277, 0.24610724274966184, 0.2106798003112694, 0.2657169349466411 +1634400000, 0.2781771785042652, 0.213692780445197, 0.20804284747927237, 0.30008719357126545 +1634400000, 0.2522072458008963, 0.25195150331035177, 0.2450878757621473, 0.2507533751266047 +1634400000, 0.25286615037609605, 0.24816376422007014, 0.2458695185454608, 0.2531005668583732 +1634400000, 0.29695589818040896, 0.2965740153283443, 0.20313415795448334, 0.20333592853676316 +1634400000, 0.2504400258350717, 0.21312377165865978, 0.2688398671156724, 0.2675963353905962 +1634400000, 0.28893140084993596, 0.23742543685647066, 0.23664166827359873, 0.23700149401999476 +1634400000, 0.2545890119067801, 0.24700489093798278, 0.24792577447259942, 0.25048032268263765 +1634400000, 0.24691539750472002, 0.25175845072868464, 0.25292382596527274, 0.24840232580132268 +1634400000, 0.23788609083427886, 0.275268656654376, 0.24766042925683307, 0.23918482325451196 +1634400000, 0.28752511196453395, 0.27379001039401846, 0.21944938415347712, 0.21923549348797056 +1634400000, 0.24987104117501543, 0.2500533239653062, 0.24990540223011093, 0.2501702326295675 +1634400000, 0.2656901663413321, 0.26575700773359334, 0.2028140475548096, 0.2657387783702649 +1638000000, 0.25226247317787104, 0.25428365752398646, 0.2412168274421064, 0.2522370418560361 +1638000000, 0.24909233342698492, 0.2511498555752138, 0.2489288109206654, 0.2508290000771358 +1638000000, 0.22673201545888966, 0.2247816794181443, 0.3178373516597298, 0.23064895346323616 +1638000000, 0.2606365691961499, 0.2607022751902126, 0.19692369420008643, 0.28173746141355105 +1638000000, 0.2490871236471965, 0.25195024005131705, 0.24943993325549607, 0.24952270304599036 +1638000000, 0.23459106212133818, 0.22755143676985826, 0.3096472426336877, 0.22821025847511586 +1638000000, 0.2554911290276039, 0.25305781366603747, 0.25375972090753307, 0.2376913363988256 +1638000000, 0.24716733607554767, 0.23734992049667378, 0.2410858092797567, 0.2743969341480218 +1638000000, 0.24756092528571547, 0.25391332545380535, 0.2472364718775267, 0.25128927738295237 +1638000000, 0.22872880459178857, 0.22681641123515353, 0.2578822171878781, 0.2865725669851797 +1638000000, 0.25122040532510104, 0.24983866022072154, 0.2494958259122163, 0.24944510854196103 +1638000000, 0.23841837666742705, 0.24535168673717928, 0.23809162118185287, 0.27813831541354084 +1638000000, 0.2704165186905217, 0.2730351886383436, 0.2483557577057762, 0.20819253496535844 +1638000000, 0.23374796728550715, 0.2274151970329995, 0.3084518531932685, 0.23038498248822487 +1638000000, 0.3140943030188366, 0.22815290362923915, 0.2286260420506395, 0.22912675130128476 +1638000000, 0.2768179021312853, 0.24413685481864275, 0.24413685510005487, 0.23490838795001717 +1638000000, 0.22432053509530972, 0.22383461364642465, 0.32756473649989254, 0.224280114758373 +1638000000, 0.2765602299652568, 0.21575855049913437, 0.28811844579475443, 0.21956277374085437 +1638000000, 0.23989383041965814, 0.23524562174292818, 0.239758672678458, 0.2851018751589557 +1638000000, 0.2499699508523138, 0.2506283986585065, 0.25150582168137314, 0.24789582880780653 +1638000000, 0.2516188082594553, 0.2454785586798595, 0.24845317248916643, 0.25444946057151885 +1638000000, 0.24762514801179625, 0.24949052132379226, 0.2476254267168656, 0.2552589039475458 +1638000000, 0.2604131989983146, 0.26041320001966356, 0.2197526155797634, 0.2594209854022585 +1638000000, 0.2526386963372494, 0.25404191989828584, 0.24523597793313368, 0.24808340583133123 +1638000000, 0.24668528143336477, 0.24671030207509562, 0.25083135461362394, 0.25577306187791576 +1638000000, 0.29993319356104475, 0.23197459214912813, 0.23431656480961066, 0.2337756494802165 +1638000000, 0.23856170773446161, 0.2710189234232845, 0.23829328738025157, 0.25212608146200227 +1638000000, 0.2212955689820984, 0.2604407717153197, 0.2592184437429184, 0.2590452155596634 +1638000000, 0.2665324851001434, 0.2602517348965415, 0.2661323894666148, 0.20708339053670022 +1638000000, 0.22747720627232662, 0.22151623838684167, 0.3236269932527164, 0.22737956208811527 +1638000000, 0.24054319665899596, 0.24109486470889813, 0.2836457489964947, 0.23471618963561117 +1638000000, 0.2613364989290193, 0.26638346107017996, 0.2645399962439577, 0.20774004375684305 +1638000000, 0.2850314214297008, 0.23405411701207762, 0.24041334767857186, 0.24050111387964979 +1638000000, 0.24767689966887937, 0.24753188280442934, 0.25239251055221845, 0.25239870697447275 +1638000000, 0.2225648050292054, 0.22024527128194674, 0.2998916886193666, 0.2572982350694813 +1638000000, 0.29528001060041537, 0.207982128667447, 0.29528005240335686, 0.20145780832878082 +1638000000, 0.24792442063364004, 0.2538475491076962, 0.2480468815140638, 0.2501811487445999 +1638000000, 0.2719059420820126, 0.19464185544104676, 0.2770121076072846, 0.256440094869656 +1638000000, 0.2456683694885958, 0.2601088187016786, 0.2461635419666364, 0.24805926984308918 +1638000000, 0.2945206467957744, 0.23586244139041357, 0.23352762203262714, 0.2360892897811849 +1638000000, 0.23227094403624682, 0.23208030547951336, 0.30337091874290945, 0.23227783174133024 +1638000000, 0.24627369197447924, 0.25472413420177986, 0.24894816344004117, 0.25005401038369973 +1638000000, 0.24761094237537642, 0.25673191988578203, 0.24804619306115172, 0.2476109446776898 +1638000000, 0.22886926110479153, 0.30975781898393007, 0.22955773151127415, 0.2318151884000042 +1638000000, 0.20269229930263322, 0.29725221336961805, 0.20280331637536766, 0.2972521709523812 +1638000000, 0.23294984620941578, 0.23195117820149824, 0.3067423468127689, 0.228356628776317 +1638000000, 0.25424599976288487, 0.24844347773523037, 0.2481278449541457, 0.24918267754773918 +1638000000, 0.285526597792596, 0.2876614314341254, 0.2132964800086103, 0.2135154907646683 +1638000000, 0.20489848837980346, 0.2402283475637843, 0.29825360026178077, 0.25661956379463147 +1638000000, 0.24134832251097432, 0.24091649211803934, 0.27665798361488336, 0.24107720175610298 +1638000000, 0.24867885122100855, 0.24771003947721368, 0.25181491728778216, 0.2517961920139955 +1638000000, 0.24230030137412745, 0.2464243682239015, 0.24466763552554951, 0.2666076948764215 +1638000000, 0.2544509951518537, 0.2500145242541236, 0.24982486170690066, 0.245709618887122 +1638000000, 0.23080356919572687, 0.3097850060996307, 0.2303999495400373, 0.2290114751646052 +1638000000, 0.24706199523749714, 0.2547073253840212, 0.25077626149045823, 0.2474544178880234 +1638000000, 0.3008020326642203, 0.21174290099365586, 0.2087131418993208, 0.2787419244428029 +1638000000, 0.2502568628517579, 0.2506722135764086, 0.25042542021526193, 0.24864550335657154 +1638000000, 0.27384391705427363, 0.23732438168150913, 0.21342753688266475, 0.2754041643815525 +1638000000, 0.2362830098270334, 0.2748837847576475, 0.2341898395053138, 0.2546433659100053 +1638000000, 0.24894472546315907, 0.24847022011208733, 0.24846354117304248, 0.2541215132517112 +1638000000, 0.24837263337007154, 0.25542297657905083, 0.24829246105289388, 0.2479119289979837 +1638000000, 0.2963742225191926, 0.21280868062589883, 0.2155117218825912, 0.27530537497231733 +1638000000, 0.2963448666749364, 0.20486659479802208, 0.20244368328300485, 0.29634485524403664 +1638000000, 0.20584551285016833, 0.2628708619021365, 0.26282784511923324, 0.26845578012846194 +1638000000, 0.3121857936245771, 0.23073042500868765, 0.2283948770332051, 0.22868890433353006 +1638000000, 0.24190485757973387, 0.23756549680659333, 0.2816209502041291, 0.2389086954095438 +1638000000, 0.2640372335673972, 0.20222621793475806, 0.26408017405185624, 0.26965637444598844 +1638000000, 0.233637641032514, 0.23319248996085645, 0.23105473829308137, 0.3021151307135481 +1638000000, 0.3260263733676477, 0.22916372275126337, 0.222440049533716, 0.22236985434737294 +1638000000, 0.2914585473687355, 0.19898372621492913, 0.23786989140480921, 0.271687835011526 +1638000000, 0.26674845979443157, 0.2685987249177769, 0.20119878587845494, 0.26345402940933654 +1638000000, 0.23836650880756563, 0.2370661375371987, 0.22994461120857007, 0.29462274244666564 +1638000000, 0.29471757692834516, 0.23570932386483906, 0.23576727858979477, 0.23380582061702093 +1638000000, 0.253604014032116, 0.2468802527028909, 0.25317854706937093, 0.24633718619562203 +1638000000, 0.2251957718010205, 0.22478661399344027, 0.22703552629210302, 0.32298208791343624 +1638000000, 0.24018299228095222, 0.24052537770885998, 0.27962463477546634, 0.23966699523472135 +1638000000, 0.2981904081096261, 0.22795318141128798, 0.2234221943223894, 0.2504342161566965 +1638000000, 0.2797858041045891, 0.20749554065466025, 0.21019403379461732, 0.3025246214461334 +1638000000, 0.2525038473968698, 0.2507818306837656, 0.24601017159430774, 0.2507041503250569 +1638000000, 0.2528263182990038, 0.24875273417504085, 0.24566794154632726, 0.25275300597962813 +1638000000, 0.2959953742553183, 0.2955391865807024, 0.20425188072202083, 0.2042135584419585 +1638000000, 0.2245401832200916, 0.23055914410163247, 0.2554218845161234, 0.2894787881621526 +1638000000, 0.2594836044598042, 0.24625231789245608, 0.2464478962498606, 0.2478161813978792 +1638000000, 0.2533930738732058, 0.2491163857519254, 0.2488989999218517, 0.2485915404530171 +1638000000, 0.2523183597099347, 0.25063679398133176, 0.2505393339218262, 0.24650551238690743 +1638000000, 0.24408272284200894, 0.2654219457544646, 0.2460845635827802, 0.24441076782074633 +1638000000, 0.2645092342336359, 0.2505994003561056, 0.24230943333604057, 0.2425819320742179 +1638000000, 0.25139795490624184, 0.24903343409334336, 0.25054579592994697, 0.24902281507046792 +1638000000, 0.24086455405793258, 0.24086401135751848, 0.27753516683512003, 0.2407362677494289 +1645200000, 0.2499999938674131, 0.24999999386704877, 0.2500000184842431, 0.24999999378129503 +1645200000, 0.24407298096037566, 0.23783937957376922, 0.2378393795737831, 0.280248259892072 +1645200000, 0.25257261157880073, 0.25257260869631243, 0.2422821514182249, 0.25257262830666183 +1645200000, 0.2606644905507229, 0.23665949356744057, 0.23426162442306867, 0.2684143914587679 +1645200000, 0.25317991137049833, 0.2489400230019355, 0.2489400213990807, 0.24894004422848542 +1645200000, 0.26142474725277315, 0.26135481174879727, 0.19592485236476145, 0.2812955886336681 +1645200000, 0.2565706480978657, 0.24659011562828256, 0.2465900723045115, 0.2502491639693404 +1645200000, 0.2500000039305398, 0.24999999796021857, 0.24999999796019262, 0.25000000014904905 +1645200000, 0.24442108058945397, 0.2591882496394481, 0.24442108058945003, 0.2519695891816478 +1645200000, 0.2525358091347513, 0.24540564150125332, 0.2525358091347513, 0.24952274022924398 +1645200000, 0.25000000170227815, 0.24999999691582622, 0.25000000170227815, 0.2499999996796175 +1645200000, 0.2500000000000011, 0.24999999999999792, 0.2500000000000011, 0.24999999999999978 +1645200000, 0.2195202284176119, 0.21987569850441963, 0.32196319714689065, 0.2386408759310777 +1645200000, 0.2291778606289023, 0.23378802803411242, 0.2337880905239725, 0.30324602081301283 +1645200000, 0.2474857150932528, 0.2575428024349448, 0.24748571815695605, 0.24748576431484623 +1645200000, 0.24894374456168852, 0.2531687663149029, 0.24894374456169038, 0.2489437445617182 +1645200000, 0.24808428774891028, 0.24808429056341616, 0.2557471339387632, 0.24808428774891028 +1645200000, 0.2499999987139903, 0.2499999987139922, 0.2500000038580271, 0.2499999987139903 +1645200000, 0.22912889916346968, 0.22912889916346968, 0.3126133025095909, 0.22912889916346968 +1645200000, 0.27093059149567345, 0.26388332555717303, 0.2709306338467136, 0.19425544910043993 +1645200000, 0.26068344443917973, 0.2464388656574934, 0.2464388703209335, 0.24643881958239347 +1645200000, 0.22961507610331006, 0.22961506732069295, 0.22961506732069584, 0.3111547892553012 +1645200000, 0.21196250870131456, 0.28741486399103144, 0.21196250870130953, 0.2886601186063444 +1645200000, 0.3244259298409161, 0.22519137167687958, 0.2251913260523442, 0.2251913724298602 +1645200000, 0.26649318360077495, 0.26679646279701563, 0.19991389080519337, 0.266796462797016 +1645200000, 0.24638485625766574, 0.2555372540253307, 0.24638481220957179, 0.25169307750743186 +1645200000, 0.22053267136626054, 0.21398801246263438, 0.2769360129653854, 0.28854330320571975 +1645200000, 0.28734921890298826, 0.28790203595993286, 0.2123743692589291, 0.21237437587814986 +1645200000, 0.27160498737977035, 0.18518511242406377, 0.27160495009808133, 0.27160495009808466 +1645200000, 0.29185492561612125, 0.23604832161078812, 0.23604837638654527, 0.23604837638654527 +1645200000, 0.25225132218585555, 0.2522512886882168, 0.2522512886882497, 0.2432461004376779 +1645200000, 0.25000000151130325, 0.2500000015112807, 0.2500000015112807, 0.24999999546613527 +1645200000, 0.22212987378962767, 0.32526067387600144, 0.2219544531766775, 0.23065499915769333 +1645200000, 0.24999998299512036, 0.25000004592008995, 0.24999998288808795, 0.24999998819670177 +1645200000, 0.23269174959431177, 0.3019247512170616, 0.2326917495943117, 0.2326917495943149 +1645200000, 0.30146799036278293, 0.23284403206309362, 0.23284398878706175, 0.23284398878706175 +1645200000, 0.2974405709193367, 0.2969589537030946, 0.20280023768878427, 0.20280023768878427 +1645200000, 0.20033606557238817, 0.2665546754427873, 0.26655462949241227, 0.26655462949241227 +1645200000, 0.22558128904083866, 0.3274023878178377, 0.22346382567450865, 0.2235524974668151 +1645200000, 0.26974480561941755, 0.2545823433495208, 0.23783642548997802, 0.2378364255410837 +1645200000, 0.25027122546846164, 0.24918636589638168, 0.2502712043175783, 0.2502712043175784 +1645200000, 0.24833547013598647, 0.2479815276824827, 0.24798152840485044, 0.25570147377668034 diff --git a/python_scripts/pyproject.toml b/python_scripts/pyproject.toml new file mode 100644 index 00000000..f9864016 --- /dev/null +++ b/python_scripts/pyproject.toml @@ -0,0 +1,7 @@ +[project] +name = "python_scripts" +version = "0.1" +dynamic = ["version"] +description = "My python scripts" +readme = "README.md" +license = "GPL-3.0-or-later" diff --git a/python_scripts/requirements.txt b/python_scripts/requirements.txt new file mode 100644 index 00000000..812865e8 --- /dev/null +++ b/python_scripts/requirements.txt @@ -0,0 +1,25 @@ +annotated-doc==0.0.4 +annotated-types==0.7.0 +anyio==4.14.0 +certifi==2026.6.17 +charset-normalizer==3.4.7 +click==8.4.1 +fastapi==0.137.2 +h11==0.16.0 +idna==3.18 +numpy==2.5.0 +pandas==3.0.3 +pyarrow==24.0.0 +pydantic==2.13.4 +pydantic_core==2.46.4 +python-dateutil==2.9.0.post0 +redis==8.0.0 +redis-streams==0.3.0 +requests==2.34.2 +six==1.17.0 +starlette==1.3.1 +tabulate==0.10.0 +typing-inspection==0.4.2 +typing_extensions==4.15.0 +urllib3==2.7.0 +uvicorn==0.49.0 diff --git a/python_scripts/src/python_scripts/__init__.py b/python_scripts/src/python_scripts/__init__.py new file mode 100644 index 00000000..e69de29b --- /dev/null +++ b/python_scripts/src/python_scripts/__init__.py diff --git a/python_scripts/src/python_scripts/__main__.py b/python_scripts/src/python_scripts/__main__.py new file mode 100644 index 00000000..76b985f9 --- /dev/null +++ b/python_scripts/src/python_scripts/__main__.py @@ -0,0 +1,152 @@ +from inspect import currentframe + +import redis +import pandas as pd +import threshold_based + +api = "http://localhost:1234" +redis_host = "localhost" +stream_key = "postgres_topic" +group = "python_consumer" +r = redis.Redis(redis_host) +PATH = "/home/matt/src/sunfish/python_scripts/src/python_scripts/resources" +similarity_scores = { + "Skype06Exp": 0, + "Skype06LogN": 0, + "Skype06Gam": 0, + "Skype06Wbl": 0, +} + + +def create_dataframes(): + Skype06Exp: str = "%s/Skype06Exp/host.parquet" % PATH + Skype06LogN: str = "%s/Skype06LogN/host.parquet" % PATH + Skype06Gam: str = "%s/Skype06Gam/host.parquet" % PATH + Skype06Wbl: str = "%s/Skype06Wbl/host.parquet" % PATH + # Keep this order whenever operating + listOfHosts = [Skype06Exp, Skype06LogN, Skype06Gam, Skype06Wbl] + listOfDataFrames = [] + + try: + for item in listOfHosts: + df_hosts = pd.read_parquet(item) + listOfDataFrames.append(df_hosts) + return listOfDataFrames + + except Exception as e: + print(f"exception caught: {e}") + exit(1) + + +""" +As suggested in https://redis.readthedocs.io/en/stable/examples/redis-stream-example.html +""" +toack = lambda k, g, e: r.xack(k, g, e) + + +def update_score( + stream, + frames, +): + global similarity_scores + host_stream = str(stream[b"host_id"].decode()) + timestamp_stream = int(stream[b"timestamp"].decode()) + current_frame = frames[0] + + for i in range(len(frames)): + current_frame = frames[i] + filtered_df = current_frame[ + (current_frame["timestamp"] == timestamp_stream) + & (current_frame["host_name"] == host_stream) + ] + + y_t = float(stream[b"downtime"].decode()) + y_t_hat = float(filtered_df["downtime"].iloc[0]) + # Abs is mandatory here! See https://en.wikipedia.org/wiki/Euclidean_distance + e_t_downtime = abs(y_t - y_t_hat) + + y_t = float(stream[b"cpuutilization"].decode()) + y_t_hat = float(filtered_df["cpu_utilization"].iloc[0]) + e_t_cpuUtil = abs(y_t - y_t_hat) + + y_t = float(stream[b"energyusage"].decode()) + y_t_hat = float(filtered_df["energy_usage"].iloc[0]) + e_t_energyUsage = abs(y_t - y_t_hat) + + y_t = float(stream[b"tasksactive"].decode()) + y_t_hat = float(filtered_df["tasks_running"].iloc[0]) + e_t_tasksActive = abs(y_t - y_t_hat) + + S = ( + e_t_downtime * 0.9 + + e_t_cpuUtil * 0.7 + + e_t_tasksActive * 0.6 + + e_t_energyUsage * 0.7 + ) + S = S / (0.9 + 0.7 + 0.6 + 0.7) + + res = list(similarity_scores)[i] + print(res) + oldS = similarity_scores[res] + print(oldS) + new_S = (oldS + S) / 2 + similarity_scores[res] = new_S + + total = 0 + for value in similarity_scores.values(): + total += value + + for key, value in similarity_scores.items(): + similarity_scores[key] = value / total + + return + + +def pretty_print(output_file, stream): + timestamp = int(stream[b"timestamp"].decode()) + output_str = f"{timestamp}, {similarity_scores['Skype06Exp']}, {similarity_scores['Skype06LogN']}, {similarity_scores['Skype06Gam']}, {similarity_scores['Skype06Wbl']}\n" + output_file.write(output_str) + + +def send_insights(stream, frames): + return + + +def print_xreadgroup_reply(reply, frames, output_file): + global threshold + for d_stream in reply: + for element in d_stream[1]: + update_score(element[1], frames) + send_insights(element[1], frames) + toack(d_stream[0], group, element[0]) + pretty_print(output_file, element[1]) + + +def read_entries(r, f, output_file): + while True: + d = r.xreadgroup( + groupname=group, + consumername="c", + block=5, + count=10, + streams={stream_key: ">"}, + ) + print_xreadgroup_reply(d, f, output_file) + + +def main(): + f = open("output.txt", "w") + try: + frames = create_dataframes() + read_entries(r, frames, output_file=f) + + f.close() + except Exception as e: + print(f"exception caught: {e}") + + finally: + f.close() + + +if __name__ == "__main__": + main() diff --git a/python_scripts/src/python_scripts/__pycache__/__main__.cpython-314.pyc b/python_scripts/src/python_scripts/__pycache__/__main__.cpython-314.pyc Binary files differnew file mode 100644 index 00000000..b353e8de --- /dev/null +++ b/python_scripts/src/python_scripts/__pycache__/__main__.cpython-314.pyc diff --git a/python_scripts/src/python_scripts/__pycache__/module2.cpython-314.pyc b/python_scripts/src/python_scripts/__pycache__/module2.cpython-314.pyc Binary files differnew file mode 100644 index 00000000..a88d583d --- /dev/null +++ b/python_scripts/src/python_scripts/__pycache__/module2.cpython-314.pyc diff --git a/python_scripts/src/python_scripts/__pycache__/threshold_based.cpython-314.pyc b/python_scripts/src/python_scripts/__pycache__/threshold_based.cpython-314.pyc Binary files differnew file mode 100644 index 00000000..8817f209 --- /dev/null +++ b/python_scripts/src/python_scripts/__pycache__/threshold_based.cpython-314.pyc diff --git a/python_scripts/src/python_scripts/module2.old b/python_scripts/src/python_scripts/module2.old new file mode 100644 index 00000000..3df7a768 --- /dev/null +++ b/python_scripts/src/python_scripts/module2.old @@ -0,0 +1,44 @@ +import pandas as pd +import requests +import numpy as np + +""" +The threshold should be set based on a OpenDC failure model with statistical distribution. +""" +confidence_red = 0.95 +confidence_yellow = 0.80 +sigma = 0 + + +def check_metric(stream, frame, threshold, dictionary): + timestamp = int(stream[b"timestamp"].decode()) + host = str(stream[b"host_id"].decode()) + new_threshold = threshold + filtered_df = frame[ + (frame["timestamp"] == timestamp) & (frame["host_name"] == host) + ] + y_t = float(stream[b"downtime"].decode()) + y_t_hat = float(filtered_df["downtime"].iloc[0]) + e_t = y_t - y_t_hat + + if e_t > threshold * confidence_red: + print(f"Red,{timestamp}") + dictionary[str(timestamp)] = dictionary.get(str(timestamp), 0) + 1 + + alert = {"Name": "Alert", "Type": "Red", "Timestamp": str(timestamp)} + alert_dt(alert) + return new_threshold + + if e_t > threshold * confidence_yellow: + print(new_threshold) + dictionary[str(timestamp)] = dictionary.get(str(timestamp), 0) + 1 + print(f"Yellow,{timestamp}") + alert = {"Name": "Alert", "Type": "Yellow", "Timestamp": str(timestamp)} + alert_dt(alert) + return new_threshold + + return new_threshold + + +def alert_dt(alert): + requests.post("http://localhost:1234/insight", json=alert) diff --git a/output/greenifier-demo-scaling/raw-output/0/seed=0/battery.parquet b/python_scripts/src/python_scripts/resources/Skype06Exp/battery.parquet Binary files differindex 93b3f21c..93b3f21c 100644 --- a/output/greenifier-demo-scaling/raw-output/0/seed=0/battery.parquet +++ b/python_scripts/src/python_scripts/resources/Skype06Exp/battery.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Exp/host.parquet b/python_scripts/src/python_scripts/resources/Skype06Exp/host.parquet Binary files differnew file mode 100644 index 00000000..e11dc8b0 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Exp/host.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Exp/powerSource.parquet b/python_scripts/src/python_scripts/resources/Skype06Exp/powerSource.parquet Binary files differnew file mode 100644 index 00000000..bf7ab98d --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Exp/powerSource.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Exp/service.parquet b/python_scripts/src/python_scripts/resources/Skype06Exp/service.parquet Binary files differnew file mode 100644 index 00000000..3d494013 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Exp/service.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Exp/task.parquet b/python_scripts/src/python_scripts/resources/Skype06Exp/task.parquet Binary files differnew file mode 100644 index 00000000..2b79e0ad --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Exp/task.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Gam/battery.parquet b/python_scripts/src/python_scripts/resources/Skype06Gam/battery.parquet Binary files differnew file mode 100644 index 00000000..93b3f21c --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Gam/battery.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Gam/host.parquet b/python_scripts/src/python_scripts/resources/Skype06Gam/host.parquet Binary files differnew file mode 100644 index 00000000..dc98c778 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Gam/host.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Gam/powerSource.parquet b/python_scripts/src/python_scripts/resources/Skype06Gam/powerSource.parquet Binary files differnew file mode 100644 index 00000000..7247d165 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Gam/powerSource.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Gam/service.parquet b/python_scripts/src/python_scripts/resources/Skype06Gam/service.parquet Binary files differnew file mode 100644 index 00000000..951429cf --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Gam/service.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Gam/task.parquet b/python_scripts/src/python_scripts/resources/Skype06Gam/task.parquet Binary files differnew file mode 100644 index 00000000..643939d1 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Gam/task.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06LogN/battery.parquet b/python_scripts/src/python_scripts/resources/Skype06LogN/battery.parquet Binary files differnew file mode 100644 index 00000000..93b3f21c --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06LogN/battery.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06LogN/host.parquet b/python_scripts/src/python_scripts/resources/Skype06LogN/host.parquet Binary files differnew file mode 100644 index 00000000..b145bb65 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06LogN/host.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06LogN/powerSource.parquet b/python_scripts/src/python_scripts/resources/Skype06LogN/powerSource.parquet Binary files differnew file mode 100644 index 00000000..466d3215 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06LogN/powerSource.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06LogN/service.parquet b/python_scripts/src/python_scripts/resources/Skype06LogN/service.parquet Binary files differnew file mode 100644 index 00000000..33795ed3 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06LogN/service.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06LogN/task.parquet b/python_scripts/src/python_scripts/resources/Skype06LogN/task.parquet Binary files differnew file mode 100644 index 00000000..e356ef75 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06LogN/task.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Wbl/battery.parquet b/python_scripts/src/python_scripts/resources/Skype06Wbl/battery.parquet Binary files differnew file mode 100644 index 00000000..93b3f21c --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Wbl/battery.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Wbl/host.parquet b/python_scripts/src/python_scripts/resources/Skype06Wbl/host.parquet Binary files differnew file mode 100644 index 00000000..9fdf3210 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Wbl/host.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Wbl/powerSource.parquet b/python_scripts/src/python_scripts/resources/Skype06Wbl/powerSource.parquet Binary files differnew file mode 100644 index 00000000..3c79b19b --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Wbl/powerSource.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Wbl/service.parquet b/python_scripts/src/python_scripts/resources/Skype06Wbl/service.parquet Binary files differnew file mode 100644 index 00000000..4d40319d --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Wbl/service.parquet diff --git a/python_scripts/src/python_scripts/resources/Skype06Wbl/task.parquet b/python_scripts/src/python_scripts/resources/Skype06Wbl/task.parquet Binary files differnew file mode 100644 index 00000000..1453b91e --- /dev/null +++ b/python_scripts/src/python_scripts/resources/Skype06Wbl/task.parquet diff --git a/python_scripts/src/python_scripts/resources/main.py b/python_scripts/src/python_scripts/resources/main.py new file mode 100644 index 00000000..accd0314 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/main.py @@ -0,0 +1,6 @@ +import pandas as pd +import numpy as np + +df = pd.read_parquet("Skype06Gam/task.parquet") + +print(df.loc[:, "num_failures"].mean()) diff --git a/python_scripts/src/python_scripts/resources/prefabs_old/battery.parquet b/python_scripts/src/python_scripts/resources/prefabs_old/battery.parquet Binary files differnew file mode 100644 index 00000000..93b3f21c --- /dev/null +++ b/python_scripts/src/python_scripts/resources/prefabs_old/battery.parquet diff --git a/python_scripts/src/python_scripts/resources/prefabs_old/host.parquet b/python_scripts/src/python_scripts/resources/prefabs_old/host.parquet Binary files differnew file mode 100644 index 00000000..e9add02b --- /dev/null +++ b/python_scripts/src/python_scripts/resources/prefabs_old/host.parquet diff --git a/python_scripts/src/python_scripts/resources/prefabs_old/powerSource.parquet b/python_scripts/src/python_scripts/resources/prefabs_old/powerSource.parquet Binary files differnew file mode 100644 index 00000000..304b09cf --- /dev/null +++ b/python_scripts/src/python_scripts/resources/prefabs_old/powerSource.parquet diff --git a/python_scripts/src/python_scripts/resources/prefabs_old/readme.txt b/python_scripts/src/python_scripts/resources/prefabs_old/readme.txt new file mode 100644 index 00000000..b00086b8 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/prefabs_old/readme.txt @@ -0,0 +1,7 @@ +This is the result of a raw simulation with the failureModels set to some statistical distribution in OpenDC. +Based on the downtime in the `host.parquet` file, we calculate the threshold. +These results are based on this failureModel: +{ + "type": "prefab", + "prefabName" : "Pl05Exp" +} diff --git a/python_scripts/src/python_scripts/resources/prefabs_old/service.parquet b/python_scripts/src/python_scripts/resources/prefabs_old/service.parquet Binary files differnew file mode 100644 index 00000000..371a66a7 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/prefabs_old/service.parquet diff --git a/python_scripts/src/python_scripts/resources/prefabs_old/task.parquet b/python_scripts/src/python_scripts/resources/prefabs_old/task.parquet Binary files differnew file mode 100644 index 00000000..1e62dae2 --- /dev/null +++ b/python_scripts/src/python_scripts/resources/prefabs_old/task.parquet diff --git a/python_scripts/src/python_scripts/threshold_based.py b/python_scripts/src/python_scripts/threshold_based.py new file mode 100644 index 00000000..13ce5e42 --- /dev/null +++ b/python_scripts/src/python_scripts/threshold_based.py @@ -0,0 +1,34 @@ +import pandas as pd +import requests +import numpy as np + +""" +The threshold should be set based on a OpenDC failure model with statistical distribution. +""" +confidence_red = 0.90 +confidence_yellow = 0.80 +sigma = 0 + + +def check_metric(stream, frame, threshold, dictionary, output_file): + timestamp = int(stream[b"timestamp"].decode()) + host = str(stream[b"host_id"].decode()) + + dictionary[str(timestamp)] = dictionary.get(str(timestamp), 0) + 1 + + e_t = dictionary.get(str(timestamp)) + # Heuristic based recognition + if e_t > threshold * confidence_red: + output_file.write(f"Timestamp: {timestamp}, Alarm: red\n") + alert = {"Name": "Alert", "Type": "Red", "Timestamp": str(timestamp)} + alert_dt(alert) + return + + if e_t > threshold * confidence_yellow: + output_file.write(f"Timestamp: {timestamp}, Alarm: yellow\n") + alert = {"Name": "Alert", "Type": "Yellow", "Timestamp": str(timestamp)} + alert_dt(alert) + + +def alert_dt(alert): + requests.post("http://localhost:1234/insight", json=alert) diff --git a/resources/experiments/experiment_failures.json b/resources/experiments/experiment_failures.json new file mode 100644 index 00000000..0b451ebe --- /dev/null +++ b/resources/experiments/experiment_failures.json @@ -0,0 +1,26 @@ +{ + "name": "greenifier-demo-scaling", + "topologies": [ + { + "pathToFile": "resources/topologies/surf.json" + } + ], + "workloads": [ + { + "pathToFile": "resources/workloads/surf_month", + "type": "ComputeWorkload" + } + ], + + "failureModels" : [ + { + "type": "trace-based", + "pathToFile": "resources/failures/Skype_user_reported.parquet" + } + ], + "exportModels": [ + { + "exportInterval": 3600 + } + ] +} diff --git a/resources/failures/Facebook_user_reported.parquet b/resources/failures/Facebook_user_reported.parquet Binary files differnew file mode 100755 index 00000000..7af770cb --- /dev/null +++ b/resources/failures/Facebook_user_reported.parquet diff --git a/resources/failures/Gmail_user_reported.parquet b/resources/failures/Gmail_user_reported.parquet Binary files differnew file mode 100755 index 00000000..db039fd1 --- /dev/null +++ b/resources/failures/Gmail_user_reported.parquet diff --git a/resources/failures/Skype_user_reported.parquet b/resources/failures/Skype_user_reported.parquet Binary files differnew file mode 100755 index 00000000..316706c7 --- /dev/null +++ b/resources/failures/Skype_user_reported.parquet diff --git a/resources/failures/Twitter_user_reported.parquet b/resources/failures/Twitter_user_reported.parquet Binary files differnew file mode 100755 index 00000000..c4679046 --- /dev/null +++ b/resources/failures/Twitter_user_reported.parquet diff --git a/resources/failures/Whatsapp_user_reported.parquet b/resources/failures/Whatsapp_user_reported.parquet Binary files differnew file mode 100755 index 00000000..0e954d70 --- /dev/null +++ b/resources/failures/Whatsapp_user_reported.parquet diff --git a/resources/failures/YouTube_user_reported.parquet b/resources/failures/YouTube_user_reported.parquet Binary files differnew file mode 100755 index 00000000..1c0c6504 --- /dev/null +++ b/resources/failures/YouTube_user_reported.parquet diff --git a/shell_scripts/kafka_connect.sh b/shell_scripts/kafka_connect.sh index 3f22b6de..fecc8f63 100755 --- a/shell_scripts/kafka_connect.sh +++ b/shell_scripts/kafka_connect.sh @@ -1,2 +1,6 @@ #!/bin/bash /opt/confluent/bin/connect-standalone /opt/confluent/etc/kafka/connect-standalone.properties /opt/confluent/share/confluent-common/connectors/sink-jdbc.properties /opt/confluent/share/confluent-common/connectors/sink-redis.properties +# Considerable effort has gone into filtering messages in situ. Please refer to: +# https://forum.confluent.io/t/invalid-value-for-filter-in-smt/10046 +# https://github.com/json-path/JsonPath +# https://docs.confluent.io/kafka-connectors/transforms/current/filter-confluent.html#properties diff --git a/shell_scripts/readme.md b/shell_scripts/readme.md deleted file mode 100644 index 2a0842b1..00000000 --- a/shell_scripts/readme.md +++ /dev/null @@ -1,13 +0,0 @@ -You need to run this each time you change `schema.proto` -```bash -cd resources/experiments/ -protoc --java_out=/home/matt/git/opendc/opendc-common/src/main/java/ schema.proto -curl -X DELETE http://localhost:8081/subjects/postgres-topic-value -``` -Confluent JDBC sink and source (includes Postgres connector) files: -https://www.confluent.io/hub/confluentinc/kafka-connect-jdbc - -Be aware the project will only work with this OpenDC as there are hidden Maven dependencies. -Kafka topic should be named `postgres_topic`. - -Confluent local (see https://www.confluent.io/installation/): |