PLUGINS.md - gbmcweb - Git at Google

 # gBMCWeb Plugin

 This document outlines the design of a plugin system for the Google BMC Redfish
 server. It also provides a user guide for developers.

 ## Background

 ### gBMCWeb and Crow

 [CrowCpp/Crow](https://github.com/CrowCpp/Crow) is a C++ framework for creating
 HTTP or Websocket web services. It
 is easy to use, thanks to its routing system which is similar to
 Python's [Flask](https://flask.palletsprojects.com/en/2.3.x/).
 The core HTTP functionality of gBMCWeb is derived from Crow.

 #### Route: URL, Method, and Handler

 In Crow, a route determines what happens when your client connects to a
 specific URL. gBMCWeb supports not only Redfish routes, but also other routes
 such as WebSocket or dynamic routes, although there are no use cases of
 non-Redfish routes inside Google. The three main components of a Redfish route
 are:

 - URL, the relative path assigned to the route. In Redfish, a URL is
   corresponding to a Redfish resource. A URL can have parameters. For instance,
   the template “/redfish/v1/Chassis/<string>” matches requests to any Chassis.

 - Method, the HTTP method of the route. In Redfish, a server must implement GET,
   POST, DELETE, and PATCH.

 - Handler, a piece of code that is executed whenever the client calls the
   associated route. In gBMCWeb, a Redfish handler is typically a lambda
   expression
   or a function that takes a "Request" object, a "Response" object, and any
   parameters in the URL template.

 In BMCWeb, you can use the `BMCWEB_ROUTE` macro to register a route.

 ### Lack of Plugin System

 One of the most frustrating drawbacks of OpenBMC/BMCweb is that it does not
 support plugins. This means that, without merging code into the upstream, there
 is no way to add new components to or modify its existing behaviors, without
 patches in the Yocto meta layer.

 ## Overview

 This document proposes a plugin system that can be used to solve the problems
 mentioned in the background section. The proposed system has the following
 features:

 - It can be loaded as static libraries at compile, which means that the logic in
   a plugin will not affect the main gBMCWeb daemon if it is not loaded.

 - It can extend existing behaviors, such as adding an OEM property to an
   existing Redfish resource.

 - It can replace existing functionality, such as replacing the execution codes
   of an existing Redfish resource completely.

 - It can add new Redfish resources or actions, such as implementing new Redfish
   resources and its associated actions, especially OEM or experimental resources
   that don’t affect other platforms.

 Using gBMCWeb plugins, you can achieve the following:

 - Faster iteration: machine-specific logic can now be implemented as plugins by
   Google or vendors, which speeds up the process.
 - Isolation: The main branch of gBMCWeb will become cleaner as hacks or
   technical debts can be platform-specific plugins. Rest of the platforms won’t
   be affected as they don’t include these plugins.

 ## Detailed Design

 ### Append

 The append plugin is supposed to be the most commonly used plugin by developers.
 It allows inserting another asynchronous callback function after the original
 handler.

 The implementation first locates the corresponding rule based on the given URL
 and HTTP method. Then, it replaces the callback function with a new function,
 inside which the old callback is executed first. When the AsyncResp is
 destructed, the newly appended logic is executed before the Redfish parameter
 handling codes. Finally, the resulting response is written back to the clients.
 The following code snippet is the proposed implementation.

 The implementation can be found
 in [this commit](https://gbmc.googlesource.com/gbmcweb/+/fa9601da5c5f414af4542c79df1d0a2d7d26b1a4)

 Users are expected to use the following macro to register an "Append" handler
 for a specific route.

 ```c++
 #define REDFISH_HANDLER_APPEND(url, verb, func)                                \
     appendRedfishHandler<crow::black_magic::getParameterTag(url)>(url, verb,   \
                                                                   func);
 ```

 For example, you can override a property of a Redfish handler.

 ```c++
 void crashdumpServiceGetPlugin(
     const crow::Request&, const std::shared_ptr<bmcweb::AsyncResp>& asyncResp,
     const std::string&)
 {
     asyncResp->res.jsonValue["MaxNumberOfRecords"] = 10;
 }


 REDFISH_HANDLER_APPEND("/redfish/v1/Systems/<str>/LogServices/Crashdump/",
                            boost::beast::http::verb::get,
                            crashdumpServiceGetPlugin);
 ```

 ### Replace

 The replace handler can supersede the original callback of an existing route.
 The implementation is rather straightforward. It first locates the corresponding
 rule, as before. Then, it completely replaces the callback function with the
 newly fed callback functions.

 The implementation can be found
 in [this commit](https://gbmc.googlesource.com/gbmcweb/+/c2761daf22230759e69ad30be61f466ecdee2f1f).

 Users are expected to use the following macro to register a "Replace" handler
 for a specific route.

 ```c++
 #define REDFISH_HANDLER_REPLACE(url, verb, func)                               \
     replaceRedfishHandler<crow::black_magic::getParameterTag(url)>(url, verb,  \
                                                                    func);
 ```

 We shall demonstrate the Append handler and Replace handler together with the
 following code snippet. The following code first replaces the existing static
 handler of “GET /redfish” with an asynchronous DBus call that is always going to
 fail via the replace plugin. Then it injects an extra code to run after the
 asynchronous call finishes via the append plugin.

 ```c++
 void redfishGetReplaced(const crow::Request& req,
                         const std::shared_ptr<bmcweb::AsyncResp>& asyncResp)
 {
     if (!redfish::setUpRedfishRoute(req, asyncResp))
     {
         return;
     }
     managedStore::GetManagedObjectStore()->PostDbusCallToIoContextThreadSafe(
         [asyncResp](const boost::system::error_code ec,
                    const std::variant<std::string>&) {
         if (ec)
         {
             asyncResp->res.jsonValue["dbus_call_result"] = "failed";
             return;
         }

         },
         "non-exist-service", "/no/exist/object",
         "org.freedesktop.DBus.Properties", "Get",
         "xyz.openbmc_project.Non.Exist.Interface", "Property");
     asyncResp->res.jsonValue["v1"] = "hacked";
 }

 void redfishGetAppended(const crow::Request&,
                         const std::shared_ptr<bmcweb::AsyncResp>& asyncResp)
 {
     if (asyncResp->res.jsonValue["dbus_call_result"] != "failed") {
       asyncResp->res.jsonValue["plugin_is_working"] = "false";
       return;
     }
     asyncResp->res.jsonValue["plugin_is_working"] = "true";
 }

 REDFISH_HANDLER_REPLACE("/redfish/", boost::beast::http::verb::get,
                         redfishGetReplaced);
 REDFISH_HANDLER_APPEND("/redfish/", boost::beast::http::verb::get,
                        redfishGetAppended);
 ```

 With a correctly implemented plugin system, we will see this result

 ```shell
 GET  http://localhost:18080/redfish/

 {
   "dbus_call_result": "failed",
   "plugin_is_working": "true",
   "v1": "hacked"
 }
 ```

 ### Add

 The add plugin is another frequently used tool that allows users to add a new
 route. The implementation of this plugin is the most straightforward. We simply
 need to leverage the existing “newRuleTagged” template function.

 The implementation can be found
 in [this commit](https://gbmc.googlesource.com/gbmcweb/+/1249173411d3d064a2d20bbaf39183d94d41437e).

 Users are expected to use the following macro to add a Redfish route.

 ```c++
 #define REDFISH_HANDLER_ADD(url, verb, privileges, func)                       \
     plugins::addRedfishHandler<crow::black_magic::getParameterTag(url)>(       \
         url, verb, privileges, func);
 ```

 ### Remove

 The remove plugin is the inverse of the add plugin and should be used sparingly.
 It permits users to delete an existing route.

 The implementation can be found
 in [this commit](https://gbmc.googlesource.com/gbmcweb/+/53d6d73c9d06f6a2400615bb71a61728f9af4abc).

 Users are expected to use the following macro to remove a Redfish route.

 ```c++
 #define REDFISH_HANDLER_REMOVE(url, verb)                                      \
     plugins::removeRedfishHandler<crow::black_magic::getParameterTag(url)>(    \
         url, verb);
 ```

 ### Load Plugins

 gBMCWeb, which inherits from Crow, leverages a Trie-like structure in the
 following sequence:

 1. All routes are added by the “BMCWEB_ROUTE” macro and stored in a container of
    type Rule.
 2. In the main thread, when the HTTP App starts running, the list of rules is
    iterated and added to a Trie. Each leaf node contains a pointer to the Rule
    object.
 3. The Trie is looked up when new HTTP requests arrive. If there is a match, the
    callback function stored in the leaf node will be invoked.

 To load plugins, we need to inject code that can modify the rule container
 before the trie is built and the HTTP App runs.

 The implementation can be found
 in [this commit](https://gbmc.googlesource.com/gbmcweb/+/c783047b85f0d041e77c4a58c22cf9641c9e7b7e).

 ### Install Headers

 To make compilation of a plugin work, a few symbols have to be declared and
 exposed to plugin libraries through headers. We have cleaned up the required
 headers so they form a complete set.

 A distinct recipe has been developed for a plugin repository. Please
 see <https://gbmc.googlesource.com/meta-gbmc-staging/+/refs/heads/master/recipes-phosphor/interfaces/gbmcweb-headers_git.bb>.

 ## Unit test

 See [macros_test.cpp](plugins/macros_test.cpp) for the implemented unit tests.
 Feel free to contribute if you found a case is not coverred.

 ## Plugin Repo and Recipe

 A plugin needs to hosted by a GIT repo. We have developed an example repo for
 users' reference. Please
 see <https://gbmc.googlesource.com/gbmcweb-platform-plugins-example/+/refs/heads/master>.

 Its recipe is available
 in <https://gbmc.googlesource.com/meta-gbmc-staging/+/refs/heads/master/recipes-phosphor/interfaces/gbmcweb-platform-plugins_git.bb>.

 ## Enable Plugin

 To enable gBMCWeb Plugin, firstly you need to enable the Meson flag in the
 bmcweb recipe.

 ```python
 # recipes-phosphor/interfaces/bmcweb_%.bbappend
 PACKAGECONFIG: append = "gbmcweb-platform-plugins"
 ```

 Secondly, replace the SRC_URI and SRCREV of the  `gbmcweb-platform-plugins`
 recipe such that it points out to your plugin repo.

 ```python
 # recipes-phosphor/interfaces/gbmcweb-platform-plugins_%.bbappend
 SRC_URI = "git://$YOUR_HOST/$YOUR_REPO;branch=master;protocol=https"
 SRCREV = "$YOUR_COMMIT"
 ```

 With above steps, the BMCWeb binary now will include and run the plugin at
 startup.
	# gBMCWeb Plugin

	This document outlines the design of a plugin system for the Google BMC Redfish
	server. It also provides a user guide for developers.

	## Background

	### gBMCWeb and Crow

	[CrowCpp/Crow](https://github.com/CrowCpp/Crow) is a C++ framework for creating
	HTTP or Websocket web services. It
	is easy to use, thanks to its routing system which is similar to
	Python's [Flask](https://flask.palletsprojects.com/en/2.3.x/).
	The core HTTP functionality of gBMCWeb is derived from Crow.

	#### Route: URL, Method, and Handler

	In Crow, a route determines what happens when your client connects to a
	specific URL. gBMCWeb supports not only Redfish routes, but also other routes
	such as WebSocket or dynamic routes, although there are no use cases of
	non-Redfish routes inside Google. The three main components of a Redfish route
	are:

	- URL, the relative path assigned to the route. In Redfish, a URL is
	corresponding to a Redfish resource. A URL can have parameters. For instance,
	the template “/redfish/v1/Chassis/<string>” matches requests to any Chassis.

	- Method, the HTTP method of the route. In Redfish, a server must implement GET,
	POST, DELETE, and PATCH.

	- Handler, a piece of code that is executed whenever the client calls the
	associated route. In gBMCWeb, a Redfish handler is typically a lambda
	expression
	or a function that takes a "Request" object, a "Response" object, and any
	parameters in the URL template.

	In BMCWeb, you can use the `BMCWEB_ROUTE` macro to register a route.

	### Lack of Plugin System

	One of the most frustrating drawbacks of OpenBMC/BMCweb is that it does not
	support plugins. This means that, without merging code into the upstream, there
	is no way to add new components to or modify its existing behaviors, without
	patches in the Yocto meta layer.

	## Overview

	This document proposes a plugin system that can be used to solve the problems
	mentioned in the background section. The proposed system has the following
	features:

	- It can be loaded as static libraries at compile, which means that the logic in
	a plugin will not affect the main gBMCWeb daemon if it is not loaded.

	- It can extend existing behaviors, such as adding an OEM property to an
	existing Redfish resource.

	- It can replace existing functionality, such as replacing the execution codes
	of an existing Redfish resource completely.

	- It can add new Redfish resources or actions, such as implementing new Redfish
	resources and its associated actions, especially OEM or experimental resources
	that don’t affect other platforms.

	Using gBMCWeb plugins, you can achieve the following:

	- Faster iteration: machine-specific logic can now be implemented as plugins by
	Google or vendors, which speeds up the process.
	- Isolation: The main branch of gBMCWeb will become cleaner as hacks or
	technical debts can be platform-specific plugins. Rest of the platforms won’t
	be affected as they don’t include these plugins.

	## Detailed Design

	### Append

	The append plugin is supposed to be the most commonly used plugin by developers.
	It allows inserting another asynchronous callback function after the original
	handler.

	The implementation first locates the corresponding rule based on the given URL
	and HTTP method. Then, it replaces the callback function with a new function,
	inside which the old callback is executed first. When the AsyncResp is
	destructed, the newly appended logic is executed before the Redfish parameter
	handling codes. Finally, the resulting response is written back to the clients.
	The following code snippet is the proposed implementation.

	The implementation can be found
	in [this commit](https://gbmc.googlesource.com/gbmcweb/+/fa9601da5c5f414af4542c79df1d0a2d7d26b1a4)

	Users are expected to use the following macro to register an "Append" handler
	for a specific route.

	```c++
	#define REDFISH_HANDLER_APPEND(url, verb, func) \
	appendRedfishHandler<crow::black_magic::getParameterTag(url)>(url, verb, \
	func);
	```

	For example, you can override a property of a Redfish handler.

	```c++
	void crashdumpServiceGetPlugin(
	const crow::Request&, const std::shared_ptr<bmcweb::AsyncResp>& asyncResp,
	const std::string&)
	{
	asyncResp->res.jsonValue["MaxNumberOfRecords"] = 10;
	}


	REDFISH_HANDLER_APPEND("/redfish/v1/Systems/<str>/LogServices/Crashdump/",
	boost::beast::http::verb::get,
	crashdumpServiceGetPlugin);
	```

	### Replace

	The replace handler can supersede the original callback of an existing route.
	The implementation is rather straightforward. It first locates the corresponding
	rule, as before. Then, it completely replaces the callback function with the
	newly fed callback functions.

	The implementation can be found
	in [this commit](https://gbmc.googlesource.com/gbmcweb/+/c2761daf22230759e69ad30be61f466ecdee2f1f).

	Users are expected to use the following macro to register a "Replace" handler
	for a specific route.

	```c++
	#define REDFISH_HANDLER_REPLACE(url, verb, func) \
	replaceRedfishHandler<crow::black_magic::getParameterTag(url)>(url, verb, \
	func);
	```

	We shall demonstrate the Append handler and Replace handler together with the
	following code snippet. The following code first replaces the existing static
	handler of “GET /redfish” with an asynchronous DBus call that is always going to
	fail via the replace plugin. Then it injects an extra code to run after the
	asynchronous call finishes via the append plugin.

	```c++
	void redfishGetReplaced(const crow::Request& req,
	const std::shared_ptr<bmcweb::AsyncResp>& asyncResp)
	{
	if (!redfish::setUpRedfishRoute(req, asyncResp))
	{
	return;
	}
	managedStore::GetManagedObjectStore()->PostDbusCallToIoContextThreadSafe(
	[asyncResp](const boost::system::error_code ec,
	const std::variant<std::string>&) {
	if (ec)
	{
	asyncResp->res.jsonValue["dbus_call_result"] = "failed";
	return;
	}

	},
	"non-exist-service", "/no/exist/object",
	"org.freedesktop.DBus.Properties", "Get",
	"xyz.openbmc_project.Non.Exist.Interface", "Property");
	asyncResp->res.jsonValue["v1"] = "hacked";
	}

	void redfishGetAppended(const crow::Request&,
	const std::shared_ptr<bmcweb::AsyncResp>& asyncResp)
	{
	if (asyncResp->res.jsonValue["dbus_call_result"] != "failed") {
	asyncResp->res.jsonValue["plugin_is_working"] = "false";
	return;
	}
	asyncResp->res.jsonValue["plugin_is_working"] = "true";
	}

	REDFISH_HANDLER_REPLACE("/redfish/", boost::beast::http::verb::get,
	redfishGetReplaced);
	REDFISH_HANDLER_APPEND("/redfish/", boost::beast::http::verb::get,
	redfishGetAppended);
	```

	With a correctly implemented plugin system, we will see this result

	```shell
	GET http://localhost:18080/redfish/

	{
	"dbus_call_result": "failed",
	"plugin_is_working": "true",
	"v1": "hacked"
	}
	```

	### Add

	The add plugin is another frequently used tool that allows users to add a new
	route. The implementation of this plugin is the most straightforward. We simply
	need to leverage the existing “newRuleTagged” template function.

	The implementation can be found
	in [this commit](https://gbmc.googlesource.com/gbmcweb/+/1249173411d3d064a2d20bbaf39183d94d41437e).

	Users are expected to use the following macro to add a Redfish route.

	```c++
	#define REDFISH_HANDLER_ADD(url, verb, privileges, func) \
	plugins::addRedfishHandler<crow::black_magic::getParameterTag(url)>( \
	url, verb, privileges, func);
	```

	### Remove

	The remove plugin is the inverse of the add plugin and should be used sparingly.
	It permits users to delete an existing route.

	The implementation can be found
	in [this commit](https://gbmc.googlesource.com/gbmcweb/+/53d6d73c9d06f6a2400615bb71a61728f9af4abc).

	Users are expected to use the following macro to remove a Redfish route.

	```c++
	#define REDFISH_HANDLER_REMOVE(url, verb) \
	plugins::removeRedfishHandler<crow::black_magic::getParameterTag(url)>( \
	url, verb);
	```

	### Load Plugins

	gBMCWeb, which inherits from Crow, leverages a Trie-like structure in the
	following sequence:

	1. All routes are added by the “BMCWEB_ROUTE” macro and stored in a container of
	type Rule.
	2. In the main thread, when the HTTP App starts running, the list of rules is
	iterated and added to a Trie. Each leaf node contains a pointer to the Rule
	object.
	3. The Trie is looked up when new HTTP requests arrive. If there is a match, the
	callback function stored in the leaf node will be invoked.

	To load plugins, we need to inject code that can modify the rule container
	before the trie is built and the HTTP App runs.

	The implementation can be found
	in [this commit](https://gbmc.googlesource.com/gbmcweb/+/c783047b85f0d041e77c4a58c22cf9641c9e7b7e).

	### Install Headers

	To make compilation of a plugin work, a few symbols have to be declared and
	exposed to plugin libraries through headers. We have cleaned up the required
	headers so they form a complete set.

	A distinct recipe has been developed for a plugin repository. Please
	see <https://gbmc.googlesource.com/meta-gbmc-staging/+/refs/heads/master/recipes-phosphor/interfaces/gbmcweb-headers_git.bb>.

	## Unit test

	See [macros_test.cpp](plugins/macros_test.cpp) for the implemented unit tests.
	Feel free to contribute if you found a case is not coverred.

	## Plugin Repo and Recipe

	A plugin needs to hosted by a GIT repo. We have developed an example repo for
	users' reference. Please
	see <https://gbmc.googlesource.com/gbmcweb-platform-plugins-example/+/refs/heads/master>.

	Its recipe is available
	in <https://gbmc.googlesource.com/meta-gbmc-staging/+/refs/heads/master/recipes-phosphor/interfaces/gbmcweb-platform-plugins_git.bb>.

	## Enable Plugin

	To enable gBMCWeb Plugin, firstly you need to enable the Meson flag in the
	bmcweb recipe.

	```python
	# recipes-phosphor/interfaces/bmcweb_%.bbappend
	PACKAGECONFIG: append = "gbmcweb-platform-plugins"
	```

	Secondly, replace the SRC_URI and SRCREV of the `gbmcweb-platform-plugins`
	recipe such that it points out to your plugin repo.

	```python
	# recipes-phosphor/interfaces/gbmcweb-platform-plugins_%.bbappend
	SRC_URI = "git://$YOUR_HOST/$YOUR_REPO;branch=master;protocol=https"
	SRCREV = "$YOUR_COMMIT"
	```

	With above steps, the BMCWeb binary now will include and run the plugin at
	startup.