nvmed: Fix nvme worker thread deadlock The issue was nvme worker can be destroyed on it's own thread which will make a deadlock saying it is waiting for itself in the destructor of the NVMeMiWorker. We have such case because we are passing sharedPtr of nvmeMi object which is the owner of worker. While the nvmeMi object is deleted from the main thread (remove entry from deviceMap) but worker is still running the job where the shared ptr of nvme mi has already passed. Now nvme mi will try to be destroyed in the worker thread which means worker thread will also be destroyed there and will create deadlock since the destructor is waiting for the thread to be done and the thread is waiting for the destructor to be done. To resolve this, we need to make sure the destruction is happening in main thread. Created sharedPtr of worker with custom deleter which will make sure the final destruction is happening in the main thread. Also made sure this is the only way of creating NVMeMiWorker. Tested: Send interface removed signal continuously and in parallel make dbus query to host initiated telemetry log. Detailed script can be found in b/438827535#comment17. Did not see any resource deadlock. Google-Bug-Id: 438827535 Change-Id: I4b42e9647a1673c1da2f73b89981b50e570a9523 Signed-off-by: Munawar Hussain <munawarhussain@google.com>
diff --git a/src/NVMeMi.cpp b/src/NVMeMi.cpp index 1b01d89..3cb030b 100644 --- a/src/NVMeMi.cpp +++ b/src/NVMeMi.cpp
@@ -286,6 +286,7 @@ } thread.join(); } + NVMeMi::~NVMeMi() { // If we're being destructed the only thing left to do is to clean up the
diff --git a/src/NVMeMi.hpp b/src/NVMeMi.hpp index b648f33..ecc4aff 100644 --- a/src/NVMeMi.hpp +++ b/src/NVMeMi.hpp
@@ -6,6 +6,7 @@ #include <cstdint> #include <thread> +#include <utility> // A worker thread for calling NVMeMI cmd. class NVMeMiWorker @@ -18,11 +19,39 @@ bool workerIsNotified = false; std::thread thread; - public: + // private to enforce the use of static create NVMeMiWorker(); + + public: NVMeMiWorker(const NVMeMiWorker&) = delete; ~NVMeMiWorker(); void post(std::function<void(void)>&& func); + + /** + will create a shared_ptr of NVMeMiWorker asserting we are in the same + thread that is running the passed io context and the shared_ptr will have + custom deleter which will ensure the NVMeMiWorker will be destroyed in + the passing io. The main purpose of this is to ensure the worker is being + destroyed in same thread it was created from following single thread + arch. + + @param io the io context of the caller thread, assertion will fail if the + io context is not from the caller thread + */ + template <typename... Args> + static std::shared_ptr<NVMeMiWorker> create(boost::asio::io_context& io, + Args&&... args) + { + assert(io.get_executor().running_in_this_thread() && + "NVMeMiWorker::create called from wrong io_context thread and " + "should be called from the same thread that is running the io " + "context."); + // Note: NVMeMiWorker constructor is private + auto* worker = new NVMeMiWorker(std::forward<Args>(args)...); + return {worker, [&io](NVMeMiWorker* objPtr) { + boost::asio::post(io, [objPtr]() { delete objPtr; }); + }}; + } }; class NVMeMi : public NVMeMiIntf, public std::enable_shared_from_this<NVMeMi>
diff --git a/src/NVMeSensorMain.cpp b/src/NVMeSensorMain.cpp index 8abea9a..e04dbf4 100644 --- a/src/NVMeSensorMain.cpp +++ b/src/NVMeSensorMain.cpp
@@ -330,7 +330,7 @@ if (res == workerMap.end() || res->second.expired()) { - worker = std::make_shared<NVMeMiWorker>(); + worker = NVMeMiWorker::create(io); workerMap[*root] = worker; } else @@ -340,7 +340,7 @@ } else { - worker = std::make_shared<NVMeMiWorker>(); + worker = NVMeMiWorker::create(io); } try