rgw/async/notifications: use common async waiter in pubsub push
* use the "yield_waiter" and "waiter" from common/async insteasd of the "waiter"
implemented inside the bucket notification code (this is so we don't
need separate investigations for 2 implementations)
* added a unit test that simulate how a separate thread (kafka or amqp) is
resuming a coroutine which is created by either the frontend or the
notification manager.
before using "defer" the unit test is passing, however,
when executed under thread sanitizer (using the WITH_TSAN cmake flag)
the following errors are observed: https://0x0.st/Xp4P.txt
after using "defer" the unit test passes under TSAN without errors.
Fixes: https://tracker.ceph.com/issues/64184
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit
2872c75f184c9e715219dfa9ad44f5b6cfe4e1fe)