common/async: remove error_codes from co_throttle interface

common/async: remove error_codes from co_throttle interface

in the initial design of co_throttle described in
https://github.com/ceph/ceph/pull/49720, the cancel_on_error option only
applied to errors from awaitable<error_code> but not to exceptions from
awaitable<void> coroutines

with the decision to use exceptions as the default method of error
handling in rgw multisite, this design choice no longer makes sense.
i've removed the error_code overloads entirely, and changed the
exception handling logic to match the previous behavior for error codes

the unit tests were rewritten with co_waiter instead of timers to make
them deterministic and faster. co_waiter's cancellation behavior
exposed some issues where the cancellation signal could cause the
completions to recurse, so on_complete() was restructured to tolerate
that

Signed-off-by: Casey Bodley <cbodley@redhat.com>