rgw: use explicit flag to cancel RGWCoroutinesManager::run()

rgw: use explicit flag to cancel RGWCoroutinesManager::run()

RGWCoroutinesManager::run() was setting ret = -ECANCELED to break out of
the loop when it sees going_down. coroutines that failed with -ECANCELED
were confusing this logic and leading to coroutine deadlock assertions
below. when we hit the going_down case, set a 'canceled' flag, and check
that flag when deciding whether to break out of the loop

Fixes: http://tracker.ceph.com/issues/17465
Signed-off-by: Casey Bodley <cbodley@redhat.com>