rgw: cancel olh modification by removing pending xattrs in set_olh error scenarios

rgw: cancel olh modification by removing pending xattrs in set_olh error scenarios

Makes request threads clean up after themselves and remove their pending
xattrs from OLH objects before they return an error from set_olh.

Fixes: https://tracker.ceph.com/issues/59663
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 38133e59d293dc8c16c92666967783d9e08a79cd)

Conflicts:
	qa/workunits/rgw/test_rgw_reshard.py
	src/common/options/rgw.yaml.in
	src/rgw/driver/rados/rgw_rados.cc
	src/rgw/driver/rados/rgw_rados.h

Cherry-pick notes:
- conflicts due to options definitions being different after Pacific
- conflicts due to rgw_rados.h/cc moving directories