mgr/dashboard: fix grafonnet build error

This PR tends to fix the issue caused by #42194

Fixes:https://tracker.ceph.com/issues/52238
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 4907c78bb732b613ae223d3a7d1a046da457d1cc)

Conflicts:
ceph.spec.in (added jsonnet package)
monitoring/grafana/dashboards/CMakeLists.txt (added grafana jsons code)
(cherry picked from commit 570160e291b9029c7998d4dd4fee33327da91312)

mgr/dashboard: monitoring: replace Grafana JSON with Grafonnet based Code

This PR intends to add grafonnet to generate grafana JSON files

Fixes: https://tracker.ceph.com/issues/45184
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit e9bd94515f865b6c8865210b6329c11f3ebe9d23)

Conflicts:
debian/control (added jsonnet package)
run-make-check.sh (switched ON the WITH_GRAFANA bool for make check)

Merge pull request #42659 from cfsnyder/wip-50386-pacific

pacific: SimpleRADOSStriper: use debug_cephsqlite

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42693 from tchaikov/pacific-pr-41215

pacific: cmake: Replace boost download url

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #42692 from tchaikov/pacific-pr-42681

pacific: cmake: link bundled fmt statically

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #42561 from smithfarm/wip-51969-pacific

pacific: rpm/luarocks: simplify conditional and support Leap 15.3

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #42544 from cybozu/pacific-kv-rocksdbstore-enrich-debug-message

pacific: kv/RocksDBStore: enrich debug message

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #42542 from smithfarm/wip-51955-pacific

pacific: compression/snappy: use uint32_t to be compatible with 1.1.9

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42224 from cfsnyder/wip-51568-pacific

pacific: mon/OSDMonitor: resize oversized Lec::epoch_by_pg, after PG merging, preventing osdmap trimming

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42536 from batrick/i51940

pacific: MDSMonitor: monitor crash after upgrade from ceph 15.2.13 to 16.2.4

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42773 from neha-ojha/wip-52244-pacific

pacific: os/bluestore: make deferred writes less aggressive for large writes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #42135 from dang/wip-dang-41991-backport

pacific: RGW - Bucket Remove Op: Pass in user

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #40696 from singuliere/wip-50156-pacific

pacific: rgw/notification: fix persistent notification hang when ack-levl=none

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #40695 from singuliere/wip-50157-pacific

pacific: rgw/notification: fixing the "persistent=false" flag

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #40693 from singuliere/wip-50214-pacific

pacific: rgw: objectlock: improve client error messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>

test/store_test: more testing for deferred writes

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 324ea6b425e5898eca6e86264fa4d34cb868a6ae)

os/bluestore: enforce one more non-inclusive comparision against prefer_deferred_size

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit cb30c9965854e7dbfe851ed0cfaf113b16bcab5e)

os/bluestore: Better handling of deferred write trigger

Now deferred write in _do_alloc_write does not depend on blob size,
but on size of extent allocated on disk.
It is now possible to set bluestore_prefer_deferred_size way larger than
bluestore_max_blob_size and still get desired behavior.
Example: for deferred=256K, blob=64K : when op write is 128K both blobs will be
written as deferred. When op write is 256K then all will go as regular write.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 06ac561a72a93bd24d05b6c2266d66fee49228b8)

os/bluestore: account for alignment with max_blob_size when splitting
write I/O into chunks.

Without the fix the following write seq:
0~4M
4096~4M

produces tons of deferred writes at the second stage.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit b06bcf99f50cf7db5856eb1f6cd56100a0d0bbbc)

os/bluestore: introduce l_bluestore_write_deferred_bytes perf counter.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit f38f18c149b9347a26d845539eaeef29979e9006)

os/bluestore: use non-inclusive comparision against prefer_deferred_size

Fixes: https://tracker.ceph.com/issues/52089
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit f1c4448fc549ed58bb58f920dfa16886e4910315)

os/bluestore: improve logging around deferred writes

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 89a20d63ec7e255d36363bd8977c3fca1d277877)

os/bluestore: fix missing 'l_bluestore_write_deferred' perf counter increment

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit dc367c4bbc701b04a5bde562e30ab5b9c1705657)

os/bluestore: cosmetic cleanups

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 080531d6813b868eda562cf27babe18d8a974413)

Merge pull request #42679 from callithea/wip-51992-pacific

pacific: monitoring/grafana/cluster: use per-unit max and limit values

Merge pull request #42755 from guits/wip-50896-pacific

pacific: ceph-volume: Consider /dev/root as mounted

ceph-volume: consider /dev/root mounted

Fixes: https://tracker.ceph.com/issues/50604
Signed-off-by: David Caro <david@dcaro.es>
(cherry picked from commit c7b959c35ff463cfbffa2abf0f56a1c1a5f594c8)

Merge pull request #42493 from dsavineau/wip-51852-pacific

pacific: ceph-volume: lvm batch: fast_allocations(): avoid ZeroDivisionError

Merge pull request #42546 from dsavineau/wip-51958-pacific

pacific: ceph-volume/tests: retry when destroying osd

Merge pull request #42606 from s0nea/wip-dashboard-pacific-translations

mgr/dashboard: update translations for pacific

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #42298 from callithea/wip-51636-pacific

pacific: monitoring: fix Physical Device Latency unit

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: clwluvw <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #42642 from cfsnyder/wip-51696-pacific

pacific: rgw: fail as expected when set/delete-bucket-website attempted on a non-exis…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42566 from cfsnyder/wip-51514-pacific

pacific: rgw/notifications: support metadata filter in CompleteMultipartUploa…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41367 from cbodley/wip-50845

pacific: rgw: deprecate the civetweb frontend

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

cmake: Replace boost download url

Boost has moved downloads to JFrog Artifactory
https://www.boost.org/users/news/boost_has_moved_downloads_to_jfr.html

Signed-off-by: Rafał Wądołowski <rwadolowski@cloudferro.com>
(cherry picked from commit c2c6678e488f41277022eaf7a929f7ef845abd5f)

Conflicts:
make-dist: trivial resolution

cmake: link bundled fmt statically

instead of building the bundled fmt library as shared library, built
static library. otherwise we'd have to link against libfmt.so.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ebbf7f0665487b9e0507f1550cd469e56895addc)

Merge pull request #42637 from cfsnyder/wip-51779-pacific

pacific: rgw : add check for tenant provided in RGWCreateRole

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42635 from cfsnyder/wip-50645-pacific

pacific: rgw: allow rgw-orphan-list to process multiple data pools

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42634 from cfsnyder/wip-51014-pacific

pacific: rgw: remove quota soft threshold

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42633 from cfsnyder/wip-50679-pacific

pacific: rgw: fix segfault related to explicit object manifest handling

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42631 from cfsnyder/wip-50465-pacific

pacific: rgw/notifications: delete bucket notification object when empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42626 from cfsnyder/wip-52007-pacific

pacific: rgw: avoid occuring radosgw daemon crash when access a conditionally …

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42625 from cfsnyder/wip-51751-pacific

pacific: RGW Zipper - Make sure bucket list progresses

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42638 from cfsnyder/wip-50708-pacific

pacific: rgw: fix bucket object listing when marker matches prefix

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42649 from cfsnyder/wip-50379-pacific

pacific: rgw/amqp/test: fix mock prototype for librabbitmq-0.11.0

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42647 from cfsnyder/wip-49555-pacific

pacific: rgw/notification: add exception handling for persistent notification thread

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42646 from cfsnyder/wip-50463-pacific

pacific: rgw/multisite: return correct error code when op fails

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42645 from cfsnyder/wip-52051-pacific

pacific: rgw: when deleted obj removed in versioned bucket, extra del-marker added

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42644 from cfsnyder/wip-51804-pacific

pacific: rgw/http/notifications: support content type in HTTP POST messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42641 from cfsnyder/wip-50367-pacific

pacific: rgw: during reshard lock contention, adjust logging

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42640 from cfsnyder/wip-51801-pacific

pacific: rgw: radosgw-admin errors if marker not specified on data/mdlog trim

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #42639 from cfsnyder/wip-51781-pacific

pacific: rgw : modfiy error XML for deleterole

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42656 from cfsnyder/wip-51785-pacific

pacific: rgw multisite: metadata sync treats all errors as 'transient' for retry

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42655 from cfsnyder/wip-50803-pacific

pacific: radosgw-admin: skip GC init on read-only admin ops

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42654 from cfsnyder/wip-50731-pacific

pacific: rgw/rgw_file: Fix the return value of read() and readlink()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42653 from cfsnyder/wip-50728-pacific

pacific: rgw : add check empty for sync url

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42652 from cfsnyder/wip-50711-pacific

pacific: rgw: fix for mfa resync crash when supplied with only one totp_pin.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42651 from cfsnyder/wip-51771-pacific

pacific: test/rgw: fix use of poll() with timers in unittest_rgw_dmclock_scheduler

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42650 from cfsnyder/wip-50642-pacific

pacific: rgw/sts: read_obj_policy() consults iam_user_policies on ENOENT

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42648 from cfsnyder/wip-50094-pacific

pacific: librgw/notifications: initialize kafka and amqp

Reviewed-by: Casey Bodley <cbodley@redhat.com>

monitoring/grafana/cluster: use per-unit max and limit values

The value we get is a perunit, so the limits and the max value should
be over 1, not 100. Note that the value being shown was correct, it
was the gauge that was not showing the correct indicators.

Signed-off-by: David Caro <david@dcaro.es>
(cherry picked from commit c98129803935e720bb068fcf3700812c550f61b4)

SimpleRADOSStriper: use debug_cephsqlite

Instead of debug_client (which is primarily used by ceph-fuse).

Fixes: https://tracker.ceph.com/issues/50307
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 484fbb51dfe1f930653dcbd5d764543fe9ad9cda)

qa/rgw: run multisite tests with metadata sync error injection

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cff6bf37e65964d2d987231598659f08a84b9410)

rgw: limit concurrency of metadata sync

limit the number of concurrent RGWMetaSyncSingleEntryCRs that each mdlog
shard is allowed to spawn. use META_SYNC_SPAWN_WINDOW=20 to match data-
and bucket sync

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a07add0a5408f0480e93ea5920bc013119ba6401)

rgw: metadata sync treats all errors as 'transient'

collect_children() had a special case for EAGAIN that it treated as
a 'transient' error, which set the can_adjust_marker = false to bail out
of RGWMetaSyncShardCR and retry from the previous marker

but the http client doesn't return EAGAIN - rgw_http_error_to_errno()
defaults to EIO - so this retry logic based on can_adjust_marker never
runs. on any other error, RGWMetaSyncSingleEntryCR would not call
marker_tracker->finish() to advance the sync status marker, and
RGWMetaSyncShardCR would continue on with full- or incremental sync
without ever attempting to retry the failed entries

a detailed comment in collect_children() describes a different strategy
for handling 'permanent' errors, but that was never fully elaborated.
i also don't think there's a reasonable way to differentiate between
transient and permanent errors, so this treats all errors as transient
to be retried

if an error really is permanent for a given metadata key, metadata sync
will get stuck there and require manual intervention

Fixes: https://tracker.ceph.com/issues/39657
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 866d66b8749b28ec626a8d0adba3d14fdd8abead)

radosgw-admin: skip GC init on read-only admin ops

Fixes: https://tracker.ceph.com/issues/50520
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 9ac1991fc798af7e0ba0fac18209b71b5ae3f02b)

Conflicts:
src/rgw/rgw_admin.cc
src/rgw/rgw_rados.h
src/rgw/rgw_sal.cc
src/rgw/rgw_sal.h

Cherry-pick notes:
- Conflicts due to move from of RGWStoreManager in rgw_sal_rados.h to StoreManager in rgw_sal.h

rgw/rgw_file: Fix the return value of read() and readlink()

Fixes: https://tracker.ceph.com/issues/49189
Signed-off-by: Dai zhiwei <daizhiwei3@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
(cherry picked from commit bfd83e8fa142873a0bdf09a4d1ad1b04127f5885)

rgw : add check empty for sync url

Fixes: https://tracker.ceph.com/issues/50103
Signed-off-by: caolei <halei15848934852@163.com>
(cherry picked from commit 3a4e0b79310b21eeee37043d5419887bb41c0cf6)

rgw: fix for mfa resync crash when supplied with only one totp_pin.

The fix returns an appropriate error message.

fixes: https://tracker.ceph.com/issues/50394

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 1b943419c45a9b17be71668bd023aeb814171b2c)

test/rgw: use spawn library for test_rgw_dmclock_scheduler

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a8e3589a2c875b6fadc853c75f20cb9256f294ca)

test/rgw: fix use of poll() with timers in unittest_rgw_dmclock_scheduler

the AsyncScheduler uses an asio timer to dispatch work to its executor
with an optional delay. when no delay is requested, it waits on the
timer with an expiration time in the past (crimson::dmclock::TimeZero)

tests are failing here because poll() is returning without executing the
handlers of those expired timers

asio implements these timers with timerfd and epoll. debugging with
strace, i see that these timers armed with timerfd_settime() are not
always immediately ready according to epoll_wait():

  eventfd2(0, EFD_CLOEXEC|EFD_NONBLOCK)   = 3
  epoll_create1(EPOLL_CLOEXEC)            = 4
  timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC) = 5
  epoll_ctl(4, EPOLL_CTL_ADD, 3, {events=EPOLLIN|EPOLLERR|EPOLLET, data={u32=14164052, u64=14164052}}) = 0
  epoll_ctl(4, EPOLL_CTL_ADD, 5, {events=EPOLLIN|EPOLLERR, data={u32=14164064, u64=14164064}}) = 0
  timerfd_settime(5, TFD_TIMER_ABSTIME, {it_interval={tv_sec=0, tv_nsec=0}, it_value={tv_sec=0, tv_nsec=1}}, {it_interval={tv_sec=0, tv_nsec=0}, it_value={tv_sec=0, tv_nsec=0}}) = 0
  epoll_wait(4, [{events=EPOLLIN, data={u32=14164052, u64=14164052}}], 128, 0) = 1
  epoll_wait(4, [], 128, 0)               = 0
  epoll_wait(4, [], 128, 0)               = 0
  epoll_wait(4, [], 128, 0)               = 0
  epoll_wait(4, [], 128, 0)               = 0
  epoll_wait(4, [{events=EPOLLIN, data={u32=14164064, u64=14164064}}], 128, 0) = 1

in this example, it took 6 calls to context.poll() before it was ready
to execute the timer's handler

to work around this, replace calls to context.poll() with calls to
context.run_for() with a very short duration

Fixes: https://tracker.ceph.com/issues/42788
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 21baed999e31c5e69c75f0cbb8757ef91585d917)

rgw: read_obj_policy() consults iam_user_policies on ENOENT

when the head object doesn't exist, read_obj_policy() has to decide
whether to return ENOENT or EACCES

when there's a bucket policy, we check whether it has s3ListBucket
permissions. when there's an assumed role, we also need to check
against the role's policies in s->iam_user_policies

Fixes: https://tracker.ceph.com/issues/49780
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5dc9375fa1888242f388f8b502f445f3ddc891f7)

rgw/amqp/test: fix mock prototype for librabbitmq-0.11.0

also use extern C for to get compilation errors when
function prototype change

Fixes: https://tracker.ceph.com/issues/50291
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 2ba598ec4c294bd09d2df18ccd2096382e303d39)

librgw/notifications: initialize kafka and amqp

Fixes: https://tracker.ceph.com/issues/49738
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 178f6bdac97b57300bbe0956633cf686a7e3ccee)

rgw/notification: add exception handling for persistent notification thread

Fixes: https://tracker.ceph.com/issues/49322
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 915963ecb9effcb1f2d38f444c1bb9307f8ffbe1)

Conflicts:
src/rgw/rgw_notify.cc

rgw/multisite: return correct error code when op fails

when trying to disable/enbale sync on non-master zone

Fixes: https://tracker.ceph.com/issues/50201
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 83e89dfa3358fe91597d6714483f96b21a234ae6)

rgw: when deleted obj removed in versioned bucket, extra del-marker added

After initial checks are complete, this will read the OLH earlier than
previously to check the delete-marker flag and under the bug's
conditions will return -ENOENT rather than create a spurious delete
marker.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 69d7589fb1305b7d202ffd126c3c835e7cd0dda3)

rgw/http/notifications: support content type in HTTP POST messages

Fixes: https://tracker.ceph.com/issues/51530
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 6a1688b57d1e329cecd0c26c494cb08c9a4c3079)

Merge pull request #42629 from rhcs-dashboard/wip-52049-pacific

pacific: mgr/dashboard: show perf. counters for rgw svc. on Cluster > Hosts

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #42628 from rhcs-dashboard/wip-52048-pacific

pacific: mgr/dashboard: fix ssl cert validation for rgw service creation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

rgw: fail as expected when set/delete-bucket-website attempted on a non-existent bucket, rgw should return HTTP 404 and NoSuchBucket.

Fixes: https://tracker.ceph.com/issues/51536
Signed-off-by: xiangrui meng <mengxr@chinatelecom.cn>
(cherry picked from commit c623aa45d35b269c6701a57e44ac05bb29a79dc8)

rgw: during reshard lock contention, adjust logging

When RGW fails to get a lock on a reshard log, we log it in such a way
that it looks like an error. Instead we'll make sure that the log
message is informational.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 6d3dee37791ad427a3435c493a1d7874ba075674)

rgw: remove quota soft threshold

Remove quota soft threshold, which causes expensive checks for sharded buckets

Fixes: 14eabd4aa7b8a2e2c0c43fe7f877ed2171277526
Signed-off-by: Zulai Wang <wangzl31@outlook.com>
(cherry picked from commit 32a39705765af0f87bec9101e5d337b797e05fea)

Conflicts:
src/common/options/rgw.yaml.in

Cherry-pick notes:
- Options defined in src/common/options.cc in Pacific vs src/common/options/rgw.yaml.in

rgw: radosgw-admin errors if marker not specified on data/mdlog trim

Check that a marker was specified and trim if we don't have one.

Also: In a world where we're parsing for generation, it doesn't really
make sense to have a 'no marker specified' as separate from a marker
that is just an empty string.

Also: Successful datalog trim returns zero, not
-ENODATA, and radosgw-admin should expect this.

Fixes: https://tracker.ceph.com/issues/51712
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 4cb6fcf7f4e9548a8a0dd017b49a6ea23bedffd6)

Conflicts:
src/rgw/rgw_admin.cc

Cherry-pick notes:
- static_cast for RadosStore not needed in Pacific

rgw : modfiy error XML for deleterole

Fixes: https://tracker.ceph.com/issues/51157
Signed-off-by: caolei <halei15848934852@163.com>
(cherry picked from commit c7ab6579c7655352d08a4c12fc3a6951217dbe6f)

rgw: fix bucket object listing when marker matches prefix

When an iniitial marker that ends with a delimiter is provided, it
prevents listing of that "subdirectory" due to new logic at the cls
level to make listing more efficient. The fix catches that situation.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 220ef4b22d1d1667eb4f2c300a0b788e87b9067d)

rgw : add check for tenant provided in RGWCreateRole

Fixes: https://tracker.ceph.com/issues/51206
Signed-off-by: caolei <halei15848934852@163.com>
(cherry picked from commit 3c99ac14080c9f5b1611c9bbe4a223a9fd2927a0)

Conflicts:
src/rgw/rgw_rest_role.cc

Cherry-pick notes:
- Pacific constructs role explicitly vs. using store->get_role(), and does not wrap in a unique_ptr

rgw: allow rgw-orphan-list to process multiple data pools

We now allow multiple data pools to be specified either on the
command-line or at the prompt. The list should be space-separated.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit bde1278a5de3e9363c0f136b6ebc913bbb5e269d)

rgw: fix segfault related to explicit object manifest handling

Fixes: https://tracker.ceph.com/issues/50467
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit ae00453515b17d25a5d42430e744855b020ccec6)

rgw/notifications: delete bucket notification object when empty

Fixes: https://tracker.ceph.com/issues/49650
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 08e9846fd5a040887e2ddbfca36da97777fc76f6)

Conflicts:
src/rgw/rgw_pubsub.cc

Cherry-pick notes:
- RGWPubSub methods take a DoutPrefixProvider as first argument in Pacific

mgr/dashboard: show perf. counters for rgw svc. on Cluster > Hosts

- Use service_map_id to retrieve perf. counters for rgw services.
- Move logic from controller to CephService.

Fixes: https://tracker.ceph.com/issues/52022
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 69a87a1dea6fe148f2a81b1d7775f5ade9626c09)

mgr/dashboard: Cluster > Hosts > Performance Counters: do not show context bar.

Do not show the context bar (Object Gateway selection).

Fixes: https://tracker.ceph.com/issues/52022
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 73acb9b60d23e79bdcdd03437b0a813487420e33)

mgr/dashboard: fix ssl cert validation for rgw service creation

Fixes: https://tracker.ceph.com/issues/51965
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 5f52c00e5291b53b4fe9ab7f9fae5d499017f91b)

rgw: avoid occuring radosgw daemon crash when access a conditionally redirected static website

Fixes: https://tracker.ceph.com/issues/51715
Signed-off-by: xiangrui meng <mengxr@chinatelecom.cn>
Signed-off-by: yupeng chen <chenyupeng@chinatelecom.cn>
(cherry picked from commit 865d13e9fa043954dce5110f4f8ae693af9604ce)

RGW Zipper - Make sure bucket list progresses

The bucket list wrapper lost the automatic advancement of the marker for
each iteration.  Some places manually set that marker, and it worked
fine, but some depended on it being automatically set.  Fix it so that
the marker advances automatically each iteration.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
(cherry picked from commit 9cd74053b8bd95d3dee6cac6723cb051768865c3)

Merge pull request #42585 from rhcs-dashboard/wip-51994-pacific

pacific: mgr/dashboard: cephadm-e2e job script: improvements

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: update translations for pacific

Note: it's not a backport because the change is not intended to land in
master. It's pacific related only.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: cephadm-e2e script: improvements

Improvements and some adaptations related to the jenkins job.

Fixes: https://tracker.ceph.com/issues/51612
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 65b75000b7694cb3cbe617bbec28c513a2522be8)

 rgw/notifications: support metadata filter in CompleteMultipartUpload events

Fixes: https://tracker.ceph.com/issues/51261
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 93b9f0fb77ca5ed0b5c89d45229732850e0a0c49)

Conflicts:
src/rgw/rgw_notify.h
src/rgw/rgw_op.cc
src/rgw/rgw_sal.h
src/rgw/rgw_sal_rados.cc
src/rgw/rgw_sal_rados.h
src/test/rgw/bucket_notification/test_bn.py

Cherry-pick notes:
- differences due to renaming of rgw::sal::RGWObject to rgw::sal::Object
- differences due to use of RadosNotification in master and reservation_t in Pacific

Merge pull request #42480 from smithfarm/wip-51836-pacific

pacific: rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>