log: fix timestap precision of log can't set to millisecond.

The option log_coarse_timestamps can be set to Log::clock
successfully,but the Log::clock has no effect on time accuracy
because the dout_impl really use is Entry::clock.So we should
set Entry::clock by log_coarse_timestamps option instead of
Log:clock.In addition, i think the Log::clock can be removed
because i didn't see what it was for.

Fixes: https://tracker.ceph.com/issues/44409
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
(cherry picked from commit a5cceeac48066dc6b0b4a21133cfdcb920d159e8)

Merge pull request #37364 from wjwithagen/wjw-fix-lazyio.cc

nautilus: test: Do not use <sys/xattr.h> under non-Linux systems

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #37232 from Vicente-Cheng/wip-47259-nautilus

nautilus: client: handle readdir reply without Fs cap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37229 from Vicente-Cheng/wip-47252-nautilus

nautilus: cephfs: osdc/Journaler: do not call onsafe->complete() if onsafe is 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37228 from Vicente-Cheng/wip-47246-nautilus

nautilus: qa: ignore expected mds failover message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37178 from batrick/i47088

nautilus: mds: recover files after normal session close

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #37301 from alvistack/nautilus-aio_graceful_reboot

nautilus: systemd: Support Graceful Reboot for AIO Node

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37558 from smithfarm/wip-47737-nautilus

nautilus: mgr/status: metadata is fetched async

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #37481 from smithfarm/wip-47605-nautilus

nautilus: mds: fix purge_queue's _calculate_ops is inaccurate

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #37550 from rhcs-dashboard/wip-47753-nautilus

nautilus: mgr/dashboard: Don't use any xlf file when building the default language

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>

Merge pull request #37505 from rhcs-dashboard/fix-46815-nautilus

nautilus: mgr/dashboard: fix broken backporting

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #37479 from smithfarm/wip-46960-nautilus

nautilus: cephfs-journal-tool: fix incorrect read_offset when finding missing objects

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #37231 from Vicente-Cheng/wip-47254-nautilus

nautilus: cephfs: client: make Client::open() pass proper cap mask to path_walk

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #37179 from batrick/i47090

nautilus: mds: After restarting an mds, its standy-replay mds remained in the "resolve" state

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36965 from Vicente-Cheng/wip-46784-nautilus

nautilus: mds/CInode: Optimize only pinned by subtrees check

Reviewed-by: Ramana Raja <rraja@redhat.com>

mgr/mgr_module: metadata is fetched async

fetch metadata using a defaultdict

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit f06e60b591d7921b0afdd40219f3866773b00683)

Conflicts:
src/pybind/mgr/status/module.py

mgr/status: metadata is fetched async

fix timing window during mds rejoin where metadata is not available

Fixes: https://tracker.ceph.com/issues/45633
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4006fff43856c851e7a07f7d8ff05364cbe03529)

Conflicts:
src/pybind/mgr/status/module.py

Merge pull request #37510 from rhcs-dashboard/wip-47717-nautilus

nautilus: mgr/dashboard: Fix pool renaming functionality

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #37241 from rhcs-dashboard/wip-47538-nautilus

nautilus: mgr/dashboard: Disabling the form inputs for the read_only modals

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #37177 from batrick/i47017

nautilus: mds: kcephfs parse dirfrag's ndist is always 0

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #37035 from smithfarm/wip-47317-nautilus

nautilus: mds: fix incorrect check for if dirfrag is being fragmented

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36967 from Vicente-Cheng/wip-46941-nautilus

nautilus: mds: place MDSGatherBuilder on the stack

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36966 from Vicente-Cheng/wip-46787-nautilus

nautilus: cephfs: client: fix extra open ref decrease

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36814 from smithfarm/wip-47081-nautilus

nautilus: messages,mds: Fix decoding of enum types on big-endian systems

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36168 from smithfarm/wip-46151-nautilus

nautilus: mds: do not defer incoming mgrmap when mds is laggy

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36968 from Vicente-Cheng/wip-46943-nautilus

nautilus: mds: resolve SIGSEGV in waiting for uncommitted fragments

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36963 from Vicente-Cheng/wip-46633-nautilus

nautilus: mds: fix mds forwarding request 'no_available_op_found'

Reviewed-by: Ramana Raja <rraja@redhat.com>

mds/CInode: Optimize only pinned by subtrees check

Fixes: https://tracker.ceph.com/issues/46727
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 6a865f805b723525358ed8e79bb3d41709e98bf3)

Conflicts:
src/mds/MDCache.cc
  - use the nautilus code flow of trim instead master

mgr/dashboard: Don't use any xlf file when building the default language

The build-i18n script no longer uses a xlf file when building the default
language. This means that we don't need to keep the messages.en-US.xlf in the repository anymore.

Fixes: https://tracker.ceph.com/issues/42693
Signed-off-by: Sebastian Krah <skrah@suse.com>
(cherry picked from commit 5efe0a3ab25e26421001033bd7ea36c86aca2b02)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/locale/messages.en-US.xlf
    - Deleted this file

Merge pull request #37435 from Vicente-Cheng/wip-47413-nautilus

nautilus: rgw: urlencode bucket name when forwarding request

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Fix pool renaming functionality

The problem was that, if a pool was renamed the routine to update tried
to work on the old pool name and didn't used the info that the pool got
renamed, therefore an error occurred in the UI saying that the pool
could not be found.

Fixes: https://tracker.ceph.com/issues/42776
Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit de933cd3c82b6a7d5c4ddfa77804509c9fcaa3db)

Conflicts:
      qa/tasks/mgr/dashboard/test_pool.py:
      - Ignore incoming quota test

mgr/dashboard: Fix pg_num counts in pool API tests

To prevent the not a number of two cluster health warning, all created
pools now have a pg_num that is a power of two.

Fixes: https://tracker.ceph.com/issues/42776
Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit ca46c9941bc6837f09aab7b4aaab799d8a9ebf48)

Conflicts:
      qa/tasks/mgr/dashboard/test_pool.py:
      - Ignore quota tests and accept 8 PG number
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: Refactor pool API tests

Renamed "__create_pool" to "__yield_pool" to make it's purpose clearer
and added a description.

Split up create pool test into three different tests.
Used "__yield_pool" for creation tests, too.

Fixes: https://tracker.ceph.com/issues/42776
Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit 96fd06b7db1d57b8384edc1af6cc39e4e532fb77)

Conflicts:
      qa/tasks/mgr/dashboard/test_pool.py:
      - Ignore incoming pool quota tests
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #37413 from jan--f/wip-47650-nautilus

nautilus: [ceph-volume]: remove unneeded call to get_devices()

Merge pull request #37377 from shyukri/wip-47283-nautilus

nautilus: ceph-volume: fix journal size argument not work

Merge PR #37508 into nautilus

* refs/pull/37508/head:
nautilus: qa: stop using kclient testing branch builds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37478 from smithfarm/wip-47345-nautilus

nautilus: qa/*/mon/mon-last-epoch-clean.sh: mark osd out instead of down

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37477 from smithfarm/wip-47250-nautilus

nautilus: tools/osdmaptool.cc: add ability to clean_temps

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37476 from smithfarm/wip-46965-nautilus

nautilus: mgr: decrease pool stats if pg was removed

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37475 from smithfarm/wip-46935-nautilus

nautilus: tools/rados: Set locator key when exporting or importing a pool

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37474 from smithfarm/wip-46738-nautilus

nautilus: mon: fix the 'Error ERANGE' message when conf "osd_objectstore" is filestore

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37473 from smithfarm/wip-46710-nautilus

nautilus: osd/PeeringState: prevent peer's num_objects going negative

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37470 from smithfarm/wip-46262-nautilus

nautilus: common, osd: add sanity checks around osd_scrub_max_preemptions

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37471 from smithfarm/wip-46461-nautilus

nautilus: pybind/mgr/balancer: use "==" and "!=" for comparing str

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37447 from badone/wip-nautilus-ca-ansible-to-2.9

nautilus: qa/ceph-ansible: Bump required ansible to 2.9

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

nautilus: qa: stop using kclient testing branch builds

... in kcephfs and multimds suites.

This is a nautilus only fix. In master and octopus we still use
kclient testing branch builds.

Fixes: https://tracker.ceph.com/issues/47642
Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge pull request #37469 from smithfarm/wip-47459-nautilus

nautilus: qa/workunits/mon: fixed excessively large pool PG count

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #37468 from smithfarm/wip-47417-nautilus

nautilus: rbd: include RADOS namespace in krbd symlinks

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mgr/dashboard: fix broken backporting

A [previous backport](https://github.com/ceph/ceph/pull/35367) from a
[master PR](https://github.com/ceph/ceph/pull/32829) changed the testing
condition for PG status from 'incomplete' to 'unknown'.

Fixes: https://tracker.ceph.com/issues/46815
Co-authored-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #37467 from smithfarm/wip-47322-nautilus

nautilus: rgw: replace '+' with "%20" in canonical query string for s3 v4 auth.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37465 from smithfarm/wip-47318-nautilus

nautilus: rgw: Expiration days can't be zero and  transition days can be zero

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37464 from smithfarm/wip-47315-nautilus

nautilus: rgw: radosgw-admin: period pull command is not always a raw_storage_op

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37463 from smithfarm/wip-46956-nautilus

nautilus: rgw: fix shutdown crash in RGWAsyncReadMDLogEntries

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37462 from smithfarm/wip-46950-nautilus

nautilus: rgw/cls: preserve olh entry's name on last unlink

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37461 from smithfarm/wip-46930-nautilus

nautilus: rgw: Empty reqs_change_state queue before unregistered_reqs

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37460 from smithfarm/wip-46594-nautilus

nautilus: rgw: add negative cache to the system object

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37459 from smithfarm/wip-47320-nautilus

nautilus: rgw: RGWObjVersionTracker tracks version over increments

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37438 from Vicente-Cheng/wip-47347-nautilus

nautilus: rgw: Swift API anonymous access should 401

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37472 from smithfarm/wip-46587-nautilus

nautilus: doc/rados: Fix osd_scrub_during_recovery default value

Reviewed-by: Neha Ojha <nojha@redhat.com>

mds: fix purge_queue's _calculate_ops is inaccurate

No need to account for removing (or zeroing) backtrace,
because if item.size is 0, num is already set to 1.

Fixes: https://tracker.ceph.com/issues/47353
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
(cherry picked from commit 8a54e03f29e19af2072751fe2816b5313ff18abe)

mon: fix the 'Error ERANGE' message when conf "osd_objectstore" is filestore

Fixes: https://tracker.ceph.com/issues/37532
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
(cherry picked from commit 4155a79f76b177ada79af746de4448773e07584a)

Conflicts:
        src/mon/OSDMonitor.cc
- in nautilus, "cmd_getval()" needs cct as first argument

cephfs-journal-tool: fix wrong read_offset when get missing objects

Fixes: https://tracker.ceph.com/issues/45575
Signed-off-by: Xue Yantao <jhonxue@tencent.com>
(cherry picked from commit bfa63666bb40c7939aa4da3c2c8f43a7022a78e8)

qa/*/mon/mon-last-epoch-clean.sh: mark osd out instead of down

The test should mark the OSD out to check if only "in" OSDs are considered by
the osdmap trimming logic.

Fixes: https://tracker.ceph.com/issues/47309
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 21c08f0be2e048edd2d3ce7ca803f94a6d32f97c)

test/cli/osdmaptool/help.t: add clean-temps

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 762966a08ab31234148fe181eb2c107ae5266906)

doc: add clean-temps to osdmaptool.rst

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 78224c21d24d68d471eb266a37624d00d1483ed8)

tools/osdmaptool.cc: add ability to clean_temps

This is particularly useful for debugging purposes when clean_temps()
takes abnormally high amount of time due to flaws in crush rules etc.

Fixes: https://tracker.ceph.com/issues/47159
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ab54d3821a61a4ff1ea9973c5f31ee86868b3009)

radosgw-admin: period pull command is not always a raw_storage_op

if a --url is given, 'period pull' does not depend on any zone/period
configuration and can be a raw_storage_op. if we get a --remote instead,
we do need to initialize the zone/period configuration to find the
correct endpoint/access keys

Fixes: https://tracker.ceph.com/issues/47217
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2b44a9d060d33dca9768c758e1908365488aac2a)

Conflicts:
src/rgw/rgw_admin.cc
- nautilus has different (but unrelatedly so) raw_storage_ops_list

mgr: decrease pool stats if pg was removed

After merge of placement groups resulting pg contains
objects from itself and merged one.
PGMap::apply_incremental treat this growth as pool stats delta,
but forget to decrease stats for removed pg.

Fixes: https://tracker.ceph.com/issues/44815
Signed-off-by: Aleksei Gutikov <aleksey.gutikov@synesis.ru>
(cherry picked from commit 6090acdae4495e11f117df2330b579744eeada2a)

tools/rados: Set locator key when exporting or importing a pool

Fixes the following error when exporting a pool that contains objects
with a locator key set:

        error getting xattr set [object name]: (2) No such file or directory
        error from export: (2) No such file or directory

Fixes: https://tracker.ceph.com/issues/46824
Signed-off-by: Iain Buclaw <iain.buclaw@dunnhumby.com>
(cherry picked from commit ecb2df9177f30213b3ff48e76e6fe31fcbb430a7)

osd/PeeringState: prevent peer's num_objects going negative

Saw it in a teuthology run:

 -5645> 2020-07-20 04:34:32.067 7f351e329700  5 osd.5 pg_epoch: 667 ... exit Started/Primary/Active/Backfilling
 -5642> 2020-07-20 04:34:32.067 7f351e329700  5 osd.5 pg_epoch: 667 ... enter Started/Primary/Active/Recovered
 -5633> 2020-07-20 04:34:32.067 7f351e329700 20 osd.5 pg_epoch: 667 ... _update_calc_stats shard 5 primary objects 0 missing 0
 -5632> 2020-07-20 04:34:32.067 7f351e329700 20 osd.5 pg_epoch: 667 ... _update_calc_stats shard 3 objects -1 missing 1
 -5631> 2020-07-20 04:34:32.067 7f351e329700 20 osd.5 pg_epoch: 667 ... _update_calc_stats shard 6 objects 0 missing 0

This will crash the choose_acting() procedure as it will mistakenly
think that peer 3 should continue to perform asynchronous recovery
(e.g., due to num_objects_missing = 1) in contrast to fully
backfill-recovered.

While I did not dig into the real cause, there are a couple of
possible explanations of how num_objects can be off. I think that
if a roll forward or log replay could delete something twice, maybe
there would be an undercount. Or maybe something as simple as a
corruption.

Since _update_calc_stats() is going to fix num_objects_missing
for that peer anyway, let's make sure it always starts with a
clean state.

Fixes: https://tracker.ceph.com/issues/46705
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 10eff2567971ca57b1e821f704de490add021c8e)

Conflicts:
src/osd/PeeringState.cc
- file does not exist in nautilus; made the change manually in
  src/osd/PG.cc instead

doc/rados: Fix osd_scrub_during_recovery default value

Since 8dca17c, `osd_scrub_during_recovery` defaults to `false`, but the
documentation was still stating that its default value is `true`.

Fixes: https://tracker.ceph.com/issues/46531
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 535b103d1848f8b5322af0815e1bf163267d7f2a)

pybind/mgr/balancer: use "==" and "!=" for comparing str

we cannot assume that two values with the same value share the same
identity in Python.

also silences warnings like:

balancer/module.py:473: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if pool_ids is '':

Fixes: https://tracker.ceph.com/issues/46406
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 65dc464977b4c2ea5ab19b8d9d0904ab31f08b94)

common, osd: add sanity checks around osd_scrub_max_preemptions

to limit maximum preempt_divisor we can use when backing off the
chunky-scrub range on preempting.

Otherwise large osd_scrub_max_preemptions values (i.e., >= 32)
would cause preempt_divisor overflow, hence the dreaded
“divide by zero error”.

Fixes: https://tracker.ceph.com/issues/46024
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit ae05de3e9b2e9868216e5168e50dfcb5074684cb)

Conflicts:
src/osd/PG.cc
- git got confused about where the change was/is

qa/workunits/mon: fixed excessively large pool PG count

Fixes: https://tracker.ceph.com/issues/47405
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2fa9442dcc7a0448ab7e3588f82f93ca2e55d686)

qa: add test for krbd symlinks created by udev

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7ccd2c0dcee175e4c5a03985f43e9259a7e4dbd4)

rbd: include RADOS namespace in krbd symlinks

Fixes: https://tracker.ceph.com/issues/40247
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0b5c11ff30dbb79690e47d5285f197f677e11bf7)

rgw: replace '+' with "%20" in canonical query string for s3
 v4 auth.

fix https://tracker.ceph.com/issues/45983

Signed-off-by: yuliyang_yewu <yuliyang_yewu@cmss.chinamobile.com>
(cherry picked from commit 9002be34aa8524816708db4f3429bfe8634b776a)

rgw: transitio days can be zero in transition check

Signed-off-by: zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit 191fc25a97cb748f1105a463e9772194ba724d97)

rgw: Expiration days can't be zero and  transition days can be zero

Signed-off-by: zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit b471fd07aa7d9ca7868572688baf89ca5a295e6f)

rgw: fix shutdown crash in RGWAsyncReadMDLogEntries

RGWAsyncReadMDLogEntries must not store pointers into coroutine memory,
because it's not guaranteed to outlive our call. store these by-value
instead, and have RGWReadMDLogEntriesCR::request_complete() copy/move
them back on completion

Fixes: https://tracker.ceph.com/issues/45771
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 13bf06dbe961132ca99f470ac026674e45fecc38)

Conflicts:
src/rgw/rgw_sync.cc
- difference in RGWAsyncReadMDLogEntries() argument list: in nautilus,
  "rgw::sal::RGWRadosStore *_store" becomes just plain "RGWRados
  *_store"

cls/rgw: preserve olh entry's name on last unlink

When rgw_bucket_unlink_instance removes the last instance of a name, it
also clears the value of rgw_bucket_olh_entry.key. However, bucket index
resharding uses this key when choosing its shard placement, so an empty
key causes all of these olh entries to be misplaced in shard 0. After
reshard, all of the olh recovery/cleanup logic would be sent to the
correct shard, and these misplaced olh entries would never be cleaned
up.

Preserving the key's name on last unlink allows the olh entry to be
resharded correctly and cleaned up normally.

Fixes: https://tracker.ceph.com/issues/46456
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit acf8f3cad9f55e34c703fdaef684853a3fb3b369)

rgw: Empty reqs_change_state queue before unregistered_reqs

In RGWHTTPManager::manage_pending_request(), before unregistering
or unlinking the http requests, empty the reqs_change_state list
to avoid use after free.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit b210437e5f28d53e770dd6938ce3c4be443da055)

rgw: ObjectCache::put() clears stale objv

if an existing object is cached with an object version, but it's
mutated without updating that version number, clear the OBJV flag so
that later cache reads asking for an object version result in a miss and
re-read the version from the osd

Fixes: https://tracker.ceph.com/issues/47306
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cf531cdd5e655a033f47e04b7dda81435a90271d)

rgw: system object cache tracks version over increments

instead of checking write_version before the write (which doesn't take
cls_version_inc() into account), check read_version after apply_write()
has been called. only cache the result if we got a read_version != 0

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ad326ffc3fba865d8c426de4be0193172b7688b7)

Conflicts:
src/rgw/services/svc_sys_obj_cache.cc
- in nautilus, the functions do not take an optional_yield argument

rgw: RGWObjVersionTracker tracks read version over increments

when no write_version is given, cls_version_inc() is used to increment
the version so other writers can use cls_version_check() to detect races

however, apply_write() will clear its cached read_version, which means
that later writes can no longer use cls_version_check() to detect other
racing writers

in cases where cls_version_inc() is used AND we know the previous version,
we can increment the cached read_version and preserve the ability to use
cls_version_check(). we know the previous version if we provided a valid
read_version to cls_version_check() and it succeeded

Fixes: https://tracker.ceph.com/issues/46849
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 298d721c1d3d8d430bfbc9e0ef5db7b44cdbe017)

Conflicts:
src/rgw/rgw_rados.cc
- nautilus does not have "RGWObjState::RGWObjState()"

rgw: add negative cache to the system object

add negative cache to the system object

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Fixes: https://tracker.ceph.com/issues/45816
(cherry picked from commit 0900bd8cf90babd6fafcb398854a4ddb071a27ee)

Conflicts:
src/rgw/services/svc_sys_obj_cache.cc
- RGWSI_SysObj_Core::raw_stat() takes different arguments in nautilus

Merge pull request #37181 from callithea/wip-47411-nautilus

nautilus: mgr: don't update pending service map epoch on receiving map from mon

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

nautilus: qa/ceph-ansible: Bump required ansible to 2.9

https://github.com/ceph/ceph-ansible/commit/fd0b9491b60303a5d27f79226f30d51c438de2ff
requires us to move to 2.9.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #37379 from shyukri/wip-46939-nautilus

nautilus: qa/tasks/ragweed: always set ragweed_repo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #37280 from ceph/nautilus-bucket-list-perf

nautilus: mgr/dashboard: fix perf. issue when listing large amounts of buckets

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #36909 from ifed01/wip-ifed-fix-bluefs-sel-nautilus

nautilus: bluestore/bluefs: make accounting resiliant to unlock()

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #37378 from shyukri/wip-47244-nautilus

nautilus: rgw: Add bucket name to bucket stats error logging

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: Swift API anonymous access should 401

There was a previous patch to fix this but turns out that only fixed it
for the Swift V1 auth. And it actaully broke keystone because it didn't
take into account the idiosyncrasies of multi tenancy. Which resulted in
the incorect behaviour for keystone. Worse, because it didn't take
tenants properly into account keystone ACLs where broken.

This patch reworks, and simplifies the original patch to work for both
auths. It even extends the ThirdPartyAccountApplier to check for an ANON
user and properly scope it to a tenant.

Fixes: https://tracker.ceph.com/issues/46295
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 67081098dc2dddd80d52d5acd166e68954cae618)

Conflicts:
src/rgw/rgw_swift_auth.h
  - only need to modify the user related code to rgw_user construct

Merge pull request #36704 from ShyamsundarR/wip-47013-nautilus

nautilus: mon: store mon updates in ceph context for future MonMap instantiation

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

rgw : add URL encode toward bucket name in forward request

Signed-off-by: caolei <halei15848934852@163.com>
(cherry picked from commit 1e4ec14dd0df4e9053ef630c831c93d392508417)

Merge pull request #37306 from p-se/wip-47546-nautilus

nautilus: mgr/dashboard: Fix many-to-many issue in host-details Grafana dashboard

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: Custom Directive for verifying if user is read_only

This commit introduces two new Custom Directives: `FormInputDisableDirective` and `FormScopeDirective`.
By using the `cdFormScope` attribute of the directive we can customize the behaviour of the modal components.

Fixes: https://tracker.ceph.com/issues/43527
Signed-off-by: Nizamudeen <nia@redhat.com>
(cherry picked from commit f6e1ed657125e0f394e80da5dfeb51617304d091)

Conflicts:
      src/pybind/mgr/dashboard/frontend/src/app/shared/directives/directives.module.ts
 - Import the FormInputDisableDirective and FormScopeDirective to the directive.module imports and exports
      src/pybind/mgr/dashboard/frontend/src/app/shared/directives/form-input-disable.directive.spec.ts
 - Changed TestBed.inject(...) to TestBed.get(...) to match the syntax for the nautilus TS version
      src/pybind/mgr/dashboard/frontend/src/app/shared/directives/form-input-disable.directive.ts
 - Changed some of the operators to match the nautilus TS version

Merge pull request #37307 from rhcs-dashboard/wip-47303-nautilus

nautilus: mgr/dashboard: REST API returns 500 when no Content-Type is specified

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #37407 from idryomov/wip-krbd-read-only-override-nautilus

nautilus: rbd: make common options override krbd-specific options

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

msg/msg_types: entity_addrvec_t: fix decode on big-endian hosts

When decoding an entity_addrvec_t with marker 1, we just have
a single (non-legacy) entity_addr_t.  This should be decoded
exactly the same as done by entity_addr_t::decode, but it
currently is not.  Specifically, the sa_family member of
the sockaddr is not converted from the on-wire little-endian
format to host byte order (as done by entity_addr_t::decode).

Fixed by using the same code as in entity_addr_t::decode.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
(cherry picked from commit 31da17378b712542e915adbf4084e0212b8bb615)