Merge pull request #35199 from smithfarm/wip-45600-nautilus

nautilus: mds: don't shallow copy when decoding xattr map

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #35185 from smithfarm/wip-45497-nautilus

nautilus: cephfs/tools: add accounted_rstat/rstat when building file dentry

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #35163 from tchaikov/nautilus-upgrade-from-luminous-fs-disable-too-few-pgs

nautilus: qa/suites/fs/upgrade: disable mon_pg_warn_min_per_osd

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #35149 from Vicente-Cheng/wip-45602-nautilus

nautilus: mds: Handle blacklisted error in purge queue

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #35118 from Vicente-Cheng/wip-45478-nautilus

nautilus: client: only set MClientCaps::FLAG_SYNC when flushing dirty auth caps

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #35117 from Vicente-Cheng/wip-45474-nautilus

nautilus: bash_completion: Do not auto complete obsolete and hidden cmds

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #35214 from shyukri/wip-43990-nautilus

nautilus: mgr/insights: fix prune-health-history

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35213 from shyukri/wip-43464-nautilus

nautilus: mgr: close restful socket after exec

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35212 from shyukri/wip-43098-nautilus

nautilus: mgr/dashboard: Display the aggregated number of request

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35211 from shyukri/wip-45469-nautilus

nautilus: monitoring: Fix "10% OSDs down" alert description

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

Merge pull request #35174 from smithfarm/wip-45637-nautilus

nautilus: mgr/dashboard: install teuthology using pip

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #35183 from smithfarm/wip-45579-nautilus

nautilus: pybind/rbd: RBD.create() method's 'old_format' parameter now defaults to False

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #35182 from smithfarm/wip-45577-nautilus

nautilus: librbd: copy API should not inherit v1 image format by default

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35086 from tspmelo/wip-45540-nautilus

nautilus: mgr/dashboard: Fix HomeTest setup

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #35137 from shyukri/wip-45208-nautilus

nautilus: monitoring: alert for pool fill up broken

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/insights: make 'insights prune-health-history 0' zap current state

The self._health_slot value needs to be updated too if the current key
is removed.

Fixes: https://tracker.ceph.com/issues/43886
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 04f25b26c5aea5b4cf5cb94e939d6fba74228b27)

mgr: close restful socket after exec

Signed-off-by: liushi <liu.shi@navercorp.com>
(cherry picked from commit ec3a696a5d328936b611b2a19fc9ab12ae183b48)

mgr/dashboard: Display the aggregated number of request

convertTimeSeries will now calculate the aggregated total number of client
request made in the last seconds, instead of the number of request per second.

Fixes: https://tracker.ceph.com/issues/42622
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 1dc483e8d9699a57ef61ea77d4a54ce93b341b17)

monitoring: Fix "10% OSDs down" alert description

The alert was triggered when less than 90% of OSDs were _up_, but then the
description took that value and described it as the percentage of OSDs being
_down_. So with 12% of OSDs down, the alert description would read:

```
88% or 88 of 100 OSDs are down (>=10%).
```

which can be panic-inducing.

This commit changes the alert expression to actually compute the ratio of OSDs
being down, which makes the correct value appear in the description.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 653c3f66823179fc5b9cbb74ff932d61a6c4178c)

Merge pull request #35180 from smithfarm/wip-45642-nautilus

nautilus: tests: src/test/compressor: Add missing gtest

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #35047 from smithfarm/wip-44686-nautilus

nautilus: qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #35049 from smithfarm/wip-45224-nautilus

nautilus: ceph_test_rados_api_{watch_notify,misc}: tolerate some timeouts

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #35048 from smithfarm/wip-44689-nautilus

nautilus: test: Add flush_pg_stats to avoid race with getting num_shards_repaired

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #35028 from shyukri/wip-44163-nautilus

nautilus:test/fio: bump to fio 3.15

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #35024 from smithfarm/wip-43919-nautilus

nautilus: osd: dispatch_context and queue split finish on early bail-out

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34957 from smithfarm/wip-44841-nautilus

nautilus: osd/PeeringState: do not trim pg log past last_update_ondisk

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #34823 from votdev/wip-45323-nautilus

nautilus: mgr/dashboard: monitoring menu entry should indicate firing alerts

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35050 from smithfarm/wip-43773-nautilus

nautilus: qa: log warning on scrub error

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35000 from smithfarm/wip-45496-nautilus

nautilus: cephfs: Client: fix Finisher assert failure

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34784 from Vicente-Cheng/wip-45221-nautilus

nautilus: cephfs-journal-tool: correctly parse --dry_run argument

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34783 from Vicente-Cheng/wip-45217-nautilus

nautilus: qa/cephfs: update TestClusterFull:test_barrier

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34435 from Vicente-Cheng/wip-44483-nautilus

nautilus: mds: handle ceph_assert on blacklisting

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34338 from Vicente-Cheng/wip-44478-nautilus

nautilus: mds: Using begin() and empty() to iterate the xlist

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

mds: don't shallow copy when decoding xattr map

Otherwise inodes' xattr maps may reference a large shared buffer
(from omap fetch or journal read). If mds trims and loads inode
repeatly, each inode can reference different large buffer in the
worst case.

Fixes: https://tracker.ceph.com/issues/45090
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit deff94c8f43bb3734b688ccb828d942b8f150638)

Conflicts:
src/mds/CInode.cc
- CInode::decode_lock_ixattr is missing in nautilus
src/mds/mdstypes.h
- nautilus has a code comment that is not in master

qa/cephfs: scrub the subdir/ after data scan

This will make sure the rstat between subdir's inode and dirfrag
are matched.

Fixes: https://tracker.ceph.com/issues/44380
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 62e9820c3eb7311b886bed1a8c06da59bd7f301a)

qa/cephfs: remove the whole subdir/ after test done

Fixes: https://tracker.ceph.com/issues/44380
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit b24edff02e6e668046b0cab6709d0ccfaf5edbc5)

tools/cephfs: Disable dirfrag auto merging and splitting

Make sure that the dirfrag auto merging and splitting are disabled
explicitly.

Fixes: https://tracker.ceph.com/issues/44380
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 39df09e52b84b688f99f4e18190683c957d40783)

tools/cephfs: scan the links to set the proper dnfirst

The ceph code will set the dnfirst=2 as default instead of CEPH_NOSNAP.
And this will avoid the MDS daemon assert crash:

ceph_assert(in->first <= straydn->first)

Fixes: https://tracker.ceph.com/issues/44380
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit b8d34cdbdc02488c9c62193a507882ec6991362a)

mds: fix debug log typo

Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 6969d41e3d4cb2b460508cee044161de2a196ebe)

pybind/rbd: RBD.create() method's 'old_format' parameter now defaults to False

The RBD v1 format has been deprecated for numerous releases and creation of
v1 format images has been disabled since the Mimic release. This fixes
the Python API's image create() method to ensure v2 images are created by
default (and no longer throw an exception that creation of v1 images are
disabled).

Fixes: https://tracker.ceph.com/issues/45504
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 45349355f4b41c6f9de594ef34a8657230113e6b)

librbd: copy API should not inherit v1 image format by default

When copying from a v1 image, by default the new destination image
would be created using the v1 format. Since the creation of v1 images
is disallowed, this has been updated to default to using the v2
image format.

Fixes: https://tracker.ceph.com/issues/45518
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 69b6d5997e8c6a11739e4d5a32564e61eb2f470f)

Conflicts:
src/librbd/internal.cc
- in nautilus, the call immediately preceding the change is
  "src->snap_lock.put_read()" whereas in master it is
  "src->image_lock.unlock_shared()"

src/test/compressor: Add missing gtest

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit a11c72d6d6c72b6801cf58c202ea92f77af14ebf)

mgr/dashboard: require enum34 only for python_version<'3.4'

This is a partial backport of 1e07237d92da3bb5eba82b49f6276d17696e6985

Backporting the entire thing would be too intrusive.

Fixes: https://tracker.ceph.com/issues/45636
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35093 from neha-ojha/wip-too-few-pgs-nautilus

nautilus: qa/suites/upgrade: disable more min pg per osd warnings

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

vstart_runner: be python3 compatible

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a9906b8cd21ef2ffea7ba9298951542843433287)

mgr/dashboard: install teuthology using pip

* install teuthology using pip
* do not install the dependencies manually, it's error-prone
  to maintain two copies of requirements.txt

Fixes: https://tracker.ceph.com/issues/42969
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 68527ab83a77d20b779759581ce0b8844991231b)

Conflicts:
src/pybind/mgr/dashboard/run-backend-api-tests.sh
- 7567948ba05a78a5a61010994e43e8a78641970c is not being backported to
  nautilus

qa/suites/fs/upgrade: disable mon_pg_warn_min_per_osd

if we upgrade from luminous, then a luminous monitor won't have the
1ac34a5ea3d1aca299b02e574b295dd4bf6167f4 or its backport, but after
upgrading to nautilus, mgr detects this issue and translates it to
a warning like

2020-05-21T05:33:26.934 DEBUG:teuthology.misc:Ceph health: HEALTH_WARN
too few PGs per OSD (12 < min 30)

if the number of pg per osd is too low. so we need to disable this
warning as we did in later releases and in master.

this change is not cherry-picked from master, as we don't do
upgrade test from luminous to master.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/upgrade: disable more min pg per osd warnings

This follows 58eb3edc8478c993c5446475df58d659d3f6d356.

This change is not cherry-picked from master since it already has
1ac34a5ea3d1aca299b02e574b295dd4bf6167f4.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35004 from smithfarm/wip-44490-nautilus

nautilus: compressor/lz4: work around bug in liblz4 versions <1.8.2

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34983 from jecluis/wip-45402-nautilus

nautilus: mon/OSDMonitor: allow trimming maps even if osds are down

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34916 from sseshasa/wip-45391-nautilus

nautilus: mon/OSDMonitor: Always tune priority cache manager memory on all mons

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34882 from hillpd/wip-45359-nautilus

nautilus: rados: prevent ShardedOpWQ suicide_grace drop when waiting for work.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34829 from rhcs-dashboard/wip-45329-nautilus

nautilus: monitoring: fix decimal precision in Grafana %percentages

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34711 from smithfarm/wip-45126-nautilus

nautilus: os/bluestore: fix extent leak after main device expand.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #34692 from votdev/wip-44475-nautilus

nautilus: mgr/dashboard: Not able to restrict bucket creation for new user

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Filer: Add error handler in _do_purge_range()

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 39b25efb26963180a7e0403174868f5a9940ff55)

mds: Handle blacklisted error in purge queue

This patch adds check to catch blacklisted error from filer. On catching this
error, mds will respawn.

Fixes: https://tracker.ceph.com/issues/43598
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit c144d66afa74e8e45a2fc96bb650219d001538bf)

monitoring: alert for pool fill up broken

Fixes: https://tracker.ceph.com/issues/44991
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit e197e4d7f481395e2cef3e7acf61254906d5c32b)

Merge pull request #34794 from smithfarm/wip-45064-nautilus

nautilus: os/bluestore: fix 'unused' calculation

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34400 from votdev/wip-44203-nautilus

nautilus: mgr/dashboard: RGW port autodetection does not support "Beast" RGW frontend

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

client: only set MClientCaps::FLAG_SYNC when flushing dirty auth caps

Sidharth noticed that we were probably doing more FLAG_SYNC cap flushes
than necessary.

This flag is intended for sync/fsync type activity, and in those cases
all we care about is cleaning dirty caps. We have no need to expedite
other types of cap calls (releases, revokes, etc.).

Fixes: https://tracker.ceph.com/issues/44963
Reported-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 63e4ee0e9fe3654dcd680e7e8271870fbd295215)

bash_completion: Do not auto complete obsolete and hidden cmds

This patch fixes two things.

1. Do not auto complete obsolete and hidden cmds.
2. sub command completions often failed due to the
   use of associative arrays which does not keep the
   order. Hence used non-associative arrays.

Fixes: https://tracker.ceph.com/issues/45141
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 41082930f33fa43391986ac05394dbfff3b5dfd7)

Merge pull request #35045 from smithfarm/wip-45501-nautilus

nautilus: rgw: anonomous swift to obj that dont exist should 401

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35019 from smithfarm/wip-45517-nautilus

nautilus: rgw: fix multipart upload's error response

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34780 from Vicente-Cheng/wip-44370-nautilus

nautilus: msg/async/rdma: fix bug event center is blocked by rdma construct connection for transport ib sync msg

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34761 from smithfarm/wip-44674-nautilus

nautilus: mgr/balancer: tolerate pgs outside of target weight map

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #34516 from shyukri/wip-43920-nautilus

nautilus: common/bl: fix memory corruption in bufferlist::claim_append()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #34515 from shyukri/wip-43087-nautilus

nautilus: bluestore: prevent BlueFS::dirty_files from being leaked when syncing metadata

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #34475 from rhcs-dashboard/wip-44997-nautilus

nautilus: rpm: add python3-saml as install dependency

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35046 from smithfarm/wip-45365-nautilus

nautilus: qa/workunits/rbd: fix list-mapped filter in unmap_device

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

mgr/dashboard: Fix HomeTest setup

'setup_controllers' was being called outside of the fake fs mock,
so the tests would fail if there was no real dist folder.

Fixes: https://tracker.ceph.com/issues/45516
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit fb9a23663371b99fb7001c8f83b753c395ae7be9)

test/objectstore/test_bluestore_types: add tests for unused bitmap

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 985497227acbdd46fefc4de5aef75aaa9630514b)

os/bluestore: do not use 'unused' bitmap if makes no sense.

The processing logic which relies on 'unused' bitmap makes sense for
bluestore setup where min alloc size is different from device block
size. Now omitting if that's not true.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit f960a018f938c70ea3d7f031611cdc5a6b9955b1)

os/bluestore: fix unused 'tail' calculation.

Fixes: https://tracker.ceph.com/issues/41901
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit c91cc3a8d689995e8554c41c9b0f652d9a3458da)

Conflicts:
    src/test/objectstore/store_test.cc
- omitted test case "TEST_P(StoreTestSpecificAUSize, ReproBug41901Test)"
  from the backport, because nautilus does not have the
  "bluestore_debug_enforce_settings" option

Merge pull request #34986 from smithfarm/wip-45499-nautilus

nautilus: rgw: fix some list buckets handle leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35003 from smithfarm/wip-44651-nautilus

nautilus: qa/suites/rbd: removed OpenStack tempest test cases

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35002 from smithfarm/wip-45502-nautilus

nautilus: rgw: fix lc does not delete objects that do not have exactly the same tags as the rule

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35001 from smithfarm/wip-45486-nautilus

nautilus: rgw: radosgw-admin: fix infinite loops in 'datalog list'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34954 from smithfarm/wip-45436-nautilus

nautilus: rgw: dmclock: wait until the request is handled

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34815 from vumrao/wip-vumrao-45316

nautilus: radosgw-admin: add support for --bucket-id in bucket stats command

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33675 from ivancich/nautilus-tune-sharded-bucket-listing

nautilus: rgw: tune sharded bucket listing

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa: log warning on scrub error

Instead of printing the (useless) traceback, just print a warning about
ignoring the failure. The traceback makes it harder to search for the
real problem in the teuthology log.

Fixes: https://tracker.ceph.com/issues/43718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b7454e423620e829e7417cdfca1faf5cd91dec3f)

Conflicts:
qa/tasks/mon_thrash.py
- whereas master has "self.manager.raw_cluster_cmd('mon', 'scrub')" in
  the try block, in nautilus it is only "self.manager.raw_cluster_cmd('scrub')"

ceph_test_watch_notify: try notify 10x if ALLOW_TIMEOUTS is set

See https://tracker.ceph.com/issues/44062

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2183edcc65c76bff075f3650ea4ebc73edc55b16)

ceph_test_rados_api_misc: ShutdownRace timeout if ALLOW_TIMEOUTS is set

See https://tracker.ceph.com/issues/44582

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3a57e70028e6226a41ef295217160c038441cd70)

qa/suites/rados/verify: set ALLOW_TIMEOUTS for workunits

When running under valgrind (and thrashing) things can be slow.  Tell
tests in case they need to tolerate timeouts.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 40a7bcea706b72388056e254db752a44d1a59fcc)

qa/suites/rados/verify/validater/valgrind: tolerate SLOW_OPS

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7c19c1534badf00adf756e3a4e342e3d009a63ca)

qa/suites/rados/verify: whitelist MON_DOWN when using valgrind

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 52d706c75f2c70c80bfc4d8182217d5831bfbf4e)

qa/suites/rados/verify/validater/valgrind: less bluestore logging

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit baeb051910744bfd7a279776afc44c8f9aa6a6e6)

qa/suites/rados/verify/validater: increase heartbeat grace

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4fda9d50f09d527262fd65eab9b9cff3fd700aad)

Revert "qa/suites/rados/verify/validator/valgrind: debug refs = 5"

This reverts commit 65e81e6eb4f136bf21b67e5de10ab49f028f9e95.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
Conflicts:
    reverted the nautilus commit instead of cherry-picking the revert of
    the master commit

test: Add flush_pg_stats to avoid race with getting num_shards_repaired

Fixes: https://tracker.ceph.com/issues/44439
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e509b7c7d0d762c20927720b08d3a5b1f8980fb4)

qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change

Fixes: https://tracker.ceph.com/issues/44518
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6edd1cb68654702aa0698b50708514105ac01632)

qa/workunits/rbd: fix list-mapped filter in unmap_device

In recent versions `rbd list-mapped` does not print the white space
at the end of the line.

Fixes: https://tracker.ceph.com/issues/45305
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d667e385a12ea4125dbf74da97a9ba4911a3f7bc)

osd: dispatch_context and queue split finish on early bail-out

If we bail out of advance_pg early because there is an upcoming merge, we
still need to dispatch_context() on rctx before we drop the PG lock.  And
the rctx that we submit needs to include the on_applied finisher comit
to call _finish_splits.

This is noticeable (at least) when there is a split and merge that are
both known.  When we process the split, the new child is added to new_pgs.
When we get to the merge epoch, we stop early and take the bail-out
path.

Fix by adding a dispatch_context call for this path.  And further make sure
that both dispatch_context callers in this function queue up the
new_pgs event.

Fixes: https://tracker.ceph.com/issues/43825
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 14c4dbb03fe3c7fb4a0a6fa40dcb89e89fb66b3c)

Conflicts:
src/osd/OSD.cc
- where master says "rctx.transaction.register_on_applied", replaced
  with nautilus equivalent: "rctx->transaction->register_on_applied"
- where master says "dispatch_context(rctx, pg, pg->get_osdmap(),
  &handle)", replaced with nautilus equivalent:
  "dispatch_context_transaction(*rctx, pg, &handle)"

rgw: anonomous swift to obj that dont exist should 401

Currently, if you attempt to GET and object in the Swift API that
doesn't exist and you don't pass a `X-Auth-Token` it will 404 instead of
401.

This is actually a rather big problem as it means someone can leak data
out of the cluster, not object data itself, but if an object exists or
not.

This is caused by the SwiftAnonymousEngine's, frankly wide open
is_applicable acceptance. When we get to checking the bucket or object
for user acceptance we  deal with it properly, but if the object doesn't
exsit, because the user has been "authorised" rgw returns a 404.

Why? Because we always override the user with the Swift account.
Meaning as far as checks are concerned the auth user is the user, not
and anonymous user.

I assume this is because a swift container could have world readable
reads or writes and in slight s3 and swift api divergents can make these
interesting edge cases leak in.

This patch doesn't change the user to the swift account if they are
anonymous. So we can do some anonymous checks when it suits later in the
request processing path.

Fixes: https://tracker.ceph.com/issues/43617
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit b03d9754e113d24221f1ce0bac17556ab0017a8a)

Conflicts:
src/rgw/rgw_swift_auth.h
- where master has "rgw_user(s->account_name)", nautilus has
  "s->account_name" only

Merge pull request #34640 from rhcs-dashboard/wip-44327-nautilus

nautilus: mgr/dashboard: fix backport #33764

Merge pull request #34450 from rhcs-dashboard/wip-44980-nautilus

nautilus: monitoring: Fix pool capacity incorrect

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34847 from bk201/wip-45273-nautilus

nautilus: mgr/dashboard: fixing RBD purge error in backend

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #35016 from rhcs-dashboard/45515-fix-py2-strptime-error

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31586 from bk201/fix-42651

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>