osd: Only scan for omap corruption once

Before
  state 2: Can have complete tables (some may be bad)
state 3: Never had complete tables
After
  state 2: Can have complete tables (some may be bad)
state 3 with legacy: Can have complete tables (bad ones are cleared)
state 3: Never had complete tables

Once OSDs boot with this change you can't downgrade to a previous release.
If someone does downgrade they could have unstable OSDs that hit assert(state.v < 3).
The following command run after shutting down the cluster but before downgrading
ceph packages would be a way to fix this.

ceph-osdomap-tool --omap-path ... --command resetv2

Fixes: http://tracker.ceph.com/issues/21328
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 8805ef53424e30fd3f24ee38f5a6bdd9e6dd8641)

tools: Add --backend option to ceph-osdomap-tool default to rocksdb

Fix hard-coded "leveldb" backend.  The command is broken in Luminous
now that "rocksdb" is the default.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit de43493990923bcdd20f88e9d04647e5ba709feb)

osd, mds, tools: drop the invalid comment and some unused variables

Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>
(cherry picked from commit 2e93424167de15e91394169f4395f5f446e710e2)

tools: Add the ability to reset state to v2

Available for testing and user downgrade.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3e4d68640cc43efc0cf10ea8119b3aa583b7f36b)

tools: Show DB state information

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3214882a95f9f70d9f6d28f4e403ee16324530f9)

Merge pull request #17734 from liewegas/wip-21089-onmount

luminous: os/bluestore: replace 21089 repair with something online (instead of fsck)

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17814 from liewegas/wip-pg-limits-luminous

luminous: mon: cluster limit on pgs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/rados/operations/health-checks: fix TOO_MANY_PGS discussion

Fiddling with pgp_num doesn't help with TOO_MANY_PGS.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 027672b777402381f6736e517ed287b38bb17abb)

mon: rename mon_pg_warn_max_per_osd -> mon_max_pg_per_osd

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 986b86fbebf9e06f9f841da8ded0bedb310fc69b)

Merge pull request #17867 from smithfarm/wip-21443-luminous

luminous: mgr: Prometheus crash when update

Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #17868 from smithfarm/wip-21452-luminous

luminous: mgr: prometheus module generates invalid output when counter names contain non-alphanum characters

Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #17896 from liewegas/wip-bluestore-tool-luminous

ceph-objectstore-tool and ceph-bluestore-tool: backports from master

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #17930 from jcsp/wip-luminous-health-warn

mon: show legacy health warning in `status` output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18031 from xiexingguo/wip-pr-17987

luminous: osd/PrimaryLogPG: kick off recovery on backoffing a degraded object

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18033 from xiexingguo/wip-pr-17698

luminous: os/bluestore: allocate entire write in one go

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18038 from jecluis/wip-21300-luminous

luminous: mon/MgrMonitor: handle cmd descs to/from disk in the absence of active mgr

Merge pull request #17862 from smithfarm/wip-21307-luminous

luminous: mon: Client client.admin marked osd.2 out, after it was down for 1504627577 seconds

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17865 from smithfarm/wip-21465-luminous

luminous: OSD metadata 'backend_filestore_dev_node' is unknown even for simple deployment

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17936 from liewegas/wip-ruleset-errors-luminous

mon/OSDMonitor: error out if setting ruleset-* ec profile property

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17946 from liewegas/wip-mgr-localpool-luminous

luminous: mgr/localpool: module to automagically create localized pools

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17998 from liewegas/wip-pr-17978-luminous

rbdmap: fix umount when multiple mounts use the same RBD

Reviewed-by: Sage Weil <sage@redhat.com>

mon/MgrMonitor: read cmd descs if empty on update_from_paxos()

If the MgrMonitor's `command_descs` is empty, the monitor will not send
the mgr commands to clients on `get_descriptions`. This, in turn, has
the clients sending the commands to the monitors, which will have no
idea how to handle them.

Therefore, make sure to read the `command_descs` from disk if the vector
is empty.

Fixes: http://tracker.ceph.com/issues/21300
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 3d06079bae0fbc096d6c3639807d9be3597e841a)

mon/MgrMonitor: populate on-disk cmd descs if empty on upgrade

During kraken, when we first introduced the mgrs, we wouldn't populate
the on-disk command descriptions on create_initial(). Therefore, if we
are upgrading from a cluster that never had a mgr, we may end up
crashing because we have no cmd descs to load from disk.

Fixes: http://tracker.ceph.com/issues/21300
Signed-off-by: Joao Eduardo Luis <joao@suse.de>

os/bluestore: allocate entire write in one go

On the first pass through the writes, compress data and calculate a final
amount of space we need to allocate.  On the second pass, assign the
extents to blobs and queue the writes.

This allows us to do a single allocation for all blobs, which will lead
to less fragmentation and a much better write pattern.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e200f358499af8e3acb6ac4f675cc167433b53ec)

osd/PrimaryLogPG: kick off recovery on backoffing a degraded object

As we are now blocking frontend ops from accessing that very object!

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit db20328b456d70d6728fd27f17da6f2f3546e84b)

rbdmap: fix umount when multiple mounts use the same RBD

When a Kubernetes Pod consumes a RBD it is mounted two
times on the same host. When the host shutdown umount will
fail leading to a hung system

Signed-off-by: Alexandre Marangone <amarango@redhat.com>
(cherry picked from commit 40825daecedb2a3481021e4d36a367c339eb9b62)

Merge pull request #17953 from ceph/wip-ceph-ansible-bport

tests - ceph-ansible backports

use ovh instead of vps

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 56d1511bdfaee5a2a0d85bd606348632bfcbf09e)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

12.2.1

Merge pull request #17970 from idryomov/wip-pr-17346-luminous

luminous: qa/suites/krbd: rbd_xfstests job overhaul

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/suites/krbd: new rbd_xfstests job definition

ext4 seems to be a better choice for our purposes -- less test churn,
rather small and reliable exclude list.

All excluded tests but generic/050 fail with no krbd in the mix, most
have popped up on linux-ext4 list at least once.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 6647c65ebe0688283cc8b9da9f638a4d6ffb75de)

qa/tasks/rbd.xfstests: take exclude list from yaml

Different filesystems (and further, different configurations of the
same filesystem) need different exclude lists.  Hard coding the list in
a wrapper script is inflexible.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 01914be5a3b35479037054293b85d96596cb870b)

Conflicts:
qa/run_xfstests_krbd.sh [ trivial, caused by commit
  3aae5ca6fda8 ("scripts: fix bash path in shebangs") ]

qa/run_xfstests.sh: quit building xfstests on test nodes

xfstests is a pain to build on trusty, xenial and centos7 with a single
script.  It is also very sensitive to dependencies, which again need to
be managed on all those distros -- different sets of supported commands
and switches, some versions have known bugs, etc.

Download a pre-built, statically linked tarball and use it instead.
The tarball was generated using xfstests-bld by Ted Ts'o, with a number
of tweaks by myself (mostly concerning the build environment).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 1a982fea53c30fb246c2ab28822af6a69165ffea)

qa/run_xfstests.sh: drop *_MKFS_OPTIONS variables

AFAICT ./check doesn't query EXT4_MKFS_OPTIONS or BTRFS_MKFS_OPTIONS,
We don't need anything special for xfs, so remove all of them to avoid
confusion.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7a0e45046d556c6624e0225ff006c31d8dbae2a8)

qa/run_xfstests.sh: kill FS_TYPE variable

./check expects FSTYP, so use that throughout.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ca99f1e8406c14e5899d393ab0bb503061c47b9b)

doc/mgr: add "local pool" plugin to toc

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0b831e7886daad5836ca3cb58c01c123f99a1cde)

qa/tests: skip test that use public bucket landsat-pds from AWS

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit e11a1f773d8795dd9bd9819aaf229c6598e22ca6)

qa/tests: update required ceph-ansbile vars

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 184d5e6bad46451b3a8c5787f56148566bb42f20)

qa/tests: Fix ceph-ansible upstream vars

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 077904a0b5dffe9d64feade94cf30ffc92f1c0e5)

Fix get_system_type failure due to invalid remote name

recent changes caused the remote name to be invalid, fix the
arg passed to get_system_type

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 9fe58d5ed6bd2338271a9e003f51d80c2e440e23)

Stop the mgr on node which is not client, this will ensure
the client.0's mgr is active mgr for workunit to work.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit b1fd615e9f507d4243744b34887366086c1890ac)

Test ceph-mgr RESTful api

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit e5b5a1b8c8d0e750d29fe41a16252dcb0adf449f)

Add additional variables as required by ceph-ansible for upstream
installation.

possible revert if those are made default
issue: https://github.com/ceph/ceph-ansible/issues/1834

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit e963bfd6ca0ea1525e37dc35bf3309548c7a3831)

Rename folders to fix task order

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 9cc00c5c1a6e99323a4655aad5c093b8d87609f6)

use bluestore with dmcrypt option

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 1041c803f1f8f7be0edd0e78df352152fa3c4e6f)

Add dmcrypt option

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit f6de5d9f9e2a3666116311c146a8f2a45f3b8dbb)

Separate the main task from options

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 60d00e0eadda3a7a7cfbac2acdc81a4d55a27570)

Catchup with recent changes with ceph-ansible

Adds osd_scenario and ceph_stable_release variables

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 0395b844884a63a2cb7480dcc40762df6915b403)

Add workaround for http://tracker.ceph.com/issues/20950

mgr bootstrap key differs on disk, rewrite the new key
using auth get.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 5c43c5972ddb04724c8659a1b8d71cda1e0acdfb)

Add kraken to luminous upgrade case

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 71ea4987e0d55cc49e90dfbaf9722769cc892f3f)

Add kernel tar/untar and systemd task

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 526d4aef1cb73ffa6f73e1353dc06eed6cccb9f9)

Restart services after upgrade

This seems to be an issue and should be fixed in our systemd scripts,
need to discuss more on this.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 2116f76a1f1fbb32ca5844c1d051e22b1b657e0f)

since the roles are mapped inside ceph-deploy, store the roles that
are mapped and use the new mapped role for upgrades during later
stage.

eg: mon.a is mapped to mon.mira002 during install, store this mapping
and durig upgrade map it back to appropriate name to find the hostname
with that role

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 8a2b9a08095dc97e0e440cc900c650d5efb55a4f)

Add doc string for missing ceph-deploy-branch

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit e7a0a4f87f2fbea0c9e27341c952245f1ad34ab8)

use mon.a as ceph-admin node, the previous get_first_mon returns
a sorted(mons)[0] as first mon.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 407ce2613d37252e48c2a648b94b065ab070ec46)

use elif instead of else to handle cases for jewel install

jewel needs neither filestore or bluestore as an option, so provide none
when running with jewel branch.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit f3ec7cabf66c76d712f77a4dfefe96f374507982)

Jewel to luminous upgrade scenario using ceph-deploy

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 9c85c76a90dcf46ec2bcc6840f4fc57ed86583b5)

Add upgrade functionality using ceph-deploy

This is to test for customer like upgrade scenarios and to find
any issues that may be related to systemd, packaging etc

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 9a73127c16d3395934d003024964ed1373d81083)

Fix old flake8 issues

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit b409c3d163194b0b65e26b0fbc93def8e17a10b4)

Add option to skip mgr install for old release

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit a9be0c387ed2ae09227dad833c8647af110bd2de)

Merge pull request #17945 from batrick/i21540

luminous: qa whitelist fixes

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #17821 from smithfarm/wip-21472-luminous

luminous: tests: kcephfs: ignorable MDS cache too large warning

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

qa/workunits/mgr/test_localpool: simple test for localpool mode

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6383fa5b3004a606c15efd2be1b7ce5d57f05086)

pybind/mgr/localpool: module to automagically create localized pools

By default, this will create a pool per rack, 3x replication, with a host
failure domain.  Those parameters can be customized via mgr config-key
options.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1a0f42b70a4c9fa68dc47f2f521d0f1e8f5bb220)

qa: whitelist expected MDS_CACHE_OVERSIZED

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 71f0066f6ec32be18abc0c220a49e203ac3b9786)

Conflicts:
   qa/suites/kcephfs/recovery/tasks/client-limits.yaml - fix
      already applied by some other commit

qa: whitelist expected rstat warning

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c53d091e5110011866aa85495ff4e5abe0442d71)

Merge pull request #17922 from batrick/kcephfs-backports

luminous: qa: kcephfs whitelist fixes

Reviewed-by: Yan, Zheng <zyan@redhat.com>

mon/OSDMonitor: error out if setting ruleset-* ec profile property

We change ruleset -> crush back in dc7a2aaf7a34b1e6af0c7b79dc44a69974c1da23.
If someone tries to use the old property, error out early, instead of
silently not doing the thing they thought they told us to do.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6376d75eda648789b5e316a1ac2883708db7128e)

mon: show legacy health warning in `status` output

Previously you only got the text of this if you were
either looking at "health detail" or if you had
already set the preluminous_compat setting (in which
case you presumably were already aware so the message
isn't doing much).

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #17855 from smithfarm/wip-21490-luminous

luminous: tests: test_rebuild_simple_altpool triggers MDS assertion

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: get config only on running MDS

Fixes: http://tracker.ceph.com/issues/21466
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8a535d9c72965c79692dccc6ff418b46fb86c3eb)

qa: whitelist cache too large in client-limits

Fixes: http://tracker.ceph.com/issues/21510
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 87039eb30a25158a192ab7099fa81fbb9ce02953)

qa: whitelist failover log warnings

Fixes: http://tracker.ceph.com/issues/21509
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 37a7737712f36422b5f035a0e02bab25ae2da890)

qa: ignore evicted client in client-recovery

Fixes: http://tracker.ceph.com/issues/21508
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8ecef57bb10a83587f02ce087b657d303ae7c589)

Merge PR #17878 into luminous

* refs/remotes/upstream/pull/17878/head:
qa/cephfs: Fix error in test_filtered_df

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #17877 into luminous

* refs/remotes/upstream/pull/17877/head:
client: Delete onsafe
client: add mountedness check inside client_lock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17870 into luminous

* refs/remotes/upstream/pull/17870/head:
mds: reset Formatter on error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #17854 into luminous

* refs/remotes/upstream/pull/17854/head:
mds: void sending cap import message when inode is frozen
client: fix message order check in handle_cap_export()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17852 into luminous

* refs/remotes/upstream/pull/17852/head:
mds: check ongoing catter-gather process before capping log

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17822 into luminous

* refs/remotes/upstream/pull/17822/head:
qa/cephfs: kill mount if it gets evicted by mds
qa/cephfs: fix test_evict_client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #17817 from smithfarm/wip-21464-luminous

luminous: qa: ignorable MDS_READ_ONLY warning

Reviewed-by: Yan, Zheng <ukernel@gmail.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #17879 from smithfarm/wip-21449-luminous

luminous: tests: test_misc creates metadata pool with dummy object resulting in WRN: POOL_APP_NOT_ENABLED

Reviewed-by: Amit Kumar amitkuma@redhat.com
Reviewed-by: Yan, Zheng <ukernel@gmail.com>

Merge pull request #17886 from smithfarm/wip-21486-luminous

luminous: tests: [kcephfs] test_client_pin times out waiting for dentry release from kernel

Reviewed-by: Amit Kumar amitkuma@redhat.com
Reviewed-by: Yan, Zheng <ukernel@gmail.com>

Merge pull request #17887 from smithfarm/wip-21487-luminous

luminous: cephfs: MDS rank add/remove log messages say wrong number of ranks

Reviewed-by: Amit Kumar amitkuma@redhat.com
Reviewed-by: Yan, Zheng <ukernel@gmail.com>

Merge pull request #17888 from smithfarm/wip-21488-luminous

luminous: tests: failures from pjd fstest

Reviewed-by: Amit Kumar amitkuma@redhat.com
Reviewed-by: Yan, Zheng <ukernel@gmail.com>

ceph-bluestore-tool: better default logging; --log-file and --log-level options

- keep derr going to stderr (so we see 'fsck error' messages)
- hide the rest
- add friendly --log-file option

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9bd98b42fea1791935280f5ad19e5420d5303e03)

ceph-bluestore-tool: add 'bluefs-bdev-expand' to expand wal or db usage

If you are using the wal or db devices, this will expand bluefs's usage to
include the entire block device.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit aa4baad529835c1999ff9cc1a2f509c52a0cc699)

ceph-bluestore-tool: add 'bluefs-bdev-sizes' command

Show bdev sizes vs owned extents.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9e492f2a2217ee94670e9e0d6cf28e4b7bff5ad0)

ceph-bluestore-tool: factor out bluefs mount

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 86db2d7b09de4362d90fba834d61978860d73309)

os/ObjectStore: add repair interface

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c7b7a1f04f78fa62890c567d0ca53874c8d75eb7)

ceph-objectstore-tool: Make pg removal require --force

Add new export-remove to combine the 2 operations

Fixes: http://tracker.ceph.com/issues/21272
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3bb20f6d750915d176c7a34367dac966a20faa76)

ceph-objectstore-tool: Better messages for bad --journal-path

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 49ca1fff7fc4360d2f3a9cac60c0ba651cbc4750)

test: Fix ceph-objectstore-tool test for standalone and latest code

vstart.sh now defaults to bluestore, so specify filestore
Set environment for run-standalone.sh and cmake build
Create td/cot_dir as test directory
Crush output format change
Change dir into test directory
Give a little time after pool creation
Check for core files as ceph-helpers.sh does

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3ac219df2d635a08be52df3ad44cf0683de46af2)

test: Move ceph-objectstore-tool test to standalone

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 495c32fd31b311d22f0d4509e896916bd2671615)

Merge pull request #17812 from liewegas/wip-21410-luminous

Revert "osd/OSDMap: allow bidirectional swap of pg-upmap-items"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph: do link/rename semantic checks after srcdn is readable

For hard link, source inode must not be directory. For rename,
types of source/destination inodes must match. If srcdn is replica
and we do these checks while it's not readble, it's possible that
wrong source inode is used in these checks.

Fixes: http://tracker.ceph.com/issues/21383
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 46962b253563a867707e7c5d7887abf2060cc4d7)

mon/MDSMonitor: fix clog msgs on MDS rank add/rm

Because these were generated before the actual
change, the get_num_in values did not make sense.

Fixes: http://tracker.ceph.com/issues/21421
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 511439b515414ae93d791f54a187e44d5fdffa7b)

client: set client_try_dentry_invalidate to false by default

By default, ceph-fuse uses side effect of 'dentry invalidation' to
trim kernel dcache if it runs on kernel < 3.18. The implemention of
kernel function d_invalidate() changed in 3.18 kernel, the method no
longer works for upstream kernel >= 3.18.

RHEL 3.10 kernel includes backport of patches that change implemention
of d_invalidate(). So checking kernel version to decide if 'dentry
invalidation' method works is unreliable.

Fixes: http://tracker.ceph.com/issues/21423
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 69adaabface27880dd6c8dbfdeeb06cf3f3d346a)

qa/tasks/cephfs: Whitelist POOL_APP_NOT_ENABLED for test_misc

test_misc verifies that ceph fs new will not create a filesystem
on a pool that already contains objects. As part of the test, it
inserts a dummy object into a pool and then attempts to use it for
CephFS. This triggers POOL_APP_NOT_ENABLED. Setting the application
metadata for the pool (and having ceph fs new fail because of the
existing metadata) would then exercise a different failure case.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit eba01c48ca8b3113b1faf353990533a5ae39232c)

qa/cephfs: Fix error in test_filtered_df

ceph df accounts for pool size, so there is no need to do it in the test.

Fixes: http://tracker.ceph.com/issues/21381
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit b059cb6290d81df07bd8ec4e1ec9ef0be24892a2)