Merge pull request #18832 from shinobu-x/wip-22075-luminous

luminous: mgr tests don't indicate failure if exception thrown from serve()

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18842 from shinobu-x/wip-21863-luminous

luminous: ceph-conf: dump parsed config in plain text or as json

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19002 from tchaikov/wip-pr-18976-luminous

luminous: udev: Fix typo in udev OSD rules file

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19023 from pdvian/wip-22164-luminous

luminous: cluster [ERR] Unhandled exception from module 'balancer' while running on mgr.x: 'NoneType' object has no attribute 'iteritems'" in cluster log

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19031 from pdvian/wip-22167-luminous

luminous: Various odd clog messages for mons

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19039 from jan--f/wip-22199-luminous

luminous: tools/crushtool: skip device id if no name exists

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19046 from shinobu-x/wip-22189-luminous

luminous: osdc/Objecter: objecter op_send_bytes perf counter always 0

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19059 from pdvian/wip-22176-luminous

luminous: osd: subscribe osdmaps if any pending pgs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19083 from smithfarm/wip-22213-luminous

luminous: On pg repair the primary is not favored as was intended

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #19084 from shinobu-x/wip-22216-luminous

 luminous: "osd status" command exception if OSD not in pgmap stats

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19118 from tangwenjun3/wip-backport-19030

luminous: os/bluestore: fix the allocate in bluefs

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Shinobu Kinjo <skinjo@redhat.com>

Merge pull request #19133 from smithfarm/wip-22194-luminous

luminous: Default kernel.pid_max is easily exceeded during recovery on high OSD-count system

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #19631 from shinobu-x/wip-22509-luminous

 luminous: osd: "sudo cp /var/lib/ceph/osd/ceph-0/fsid ..." fails

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19839 from tchaikov/wip-22558-luminous

luminous: rpm: adjust ceph-{osdomap,kvstore,monstore}-tool feature move

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #18838 from shinobu-x/wip-21973-luminous

luminous: [test] UpdateFeatures RPC message should be included in test_notify.py

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18840 from shinobu-x/wip-21970-luminous

luminous: [journal] tags are not being expired if no other clients are registered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18949 from jcsp/wip-luminous-rgw-config-desc

luminous: options.cc: document rgw config options

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19051 from shinobu-x/wip-22185-luminous

luminous: rbd: abort in listing mapped nbd devices when running in a container

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20062 from cbodley/wip-luminous-rgw-bilog-trim

luminous: rgw multisite: automated trimming for bucket index logs

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #19848 from cbodley/wip-luminous-boost-166

luminous: backport beast frontend and boost 1.66 update

Merge pull request #19500 from shinobu-x/wip-22401-luminous

luminous: rgw: make HTTP dechunking compatible with Amazon S3

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17816 from smithfarm/wip-hadoop-s3a

luminous: tests: qa/rgw: hadoop-s3a suite targets centos_latest

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #18972 from linuxbox2/luminous-22084

luminous: rgw: Fix swift object expiry not deleting objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19086 from shinobu-x/wip-22210-luminous

luminous: radosgw-admin zonegroup get and zone get should return defaults when there is no realm

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19088 from pdvian/wip-22177-luminous

luminous: rgw: check going_down() when lifecycle processing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #19636 from shinobu-x/wip-22506-luminous

luminous: rgw usage trim only trims a few entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19784 from linuxbox2/luminous-20201

luminous: RGW: S3 POST policy should not require Content-Type

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #19799 from smithfarm/wip-22581-luminous

luminous: rgw: multisite: 'radosgw-admin sync error list' contains temporary EBUSY errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19867 from smithfarm/wip-22591-luminous

luminous: rgw: refuses upload when Content-Type missing from POST policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20051 from ceph/revert-19538-wip-22434-luminous

Revert "luminous: rgw: user stats increased after bucket reshard"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20056 from linuxbox2/luminous-rgw-latchfix

luminous: rgw_file: alternate fix deadlock on lru eviction

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20073 from smithfarm/wip-22767-luminous

luminous: rgw: librgw: fix shutdown error with resources uncleaned

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20075 from smithfarm/wip-22773-luminous

luminous: rgw: file deadlock on lru evicting

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #20107 from dreamhost/wip-22792

luminous: rgw: When a system object is created exclusively, do not distribute the

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd: Scrub of object with only object error(s) are not logged

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1ad05b1068ddd5d3312af45af1a60587200ddcd7)

osd: Primary shard needs to be at the front to be preferred

Caused by: cd0d8b0714d8684cf61b4650e170027ef46f489b

Fixes: http://tracker.ceph.com/issues/21907
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b7d2f27da1a68046a688749f95093a88e6362b28)

ceph-osdomap-tool: Add compact comand option

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b1f397f4fc3313ee0325722156759bfe32286c6b)

osd: Fixes for mark_unfound_lost

Let recovery requeue operations and release backoff
For "delete" mark object context, if present, to non-existent

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0c106f1a46815bb769e7a503b1adb265480b6779)

ceph-objectstore-tool: Fix debug output

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 090c773fc9b7f3316ed88edd24b85a703224ddc5)

test: Remove bogus check in ceph_objectstore_tool.py

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f918b1fac1f3cdb4cf316fb4c7afc90de49ce06e)

Merge pull request #20136 from dillaman/wip-22815-luminous

luminous: qa/workunits/rbd: switch devstack to pike release

Reviewed-by: Mykola Golub <mgolub@suse.com>

qa/workunits/rbd: switch devstack to pike release

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7ff1e0bc2193acd009a74db81cf8e8cc8639ad12)

Merge pull request #19105 from batrick/i22228

luminous: client: anchor Inode while trimming caps

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #19360 from ukernel/luminous-22219

luminous: mds: ignore export pin for unlinked directory

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19646 from pdvian/wip-22503-luminous

luminous: mds: properly eval locks after importing inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20055 from dzafman/wip-22724

luminous: miscounting degraded objects and PG stuck in recovery_unfound

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'luminous' into wip-22724

Merge pull request #20081 from dzafman/wip-recovering-luminous

luminous: osd/ReplicatedPG.cc: recover_replicas: object added to missing set for backfill, but is not in recovering, error!

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18008 from ukernel/luminous-21584

luminous: mds: fix CDir::log_mark_dirty()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #18782 from ukernel/luminous-21985

luminous: mds: fix MDS_FEATURE_INCOMPAT_FILE_LAYOUT_V2 definition

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18783 from ukernel/luminous-21975

luminous: mds: trim 'N' log segments according to how many log segments are there

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18831 from shinobu-x/wip-22076-luminous

luminous: cephfs: "ceph tell mds" commands result in "File exists" errors on client admin socket

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18835 from shinobu-x/wip-22074-luminous

luminous: mds: don't check gid when none specified in auth caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18869 from shinobu-x/wip-21952-luminous

luminous: mds: no assertion on inode being purging in find_ino_peers()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18871 from shinobu-x/wip-21947-luminous

luminous: mds: preserve order of requests during recovery of multimds cluster

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18912 from shinobu-x/wip-22077-luminous

luminous: src/mds/MDCache.cc: 7421: FAILED assert(CInode::count() == inode_map.size() + snap_inode_map.size())

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19055 from shinobu-x/wip-22192-luminous

luminous: MDSMonitor: monitor gives constant "is now active in filesystem cephfs as rank" cluster log info messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #19326 from ukernel/luminous-22263

luminous: mds: handle client reconnect gather race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19480 from smithfarm/wip-22407-luminous

luminous: cephfs: client: implement delegation support in userland cephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #19775 from ukernel/luminous-19578

luminous: mds: track dirty dentries in separate list

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19827 from pdvian/wip-22587-luminous

luminous: mds: reduce debugging level for balancer messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #20082 from smithfarm/wip-21525-luminous

luminous: cephfs: client: dual client segfault with racing ceph_shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20085 from pdvian/wip-22763-luminous

luminous: cephfs-journal-tool: add "set pool_id" option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20086 from pdvian/wip-22765-luminous

luminous: client: avoid recursive lock in ll_get_vino

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/rgw: hadoop suite targets centos_latest

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a97d5bea42a40909aa4e9672505b9ad994cec184)

Conflicts:
    qa/suites/rgw/hadoop-s3a/s3a-hadoop.yaml (changes already backported
        via 6b5f212b9170dbd99779527b41be233b9d40b5e4)

qa/rgw: use overrides for hadoop version

instead of having two separate tasks, add v27.yaml and v28.yaml with
overrides for hadoop_version

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4a953a16d7b80d343899de39a8cf55dc761847de)

Merge pull request #18019 from ukernel/luminous-21091

luminous: mds: fix StrayManager::truncate()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19485 from dillaman/wip-21788-luminous

luminous: librbd: refresh image after applying new/removing old metadata

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #18834 from shinobu-x/wip-22073-luminous

luminous: [api] compare-and-write methods not properly advertised

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19058 from shinobu-x/wip-22190-luminous

luminous: class rbd.Image discard----OSError: [errno 2147483648] error discarding region

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19066 from pdvian/wip-22172-luminous

luminous: [rbd-nbd] Fedora does not register resize events

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #19107 from pdvian/wip-22208-luminous

luminous: rbd: disk usage on empty pool no longer returns an error message

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19305 from pdvian/wip-21700-luminous

luminous: rbd-mirror: Allow a different data-pool to be used on the secondary cluster

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19447 from smithfarm/wip-21646-luminous

luminous: librbd: Image-meta should be dynamically refreshed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19479 from smithfarm/wip-22376-luminous

luminous: rbd: Python RBD metadata_get does not work

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #19484 from dillaman/wip-21644-luminous

luminous: rbd-mirror: sync image metadata when transfering remote image

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #19503 from shinobu-x/wip-22395-luminous

luminous: librbd: cannot clone all image-metas if we have more than 64 key/value pairs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19504 from shinobu-x/wip-22393-luminous

luminous: librbd: cannot copy all image-metas if we have more than 64 key/value pairs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19554 from shinobu-x/wip-22454-luminous

 luminous: cluster resource agent ocf:ceph:rbd - wrong permissions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19625 from shinobu-x/wip-22497-luminous

 luminous: [rbd-mirror] new pools might not be detected

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19800 from smithfarm/wip-21641-luminous

luminous: rbd ls -l crashes with SIGABRT

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #19802 from smithfarm/wip-21690-luminous

luminous: tests: rbd_mirror_helpers.sh request_resync_image function saves image id to wrong variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19853 from pdvian/wip-22593-luminous

luminous: rbd: librbd: filter out potential race with image rename

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Revert "luminous: rgw: user stats increased after bucket reshard"

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

When a system object is created exclusively, do not distribute the
cache information to the other rados gateways and instead let them
lazily retrieve the information when they need it.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 647ce3387312fc683660c1f3c7571c577379be1c)
Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #18722 from shinobu-x/wip-luminous-21946

luminous: mds: set PRIO_USEFUL on num_sessions counter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18730 from shinobu-x/wip-luminous-22030

luminous: List of filesystems does not get refreshed after a filesystem deletion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18754 from shinobu-x/wip-21636-luminous

luminous: ceph-monstore-tool --readable mode doesn't understand FSMap, MgrMap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20053 from dillaman/wip-22740-luminous

luminous: qa/suites/upgrade: disable broken kraken upgrade cls_rbd test

Reviewed-By: Yuri Weinstein <yuriw@redhat.com>

Merge pull request #18564 from kmroz/wip-21688-luminous

luminous: Possible deadlock in 'list_children' when refresh is required

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18698 from shinobu-x/wip-luminous-21808

luminous: rbd: fix crash during map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19038 from pdvian/wip-22183-luminous

luminous: rgw: set sync_from_all as true when no value is seen

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19446 from smithfarm/wip-22404-luminous

luminous: doc: crush_ruleset is invalid command in luminous

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #18747 from shinobu-x/wip-21794-luminous

luminous: backoff causes out of order op

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #20087 from dzafman/wip-misc-luminous

qa/standalone: Minor fixes for ceph_objectstore_tool.py test

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: Cleanup test-erasure-eio.sh code

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 69b5fc54feb59f8b0a26a3ca3e925980c91b5b75)

test: Add replicated recovery/backfill test

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c2572bee3c6256419e0b265e9e2829e7f3afb76d)

osd: Better recovery/backfill restart for mark_unfound_lost delete/revert

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 689bff354a6bfa1cf47b0d0a04fa9f1b2ef68f75)

osd, mon: Add new pg states recovery_unfound and backfill_unfound

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 7f8b0ce9e681f727d8217e3ed74a1a3355f364f3)

Conflicts:
src/osd/PG.h (trivial)
src/osd/osd_types.h (trivial)

osd: Add new UnfoundBackfill and UnfoundRecovery pg transitions

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bb2bcb95f51abc206e005e44ef383ee45b8f2209)

Conflicts:
src/osd/PG.cc (trivial, no MBackfillReserve::CANCEL)
src/osd/PG.h (trivial)

fixup! osd: Add new UnfoundBackfill and UnfoundRecovery pg transitions