osd: vary tick interval +/- 5% to avoid scrub livelocks

If you have two pgs that need to scrub on two OSDs, each the primary
for one pg and the replica for the other, you can end up in a livelock:

- both osds locally reserve a scrub slot
- both osds send a scrub schedule request
- both scrub requests are rejected
- both osds wait exactly 1 second
- repeat

Seems a bit unlikely, but I've seen test cases where it goes on more an
hour.

Fixes: http://tracker.ceph.com/issues/26890
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2011377c379c9d53a3a0a693a7874fc330278898)

Conflicts:
src/osd/OSD.cc
- luminous does not have src/include/random.h; use #include <random>
  instead, seeding with whoami so each OSD gets a different series
  of pseudo-random numbers

osd: tick at OSD_TICK_INTERVAL, not heartbeat interval

Heartbeat inveral is not related!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7d76458354661f7575c4a2cae251a9b828513580)

Merge pull request #24327 from smithfarm/wip-24478-luminous

luminous: read object attrs failed at EC recovery

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #24395 from smithfarm/wip-25145-luminous

luminous: mon: Automatically set expected_num_objects for new pools with >=100 PGs per OSD

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #24397 from smithfarm/wip-36137-luminous

luminous: rgw: multisite: update index segfault on shutdown/realm reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24398 from smithfarm/wip-36202-luminous

luminous: multisite: intermittent test_bucket_index_log_trim failures

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24393 from smithfarm/wip-23998-luminous

luminous: osd/EC: slow/hung ops in multimds suite test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24375 into luminous

* refs/pull/24375/head:
mds: use monotonic waits in Beacon

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24391 from smithfarm/wip-24630-luminous

luminous: cls/rgw: don't assert in decode_list_index_key()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24387 from pdvian/wip-36126-luminous

luminous: msg/async: clean up local buffers on dispatch

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #24389 from pdvian/wip-36128-luminous

luminous: rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24316 from smithfarm/wip-26979-luminous

luminous: multisite: intermittent failures in test_bucket_sync_disable_enable

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24317 from smithfarm/wip-35703-luminous

luminous: multisite: out of order updates to sync status markers

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24318 from smithfarm/wip-35980-luminous

luminous: multisite: data sync error repo processing does not back off on empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24361 from pdvian/wip-36124-luminous

luminous: rgw: fix chunked-encoding for chunks >1MiB

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24123 from pdvian/wip-35713-luminous

luminous: librbd: ensure exclusive lock acquired when removing sync point snaps…

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #24320 from smithfarm/wip-36119-luminous

luminous: [rbd-mirror] failed assertion when updating mirror status

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24390 from smithfarm/wip-24946-luminous

luminous: librbd: image create request should validate data pool for self-managed snapshot support

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #23877 from smithfarm/wip-24842-luminous

luminous: qa: move mds/client config to qa from teuthology ceph.conf.template

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24086 from batrick/i35976

luminous: mds: configurable timeout for client eviction

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

Merge pull request #24376 from smithfarm/wip-35939-luminous

luminous: client: statfs inode count odd

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24378 from smithfarm/wip-36135-luminous

luminous: mds: rctime may go back

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use monotonic waits in Beacon

This guarantees that the sender thread cannot be disrupted by system clock
changes. This commit also simplifies the sender thread by manually managing the
thread and avoiding unnecessary context creation.

Fixes: http://tracker.ceph.com/issues/26962
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a5fc29b95281c6ca58c9177c665c379846beb4b3)

Conflicts:
src/mds/Beacon.cc
- g_conf->foo instead of g_conf()->foo
- boost::string_view instead of std::string_view
- always specify template type std::unique_lock<std::mutex>
src/mds/Beacon.h
- time::min() instead of clock::zero()
- always specify template type std::unique_lock<std::mutex>
- std::chrono::seconds instead of "1s" in std::chrono_literals namespace
  (which is a C++14ism)

Merge pull request #24382 from alfredodeza/luminous-rm36247

luminous ceph-volume: skip processing devices that don't exist when scanning system disks

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rgw: remove BucketChangeObserver from data sync thread

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f05db89637d280505321708683182f0f2c886208)

Conflicts:
src/rgw/rgw_data_sync.cc
src/rgw/rgw_data_sync.h
- argument lists are different in luminous, compared to master

rgw: add BucketChangeObserver to RGWDataChangesLog

this means that BucketTrimManager will track active buckets based on
local changes, rather than changes in remote datalogs or error repos

Fixes: http://tracker.ceph.com/issues/36034
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f3c258c49ff6899433e742b10554c83413d64a8a)

rgw: async sync_object and remove_object does not access coroutine memory

Fixes: http://tracker.ceph.com/issues/35905
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 2d38306e9333772a21ffdc9d92838e3b6b5c3148)

mon/OSDMonitor: Warn if missing expected_num_objects

When creating a pool on filestore, warn if the user appears to be
creating a pool to store a large number of objects but omitted the
expected_num_objects parameter. Create the pool anyway.

Fixes: http://tracker.ceph.com/issues/24687
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit 69fb2293c4d38012e7c4781aaa39a47596125bbb)

mon/OSDMonitor: Warn when expected_num_objects will have no effect

The expected_num_objects argument to ceph osd pool create is
only effective on filestore pools when merging is disabled
(filestore_merge_threshold < 0). Warn and disallow pool creation
in this situation.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit 4c108a50e5f74a56965d49687a8c817f4a5ce42b)

librbd: validate data pool for self-managed snapshot support

Fixes: https://tracker.ceph.com/issues/24675
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 08ea7d62ba6eedf614d72ff9d33f2e6a1c0b81fe)

Conflicts:
src/librbd/image/CreateRequest.cc
src/librbd/image/CreateRequest.h
- luminous uses m_ioctx where master has m_io_ctx
- luminous IoCtx does not have get_namespace/set_namespace
- in luminuos CreateRequest state machine is a little different than in
the master (see the diagram in CreateRequest.h). In luminous the next
state after VALIDATE_DATA_POOL is CREATE_ID_OBJECT, so call
create_id_object() instead of add_image_to_directory().

mon/MDSMonitor: no_reply on MMDSLoadTargets

If we don't note that we don't reply then we don't close out the routed
mon request and the op will appear as slow on the forwarding mon.

Fixes: http://tracker.ceph.com/issues/23769
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b462b59065424520170956581e72e16481b16f0a)

Conflicts:
src/mon/MDSMonitor.cc
- luminous has "ignore:" instead of "done:" in
MDSMonitor::preprocess_offload_targets() and this part of the backport
was already done unintentionally in
7bbc0a7b1670d99e42149fd3a25c24600314ca94

cls/rgw: don't assert in decode_list_index_key()

Fixes: http://tracker.ceph.com/issues/24117
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit e71fba0c6f5b3b0572b5136840cf0ed3c9186569)

Conflicts:
src/cls/rgw/cls_rgw.cc
- use original non-C++17 version of escape_str() function in luminous

rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors

if the bucket index lists multipart meta objects that don't actually
exist in rados, this error prevents the bucket from being deleted

Fixes: http://tracker.ceph.com/issues/35986
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 764d6a8599eb5ea5a6382fea57e4b28f97e26d93)

Conflicts:
src/rgw/rgw_multi.cc : Resolved in abort_bucket_multiparts

msg/async: clean up local buffers on dispatch

The AsyncConnection keeps local (member variable) bufferlists of incoming
messages before they're placed into the Message's front/data/middle buffers.
Previously these were reset only when a new Message is being received, which
means in steady state we store a full Message for every Connection even if
it's inactive!

Instead we obviously want to drop our local references to Message state
once it's been dispatched, so that it can go away.

Fixes: http://tracker.ceph.com/issues/35987
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 47ed036753223c44c7bf66c64d4a4adfe7267c0a)

Merge pull request #24347 from pdvian/wip-35981-luminous

luminous: ceph-disk: compatibility fix for python 3

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #24311 from batrick/i35838

luminous: mds: use monotonic clock for beacon message timekeeping

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24323 from smithfarm/wip-36133-luminous

luminous: client: update ctime when modifying file content

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24328 from smithfarm/wip-24912-luminous

luminous: qa: multifs requires 4 mds but gets only 2

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove check using method from master

Not essential we check this and it breaks tests in Luminous.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24329 from smithfarm/wip-32103-luminous

luminous: mds: allows client to create .. and . dirents

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume util.disk when there are no devices mapped, skip to the next one

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9b0f472abadde26fce2a603fca5c466ebb770d4a)

ceph-volume tests.util verify devices that don't exist don't break get_devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0176c1965e6ed532b38dfee03af0e9e3a85149d6)

Merge pull request #24136 from gregsfortytwo/wip-luminous-make-check

luminous: build/ops: rpm: selinux-policy fixes

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #24342 from tchaikov/lumious-21769

luminous: osd/ECBackend: don't get result code of subchunk-read overwritten

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

include: add utime_t ctor for ceph::coarse_real_time

this change differs from the upstream commit 61fb24883e812c11016acea0654f6aef7ddab1f7
because it relied on the std::void_t template, which isn't available in
c++11. just add another explicit constructor for coarse_real_time as a workaround

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: data sync respects error_retry_time for backoff on error_repo

don't restart processing the error_repo until error_retry_time. when
data sync is otherwise idle, don't sleep past error_retry_time

Fixes: http://tracker.ceph.com/issues/26938
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit eb655323781ce4d23d6983aa5164d9dc367497e9)

Conflicts:
src/rgw/rgw_data_sync.cc

mds: prevent rctime from going back

Fixes: http://tracker.ceph.com/issues/35916
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 920ef964311a61fcc6c0d6671b77ffe98522863d)

Conflicts:
src/mds/Server.cc
- luminous does not increment or decrement pi.inode.rstat.rsnaps

client: set f_files to the total number of files in the filesystem

Fixes: http://tracker.ceph.com/issues/24849
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 39467a2c95815a495d75a9ced119975bfe62616c)

Conflicts:
src/client/Client.cc

common: adding missing ceph::coarse_real_clock helpers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 11cd4254ff645a306442f88356e8ac3d493c9a3d)

rgw: data sync uses coarse clock for error_retry_time

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 233ee9cf291194f3d8d291a5e4632612612b7731)

qa: fix kcephfs/recovery suite

This is a luminous-only commit.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #24242 from jonsger/luminous-backport-pr#23596

luminous: rgw: incremental data sync uses truncated flag to detect end of listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24358 from alfredodeza/luminous-rm36249

luminous ceph-volume:  activate option --auto-detect-objectstore respects --no-systemd

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rgw: fix chunked-encoding for chunks >1MiB

For HTTP responses sent with chunked-encoding, and greater than 1MiB in
size, the chunk-size field was being printed wrong.

Specifically, the chunk-size field was being sent with a mangled or
missing trailer of '\r\n'.

This bug manifested as HTTP clients being unable to read the response:
Chrome generates ERR_INCOMPLETE_CHUNKED_ENCODING
Python/boto generates httplib.LineTooLong: got more than 65536 bytes when reading chunk size

The wrong variable was being used to determine the size of the buffer
used for the chunk-size field.

Fix it by using the correct variable, and rename the variables to
clearly reflect their purpose.

Prior to PR#23940, this would only have been seen in some Swift
operations. PR#23940 changed some S3 operations to also use chunked
encoding to get responses sent faster, and made the bug easier to
detect. It was initially reported for a ListBucket call with a high
max-keys argument.

Backport: luminous, mimic
Reference: https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
Reference: https://github.com/ceph/ceph/pull/23940
Fixes: http://tracker.ceph.com/issues/35990
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
(cherry picked from commit 3b864482d6aef2efe0b03be70ea83c38f7a6d99b)

qa: add qa helper methods from master

For Luminous. This is needed by tests in this branch.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: whitelist cap revoke warning

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4367de377e68102f3c17c8dd85321c221d06d9dd)

doc: document cap revoke non-responders client eviction

Fixes: http://tracker.ceph.com/issues/25188
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 28a52d6fa14425fc877a69055dabe4e7c00f6b14)

test: validate client eviction for cap revoke non-responders

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit c0b1dacc9f9c9b5af07a1b83a0adb53d001c2b79)

mds: add counter for tracking cap non-responding clients

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 8f2de92712a98568b0d07a795f1158868caae550)

Conflicts:
src/mds/Server.cc
src/mds/Server.h

mds: evict clients that do not respond to cap revoke by MDS

By default, preserve old behaviour. When configured with a non
default value, evict clients that have not responded to cap
revoke by MDS for the configured amount of seconds.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 4cf7815cdcd8efbbb981ef45b3eabee387b4de21)

Conflicts:
src/common/options.cc
src/mds/MDSDaemon.cc
src/mds/MDSRank.h
src/mds/Server.cc
src/mds/Server.h

mds: pass timeout argument for fetching late clients

This would be required when fetching clients that have not
responded to cap revoke by MDS for a configured timeout
value.

Additionally, make member functions private which are called
from the Locker class itself.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 005cf6c76f2d25ee88dd6ac9d0c67cfa88a9d58e)

Conflicts:
src/mds/Locker.cc

mds: use monotonic clock in beacon

Also update other parts of MDS which interact.

Fixes: http://tracker.ceph.com/issues/26959
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e77a2f5da71f1ba62b69b8d0fb9f34abe426da79)

Conflicts:
src/mds/Beacon.cc
src/mds/Beacon.h
src/mds/MDCache.h
src/mds/Server.cc

mds: simplify beacon init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b7448f8fe0e4d12dd1070de4c3380e21d9da310a)

Conflicts:
src/mds/Beacon.cc
src/mds/Beacon.h

Merge pull request #24089 from batrick/i26990

luminous: mds: curate priority of perf counters sent to mgr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

Merge pull request #24310 from batrick/i36210

luminous: mds: runs out of file descriptors after several respawns

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume lvm.activate pass no_systemd args when detecting objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 29de6820a9a7108c8de41036add9332f9b910359)

ceph-volume tests.lvm verify activate works with --auto-detect-objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 60dbeb6184797ccffeaeea2e85ef2d7fb4a1c54c)

librbd: ensure exclusive lock acquired when removing sync point snapshots

Fixes: http://tracker.ceph.com/issues/24898
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 5bb823098ed0728cf9b88c654ba4528f3e42d695)

librbd: fixed memory leak and use-after-free in group snap API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8d68fa2a8e0c292d74d5f4d0080e4db370f4685e)

Conflicts:
src/librbd/api/Group.cc: removed file

Merge pull request #24335 from alfredodeza/luminous-rm36098

luminous ceph-volume:  lsblk can fail to find PARTLABEL, must fallback to blkid

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24070 from smithfarm/wip-25025-luminous

luminous: cls_rgw test is only run in rados suite: add it to rgw suite as well

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24299 from pdvian/wip-35962-luminous

luminous: osd/PG: avoid choose_acting picking want with > pool size items

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #24308 from neha-ojha/wip-36239-luminous

luminous: osd/PrimaryLogPG: fix potential pg-log overtrimming

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph-disk: compatibility fix for python 3

In python 3, dev is a string, but mounts_dev is bytes (because
/proc/mounts was opened with mode 'rb') so they can't compare
equal, resulting in is_mounted() returning None for mounted OSDs.
The safest fix for this we could come up with was to normalize
dev to a str using _bytes2str() (just in case), and open
/proc/mounts in mode 'r', so its lines are interpreted as strs.

Fixes: https://tracker.ceph.com/issues/35906
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 8e3d948ff4c3b19dc55b9a5ae63f5b63f0c476a3)

osd/ECBackend: don't get result code of subchunk-read overwritten

Fixes: http://tracker.ceph.com/issues/21769
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 4692f995ba6967796d5dd2f5fd85fde8705a1ccc)

Conflicts:
src/osd/ECBackend.cc:
          master contains changes from
          33d79f303d945c8d6d2e9e9a471efc4df57617a6, but luminous
          does, so we don't need to consider the case of fragmented
          read.

mds: reorganize and curate perf log counters

Categorize perf counters into high (PRIO_{CRITICAL/INTERESTING}),
medium (PRIO_USEFUL) and low priority (PRIO_DEBUGONLY). High and
medium priority stats are transmitted to ceph mgr and included in
prometheus output.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
Fixes: http://tracker.ceph.com/issues/24004
(cherry picked from commit 48a33f6f06802fee998f60a2ec35d9041a7c51a6)

Conflicts:
src/mds/MDSRank.cc

mds: show session load average in dump session

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 682dc08cbd521342b4abc45fd85edb2dd7bc5682)

Conflicts:
    src/mds/SessionMap.h

mds: allow request load average decay rate configurable via admin socket

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 3809d96a5b53721b88602d662c4c1cd00dd727a0)

Conflicts:
src/mds/MDSRank.h
src/mds/SessionMap.h
src/mds/SessionMap.cc

mds: decay counter for tracking request load average

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 816d8aaea01aef81467e3df7a4b3ab3eff86d27a)

Conflicts:
src/common/options.cc
src/mds/SessionMap.cc
src/mds/SessionMap.h

mds: add perf counters to track current open/stale sessions

Signed-off-by: Venky Shankar <vshankar@redhat.com>

mds:improve mds perf counters can statistics filesystem operations number and latency

Dumping the latency of all operations which will help we analysis of
what kind of operation is the most waste of time which can do a simple
performence analysis.

Fixes: http://tracker.ceph.com/issues/22097
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
(cherry picked from commit 20760f26bae7d9c599019a0295d66386e4281af2)

Merge PR #23553 into luminous

* refs/pull/23553/head:
mds: report lagginess at lower debug
MDSMonitor: note beacons and cluster changes at low dbg level

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Conflicts:
    src/mds/Beacon.cc

ceph-volume simple.scan better detection for ceph-disk data devices

Uses the new ``CephDiskDevice`` to look into PARTLABEL from both lsblk
(the default) falling back to blkid, which in some cases has the right
value when lsblk fails.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c502e5f64aa4f0d832c14680e1731e58d8a3770b)

ceph-volume tests.util verify CephDiskDevice behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8f072eb3f940a6714089b37a6fbacd549420a454)

ceph-volume tests.conftest add blkid patching to device_info fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5bfc8c26e639dba955f83f53893ce311e6e047f1)

ceph-volume util.device add a ceph-disk device object to detect types

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 200e6bfd934b34aa8175330aef32c51505ec3392)

ceph-volume tests.util verify new blkid API calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ead3c7e7d19635a741a9da6fc330ec12a68b3d77)

ceph-volume util.disk expand blkid's API calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dbde6a5b611fdd4bcd942df5dbab94ab5076fdff)

Merge pull request #24083 from batrick/i26851

luminous: ceph_volume_client: py3 compatible

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24313 from batrick/i36198

luminous: ceph-fuse: add SELinux policy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24119 from ukernel/luminous-22504

luminous: client: drop null child dentries before try pruning inode's alias

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24138 from pdvian/wip-35718-luminous

luminous: mds: print is_laggy message once

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24268 from smithfarm/wip-36101-luminous

luminous: qa: remove knfs site from future releases

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #24285 from iridescent-rsy/luminous-35958

luminous: librbd: fixed assert when flattening clone with zero overlap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24303 from vshankar/wip-35931

luminous: client: retry remount on dcache invalidation failure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

*: set missing CLOEXEC on opened fds

Otherwise these descriptors may leak across execve() during e.g. MDS respawn.

Fixes: http://tracker.ceph.com/issues/35850
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 86dbd9e755ae0472db3d8c04bc444c502193dc13)

Conflicts:
CMakeLists.txt
src/auth/Crypto.cc
src/ceph_osd.cc
src/common/CMakeLists.txt
src/common/Preforker.h
src/common/SubProcess.cc
src/common/admin_socket.cc
src/common/admin_socket_client.cc
src/common/blkdev.cc
src/crimson/CMakeLists.txt
src/global/signal_handler.cc
src/include/random.h
src/include/uuid.h
src/log/Log.cc
src/mon/MonitorDBStore.h
src/msg/simple/Accepter.cc
src/os/bluestore/BlueStore.cc
src/os/bluestore/bluestore_tool.cc
src/os/filestore/FileStore.cc
src/os/kstore/KStore.cc
src/test/objectstore/test_bluefs.cc

Merge pull request #24338 from alfredodeza/luminous-rm36251

luminous ceph-volume add new ceph-handlers role from ceph-ansible

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests.functional add new ceph-handlers role from ceph-ansible

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 956dc5e65c6e35e27ab6524d4fb9fdd05d616cf6)

mds: access last std::vector element with `back()`

... rather than accessing it as an array. also convert
to range loops whereever necessary. plus some style
fixes.

Fixes: https://tracker.ceph.com/issues/25113
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f9b8ad5793d8265f131566233080613f475004aa)

mds: disallow certain file operations to "." and ".." dirents

Also, fixup return value for file operations such as rmdir()
and rename() on these directories.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bae6c9db656afeff1dede6253300d22082313187)

Conflicts:
src/mds/Server.cc
src/test/libcephfs/test.cc