qa/cephfs: test if evicted client unmounts without hanging

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit b14302d1fed101beefd81d5457c14c2a88a13e09)

qa/tasks: allow custom timeout for umount_wait()

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 18a9d0c49175c9aacfaec15d28125c8fb2a77b51)

client: don't hang when MDS sessions are evicted

Currently, a filesystem client hangs if a request is made after it's
eviction. Prevent the client from hanging and allow a manual unmount
in such cases.

Fixes: http://tracker.ceph.com/issues/10915
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 0f56c7e8e5f45112c5924797a7798b3c926d8492)

Merge pull request #21674 from pdvian/wip-23865-luminous

luminous: rgw: fix error handling for GET with ?torrent

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #19884 from pdvian/wip-22637-luminous

luminous: rgw:lc: RGWPutLC return ERR_MALFORMED_XML  when missing <Rule> tag in…

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #21382 from smithfarm/wip-23681-luminous

luminous: mg_read() call has wrong arguments

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21435 from pdvian/wip-23682-luminous

luminous: rgw: return EINVAL if max_keys can not convert correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21438 from pdvian/wip-23685-luminous

luminous: rgw_file: conditionally unlink handles when direct deleted

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21660 from pdvian/wip-23862-luminous

luminous: rgw: aws4 auth supports PutBucketRequestPayment

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #21661 from pdvian/wip-23861-luminous

luminous: rgw: admin rest api shouldn't return error when getting user's stats if

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21669 from oritwas/wip-rgw-luminous-22721

luminous: rgw: fix bi_list to reset is_truncated flag if it skips entires

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21675 from pdvian/wip-23866-luminous

luminous: rgw: move all pool creation into rgw_init_ioctx

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21735 from pdvian/wip-23868-luminous

luminous: rgw: do not reflect period if not current

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21736 from pdvian/wip-23869-luminous

luminous: rgw: es module: set compression type correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21758 from cbodley/wip-23864

luminous: rgw: add buffering filter to compression for fetch_remote_obj

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21792 from cbodley/wip-23977

luminous: rgw: raise log level on coroutine shutdown errors

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21908 from cbodley/wip-24060

luminous: radosgw-admin sync status improvements

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21917 from smithfarm/wip-24063-luminous

luminous: rgw: Misnamed S3 operation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21741 from pdvian/wip-23902-luminous

luminous: [rbd-mirror] local tag predecessor mirror uuid is incorrectly replaced with remote

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21869 from pdvian/wip-23945-luminous

luminous: qa/workunits/rbd: potential race in mirror disconnect test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21944 from dillaman/wip-23809-luminous

luminous: qa/workunits/rbd/import_export: update unit formatting

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21995 from tchaikov/luminous-24051

luminous: os/bluestore: fix the demotion in StupidAllocator::init_rm_free

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
ifed01

os/bluestore: fix the demotion in StupidAllocator::init_rm_free

* we should update the _size field when demotion kicks in, otherwise the
  interval_set::m and interval_set::_size won't be consistent.
* also rename post_process to claim, as it "steals" the leftover(s) of
chopped segment.

Fixes: https://tracker.ceph.com/issues/24051
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d43c8da83329dd743c635869b1443366d03839fd)

Merge pull request #21872 from tchaikov/wip-luminous-jdk

luminous: cmake: fix the cepfs java binding build on Bionic

Reviewed-by: Robin H. Johnson <robin.johnson@dreamhost.com>

radosgw-admin: add pretty 'bucket sync status' command

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 650aef59cfaf429951a82a9d65a37b8c4bd1b942)

Conflicts:
src/rgw/rgw_admin.cc (no readonly_ops_list, no std::string_view)

rgw: expose struct bucket_index_marker_info in header

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6274368d9cc6566e85b52e53d6949a7814a5ea27)

rgw: rgw_bucket_sync_status takes bucket info

rgw_bucket_sync_status() no longer reads the bucket instance info, and
instead requires the caller to pass it in

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3640ee844a9d77f0fa6a6849c4d8b51cc932f4e2)

radosgw-admin: rename 'bucket sync status' to 'bucket sync markers'

renames the command to 'bucket sync markers' because it only returns the
local sync status markers for use in multisite tests

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2df2b0665d12dd9e47bf7155a1c6335c2d1a3bf4)

Conflicts:
src/rgw/rgw_admin.cc (no readonly_ops_list)

rgw: translate the state in rgw_data_sync_marker

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 7ddca6e50d85380505134481d37734b4f1433f1a)

doc: update radosgw-admin.rst and help.t about data sync status

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 33a0ec7164eff9fb66f05d984eb92b0de4ced9b1)

rgw: add --shard-id for data sync status

when specified --shard-id with data sync status, it'll
display the realated pending and recovering bucket shards, as well as sync
marker, besides this, we can specified --max-entries to limit the number
of datalog entries(for pending buckets) or omapkey entries(for recovering
buckets) be consumed.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 1dc7f5ea207dc17c743bfa1e06db446e16d31aa6)

rgw: read behind bucket shards of a specified data log shard

add read_shard_status to read pending bucket shards,
recovering bucket shards and sync marker

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 9c5ff46ad9c849eb0c99a4d78be3f9dd2299779c)

Conflicts:
rgw_data_sync.cc (make_move_iterator fails to build with gcc<5)

Merge pull request #21936 from ceph/wip-yuriw-24021-luminous

qa/test: added client-upgrade-luminous suite

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21871 from tchaikov/wip-luminous-18649

luminous: mgr/restful: A couple of restful fixes

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #21911 from dzafman/backport_23850-zafman

osd/ECBackend: only check required shards when finishing recovery reads

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

qa/test: added client-upgrade-luminous suite
(whitelisted CACHE_POOL_NO_HIT_SET)

Fixes https://tracker.ceph.com/issues/24021

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/workunits/rbd/import_export: update unit formatting

Mimic changed the unit formating output for rbd CLI commands, so
for upgrade tests we need to support both Luminous and Mimic style.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21725 from smithfarm/wip-23631-luminous

luminous: rbd: python bindings fixes and improvements

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21726 from smithfarm/wip-23900-luminous

luminous: [rbd-mirror] asok hook for image replayer not re-registered after bootstrap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21870 from tchaikov/wip-luminous-24041

luminous: ceph-disk: write log to /var/log/ceph not to /var/run/ceph

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #21919 from smithfarm/wip-24070-luminous

luminous: build-integration-branch script

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados: add coverage for osd_recovery_max_single_start > 1

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 234d652317baefb28fd498f476f1db573974db4c)

osd/ECBackend: preserve requests for other objects when sending extra reads

When multiple objects are in flight for the same ReadOp, swap() on the
map<hobject_t, read_request_t> would remove requests for all objects.

We just want to replace the requests for the single object we're
dealing with in send_all_remaining_reads().

This prevents crashing trying to look up rop.to_read[hoid] when another
object in the same ReadOp gets an EIO and tries to send more requests.

Test this by using osd-recovery-max-single-start to bundle multiple
reads into one ReadOp. Save and restore CEPH_ARGS so custom settings
are reset for each test.

Fixes: http://tracker.ceph.com/issues/23195 (the 2nd crash there)
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit d4808256d25851f85ce5e86704cc4e867e2fc905)

osd/ECBackend: recover from EIO based on the minimum data necessary

Discount shards that already returned EIO, and use minimum_to_decode()
to request just what is necessary to recover or read the originally
requested extents of the object.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit b162a5478d6a907cc0a9ddd5ae8442e81f8d8fb3)

Conflicts:
src/osd/ECBackend.cc (Adjust for Luminous not having subchunks)
src/osd/ECBackend.h (trivial)

osd/ECBackend: only check required shards when finishing recovery reads

1235810c2ad08ccb7ef5946686eb2b85798f5bca allowed recovery to use
multiple passes of reads to handle EIO, but the end condition for
checking whether we finished reading requires the full data to be
decodable (this is what get_want_to_read_shards returns).

This is just a loss of efficiency normally, since when there is only
one object the subsequent read works, and grabs all the data
necessary. The crash comes from having multiple objects in the same
ReadOp - in this case the sequence of events is:

- start recovery of two objects (osd_recovery_max_single_start > 1)
- read object a shard 3
- read object b shard 3
- fail minimum_to_decode because shard 3 can't reconstruct all of object a
- re-read all of object a, marking more reads in progress
- fail minimum_to_decode because shard 3 can't reconstruct all of object b
- skip re-reading object because there are now reads in progress
- finish reading k shards of object a
- still fail minimum_to_decode for object b, so no extra data was read
- send_all_remaining_reads tries to lookup object b in ReadOp object
- crash dereferencing to_read[object b], since this was cleared after handling the original object b read reply

This patch fixes the immediate inefficiency and crash by only checking
for the missing shards that were requested, rather than the entire
object, for recovery reads.

Fixes: http://tracker.ceph.com/issues/23195 (first crash)
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 468ad4b41010488c8d48ef65ccbebfdb4270690f)

Conflicts:
src/osd/ECBackend.cc (trivial)

osd/ECBackend: store the original needed shards in a ReadOp

This avoids extra logic recomputing them due to EIO or redundant
reads.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 0f9474d4e8e54cb8784cc2dbaa50318c2ff8e667)

Conflicts:
src/osd/ECBackend.cc (trivial)

qa/standalone: s/delete_erasure_pool/delete_erasure_coded_pool/

it's a regression introduced by ac56a202

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fc43ae1724f6cdf3a00e1995281e1e4d414093da)

Brought into Luminous to make backport clean

qa/standalone: extract delete_pool()

some tests, like osd-backfill-stats.sh are using delete_pool(), but
they don't have this function defined. and this function is defined
in standalone tests separately, so would be simpler if we can
consolidate them in ceph-helper.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ac56a202fd57d6a18d3852d26a4e5cf4a214bf8a)

Brought into Luminous to make backport clean

script: build-integration-branch: avoid Unicode error

When run on PRs that have non-ASCII characters in their titles, the script
fails like this when run in a non UTF-8 environment:

UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2026' in
position 651: ordinal not in range(256)

Since failing like that is not useful, avoid the error by forcing ASCII
encoding and filtering out any non-ASCII characters.

Fixes: http://tracker.ceph.com/issues/24003
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 6dd31fdbec43e37bece24c0de886d2c4083901c8)

script/build-integration-branch: python3 compatible and pep8 clean

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4e8e3116949d11f62ac027e300c09dc5f929cfc5)

script/build-integration-branch: check errors

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 686f2ae7fa8050d6113674ea5570ad956792e49a)

script/build-integration-branch: print pr url list with titles

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5916683c52fe6c1d54e91415ba0c4af790d48485)

src/script/build-integration-branch

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b5c27826158f699960fcf887e901545c359ad68c)

rgw: policy: modify s3:ListBucketMultiPartUploads to s3:ListBucketMultipartUploads

according to AWS S3, the permission keyword should be s3:ListBucketMultipartUploads
rather than s3:ListBucketMultiPartUploads.

all operation permission list in AWS S3 as follows:
  https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html?shortFooter=true

Signed-off-by: xiangxiang <xiangxiang@xsky.com>
(cherry picked from commit 41ef62d9047ca7469023181f203e79c63ec30d64)

rgw: display data sync recovering shards in radosgw-admin sync status

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 8c3ea263b910db99dfe7e7aa5f17193941c70d1d)

rgw: add RGWReadDataSyncRecoveringShardsCR to read recovering shards

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit f4de5b58e27f320e8eed6a72684ad7621a7c1110)

rgw: display errors of object sync failed in sync error list

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 39fd90c8ad2ce5af8d074a7f52d12dea3b06c730)

rgw: add lagging shard ids in rgw sync status

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 99f3c3f8e97159cda401b280ea25f9822b14b90d)

rgw: RGWRadosGetOmapKeysCR uses omap_get_keys2

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0ccf50fc9df95bf2b125e4b3cb1b641aef8c23b5)

Conflicts: (no sync tracing)
src/rgw/rgw_data_sync.cc
src/rgw/rgw_sync.cc

rgw: RGWRadosGetOmapKeysCR uses completion return code

when this operates on a nonexistent object, the osd will reject the
request with ENOENT before trying to process the subops. so Objecter
will get back a subop return code of 0, try to decode an empty
bufferlist into the result and map that subop return code to EIO

by using the AioCompletion's return code, we get the correct result of
ENOENT instead

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2f94f63e705c33c4bf57923d96c747c5c6681e2d)

Merge pull request #21109 from pdvian/wip-23474-luminous

luminous: client: add ceph_ll_sync_inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #21346 from pdvian/wip-23632-luminous

luminous: mds: mds shutdown fixes and optimization

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21354 from pdvian/wip-23635-luminous

luminous: client: fix request send_to_auth was never really used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21638 from smithfarm/wip-23852-luminous

luminous: osd: do not crash on empty snapset

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21359 from xiexingguo/backport-pr-21315

luminous: crush/CrushWrapper: fix potential invalid use of iterator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21366 from pdvian/wip-23636-luminous

luminous: mds: fix unhealth heartbeat during rejoin

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21372 from pdvian/wip-23637-luminous

luminous: mds: mark damaged if sessions' preallocated inos don't match inotable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21441 from pdvian/wip-23702-luminous

luminous: mds: properly dirty sessions opened by journal replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21447 from pdvian/wip-23703-luminous

luminous: MDSMonitor: fix mds health printed in bad format

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21448 from pdvian/wip-23750-luminous

luminous: mds: make rstat.rctime follow inodes' ctime

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21512 from pdvian/wip-23791-luminous

luminous: MDSMonitor: initialize new Filesystem epoch from pending

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21542 from pdvian/wip-23802-luminous

luminous:client: flush the mdlog in _fsync before waiting on unstable reqs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21596 from joscollin/wip-23771-luminous

luminous: client: Fix the gid_count check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21732 from batrick/i23792

luminous: MDSMonitor: cleanup and protect fsmap access

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21796 from dalgaaf/luminous-issue-23873

luminous: client: use common interp of st_nlink for dirs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21840 from pdvian/wip-23930-luminous

luminous: mds: properly trim log segments after scrub repairs something

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21844 from pdvian/wip-23934-luminous

luminous: mds: properly check auth subtree count in MDCache::shutdown_pass()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

cmake: use javac -h for creating JNI native headers

JDK 1.10 does not offer javah anymore, so we need to use "javac -h" or
add_jar(... GENERATE_NATIVE_HEADERS) instead.

Fixes: http://tracker.ceph.com/issues/24012
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit edc504b9d446cc1739f3e8e087914cfe66d6cac3)

Conflicts:
src/java/CMakeLists.txt: in master, we dropped the support for
trusty, but luminous still supports trusty. while trusty ships JDK-7,
where javac does not support "-h" option yet. so we need to handle this
case. "-h" was introduced in Java SE 8, see
https://docs.oracle.com/javase/8/docs/technotes/guides/javac/index.html
.

cmake: bump target jdk to 1.7

jdk 1.7 was released on 2011. For CentOS 6/7, both have openjdk
1.8. For Debian 8(jessie), Ubuntu 14.04(trusty) have openjdk 1.7.
These are the oldest distributions officially built for during
luminous release.

Since jdk 1.9/9, the minimal supported version is 1.6/6.
jdk 1.0/10(recently released) only supports 1.7/7.
http://openjdk.java.net/jeps/182

Fixes: http://tracker.ceph.com/issues/23458
Signed-off-by: Shengjing Zhu <zhsj@umcloud.com>
(cherry picked from commit 7c511d79240ac2ab20a3f9360d90a766ee1b9682)

qa/restful: Test pg_num/pgp_num modifications

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 059588e758d4dad429012b8c452926781d6e5fea)

restful: Support auid pool argument

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 880c53c681e2b4fbf22160fcf34d7de527832d2a)

Conflicts:
src/pybind/mgr/restful/common.py: trivial resolution

restful: Set the value of the argument

Previously, we forgot to set the value of the argument for the second
group of pool variables.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 64ac3af77e0e6c81b842f72678829a15e1a60087)

restful: Fix jsonification

The CommandsRequest.waiting is an array of arrays, not an array of
command results and so it cannot be represented that way. This commit
fixes that by outputting the array in json diretly.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 7005c6c0f68d82776c11cdfdd1a56e06f27d5e66)

ceph-disk: write log to /var/log/ceph not to /var/run/ceph

Fixes: http://tracker.ceph.com/issues/24041
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e0856bd7e78828ea352290bd325aa4f5e06c3029)

qa/workunits/rbd: potential race in mirror disconnect test

(due to a typo in get_image_id command arg)

Fixes: http://tracker.ceph.com/issues/23938
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 5bc1d4a51a5a3c1e887c3c1054ed17b5f15ce2ed)

mds: properly check auth subtree count in MDCache::shutdown_pass()

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/23813
(cherry picked from commit cb60472bd2803878c04dbb961fd7f7f54707f25b)

mds: properly trim log segments after scrub repairs something

Before calling MDLog::trim_all(), we need to make sure mdlog get flushed
Otherwise, there can be zero expiring log segments. This can cause
MDSGatherBuilder to malfunction. (MDSGatherBuilder::activate() does not
execute empty MDSGatherBuilder's finish context)

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
http://tracker.ceph.com/issues/23880
(cherry picked from commit 5be40f35c3365265b76a12a3bd0e17c3f8761bb9)

Merge pull request #21740 from pdvian/wip-23881-luminous

luminous: os/bluestore/bluefs_types: make block_mask 64-bit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21752 from pdvian/wip-23904-luminous

luminous: osdc/Objecter: prevent double-invocation of linger op callback

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21778 from pdvian/wip-23924-luminous

luminous: ceph_test_rados_api_aio: fix race with full pool and osdmap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21737 from pdvian/wip-23870-luminous

luminous: osd: publish osdmap to OSDService before starting wq threads

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21518 from smithfarm/wip-23784-luminous

luminous: osd: Warn about objects with too many omap entries

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #21655 from liewegas/wip-23786

luminous: qa: disable -Werror when compiling env_librados_test

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21760 from badone/wip-summarise-osds-when-issuing-slow-request-warning-luminous

luminous: mon/PGMap: Summarise OSDs in blocked/stuck requests

Reviewed-by: John Spray <john.spray@redhat.com>

mon/PGMap: Summarise OSDs in blocked/stuck requests

This fix is luminous specific since this code was changed in Mimic to
use the new SLOW_OPS errors.

Fixes: https://tracker.ceph.com/issues/23205
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

client: use common interp of st_nlink for dirs

Apparently some applications use this (like mail servers) and since it's
trivial to support, let's do it. Idea is that st_nlinks for a directory is
either 0 (it is unlinked) or 2 + the number of sub-directories (which have ..
parent links).

Fixes: https://tracker.ceph.com/issues/23873
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 67c7e4619188cfd684e59e3cb538948ae8f2b74c)

Conflicts:
        src/client/Inode.cc

qa/workunits/rados/test_large_omap_detection: Scrub pgs instead of OSDs

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit aba1e74e03c753366009996dedcb59ef83e155de)

osd: Add a flag to ScrubMap to signal check needed

If we know none of the scrub maps have errors early in
be_large_omap_check() we can return without doing a lot of unnecessary
work.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 2094fda7655ddd82b56850b3ded961a8c5c66966)

osd: Warn about objects with too many omap entries

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 71bf04775bef90c9291bd825da626bc9de6f9ec1)

Conflicts:
src/osd/osd_types.h (trivial resolution)

osd: Move creation of 'master_set' to scrub_compare_maps

Moving it to scrub_compare_maps allows us to pass it to multiple
subsequently called functions.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 63ef97b3db81c0c725119943c8594a4a0d12154e)