mds: issue new caps when sending reply to client

After Locker::issue_new_caps() adds new Capability data struct,
do not issue caps immediately. Let CInode::encode_inodestate()
do the job instead. This can avoid various races that early reply
is not allowed, caps that haven't been sent to client gets revoked.

Fixes: http://tracker.ceph.com/issues/19635
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 799703a4acb49db0b6cc99a23e4326767e694c3a)

Merge pull request #14752 from cbodley/wip-19474

jewel: rgw: allow system users to read SLO parts

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14672 from smithfarm/wip-19334-jewel

jewel: MDS heartbeat timeout during rejoin, when working with large amount of caps/inodes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14677 from smithfarm/wip-19665-jewel

jewel: mds: C_MDSInternalNoop::complete doesn't free itself

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15466 from smithfarm/wip-19762-jewel

jewel: cephfs: non-local quota changes not visible until some IO is done

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14700 from smithfarm/wip-19709-jewel

jewel: mds: enable start when session ino info is corrupt

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14685 from smithfarm/wip-19675-jewel

jewel: cephfs: Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14684 from smithfarm/wip-19673-jewel

jewel: cephfs: mds is crushed, after I set about 400 64KB xattr kv pairs to a file

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14683 from smithfarm/wip-19671-jewel

jewel: cephfs: MDS assert failed when shutting down

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14682 from smithfarm/wip-19668-jewel

jewel: cephfs: MDS goes readonly writing backtrace for a file whose data pool has been removed

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14679 from smithfarm/wip-19666-jewel

jewel: cephfs: The mount point break off when mds switch hanppened.

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14676 from smithfarm/wip-19619-jewel

jewel: cephfs: MDS server crashes due to inconsistent metadata.

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14674 from smithfarm/wip-19482-jewel

jewel: cephfs: No output for ceph mds rmfailed 0 --yes-i-really-mean-it command

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14671 from smithfarm/wip-19044-jewel

jewel: tests: buffer overflow in test LibCephFS.DirLs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14670 from smithfarm/wip-18949-jewel

jewel: mds: avoid reusing deleted inode in StrayManager::_purge_stray_logged

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14669 from smithfarm/wip-18900-jewel

jewel: cephfs: Test failure: test_open_inode

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14668 from smithfarm/wip-18705-jewel

jewel: mds: fragment space check can cause replayed request fail

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14698 from smithfarm/wip-19677-jewel

jewel: cephfs: ceph-fuse does not recover after lost connection to MDS

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14766 from smithfarm/wip-19757-jewel

jewel: rgw: fix failed to create bucket if a non-master zonegroup has a single zone

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14787 from linuxbox2/jewel-rgw-shard-limit-ck

jewel: rgw: add bucket size limit check to radosgw-admin

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14789 from mdw-at-linuxbox/wip-jewel-rgw-rvk

jewel: rgw: swift: disable revocation thread if sleep == 0 || cache_size == 0

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14815 from smithfarm/wip-19786-jewel

jewel: rgw: failure to create s3 type subuser from admin rest api

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15312 from theanalyst/wip-20078

jewel: rgw: only append zonegroups to rest params if not empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15382 from theanalyst/wip-mem-leak2

jewel: rgw:fix memory leaks in data/md sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Client.cc: adjust Client::_getattr calls

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/cephfs: use getfattr/setfattr helpers

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit dd43d3bc646aeab88486b0963fc83de0b18800c4)

Conflicts:
        qa/tasks/cephfs/test_data_scan.py: difference in the
            self._mount.run_shell() call in NonDefaultLayout::write (which is
            being dropped by this commit) - in jewel it has "sudo", and in
            master it doesn't

tasks/cephfs: fix race while mounting

This could fail if the mount hadn't finished
coming up.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit adfb757c898a80f18c15dafd02e29840c5931c87)

qa: add test for reading quotas from different clients

Fixes: http://tracker.ceph.com/issues/17939
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 61617f8f10a6322603a9add77980865cd972ef97)

client: _getattr on quota_root before using in statfs

...so that after someone adjusts the quota settings
on an inode that another client is using as its mount root,
the change is visible immediately on the other client.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 3d25941aadd223669448d0f5d3c0bd1fefa72308)

client: getattr before read on ceph.* xattrs

Previously we were returning values for quota, layout
xattrs without any kind of update -- the user just got
whatever happened to be in our cache.

Clearly this extra round trip has a cost, but reads of
these xattrs are fairly rare, happening on admin
intervention rather than in normal operation.

Fixes: http://tracker.ceph.com/issues/17939
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 532dc4b68e538c189ef828f67cecd0d647a62250)

Merge pull request #15416 from dzafman/wip-20126

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd: Object level shard errors are tracked and used if no auth available

Shards with object mismatch are tracked to mark them inconsistent
Fix test because storing omap_digest in object_info not behaving as before

Fixes: http://tracker.ceph.com/issues/20089
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1cacbea763c7aabfeaaf4bd5e878301044184117)

Conflicts:
src/test/osd/osd-scrub-repair.sh (no alloc_hint in object_info)

rgw:fix memory leaks

Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>
(cherry picked from commit 73e5be2b6133cf4caa0e5e5c8c9eae748b785dbf)

Merge pull request #15360 from liewegas/wip-jewel-master-mixed

qa/workunits/rados/test-upgrade-*: whitelist tests for master

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits/rados/test-upgrade-*: whitelist tests for master

The jewel-x upgrade test now runs this script against a mixed cluster on
a machine with code from master installed.  That means we have to skip
any new tests that will fail on a mixed cluster.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15208 from liewegas/wip-sortbitwise-jewel

mon: fix 'sortbitwise' warning on jewel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14851 from yehudasa/wip-rgw-support-ragweed-jewel

jewel: rgw: add apis to support ragweed suite

Reviewed-by: Nathan Cutler <ncutler@suse.com>

rgw: rest conn functions cleanup, only append zonegroup if not empty

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 4e41af19846db75081cb0ddb7b33dc2bb9321ace)

rgw: rest and http client code to use param vectors

Replaces param/header lists with vectors. In these cases, we're only ever
adding to the back of the list, so a vector should be more efficient.
Also moves param_pair_t/param_vec_t higher up the include chain for
cleaner function signatures.

Signed-off-by: Karol Mroz <kmroz@suse.com>
(cherry picked from commit d4a2527872e0f5c3ae2874bb7d0ff459ae83cfd4)

Conflicts:
src/rgw/rgw_http_client.cc
trivial ws conflict on rebase

Merge pull request #14765 from smithfarm/wip-18972-jewel

jewel: ceph-disk does not support cluster names different than 'ceph'

Reviewed-by: Alfredo Deza <adeza@redhat.com>

qa/suites/rados/singleton-nomsgr/*: set sortbitwise after upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: fixup sortbitwise flag warning

"ceph -s" does not report warning when using
command "ceph osd unset sortbitwise" to drop
sortbitwise flag.
we should use "osdmap.get_up_osd_features() &
CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT"
instead of "(osdmap.get_features(CEPH_ENTITY_TYPE_OSD, NULL) &
CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT)",
because osdmap.get_features only get local "features"

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
(cherry picked from commit c25ee187e28724846d0011cd8145e16956d3636e)

mon: remove config option to disable no sortbitwise warning

We'll require this soon.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 04e3319e8aecde9ca58ccb7c89016f9079c7d657)

Merge pull request #13450 from dreamhost/wip-18887-jewel

jewel: msg: IPv6 Heartbeat packets are not marked with DSCP QoS - simple messenger

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

msg/simple/Pipe: manual backport of fix in PR#14795

Manual backport of errno fixup from PR#14795
(6f1037e22c2a304795895498cdc955e0ef80f8e3), as noted by
https://github.com/ceph/ceph/pull/13450#discussion_r114696885.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #14667 from smithfarm/wip-18699-jewel

jewel: client: fix the cross-quota rename boundary check conditions

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #14332 from shinobu-x/wip-19396-jewel

 jewel: Objecter::epoch_barrier isn't respected in _op_submit()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14204 from dzafman/wip-18533-jewel

jewel: core: two instances of omap_digest mismatch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13884 from shinobu-x/wip-19119-jewel

jewel: pre-jewel "osd rm" incrementals are misinterpreted

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13647 from mslovy/wip-19083-jewel

jewel: osd: preserve allocation hint attribute during recovery

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #14791 from smithfarm/wip-19774-jewel

jewel: osd: promote throttle parameters are reversed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14763 from smithfarm/wip-19562-jewel

jewel: api_misc: [  FAILED  ] LibRadosMiscConnectFailure.ConnectFailure

Reviewed-by: Sage Weil <sage@redhat.com>

rgw: new rest api to retrieve object layout

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 2768583dc486109e49d209243675b99fdd39e92c)

rgw: rest api to read zone config params

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit a220a2efbfd675e6abf14ae33c21005bcbf6dadf)

Merge pull request #14481 from dillaman/wip-19468-jewel

jewel: librbd: is_exclusive_lock_owner API should ping OSD

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14666 from smithfarm/wip-19612-jewel

jewel: librbd: Issues with C API image metadata retrieval functions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14664 from smithfarm/wip-19325-jewel

jewel: rbd: [api] temporarily restrict (rbd_)mirror_peer_add from adding multiple peers

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14694 from ceph/wip-bp-systemd

[backport] qa/tasks: systemd test backport to jewel

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13544 from shinobu-x/wip-18932-jewel

jewel: tests: 'ceph auth import -i' overwrites caps, should alert user before overwrite

Reviewed-by: Kefu Chai <kchai@redhat.com>

filestore, tools: Fix logging of DBObjectMap check() repairs

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1704f62c0831e6b07138f7dd14a89fef3c9ed2c1)

osd: Simplify DBObjectMap by no longer creating complete tables

Bump the version for new maps to 3
Make clone less efficient but simpler
Add rename operation (use instead of clone/unlink)
For now keep code that understands version 2 maps

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 738156a99ed1caf61e5a8230eb8048360056c08e)

No ghobject_t::operator>() so use Kraken cmp_bitwise() instead
Need to use MIN_GHOBJ/MAX_GHOBJ instead of std::min/std::max

ceph-osdomap-tool: Fix seg fault with large amount of check error output

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1dda0411f4fbb14ce1e0062da9f14ec3af505d39)

osd: Add automatic repair for DBObjectMap bug

Add repair command to ceph-osdomap-tool too

Under some situations the previous rm_keys() code would
generated a corrupt complete table.  There is no way to
figure out what the table should look like now.  By removing
the entries we fix the corruption and aren't much worse off
because the corruption caused some deleted keys to re-appear.

This doesn't breaking the parent/child relationship during
repair because some of the keys may still be contained
in the parent.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 4cd3c74c928a32e065ed9543d6c91d8718a6ae3d)

Conflicts:
src/os/filestore/DBObjectMap.h (trivial)

ceph-osdomap-tool: Fix tool exit status

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 666f14ed90655a2d1bedde8561949625db7a9e6c)

DBObjectMap: rewrite rm_keys and merge_new_complete

Leverage the updated in_complete_region and needs_parent to simplify
these methods.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit c4dffb68eaafe724f7fdae93a4285a7f8003ea29)

DBObjectMap: strengthen in_complete_region post condition

Previously, in_complete_region didn't guarantee anything about
where it left complete_iter pointing.  It will be handy for
complete_iter to be pointing at the lowest interval which ends
after to_test.  Make it so.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 97b35f4d7d4862da4b6f50ecaef0d292a671fd04)

DBObjectMap: fix next_parent()

The previous implementation assumed that
lower_bound(parent_iter->key()) always leaves the iterator
on_parent().  There isn't any guarantee, however, that that
key isn't present on the child as well.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 74a7631d0938d7b44894f022224eab10a90d5cec)

test_object_map: add tests to trigger some bugs related to 18533

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit f131dbcf5bb17107c029f942a57e9bf4432a26ee)

test: Add ceph_test_object_map to make check tests

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0e97a01bd7291458881ee53cece2d887f6333669)

ceph-osdomap-tool: Add --debug and only show internal logging if enabled

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 5fb2b2d13953979e5da9f571ab8c4b0b510b8368)

osd: DBOjectMap::check: Dump complete mapping when inconsistency found

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fcf1e17c645e8fad5216c3e59627c817e5c858c7)

test_object_map: Use ASSERT_EQ() for check() so failure doesn't stop testing

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 053a273cbc02d6902a4bb1f11db1ea946498df3a)

tools: Check for overlaps in internal "complete" table for DBObjectMap

Changed check to return an error count and fix tool error message

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e5e8eb962db6187ea19b96ba29ac83469c90b4ea)

Conflicts:
src/os/filestore/DBObjectMap.h (trivial)

tools: Add dump-headers command to ceph-osdomap-tool

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f4101591ad701a62fe027c4744ca8ea505f44bdc)

Conflicts:
src/os/filestore/DBObjectMap.h (trivial)

tools: Add --oid option to ceph-osdomap-tool

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2d94889e9ee3359017b1efd560f3557ce03ccee6)

osd: Remove unnecessary assert and assignment in DBObjectMap

Fix and add comment(s)

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 937e6a03ea4692cc44d53faa0615f8e808c9eb03)

rgw: add suport for creating S3 type subuser of admin rest api

Fixes: http://tracker.ceph.com/issues/16682
The original code cannot support create s3 type subuser of admin rest api as when i execute the following command:

  ./s3curl.pl --id=personal --put -- http://radosgw.h3c.com:8000/admin/user?subuser\&uid=yrf2\&subuser=yrf2:yrf1\&key-type=s3 -v

it would return msg as follows :

 < HTTP/1.1 403 Forbidden
< Date: Thu, 14 Jul 2016 07:04:40 GMT
* Server Apache/2.4.7 (Ubuntu) is not blacklisted
< Server: Apache/2.4.7 (Ubuntu)
< x-amz-request-id: tx00000000000000006608f-0057873988-8551-slave
< Accept-Ranges: bytes
< Content-Length: 114
< Content-Type: application/json
<
* Connection #0 to host slave.com left intact
{"Code":"InvalidAccessKeyId","RequestId":"tx00000000000000006608f-0057873988-8551-slave","HostId":"8551-slave-us"}

 But i have modified the codes for support it ,and it will return actual msg as follows :

  "subusers": [
        {
            "id": "yrf2:yrf1",
            "permissions": "<none>"
        }
    ],
    "keys": [
        {
            "user": "yrf2",
            "access_key": "B46PXYFEWUX0IMHGKP8C",
            "secret_key": "2JYxywXizqwiiMd74UXrJdSJMPNlBtYwF5z8rNvh"
        },
        {
            "user": "yrf2:yrf1",
            "access_key": "INO55WXJ7JQ1ZZGSAB6B",
            "secret_key": "GgCKEfF9hArV2hglunbO7KtvKZnbhmsDpqjSj5DL"
        }
    ],

Please check it ,thanks .

Signed-off-by: snakeAngel2015 <li.jieA@h3c.com>
(cherry picked from commit 6535f6ad2137ee55bf5531e865c05aa10bd39bd0)

Merge pull request #14809 from tchaikov/wip-18193-jewel

jewel: tests: test/librados/tmap_migrate: g_ceph_context->put() upon return

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #14701 from smithfarm/wip-18193-jewel

jewel: core: transient jerasure unit test failures

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14776 from linuxbox2/jewel-pullup-civet-chunked

[DNM] jewel: pullup civet chunked

Merge pull request #14416 from smithfarm/wip-19557-jewel

jewel: tests: upgrade/hammer-x failing with OSD has the store locked when Thrasher runs ceph-objectstore-tool on down PG

Reviewed-by: Kefu Chai <kchai@redhat.com>

civetweb:  pullup chunked encoding by Marcus

Fixes: http://tracker.ceph.com/issues/19736
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

os: make zero values noops for set_alloc_hint() in FileStore

Signed-off-by: yaoning <yaoning@unitedstack.com>
(cherry picked from commit e2ec24f61b55457caccefecd56f9f08b98264802)

test/librados/tmap_migrate: g_ceph_context->put() upon return

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflict: test/librados/tmap_migrate.cc
  this change is not cherry-picked from master, because tmap_migrate was
removed in master. so we are applying the same change in
cb1cda96713b2ec0f6418c4cbe3d964c2020729c to this test.

Fix reveresed promote throttle default parameters.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 793ceac2f3d5a2c404ac50569c44a21de6001b62)

rgw: swift: disable revocation thread if sleep == 0 || cache_size == 0

Keystone tokens can be revoked.  This causes them to fail
validation.  However, in ceph, we cache them.  As long as
they're in the cache we trust them.  To find revoked tokens
there's a call OSI-PKI/revoked but that's only useful for
pki tokens.  Installations using fernet/uuid may not even
have the proper credentials to support the call, in which
case the call blows up in various ways filling up logs
with complaints.

This code makes the revocation thread optional; by disabling it,
the complaints go away.  A further fix is in the works
to use other more modern calls available in modern keystone
installations to properly deal with non-PKI/PKIZ tokens.

(NB: jewel has this logic in src/rgw/rgw_swift.cc not in src/rgw/rgw_keystone.h)

To disable the revocation thread, use at least one of these:
        rgw_keystone_token_cache_size = 0
using this will cause tokens to be validated on every call.
You may instead want to set
        rgw_keystone_revocation_interval = 0
using just this will disable the revocation thread,
but leaves the cache in use.  That avoids the extra
validation overhead, but means token revocation won't
work very well.

Fixes: http://tracker.ceph.com/issues/9493
Fixes: http://tracker.ceph.com/issues/19499
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit 003291a8cbca455c0e8731f66759395a0bb1f555)

Merge pull request #14686 from smithfarm/wip-19686-jewel

jewel: osd: Give requested scrubs a higher priority

Reviewed-by: David Zafman <dzafman@redhat.com>

rgw: add bucket size limit check to radosgw-admin

The change adds a new list of all buckets x all users, with
fields for bucket name, tenant name, current num_objects,
current num_shards, current objects per shard, and the
corresponding fill_status--the latter consisting of 'OK',
'WARN <n>%', or 'OVER <n>%.'

The warning check is relative to two new tunables.  The threshold
max objects per shard is set as rgw_bucket_safe_max_objects_per_shard,
which defaults to 100K.  The value rgw_bucket_warning_threshold is
a percent of the current safe max at which to warn (defaults to
90% of full).

From review:

* fix indentation (rgw_admin)
* if user a user_id is provided, check only buckets for that user
* update shard warn pct to be pct-of-fill (not 100 - pct-of-fill)
* print only buckets near or over per-shard limit, if --warnings-only
* s/bucket limitcheck/bucket limit check */
* sanity shard limit should be 90, not 10 (because that changed)
* fixes for memleaks and other points found by cbodley

Fixes: http://tracker.ceph.com/issues/17925
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 7bc144ce36fedc16a3dedc54598b0d75fb8c68bc)

Merge pull request #14605 from asheplyakov/19476-jewel

jewel: rgw: don't return skew time in pre-signed url

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14660 from smithfarm/wip-19478-jewel

jewel: rgw: zonegroupmap set does not work

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14607 from asheplyakov/19607-jewel

jewel: rgw: multisite: fetch_remote_obj() gets wrong version when copying from remote

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14587 from asheplyakov/19617-jewel

jewel: mon/MonClient: make get_mon_log_message() atomic

Reviewed-by: Sage Weil <sage@redhat.com>

test/ceph_crypto: do not read ceph.conf in global_init()

ForkDeathTest.MD5 expect an empty output while global_init() complains
if ceph.conf is missing if 0 is passed in as the `flags`. this test
passes if ceph.conf is in current working directory, but jenkins does
not prepare this file for this test.

Fixes: http://tracker.ceph.com/issues/18128
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c72a2271a8012a66d7bbccf5766a73da5bb878d6)

Conflicts:
        src/test/ceph_crypto.cc - jewel does not have
             5af29540675b674c1985ff98b28a783ed124acf6

tests: fix erasure-code premature deallocation of cct

The setup function returns before the run function, the cct variable
must be a data member, not a local variable that gets de-allocated
before run() starts.

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit efa1e54362423d4cfd1541fb8c68237b7b9ebbe3)

rbd-nbd: no need create asok file for unmap and list-mapped commands.

Fixes: http://tracker.ceph.com/issues/17951
Signed-off-by: Pan Liu <liupan1111@gmail.com>
(cherry picked from commit 72352653d585ef89043a4ece371b5c0cb3f6f32a)

rbd-nbd: restart parent process logger after forking

Fixes: http://tracker.ceph.com/issues/18070
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 29baf254d72cc593572b5a6215360ba51e3be198)

crushtool: do not release g_ceph_context at exit

it is but a work around of occasionally timeout.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d305cc51b18cbf4b2757bbacb5d43324461306a9)

common,test: g_ceph_context->put() upon return

prior to this change, global_init() could create a new CephContext
and assign it to g_ceph_context. it's our responsibilty to release
the CephContext explicitly using cct->put() before the application
quits. but sometimes, we fail to do so.

in this change, global_init() will return an intrusive_ptr<CephContext>,
which calls `g_ceph_context->put()` in its dtor. this ensures that
the CephContext is always destroyed before main() returns. so the
log is flushed before _log_exp_length is destroyed.

there are two cases where global_pre_init() is called directly.
- ceph_conf.cc: g_ceph_context->put() will be called by an intrusive_ptr<>
  deleter.
- rgw_main.cc: global_init() is called later on on the success code
  path, so it will be taken care of.

Fixes: http://tracker.ceph.com/issues/17762
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cb1cda96713b2ec0f6418c4cbe3d964c2020729c)

Conflicts:
src/ceph_fuse.cc
src/ceph_mgr.cc
src/global/global_init.cc
src/rgw/rgw_main.cc
src/test/compressor/test_compression.cc
src/test/compressor/test_compression_plugin.cc
src/test/compressor/test_compression_plugin_snappy.cc
src/test/compressor/test_compression_plugin_zlib.cc
src/test/compressor/test_compression_snappy.cc
src/test/compressor/test_compression_zlib.cc
src/test/erasure-code/TestErasureCode.cc
src/test/erasure-code/TestErasureCodeExample.cc
src/test/erasure-code/TestErasureCodeIsa.cc
src/test/erasure-code/TestErasureCodeJerasure.cc
src/test/erasure-code/TestErasureCodeLrc.cc
src/test/erasure-code/TestErasureCodePlugin.cc
src/test/erasure-code/TestErasureCodePluginIsa.cc
src/test/erasure-code/TestErasureCodePluginJerasure.cc
src/test/erasure-code/TestErasureCodePluginLrc.cc
src/test/erasure-code/TestErasureCodePluginShec.cc
src/test/erasure-code/TestErasureCodeShec.cc
src/test/erasure-code/TestErasureCodeShec_thread.cc
src/test/fio/fio_ceph_objectstore.cc
src/test/librados/misc.cc
src/test/mon/PGMap.cc
src/test/msgr/test_async_networkstack.cc
src/test/msgr/test_userspace_event.cc
src/test/objectstore/Allocator_test.cc
src/test/objectstore/BitAllocator_test.cc
src/test/objectstore/test_bluefs.cc
src/test/objectstore/test_bluestore_types.cc
src/test/objectstore/test_memstore_clone.cc
src/test/osd/TestPGLog.cc
src/test/rgw/test_http_manager.cc
src/test/rgw/test_rgw_compression.cc
src/test/test_mempool.cc
src/tools/rados/rados.cc

Merge pull request #14665 from smithfarm/wip-19610-jewel

jewel: [librados_test_stub] cls_cxx_map_get_XYZ methods don't return correct value

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14653 from smithfarm/wip-19662-jewel

jewel: rgw_file: fix event expire check, don't expire directories being read

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>