Merge pull request #16365 from jcsp/wip-osd-marked-me-down

osd: rephrase "wrongly marked me down" clog message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: update whitelist for "wrongly marked me down"

Signed-off-by: John Spray <john.spray@redhat.com>

osd: rephrase "wrongly marked me down" clog message

Rephrase as "Monitor daemon marked osd.<id> down, but it is
still running", which is more descriptive.  Demote the
epoch information to DEBUG level.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #16326 from liewegas/wip-weight-set

crush,mon: add weight-set introspection and manipulation commands

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com

Merge pull request #16360 from dillaman/wip-rbd-mirror-service-daemon

mgr/dashboard: rbd mirroring status page

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16529 from Liuchang0812/wip-20750

mgr/status: row has incorrect number of values

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16503 from renhwztetecs/renhw-wip-mon-rebuild-doc

doc/mon: fix ceph-authtool command in rebuild mon's sample

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16523 from tchaikov/wip-ceph-disk-asok-dir

tests: add setup/teardown for asok dir

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

mgr/status: row has incorrect number of values

Fixes: http://tracker.ceph.com/issues/20750
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #16507 from scienceluo/patch-1

Update .organizationmap

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc/mon: fix ceph-authtool create keyring

if use "ceph-authtool keyring -n mon. --cap mon allow 'allow *'"
as err:
[root]# ceph-authtool keyring -n mon. --cap mon allow 'allow *'
ceph-authtool: unexpected 'allow *'

Excessive "allow" is the cause of the error

Signed-off-by huanwen ren <ren.huanwen@zte.com.cn>

vstart.sh: do not try to remove default asok_dir

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: add setup/teardown for asok dir

ceph-disk.sh has its own setup/teardown, so update them also. so asok
files will have their parent directory when being created.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16502 from weiqiaomiao/wip_weiqm_unlock_bucket

rgw: should unlock when reshard_log->update() reture non-zero in RGWB…
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: should unlock when reshard_log->update() reture non-zero in RGWBucketReshard::execute()

Signed-off-by: Wei Qiaomiao <wei.qiaomiao@zte.com.cn>

Merge pull request #16508 from liewegas/wip-mgr-lock-cycle

mgr: fix lock cycle

Merge pull request #16488 from liewegas/wip-skewed

common/options: remove mon_warn_osd_usage_min_max_delta from options.cc too

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16490 from jdurgin/wip-20739

osd/PrimaryLogPG: skip deleted missing objects in pg[n]ls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16501 from liewegas/wip-fix-mgr-commands

mon: do not dereference empty mgr_commands

Merge pull request #16500 from liewegas/wip-compact-sudo

qa/workunits/cephtool/test.sh: add sudo for daemon compact

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/PyModules: fix lock name

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: fix lock cycle

This breaks a lock cycle with DaemonState::lock and PyModules::lock.

Signed-off-by: Sage Weil <sage@redhat.com>

Update .organizationmap

The list of contributors to ZTE looks better if it is in alphabetical order.

Signed-off-by: Luo Kexue luo.kexue@zte.com.cn

Merge pull request #15991 from dillaman/wip-rbd-auth-profile

mon,osd: new rbd-based cephx cap profiles

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16432 from liewegas/wip-mgr-daemonstate-lock

mgr: add per-DaemonState lock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16460 from liewegas/wip-mgr-metadata

mon: add mgr metdata commands, and overall 'versions' command for all daemon versions

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16465 from xiexingguo/wip-shutup-noscrub-warn

mon: stop issuing not-[deep]-scrubbed warnings if disabled

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16477 from tchaikov/wip-health-mon

mon/HealthMonitor: trigger a proposal if stat updated

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16478 from xiexingguo/wip-drop-empty-message

mon/HealthMonitor: avoid sending unnecessary MMonHealthChecks to leader

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16480 from jcsp/wip-rm-convert

common: remove config opt conversion utility

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mon: do not dereference empty mgr_commands

This may be empty before the initial quorum commits the static command
list.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephtool/test.sh: add sudo for daemon compact

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR 16390 into master

* refs/remotes/upstream/pull/16390/head:
mds: cleanup with unique_ptr
mds: add to get version in the mds_commands
mds: cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR 16379 into master

* refs/remotes/upstream/pull/16379/head:
qa: fix MDS_CLIENT_RECALL copy error

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16337 into master

* refs/remotes/upstream/pull/16337/head:
mds: clear dirty rstat flag on non-auth inode when finishing scatter update

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16280 into master

* refs/remotes/upstream/pull/16280/head:
Client: mkdirs bugs when two clients send mkdir request for a same dir

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16278 into master

* refs/remotes/upstream/pull/16278/head:
MDSMonitor: show laggy MDSs at higher debug level

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16270 into master

* refs/remotes/upstream/pull/16270/head:
mds: improve wording of mdsmap dne logging

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16232 into master

* refs/remotes/upstream/pull/16232/head:
MDS: update the mlogger of mds in function check_memory_usage

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16231 into master

* refs/remotes/upstream/pull/16231/head:
Client: get rid of variable "butnot" in "client::get_caps"

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16226 into master

* refs/remotes/upstream/pull/16226/head:
qa: wait for OSDMap to propagate for snap purge

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16149 into master

* refs/remotes/upstream/pull/16149/head:
mds: misc cleanup

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 15942 into master

* refs/remotes/upstream/pull/15942/head:
client: add unlock_fh_pos when abort

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crush: assume weight_set != null imples weight_set_size > 0

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: skip deleted missing objects in pg[n]ls

Fixes: http://tracker.ceph.com/issues/20739
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

PendingReleaseNotes: added blurb for new RBD cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: expand profile caps upon construction to avoid potential race

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: rbd-mirror tests should use 'mirror' user

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/suites/rbd: mirroring tests should use rbd cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: devstack test should use auth profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: include details for new RBD cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon,osd: drop the "allow" prefix for profile caps

The use of the a profile implies that it sets everything up. If
support for deny is ever added, it doesn't make much sense to
deny a profile.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: new 'rbd'/'rbd-read-only' osd cap profiles

The 'rbd' profile provides read-only class access on all pools
to the 'rbd_children' object and write access to the optionally
specified pool. The 'rbd-read-only' profile does as its name
implies.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: primitive cephx osd profile cap support

The two new example profiles are read-only and read-write

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: added 'rbd' profile

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: support regex-based restrictions on command caps

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16445 from tchaikov/wip-16895

test: create asok files in a temp directory under $TMPDIR

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

crush/CrushWrapper: only allocate weight_set if non-empty

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/mon/crush_ops.sh: fix in-use rule removal test

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/mon/crush_ops: require luminous clients for test

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: creating rules

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: document weight set commands

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map*: restructure and clean up

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: warn off manual crush map edits

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: update crush location section

Fix various inaccuracies, simplify.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: prune intro

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: 'osd crush rule ls' plaintext output

More suitable for scripting.

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: remove_item_under: weight down weight-sets

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: remove_item_under whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: link_bucket whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: detach_bucket: weight down weight_sets

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: detach_bucket uninline

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: remove_item: weight down weight-sets

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: create_or_move_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: remove_item, bucket_remove_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: update_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: adjust_item_weight* whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: bucket_add_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: add_bucket: populate weight-sets

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: get_choose_args_positions helper

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: insert_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

mon: 'osd crush weight-set {ls,dump,create[-compat],rm[-compat],reweight[-compat]}' commands

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: choose_arg_update_item_weight

Signed-off-by: Sage Weil <sage@redhat.com>

common/cmdparse: allow vector<double>

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: {create,rm}_choose_args(index)

Signed-off-by: Sage Weil <sage@redhat.com>

test/mon/osd-crush: remove xml osd-crush-tree.rng test

I don't know how to generate or update this, and am not *that* interested
in learning more xml.

Signed-off-by: Sage Weil <sage@redhat.com>

test/cli/crushtool: fix osd tree output

Signed-off-by: Sage Weil <sage@redhat.com>

test/ceph_objectstore_tool.py: fix parsing of osd tree output

Signed-off-by: Sage Weil <sage@redhat.com>

crush: dump weight-set values with 'osd crush tree'

For plain, include a column for each pool.  Show only the first weight if
it is positional.

For formatted, include the full set of weights.

As far of this we switch 'ceph osd crush tree' to have a plaintext mode.

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushTreeDumper: include parent in Item

This gives us info we need for the weight_sets...

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: make a compat/default choose_args

We can make a backward-compatible crush map for legacy clients only if we
have a single global choose_args for the whole map.  Add a special
slot to store that of -1.  (Normally the index is the pool id, so -1 is
unused by ceph.)

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: have_choose_args(index)

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: has_non_straw2_buckets()

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: whitespace cleanup in decode

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: fix CHOOSE_ARGS feature detection in get_features()

We need to include the mask in the CEPH_FEATURES_CRUSH value, and | in the
FEATUREMASK later, because this is a second-generation feature bit usage.

This fixes get_min_compat_client(), which was returning 'jewel' when
incompat choose_args were in use instead of 'luminous'.  Now it is correct.

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: fix decoding of choose_args weight-set

Assert that ids_size matches the bucket while we are at it.

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: remove mon_warn_osd_usage_min_max_delta from options.cc too

Signed-off-by: Sage Weil <sage@redhat.com>

test: create asok files in a temp directory under $TMPDIR

to shorten the pathname of unix domain socket created for admin socket,
so it does not exceed the limit of 107 on GNU/Linux:

* ceph-helper.sh: the temp directory is named ${TMPDIR:-/tmp}/ceph-asok.$$
* vstart.sh: the temp directory is named `mktemp -u -d "${TMPDIR:-/tmp}/ceph-asok.XXXXXX"`

Fixes: http://tracker.ceph.com/issues/16895
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16408 from yanghonggang/master

osd: fix pg ref leaks when osd shutdown

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16449 from smithfarm/wip-obsolete-xio

build/ops: rpm: Drop legacy libxio support

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>