ceph-disk: s|/dev/fd/0|-|

3a4931b allows ceph cli to use '-' to denote stdin/stdout, so use it
instead of /dev/fd/0.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/ceph-disk: add docs for ceph-disk

we might want to move part of it to user-facing docs.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16417 from fangyuxiangGL/rgw-hang

rgw: fix rgw hang when do RGWRealmReloader::reload after go SIGHUP

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16422 from theanalyst/rgw-lc-log-fix

rgw: fix asctime when logging in rgw_lc

Reviewed-by: Daniel Gryniewicz <dang@fprintf.net>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@linuxbox.com>

Merge pull request #16524 from dingdangzhang/master

msg/async: use auto iterator having more simple code and good performance

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16481 from nikitych/bug20706

ceph-disk: Fix for missing 'not' in *_is_diskdevice checks

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16247 from shangzhong/wip-20556

ceph-disk: change the lockbox partition number to 5

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13723 from ovh/bp-forced-recovery

osd/PG: make prioritized recovery possible

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16504 from Yan-waller/wip-walle-0721osdclients

osd: fix OpRequest and tracked op dump information

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16512 from liupan1111/wip-fix-latency

common: the latency dumped by "ceph osd perf" is not real

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16518 from liewegas/wip-20751

mon/OSDMonitor: remove zeroed new_state updates

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16365 from jcsp/wip-osd-marked-me-down

osd: rephrase "wrongly marked me down" clog message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: update whitelist for "wrongly marked me down"

Signed-off-by: John Spray <john.spray@redhat.com>

osd: rephrase "wrongly marked me down" clog message

Rephrase as "Monitor daemon marked osd.<id> down, but it is
still running", which is more descriptive.  Demote the
epoch information to DEBUG level.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #16326 from liewegas/wip-weight-set

crush,mon: add weight-set introspection and manipulation commands

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com

use auto iterator having more simple code and good performence

Signed-off-by: dingdangzhang <boqian.zy@alibaba-inc.com>

Merge pull request #16360 from dillaman/wip-rbd-mirror-service-daemon

mgr/dashboard: rbd mirroring status page

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16529 from Liuchang0812/wip-20750

mgr/status: row has incorrect number of values

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16503 from renhwztetecs/renhw-wip-mon-rebuild-doc

doc/mon: fix ceph-authtool command in rebuild mon's sample

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16523 from tchaikov/wip-ceph-disk-asok-dir

tests: add setup/teardown for asok dir

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

mgr/status: row has incorrect number of values

Fixes: http://tracker.ceph.com/issues/20750
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #16507 from scienceluo/patch-1

Update .organizationmap

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc/mon: fix ceph-authtool create keyring

if use "ceph-authtool keyring -n mon. --cap mon allow 'allow *'"
as err:
[root]# ceph-authtool keyring -n mon. --cap mon allow 'allow *'
ceph-authtool: unexpected 'allow *'

Excessive "allow" is the cause of the error

Signed-off-by huanwen ren <ren.huanwen@zte.com.cn>

vstart.sh: do not try to remove default asok_dir

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: add setup/teardown for asok dir

ceph-disk.sh has its own setup/teardown, so update them also. so asok
files will have their parent directory when being created.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16502 from weiqiaomiao/wip_weiqm_unlock_bucket

rgw: should unlock when reshard_log->update() reture non-zero in RGWB…
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: should unlock when reshard_log->update() reture non-zero in RGWBucketReshard::execute()

Signed-off-by: Wei Qiaomiao <wei.qiaomiao@zte.com.cn>

osd/OSDMap: fix Incremental::dump

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: remove zeroed new_state updates

These are interpreted as CEPH_OSD_UP in apply_incremental for legacy
reasons--we don't want them!

Fixes: http://tracker.ceph.com/issues/20751
Signed-off-by: Sage Weil <sage@redhat.com>

common: the latency dumped by "ceph osd perf" is not real

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

Merge pull request #16508 from liewegas/wip-mgr-lock-cycle

mgr: fix lock cycle

Merge pull request #16488 from liewegas/wip-skewed

common/options: remove mon_warn_osd_usage_min_max_delta from options.cc too

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16490 from jdurgin/wip-20739

osd/PrimaryLogPG: skip deleted missing objects in pg[n]ls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16501 from liewegas/wip-fix-mgr-commands

mon: do not dereference empty mgr_commands

Merge pull request #16500 from liewegas/wip-compact-sudo

qa/workunits/cephtool/test.sh: add sudo for daemon compact

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/PyModules: fix lock name

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: fix lock cycle

This breaks a lock cycle with DaemonState::lock and PyModules::lock.

Signed-off-by: Sage Weil <sage@redhat.com>

Update .organizationmap

The list of contributors to ZTE looks better if it is in alphabetical order.

Signed-off-by: Luo Kexue luo.kexue@zte.com.cn

osd: store op request source info

if we use command 'ceph daemon osd.# dump_historic_ops' to check the history
 tracked ops, the 'client_addr' is always dumped as '-' which result in clients
indistinguishable from each other.

"description": "osd_op(client.24123.0:328 3.36 3:6ec0d61b:::rbd_data.10472ae8944a.0000000000000289:head [write 4141056~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected e36)",
            "initiated_at": "2017-07-22 14:24:25.925633",
            "age": 4.893190,
            "duration": 0.272642,
            "type_data": {
                "flag_point": "commit sent; apply or cleanup",
                "client_info": {
                    "client": "client.24123",
                    "client_addr": "-",
                    "tid": 328
                },

It is because 'connection' was reset to nullptr in function '_unregistered'
before op has been inserted into history, so the 'get_source_addr' always return
empty entity addr.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

common: dump field name of 'flag_point' and 'client_info'

we used to open "type_data" section as a array (that's open_array_section),
some key field names can not be dumped out. use open_object_section to dump them
as following:

before this change:
------------------
"description": "osd_op(client.4202.0:4958 3.28 3:163e8348:::rbd_data.10472ae8944a.0000000000000164:head [write 1691648~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected e24)",
            "initiated_at": "2017-07-22 14:22:13.469324",
            "age": 17.595664,
            "duration": 0.460406,
            "type_data": [
                "commit sent; apply or cleanup",
                {
                    "client": "client.4202",
                    "client_addr": "-",
                    "tid": 4958
                },
-----------------
after:
-----------------
"description": "osd_op(client.24123.0:328 3.36 3:6ec0d61b:::rbd_data.10472ae8944a.0000000000000289:head [write 4141056~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected e36)",
            "initiated_at": "2017-07-22 14:24:25.925633",
            "age": 4.893190,
            "duration": 0.272642,
            "type_data": {
                "flag_point": "commit sent; apply or cleanup",
                "client_info": {
                    "client": "client.24123",
                    "client_addr": "-",
                    "tid": 328
                },
----------------

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #15991 from dillaman/wip-rbd-auth-profile

mon,osd: new rbd-based cephx cap profiles

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16432 from liewegas/wip-mgr-daemonstate-lock

mgr: add per-DaemonState lock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16460 from liewegas/wip-mgr-metadata

mon: add mgr metdata commands, and overall 'versions' command for all daemon versions

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16465 from xiexingguo/wip-shutup-noscrub-warn

mon: stop issuing not-[deep]-scrubbed warnings if disabled

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16477 from tchaikov/wip-health-mon

mon/HealthMonitor: trigger a proposal if stat updated

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16478 from xiexingguo/wip-drop-empty-message

mon/HealthMonitor: avoid sending unnecessary MMonHealthChecks to leader

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16480 from jcsp/wip-rm-convert

common: remove config opt conversion utility

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mon: do not dereference empty mgr_commands

This may be empty before the initial quorum commits the static command
list.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephtool/test.sh: add sudo for daemon compact

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR 16390 into master

* refs/remotes/upstream/pull/16390/head:
mds: cleanup with unique_ptr
mds: add to get version in the mds_commands
mds: cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR 16379 into master

* refs/remotes/upstream/pull/16379/head:
qa: fix MDS_CLIENT_RECALL copy error

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16337 into master

* refs/remotes/upstream/pull/16337/head:
mds: clear dirty rstat flag on non-auth inode when finishing scatter update

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16280 into master

* refs/remotes/upstream/pull/16280/head:
Client: mkdirs bugs when two clients send mkdir request for a same dir

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16278 into master

* refs/remotes/upstream/pull/16278/head:
MDSMonitor: show laggy MDSs at higher debug level

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16270 into master

* refs/remotes/upstream/pull/16270/head:
mds: improve wording of mdsmap dne logging

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16232 into master

* refs/remotes/upstream/pull/16232/head:
MDS: update the mlogger of mds in function check_memory_usage

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16231 into master

* refs/remotes/upstream/pull/16231/head:
Client: get rid of variable "butnot" in "client::get_caps"

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16226 into master

* refs/remotes/upstream/pull/16226/head:
qa: wait for OSDMap to propagate for snap purge

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16149 into master

* refs/remotes/upstream/pull/16149/head:
mds: misc cleanup

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 15942 into master

* refs/remotes/upstream/pull/15942/head:
client: add unlock_fh_pos when abort

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crush: assume weight_set != null imples weight_set_size > 0

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: skip deleted missing objects in pg[n]ls

Fixes: http://tracker.ceph.com/issues/20739
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

PendingReleaseNotes: added blurb for new RBD cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: expand profile caps upon construction to avoid potential race

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: rbd-mirror tests should use 'mirror' user

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/suites/rbd: mirroring tests should use rbd cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: devstack test should use auth profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: include details for new RBD cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon,osd: drop the "allow" prefix for profile caps

The use of the a profile implies that it sets everything up. If
support for deny is ever added, it doesn't make much sense to
deny a profile.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: new 'rbd'/'rbd-read-only' osd cap profiles

The 'rbd' profile provides read-only class access on all pools
to the 'rbd_children' object and write access to the optionally
specified pool. The 'rbd-read-only' profile does as its name
implies.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: primitive cephx osd profile cap support

The two new example profiles are read-only and read-write

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: added 'rbd' profile

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: support regex-based restrictions on command caps

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16445 from tchaikov/wip-16895

test: create asok files in a temp directory under $TMPDIR

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

crush/CrushWrapper: only allocate weight_set if non-empty

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/mon/crush_ops.sh: fix in-use rule removal test

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/mon/crush_ops: require luminous clients for test

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: creating rules

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: document weight set commands

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map*: restructure and clean up

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: warn off manual crush map edits

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: update crush location section

Fix various inaccuracies, simplify.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: prune intro

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: 'osd crush rule ls' plaintext output

More suitable for scripting.

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: remove_item_under: weight down weight-sets

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: remove_item_under whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: link_bucket whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: detach_bucket: weight down weight_sets

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: detach_bucket uninline

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: remove_item: weight down weight-sets

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: create_or_move_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: remove_item, bucket_remove_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: update_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: adjust_item_weight* whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: bucket_add_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: add_bucket: populate weight-sets

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: get_choose_args_positions helper

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: insert_item whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

mon: 'osd crush weight-set {ls,dump,create[-compat],rm[-compat],reweight[-compat]}' commands

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: choose_arg_update_item_weight

Signed-off-by: Sage Weil <sage@redhat.com>