os/bluestore/BlueStore: remove unused code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #16545 from asomers/install-deps

install-deps.sh: add missing dependencies for FreeBSD

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #16564 from liewegas/wip-vstart-mgr

vstart.sh: fix mgr vs restful command startup race

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

vstart.sh: fix mgr vs restful command startup race

If the mgr hasn't activated yet we won't have registered the python
commands.  Use 'ceph tell mgr ...' to ensure we block.  (This is only a
problem right after mkfs when the first mgr hasn't started yet.)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16497 from weiqiaomiao/wip_weiqm_rgw_compile_warn

rgw: Drop dump_usage_bucket_info() to silence warning from -Wunused-function

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16394 from jcsp/wip-pr-docs

doc: add doc requirements on PR submitters

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16038 from ztczll/ztczll

.mailmap, .organizationmap: Update ztczll affiliation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

.mailmap, .organizationmap: Update ztczll affiliation

Signed-off-by: zhanglei <243290414@qq.com>

Merge pull request #16468 from gmayyyha/mgr-show-rbd-features

mgr/dashboard: show rbd image features

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

mgr/dashboard: rbd image show features

Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>

Merge pull request #16537 from shangzhong/wip-update-organization

mailmap: add affiliation for Zhu Shangzhong

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #16427 from shangzhong/wip-20685

ceph-disk: s/ceph_osd_mkfs/command_check_call/

Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #16362 from tchaikov/wip-ceph-disk

ceph-disk: use '-' for feeding ceph cli with stdin

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

install-deps.sh: add missing dependencies for FreeBSD

Signed-off-by: Alan Somers <asomers@gmail.com>

Merge pull request #16399 from batrick/cephfs-ptl-change

AUTHORS: update CephFS PTL

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

doc: update CephFS PTL

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16509 from liewegas/wip-rgw-wait

qa/suits/rados/basic/tasks/rgw_snaps: wait for pools to be created

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph-disk: s|/dev/fd/0|-|

3a4931b allows ceph cli to use '-' to denote stdin/stdout, so use it
instead of /dev/fd/0.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/ceph-disk: add docs for ceph-disk

we might want to move part of it to user-facing docs.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16437 from tchaikov/wip-doc-fixes

doc: update ceph(8) man page with new sub-commands

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16417 from fangyuxiangGL/rgw-hang

rgw: fix rgw hang when do RGWRealmReloader::reload after go SIGHUP

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16422 from theanalyst/rgw-lc-log-fix

rgw: fix asctime when logging in rgw_lc

Reviewed-by: Daniel Gryniewicz <dang@fprintf.net>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@linuxbox.com>

ceph-disk: s/ceph_osd_mkfs/command_check_call/
Fixes: http://tracker.ceph.com/issues/20685
Signed-off-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn>

Update my organization and mail

Signed-off-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn>

Merge pull request #16524 from dingdangzhang/master

msg/async: use auto iterator having more simple code and good performance

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16481 from nikitych/bug20706

ceph-disk: Fix for missing 'not' in *_is_diskdevice checks

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16247 from shangzhong/wip-20556

ceph-disk: change the lockbox partition number to 5

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13723 from ovh/bp-forced-recovery

osd/PG: make prioritized recovery possible

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16504 from Yan-waller/wip-walle-0721osdclients

osd: fix OpRequest and tracked op dump information

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16512 from liupan1111/wip-fix-latency

common: the latency dumped by "ceph osd perf" is not real

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16518 from liewegas/wip-20751

mon/OSDMonitor: remove zeroed new_state updates

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16365 from jcsp/wip-osd-marked-me-down

osd: rephrase "wrongly marked me down" clog message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: update whitelist for "wrongly marked me down"

Signed-off-by: John Spray <john.spray@redhat.com>

osd: rephrase "wrongly marked me down" clog message

Rephrase as "Monitor daemon marked osd.<id> down, but it is
still running", which is more descriptive.  Demote the
epoch information to DEBUG level.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #16326 from liewegas/wip-weight-set

crush,mon: add weight-set introspection and manipulation commands

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com

use auto iterator having more simple code and good performence

Signed-off-by: dingdangzhang <boqian.zy@alibaba-inc.com>

Merge pull request #16360 from dillaman/wip-rbd-mirror-service-daemon

mgr/dashboard: rbd mirroring status page

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16529 from Liuchang0812/wip-20750

mgr/status: row has incorrect number of values

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16503 from renhwztetecs/renhw-wip-mon-rebuild-doc

doc/mon: fix ceph-authtool command in rebuild mon's sample

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16523 from tchaikov/wip-ceph-disk-asok-dir

tests: add setup/teardown for asok dir

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

doc/rados/operations/add-or-rm-osds: fix dead hyper link

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: document "ceph osd set-require-min-compat-client"

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/status: row has incorrect number of values

Fixes: http://tracker.ceph.com/issues/20750
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #16507 from scienceluo/patch-1

Update .organizationmap

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc/mon: fix ceph-authtool create keyring

if use "ceph-authtool keyring -n mon. --cap mon allow 'allow *'"
as err:
[root]# ceph-authtool keyring -n mon. --cap mon allow 'allow *'
ceph-authtool: unexpected 'allow *'

Excessive "allow" is the cause of the error

Signed-off-by huanwen ren <ren.huanwen@zte.com.cn>

doc: document "ceph features"

Signed-off-by: Kefu Chai <kchai@redhat.com>

vstart.sh: do not try to remove default asok_dir

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: add setup/teardown for asok dir

ceph-disk.sh has its own setup/teardown, so update them also. so asok
files will have their parent directory when being created.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16502 from weiqiaomiao/wip_weiqm_unlock_bucket

rgw: should unlock when reshard_log->update() reture non-zero in RGWB…
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: Drop Unused function in rgw_usage.cc

fix compile warnning in any PR (Ex: https://jenkins.ceph.com/job/ceph-pull-requests/29193/console):
/home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_usage.cc:33:13: warning: ‘void dump_usage_bucket_info(ceph::Formatter*, std::__cxx11::string&, cls_user_bucket_entry&)’ defined but not used [-Wunused-function]
 static void dump_usage_bucket_info(Formatter *formatter, string& name, cls_user_bucket_entry& entry)

Signed-off-by: Wei Qiaomiao <wei.qiaomiao@zte.com.cn>

rgw: should unlock when reshard_log->update() reture non-zero in RGWBucketReshard::execute()

Signed-off-by: Wei Qiaomiao <wei.qiaomiao@zte.com.cn>

osd/OSDMap: fix Incremental::dump

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: remove zeroed new_state updates

These are interpreted as CEPH_OSD_UP in apply_incremental for legacy
reasons--we don't want them!

Fixes: http://tracker.ceph.com/issues/20751
Signed-off-by: Sage Weil <sage@redhat.com>

common: the latency dumped by "ceph osd perf" is not real

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

Merge pull request #16508 from liewegas/wip-mgr-lock-cycle

mgr: fix lock cycle

qa/suites/rados/basic/tasks/rgw_snaps: wait for pools to be be created

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/basic/tasks/rgw_snaps: fix pool list

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16488 from liewegas/wip-skewed

common/options: remove mon_warn_osd_usage_min_max_delta from options.cc too

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16490 from jdurgin/wip-20739

osd/PrimaryLogPG: skip deleted missing objects in pg[n]ls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16501 from liewegas/wip-fix-mgr-commands

mon: do not dereference empty mgr_commands

Merge pull request #16500 from liewegas/wip-compact-sudo

qa/workunits/cephtool/test.sh: add sudo for daemon compact

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/PyModules: fix lock name

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: fix lock cycle

This breaks a lock cycle with DaemonState::lock and PyModules::lock.

Signed-off-by: Sage Weil <sage@redhat.com>

Update .organizationmap

The list of contributors to ZTE looks better if it is in alphabetical order.

Signed-off-by: Luo Kexue luo.kexue@zte.com.cn

osd: store op request source info

if we use command 'ceph daemon osd.# dump_historic_ops' to check the history
 tracked ops, the 'client_addr' is always dumped as '-' which result in clients
indistinguishable from each other.

"description": "osd_op(client.24123.0:328 3.36 3:6ec0d61b:::rbd_data.10472ae8944a.0000000000000289:head [write 4141056~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected e36)",
            "initiated_at": "2017-07-22 14:24:25.925633",
            "age": 4.893190,
            "duration": 0.272642,
            "type_data": {
                "flag_point": "commit sent; apply or cleanup",
                "client_info": {
                    "client": "client.24123",
                    "client_addr": "-",
                    "tid": 328
                },

It is because 'connection' was reset to nullptr in function '_unregistered'
before op has been inserted into history, so the 'get_source_addr' always return
empty entity addr.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

common: dump field name of 'flag_point' and 'client_info'

we used to open "type_data" section as a array (that's open_array_section),
some key field names can not be dumped out. use open_object_section to dump them
as following:

before this change:
------------------
"description": "osd_op(client.4202.0:4958 3.28 3:163e8348:::rbd_data.10472ae8944a.0000000000000164:head [write 1691648~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected e24)",
            "initiated_at": "2017-07-22 14:22:13.469324",
            "age": 17.595664,
            "duration": 0.460406,
            "type_data": [
                "commit sent; apply or cleanup",
                {
                    "client": "client.4202",
                    "client_addr": "-",
                    "tid": 4958
                },
-----------------
after:
-----------------
"description": "osd_op(client.24123.0:328 3.36 3:6ec0d61b:::rbd_data.10472ae8944a.0000000000000289:head [write 4141056~4096 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected e36)",
            "initiated_at": "2017-07-22 14:24:25.925633",
            "age": 4.893190,
            "duration": 0.272642,
            "type_data": {
                "flag_point": "commit sent; apply or cleanup",
                "client_info": {
                    "client": "client.24123",
                    "client_addr": "-",
                    "tid": 328
                },
----------------

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #15991 from dillaman/wip-rbd-auth-profile

mon,osd: new rbd-based cephx cap profiles

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16432 from liewegas/wip-mgr-daemonstate-lock

mgr: add per-DaemonState lock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16460 from liewegas/wip-mgr-metadata

mon: add mgr metdata commands, and overall 'versions' command for all daemon versions

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16465 from xiexingguo/wip-shutup-noscrub-warn

mon: stop issuing not-[deep]-scrubbed warnings if disabled

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16477 from tchaikov/wip-health-mon

mon/HealthMonitor: trigger a proposal if stat updated

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16478 from xiexingguo/wip-drop-empty-message

mon/HealthMonitor: avoid sending unnecessary MMonHealthChecks to leader

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16480 from jcsp/wip-rm-convert

common: remove config opt conversion utility

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mon: do not dereference empty mgr_commands

This may be empty before the initial quorum commits the static command
list.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephtool/test.sh: add sudo for daemon compact

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR 16390 into master

* refs/remotes/upstream/pull/16390/head:
mds: cleanup with unique_ptr
mds: add to get version in the mds_commands
mds: cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR 16379 into master

* refs/remotes/upstream/pull/16379/head:
qa: fix MDS_CLIENT_RECALL copy error

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16337 into master

* refs/remotes/upstream/pull/16337/head:
mds: clear dirty rstat flag on non-auth inode when finishing scatter update

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16280 into master

* refs/remotes/upstream/pull/16280/head:
Client: mkdirs bugs when two clients send mkdir request for a same dir

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16278 into master

* refs/remotes/upstream/pull/16278/head:
MDSMonitor: show laggy MDSs at higher debug level

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16270 into master

* refs/remotes/upstream/pull/16270/head:
mds: improve wording of mdsmap dne logging

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16232 into master

* refs/remotes/upstream/pull/16232/head:
MDS: update the mlogger of mds in function check_memory_usage

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16231 into master

* refs/remotes/upstream/pull/16231/head:
Client: get rid of variable "butnot" in "client::get_caps"

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16226 into master

* refs/remotes/upstream/pull/16226/head:
qa: wait for OSDMap to propagate for snap purge

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 16149 into master

* refs/remotes/upstream/pull/16149/head:
mds: misc cleanup

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR 15942 into master

* refs/remotes/upstream/pull/15942/head:
client: add unlock_fh_pos when abort

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crush: assume weight_set != null imples weight_set_size > 0

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: skip deleted missing objects in pg[n]ls

Fixes: http://tracker.ceph.com/issues/20739
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

PendingReleaseNotes: added blurb for new RBD cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: expand profile caps upon construction to avoid potential race

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: rbd-mirror tests should use 'mirror' user

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/suites/rbd: mirroring tests should use rbd cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: devstack test should use auth profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: include details for new RBD cap profiles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon,osd: drop the "allow" prefix for profile caps

The use of the a profile implies that it sets everything up. If
support for deny is ever added, it doesn't make much sense to
deny a profile.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: new 'rbd'/'rbd-read-only' osd cap profiles

The 'rbd' profile provides read-only class access on all pools
to the 'rbd_children' object and write access to the optionally
specified pool. The 'rbd-read-only' profile does as its name
implies.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: primitive cephx osd profile cap support

The two new example profiles are read-only and read-write

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: added 'rbd' profile

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: support regex-based restrictions on command caps

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16445 from tchaikov/wip-16895

test: create asok files in a temp directory under $TMPDIR

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>