qa: fix wait for wrong health message

Fixes: http://tracker.ceph.com/issues/20805
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16629 from johnwilkins/wip-doc-remove-contractions

Remove contractions from the documentation.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16572 from liewegas/wip-pidfile

test: add separate ceph-helpers-based smoke test

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/cephfs: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/start: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/man: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/install: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/rgw: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/rbd: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/rados: Removed contractions to improve readabilty for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #16569 from liewegas/wip-set-not-put

mon: 'config-key put' -> 'config-key set'

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

qa/standalone/crush/crush-classes: fix test

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/README

Signed-off-by: Sage Weil <sage@redhat.com>

test/smoke.sh: add simple ceph-helpers-based smoke test

This is just enough during make check to know things aren't totally
broken.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16388 from xiexingguo/wip-class-misc-fixes

crush, mon: simplify device class manipulation commands

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16172 from ivancich/wip-dup-ops

PGLog: store extra duplicate ops beyond the normal log entries

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16558 from hjwsm1989/fix-recovering-hang-with-unfound-objects

  osd/PrimaryLogPG: fix recovering hang when have unfound objects

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16561 from Yan-waller/wip-walle-0725osddumpclinfo

osd: add dump filter for tracked ops

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

PGLog: continuation, store extra duplicate ops beyond the normal log entries

This helps us avoid replaying non-idempotent client operations when
the pg log is very short, e.g. in an effort to force OSDs to use
backfill rather than regular recovery. This can be advantageous to
avoid blocking i/o to objects, at the cost of longer total time to
become clean (since backfill requires scanning the objects to see what
is missing).

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #16384 from mikulely/2-checkout-placmenet-target-existness

rgw: check placement target existence during bucket creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16389 from prallabh/wip-20661

rgw: Do not decrement stats cache when the cache values are zero

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16463 from cbodley/wip-20708

rgw: fix parse/eval of policy conditions with IfExists

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #16495 from weiqiaomiao/wip_weiqm_rgw_crash

rgw: fix radosgw will crash when service is restarted during lifecycl…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #16531 from rzarzynski/wip-rgw-20757

rgw: fix memory leaks during Swift Static Website's error handling.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16601 from rzarzynski/wip-rgw-20774

rgw: never let http_redirect_code of RGWRedirectInfo to stay uninitialized

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #16604 from rzarzynski/wip-rgw-20779

rgw: fix the UTF8 check on bucket entry name in rgw_log_op().

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16575 from liewegas/wip-20693

qa/suites/rados: at-end: ignore PG_{AVAILABILITY,DEGRADED}

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16398 from dillaman/wip-20655

rbd-mirror: guard the deletion of non-primary images

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #16574 from batrick/rgw-rbd-stripped-build

CMakeLists: trim rbd/rgw forced dependencies

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16603 from dillaman/wip-rbd-mirror-valgrind

rbd-mirror: initialize timer context pointer to null

Merge pull request #16613 from liewegas/wip-vstart-mgr

vstart.sh: start mgr after mon, before osds

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #16584 from gregsfortytwo/wip-deletes-coverity

messages: default-initialize MOSDPGRecoveryDelete[Reply] members

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

vstart.sh: start mgr after mon, before osds

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16610 from dzafman/wip-fix-reg11184

test: reg11184 might not always find pg 2.0 prior to import

Reviewed-by: Sage Weil <sage@redhat.com>

PendingReleaseNotes: note about put -> set

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephtool/test.sh: add some config-key tests

Signed-off-by: Sage Weil <sage@redhat.com>

mon: 'config-key put' -> 'config-key set'

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16530 from xiexingguo/wip-fix-pgtemp

mon: prime pg_temp and a few health warning fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16587 from dzafman/wip-20770

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16600 from dingdangzhang/master

bluestore/BlueFS: pass string as const ref

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16563 from hjwsm1989/readable-thread-info

common/WorkQueue: use threadpoolname + threadaddr for heartbeat_han…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16576 from liewegas/wip-20371

mon/MonClient: scale backoff interval down when we have a healthy mon session

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16559 from hjwsm1989/dump-stuck

qa/tasks/dump_stuck: fix dump_stuck test bug

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16516 from Liuchang0812/nit-close-formatter-section

mon/MDSMonitor: close object section of formatter

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16553 from wumingqiao/wip-leveldb-store-crash

kv: resolve a crash issue in ~LevelDBStore()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16578 from liewegas/wip-20771

common/common_init: disable default dout logging for UTILITY_NODOUT too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16522 from majianpeng/bluestore-remove-unused-code

os/bluestore/BlueStore: remove unused code.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

test: reg11184 might not always find pg 2.0 prior to import

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #16571 from ceph/wip-cd-bluestore-2

qa/tasks/ceph-deploy: Fix bluestore options for ceph-deploy

Reviewed-by: Tamil Muthamizhan <tmuthami@redhat.com>

Merge pull request #16452 from wjwithagen/wjw-warning-unused

buffer: silence unused var warning on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16483 from ddiss/cmake_install_includedir

cmake: use CMAKE_INSTALL_INCLUDEDIR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15033 from rzarzynski/wip-rgw-19831

rgw: fix segfault in RevokeThread during its shutdown procedure.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

PGLog: store extra duplicate ops beyond the normal log entries

This helps us avoid replaying non-idempotent client operations when
the pg log is very short, e.g. in an effort to force OSDs to use
backfill rather than regular recovery. This can be advantageous to
avoid blocking i/o to objects, at the cost of longer total time to
become clean (since backfill requires scanning the objects to see what
is missing).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16013 from xiexingguo/wip-blue-tool

os/bluestore/bluestore_tool: add sanity check to get rid of occasionally crash

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16528 from tchaikov/wip-ceph-release-notes

tools: ceph-release-notes: escape _ for unintended links

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #16404 from dzafman/wip-20041

osd: scrub_to specifies clone ver, but transaction include head write…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/rados/operations/crush-map: s/die/dice/

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: be more helpful for new weight-set CLI users

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc/release-notes: update device-class cli family

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test/crush: fix memory leak

use unique_ptr<> instead, otherwise static analyzer will complain.
as ASSERT_EQ will bail out without executing delete c

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test/crush: kill dead class_is_in_use test

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: kill 'class rename'

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: kill "class create" command

The device class is now self and automatically managed.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: allow "crush class rm" to automatically recycle shadow tree(s)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: rm-device-class support

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: tidy up class_map on remove_root()

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: implement find_shadow_roots

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: use is_shadow_item() wrapper if possible

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: reuse find_roots to implement find_nonshadow_roots

to reduce code redundance...

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: guard set-device-class

If a device has already been bounded to a class,
do not allow to change its class silently.
Require user call rm-device-class first.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: fix class_is_in_use()

A class can be considered as in-use only if it is referenced by
any of the existing crush rules.

The patch also makes the output more human readable. For example:

./bin/ceph osd crush rule create-replicated myrule default host ssd
./bin/ceph osd crush class rm ssd
Error EBUSY: class 'ssd' still referenced by crush_rule 'myrule'

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: rebuild shadow tree on "crush create-or-move/move"

This patch solves the problem below:

./bin/ceph osd crush move osd.0 root=foo rack=foo-rack host=foo-host
moved item id 0 name 'osd.0' to location {host=foo-host,rack=foo-rack,root=foo} in crush map

 ./bin/ceph osd crush rule create-replicated foo-rule foo host ssd
Error EINVAL: root foo has no devices with class ssd

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: fix "crush create-or-move/move" would drop osd's class

    Was:
     ./bin/ceph osd tree
    ID CLASS WEIGHT  TYPE NAME                                        UP/DOWN REWEIGHT PRI-AFF
    -1       3.00000 root default
    -2       3.00000     host gitbuilder-ceph-rpm-centos7-amd64-basic
     0   ssd 1.00000         osd.0                                         up  1.00000 1.00000
     1   ssd 1.00000         osd.1                                         up  1.00000 1.00000
     2   ssd 1.00000         osd.2                                         up  1.00000 1.00000

    ./bin/ceph osd crush move osd.0 root=foo rack=foo-rack  host=foo-host
    moved item id 0 name 'osd.0' to location {host=foo-host,rack=foo-rack,root=foo} in crush map

     ./bin/ceph osd tree
    ID CLASS WEIGHT  TYPE NAME                                        UP/DOWN REWEIGHT PRI-AFF
    -7       1.00000 root foo
    -6       1.00000     rack foo-rack
    -5       1.00000         host foo-host
     0       1.00000             osd.0                                     up  1.00000 1.00000
    -1       2.00000 root default
    -2       2.00000     host gitbuilder-ceph-rpm-centos7-amd64-basic
     1   ssd 1.00000         osd.1                                         up  1.00000 1.00000
     2   ssd 1.00000         osd.2                                         up  1.00000 1.00000

    Now:
    ./bin/ceph osd tree
    ID CLASS WEIGHT  TYPE NAME                                        UP/DOWN REWEIGHT PRI-AFF
    -1       3.00000 root default
    -2       3.00000     host gitbuilder-ceph-rpm-centos7-amd64-basic
     0   ssd 1.00000         osd.0                                         up  1.00000 1.00000
     1   ssd 1.00000         osd.1                                         up  1.00000 1.00000
     2   ssd 1.00000         osd.2                                         up  1.00000 1.00000

    ./bin/ceph osd crush move osd.0 root=foo rack=foo-rack  host=foo-host
    moved item id 0 name 'osd.0' to location {host=foo-host,rack=foo-rack,root=foo} in crush map

    ./bin/ceph osd tree
    ID CLASS WEIGHT  TYPE NAME                                        UP/DOWN REWEIGHT PRI-AFF
    -7       1.00000 root foo
    -6       1.00000     rack foo-rack
    -5       1.00000         host foo-host
     0   ssd 1.00000             osd.0                                     up  1.00000 1.00000
    -1       2.00000 root default
    -2       2.00000     host gitbuilder-ceph-rpm-centos7-amd64-basic
     1   ssd 1.00000         osd.1                                         up  1.00000 1.00000
     2   ssd 1.00000         osd.2                                         up  1.00000 1.00000

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: optional show-shadow for "crush tree" command

 ./bin/ceph osd crush tree --show-shadow
ID WEIGHT  TYPE NAME
-4 3.00000 root default~ssd
-3 3.00000     host gitbuilder-ceph-rpm-centos7-amd64-basic~ssd
 0 1.00000         osd.0
 1 1.00000         osd.1
 2 1.00000         osd.2
-1 3.00000 root default
-2 3.00000     host gitbuilder-ceph-rpm-centos7-amd64-basic
 0 1.00000         osd.0
 1 1.00000         osd.1
 2 1.00000         osd.2

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16579 from liewegas/wip-fix-nonregression

qa/suites/rados/singleton/all/erasure-code-nonregression: fix typo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amik Kumar <amitkuma@redhat.com>

qa/workunits/erasure-code/encode-decode-nonregression: do not require git checkout

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: fix the UTF8 check on bucket entry name in rgw_log_op().

Fixes: http://tracker.ceph.com/issues/20779
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #16580 from liewegas/wip-20767

pybind/mgr/zabbix: fix health in non-compat mode

Reviewed-by: Wido den Hollander <wido@widodh.nl>

Merge pull request #16573 from liewegas/wip-osd-manual

doc/install/manual-deployment: update osd creation steps

Reviewed-by: Alfredo Deza <adeza@redhat.com>

rbd-mirror: initialize timer context pointer to null

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

 bluestore/BlueFS: pass string as const ref

Signed-off-by: dingdangzhang <boqian.zy@alibaba-inc.com>

test: Fixes for test_pidfile

Add a short delay for one test
Fix test that had unreliable log output

Fixes: http://tracker.ceph.com/issues/20770
Signed-off-by: David Zafman <dzafman@redhat.com>

rgw: rgw_website.h doesn't assume inclusion of the std namespace anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: never let http_redirect_code of RGWRedirectInfo to stay uninitialized.

Fixes: http://tracker.ceph.com/issues/20774
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #14868 from wangzhengyong/librbd

librbd: add compare and write API

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: fixed compiler warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: improved compare_and_write testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: add compare_and_write test for fsx
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: add compare_and_write testcase
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: add Mock Image cache for compare_and_write
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: handle compare_and_write op in test_mock_Replay
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd/journal: handle compare_and_write event
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: handle cmpext request in LibradosTestStub
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write/aio_compare_and_write API
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write perfcounter
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: handle compare_and_write imagerequest in ImageRequestWQ
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write ImageRequest
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add CompareAndWrite ObjectRequest
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

Merge pull request #16595 from wjwithagen/wip-wjw-freebsd-run-standalone

qa: make run-standalone work on FreeBSD

Reviewed-by: Sage Weil <sage@redhat.com>

mon/MonClient: do not send_log if conn is not active anymore

the log message to be sent will be appended to waiting_for_session
instead. but we will send the logs anyway when the MonClient is
authorized with the new connection. so, avoid doing this as it's not
necessary.

also refactor the schedule_tick() call into a scope_guard, so it is
always called upon the return of the tick() method.

Signed-off-by: Kefu Chai <kchai@redhat.com>