Merge PR #16413 into master

* refs/remotes/upstream/pull/16413/head:
qa/cephfs: lsof if umount fails

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #16410 into master

* refs/remotes/upstream/pull/16410/head:
mds: refuse 'export dir' request when mds is stopping

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16645 from tchaikov/wip-bluefs-allocator-is-dev-only-option

common/options: make "blue{fs,store}_allocator" LEVEL_DEV

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16489 from liewegas/wip-bluestore-options

common/options.cc: document bluestore config options

Merge pull request #16520 from xiexingguo/wip-application-warn

mon/OSDMonitor: issue pool application related warning

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16568 from liewegas/wip-application-warn

qa,doc: document and fix tests for pool application warnings

Merge pull request #16634 from neha-ojha/wip-default-device-class

osd: add default_device_class to metadata

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16638 from liewegas/wip-20804

osd/PG: ignore CancelRecovery in NotRecovering

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16646 from tchaikov/wip-qa-run-standalone-work-with-gnu-find

qa/run-standalone.sh: fix the find option to be compatible with GNU find

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #16487 from liewegas/wip-mgr-module-health

mgr: mgr_module interface to report health alerts

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16641 from liewegas/wip-mgr-configs

mgr/PyState: shut up about get_config on nonexistent keys

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16582 from liewegas/wip-20633

mon/MgrMonitor: reset mgrdigest timer with new subscription

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/run-standalone.sh: fix the find option to be compatible with GNU find

also re-indent to be consistent with other part of this script

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: make "blue{fs,store}_allocator" LEVEL_DEV

See-also: http://tracker.ceph.com/issues/20660
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16599 from liewegas/wip-standalone-fixes

qa/workunits: adjust path to ceph-helpers.sh

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16482 from tchaikov/wip-opt

common/options: refactors to set the properties in a more structured way

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16589 from Yan-waller/wip-walle-0726testwatchnotify

test/rados: fix wrong parameter order of RETURN1_IF_NOT_VAL

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16609 from wjwithagen/wip-wjw-FreeBSD-group-wheel

ceph_disk/main.py: FreeBSD root has wheel for group

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16593 from shangzhong/wip-20660

os/bluestore/StupidAllocator: rounded down len to an align boundary

Reviewed-by: Sage Weil <sage@redhat.com>

common/options: document bluestore options

Signed-off-by: Sage Weil <sage@redhat.com>

compressor: accept 'none' or '' for none

This makes the parse and output methods match up.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: observe bluestore_compression_required_ratio

Signed-off-by: Sage Weil <sage@redhat.com>

doc/dev/config: some guidance around defining config options

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #16435 into master

* refs/remotes/upstream/pull/16435/head:
qa: whitelist trim error during powercycle tests

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/PyState: shut up about get_config on nonexistent keys

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16450 from markhpc/wip-rocksdb-bloomsettings

kv/RocksDBStore: Table options for indexing and filtering

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16281 from badone/wip-PG-cluster-log-audit

osd: Log audit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16514 from liewegas/wip-20744

qa/tasks/ceph: wait for mgr to activate and pg stats to flush in health()

Reviewed-by: John Spray <john.spray@redhat.com>

osd/PG: ignore CancelRecovery in NotRecovering

It's possible to get >1 CancelRecovery events in the queue.
The first will transition us from Recovering to NotRecovering.
Currently the second will make us crash.

This is harmless; instead just swallow the event.

Fixes: http://tracker.ceph.com/issues/20804
Signed-off-by: Sage Weil <sage@redhat.com>

qa: whitelist trim error during powercycle tests

Fixes: http://tracker.ceph.com/issues/20566
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16635 from dzafman/wip-fix-test-pidfile

test: test_pidfile running 2nd mon has unreliable log output

Reviewed-by: Sage Weil <sage@redhat.com>

test: test_pidfile running 2nd mon has unreliable log output

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: add default_device_class to metadata

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #16629 from johnwilkins/wip-doc-remove-contractions

Remove contractions from the documentation.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16572 from liewegas/wip-pidfile

test: add separate ceph-helpers-based smoke test

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/cephfs: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/start: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/man: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/install: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/rgw: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/rbd: Removed contractions for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc/rados: Removed contractions to improve readabilty for ESL speakers.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #16569 from liewegas/wip-set-not-put

mon: 'config-key put' -> 'config-key set'

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

qa/standalone/crush/crush-classes: fix test

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/README

Signed-off-by: Sage Weil <sage@redhat.com>

test/smoke.sh: add simple ceph-helpers-based smoke test

This is just enough during make check to know things aren't totally
broken.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/upgrade: fix upgrade wait for healthy

There is no mgr, so we can't call ceph.healthy.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/ceph: wait for pg stats to flush in healthy check

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/ceph_manager: wait_for_all_up -> wait_for_all_osds_up

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/ceph_manager: expose flush_all_pg_stats

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16388 from xiexingguo/wip-class-misc-fixes

crush, mon: simplify device class manipulation commands

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16172 from ivancich/wip-dup-ops

PGLog: store extra duplicate ops beyond the normal log entries

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16558 from hjwsm1989/fix-recovering-hang-with-unfound-objects

  osd/PrimaryLogPG: fix recovering hang when have unfound objects

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16561 from Yan-waller/wip-walle-0725osddumpclinfo

osd: add dump filter for tracked ops

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

PGLog: continuation, store extra duplicate ops beyond the normal log entries

This helps us avoid replaying non-idempotent client operations when
the pg log is very short, e.g. in an effort to force OSDs to use
backfill rather than regular recovery. This can be advantageous to
avoid blocking i/o to objects, at the cost of longer total time to
become clean (since backfill requires scanning the objects to see what
is missing).

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #16384 from mikulely/2-checkout-placmenet-target-existness

rgw: check placement target existence during bucket creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16389 from prallabh/wip-20661

rgw: Do not decrement stats cache when the cache values are zero

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16463 from cbodley/wip-20708

rgw: fix parse/eval of policy conditions with IfExists

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #16495 from weiqiaomiao/wip_weiqm_rgw_crash

rgw: fix radosgw will crash when service is restarted during lifecycl…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #16531 from rzarzynski/wip-rgw-20757

rgw: fix memory leaks during Swift Static Website's error handling.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16601 from rzarzynski/wip-rgw-20774

rgw: never let http_redirect_code of RGWRedirectInfo to stay uninitialized

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #16604 from rzarzynski/wip-rgw-20779

rgw: fix the UTF8 check on bucket entry name in rgw_log_op().

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16575 from liewegas/wip-20693

qa/suites/rados: at-end: ignore PG_{AVAILABILITY,DEGRADED}

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/suites: more whitelisting

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16398 from dillaman/wip-20655

rbd-mirror: guard the deletion of non-primary images

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #16574 from batrick/rgw-rbd-stripped-build

CMakeLists: trim rbd/rgw forced dependencies

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16603 from dillaman/wip-rbd-mirror-valgrind

rbd-mirror: initialize timer context pointer to null

Merge pull request #16613 from liewegas/wip-vstart-mgr

vstart.sh: start mgr after mon, before osds

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #16584 from gregsfortytwo/wip-deletes-coverity

messages: default-initialize MOSDPGRecoveryDelete[Reply] members

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

vstart.sh: start mgr after mon, before osds

Signed-off-by: Sage Weil <sage@redhat.com>

src/ceph-disk/ceph_disk/main.py: FreeBSD root has wheel for group

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

kv/RocksDBStore: Add table options for filter and index tuning.

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #16610 from dzafman/wip-fix-reg11184

test: reg11184 might not always find pg 2.0 prior to import

Reviewed-by: Sage Weil <sage@redhat.com>

PendingReleaseNotes: note about put -> set

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephtool/test.sh: add some config-key tests

Signed-off-by: Sage Weil <sage@redhat.com>

mon: 'config-key put' -> 'config-key set'

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16530 from xiexingguo/wip-fix-pgtemp

mon: prime pg_temp and a few health warning fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16587 from dzafman/wip-20770

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16600 from dingdangzhang/master

bluestore/BlueFS: pass string as const ref

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16563 from hjwsm1989/readable-thread-info

common/WorkQueue: use threadpoolname + threadaddr for heartbeat_han…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16576 from liewegas/wip-20371

mon/MonClient: scale backoff interval down when we have a healthy mon session

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16559 from hjwsm1989/dump-stuck

qa/tasks/dump_stuck: fix dump_stuck test bug

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16516 from Liuchang0812/nit-close-formatter-section

mon/MDSMonitor: close object section of formatter

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16553 from wumingqiao/wip-leveldb-store-crash

kv: resolve a crash issue in ~LevelDBStore()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16578 from liewegas/wip-20771

common/common_init: disable default dout logging for UTILITY_NODOUT too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16522 from majianpeng/bluestore-remove-unused-code

os/bluestore/BlueStore: remove unused code.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

test: reg11184 might not always find pg 2.0 prior to import

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #16571 from ceph/wip-cd-bluestore-2

qa/tasks/ceph-deploy: Fix bluestore options for ceph-deploy

Reviewed-by: Tamil Muthamizhan <tmuthami@redhat.com>

common/config: change the element type of lists to "const char*"

there lists holds constant string literals. and they are not changeable
at runtime. so make std::list<const char*> for smaller memory foot
print.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config: accept initialize_list when adding option properties

it's syntax sugar, but more handly this way. and it helps to reduce
memory foot print.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16452 from wjwithagen/wjw-warning-unused

buffer: silence unused var warning on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16483 from ddiss/cmake_install_includedir

cmake: use CMAKE_INSTALL_INCLUDEDIR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15033 from rzarzynski/wip-rgw-19831

rgw: fix segfault in RevokeThread during its shutdown procedure.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

PGLog: store extra duplicate ops beyond the normal log entries

This helps us avoid replaying non-idempotent client operations when
the pg log is very short, e.g. in an effort to force OSDs to use
backfill rather than regular recovery. This can be advantageous to
avoid blocking i/o to objects, at the cost of longer total time to
become clean (since backfill requires scanning the objects to see what
is missing).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16013 from xiexingguo/wip-blue-tool

os/bluestore/bluestore_tool: add sanity check to get rid of occasionally crash

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16528 from tchaikov/wip-ceph-release-notes

tools: ceph-release-notes: escape _ for unintended links

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #16404 from dzafman/wip-20041

osd: scrub_to specifies clone ver, but transaction include head write…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/rados/operations/crush-map: s/die/dice/

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: be more helpful for new weight-set CLI users

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc/release-notes: update device-class cli family

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>