Merge pull request #16578 from liewegas/wip-20771

common/common_init: disable default dout logging for UTILITY_NODOUT too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16522 from majianpeng/bluestore-remove-unused-code

os/bluestore/BlueStore: remove unused code.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16571 from ceph/wip-cd-bluestore-2

qa/tasks/ceph-deploy: Fix bluestore options for ceph-deploy

Reviewed-by: Tamil Muthamizhan <tmuthami@redhat.com>

Merge pull request #16452 from wjwithagen/wjw-warning-unused

buffer: silence unused var warning on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16483 from ddiss/cmake_install_includedir

cmake: use CMAKE_INSTALL_INCLUDEDIR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15033 from rzarzynski/wip-rgw-19831

rgw: fix segfault in RevokeThread during its shutdown procedure.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16013 from xiexingguo/wip-blue-tool

os/bluestore/bluestore_tool: add sanity check to get rid of occasionally crash

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16528 from tchaikov/wip-ceph-release-notes

tools: ceph-release-notes: escape _ for unintended links

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #16404 from dzafman/wip-20041

osd: scrub_to specifies clone ver, but transaction include head write…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16579 from liewegas/wip-fix-nonregression

qa/suites/rados/singleton/all/erasure-code-nonregression: fix typo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amik Kumar <amitkuma@redhat.com>

qa/workunits/erasure-code/encode-decode-nonregression: do not require git checkout

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16580 from liewegas/wip-20767

pybind/mgr/zabbix: fix health in non-compat mode

Reviewed-by: Wido den Hollander <wido@widodh.nl>

Merge pull request #16573 from liewegas/wip-osd-manual

doc/install/manual-deployment: update osd creation steps

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #14868 from wangzhengyong/librbd

librbd: add compare and write API

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: fixed compiler warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: improved compare_and_write testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: add compare_and_write test for fsx
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: add compare_and_write testcase
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: add Mock Image cache for compare_and_write
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: handle compare_and_write op in test_mock_Replay
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd/journal: handle compare_and_write event
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: handle cmpext request in LibradosTestStub
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write/aio_compare_and_write API
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write perfcounter
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: handle compare_and_write imagerequest in ImageRequestWQ
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write ImageRequest
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add CompareAndWrite ObjectRequest
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

Merge pull request #16595 from wjwithagen/wip-wjw-freebsd-run-standalone

qa: make run-standalone work on FreeBSD

Reviewed-by: Sage Weil <sage@redhat.com>

qa: make run-standalone work on FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #16543 from dillaman/wip-service-daemon-c-api

librados: add missing implementations for C service daemon API methods

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16519 from liewegas/wip-20754

osd: populate last_epoch_split during build_initial_pg_history

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16446 from xiexingguo/wip-destroyed

mon: show destroyed status in tree view; do not auto-out destroyed osds

Reviewed-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: introduce mon_osd_destroyed_out_interval for destroyed out control

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSD: abort booting if osd is marked as destroyed

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: apply new 'destroyed' status to 'osd tree' filter

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: show "destroyed" status in tree view

It would be a pain if we destroyed an osd and there is no obvious way
to find it out.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16123 from ddiss/fix-journal-supression

ceph-disk: don't activate suppressed journal devices

Reviewed-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/zabbix: fix health in non-compat mode

This was apparently written/tested with mon_health_preluminous_compat
enabled.  Fix to behave with or without that option.

Fixes: http://tracker.ceph.com/issues/20767
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16498 from Songweibin/wip-doc-rbd-trash-cli

doc: add rbd new trash cli and cleanups  in release-notes.rst

Merge pull request #16552 from tchaikov/wip-doc-fixes

doc: document mClock related options

Reviewed-by: Sage Weil <sage@redhat.com>

doc/install/manual-deployment: update osd creation steps

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16534 from liewegas/wip-20751

mon/OSDMonitor: ensure UP is not set for newly-created OSDs

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/singleton/all/erasure-code-nonregression: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

common/common_init: disable default dout logging for UTILITY_NODOUT too

This was broken in 4d06719f101f49019be25bd6137f72f2cb85bd5a.

Fixes: http://tracker.ceph.com/issues/20771
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16469 from xiexingguo/wip-fix-test

test: s/osd_objectstore_type/osd_objectstore

doc: add new trash cli in rados-rbd-cmds.rst

Signed-off-by: songweibin <song.weibin@zte.com.cn>

doc: cleanups in release-notes.rst

Signed-off-by: songweibin <song.weibin@zte.com.cn>

os/bluestore/BlueStore: remove unused code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Add both filestore and bluestore options for tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add ceph-deploy overrides options

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #15693 from dillaman/wip-osd-cmpext-ec-pool

osd: support cmpext operation on EC-backed pools

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16527 from jcsp/wip-options-followup

common: follow up to new options infrastructure

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16511 from c744402859/master

OSDMonitor: check mon_max_pool_pg_num when set pool pg_num

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16560 from nrdmn/patch-3

pybind/mgr/restful: fix typo

Reviewed-by: Sage Weil <sage@redhat.com>

Fixed typo in src/pybind/mgr/restful/api/__init__.py

Signed-off-by: Nick Erdmann <n@nirf.de>
Fixed typo

bluestore config options for tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add bluestore overrides for ceph-deploy

ceph-deploy doesn't use ceph overrides, Add same overrides for ceph-deploy

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Move ceph-deploy config options into its own folder

The old structure of link at top folder is pretty much outdated, the test
config option needs to be specific to cluster yaml.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add option to specify bluestore/filestore options

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #16536 from jdurgin/wip-20747

osd: PrimaryLogPG, PGBackend: complete callback even if interval changes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16539 from jdurgin/wip-20753

osd/PGLog: fix inaccurate missing assert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16541 from liewegas/wip-20761

qa/workunits/cephtool/test.sh: disable 'fs status' until bug is fixed

Merge pull request #16544 from dillaman/wip-service-daemon-null-dereference

mgr/MgrClient: do not attempt to access a global variable for config

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16385 from mikulely/3-check-target-when-create-bucket

rgw: check placement existence when create bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16540 from cbodley/wip-mrgw-admin-socket

rgw: restore admin socket path in mrgw.sh

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16513 from liewegas/wip-standalone

qa: move ceph-helpers-based make check tests to qa/standalone; run via teuthology

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16566 from jcsp/wip-doc-health

doc/rados: add page for health checks and update monitoring.rst

Reviewed-by: Sage Weil <sage@redhat.com>

rgw: fix segfault in RevokeThread during its shutdown procedure.

Fixes: http://tracker.ceph.com/issues/19831
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: dout can work also with non-raw pointers to CephContext.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/rados: add page for health checks and update monitoring.rst

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #16545 from asomers/install-deps

install-deps.sh: add missing dependencies for FreeBSD

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #16564 from liewegas/wip-vstart-mgr

vstart.sh: fix mgr vs restful command startup race

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

ceph-disk/tests/ceph-disk.sh: wait for right number of pgs

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: fix mgr vs restful command startup race

If the mgr hasn't activated yet we won't have registered the python
commands.  Use 'ceph tell mgr ...' to ensure we block.  (This is only a
problem right after mkfs when the first mgr hasn't started yet.)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16497 from weiqiaomiao/wip_weiqm_rgw_compile_warn

rgw: Drop dump_usage_bucket_info() to silence warning from -Wunused-function

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16394 from jcsp/wip-pr-docs

doc: add doc requirements on PR submitters

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16038 from ztczll/ztczll

.mailmap, .organizationmap: Update ztczll affiliation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: document mClock related options

* rados/configuration/osd-config-ref.rst: document mClock related
  options
* release-notes.rst: fix syntax errors. link QoS to mClock related docs.

Signed-off-by: Kefu Chai <kchai@redhat.com>

.mailmap, .organizationmap: Update ztczll affiliation

Signed-off-by: zhanglei <243290414@qq.com>

qa/run-standalone.sh: helper to run all standalone tests

Nothing fancy, but documents how these are run.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSMap: init mds_features

This can lead to ceph-dencoder reencode failures.  Doesn't matter too
much in the real world since body decodes these ancient mds_info_t
structs.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/scrub: separate scrub/repair tests from rest of osd/

They are slow.  Run them separately.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/ceph-helpers: factor rbd pool create out of run_mon

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/mon/osd-pool-create: stop testing create pool output

Signed-off-by: Sage Weil <sage@redhat.com>

qa: move ceph-helpers and misc src/test/*.sh tests to qa/standalone

- stop running via make check
- add teuthology yamls to run them
- disable ceph_objecstore_tool.py for now (too slow for make check, and
we can't use vstart in teuthology via a package install)
- drop cephtool tests since those are already covered by other teuthology
tests
- leave a handful of (fast!) ceph-helpers tests for make check for minimal
integration tests.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16468 from gmayyyha/mgr-show-rbd-features

mgr/dashboard: show rbd image features

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

mgr/dashboard: rbd image show features

Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>

Merge pull request #16537 from shangzhong/wip-update-organization

mailmap: add affiliation for Zhu Shangzhong

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #16427 from shangzhong/wip-20685

ceph-disk: s/ceph_osd_mkfs/command_check_call/

Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #16362 from tchaikov/wip-ceph-disk

ceph-disk: use '-' for feeding ceph cli with stdin

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

install-deps.sh: add missing dependencies for FreeBSD

Signed-off-by: Alan Somers <asomers@gmail.com>

mgr/MgrClient: do not attempt to access a global variable for config

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados: add missing implementations for C service daemon API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/cephtool/test.sh: disable 'fs status' until bug is fixed

See http://tracker.ceph.com/issues/20761
Signed-off-by: Sage Weil <sage@redhat.com>

deb,rpm: require socat for ceph-test

Used by qa/standalone/mon/mon-bind.sh.

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_compression: speed it up

~4m -> ~25s

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/workunit: allow alt basedir

Instead of 'qa/workunits' allow something like 'qa/standalone'.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph: drop --admin-socket warning/error on ceph cli

'ceph daemon' has existed for a long time; this has outlived its
usefullness.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: restore admin socket path in mrgw.sh

ceph-conf was rejecting the entity name with:

error parsing 'radosgw.8000': expected string of the form TYPE.ID, valid
types are: auth, mon, osd, mds, mgr, client

these mstart/mrun/mrgw scripts carefully place their output files under
their respective run/<cluster-name>/out subdirectory to keep them from
clashing

Signed-off-by: Casey Bodley <cbodley@redhat.com>

osd: populate last_epoch_split during build_initial_pg_history

If we get a pg create and have to generate the pg_history_t fields,
populate the last_epoch_split field too.  This is needed discard ops sent
from before the last split.  Specifically, PG::can_discard_op() looks at
it when the client has the RESEND_ON_SPLIT feature bit.  If we don't
discard, we may run afoul of assertions later (e.g., that the object
belongs to the PG at all) or potentially process an op out of order.

Fixes: http://tracker.ceph.com/issues/20754
Signed-off-by: Sage Weil <sage@redhat.com>