Merge pull request #17654 from yuyuyu101/wip-spdk-doc

doc/bluestore: add SPDK usage for bluestore

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17658 from ukernel/wip-cephfs-seekdir

doc/cephfs/posix: remove stale information for seekdir

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17669 from ygtzf/parameter-default-value-error

doc: update default value of parameter mon_subscribe_interval

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17672 from FrankYu/master

doc: the client inputs the pool name instead of pool ID

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge PR #17679 into master

* refs/remotes/upstream/pull/17679/head:
qa: get asok path from ceph.conf
qa: use config_path property instead of literal

Reviewed-by: John Spray <john.spray@redhat.com>

qa: get asok path from ceph.conf

The asok path was recently moved to /tmp to resolve unix path too long errors.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use config_path property instead of literal

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17650 from ceph/wip-ceph-deploy-fix

qa/tests: Fix get_system_type failure due to invalid remote name

Merge pull request #17636 from dillaman/wip-21319

rbd: clean up warnings when mirror commands used on non-setup pool

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

doc: the client inputs the pool name instead of pool ID

Signed-off-by: Frank Yu <flyxiaoyu@gmail.com>

Merge pull request #13383 from wonzhq/wip-update-calc-stats

osd: change the conditional in _update_calc_stats

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17640 from tchaikov/wip-diagram-py3

doc/scripts: py3 compatible

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: update default value of parameter mon_subscribe_interval

Signed-off-by: yaoguotao <yaoguot@gmail.com>

Merge pull request #17639 from tchaikov/wip-doc-ceph-disk

doc: update ceph-disk with a state-transition diagram

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #17538 from dzafman/wip-21272

Add export and remove ceph-objectstore-tool command option

Reviewed-by: Sage Weil <sage@redhat.com>

doc/bluestore: add SPDK usage for bluestore

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17537 from mychoxin/yuanxin_remove_submit

os/bluestore: remove useless function submit

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

doc/cephfs/posix: remove stale information for seekdir

Current cephfs can support seekdir efficiently. The diverge was
fixed by https://github.com/ceph/ceph/pull/14317

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #17606 from ceph/wip-bz1478598

ceph-volume allow filtering by `uuid`, do not require osd id

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume lvm.trigger just split the osd_id once

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests catch improper parsing of osd uuid

When the osd ID existed in the UUID in the same format (e.g. '1-') the
parsing would break returning a bogus UUID that is impossible to find

Signed-off-by: Alfredo Deza <adeza@redhat.com>

tests: remove 'ceph-' prefix from test node hostnames

This updates our Vagrantfile to match the changes in ceph-ansible
introduced by https://github.com/ceph/ceph-ansible/commit/298a63c43

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume lvm.api PVolumes should filter by all matching tags, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests pv API should also match all pv_tags

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests prove all volume tags have to match in vgs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #17377 from cbodley/wip-rgw-test-multi-encryption

test/rgw: fix test_encrypted_object_sync for 3+ zones

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #17102 from mychoxin/yuanxin_print_log_if_add_event_fail

msg/async: print error log if add_event fail

Reviewed-by: Pan Liu <liupan1111@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16492 from yehudasa/wip-rgw-multi-log

rgw: multisite log tracing

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17400 from theanalyst/fix/rgw-set-attrs-mtime

rgw: rgw_rados: set_attrs now sets the same time for BI & object

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

admon/build-doc: be compatible with py3

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados: fix the method names

otherwise, sphinx fails to run with py3

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: switch to upstream sphinx-ditaa

it works with setuptools and is now compatible with py3

the py3 branch is created to track the upstream's master branch

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17638 from idryomov/wip-rbd-already-mapped-fixes

krbd: rewrite "already mapped" code

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

ceph-volume lvm.api all vg tags should match, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests prove all tags have to match in lvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.api all lv tags should match, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.activate allow not using osd_id

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests prove missing args.osd_id is fine

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests add is_root and lv fixtures to conftest

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/scripts/gen_state_diagram.py: port to py3

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: update ceph-disk with a state transition diagram

also, quote the variabls in source code with double quote.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: update unmap.t for "already mapped" warning

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #17621 from idryomov/wip-krbd-xfstests-blockdev

qa/suites/krbd: enable xfstests blockdev tests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

krbd: rewrite "already mapped" code

The "already mapped" code, introduced in commit d6a66fc8f49b ("rbd:
before rbd map, warn if the image is already mapped") is broken:
because of a use-after-free on attribute strings, the warning isn't
even printed half the time.

Rewrite making use of udev enumeration filters and fix the interface
while at it.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Fix get_system_type failure due to invalid remote name

recent changes caused the remote name to be invalid, fix the
arg passed to get_system_type

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #17453 from kungf/eversion_t_opt_max

osd_types: define max in eversion_t::max() to static

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17571 from jcsp/wip-21260

mon: fix dropping mgr metadata for active mgr (#21260)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17572 from xiexingguo/wip-os-tool

tools/ceph_objectstore_tool: fix 'dup' unable to duplicate meta PG

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17585 from jdurgin/wip-recovery-delete-encoding-master

MOSDPGRecoveryDelete[Reply]: bump header version

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17588 from kungf/aio_submit_size_change

os/bluestore: shrink aio submit size to pending value

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17599 from tchaikov/wip-encoding-optional

encoding: reset optional<> if it is uninitialized

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17600 from jecluis/wip-21309

mon/OSDMonitor: don't create pgs if pool was deleted

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

krbd: factor out enumerate_devices()

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #17298 from xiexingguo/wip-pg-null-obc-access

osd/PrimaryLogPG: maybe_handle_manifest_detail - sanity check obc existence

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #17559 from dillaman/wip-21181

qa/workunits/rbd: relax greps to support upgrade formatting change

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #17324 from smithfarm/wip-rbd-symlink

build/ops: rpm: drop legacy librbd.so.1 symlink in /usr/lib64/qemu

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Ademar de Souza Reis Jr <areis@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #17512 from tchaikov/wip-chmod-x

common,mon,osd,pybind: silence warning and remove executable mode bit

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #17539 from tchaikov/wip-21204

mon,option: set default value for mon_dns_srv_name

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17549 from iliul/fix-texttable-align

tools: Modify "rados df" header's alignment

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph-objectstore-tool: Make pg removal require --force

Add new export-remove to combine the 2 operations

Fixes: http://tracker.ceph.com/issues/21272
Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Better messages for bad --journal-path

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Fix ceph-objectstore-tool test for standalone and latest code

vstart.sh now defaults to bluestore, so specify filestore
Set environment for run-standalone.sh and cmake build
Create td/cot_dir as test directory
Crush output format change
Change dir into test directory
Give a little time after pool creation
Check for core files as ceph-helpers.sh does

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Move ceph-objectstore-tool test to standalone

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #17096 from ceph/wip-ansbile-fixes

[qa/tests]: misc ceph-ansible fixes and udpate

rbd: mirror "get" actions now have cleaner error messages

Fixes: http://tracker.ceph.com/issues/21319
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17513 from Liuchang0812/wip-max-avail-in-df

mon: incorrect MAX AVAIL in "ceph df"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

cls/rbd: avoid recursively listing the watchers on rbd_mirroring object

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #17533 into master

* refs/remotes/upstream/pull/17533/head:
qa: log ceph-fuse kill/cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17506 into master

* refs/remotes/upstream/pull/17506/head:
mds: reset Formatter on error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17466 into master

* refs/remotes/upstream/pull/17466/head:
qa: whitelist expected MDS_READ_ONLY for kcephfs test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17456 into master

* refs/remotes/upstream/pull/17456/head:
mds: print subtrees only after merge

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #17452 into master

* refs/remotes/upstream/pull/17452/head:
Make standby-replay mds not migrate empty subtree when trimming cache
Make standby-replay mds avoid initiating subtree export

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #17384 into master

* refs/remotes/upstream/pull/17384/head:
ceph.in: validate service glob

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #16752 into master

* refs/remotes/upstream/pull/16752/head:
client: NULL check before append value to bufferlist

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Piotr Dałek <piotr.dalek@corp.ovh.com>

Stop the mgr on node which is not client, this will ensure
the client.0's mgr is active mgr for workunit to work.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #16272 from liewegas/wip-balancer

mgr/balancer: mgr module to automatically balance PGs across OSDs

Reviewed-by: John Spray <john.spray@redhat.com>

os/bluestore: shrink aio submit size to pending value

Signed-off-by: kungf <yang.wang@easystack.cn>

Merge pull request #17403 from ceph/wip-bz1485011

ceph-volume Persist non-lv devices for journals

Reviewed-by: Andrew Schoen aschoen@redhat.com

Merge pull request #17603 from liewegas/wip-migration-twiddle

doc: restructure bluestore migration insructions

Reviewed-by: David Galloway <dgallowa@redhat.com>

doc: restructure bluestore migration insructions

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17602 from ceph/wip-blumig-doc

doc: Updates to bluestore migration doc

Reviewed-by: Sage Weil <sage@redhat.com>

doc/rados/operations/bluestore-migration: Add instruction for evacuating
host

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #17011 from liewegas/wip-doc-bluestore-migration

doc/rados/operations/bluestore-migration: update docs a bit

Merge pull request #17582 from ivancich/wip-fix-mclock-unittests

test/common: unittest_mclock_priority_queue builds with "make" command

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/rados/operations/bluestore-migration: Fix typo in bluestore migration doc

Signed-off-by: David Galloway <dgallowa@redhat.com>

mon: fix dropping mgr metadata for active mgr

drop_standby() was killing it and it was only getting added
back in certain locations.  Instead, make the metadata
drop conditional and only do it in the places we're
really dropping the daemon, not when we're promoting
it to active.

Fixes: http://tracker.ceph.com/issues/21260
Signed-off-by: John Spray <john.spray@redhat.com>

qa/tasks/mgr: regression test for 21260

(aka http://tracker.ceph.com/issues/21260)

Fixes: http://tracker.ceph.com/issues/21260
Signed-off-by: John Spray <john.spray@redhat.com>

qa/suites/krbd: enable xfstests blockdev tests

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mon/OSDMonitor: don't create pgs if pool was deleted

A pool may be deleted while in the process of creating pgs. If that
happens, we should not queue additional pgs from that pool to be
created.

Fixes: http://tracker.ceph.com/issues/21309
Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #17569 from liewegas/wip-21293

Revert "os/bluestore: pass strict flag to bluestore_blob_use_tracker_t::equal()"

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17352 from liewegas/wip-21171

os/bluestore: drop deferred_submit_lock, fix aio leak

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Pan Liu <liupan1111@gmail.com>

encoding: reset optional<> if it is uninitialized

* should reset it, in case we reuse it after initializing it.
* initialize the value of `p` using the C++11 style initializer, so it
  is zero-initialized.
* revert 2a83ef3c which disables a warning of:
 ./include/encoding.h:317:7: warning: 't' may be used uninitialized in
    this function [-Wmaybe-uninitialized]
 where the `t` is the temporary variable for initializing the value of
 `p`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17593 from shashalu/fix-clean-up-apply_epoch

rgw: drop unused function apply_epoch

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17590 from xiaoxichen/minor_doc_fix

doc: fix typo on specify db block device

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17573 from jcsp/wip-21292

mgr/restful: cleaner message when not configured

Reviewed-by: Boris Ranto <branto@redhat.com>

rgw: drop unused function apply_epoch

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>

doc: fix typo on specify db block device

Should use block.db instead of block-db

Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>

Test ceph-mgr RESTful api

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add additional variables as required by ceph-ansible for upstream
installation.

possible revert if those are made default
issue: https://github.com/ceph/ceph-ansible/issues/1834

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #17536 from ceph/wip-cd-rbd-pool

qa/tests: fix rbd pool creation for systemd tests