osd: fix tests of mclock implementations to track opimizations for op wq

wip-osd-deq changes client identifiers from entity_inst_t to
uint64_t. Updates tests to do this as well.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

osd: use deque instead of list for op wq to_process

Fewer allocations.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use global id instead of entity_inst_t as client identifier

entity_inst_t is huge; use uint64_t instead.

Note that this will currently confuse mds.0 vs osd.0, but we plan
to switch the mds to use mds.$gid soon anyway so that will go
away.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17641 from amitkumar50/cov-messages-12

messages: Initialization of variable beat

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #10460 from weiqiaomiao/wqm-wip-rgw-dellc

rgw: add tenant to shard_id in RGWDeleteLC::execute()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #17061 from shashalu/fix-rgw-lc

rgw: fix lc process only schdule the first item of lc objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: fang.yuxiang <fang.yuxiang@eisoo.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #17631 from joke-lee/RGW-CORS-Access-Control-Allow-Origin-HEADER

rgw: return 'Access-Control-Allow-Origin' header when the set bucket versioning through XMLHttpRequest

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17632 from joke-lee/RGW-CORS-Access-Control-Allow-Origin-HEADER-WEBSITE

rgw: return 'Access-Control-Allow-Origin' header when the set and delete bucket website through XMLHttpRequest

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17663 from shashalu/fix-rgw-LCFilter

rgw: Fix closing tag for Prefix

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17543 from iliul/fix-dup-global-section

vstart.sh: Remove duplicate global section

Reviewed-by: Sage Weil <sage@redhat.com>

vstart.sh: Remove duplicate global section

Signed-off-by: iliul <liul124@chinaunicom.cn>

Merge pull request #17544 from joelin/wip-delete-forgot-mark

msg: remove the ),it's redundant.

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #17515 from xiexingguo/wip-data-digest

osd/PrimaryLogPG: do not set data digest for bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17366 from smithfarm/wip-21176

build/ops: python-numpy-devel build dependency for SUSE

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Ewaznazari <Nazari@keyinternational.academy>

Merge pull request #17685 from ygtzf/HashIndex-clerical-error

os/filestore/HashIndex.h: fixed a typo in comment

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17688 from TsaiJin/wip-fix-assert-bug

msg/asyc/rmda: fix the bug of assert when Infiniband::recv_msg receives disconnect message

Reviewed-by: Haomai Wang <haomai@xsky.com>

osd/osd_types.h: default to no data/omap digest for new object

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/PrimaryLogPG: do not generate data digest for BlueStore by default

BlueStore enables CRC by default, so this is a dup and gains
no more benefits.

Turn this off by default, which is good for performance.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17687 from yonghengdexin735/wip-zzz-error

common: fix typo deamon in comments

Reviewed-by: Jos Collin <jcollin@redhat.com>

msg/asyc/rmda: fix the bug of assert when Infiniband::recv_msg receives
disconnect message

Signed-off-by: Jin Cai <caijin.caij@alibaba-inc.com>

common:fix error word

Signed-off-by: yonghengdexin735 <zhang.zezhu@zte.com.cn>

Merge pull request #17613 from smithfarm/wip-12056

doc: document include/ipaddr.h

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17644 from Vogtinator/readme-vstart

README: Improve vstart.sh usage

os/filestore/HashIndex.h: fixed a clerical error

Signed-off-by: yaoguotao <yaoguot@gmail.com>

Merge pull request #17654 from yuyuyu101/wip-spdk-doc

doc/bluestore: add SPDK usage for bluestore

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17658 from ukernel/wip-cephfs-seekdir

doc/cephfs/posix: remove stale information for seekdir

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17669 from ygtzf/parameter-default-value-error

doc: update default value of parameter mon_subscribe_interval

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17672 from FrankYu/master

doc: the client inputs the pool name instead of pool ID

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge PR #17679 into master

* refs/remotes/upstream/pull/17679/head:
qa: get asok path from ceph.conf
qa: use config_path property instead of literal

Reviewed-by: John Spray <john.spray@redhat.com>

qa: get asok path from ceph.conf

The asok path was recently moved to /tmp to resolve unix path too long errors.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use config_path property instead of literal

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17650 from ceph/wip-ceph-deploy-fix

qa/tests: Fix get_system_type failure due to invalid remote name

Merge pull request #17636 from dillaman/wip-21319

rbd: clean up warnings when mirror commands used on non-setup pool

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

doc: the client inputs the pool name instead of pool ID

Signed-off-by: Frank Yu <flyxiaoyu@gmail.com>

Merge pull request #13383 from wonzhq/wip-update-calc-stats

osd: change the conditional in _update_calc_stats

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17640 from tchaikov/wip-diagram-py3

doc/scripts: py3 compatible

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: update default value of parameter mon_subscribe_interval

Signed-off-by: yaoguotao <yaoguot@gmail.com>

Merge pull request #17639 from tchaikov/wip-doc-ceph-disk

doc: update ceph-disk with a state-transition diagram

Reviewed-by: Alfredo Deza <adeza@redhat.com>

rgw: LCFilter_S3::to_xml the second "prefix" missed "/"

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>

Merge pull request #17538 from dzafman/wip-21272

Add export and remove ceph-objectstore-tool command option

Reviewed-by: Sage Weil <sage@redhat.com>

doc/bluestore: add SPDK usage for bluestore

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17537 from mychoxin/yuanxin_remove_submit

os/bluestore: remove useless function submit

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

doc/cephfs/posix: remove stale information for seekdir

Current cephfs can support seekdir efficiently. The diverge was
fixed by https://github.com/ceph/ceph/pull/14317

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #17606 from ceph/wip-bz1478598

ceph-volume allow filtering by `uuid`, do not require osd id

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

messages: Initialization of variable beat

Fixes the coverity issue:

** 717278 Uninitialized scalar field

CID 717278 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member beat is not
initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>

ceph-volume lvm.trigger just split the osd_id once

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests catch improper parsing of osd uuid

When the osd ID existed in the UUID in the same format (e.g. '1-') the
parsing would break returning a bogus UUID that is impossible to find

Signed-off-by: Alfredo Deza <adeza@redhat.com>

tests: remove 'ceph-' prefix from test node hostnames

This updates our Vagrantfile to match the changes in ceph-ansible
introduced by https://github.com/ceph/ceph-ansible/commit/298a63c43

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume lvm.api PVolumes should filter by all matching tags, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests pv API should also match all pv_tags

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests prove all volume tags have to match in vgs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #17377 from cbodley/wip-rgw-test-multi-encryption

test/rgw: fix test_encrypted_object_sync for 3+ zones

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #17102 from mychoxin/yuanxin_print_log_if_add_event_fail

msg/async: print error log if add_event fail

Reviewed-by: Pan Liu <liupan1111@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16492 from yehudasa/wip-rgw-multi-log

rgw: multisite log tracing

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17400 from theanalyst/fix/rgw-set-attrs-mtime

rgw: rgw_rados: set_attrs now sets the same time for BI & object

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

admon/build-doc: be compatible with py3

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados: fix the method names

otherwise, sphinx fails to run with py3

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: switch to upstream sphinx-ditaa

it works with setuptools and is now compatible with py3

the py3 branch is created to track the upstream's master branch

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17638 from idryomov/wip-rbd-already-mapped-fixes

krbd: rewrite "already mapped" code

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

README: Improve vstart.sh usage

Using long options where possible to make it easier to understand directly
and adding bluestore by default as it is the default storage backend.

Signed-off-by: Fabian Vogt <fvogt@suse.com>

ceph-volume lvm.api all vg tags should match, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests prove all tags have to match in lvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.api all lv tags should match, not just one

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.activate allow not using osd_id

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests prove missing args.osd_id is fine

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests add is_root and lv fixtures to conftest

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/scripts/gen_state_diagram.py: port to py3

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: update ceph-disk with a state transition diagram

also, quote the variabls in source code with double quote.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: update unmap.t for "already mapped" warning

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #17621 from idryomov/wip-krbd-xfstests-blockdev

qa/suites/krbd: enable xfstests blockdev tests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

krbd: rewrite "already mapped" code

The "already mapped" code, introduced in commit d6a66fc8f49b ("rbd:
before rbd map, warn if the image is already mapped") is broken:
because of a use-after-free on attribute strings, the warning isn't
even printed half the time.

Rewrite making use of udev enumeration filters and fix the interface
while at it.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rgw: return 'Access-Control-Allow-Origin' header when the set and delete bucket website through XMLHttpRequest

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

rgw: return 'Access-Control-Allow-Origin' header when the set bucket versioning through XMLHttpRequest

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Fix get_system_type failure due to invalid remote name

recent changes caused the remote name to be invalid, fix the
arg passed to get_system_type

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #17453 from kungf/eversion_t_opt_max

osd_types: define max in eversion_t::max() to static

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17571 from jcsp/wip-21260

mon: fix dropping mgr metadata for active mgr (#21260)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17572 from xiexingguo/wip-os-tool

tools/ceph_objectstore_tool: fix 'dup' unable to duplicate meta PG

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17585 from jdurgin/wip-recovery-delete-encoding-master

MOSDPGRecoveryDelete[Reply]: bump header version

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17588 from kungf/aio_submit_size_change

os/bluestore: shrink aio submit size to pending value

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17599 from tchaikov/wip-encoding-optional

encoding: reset optional<> if it is uninitialized

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17600 from jecluis/wip-21309

mon/OSDMonitor: don't create pgs if pool was deleted

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

krbd: factor out enumerate_devices()

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #17298 from xiexingguo/wip-pg-null-obc-access

osd/PrimaryLogPG: maybe_handle_manifest_detail - sanity check obc existence

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #17559 from dillaman/wip-21181

qa/workunits/rbd: relax greps to support upgrade formatting change

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

doc: document include/ipaddr.h

Fixes: http://tracker.ceph.com/issues/12056
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #17324 from smithfarm/wip-rbd-symlink

build/ops: rpm: drop legacy librbd.so.1 symlink in /usr/lib64/qemu

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Ademar de Souza Reis Jr <areis@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #17512 from tchaikov/wip-chmod-x

common,mon,osd,pybind: silence warning and remove executable mode bit

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #17539 from tchaikov/wip-21204

mon,option: set default value for mon_dns_srv_name

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17549 from iliul/fix-texttable-align

tools: Modify "rados df" header's alignment

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph-objectstore-tool: Make pg removal require --force

Add new export-remove to combine the 2 operations

Fixes: http://tracker.ceph.com/issues/21272
Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Better messages for bad --journal-path

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Fix ceph-objectstore-tool test for standalone and latest code

vstart.sh now defaults to bluestore, so specify filestore
Set environment for run-standalone.sh and cmake build
Create td/cot_dir as test directory
Crush output format change
Change dir into test directory
Give a little time after pool creation
Check for core files as ceph-helpers.sh does

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Move ceph-objectstore-tool test to standalone

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #17096 from ceph/wip-ansbile-fixes

[qa/tests]: misc ceph-ansible fixes and udpate

rbd: mirror "get" actions now have cleaner error messages

Fixes: http://tracker.ceph.com/issues/21319
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17513 from Liuchang0812/wip-max-avail-in-df

mon: incorrect MAX AVAIL in "ceph df"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

cls/rbd: avoid recursively listing the watchers on rbd_mirroring object

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #17533 into master

* refs/remotes/upstream/pull/17533/head:
qa: log ceph-fuse kill/cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17506 into master

* refs/remotes/upstream/pull/17506/head:
mds: reset Formatter on error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #17466 into master

* refs/remotes/upstream/pull/17466/head:
qa: whitelist expected MDS_READ_ONLY for kcephfs test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>