qa/workunits: silence py warnings for ceph-disk tests

ceph-disk now prints "depreacted" warning message when it starts. but
the tests parses its stdout and stderr for a json string. so we need to
silence the warnings for the tests.

Fixes: http://tracker.ceph.com/issues/22154
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18980 from shinobu-x/os_filestore_filestore_nit_cleanup

os/filestore/FileStore: Initialized by nullptr, NULL or 0 instead

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18982 from neha-ojha/wip-const-cleanup

osd/PG: const cleanup for recoverable/readable predicates

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18934 from tchaikov/wip-option-return-by-ref

Revert "common/config: return const reference instead of a copy"

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #18916 from tchaikov/wip-22113

osd: subscribe osdmaps if any pending pgs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18816 from amitkumar50/cov-common-2

common: Check this->data.op_size before use

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18810 from majianpeng/fix-21982

osdc/Objecter: record correctly value for l_osdc_op_send_bytes.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18777 from majianpeng/bluestore-collection-list

os/bluestore: optimize _collection_list.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18544 from ddiss/pid_max

sysctl.d: set kernel.pid_max=4194304

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18718 from shinobu-x/sk-ceph_osd

ceph_osd.cc: Drop legacy or redundant code

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18776 from JianyuLi/joa-mul-mds

MDS: make popular counter decay at proper rate

Merge pull request #18784 from taodd/client-bugfix

Client:Fix readdir bug

Merge pull request #18786 from taodd/master

Client: setattr should drop "Fs" rather than "As" for mtime and size

Merge pull request #18836 from taodd/cephfs-hadoop

MDS: correct the error msg when init mon client

Merge pull request #18892 from taodd/client-bugfix2

MDS: add null check before we push_back "onfinish"

Merge pull request #18893 from taodd/mds-inotable-cleanup

mds: simplify the code logic in replay_alloc_ids

Merge pull request #18904 from taodd/mds-sesseionmap-cleanup

mds: get rid of the "if" check which is unnecessary inside a loop

Merge pull request #19003 from dillaman/wip-22158

librbd: set deleted parent pointer to null

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

librbd: set deleted parent pointer to null

Fixes: http://tracker.ceph.com/issues/22158
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18963 from amitkumar50/test-cov-test-9

test: Incorrect conversion to double

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18901 from jan--f/jan-crushtool-decompile-missing-ids

tools/crushtool: skip device id if no name exists

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18939 from majianpeng/bluestore-reap-collection

os/bluestore: fix the wrong usage for map_any

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18913 from yaozongyou/wip-rados-add-touch-command

tools/rados: add touch command to change object modification time

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18848 from jcsp/wip-22098

mgr/dashboard: fix audit log loading

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18847 from jcsp/wip-mgr-mon-upness

mgr/dashboard: show warnings if data is out of date or mons are down

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #18484 from Liuchang0812/wip-mgr-request-osd-metadata

mgr: request daemon's metadata when receiving a report from an unknown server

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18976 from yahooguntu/patch-1

udev: Fix typo in udev OSD rules file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18944 from tchaikov/wip-dpdk-spdk

cmake: WITH_SPDK=ON by default

Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>

msg/async: rebase msg/async/dpdk to spdk/dpdk

Signed-off-by: Liu-Chunmei <chunmei.liu@intel.com>

cmake: use BuildDPDK if libdpdk not found

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add rte_mempool_{ring,stack} to Finddpdk.cmake

* also refactor it a little bit

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: BuildDPDK: do not build kernel modules

see also http://pkgs.fedoraproject.org/cgit/rpms/dpdk.git/tree/dpdk.spec

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: remove duplicated spdk linkage

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: refactor dpdk related code

* move the check of `USE_CRYPTOPP` to $top_srcdir/CMakeLists.txt
* remove reference of DPDK_LIBRARY, it's defined nowhere
* move the dpdk code to a single place

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18988 from ceph/wip-rm22154

ceph-disk: create deprecation warnings

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc/ceph-volume create a migration section to help users coming from ceph-disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-disk add deprecation warnings in favor of ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Fixes: http://tracker.ceph.com/issues/22154

Merge pull request #18917 from ZVampirEM77/wip-em-copyobjdata-cleanup

rgw: cleanup unused parameters in RGWRados::copy_obj_data

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

udev: Fix typo in udev OSD rules file

The rule for lockbox partitions had an invalid UUID for the "change" action.

Signed-off-by: Mitch Birti <yahooguntu@gmail.com>

Merge pull request #18922 from Synesis-LLC/fix-22122

rgw: fix radosgw-admin bucket rm with --purge-objects and --bypass-gc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18956 from Jeegn-Chen/wip-issue-22129

rgw: AWS v4 authorization work when INIT_MULTIPART is chunked

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #18984 from ceph/wip-cv-unittest-fix

ceph-volume tests.devices.lvm prepare isn't bluestore specific anymore

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18968 from joscollin/wip-rgwadmin-rewrite

rgw: fix rewrite options usage text

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17797 from amitkumar50/cov-msg-3

msg: Initialize lkey,bound,port_cnt,num_chunk,gid_idx

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17874 from ShowingCloud/master

ceph-bluestore-tool: check if bdev is empty on 'bluefs-bdev-expand'

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18090 from amitkumar50/test-cov-8

test: checking negative returns from creat()

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-volume tests.devices.lvm prepare isn't bluestore specific anymore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

sysctl.d: set kernel.pid_max=4194304 on 64-bit systems

For CONFIG_BASE_FULL Linux kernels, the maximum number of proc/thread
IDs is set to 32768 by default. This default limit can be quite easily
hit during recovery on nodes with high OSD counts.
To avoid hitting the pid_max default limit, attempt to configure it to
4194304, which corresponds to the maximum limit possible on 64-bit
CONFIG_BASE_FULL kernels.

Fixes: http://tracker.ceph.com/issues/21929
Signed-off-by: David Disseldorp <ddiss@suse.de>

sysctl: relocate 90-ceph-osd.conf under etc/sysctl/

Signed-off-by: David Disseldorp <ddiss@suse.de>

os/filestore: remove redudant check.

If next is null, we already set a non-null to it. So don't check again.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

os/bluestore: using ghobject_t::is_max to judge.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/PG: const cleanup for recoverable/readable predicates

Signed-off-by: Neha Ojha <nojha@redhat.com>

os/bluestore: bypass check if collection has more objects than nonexistent_count.

If next != ghobject_t::get_max(), this mean collection has more data
than nonexistent_count. For this we should bypass check.
And fix wrong comments because _collection_list don't return object num.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

cmake: WITH_SPDK=ON by default

Signed-off-by: wanjun.lp <wanjun.lp@alibaba-inc.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm,deb: add cunit1 as build dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: build spdk/dpdk in cmake modules

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: wanjun.lp <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

os/filestore/FileStore: [Cleanup (trivial)] Initialized by nullptr, NULL or 0 instead

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

rgw: fix rewrite options usage text

Fixed rewrite options usage text properly.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18950 from sknorr/fix-rn-typos

doc: Fix typos in release notes

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17977 from cbodley/wip-rgw-beast-qa

qa/rgw: add beast frontend to some rgw suites

Merge PR #18600 into master

* refs/pull/18600/head:
MDSMonitor: only clog changes to active

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

githubmap: update contributors

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18774 into master

* refs/pull/18774/head:
mds: fix CDir::log_mark_dirty()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18887 from dillaman/wip-20789

librbd: compare and write against a clone can result in failure

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18924 from ceph/wip-cv-filestore-devices

ceph-volume: allow using a device or partition for `lvm --data`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18700 from tianshan/fix_cls_bucket_head

rgw: fix cls_bucket_head result order consistency

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18926 from theanalyst/wip-mp-jewel-fix

rgw: set sync_from_all as true when no value is seen

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph-volume tests.functional create a common playbook directory for setups

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume lvm prepare can accept devices for --data

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add setup playbook for xenial filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add data device and journal to xenial filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add setup playbook for xenial bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add raw data device to xenial bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional tox.ini addition of setup playbook for partitions

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add raw data device to centos7 bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add setup playbook for centos7 bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add setup playbook for centos7 filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add raw device to centos7 filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.common update --data flag to reflect device acceptance

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #18969 from vshankar/rbd-mirror-image-map-state-transition

rbd-mirror: use next transition state to check transition completeness

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18960 from Liuchang0812/mailmap-update

.mailmap, .organizationmap: update JingChen, ZongyouYao, ShanchunLv's…

test/librbd: disable compare and write if skipping partial discard

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: refactor object write request state machines

Fixes: http://tracker.ceph.com/issues/20789
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: consolidate all object discard-related logic into single state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: simplify interface between object IO and copyup state machines

The initial copyup was not receiving a write hint and the
code for hints was duplicated multiple times. Additionally,
the object map state should match the last executed write op.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: copyup state machine needs to handle empty write ops

The compare-and-write object operation cannot be executed
concurrently within a copyup operation since the object might not
exist yet (if not performing a deep-copy).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: object cacher should re-use read state machine

This adds support for sparse-reads and ensures all object reads
utilize a single, tested code path.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: refactor io::ObjectReadRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: default template parameter for C_SparseReadRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: reduce lock scope when assembling read results

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ObjectReadRequest does not require buffer extents

The ReadResult wrapper requires the buffer extents to know
how to properly reconstruct the out buffer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: templatize IO object request state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados_test_stub: mocked remainder of IO ops used by librbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados_test_stub: simulate set_alloc_hint creating a non-existent file

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Fix typos in release notes

Signed-off-by: Stefan Knorr <sknorr@suse.de>

rbd-mirror: use next transition state to check transition completeness

Signed-off-by: Venky Shankar <vshankar@redhat.com>

test: Incorrect conversion to double

Fixes the coverity issue:

CID 1417062 (#1 of 1): Result is not floating-point
(UNINTENDED_INTEGER_DIVISION)
integer_division: Dividing integer expressions 100 and temp
, and then converting the integer quotient to type double.
Any remainder, or fractional part of the quotient, is ignored.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #18952 from dillaman/wip-test-librbd

test/librbd: clean up for several mock function tests

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18947 from dillaman/wip-22131

rbd-nbd: certain kernels may not discover resized block devices

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18909 from dillaman/wip-22120

librbd: possible deadlock with synchronous maintenance operations

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>