doc/ceph-volume create a migration section to help users coming from ceph-disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-disk add deprecation warnings in favor of ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Fixes: http://tracker.ceph.com/issues/22154

Merge pull request #18922 from Synesis-LLC/fix-22122

rgw: fix radosgw-admin bucket rm with --purge-objects and --bypass-gc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18956 from Jeegn-Chen/wip-issue-22129

rgw: AWS v4 authorization work when INIT_MULTIPART is chunked

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #18984 from ceph/wip-cv-unittest-fix

ceph-volume tests.devices.lvm prepare isn't bluestore specific anymore

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18968 from joscollin/wip-rgwadmin-rewrite

rgw: fix rewrite options usage text

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17797 from amitkumar50/cov-msg-3

msg: Initialize lkey,bound,port_cnt,num_chunk,gid_idx

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17874 from ShowingCloud/master

ceph-bluestore-tool: check if bdev is empty on 'bluefs-bdev-expand'

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18090 from amitkumar50/test-cov-8

test: checking negative returns from creat()

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-volume tests.devices.lvm prepare isn't bluestore specific anymore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

rgw: fix rewrite options usage text

Fixed rewrite options usage text properly.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18950 from sknorr/fix-rn-typos

doc: Fix typos in release notes

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #17977 from cbodley/wip-rgw-beast-qa

qa/rgw: add beast frontend to some rgw suites

Merge PR #18600 into master

* refs/pull/18600/head:
MDSMonitor: only clog changes to active

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

githubmap: update contributors

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18774 into master

* refs/pull/18774/head:
mds: fix CDir::log_mark_dirty()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18887 from dillaman/wip-20789

librbd: compare and write against a clone can result in failure

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18924 from ceph/wip-cv-filestore-devices

ceph-volume: allow using a device or partition for `lvm --data`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18700 from tianshan/fix_cls_bucket_head

rgw: fix cls_bucket_head result order consistency

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18926 from theanalyst/wip-mp-jewel-fix

rgw: set sync_from_all as true when no value is seen

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph-volume tests.functional create a common playbook directory for setups

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume lvm prepare can accept devices for --data

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add setup playbook for xenial filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add data device and journal to xenial filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add setup playbook for xenial bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add raw data device to xenial bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional tox.ini addition of setup playbook for partitions

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add raw data device to centos7 bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add setup playbook for centos7 bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add setup playbook for centos7 filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional add raw device to centos7 filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.common update --data flag to reflect device acceptance

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #18969 from vshankar/rbd-mirror-image-map-state-transition

rbd-mirror: use next transition state to check transition completeness

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18960 from Liuchang0812/mailmap-update

.mailmap, .organizationmap: update JingChen, ZongyouYao, ShanchunLv's…

test/librbd: disable compare and write if skipping partial discard

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: refactor object write request state machines

Fixes: http://tracker.ceph.com/issues/20789
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: consolidate all object discard-related logic into single state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: simplify interface between object IO and copyup state machines

The initial copyup was not receiving a write hint and the
code for hints was duplicated multiple times. Additionally,
the object map state should match the last executed write op.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: copyup state machine needs to handle empty write ops

The compare-and-write object operation cannot be executed
concurrently within a copyup operation since the object might not
exist yet (if not performing a deep-copy).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: object cacher should re-use read state machine

This adds support for sparse-reads and ensures all object reads
utilize a single, tested code path.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: refactor io::ObjectReadRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: default template parameter for C_SparseReadRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: reduce lock scope when assembling read results

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ObjectReadRequest does not require buffer extents

The ReadResult wrapper requires the buffer extents to know
how to properly reconstruct the out buffer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: templatize IO object request state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados_test_stub: mocked remainder of IO ops used by librbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados_test_stub: simulate set_alloc_hint creating a non-existent file

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Fix typos in release notes

Signed-off-by: Stefan Knorr <sknorr@suse.de>

rbd-mirror: use next transition state to check transition completeness

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #18952 from dillaman/wip-test-librbd

test/librbd: clean up for several mock function tests

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18947 from dillaman/wip-22131

rbd-nbd: certain kernels may not discover resized block devices

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18909 from dillaman/wip-22120

librbd: possible deadlock with synchronous maintenance operations

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18923 from dillaman/wip-21966

librbd: prevent overflow of discard API result code

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

rgw: fix radosgw-admin bucket rm with --purge-objects and --bypass-gc

Call RGWRados::delete_bucket() from rgw_remove_bucket_bypass_gc()
instead of partial copy of code of RGWRados::delete_bucket().

Fix updating user stats after radosgw-admin bucket rm --purge-objects and --bypass-gc
Due to rgw_user(const std::string& s) was called incorrect version of rgw_bucket_sync_user_stats().

Fixes: http://tracker.ceph.com/issues/22122
Fixes: http://tracker.ceph.com/issues/19959
Signed-off-by: Aleksei Gutikov <aleksey.gutikov@synesis.ru>

Merge pull request #18918 from ZVampirEM77/wip-em-rewrite-cmd

rgw: add rewrite cmd and options into radosgw-admin usage and doc

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

.mailmap, .organizationmap: update JingChen, ZongyouYao, ShanchunLv's mailmap and affiliation

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

rgw: add rewrite related cmd and parameter into radosgw-admin help info

+ radosgw-admin object rewrite
+ radosgw-admin bucket rewrite
+ --min-rewrite-size
+ --max-rewrite-size
+ --min-rewrite-stripe-size

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

Merge pull request #18808 from tchaikov/wip-qa-max-pg-per-osd

qa: do not wait for down/out osd for pg convergence

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15788 from vshankar/mirror-ha-poolwatcher-policy

rbd-mirror: track images in policy map in support of A/A

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: AWS v4 authorization work when INIT_MULTIPART is chunked

Add RGW_OP_INIT_MULTIPART as a the single chunk special case
like RGW_OP_COMPLETE_MULTIPART.

Fixes: http://tracker.ceph.com/issues/22129
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>

Merge pull request #18953 from cbodley/wip-22139

common: fix BoundedKeyCounter const_pointer_iterator

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

common: fix BoundedKeyCounter const_pointer_iterator

with libc++, clang fails to compile a call to:

  vector::assign(const_pointer_iterator, const_pointer_iterator)

because const_pointer_iterator does not satisfy the InputIterator
concept. added the necessary typedefs for value_type and reference
to reflect the pointer type

Fixes: http://tracker.ceph.com/issues/22139
Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/librbd: fixed gmock uninteresting function call warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: eliminate sleeps within deep-copy mock tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18883 from cbodley/wip-22083+

rgw: set num_shards on 'radosgw-admin data sync init'

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: set num_shards on 'radosgw-admin data sync init'

Fixes: http://tracker.ceph.com/issues/22083
Signed-off-by: Casey Bodley <cbodley@redhat.com>

dencoder/rgw: expose rgw sync status types

Signed-off-by: Casey Bodley <cbodley@redhat.com>

librbd: added missing locks for snap rollback and rename

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18940 from trociny/wip-valgrind-DeadLockOwner

test/librbd: valgrind warning in TestMockManagedLockBreakRequest.DeadLockOwner

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17761 from cbodley/wip-rgw-bilog-trim

rgw multisite: automated trimming for bucket index logs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #18811 from oritwas/wip-rgw-22046

tenant fixes for dynamic resharding

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18920 from yaozongyou/wip-ceph-dencoder-add-types

ceph-dencoder: add support for cls_rgw_lc_obj_head

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18946 from joscollin/wip-add-cbodley

.githubmap: Add cbodley

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18900 from qrGitHub/master

rgw.iam: change '1' to '1ULL' in function print_actions

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

librbd: invalidating the cache shouldn't hold write lock

This can cause deadlock when readahead is in-progress since neither
can make forward progress.

Fixes: http://tracker.ceph.com/issues/22131
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-nbd: rescan partition table after image resize event

Fixes: http://tracker.ceph.com/issues/22131
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

.githubmap: Add cbodley

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18921 from Leeshine/wip-data-sync-cmd

rgw: add metadata and data sync related cmd into radosgw-admin usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

rgw: add data sync related cmd into radosgw-admin help info

+ radosgw-admin data sync status
+ radosgw-admin data sync init
+ radosgw-admin data sync run

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: add metadata sync related cmd into radosgw-admin help info

+ radosgw-admin metadata sync status
+ radosgw-admin metadata sync init
+ radosgw-admin metadata sync run

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

librbd: prevent overflow of discard API result code

Prevent discard/writesame lengths larger than 2GB.

Fixes: http://tracker.ceph.com/issues/21966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: valgrind warning in TestMockManagedLockBreakRequest.DeadLockOwner

When running with 'debug_rbd >= 20':

  Conditional jump or move depends on uninitialised value(s)
    by 0x94277B: operator<< (ostream:167)
    by 0x94277B: librbd::managed_lock::BreakRequest<librbd::(anonymous namespace)::MockTestImageCtx>::handle_get_watchers(int) (BreakRequest.cc:101)

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

rgw: fix cls_bucket_head result order consistency

headers's key is string but bucket_instance_ids's key is int
looked around and found no one use header's key, so change
header from map to vector

Signed-off-by: Tianshan Qu <tianshan@xsky.com>

Merge pull request #18905 from ifed01/wip-bluestore-recalc-alloc

os/bluestore: remove ineffective BlueFS fnode extent calculation.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18903 from jan--f/jan-mgr-prometheus-unkown-pgstate

pybind/mgr/prometheus: don't crash when encountering an unknown PG state

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18885 from dzafman/wip-22112

ceph-objectstore-tool: Fix set-size to clear data_digest if changing …

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18622 from majianpeng/bluestore-opseq

os/bluestore: OpSequencer: reduce kv_submitted_waiters if _is_all_kv_submitted() return true.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #17032 from yangdongsheng/rbd_qos

rbd: implement image qos in tokenbucket algorithm

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18483 from Songweibin/wip-list-children2

 rbd: fix rbd children listing when child is in trash

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18007 from yehudasa/wip-rgw-config-desc

options.cc: document rgw config options

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Abhishek Lekshmanan <alekshmanan@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: set sync_from_all as true when no value is seen

In order to support jewel-luminous multisite scenarios where when a
jewel cluster is the master, the sync-from-all json field wouldn't be
set leading to the secondary not syncing data as this value can't be
overriden to true, so defaulting the value of sync_from_all to true when
we don't encounter it in the json

Fixes: http://tracker.ceph.com/issues/22062
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #18894 from liewegas/wip-fix-balancer

mgr/balancer: skip CRUSH_ITEM_NONE

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18763 from cfanz/master

rgw: don't write bucket_header when it is not changed in rgw_bucket_prepare_op

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18852 from theanalyst/wip-22083

rgw: data sync: set num_shards when building full maps

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #18866 from cbodley/wip-22101

rgw: RGWEnv::set() takes std::string

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #18875 from yaozongyou/rgw-cleanup

rgw: remove unused disable_signal_fd

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18895 from cooboos/wip-opt-maperase

rgw: optimize the rgw-attr del code logic

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18845 from yaozongyou/wip-lifecycle-log-level

rgw: update life cycle related log level

Reviewed-by: Casey Bodley <cbodley@redhat.com>

librbd: possible deadlock with synchronous maintenance operations

Fixes: http://tracker.ceph.com/issues/22120
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph-dencoder: adding support for cls_rgw_lc_obj_head

After adding cls_rgw_lc_obj_head, we can using the following command
to look into lc processing related information:
rados -p default.rgw.log getomapheader lc.0 ./header.bin --namespace=lc
ceph-dencoder type cls_rgw_lc_obj_head import ./header.bin decode dump_json

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>