pybind/rbd: create/clone/copy: default to None for features param

If None is specified don't set the features so that the defaults is
used.

Fixes: http://tracker.ceph.com/issues/15982
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: clone: default to parent features

Fixes: http://tracker.ceph.com/issues/15982
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8473 from rjfd/wip-14421

rbd-mirror: Delete local image mirror when remote image mirroring is disabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: test: added image-deleter test case when mirroring is disabled

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Unregister clients from non-primary images journal

A non-primary image may have registered clients on its journal
(for instance a primary image that was later demoted). We must
unregister the clients when disabling image mirroring with the
force option.

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: test: Added unit test for testing image-deleter thread

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: test: Fixed test_PoolWatcher to support the new changes in PoolWatcher class

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Replayer: bootstrap existing mirrored images

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: replayer: Added image-deleter thread to replayer

Fixes: http://tracker.ceph.com/issues/14421
Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Added image-deleter thread to rbd-mirror

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Added implementation of image-deleter thread

image-deleter thread will handle the deletion of local images that have
been disabled for mirroring by the primary site.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #9179 from vshankar/rbd-min-commit-position

cls_journal: initialize empty commit position upon client register

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7836 from yuyuyu101/wip-13938

TaskFinisher: use singleton SafeTimer and Finisher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osdc/Objecter: move C_TwoContexts class to cc fileReviewed-by: Sage Weil <sage@redhat.com>

osdc/Objecter: move C_TwoContexts class to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Objecter: move C_TwoContexts class to cc file

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9241 from ceph/wip-fix-q-tests

common/: fix tests for prioritized_queue and wpq

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9102 from yehudasa/wip-rgw-leak-4

rgw: don't unregister request if request is not connected to manager

Reviewed-by: Casey Bodley <cbodley@redhat.com>

common/: fix tests for prioritized_queue and wpq

Introduced: 27b6ec40b93c05fbaa228783e11e2fac8f1a1ee1
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #9216 from dzafman/wip-test-fix

test: Disable tests due to recovery race

test: Disable tests due to recovery race

Disable:
osd-scrub-repair.sh
test-erasure-eio.sh
test-erasure-code.sh

These test are failing due to race conditions that is exposing
bug http://tracker.ceph.com/issues/13937.

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #9211 from dillaman/wip-15938

librbd: write-after-write might result in an inconsistent replicated image

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9231 from athanatos/wip-recovery-wq

osd: Put background recovery tasks in the main OpWQ

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6208 from liewegas/wip-alloc-hints

librados,osd: rados hints for future access pattern

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7070 from majianpeng/headerbeatmap-check

common/HeartbeatMap: print stack info of unhealth thread.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7843 from branch-predictor/bp-smart-async-wp

msg/async: Implement smarter worker thread selection

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8141 from songbaisen/song14

crush: fix the problem can not find the define item below the bucket

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8840 from tchaikov/wip-opt-u32

common/config: cast OPT_U32 options using uint32_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9072 from liewegas/wip-15840

osd: fix crush location update on startup

Merge pull request #9116 from ddiss/librados_sparse_read_use_snapid

librados: use caller provided snapid for aio_sparse_read()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #9002 from rzarzynski/wip-rgw-swift-acct_cont_md-fix

rgw: fix updating account/container metadata of Swift API.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

cls_journal: Select min commit position for new clients

When a new client registers, its commit position is initialized
to the least commit position of all other clients.

Fixes: http://tracker.ceph.com/issues/15757
Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #9225 from dachary/wip-run-make-check

tests: rm -fr /tmp/*virtualenv*

Reviewed-by: Erwan Velu <erwan@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9165 from mikulely/fix-rgw-multisite-test

rgw: misc fix for test_multi.py

Reviewed-by: Casey Bodley <cbodley@redhat.com>

tests: rm -fr /tmp/*virtualenv*

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #9139 from majianpeng/bluestore-statfs

os/bluestore/BlueStore: free space of bluestore should reduce spaces …

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9161 from cxwshawn/wip-tc1-fix

osd: optimize header file dependency to decreace compile time when modify any file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9193 from cxwshawn/wip-tc2-fix

mon: optimize header file dependency.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9068 from trociny/wip-pybind-rbd-mirror

librbd: initial python APIs to support mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9088 from trociny/wip-15500

qa: dynamic_features.sh: return error only if it failed on alive QEMU

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9140 from zhouyuan/fix_disable_journaling

librbd: close journal before removing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9177 from trociny/wip-15916

rbd-mirror: it might got stuck in calculating "behind master" position

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

journal: replay position might change after pruning stale tags

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: delay commit of overwritten journal event

With the cache enabled and write-after-write IOs to the same
object extents, it was possible for the overwritten journal event
to be committed before the overwriter journal event was written
to disk.  If a client crash occurs before the event is written,
the image will be inconsistent on replay.

Fixes: http://tracker.ceph.com/issues/15938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9196 from tchaikov/wip-cmake-vstart

cmake: add cython_rados to vstart target

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8164 from rzarzynski/wip-rgw-bulkdel-slashfix

rgw: handle initial slashes properly in BulkDelete of Swift API.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: handle initial slashes properly in BulkDelete of Swift API.

Fixes: http://tracker.ceph.com/issues/15948
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #9220 from cxwshawn/wip-tc3-fix

auth: optimize header file dependency.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9212 from liewegas/wip-bit-alloc-test

tests: disable unittest_bit_alloc for now

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #9219 from renhwztetecs/renhw-wip-logmonitor-cleanup

mon/log: clean unused function and comments

Reviewed-by: Sage Weil <sage@redhat.com>

test: file change affected by auth file header optimization.

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

AUTH: optimize header file dependency.
same work as PR: https://github.com/ceph/ceph/pull/9193, https://github.com/ceph/ceph/pull/9161

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

mon/log: using standard macros instead of -1

using standard macros instead of -1

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mon/log: clean unused function and comments

clean unused function and comments

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #8656 from kmroz/wip-rgw-handler-cleanup-1

rgw-rest: remove redundant virtual functions
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8886 from cbodley/wip-rgw-zone-short-ids

rgw: check for short_zone_id collisions on period update
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9148 from rzarzynski/wip-rgw-15902

rgw: camelcase names of custom attributes in Swift's responses.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9042 from rzarzynski/wip-rgw-15812

rgw: handle errors properly during GET on Swift's DLO.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9149 from cbodley/wip-rgw-zone-modify-master

rgw: add_zone only clears master_zone if --master=false
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9154 from yehudasa/wip-15886

rgw: keep track of written_objs correctly
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9190 from dillaman/wip-15928

librbd: metadata retrieval added to open image state machine

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9191 from dillaman/wip-15930

rbd-mirror: disable librbd caching for replicated images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9173 from dillaman/wip-15909

rbd-mirror: potential crash during image status update

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8983 from dzafman/wip-li-rados

tools/rados: Improve list-inconsistent json format

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: misc fix for test_multi.p

remove redundent meta_checkpoint, cause realm.create_user(user) will
call meta_checkpoint by default(via wait_meta = True).

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

TaskFinisher: use singleton SafeTimer and Finisher

Fix #13938
Signed-off-by: Haomai Wang <haomai@xsky.com>

OSD: optimize header file dependency to decreace compile time when modify any file
    OSD's left header dependency optimize.
Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

PG: optimize header file dependency to decreace compile time when modify any file
PG Layer's header dependency optimize.
Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

PGBackend: optimize header file dependency to decreace compile time when modify any file

hmm, everytime modify any file, then compile, then wait&wait&wait, so trying to optimize
this part, wish to decrease header dependency, try to improve compile speed.

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

tool: change affected file include relation by mon file header optimize

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

MON: optimize header file dependency.

same work as PR: https://github.com/ceph/ceph/pull/9161

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

crush: fix the problem that can not find the define item below the bucket

Fixes: http://tracker.ceph.com/issues/15154
Signed-off-by: song baisen <song.baisen@zte.com.cn>

Merge pull request #9108 from tchaikov/wip-13990

osd: reset session->osdmap if waiting_for_pg is empty

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8908 from jcsp/wip-15705

mds: fix mdsmap print_summary with standby replays

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tests: disable unittest_bit_alloc for now

See http://tracker.ceph.com/issues/15941

Signed-off-by: Sage Weil <sage@redhat.com>

ObjectCacher: pass new journal tid when overwriting extent

librbd needs to ensure the new journal event has been safely
committed to disk before it can mark an overwritten journal
event as committed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: record rbd CLI debug messages during mirror stress

The debug messages from 'rbd bench-write' and 'rbd snap create',
in addition to the existing debug messages from rbd-mirror, make
it possible to determine the source of any image inconsistency.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9199 from jmunhoz/mc-brp-aws4

doc: rgw: minor cleanup and doc update

Merge pull request #9133 from tchaikov/wip-cmake-selinux

cmake: add selinux support

Reviewed-by: Casey Bodley <cbodley@redhat.com>

journal: reset watch step after pruning expired tag

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: additional debug messages during image replayer start/stop

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: ensure proper handling of status updates during shutdown

Previously, several shutdown race conditions could occur due to the
use of the async work queue for scheduling updates.

Fixes: http://tracker.ceph.com/issues/15909
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: track bootstrap state within image status

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: combine ImageReplayer stopped and uninitialized states

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: lock ordering issue in status update callback

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9192 from tchaikov/cmake-object-library

cmake: use object library instead recompiling popular .cc files

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd-mirror: disable librbd caching for replicated images

Each image has its own cache and each cache uses its own thread. With
a large replicated cluster, this could result in thousands of extra
threads and gigabytes of extra memory.

Fixes: http://tracker.ceph.com/issues/15930
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9146 from cbodley/wip-rgw-period-push

radosgw-admin: fix 'period push' handling of --url
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: doc: bucket request payment update

S3 doc update.

The bucket request payment feature is available in Jewel

commits:

899d7754fe15ab8635fb506fcadafa8e5e92c376
461bd7fa08710be994c522c4d173a4151dbc173a

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

test: initial python APIs to support mirroring

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd: initial python APIs to support mirroring

Fixes: http://tracker.ceph.com/issues/15656
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rgw: aws4: fix log message

With the UNSIGNED-PAYLOAD bugfix in place this log message requires an update.

Requests asking for unsigned payloads don't need any completion handler so
notifying about body detection in this point is not accurate.

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

Merge pull request #8553 from yatinkumbhare/doc-fix

doc: fix broken link for bootstrapping monitors

cmake: add cython_rados to vstart target

"ceph" cli imports rados python module

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9182 from objoo/YD-mailmap-updates-MAY

Leftover of previous mailmap not merged.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #9143 from tchaikov/wip-osdmaptool-man

man: osdmaptool: document test-map-pgs* options

Reviewed-by: Vikhyat Umrao <vumrao@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

cmake: make common/TextTable.cc an object library

so we don't recompile it whenever it is used.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: reuse secret.c's object library

so we don't need to recompile it when using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9153 from ceph/openstack_wusui

OpenStack setup scripts.

librbd: metadata retrieval added to open image state machine

Fixes: http://tracker.ceph.com/issues/15928
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: async version of metadata_list helper method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>