rgw: remove unnecessary data copying in RGWPutMetadataBucket.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: Fix updating CORS/ACLs during POST on Swift's container.

Introduced in: 7a7de9b75265b978ba4e53966f614fac033972cb
Fixes: http://tracker.ceph.com/issues/15976
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #9002 from rzarzynski/wip-rgw-swift-acct_cont_md-fix

rgw: fix updating account/container metadata of Swift API.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9225 from dachary/wip-run-make-check

tests: rm -fr /tmp/*virtualenv*

Reviewed-by: Erwan Velu <erwan@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9165 from mikulely/fix-rgw-multisite-test

rgw: misc fix for test_multi.py

Reviewed-by: Casey Bodley <cbodley@redhat.com>

tests: rm -fr /tmp/*virtualenv*

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #9139 from majianpeng/bluestore-statfs

os/bluestore/BlueStore: free space of bluestore should reduce spaces …

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9161 from cxwshawn/wip-tc1-fix

osd: optimize header file dependency to decreace compile time when modify any file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9193 from cxwshawn/wip-tc2-fix

mon: optimize header file dependency.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9068 from trociny/wip-pybind-rbd-mirror

librbd: initial python APIs to support mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9088 from trociny/wip-15500

qa: dynamic_features.sh: return error only if it failed on alive QEMU

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9140 from zhouyuan/fix_disable_journaling

librbd: close journal before removing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9177 from trociny/wip-15916

rbd-mirror: it might got stuck in calculating "behind master" position

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9196 from tchaikov/wip-cmake-vstart

cmake: add cython_rados to vstart target

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8164 from rzarzynski/wip-rgw-bulkdel-slashfix

rgw: handle initial slashes properly in BulkDelete of Swift API.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: handle initial slashes properly in BulkDelete of Swift API.

Fixes: http://tracker.ceph.com/issues/15948
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #9220 from cxwshawn/wip-tc3-fix

auth: optimize header file dependency.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9212 from liewegas/wip-bit-alloc-test

tests: disable unittest_bit_alloc for now

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #9219 from renhwztetecs/renhw-wip-logmonitor-cleanup

mon/log: clean unused function and comments

Reviewed-by: Sage Weil <sage@redhat.com>

test: file change affected by auth file header optimization.

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

AUTH: optimize header file dependency.
same work as PR: https://github.com/ceph/ceph/pull/9193, https://github.com/ceph/ceph/pull/9161

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

mon/log: using standard macros instead of -1

using standard macros instead of -1

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mon/log: clean unused function and comments

clean unused function and comments

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #8656 from kmroz/wip-rgw-handler-cleanup-1

rgw-rest: remove redundant virtual functions
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8886 from cbodley/wip-rgw-zone-short-ids

rgw: check for short_zone_id collisions on period update
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9148 from rzarzynski/wip-rgw-15902

rgw: camelcase names of custom attributes in Swift's responses.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9042 from rzarzynski/wip-rgw-15812

rgw: handle errors properly during GET on Swift's DLO.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9149 from cbodley/wip-rgw-zone-modify-master

rgw: add_zone only clears master_zone if --master=false
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9154 from yehudasa/wip-15886

rgw: keep track of written_objs correctly
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9190 from dillaman/wip-15928

librbd: metadata retrieval added to open image state machine

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9191 from dillaman/wip-15930

rbd-mirror: disable librbd caching for replicated images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9173 from dillaman/wip-15909

rbd-mirror: potential crash during image status update

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8983 from dzafman/wip-li-rados

tools/rados: Improve list-inconsistent json format

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: misc fix for test_multi.p

remove redundent meta_checkpoint, cause realm.create_user(user) will
call meta_checkpoint by default(via wait_meta = True).

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

OSD: optimize header file dependency to decreace compile time when modify any file
    OSD's left header dependency optimize.
Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

PG: optimize header file dependency to decreace compile time when modify any file
PG Layer's header dependency optimize.
Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

PGBackend: optimize header file dependency to decreace compile time when modify any file

hmm, everytime modify any file, then compile, then wait&wait&wait, so trying to optimize
this part, wish to decrease header dependency, try to improve compile speed.

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

tool: change affected file include relation by mon file header optimize

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

MON: optimize header file dependency.

same work as PR: https://github.com/ceph/ceph/pull/9161

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

Merge pull request #9108 from tchaikov/wip-13990

osd: reset session->osdmap if waiting_for_pg is empty

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8908 from jcsp/wip-15705

mds: fix mdsmap print_summary with standby replays

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tests: disable unittest_bit_alloc for now

See http://tracker.ceph.com/issues/15941

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9199 from jmunhoz/mc-brp-aws4

doc: rgw: minor cleanup and doc update

Merge pull request #9133 from tchaikov/wip-cmake-selinux

cmake: add selinux support

Reviewed-by: Casey Bodley <cbodley@redhat.com>

journal: reset watch step after pruning expired tag

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: additional debug messages during image replayer start/stop

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: ensure proper handling of status updates during shutdown

Previously, several shutdown race conditions could occur due to the
use of the async work queue for scheduling updates.

Fixes: http://tracker.ceph.com/issues/15909
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: track bootstrap state within image status

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: combine ImageReplayer stopped and uninitialized states

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: lock ordering issue in status update callback

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9192 from tchaikov/cmake-object-library

cmake: use object library instead recompiling popular .cc files

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd-mirror: disable librbd caching for replicated images

Each image has its own cache and each cache uses its own thread. With
a large replicated cluster, this could result in thousands of extra
threads and gigabytes of extra memory.

Fixes: http://tracker.ceph.com/issues/15930
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9146 from cbodley/wip-rgw-period-push

radosgw-admin: fix 'period push' handling of --url
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: doc: bucket request payment update

S3 doc update.

The bucket request payment feature is available in Jewel

commits:

899d7754fe15ab8635fb506fcadafa8e5e92c376
461bd7fa08710be994c522c4d173a4151dbc173a

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

test: initial python APIs to support mirroring

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd: initial python APIs to support mirroring

Fixes: http://tracker.ceph.com/issues/15656
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rgw: aws4: fix log message

With the UNSIGNED-PAYLOAD bugfix in place this log message requires an update.

Requests asking for unsigned payloads don't need any completion handler so
notifying about body detection in this point is not accurate.

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

Merge pull request #8553 from yatinkumbhare/doc-fix

doc: fix broken link for bootstrapping monitors

cmake: add cython_rados to vstart target

"ceph" cli imports rados python module

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9182 from objoo/YD-mailmap-updates-MAY

Leftover of previous mailmap not merged.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #9143 from tchaikov/wip-osdmaptool-man

man: osdmaptool: document test-map-pgs* options

Reviewed-by: Vikhyat Umrao <vumrao@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

cmake: make common/TextTable.cc an object library

so we don't recompile it whenever it is used.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: reuse secret.c's object library

so we don't need to recompile it when using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9153 from ceph/openstack_wusui

OpenStack setup scripts.

librbd: metadata retrieval added to open image state machine

Fixes: http://tracker.ceph.com/issues/15928
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: async version of metadata_list helper method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8669 from dreamhost/wip-15532

rgw/s3website: ErrorDoc & Double-fault support

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #9150 from stiopaa1/compatset_removeIncludeVector_mc

common: CompatSet.h: remove unneeded #include<vector>

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9158 from cxwshawn/wip-tc-fix

osd: erase duplicated header include

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9169 from stiopaa1/osd_hitSetRemoveFormatterInclude

osd/HitSet: remove unneeded #include - Formatter

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9186 from stiopaa1/osd_fixTypoInComment

osd/OSD.h: fix typo in comment

osd/OSD.h: fix typo in comment

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

mailmap: Luo Runbing affiliation (was "runsisi", name normalization between Huazhong University of Science and Technology part & ZTE .)

Signed-off-by: Yann Dupont <yann@objoo.org>

xio: rename enqueue_for_send() to enqueue()Reviewed-by: Sage Weil <sage@redhat.com>

xio: rename enqueue_for_send() to enqueue()

Reviewed-by: Sage Weil <sage@redhat.com>

mailmap: Jenkis affiliation (name normalization)

Signed-off-by: Yann Dupont <yann@objoo.org>

Merge pull request #9181 from dillaman/wip-rbd-mirror-test

qa/workunits/rbd: fixed rbd_mirror teuthology runtime errors

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

mailmap: Fangchen Sun affiliation (and name normalisation, was "sunspot")

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Jacek J. Łakis affiliation (name normalisation)

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Sangdi Xu affiliation (name normalisation)

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Roland Mechler  affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Etienne Menguy affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

radosgw-admin: fix 'period push' handling of --url

was calling send_to_remote_gateway(), but passing 'url' instead of
'remote'. now uses send_to_remote_or_url() to accept either

Fixes: http://tracker.ceph.com/issues/15926
Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/workunits/rbd: fixed rbd_mirror teuthology runtime errors

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Add date to README

Merge pull request #8307 from songbaisen/song16

mon: paxos small optimization on store_state

Reviewed-by: Kefu Chai <kchai@redhat.com>

Pipe: take a ref to existing while we are waiting

Pipe: take a ref to existing while we are waiting

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9159 from tchaikov/wip-kill-warnings

osd,auth: silence "-Wmisleading-indentation" warnings

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

journal: live replay might skip entries from previous object setReviewed-by: Mykola Golub <mgolub@mirantis.com>

journal: live replay might skip entries from previous object set

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9092 from trociny/wip-15746

doc: update mirroring guide to include pool/image status commands

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: rbd-mirror daemon stress test

This test repeatedly runs rbd bench-write, kills the process
randomly to create an unclean journal shutdown, and verifies
that the image content replicates correctly.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: skip partially complete tag entries during playback

If a journal client does not fully write out its buffered entries
before quiting, replay should skip over all remaining out-of-
sequence entries for the tag.

Fixes: http://tracker.ceph.com/issues/15864
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: helper method to detect newer tags

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: close, advance, and open object set ordering

Flush in-flight appends to open objects before advancing the
active object set.  Additionally, don't start recording to the
new objects until after advancing the active set.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: new ObjectRecorder closed callback

The callback will be invoked if there were in-flight appends
when the close was requested.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: do not flag append as full if already known to be full

Once an object has overflowed or a close is in-progress, re-attempting
to close the object is not needed since the async process is already
underway.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: delay object overflow event until in-flight appends settled

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: ignore flush on closed/overflowed object

The journal would be in-progress on transitioning to a new
object recorder in a newer object set.  Once the records
re-attach to the new object player they will automatically
flush.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: implicitly detach future's flush handler on append

If the future is already in-flight, there is no purpose served
by requesting the future be flushed to disk.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: async callback for advancing the active object set

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: re-fetch active object before advancing set during replay

During a live replay, it's possible that an append and and overflow
into the next object could race with the live playback of the same
object.  Re-fetch an "empty" object at least once before advancing
to next set to ensure all records have been read.

Fixes: http://tracker.ceph.com/issues/15665
Signed-off-by: Jason Dillaman <dillaman@redhat.com>