doc: fix description for rsize and rasize

Signed-off-by: Andreas Gerstmayr <andreas.gerstmayr@gmail.com>
(cherry picked from commit a004254a02cafdee848a382f893f89c3c5a9845a)

Merge pull request #10804 from dachary/wip-17089-jewel

jewel: OSD failed to subscribe skipped osdmaps after ceph osd pause

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10760 from dachary/wip-16865-jewel

jewel: saw valgrind issues in ReplicatedPG::new_repop

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10662 from dachary/wip-16901-jewel

jewel: segfault in RGWOp_MDLog_Notify

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>

Merge pull request #10763 from dachary/wip-17034-jewel

jewel: rgw: object expirer's hints might be trimmed without processing in some circumstances
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10658 from dachary/wip-16862-jewel

jewel: default zone and zonegroup cannot be added to a realm
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10659 from dachary/wip-16863-jewel

jewel: use zone endpoints instead of zonegroup endpoints
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10660 from dachary/wip-16864-jewel

jewel: multisite segfault on ~RGWRealmWatcher if realm was deleted
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10847 from gregsfortytwo/jewel-fix-mds

mds: fix double-unlock on shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #10663 from dachary/wip-16934-jewel

jewel: Add zone rename to radosgw_admin

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10664 from dachary/wip-16958-jewel

jewel: Bug when using port 443s in rgw.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10764 from dachary/wip-17032-jewel

jewel: multisite: RGWPeriodPuller tries to pull from itself

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10765 from dachary/wip-17006-jewel

jewel: Increase log level for messages occuring while running rgw admin command

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10845 from cbodley/wip-16702

jewel: rgw: add tenant support to multisite sync

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #9405 from SUSE/wip-16083-jewel

jewel: mds: wrongly treat symlink inode as normal file/dir when symlink inode is stale on kcephfs

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10654 from dachary/wip-9577-jewel

jewel: mon: "mon metadata" fails when only one monitor exists

Reviewed-by: John Spray <john.spray@redhat.com>

mds: fix double-unlock on shutdown

We did a bad backport or something and accidentally ended up with two Unlock()
calls on mds_lock. Don't.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

rgw: collect skips a specific coroutine stack

Fixes: http://tracker.ceph.com/issues/16665
Instead of drain_all_but() that specifies number of stacks to leave behind,
added drain_all_but_stack() that has a specific stack specified. This is needed
so that we don't call wakeup() through lease_cr->go_down() on a cr stack that
was already collected.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 5a2e8f0526db92a290c711f82627fc5042c290ea)

Merge pull request #10791 from dachary/wip-17004-jewel

jewel: rbd-mirror: FAILED assert(m_state == STATE_STOPPING)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10827 from dillaman/wip-16950-jewel

jewel: librbd: delay acquiring lock if image watch has failed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: fix compilation

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 874de58024e12d893ae050cd421ef67c9521c753)

rgw: remove datalog keys from error repo on ENOENT

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 16976eedb70292e821193e39bb577a68df1bc95a)

test/rgw: add optional --tenant flag to test_multi.py

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6f65d192cd3917cdcf7d9deb5396750d2c641842)

rgw: add tenant id to GetBucketInstanceInfo

use the rgw_bucket overload of get_bucket_instance_info() so it can
supply the tenant id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3cf715c6828d7d9732bf6a6fd9b46bbd8c08de5f)

rgw: carry tenant id with data sync

use rgw_bucket_shard to track buckets, which includes tenant id

Fixes: http://tracker.ceph.com/issues/16469
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d945e2099525e39588b572e357b115df98c8cdca)

rgw: add tenant to url for RGWRESTStreamWriteRequest

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2a1e1a776c58f62fbcb57877d56b92eb1e8933e6)

rgw: add tenant to rgw_bucket json format

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fc0df4802b48fa095587b93557c82da5446d9ca3)

rgw: add rgw_bucket_parse_bucket_key()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d5ac140040ea2f17e33d2846870cb033c7ae6ffa)

rgw: add tenant name to datalog entries

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c8aa668fc3e34a7651ce6e2dda62902ced808541)

rgw: convert bucket instance listings back to metadata key format

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3eae201c8a1513bae23233700c754ec076360664)

rgw: use tenant/ for bucket instance metadata keys

to work around the ambiguity of parsing tenant: and :shard in the same
bucket instance metadata key, use tenant/ instead

to preserve backward compatibility with existing objects, new helper
function rgw_bucket_instance_key_to_oid() converts this / back to a :
before being used as an object name

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a0befa1e7be3aeb37209bac2e2a7de0edf5d5a95)

rgw: add get_key() methods to format rgw_buckets

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 51ff8ef7c6bb7c9f55f3ca8ad059b029a4088cdf)

rgw: data sync debug logging

a high level debug logging of the data sync process.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 97ef16c0cc0ad8345b5a897108717d83ab0aa9ab)

rgw: modifying multi-site log messages.

Increasing the log level and removing 'ERROR' from messages that appear during normal multi-site operation.
Fixes: http://tracker.ceph.com/issues/16121
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit e1737ddaa035f23757d021dd34b0c87d24ce372c)

Merge pull request #10653 from dachary/wip-10495-jewel

jewel : 60-ceph-partuuid-workaround-rules still needed by debian jessie (udev 215-17)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10357 from SUSE/wip-16748-jewel

jewel: mount.ceph: move from ceph-base to ceph-common and add symlink in /sbin for SUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10790 from dachary/wip-17005-jewel

jewel: ImageReplayer::is_replaying does not include flush state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10792 from dachary/wip-16978-jewel

jewel: rbd-mirror: FAILED assert(m_on_update_status_finish == nullptr)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10646 from dachary/wip-16576-jewel

jewel: rbd-mirror: FAILED assert(m_local_image_ctx->object_map != nullptr)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10647 from dachary/wip-16593-jewel

jewel: FAILED assert(object_no < m_object_map.size())

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10786 from dachary/wip-17061-jewel

jewel: bashism in src/rbdmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10421 from SUSE/wip-16659-jewel

jewel: ReplicatedBackend doesn't increment stats on pull, only push

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10501 from Abhishekvrshny/wip-16621-jewel

jewel: mds:  tell command blocks forever with async messenger (TestVolumeClient.test_evict_client failure)

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10816 from SUSE/wip-17092-jewel

jewel: build/ops: need rocksdb commit 7ca731b12ce for ppc64le build

Reviewed-by: Loic Dachary <ldachary@redhat.com>

librbd: delay acquiring exclusive lock if watch has failed

Fixes: http://tracker.ceph.com/issues/16923
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit dfe9f3eac9cca3b83962e0e1b7eac38e6e76d7a5)

librbd: convert ImageWatcher class to template

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 814c305ce8c35b5ce01d7e29a912d5ef3978754b)

 Conflicts:
src/librbd/ImageWatcher.cc: no shrink guard
src/librbd/Operations.cc: no shrink guard

Merge pull request #10103 from xiaoxichen/wip-16037-jewel

jewel: MDSMonitor::check_subs() is very buggy

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10105 from xiaoxichen/wip-16515-jewel

jewel: Session::check_access() is buggy

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10106 from xiaoxichen/wip-16215-jewel

jewel: client: crash in unmount when fuse_use_invalidate_cb is enabled

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10108 from xiaoxichen/wip-16320-jewel

jewel: fuse mounted file systems fails SAMBA CTDB ping_pong rw test with v9.0.2

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10199 from SUSE/wip-16625-jewel

jewel: Failing file operations on kernel based cephfs mount point leaves unaccessible file behind on hammer 0.94.7

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10499 from Abhishekvrshny/wip-16299-jewel

jewel: mds: fix SnapRealm::have_past_parents_open()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10500 from Abhishekvrshny/wip-16620-jewel

jewel: Fix shutting down mds timed-out due to deadlock

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10502 from Abhishekvrshny/wip-16797-jewel

jewel: MDS Deadlock on shutdown active rank while busy with metadata IO

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10104 from xiaoxichen/wip-16560-jewel

jewel: mds: enforce a dirfrag limit on entries

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #10074 from ceph/jewel-16002

jewel: ObjectCacher split BufferHead read fix

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

build/ops: bump rocksdb submodule

Fixes a FTBFS on ppc64le

Fixes: http://tracker.ceph.com/issues/17092
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10537 from theanalyst/wip-16778-jewel

jewel: rgw multisite: preserve zone's extra pool

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10655 from dachary/wip-16163-jewel

jewel: rgw: can set negative max_buckets on RGWUserInfo
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10580 from dreamhost/wip-16928-jewel

jewel: rgw: Fix civetweb IPv6
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10216 from SUSE/wip-16637-jewel

jewel: add socket backlog setting for via ceph.conf
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10167 from jmunhoz/aws4-streaming-backport-jewel

jewel: rgw: aws4: add STREAMING-AWS4-HMAC-SHA256-PAYLOAD support
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9544 from Abhishekvrshny/wip-16085-jewel

jewel: A query on a static large object fails with 404 error
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10525 from Abhishekvrshny/wip-16732-jewel

jewel: Bucket index shards orphaned after bucket delete
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10188 from dreamhost/wip-16618-jewel

jewel: rgw: fix multi-delete query param parsing.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9266 from Abhishekvrshny/wip-15964-jewel

jewel: rgw: realm pull fails when using apache frontend
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10710 from rzarzynski/wip-16393-jewel

jewel: rgw: improve support for Swift's object versioning.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

osd: set objecter.osdmap when starts

if any OSD is full or the pause flag is set in cluster, objecter will
always try to subscribe the next osdmap using the mon client shared with
OSD. but if the OSD's osdmap is very far behind on osd maps due to some
reasons, `MonClient::sub_want_increment()` will reject the subscribe
request from OSD, because it's asking for older maps while the mon client
is about to send the request from objecter for the *next* osdmap.

so we need to update objecter with the latest local osdmap, so it is
able to skip the new osdmaps if OSD needs to catch up with the cluster
first.

Fixes: http://tracker.ceph.com/issues/17023
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1515e08e7d425f01d2e9d02c34b2ae40cdf2611a)

mon/MonClient: should update sub_sent with sub_new

std::map::insert() does not overwrite existing items with the same key.
so we need to do this in a different way. and we will check sub_sent
aftewards, so we need to keep an updated sub_sent around.

Fixes: http://tracker.ceph.com/issues/17023
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 550814900ebca9dd89e088b9abe24f66eb5e3e6d)

Merge pull request #10797 from trociny/wip-17080-jewel

jewel: the option 'rbd_cache_writethrough_until_flush=true' dosn't work

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

librbd: cache was not switching to writeback after first flush

Fixes: http://tracker.ceph.com/issues/16654
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2d9840af39555ce00246b50e4a5c186798bd88ff)

test: unit test cases for disabling librbd journal by policy

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4a256f5044ac54a8a0a9598ee7fd151b8ea08f84)

Conflicts:
src/test/librbd/CMakeLists.txt (test_ConsistencyGroups.cc)

librbd: utilize factory methods to create AioObjectRequest objects

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2e5076eb19172919deeb0f4d11f2b2b6cc03fab3)

librbd: convert AioObjectRequest/AioObjectRead classes to templates

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 840a473e7fb94124ca7c571fc3dd2c787e0d265a)

librbd: move read callback helpers to narrowest scope

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 65b336f68596a430629692a8682c5cfe883c31f1)

librbd: convert AioImageRequest derived classes to templates

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a945c2c235d9259b1d4a8a579d7e6efc2ed35916)

librbd: removed namespace indentation from legacy classes

Better follows the Ceph C++ style guide

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f1e391982b43ddfb363ff913260460368a6d5834)

librbd: do not record journal events if append is disabled by policy

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 405142c615352613e8bacee46e92484eb0c08f26)

librbd: remove unused journal replay canceled callback

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f453554edcb16ec72714b2456c08bab9e339b1eb)

librbd: optionally support disabling journal appends via policy

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f7eeacd02f8f80c5d1d5d9cfed35bbd23f64b018)

librbd: optimize header file dependency & modify related file.

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
(cherry picked from commit 2185a6275c89d9462611e965da407ea0d504987d)

librbd: optimize operation header file dependency

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
(cherry picked from commit 88afa408683238b803606f3d63de326ead8cefee)

librbd: optimize journal header file dependency

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
(cherry picked from commit 59397e27a04fb0f2b990a86dcaf3c418457d1b72)

librbd: optimize image_watcher header file dependency

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
(cherry picked from commit aee1559f312e9493d6580e5ffcb606fe21ee3c59)

librbd: optimize image header file dependency

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
(cherry picked from commit 461958ebc7e8ccef76b3f08a979482762267c10b)

librbd: optimize exclusive_lock  header file dependency

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
(cherry picked from commit 01282eb5cdeda7bbbb77438c7d1953d023ea54b6)

librbd: optimize object-map header file dependency

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
(cherry picked from commit 1a2276927d5956918f7a4830b4a44048ac090328)

test: fix librbd tests for rbd_skip_partial_discard

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 9717417b7b8aa1b24a82a0bfddbfc23748188641)

Conflicts:
src/rocksdb this was a mistake in the original commit

librbd: discard hangs when 'rbd_skip_partial_discard' is enabled

Fixes: http://tracker.ceph.com/issues/16386
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit dc41731fbfd73d9fbb63d3ff360d4c5dd62deaf1)

librbd: object_may_exist always return true when you write an empty object

if you write an empty object, object map is updated firstly

Signed-off-by: xinxin shu <shuxinxin@chinac.com>
(cherry picked from commit a54073808146d205d54d6a932a6e0fd750f1dc38)

librbd: ensure that AIO ops are started prior to request

Fixes: http://tracker.ceph.com/issues/16708
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6064f2346de0a8bf2878bf5bfac9a992cda7c4ca)

librbd: helper method for creating and starting AioCompletions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3df7213c0a9f0186e3a37c9e4a10f1c8bc84446e)

Merge pull request #10649 from dachary/wip-16867-jewel

jewel: mkfs.xfs slow performance with discards and object map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: potential assertion failure during error-induced shutdown

Fixes: http://tracker.ceph.com/issues/16956
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6a465d9dad417e8b52909c5478f7e3e433748948)

rbd-mirror: potential race condition during failure shutdown

Fixes: http://tracker.ceph.com/issues/16980
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 74ec7c91f17630c77647cfc9813090d688b3410d)

rbd-mirror: replaying state should include flush action

Fixes: http://tracker.ceph.com/issues/16970
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7246f1b771b4d1f336ac11b9e881c9eb32bcd4e1)

doc: added rbdmap man page to RBD restructured index

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4e05cbf7e6eda797a9b08e0081aead19523d5762)

rbdmap: specify bash shell interpreter

Fixes: http://tracker.ceph.com/issues/16608
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 51237c33d1ec4034c5c07f5d63d63838a76bce88)

Merge pull request #10684 from dillaman/wip-16904-jewel

jewel: rbd-mirror: reduce memory footprint during journal replay

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: remove ceph_test_rbd_mirror_image_replay test case

Fixes: http://tracker.ceph.com/issues/16539
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 06a333b116383de4d39c9e098e6e5fa195ceb370)