Merge pull request #13143 from linuxbox2/jewel-mdw-rgw-lf

jewel: rgw: radosgw/swift: clean up flush / newline behavior.

Reviewed-by: Nathan Cutler <ncutler@suse.com>

radosgw/swift: clean up flush / newline behavior.

The current code emits a newline after swift errors, but fails
to account for it when it calculates 'content-length'.  This results in
some clients (go github.com/ncw/swift) producing complaints about the
unsolicited newline such as this,
Unsolicited response received on idle HTTP channel starting with "\n"; err=<nil>

This logic eliminates the newline on flush.  This makes the content length
calculation correct and eliminates the stray newline.

There was already existing separator logic in the rgw plain formatter
that can emit a newline at the correct point.  It had been checking
"len" to decide if previous data had been emitted, but that's reset to 0
by flush().  So, this logic adds a new per-instance variable to separately
track state that it emitted a previous item (and should emit a newline).

Fixes: http://tracker.ceph.com/issues/18473
Signed-off-by: Marcus Watts <mwatts@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 5f229d6a33eae4906f22cdb90941835e47ee9f02)

Merge pull request #11865 from dachary/wip-17710-jewel

jewel: multisite: race between ReadSyncStatus and InitSyncStatus leads to EIO errors

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #12033 from dachary/wip-17926-jewel

jewel: ceph-disk --dmcrypt create must not require admin key

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11968 from ddiss/jewel_next_flush_evict_snaps

jewel: tools: snapshotted RBD extent objects can't be manually evicted from a cache tier

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12151 from dachary/wip-18011-jewel

jewel: test fails due to The UNIX domain socket path

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12296 from SUSE/wip-18133-jewel

jewel: build/ops: fix undefined crypto references with --with-xio

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11997 from Abhishekvrshny/wip-17876-jewel

jewel: osd: update_log_missing does not order correctly with osd_ops

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11944 from SUSE/wip-17866-jewel

jewel: osd: Add config option to disable new scrubs during recovery

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11672 from linuxbox2/jewel-17663

jewel: rgw_rest_s3:  apply missed base64 try-catch

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #11953 from SUSE/wip-17885-jewel

jewel: test: temporarily disable fork()'ing tests

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11884 from SUSE/wip-17754-jewel

jewel: ceph-create-keys loops forever

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11529 from SUSE/wip-17600-jewel

jewel: common: Improve linux dcache hash algorithm

Reviewed-by: Kefu Chai <kchai@redhat.com>

build/ops: fix undefined crypto references with --with-xio

Only with --with-xio, RPM build fails due to undefined references to various
symbols starting with "PK11_" in ./.libs/libcommon.a(Crypto.o) in several
of the unit tests.

Fixes: http://tracker.ceph.com/issues/18133
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11999 from Abhishekvrshny/wip-17904-jewel

jewel: Error EINVAL: removing mon.a at 172.21.15.16:6789/0, there will be 1 monitors

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12174 from SUSE/wip-18025-jewel

rgw: don't store empty chains in gc

Merge pull request #11758 from Abhishekvrshny/wip-17784-jewel

jewel: osd crashes when radosgw-admin bi list --max-entries=1 command runing

rgw: don't store empty chains in gc

Fixes: http://tracker.ceph.com/issues/17897
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 7c8f1b9edc921711fa30345bbecea2dcd5de1229)

Merge pull request #11855 from dachary/wip-17472-jewel

jewel: rpm: /etc/ceph/rbdmap is packaged with executable access rights

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11660 from Abhishekvrshny/wip-17601-jewel

jewel: mon: health does not report pgs stuck in more than one state

Reviewed-by: Sage Weil <sage@redhat.com>

tests: save 9 characters for asok paths

For vstart.sh powered tests, save 9 characters in the path name
by replacing testdir/test- with td/t-

60 characters imposed by jenkins
9 characters for src/test
5 characters for td/t-

33 left (instead of 24) for the test to create asok such as out/client.admin.25327.asok

Moving these files outside of the build directory is a bad idea because
tests should only create/use files within the builddir and not write
outside of this directory. Doing so would make things more complicated
for cleanup in case the test fail and create other problems as a
consequence (filling out disk space, conflicting directories between
runs etc.).

For ceph-helpers.sh tests replace testdir with td, saving 5 characters.
This is not strictly necessary but keeps the directory names consistent:
if the developer wants to get rid of all the test leftovers, it is
enough to remove the a single directory: td.

Fixes: http://tracker.ceph.com/issues/16014
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit cd72ff9f746fe13a245324dce2aa457d3c876331)

Conflicts:
.gitignore : test directories are not ignored in jewel
src/test/vstart_wrapper.sh : trivial context resolution

mon,ceph-disk: add lockbox permissions to bootstrap-osd

ceph-disk --dmcrypt needs to put a config-key and authorize
the OSD to get it back. The corresponding permissions are
added to the bootstrap-osd profile in the monitor.

When preparing the OSD lockbox, use the bootstrap-osd profile instead of
implicitly requiring admin permissions to perform the initial config-key
and auth get-or-create operations.

Fixes: http://tracker.ceph.com/issues/17849
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit c1c6224bf8df788cd34e9ba7294a6515a9ab97e3)

mon: MonmapMonitor: drop unnecessary 'goto' statements

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 20dcb597e35e6961db81831facefbe22cecddec3)

mon: MonmapMonitor: return success when monitor will be removed

Fixes: http://tracker.ceph.com/issues/17725
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit c9d46cfbf2512bc3495c6901de2b8f711bef9bae)

ReplicatedPG::do_update_log_missing: take the pg lock in the callback

Fixes: http://tracker.ceph.com/issues/17789
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 3ba2e2e21a187474d556722d1ef30c79628f1c50)

Merge pull request #11976 from dachary/wip-17903-jewel

ceph-disk: fix flake8 errors

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

ceph-disk: fix flake8 errors

flake8 3.1.1 surfaces the following issues:

ceph_disk/main.py:173:1: E305 expected 2 blank lines after class or
function definition, found 1
ceph_disk/main.py:5011:1: E305 expected 2 blank lines after class or
function definition, found 1

Fixes: http://tracker.ceph.com/issues/17898
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 9f3403b1e1db88ff966860da13d18f89f3a8b09d)

osd: Add config option to disable new scrubs during recovery

The osd_scrub_during_recovery config option allows for configuring
if the OSD will schedule a new scrub while recovery is active.

When set to false no new scrubs will be initiated by the OSD while
there are recovery threads active on that OSD.

Signed-off-by: Wido den Hollander <wido@42on.com>
(cherry picked from commit 33f1f6077804dbcbb82a41ef7f6803b5f3365831)

Conflicts:
src/osd/OSD.h (between jewel and kraken, pg recovery methods were moved
        from the OSD class to OSDService)

ceph-create-keys: wait 10 minutes to get or create the bootstrap key, not forever

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bea802bd13df883d06b7b67a83ac6540e44ab6c1)

ceph-create-keys: wait 10 minutes to get or create a key, not forever

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8f84681a4d97a7ea6bc04e759eccfc40204a2fdb)

ceph-create-keys: wait for quorum for ten minutes, not forever

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 32cedd2c18e48d5f9a1547e5fd69c24c7eac1638)

Merge pull request #11916 from trociny/wip-17767-jewel

jewel: librbd: restore journal access when force disabling mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits: update test_cache_pool.sh

Signed-off-by: Mingxin Liu <mingxin@xsky.com>
(cherry picked from commit 383177be07700f9fee067ecb7a27155c322f98dd)

tools/rados: add --with-clones option to include clones for cache-flush/cache-evict

Signed-off-by: Mingxin Liu <mingxin@xsky.com>
(cherry picked from commit 3dd9fa2fb39c76ba3977d7bc09cd28bb3465d56c)

tools/rados: default to include clone objects when excuting "cache-flush-evict-all"

Signed-off-by: Mingxin Liu <mingxin@xsky.com>
(cherry picked from commit b1cf2d9276834d04172d45d5f4dd48ca3e99d338)

Merge pull request #11875 from dachary/wip-17842-jewel

jewel: Remove the runtime dependency on lsb_release

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11737 from Abhishekvrshny/wip-17765-jewel

jewel: collection_list shadow return value #

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11873 from dachary/wip-17841-jewel

jewel: mds fails to respawn if executable has changed

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11862 from dachary/wip-17582-jewel

jewel: monitor assertion failure when deactivating mds in (invalid) fscid 0

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11861 from dachary/wip-17615-jewel

jewel: mds: false failing to respond to cache pressure warning

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11860 from dachary/wip-17617-jewel

jewel: [cephfs] fuse client crash when adding a new osd

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11858 from dachary/wip-17697-jewel

jewel: MDS long-time blocked ops. ceph-fuse locks up with getattr of file

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11857 from dachary/wip-17706-jewel

jewel: multimds: mds entering up:replay and processing down mds aborts

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11856 from dachary/wip-17720-jewel

jewel: MDS: false failing to respond to cache pressure warning

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11413 from dachary/wip-17478-jewel

jewel: MDS goes damaged on blacklist (failed to read JournalPointer: -108 ((108) Cannot send after transport endpoint shutdown)

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11743 from tchaikov/wip-17558-jewel

jewel: mon: send updated monmap to its subscribers

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11736 from Abhishekvrshny/wip-17732-jewel

jewel: ceph daemons DUMPABLE flag is cleared by setuid preventing coredumps

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11735 from Abhishekvrshny/wip-17721-jewel

jewel: osd_max_backfills default has changed, documentation should reflect that.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11606 from dzafman/wip-17666

jewel: osd: Remove extra call to reg_next_scrub() during splits

Reviewed-by: Loic Dachary <ldachary@redhat.com>

test: temporarily disable fork()'ing tests

Fixes: http://tracker.ceph.com/issues/16556
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 1f3d96c501ad666b9878269de25bb0172516fc53)

Conflicts:
src/test/libcephfs/recordlock.cc (file does not exist in jewel)

Merge pull request #10865 from dachary/wip-17057-jewel

jewel: The request lock RPC message might be incorrectly ignored

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

librbd: restore journal access when force disabling mirroring

If mirroring is force disabled on a demoted image, the journal was
being left in an inconsistent ownership state.

This is a direct commit to jewel as the fix in the master was
against the newly added async version of mirror disable, which is
not going to be merged to jewel.

Fixes: http://tracker.ceph.com/issues/17767
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11854 from dachary/wip-17712-jewel

jewel: 'rbd du' of missing image does not return error

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11870 from dachary/wip-17844-jewel

jewel: rbd-nbd: disallow mapping images >2TB in size

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11869 from dachary/wip-17845-jewel

jewel: rbd-mirror: snap protect of non-layered image results in split-brain

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11574 from gaurav36/jewel-backport-period-prepare-command

jewel: rgw multisite: obsolete 'radosgw-admin period prepare' command

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11864 from dachary/wip-17733-jewel

jewel: multisite: after finishing full sync on a bucket, incremental sync starts over from the beginning

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11675 from asheplyakov/jewel-17674

jewel: rgw: fix put_acls for objects starting and ending with underscore

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11852 from dachary/wip-17766-jewel

jewel: Exclusive lock improperly initialized on read-only image when using snap_set API

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11853 from dachary/wip-17763-jewel

jewel: TestLibRBD.DiscardAfterWrite doesn't handle rbd_skip_partial_discard = true

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11760 from Abhishekvrshny/wip-17769-jewel

jewel: disable virtual hosting of buckets when no hostnames are configured
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11867 from dachary/wip-17675-jewel

jewel: RGWRados::get_system_obj() sends unnecessary stat request before read
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11863 from dachary/wip-17735-jewel

jewel: RGW will not list Argonaut-era bucket via HTTP (but radosgw-admin works)
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11627 from cbodley/wip-17343

jewel: rgw: work around curl_multi_wait bug with non-blocking reads

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: add missing mutex header for std::once_flag

this fix was added directly to the jewel branch rather than backporting
from master, because the code on master compiles without this specific
include - it's likely included by another header, but backporting would
involve pulling in unrelated changes

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11822 from SUSE/wip-17816-jewel

jewel: Missing comma in ceph-create-keys causes concatenation of arguments

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: remove suggestion to upgrade libcurl

Reported-by: Ken Dreyer <kdreyer@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit dcff120edaabb6fb0ebba73a716e4d42fa10dcd8)

rgw/rgw_http_client: add compat.h include for TEMP_FAILURE_RETRY

Signed-off-by: John Coyle <dx9err@gmail.com>
(cherry picked from commit 00f4554aa5511d86ea45aec90c16617c419ca7c3)

rgw: detect and work around a curl_multi_wait bug

Fixes: http://tracker.ceph.com/issues/15915
Fixes: http://tracker.ceph.com/issues/16695
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0359be62b79857357f6d0d4fb81c6bb4e0c381a6)

rgw: use non-blocking reads for clear_signal

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1101fcc750cdd8702b2ca570da2cecb47d0ddeae)

rgw: factored clear_signal out of do_curl_wait

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f2f5cdf3aadfa265e2e834a9354c2aacf274212d)

rgw: do_curl_wait uses ldout

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 208098441366cc7446138cc15afd4a99cd30a2fc)

rgw: add pipe fd to set for select() in do_curl_wait()

when HAVE_CURL_MULTI_WAIT is 0, the pipe fd is never added to the
readfds for select(), so FD_ISSET() is always false. this prevents us
from ever trying to read from the fd, and the pipe's buffer eventually
fills up and deadlocks callers of RGWHTTPManager::signal_thread() when
they try to write to the pipe

Fixes: http://tracker.ceph.com/issues/16368
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 75897f82abde818fde6e1625c6977dcdd639a1f0)

rgw: clean up thread_pipe in RGWHTTPManager::stop

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c93959b182c4394517a9ec9e29e3b14ee8030fc6)

rgw: create thread_pipe before RGWHTTPManager::ReqsThread

closes a potential race between pipe creation and
RGWHTTPManager::reqs_thread_entry()'s access of thread_pipe

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9161e9fdbfb2aa529a87197ef1a34f03917b80d9)

common: Remove the runtime dependency on lsb_release

With modern releases we should be able to make do with the call to
os_release_parse only which uses /etc/os-release which should be available on
most (all?) releases we currently support. this then allows us to remove the
runtime dependency which pulls in several other packages and would be nice to
avoid.

Fixes: http://tracker.ceph.com/issues/17425
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 8b1a57fd34cdc28ae688dcdf73fe70443e0463ac)

Conflicts:
debian/control: trivial context resolution

common/util: add support for distro info from /etc/os-release file

test/cephtool-test-mon.sh tests osd metadata for distro info. This fails on Alpine because lsb_release is not supported.

Updated collect_sys_info() to detect distro info in /etc/os-release as a fallback to lsb_release.

Added unit test for distro info detection.

Signed-off-by: John Coyle <dx9err@gmail.com>
(cherry picked from commit a9487e2ad09546f8cd06a371cfed9e4ba992c679)

mds: respawn using /proc/self/exe

This allows the MDS to respawn using the same executable file even if it
has since been deleted (on Linux). Otherwise, the execv fails because
the readlink returns "/path/to/deleted (deleted)". (There is no path to
the old executable.)

Fixes: http://tracker.ceph.com/issues/17531
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 66a122025f6cf023cf7b2f3d8fbe4964fb7568a7)

rbd-nbd: disallow mapping images >2TB in size

Fixes: http://tracker.ceph.com/issues/17219
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 2012b4dfc65414751e3da8386a3a14da8b7c1249)

rbd-mirror: snap protect of non-layered image results in split-brain

Fixes: http://tracker.ceph.com/issues/16962
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 8e1cc88e068fe57e9bbfa1ebbc6bbf89fb62aaac)

rgw: get_system_obj does not use result of get_system_obj_state

get_system_obj() calls get_system_obj_state() to send a [getxattrs,stat]
request and fill out an RGWObjState - but the results are not used

Fixes: http://tracker.ceph.com/issues/17580
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 973d0076958ec9416cbe9ebc1cec03ee147ca4f9)

rgw: RGWSimpleRadosReadCR tolerates empty reads

Fixes: http://tracker.ceph.com/issues/17568
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c58ac1b0cc33505692f6244abc34ecea05e6d072)

rgw: fix for passing temporary in InitBucketSyncStatus

Fixes: http://tracker.ceph.com/issues/17661
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 32906353c3136bc500ff06a3732a0946591118fa)

rgw multisite: fix the increamtal bucket sync init

in the `RGWBucketShardFullSyncCR::operate`, inc_marker will assigned with remote bilog's max_marker.
but the sync_status's inc_marker cant be assigned.so the next step inc sync will always sync
from null log,which means at beginning log.

Fixes: http://tracker.ceph.com/issues/17624
Signed-off-by: Zengran Zhang <zhangzengran@h3c.com>
(cherry picked from commit 0d928c26408f2fa0b276304c4c00e9c41e0777fa)

rgw: get_zonegroup() uses "default" zonegroup if empty

Fixes: http://tracker.ceph.com/issues/17372
An empty zonegroup should be replaced with the "default" zonegroup.
This is needed when dealing with zonegroup set in old bucket info,
that predated setting the buckets' region.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 3d7169af3f5eeb6bc8a89ee873ec43f57f206f3a)

mds: use parse_filesystem in parse_role

This allows us to reuse code in parse_filesystem and avoid
get_filesystem which may fail if the fscid does not exist. This would
result in the program (mon) aborting due to the uncaught exception.

Fixes: http://tracker.ceph.com/issues/17518
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit edc78e46cee356da1e45247c38b7428dd6c965cb)

mds: group filesystem access methods

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 59de9919a0e8388c1b8e7968187de6b1b6492425)

mds: use reference to avoid copy

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8934e95d75acadc1a82347272ac18175751a92de)

mds: fully encapsulate filesystems map

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit de6c36af022da9599b04b59e5300fbb6a3de938f)

mds: fix false "failing to respond to cache pressure" warning

the false warning happens in following sequence of events
- MDS has cache pressure, sends recall state messages to clients
- Client does not trim as many caps as MDS expected. So MDS
  does not reset session->recalled_at
- MDS no longer has cache pressure, it stop sending recall state
  messages to clients.
- Client does not release its caps. So session->recalled_at in
  MDS keeps unchanged

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 51c926a74e5ef478c11ccbcf11c351aa520dde2a)

Revert "osdc: After write try merge bh."

This reverts commit 1a48a8a2b222e41236341cb1241f0885a1b0b9d8.

Fixes: http://tracker.ceph.com/issues/17270
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 88dbde2fa7ce89ab58724833d436c563ba325682)

osdc/ObjectCacher: wake up dirty stat waiters after removing buffers

Fixes: http://tracker.ceph.com/issues/17275
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit a684dc50873ca968131a68db65f1a1df36767d44)

mds: check if down mds is known

This avoids an assertion failure where an MDS receives an mdsmap that
causes it to enter up:replay and also see another MDS go down.

Fixes: http://tracker.ceph.com/issues/17670
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 99a21133cd3133088e3977d46aab7120524eb350)

mds: fix false "failing to respond to cache pressure" warning

the false warning happens in following sequence of events
- MDS has cache pressure, sends recall state messages to clients
- Client does not trim as many caps as MDS expected. So MDS
  does not reset session->recalled_at
- MDS no longer has cache pressure, it stop sending recall state
  messages to clients.
- Client does not release its caps. So session->recalled_at in
  MDS keeps unchanged

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 51c926a74e5ef478c11ccbcf11c351aa520dde2a)

rpm: fix permissions for /etc/ceph/rbdmap

Prior to this change, the RPM packaging would install /etc/ceph/rbdmap
with exectuable permissions. The execute bit is not necessary and does
not match what the Debian packaging does. Remove the execute bit in this
case.

Fixes: http://tracker.ceph.com/issues/17395
Reported-by: Martin Bukatovic <mbukatov@redhat.com>
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit d4b84a13960f9f46593bf89dc92bfc3e54b4851e)

Conflicts:
ceph.spec.in : trivial context resolution

rbd: return error if we specified a wrong image name for rbd du

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
(cherry picked from commit ce4c801cfc114f718ca51c32b657fec638ca9aaf)

test: skip TestLibRBD.DiscardAfterWrite if skip partial discard enabled

Fixes: http://tracker.ceph.com/issues/17750
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 20406aef3e0bd442d17cbb7acf5db7c42a0d8a11)

librbd: exclusive lock incorrectly initialized when switching to HEAD

Fixes: http://tracker.ceph.com/issues/17618
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 039716db053f91a8ee52d669cbe992d95af6a6c6)

common: Improve linux dcache hash algorithm

In ceph_str_hash_linux(), hash value is defined as unsigned long,
which is 8 bytes on 64-bit platforms when compiled with gcc. But
the return value is truncated to 4 bytes, and there's no need to
preserve 8 bytes intermediate value in the algorithm. The compiler
doesn't figure out this fact and produces redundant code.

After removing the "long" definition, this route runs much faster.
Following result are tested on x86_64 and aarch64 platforms, built
by gcc 5.3.1 with optimization level -O2. Same output is observed
with gcc 4.8.4 and -O3 optimization.

ARM Cortex-A57
+---------------+--------------+---------------+-------------+
| String Length | Time w/ long | Time w/o long | Improvement |
+---------------+--------------+---------------+-------------+
| 32            | 0.088 us     | 0.067 us      | 24%         |
+---------------+--------------+---------------+-------------+
| 4096          | 10.26 us     | 8.20 us       | 20%         |
+---------------+--------------+---------------+-------------+
| 65536         | 164 us       | 131 us        | 20%         |
+---------------+--------------+---------------+-------------+
| 1048576       | 2624 us      | 2099 us       | 20%         |
+---------------+--------------+---------------+-------------+

Intel i7-4790
+---------------+--------------+---------------+-------------+
| String Length | Time w/ long | Time w/o long | Improvement |
+---------------+--------------+---------------+-------------+
| 32            | 0.033 us     | 0.028 us      | 16.3%       |
+---------------+--------------+---------------+-------------+
| 4096          | 3.87 us      | 3.64 us       | 6.2%        |
+---------------+--------------+---------------+-------------+
| 65536         | 61.3 us      | 57.8 us       | 5.7%        |
+---------------+--------------+---------------+-------------+
| 1048576       | 973 us       | 917 us        | 5.8%        |
+---------------+--------------+---------------+-------------+

Signed-off-by: Yibo Cai <yibo.cai@linaro.org>
(cherry picked from commit 0cdee4bf0a55f63321957729c50caf17e7cd3b4a)

mds: handle blacklisting during journal recovery

EBLACKLISTED was being incorrectly handled as an
indication of metadata damage.

Fixes: http://tracker.ceph.com/issues/17236
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 19bb8c0df9b48ebbccfd1913126bb48b6337319e)