MDSMonitor.cc: fix mdsmap.<namespace> subscriptions

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9282 from dillaman/wip-15993

rbd-mirror: image live-replay might become stuck

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9285 from mikulely/rgw-clean-multipart-macro

rgw: cleanup RGW_MIN_MULTIPART_SIZE

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8672 from zaitcev/wip-5073-multen-2

rgw: Add access to multi-tenancy test

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8533 from theanalyst/rgw/rest-user-get

rgw_rest_admin: return -EINVAL for improper user info requests
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8186 from rzarzynski/wip-rgw-bulkdel-cosm

rgw: optimize BulkDelete authorization and response printing
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8093 from zaitcev/swift-root-prefix

rgw: Allow to serve Swift off the URL root
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8053 from rzarzynski/wip-rgw-aws4-memalloc

rgw: improve memory management in AWS4 auth
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

test: use randomized write sizes for rbd-mirror stress test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: eliminate watch delay for object refetches

The randomized write sizes of the modified rbd-mirror stress
test results in a lot of journal object with few entries.
Immediately fetch objects when performing a refetch check prior
to closing an empty object.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: keep active tag to assist with pruning watched objects

It's possible that there might be additional entries to prune in
objects that haven't been prefetched yet. Keep the active tag
to allow these entries to be pruned after they have been loaded.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: update commit entry object number upon overflow

Otherwise the recorded object positions might point to an older
object that doesn't contain the actual entry.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: cleanup watch refetch flag handling

Clear the refetch required flag while scheduling the watch
and remove the stale object after the watch completes if still
empty. Previously, it was possible for the flag to become
out-of-sync with whether or not it was actually refreshed
and pruned.

Fixes: http://tracker.ceph.com/issues/15993
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9279 from majianpeng/bluestore-uniquelock

os/bluestore/BlueStore: Fix wrong usage of std::unique_lock for OpSequencer

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8741 from rzarzynski/wip-rgw-keystonev3-doc

rgw, doc: improve documentation of Keystone v3 config options

os/bluestore/BlueStore: Fix wrong usage of std::unique_lock for OpSequenecer

When set bluestore_sync_wal_apply = false, the osd must be crashed.
I add some debug message in WALWQ::_dequeue():
// preserve wal ordering for this sequencer by taking the lock
/ while still holding the queue lock
-      i->osr->wal_apply_lock.lock();
+      try {
+       i->osr->wal_apply_lock.lock();
+      } catch (const system_error& e) {
+       if (e.code() == std::errc::resource_deadlock_would_occur) {
+         assert(0 == "would deadlock");
+       } else
+         assert(0 == "other system_error");
+      }
       return i;
     }

2016-05-24 00:02:29.107165 7f0f32ea6700 -1 os/bluestore/BlueStore.h: In
function 'virtual BlueStore::TransContext* BlueStore::WALWQ::_dequeue()'
thread 7f0f32ea6700 time 2016-05-24 00:02:29.103360
os/bluestore/BlueStore.h: 489: FAILED assert(0 == "would deadlock")

 ceph version 10.2.0-1232-g21c7599 (21c759936ddcbfec50cf177a577206a2c443c212)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x80) [0x560196215ac0]
 2: (ThreadPool::WorkQueue<BlueStore::TransContext>::_void_dequeue()+0x234)[0x560195e618b4]
 3: (ThreadPool::worker(ThreadPool::WorkThread*)+0x654) [0x560196207324]
 4: (ThreadPool::WorkThread::entry()+0x10) [0x560196208d70]
 5: (()+0x760a) [0x7f0f411d260a]
 6: (clone()+0x6d) [0x7f0f3f17a59d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is
needed to interpret this.

This because the wron usage of std::unique_lock.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #9218 from ceph/wip-asok-denials

rpm: Packaging fixes

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cuthler <ncutler@suse.com>

Merge pull request #8906 from david-z/wip-fix-kcephfs-symlink

mds: wrongly treat symlink inode as normal file/dir when symlink inode is stale on kcephfs

rgw: cleanup RGW_MIN_MULTIPART_SIZE

As the commit 11cf9bbb298768b5307616db5977cc4985072a6e
make rgw_multipart_min_part_size as a config option,
we did not use RGW_MIN_MULTIPART_SIZE anymore.

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Merge pull request #9281 from songbaisen/song29

msg: fix the wrong para in cpp_strerror

Reviewed-by: Kefu Chai <kefu@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

msg: fix the wrong para in cpp_strerror

Signed-off-by:song baisen <song.baisen@zte.com.cn>

Merge branch 'xio-rename_XioRsp' of https://github.com/Adirl/ceph into Adirl-xio-rename_XioRsp

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #9235 from athanatos/wip-throttle

BackoffThrottle: fix for freebsd and avoid spinning when full

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #9276 from linuxbox2/xio-enqueue-2

xio: follow 03d33e59 (enqueue_for_send()->enqueue())

xio: follow 03d33e59 (enqueue_for_send()->enqueue())

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #9275 from ktdreyer/wip-doc-cephfs-jcsp

doc: John Spray is now CephFS lead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc: John Spray is now CephFS lead

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #8856 from avnerbh/fixes_4_keepalive_to_xio

xio: provide keepalive(s) and their ack(s)

Merge pull request #9171 from weiqiaomiao/wqm-wip-rgw-mem

rgw: fix some memory leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8873 from roidayan/xio-use_xio_session_event_str

xio: use accelio api to get event str

(this is a decision for the Accelio API, we need to follow it)

Merge pull request #9205 from Adirl/fix_compiltaion_entity_addr_t3

Fix compiltaion entity addr t

Merge pull request #9229 from oritwas/wip-15937

rgw: add missing metadata_heap pool to old zones

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rpm: Restore /var/run/ceph context manually

The fixfiles command won't fix label for /var/run/ceph (/run is probably
excluded from relabel), we need to restore the context manually.

Signed-off-by: Boris Ranto <branto@redhat.com>

rpm: Mute semodule in postun script

Currently, we don't mute the semodule output in postun script. This
results in the following message when removing ceph-selinux package:

libsemanage.semanage_direct_remove_key: Removing last ceph module (no
other ceph module exists at another priority).

The fix is to simply mute the output of the command.

Signed-off-by: Boris Ranto <branto@redhat.com>

Revert "rpm: put /sbin/ldconfig into -p"

This reverts commit a53d932fc4a77a1fac33c4f650f2bf629a2f1234.

If the interpret is passed to postun script, everything until the next
rpm section gets interpreted as the input for the interpret (in this
case ldconfig). This results in warnings like

/sbin/ldconfig: relative path `0' used to build cache

when removing the packages that pass the interpret because we use
several '#' characters to separate the subpackage sections and these get
passed to the ldconfig.

Using the default interpret (bash) fixes the warning. I'm also doing the
same to %post script for consistency.

Signed-off-by: Boris Ranto <branto@redhat.com>

BackoffThrottle: wait() if past max

Otherwise, we risk spinning on wait_for() with a small
delay.

Fixes: http://tracker.ceph.com/issues/15953
Signed-off-by: Samuel Just <sjust@redhat.com>

BackoffThrottle: use wait_for instead of wait_until

On some platforms, wait_until won't surrender the lock
with a negative argument.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #9249 from tchaikov/wip-cmake-ocf

cmake: add ocf support

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9227 from ceph/rpm-autosetup-kdreyer

ceph.spec.in: use %autosetup macro

Reviewed-by: Nathan Cuthler <ncutler@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #9147 from flaf/master

doc: update required packages list to build doc

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9247 from cxwshawn/wip-tc4-fix

global: optimize header file dependency.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9251 from cxwshawn/wip-tc5-fix

client: optimize header file dependency.

Reviewed-by: Sage Weil <sage@redhat.com>

osd/HitSet.h: add 'override' to virtual functionsReviewed-by: Kefu Chai <kchai@redhat.com>

osd/HitSet.h: add 'override' to virtual functions

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9255 from yuyuyu101/wip-test-msgr-fix

test_msgr: add verbose info for pending connections

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8929 from ajarr/wip-15614

ceph_volume_client: allow read-only authorization for volumes

Reviewed-by: John Spray <john.spray@redhat.com>

ceph_volume_client: allow read-only authorization for volumes

Allow clients to be restricted to read-only mount of the volume
by restricting their ceph auth ID's MDS and OSD caps to read-only.

Fixes: http://tracker.ceph.com/issues/15614
Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge pull request #8473 from rjfd/wip-14421

rbd-mirror: Delete local image mirror when remote image mirroring is disabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: test: added image-deleter test case when mirroring is disabled

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Unregister clients from non-primary images journal

A non-primary image may have registered clients on its journal
(for instance a primary image that was later demoted). We must
unregister the clients when disabling image mirroring with the
force option.

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: test: Added unit test for testing image-deleter thread

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: test: Fixed test_PoolWatcher to support the new changes in PoolWatcher class

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Replayer: bootstrap existing mirrored images

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: replayer: Added image-deleter thread to replayer

Fixes: http://tracker.ceph.com/issues/14421
Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Added image-deleter thread to rbd-mirror

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Added implementation of image-deleter thread

image-deleter thread will handle the deletion of local images that have
been disabled for mirroring by the primary site.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #9179 from vshankar/rbd-min-commit-position

cls_journal: initialize empty commit position upon client register

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7836 from yuyuyu101/wip-13938

TaskFinisher: use singleton SafeTimer and Finisher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test_msgr: add verbose info for pending connections

Signed-off-by: Haomai Wang <haomai@xsky.com>

client: optimize header file dependency.

same work as https://github.com/ceph/ceph/pull/9247

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

osd/HitSet.h: add 'override' to virtual functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

osdc/Objecter: move C_TwoContexts class to cc fileReviewed-by: Sage Weil <sage@redhat.com>

osdc/Objecter: move C_TwoContexts class to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Objecter: move C_TwoContexts class to cc file

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

cmake: add ocf support

a new option is added: WITH_OCF. it is OFF by default.

Signed-off-by: Kefu Chai <kchai@redhat.com>

global: optimize header file dependency.

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

Merge pull request #9241 from ceph/wip-fix-q-tests

common/: fix tests for prioritized_queue and wpq

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9102 from yehudasa/wip-rgw-leak-4

rgw: don't unregister request if request is not connected to manager

Reviewed-by: Casey Bodley <cbodley@redhat.com>

common/: fix tests for prioritized_queue and wpq

Introduced: 27b6ec40b93c05fbaa228783e11e2fac8f1a1ee1
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #9216 from dzafman/wip-test-fix

test: Disable tests due to recovery race

test: Disable tests due to recovery race

Disable:
osd-scrub-repair.sh
test-erasure-eio.sh
test-erasure-code.sh

These test are failing due to race conditions that is exposing
bug http://tracker.ceph.com/issues/13937.

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #9211 from dillaman/wip-15938

librbd: write-after-write might result in an inconsistent replicated image

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9231 from athanatos/wip-recovery-wq

osd: Put background recovery tasks in the main OpWQ

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6208 from liewegas/wip-alloc-hints

librados,osd: rados hints for future access pattern

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7070 from majianpeng/headerbeatmap-check

common/HeartbeatMap: print stack info of unhealth thread.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7843 from branch-predictor/bp-smart-async-wp

msg/async: Implement smarter worker thread selection

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8141 from songbaisen/song14

crush: fix the problem can not find the define item below the bucket

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8840 from tchaikov/wip-opt-u32

common/config: cast OPT_U32 options using uint32_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9072 from liewegas/wip-15840

osd: fix crush location update on startup

Merge pull request #9116 from ddiss/librados_sparse_read_use_snapid

librados: use caller provided snapid for aio_sparse_read()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #9002 from rzarzynski/wip-rgw-swift-acct_cont_md-fix

rgw: fix updating account/container metadata of Swift API.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: add missing metadata_heap pool to old zones

Fixes: http://tracker.ceph.com/issues/15937
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

cls_journal: Select min commit position for new clients

When a new client registers, its commit position is initialized
to the least commit position of all other clients.

Fixes: http://tracker.ceph.com/issues/15757
Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #9225 from dachary/wip-run-make-check

tests: rm -fr /tmp/*virtualenv*

Reviewed-by: Erwan Velu <erwan@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph.spec.in: use %autosetup macro

New in rpm 4.11,
http://www.rpm.org/wiki/PackagerDocs/Autosetup

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #9165 from mikulely/fix-rgw-multisite-test

rgw: misc fix for test_multi.py

Reviewed-by: Casey Bodley <cbodley@redhat.com>

tests: rm -fr /tmp/*virtualenv*

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #9139 from majianpeng/bluestore-statfs

os/bluestore/BlueStore: free space of bluestore should reduce spaces …

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9161 from cxwshawn/wip-tc1-fix

osd: optimize header file dependency to decreace compile time when modify any file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9193 from cxwshawn/wip-tc2-fix

mon: optimize header file dependency.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9068 from trociny/wip-pybind-rbd-mirror

librbd: initial python APIs to support mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9088 from trociny/wip-15500

qa: dynamic_features.sh: return error only if it failed on alive QEMU

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9140 from zhouyuan/fix_disable_journaling

librbd: close journal before removing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9177 from trociny/wip-15916

rbd-mirror: it might got stuck in calculating "behind master" position

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

journal: replay position might change after pruning stale tags

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: delay commit of overwritten journal event

With the cache enabled and write-after-write IOs to the same
object extents, it was possible for the overwritten journal event
to be committed before the overwriter journal event was written
to disk.  If a client crash occurs before the event is written,
the image will be inconsistent on replay.

Fixes: http://tracker.ceph.com/issues/15938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9196 from tchaikov/wip-cmake-vstart

cmake: add cython_rados to vstart target

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8164 from rzarzynski/wip-rgw-bulkdel-slashfix

rgw: handle initial slashes properly in BulkDelete of Swift API.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: handle initial slashes properly in BulkDelete of Swift API.

Fixes: http://tracker.ceph.com/issues/15948
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #9220 from cxwshawn/wip-tc3-fix

auth: optimize header file dependency.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9212 from liewegas/wip-bit-alloc-test

tests: disable unittest_bit_alloc for now

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #9219 from renhwztetecs/renhw-wip-logmonitor-cleanup

mon/log: clean unused function and comments

Reviewed-by: Sage Weil <sage@redhat.com>