Merge pull request #9280 from zhuangzeqiang/wip15522

rbd: check value of config override when setting image-meta

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: check value of config override when setting image-meta

Fixes: http://tracker.ceph.com/issues/15522
Signed-off-by: zhuangzeqiang <zhuang.zeqiang@h3c.com>

Merge pull request #9326 from marcan/rbd-aio-completion

librbd: only mark async completion as done after callback

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

pybind: misc fixesReviewed-by: Josh Durgin <jdurgin@redhat.com>

pybind: misc fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #9322 from trociny/wip-16011

test: failure in journal.sh workunit test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9344 from renhwztetecs/renhw-wip-mdsdaemon

mds: mds-daemon init fixup

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9112 from renhwztetecs/renhw-wip-mdbalancer

mds/mdbalancer: fixup assignment type in the hit_dir

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9335 from ceph/wip-16032-radosgw-flask

packaging: drop python-flask dependency from radosgw

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #9309 from dillaman/wip-15949

journal: support asynchronous shutdown

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9156 from ceph/wip-cmake

cmake: Added #defines to config-h.in.cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: remove verbose output from

Signed-off-by: Ali Maredia <amaredia@redhat.com>

cmake: Added #defines to config-h.in.cmake

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #8941 from xiexingguo/xxg-wip-reweight-filter

mon/OSDMonitor: more fixes for reweight_by_* command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9120 from runsisi/wip-rpg-minor-cleanup

osd/ReplicatedPG: remove unused local variable update

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9126 from xiexingguo/xxg-wip-fix-precisionlost

osd: fix sched_time not actually randomized

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9324 from rzarzynski/wip-16015

rgw: properly handle initial slashes in SLO's segment path.

Merge pull request #9353 from stiopaa1/common_weightedPriorityQueue

common/WeightedPriorityQueue: add override to func

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7733 from xinxinsh/wip-14701

osdmaptool: allow add --pg-num to osdmaptool --test-map-pgs to override the map's pg_num value

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9065 from xiexingguo/xxg-wip-texttable

mon/PGMonitor: use TextTable for pg relevant dump commands.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9097 from jcsp/wip-15866

mon: fix 'mon metadata' for lone monitors, and enable listing all metadata at once

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9198 from xiexingguo/xxg-wip-filestore-trackedkey

os/filestore: add xattr option family to tracked keys

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9204 from xiexingguo/xxg-wip-fix-omapfsid

os/filestore: fix omap fsid related fd leak

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9223 from vumrao/wip-vumrao-15762

osd: add peer_addr in heartbeat_check log message

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9302 from ceph/wip_update_cmake_readme

Improves the README for cmake instructions and slightly improved markdown syntax.

Reviewed-by: Ali Maredia <amaredia@redhat.com>

rgw: properly handle initial slashes in SLO's segment path.

Fixes: http://tracker.ceph.com/issues/16015
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

librbd: only mark async completion as done after callback

This change in behavior was introduced in fde9f78.

Signed-off-by: Hector Martin <marcan@marcan.st>

rbd-mirror: image replay now uses asynchronous journal shutdown

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: integrate with async journaler shutdown API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mds: drop reference if message in laggy queue is unrecognized

drop reference if message in laggy queue is unrecognized

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds: make a quick exit if suicide

make a quick exit if suicide

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds: set stopping to true during suicide()

We are using stopping flag everywhere to indicate that a shutdown
procedure of the mds-daemon is already in progress, but we never
set stopping to true correctly.

The suicide() method is the proper place to set the stopping flag
as it is the only exit for mds-daemon.

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #9342 from vumrao/wip-vumrao-16023

rgw : cleanup radosgw-admin temp command
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9244 from dreamhost/wip-15974

rgw: fix manager selection when APIs customized
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9263 from rzarzynski/wip-rgw-15976

rgw: fix CORS/ACLs metadata processing in Swift API
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9246 from rzarzynski/wip-15977

rgw: fix update of already existing account/bucket's custom attributes.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: we should probably return InvalidArgument when XML_parser fails of RGWPutACLs opReviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: we should probably return InvalidArgument when XML_parser fails of RGWPutACLs op
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #7273 from linuxbox2/rgw-email

Rgw email (#1398)
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw : cleanup radosgw-admin temp command as it was deprecated
and also implementation code for this command was removed in commit
8d7c8828b02c46e119adc4b9e8f655551512fc2d

Fixes: http://tracker.ceph.com/issues/16023
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

Merge pull request #9338 from songbaisen/a1

cephfs: should give a value before use cpp_strerror

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9323 from ukernel/wip-mdsmap-subs

MDSMonitor.cc: fix mdsmap.<namespace> subscriptions

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9033 from renhwztetecs/renhw-wip-cephfs_addlink_python

cephfs: add link function in python

Reviewed-by: John Spray <john.spray@redhat.com>

mds: fix race between StrayManager::{eval_stray,reintegrate_stray}Reviewed-by: John Spray <

mds: fix race between StrayManager::{eval_stray,reintegrate_stray}

Reviewed-by: John Spray <john.spray@redhat.com>

cephfs: should give a value before use cpp_strerror

Signed-off-by:song baisen <song.baisen@zte.com.cn>

packaging: drop python-flask dependency from radosgw

The PowerDNS integration code itself is not packaged at this point, so
remove the python-flask dependency from the radosgw package.

Maybe the PowerDNS integration bits could live in a separate
sub-package, eventually.

Fixes: http://tracker.ceph.com/issues/16032
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

common/WeightedPriorityQueue: add override to func

Add override to virtual functions.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9258 from trociny/wip-15982

pybind: RBD API should default to None for features

Reviewed-by: Jason Dillaman <dillaman@redhat.com >

Renamed file to better indicate it contains markdown syntax.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #9300 from tchaikov/wip-cmake-java

cmake: add libcephfs-java support

Reviewed-by: Casey Bodley <cbodley@redhat.com>

test: workaround failure in journal.sh

With the changes to ensure that the commit position of a new
client is initialized to the minimum position of other clients,
the 'journal inspect/export' commands return zero records because
the master client has committed all of its entries.

Workaround this by restoring the initial commit position after
writing to the image.

Fixes: http://tracker.ceph.com/issues/16011
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

cls::journal: treat empty commit position as minimal

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

journal: extend asynchronous shutdown to facade

Fixes: http://tracker.ceph.com/issues/14530
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: trimmer now has asynchronous shutdown

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: metadata init and shutdown now asynchronous

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: player shutdown is now handled asynchronously

Fixes: http://tracker.ceph.com/issues/15949
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

MDSMonitor.cc: fix mdsmap.<namespace> subscriptions

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9122 from cbodley/wip-rgw-httpmgr-pipe

rgw: clean up pipe fds in RGWHTTPManager::stop()
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8765 from cbodley/wip-rgw-usr2

rgw: extend SIGHUP handler for realm reconfiguration

Merge pull request #9167 from rjfd/wip-ceph-tool-non-ascii-chars

ceph.in: fix exception when pool name has non-ascii characters

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #9282 from dillaman/wip-15993

rbd-mirror: image live-replay might become stuck

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9285 from mikulely/rgw-clean-multipart-macro

rgw: cleanup RGW_MIN_MULTIPART_SIZE

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Improves the README for cmake by including quick-start developer
instructions and slightly improved markdown syntax.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

cmake: add "git-update" target for syncing git submodules

it maps the same functionality in autogen.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add libcephfs-java support

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8672 from zaitcev/wip-5073-multen-2

rgw: Add access to multi-tenancy test

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8533 from theanalyst/rgw/rest-user-get

rgw_rest_admin: return -EINVAL for improper user info requests
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8186 from rzarzynski/wip-rgw-bulkdel-cosm

rgw: optimize BulkDelete authorization and response printing
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8093 from zaitcev/swift-root-prefix

rgw: Allow to serve Swift off the URL root
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8053 from rzarzynski/wip-rgw-aws4-memalloc

rgw: improve memory management in AWS4 auth
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

test: use randomized write sizes for rbd-mirror stress test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: eliminate watch delay for object refetches

The randomized write sizes of the modified rbd-mirror stress
test results in a lot of journal object with few entries.
Immediately fetch objects when performing a refetch check prior
to closing an empty object.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: keep active tag to assist with pruning watched objects

It's possible that there might be additional entries to prune in
objects that haven't been prefetched yet. Keep the active tag
to allow these entries to be pruned after they have been loaded.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: update commit entry object number upon overflow

Otherwise the recorded object positions might point to an older
object that doesn't contain the actual entry.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: cleanup watch refetch flag handling

Clear the refetch required flag while scheduling the watch
and remove the stale object after the watch completes if still
empty. Previously, it was possible for the flag to become
out-of-sync with whether or not it was actually refreshed
and pruned.

Fixes: http://tracker.ceph.com/issues/15993
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9279 from majianpeng/bluestore-uniquelock

os/bluestore/BlueStore: Fix wrong usage of std::unique_lock for OpSequencer

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8741 from rzarzynski/wip-rgw-keystonev3-doc

rgw, doc: improve documentation of Keystone v3 config options

pybind/rados: s/__del__/__dealloc__/

Signed-off-by: Hector Martin <marcan@marcan.st>

test/pybind: explicitly remove references in tearDown

Looks like nose likes to leave references to the test instances around.
This doesn't break anything, but does cause Ioctx/Image objects to stick
around which makes debugging more complicated, as everything gets dealloced
together at the end.

Signed-off-by: Hector Martin <marcan@marcan.st>

pybind/rbd: keep a ref to the Image in SnapIterator

Signed-off-by: Hector Martin <marcan@marcan.st>

os/bluestore/BlueStore: Fix wrong usage of std::unique_lock for OpSequenecer

When set bluestore_sync_wal_apply = false, the osd must be crashed.
I add some debug message in WALWQ::_dequeue():
// preserve wal ordering for this sequencer by taking the lock
/ while still holding the queue lock
-      i->osr->wal_apply_lock.lock();
+      try {
+       i->osr->wal_apply_lock.lock();
+      } catch (const system_error& e) {
+       if (e.code() == std::errc::resource_deadlock_would_occur) {
+         assert(0 == "would deadlock");
+       } else
+         assert(0 == "other system_error");
+      }
       return i;
     }

2016-05-24 00:02:29.107165 7f0f32ea6700 -1 os/bluestore/BlueStore.h: In
function 'virtual BlueStore::TransContext* BlueStore::WALWQ::_dequeue()'
thread 7f0f32ea6700 time 2016-05-24 00:02:29.103360
os/bluestore/BlueStore.h: 489: FAILED assert(0 == "would deadlock")

 ceph version 10.2.0-1232-g21c7599 (21c759936ddcbfec50cf177a577206a2c443c212)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x80) [0x560196215ac0]
 2: (ThreadPool::WorkQueue<BlueStore::TransContext>::_void_dequeue()+0x234)[0x560195e618b4]
 3: (ThreadPool::worker(ThreadPool::WorkThread*)+0x654) [0x560196207324]
 4: (ThreadPool::WorkThread::entry()+0x10) [0x560196208d70]
 5: (()+0x760a) [0x7f0f411d260a]
 6: (clone()+0x6d) [0x7f0f3f17a59d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is
needed to interpret this.

This because the wron usage of std::unique_lock.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #9218 from ceph/wip-asok-denials

rpm: Packaging fixes

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cuthler <ncutler@suse.com>

Merge pull request #8906 from david-z/wip-fix-kcephfs-symlink

mds: wrongly treat symlink inode as normal file/dir when symlink inode is stale on kcephfs

rgw: cleanup RGW_MIN_MULTIPART_SIZE

As the commit 11cf9bbb298768b5307616db5977cc4985072a6e
make rgw_multipart_min_part_size as a config option,
we did not use RGW_MIN_MULTIPART_SIZE anymore.

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Merge pull request #9281 from songbaisen/song29

msg: fix the wrong para in cpp_strerror

Reviewed-by: Kefu Chai <kefu@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

mds: fix race between StrayManager::{eval_stray,reintegrate_stray}

StrayManager::eval_stray() is called after Server::respond_to_request()
drops locks. So it can race with StrayManager::reintegrate_stray()

Fixes: http://tracker.ceph.com/issues/15920
Signed-off-by: Yan, Zheng <zyan@redhat.com>

msg: fix the wrong para in cpp_strerror

Signed-off-by:song baisen <song.baisen@zte.com.cn>

Merge branch 'xio-rename_XioRsp' of https://github.com/Adirl/ceph into Adirl-xio-rename_XioRsp

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #9235 from athanatos/wip-throttle

BackoffThrottle: fix for freebsd and avoid spinning when full

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #9276 from linuxbox2/xio-enqueue-2

xio: follow 03d33e59 (enqueue_for_send()->enqueue())

xio: follow 03d33e59 (enqueue_for_send()->enqueue())

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #9275 from ktdreyer/wip-doc-cephfs-jcsp

doc: John Spray is now CephFS lead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc: John Spray is now CephFS lead

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #8856 from avnerbh/fixes_4_keepalive_to_xio

xio: provide keepalive(s) and their ack(s)

Merge pull request #9171 from weiqiaomiao/wqm-wip-rgw-mem

rgw: fix some memory leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8873 from roidayan/xio-use_xio_session_event_str

xio: use accelio api to get event str

(this is a decision for the Accelio API, we need to follow it)

Merge pull request #9205 from Adirl/fix_compiltaion_entity_addr_t3

Fix compiltaion entity addr t

Merge pull request #9229 from oritwas/wip-15937

rgw: add missing metadata_heap pool to old zones

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rpm: Restore /var/run/ceph context manually

The fixfiles command won't fix label for /var/run/ceph (/run is probably
excluded from relabel), we need to restore the context manually.

Signed-off-by: Boris Ranto <branto@redhat.com>

rpm: Mute semodule in postun script

Currently, we don't mute the semodule output in postun script. This
results in the following message when removing ceph-selinux package:

libsemanage.semanage_direct_remove_key: Removing last ceph module (no
other ceph module exists at another priority).

The fix is to simply mute the output of the command.

Signed-off-by: Boris Ranto <branto@redhat.com>

Revert "rpm: put /sbin/ldconfig into -p"

This reverts commit a53d932fc4a77a1fac33c4f650f2bf629a2f1234.

If the interpret is passed to postun script, everything until the next
rpm section gets interpreted as the input for the interpret (in this
case ldconfig). This results in warnings like

/sbin/ldconfig: relative path `0' used to build cache

when removing the packages that pass the interpret because we use
several '#' characters to separate the subpackage sections and these get
passed to the ldconfig.

Using the default interpret (bash) fixes the warning. I'm also doing the
same to %post script for consistency.

Signed-off-by: Boris Ranto <branto@redhat.com>

BackoffThrottle: wait() if past max

Otherwise, we risk spinning on wait_for() with a small
delay.

Fixes: http://tracker.ceph.com/issues/15953
Signed-off-by: Samuel Just <sjust@redhat.com>