Merge remote-tracking branch 'origin/jewel'

Merge pull request #7026 from xdonghai/master

rbd: must specify both of stripe-unit and stripe-count when specifying stripingv2 feature

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6998 from xiexingguo/xxg-wip-clsrbd

stringify outputted error code and fix unmatched parentheses.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6983 from xiexingguo/xxg-wip-14126

librbd: small fixes for error messages and readahead counter

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6298 from guangyy/wip-13441

osd: make list_missing query missing_loc.needs_recovery_map

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6572 from liewegas/wip-crush-chooseleaf-stable

crush: add chooseleaf_stable tunable

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #7043 from dillaman/wip-14170-jewel

librbd: do not ignore self-managed snapshot release result

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: do not ignore self-managed snapshot release result

Fixes: #14170
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7042 from dillaman/wip-14164-jewel

librbd: properly handle replay of snap remove RPC message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7041 from dillaman/wip-14165

qa/workunits: merge_diff shouldn't attempt to use striping

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7040 from dillaman/wip-14092-jewel

librbd: ensure librados callbacks are flushed prior to destroying

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7036 from xiexingguo/xxg-wip-kvstore

KeyValueStore: fix return code of mkfs

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #7020 from chenyehua11692/master

doc:adding "--allow-shrink" in decreasing the size of the rbd block to distinguish from the increasing option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7035 from dillaman/wip-14122-jewel

librbd: clear error when older OSD doesn't support image flags

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: properly handle replay of snap remove RPC message

Fixes: #14164
Backport: infernalis
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits: merge_diff shouldn't attempt to use striping v2

Fixes: #14165
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ensure librados callbacks are flushed prior to destroying image

Fixes: #14092
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

KeyValueStore: fix return code of mkfs

It shall return a negative result code instead.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librbd: clear error when older OSD doesn't support image flags

Fixes: #14122
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6996 from H3C/wip-mds-f11322

mds: we should wait messenger when MDSDaemon suicide

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #7015 from chipitsine/master

ceph-fuse: fix double free of args

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7004 from jazeltq/master

doc: fix typo

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6870 from mslovy/wip-pgmeta-object

osd: os: skip checking pg_meta object existance in FileStore

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6744 from majianpeng/scrub-fix

osd: release related sources when scrub is interrupted

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6961 from liewegas/wip-lockdep-max

common/lockdep: increase max lock names

Reviewed-by: Kefu Chai <kchai@redhat.com>

mds: we should wait messenger when MDSDaemon suicide

Signed-off-by: Wei Feng <feng.wei@h3c.com>

Merge pull request #7025 from tchaikov/wip-ceph-detect-init-py3

ceph-detect-init: fix py3 test

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph-detect-init: fix py3 test

* in python3, None can not be compared with a str. and
  we should always set codename with a non empty str, so update
  TestCephDetectInit.test_debian so it always set a code name.
  and assert(codename and distroname) in choose_init().

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7023 from tchaikov/wip-doc-cachemode

doc: document "readforward" and "readproxy" cache mode

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rbd:must specify both of stripe-unit and stripe-count when specify striping features

when create a rbd image with striping features, Be sure specify both of stirpe-unit and stripe-count too, If not, refuse to excute and give a message

Signed-off-by:Donghai Xu <xu.donghai@h3c.com>

doc: document "readforward" and "readproxy" cache mode

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc:adding "--allow-shrink" in decreasing the size of the rbd block to distinguish from the increasing option

In the original file, the increasing and decreaing of the size of the rbd block shares the same option:
    "rbd resize --size 2048 foo".
However, it is not proper, as the "--allow-shrink" needs to be added while decreasing the size of the rbd block.
As a result, it is necessary to make a distinguish between these two options as follows:
    "rbd resize --size 2048 foo (to increase)"
    "rbd resize --size 2048 foo --allow-shrink (to decrease)"

Signed-off-by: Yehua <chen.yehua@h3c.com>

ceph doc fix slip of pen

mon client hung interval -> mon client hunt interval

Signed-off-by: tianqing <tianqing@unitedstack.com>

ceph-fuse: fix double free of args

Checking src/ceph_fuse.cc...
[src/ceph_fuse.cc:55]: (error) Memory pointed to by 'argv' is freed twice.
[src/ceph_fuse.cc:55]: (error) Deallocating a deallocated pointer: argv

Signed-off-by: Ilya Shipitsin <chipitsine@gmail.com>

Merge pull request #6988 from xiexingguo/xxg-wip-14134

FileJournal: fix return code of create method

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #7005 from YankunLi/patch-4

delete default zone

Merge pull request #7006 from YankunLi/patch-5

correct radosgw-admin command

Merge pull request #6997 from zhouyuan/evict_check_range

osd: cache tier: add config option for eviction check list size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6990 from linuxbox2/master-fixes

Fixes some small issues

Reviewed-by: Kefu Chai <kchai@redhat.com>

correct radosgw-admin command

the command to disable users quota should be 'radosgw-admin quota disable --quota-scope=<user | bucket> --uid=<uid>'

delete default zone

if the default zone exist, delete the default zone from both the east and west pools.

Merge remote-tracking branch 'origin/jewel'

Merge pull request #6926 from dachary/wip-14080-ceph-disk-udevadm

ceph-disk: fix failures when preparing disks with udev > 214

On CentOS 7.1 and other operating systems with a version of udev greater or equal to 214,
running ceph-disk prepare triggered unexpected removal and addition of partitions on
the disk being prepared. That created problems ranging from the OSD not being activated
to failures because /dev/sdb1 does not exist although it should.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7002 from dillaman/wip-14092-jewel

tests: flush op work queue prior to destroying MockImageCtx

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6986 from xiexingguo/xxg-wip-14129

librbd: fix snap_exists API return code overflow

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: update test cases with the new snap_exists API

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #6938 from wuxiangwei/wip-wxw-rbdfuserename

rbd-fuse: implement mv operation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

ECBackend: fix unmatched parenttheses

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rbd: stringify outputted error code

As it is more human-readable.

Signed-off-by: xie.xingguo <xie.xingguo@zte.com.cn>

tests: flush op work queue prior to destroying MockImageCtx

Fixes: #14092
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph-disk: protect deactivate with activate lock

When ceph-disk prepares the disk, it triggers udev events and each of
them ceph-disk activate. If systemctl stop ceph-osd@2 happens while
there still are ceph-disk activate in flight, the systemctl stop may be
cancelled by the systemctl enable issued by one of the pending ceph-disk
activate.

This only matters in a test environment where disks are destroyed
shortly after they are activated.

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-disk: use blkid instead of sgdisk -i

sgdisk -i 1 /dev/vdb opens /dev/vdb in write mode which indirectly
triggers a BLKRRPART ioctl from udev (starting version 214 and up) when
the device is closed (see below for the udev release note). The
implementation of this ioctl by the kernel (even old kernels) removes
all partitions and adds them again (similar to what partprobe does
explicitly).

The side effects of partitions disappearing while ceph-disk is running
are devastating.

sgdisk is replaced by blkid which only opens the device in read mode and
will not trigger this unexpected behavior.

The problem does not show on Ubuntu 14.04 because it is running udev <
214 but shows on CentOS 7 which is running udev > 214.

git clone git://anonscm.debian.org/pkg-systemd/systemd.git
systemd/NEWS:
CHANGES WITH 214:

        * As an experimental feature, udev now tries to lock the
          disk device node (flock(LOCK_SH|LOCK_NB)) while it
          executes events for the disk or any of its partitions.
          Applications like partitioning programs can lock the
          disk device node (flock(LOCK_EX)) and claim temporary
          device ownership that way; udev will entirely skip all event
          handling for this disk and its partitions. If the disk
          was opened for writing, the close will trigger a partition
          table rescan in udev's "watch" facility, and if needed
          synthesize "change" events for the disk and all its partitions.
          This is now unconditionally enabled, and if it turns out to
          cause major problems, we might turn it on only for specific
          devices, or might need to disable it entirely. Device Mapper
          devices are excluded from this logic.

http://tracker.ceph.com/issues/14094 Fixes: #14094

Signed-off-by: Ilya Dryomov <idryomov@redhat.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-disk: dereference symlinks in destroy and zap

The behavior of partprobe or sgdisk may be subtly different if given a
symbolic link to a device instead of an actual device. The debug output
is also more confusing when the symlink shows instead of the device it
points to.

Always dereference the symlink before running destroy and zap.

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-disk: increase partprobe / udevadm settle timeouts

The default of 120 seconds may be exceeded when the disk is very slow
which can happen in cloud environments. Increase it to 600 seconds
instead.

The partprobe command may fail for the same reason but it does not have
a timeout parameter. Instead, try a few times before failing.

The udevadm settle guarding partprobe are not necessary because
partprobe already does the same. However, partprobe does not provide a
way to control the timeout. Having a udevadm settle after another is
going to be a noop most of the time and not add any delay. It matters
when the udevadm settle run by partprobe fails with a timeout because
partprobe will silentely ignores the failure.

http://tracker.ceph.com/issues/14080 Fixes: #14080

Signed-off-by: Loic Dachary <loic@dachary.org>

tests: ceph-disk workunit increase verbosity

So that reading the teuthology log is enough in most cases to figure out
the cause of the error.

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-disk: fix typo

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-disk: log parted output

Should parted output fail to parse, it is useful to get the full output
when running in verbose mode.

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-disk: do not discard stderr

Signed-off-by: Loic Dachary <loic@dachary.org>

cache-tier: Allow to config eviction check max size

This patch adds the option for eviction check size in cache-tier.
On a busy setup, it's better to check bigger number of objects so
the eviction is faster.

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Dispatcher.h: include assert.h

The inline ms_fast_dispatch implementation calls assert, which
puts asset.h in the interface.

The fact that many Dispatcher descendants don't implement
ms_fast_dispatch prevents making the method pure virtual--which
suggests that maybe there is a need for a FastDispatcher
interface that inherits Dispatcher and introduces ms_fast_dispatch
and related?

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

xio: avoid conversion warning w/xio_queue_depth

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

xio: remove static declspec on buffer::create_msg

Likely an update missed from PR #6686.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

FileJournal: fix return code of create method

Shall return negative error code instead.

Fixes: #14134
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librbd: fix wrongly reported error code

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rbd-fuse: discard space restriction for mv operation

discard space restriction for destination image name of
the mv operation.

Signed-off-by: Xiangwei Wu wuxiangwei@h3c.com

librbd: string standard error number

It's more human-readable.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librbd: fix wrong tip message

As equivalent size is ok for copy.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

tools: replace snap_exists with a new safer version

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librbd: fix snap_exists API overflow issue

The original one may overflow and thus not be safe.

Fixes: #14129
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librbd: fix readahead counter update logic

Fixes: #14127
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #6766 from xiexingguo/xxg-wip-cleanup

osd: fix wrongly placed assert and some cleanups

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6708 from xiaoxichen/shutdown_when_flipping

osd: shut down if we flap too many times in a short period

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5782 from ceph/wip-cxx11time

common: new timekeeping common code, and Objecter conversion

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6796 from dx9/wip-12406-sys_siglist

common: signal_handler: added support for using reentrant strsignal() implementations vs. sys_siglist[]

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6814 from stiopaa1/rem_inc

msg: removed unneeded includes from Dispatcher

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6815 from badone/wip-13993

common: log: Assign LOG_DEBUG priority to syslog calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6881 from wido/mon-codestyle-fix

mon: fix coding-style on PG related Monitor files

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6921 from liewegas/wip-kv-log

osd,mon: log leveldb and rocksdb to ceph log

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6950 from mslovy/wip-optimaze-map

osd: misc optimization for map utilization

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6963 from liewegas/wip-memstore

osd: memstore: fix two bugs

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #6982 from dillaman/wip-librbd-valgrind

librbd: uninitialized state in snap remove state machine

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: uninitialized state in snap remove state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6979 from dillaman/wip-14123-jewel

librbd: correct issues discovered when cache is disabled

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6974 from dillaman/wip-14108-jewel

librbd: correct issues discovered during teuthology testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6978 from trociny/fix-rbd-init-order

rbd: fix static initialization ordering issues

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

time: Update OSDC for C++11 Time

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

osdc: Whitespace

Since I'm making a pervasive change anyway, knock off end-of-line spaces
and wrap overly long lines.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

perfcounters: Allow C++11 time increment/decrement

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

timer: Another timer using new time/lock standards

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

time: Overload for std:chrono in Cond::TimedWait

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

time: Add basic support for std::chrono based time

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

librbd: skip journal event if write/discard was no-op

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: object map ENOENT optimizations should complete asynchronously

This ensures a consistent lock ordering between the optimized path and
a librados callback. Remove the cache writebach handler's custom finisher
and re-use the op_work_queue.

Fixes: #14123
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6701 from XinzeChi/wip-batch-peek-journal

osd: FileJournal: support batch peak and pop from writeq

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6804 from ceph/putting-the-fail-in-assertion-failed

common: assert: abort() rather than throw

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6900 from liewegas/wip-13988

mon: fix reuse of osd ids (clear osd info on osd deletion)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6902 from XinzeChi/wip-pg_stat_t-encoding

osd: utime_t, eversion_t, osd_stat_sum_t encoding optimization

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6909 from mslovy/wip-shared-cache

common: improve shared_cache and simple_cache efficiency with hash table

Reviewed-by: Sage Weil <sage@redhat.com>

rbd: fix "static initialization order fiasco"

The order in which static objects in different compilation units are
initialized is undefined. Use static functions as a workaround.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: update exclusive lock state when shutting down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>