mds: we should wait messenger when MDSDaemon suicide

Signed-off-by: Wei Feng <feng.wei@h3c.com>

Merge pull request #6766 from xiexingguo/xxg-wip-cleanup

osd: fix wrongly placed assert and some cleanups

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6708 from xiaoxichen/shutdown_when_flipping

osd: shut down if we flap too many times in a short period

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5782 from ceph/wip-cxx11time

common: new timekeeping common code, and Objecter conversion

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6796 from dx9/wip-12406-sys_siglist

common: signal_handler: added support for using reentrant strsignal() implementations vs. sys_siglist[]

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6814 from stiopaa1/rem_inc

msg: removed unneeded includes from Dispatcher

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6881 from wido/mon-codestyle-fix

mon: fix coding-style on PG related Monitor files

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6921 from liewegas/wip-kv-log

osd,mon: log leveldb and rocksdb to ceph log

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6950 from mslovy/wip-optimaze-map

osd: misc optimization for map utilization

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6963 from liewegas/wip-memstore

osd: memstore: fix two bugs

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #6982 from dillaman/wip-librbd-valgrind

librbd: uninitialized state in snap remove state machine

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: uninitialized state in snap remove state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6978 from trociny/fix-rbd-init-order

rbd: fix static initialization ordering issues

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

time: Update OSDC for C++11 Time

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

osdc: Whitespace

Since I'm making a pervasive change anyway, knock off end-of-line spaces
and wrap overly long lines.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

perfcounters: Allow C++11 time increment/decrement

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

timer: Another timer using new time/lock standards

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

time: Overload for std:chrono in Cond::TimedWait

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

time: Add basic support for std::chrono based time

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #6701 from XinzeChi/wip-batch-peek-journal

osd: FileJournal: support batch peak and pop from writeq

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6804 from ceph/putting-the-fail-in-assertion-failed

common: assert: abort() rather than throw

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6900 from liewegas/wip-13988

mon: fix reuse of osd ids (clear osd info on osd deletion)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6902 from XinzeChi/wip-pg_stat_t-encoding

osd: utime_t, eversion_t, osd_stat_sum_t encoding optimization

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6909 from mslovy/wip-shared-cache

common: improve shared_cache and simple_cache efficiency with hash table

Reviewed-by: Sage Weil <sage@redhat.com>

rbd: fix "static initialization order fiasco"

The order in which static objects in different compilation units are
initialized is undefined. Use static functions as a workaround.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #6968 from xiexingguo/xxg-wip-14118

librbd: exit if parent's snap is gone during clone

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: exit if parent snap is gone during clone

Fixes: #14118
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #6914 from x11562/x11562

rbd: additional validation for striping parameters

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6958 from xiexingguo/patch-1

librbd: return error if we fail to delete object_map head object

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6965 from kylinstorage/wip-rbd-default-order

rbd: use default order from configuration when not specified

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6928 from HeyoQiang/heyongqiang1215

Monitor: modify a dout level in OSDMonitor.cc

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

mon: modify the level of a log about OSD's condition in OSDMonitor.cc

    In actual use, we replace old hard disk by new or do performance testing regularly and need to down OSDs manually, but there are some OSDs can not be marked down, and there is no obvious information.
The log { dout(5) << "can_mark_down current up_ratio " << up_ratio << " < min "<< g_conf->mon_osd_min_up_ratio<< ", will not mark osd." << i << "down" << dendl ; }" can explain  why it happened.
   In addition, we can change the value of mon_osd_min_up_ratio more reasonable in our operating environment.so it is necessary to adjust the log's level lower.

For example:
   There are 6 OSDs , wen have marked down 5 of them and the mon_osd_min_up_ratio = 0.3.
   In this situation, when we mark the last OSD to down, it will show "ceph-osd stop/waiting", but in actually, the OSD is still up.

Signed-off-by: Yongqiang He <he.yongqiang@h3c.com>

rbd: use default order value from g_conf

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

Merge pull request #6847 from oritwas/wip-fix-warning

build: fix a few warnings

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6924 from stiopaa1/removefriend

msg/simple: remove unneeded friend declarations

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6405 from jcsp/wip-9963

librados: new style (sharded) object listing

Reviewed-by: Sage Weil <sage@redhat.com>

os/MemStore: fix use-after-free on iterator

Signed-off-by: Sage Weil <sage@redhat.com>

memstore: fix loop in PageSetObject::clone()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #6945 from jcsp/wip-blacklist-clear

mon: add `osd blacklist clear`

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #6959 from oritwas/wip-fix-run-cmake-check

cmake: no need to run configure from run-cmake-check.sh

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6564 from branch-predictor/bp-fix-mon-kv-store

mon: fixes related to mondbstore->get() changes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6202 from cxwshawn/wip-ful-fix

rados: implement rm --force option to force remove when full

Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd: fix "was hidden" warning

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

newstore: fix was hidden warning

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: use unsigned long to print to log

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #6956 from somnathr/wip-conditional-drive-metadata-collection

osd: FileStore: conditional collection of drive metadata

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6929 from clever215/mon-log

mon: degrade a log message to level 2

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

cmake: no need to run configure from run-cmake-check.sh

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #6864 from scienceluo/wip-cleanup

osdmap: remove unused local variables

Reviewed-by: Loic Dachary <ldachary@redhat.com>

librbd: return error if we fail to delete object_map head object

Fixes: #14098
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osdmap: remove unused local variables

Signed-off-by: luo kexue <luo.kexue@zte.com.cn>

Merge pull request #6904 from wuxiangwei/wip-wxw-rbdadminsocket

rbd: add pool name to disambiguate rbd admin socket commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6942 from x11562/stripe

rbd: striping parameters should support 64bit integers

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

FileStore: Conditional collection of drive metadata

get_device_by_uuid->blkid_find_dev_with_tag() call from
FileStore::collect_metadata() is hanging for ~3min before returning
EINVAL in case the drive is visible but reserved for some other host.
This is probably is bug within blkid* calls. fdisk/lsblk call is coming
out immediately saying device is inaccessible. This call is now
protected by config option filestore_collect_device_partition_information

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>

doc/release-notes: v10.0.1

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/jewel'

Merge pull request #6932 from aiicore/test_fix

tests: allow object corpus readable test to skip specific incompat instances

Reviewed-by: Sage Weil <sage@redhat.com>

mon: add `osd blacklist clear`

This is just like 'blacklist rm' except it removes
everything.  Useful if you've got a whole bunch of
things in your blacklist and you don't want to wait
for N "blacklist rm" commands to run.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6941 from jcsp/wip-14029

mds: fix setvxattr (broken in a536d114)

osd: use unordered_map for repop_map
With a prediction of expected per pg maximum ops to initialize the number of hash bucket

Signed-off-by: Ning Yao <zay11022@gmail.com>

rbd: process crushed, rbd creating with striping parameters

Signed-off-by: Na Xie <xie.na@h3c.com>

Merge pull request #6948 from ceph/revert-6939-bp-remove-bz2dev

Revert "makefiles: remove bz2-dev from dependencies"

Reviewed-by: Dan Mick <dmick@redhat.com>

Revert "makefiles: remove bz2-dev from dependencies"

osd: combine map.count() and map.find() into one searching operation

Signed-off-by: Ning Yao <zay11022@gmail.com>

Merge pull request #6692 from SUSE/wip-13860

ceph.spec.in: add BuildRequires: systemd

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #6939 from branch-predictor/bp-remove-bz2dev

makefiles: remove bz2-dev from dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6943 from smithfarm/wip-doc-dev-typo

doc: fix typo in developer guide

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: fix typo in developer guide

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rbd: specify pool name for rbd admin socket commands

Add the pool name for a given rbd imgae when executing rbd admin socket
commands in case there are more than one images with the same name in
different pools.

Signed-off-by: Xiangwei Wu wuxiangwei@h3c.com

mds: fix setvxattr (broken in a536d114)

There was an accidental move of this line
when adding the MAY_SET_POOL check, which
was causing setxattr to proceed before
it had the right locks, and thereby apply
its checks on bad data (symptom was failing
to detect that the file had data written to it).

Fixes: #14029
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6821 from H3C/wip-cephfs-f11322

ceph-fuse: add process to ceph-fuse --help

Reviewed-by: John Spray <john.spray@redhat.com>

makefiles: remove bz2-dev from dependencies

The only thing that uses bzip2-devel is RocksDB, and it's optional, not
requirement. Drop the bzip2-devel/libbz2-dev dependency entirely, and
let RocksDB use it only if it is already present.

Fixes: #13981
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

ceph-fuse: add process to ceph-fuse --help

Signed-off-by: Wei Feng <feng.wei@h3c.com>

Merge pull request #6934 from smithfarm/wip-14070

doc: Update list of admin/build-doc dependencies

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: Update list of admin/build-doc dependencies

http://tracker.ceph.com/issues/14070 Fixes: #14070

Signed-off-by: Nathan Cutler <ncutler@suse.com>

test/encoding/readable.sh: add non-whole type skip

With this modification there will be possibility to skip only one/few
object/s of particular type, not whole type as it was before.

Before:
- To skip whole TYPE create file named TYPE in forward_incompat
directory.

Now:
- To skip whole TYPE create file or empty directory named TYPE in
forward_incompat directory.
- To skip one/few object/s of TYPE create directory named TYPE in
forward_incompat and put into symbolic links to objects that
you wantt to skip.

Signed-off-by: Igor Podoski <igor.podoski@ts.fujitsu.com>

Merge pull request #6910 from dachary/wip-dev-guide-ceph-qa-suite

doc: dev: introduction to tests

Reviewed-by: Nathan Cutler <ncutler@suse.com>

check for striping parameters

stripe_unit is same to object size or stripe_count is equal to 1,the remainder for it is misleading.

Signed-off-by: Na Xie <xie.na@h3c.com>

Merge pull request #6927 from tchaikov/wip-fix-ftbfs

rgw: fix the build failure

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: remove unused variable

to fix the compiler warning of

 rgw/rgw_formats.cc: In member function 'virtual void
 RGWFormatter_Plain::dump_format_va(const char*, const char*, bool, const
 char*, __va_list_tag*)':
 rgw/rgw_formats.cc:143:15: warning: unused variable 'format'
 [-Wunused-variable]
    const char *format;

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: fix the build failure

* s/bucket_name_str/bucket_name/
* the member variable name of `req_state` was changed to `bucket_name`
  in f7ca00a. but some commits was still using the old name before the
  commit got merged.
* fixes for the tenant related change
* also fixes a typo

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6859 from dillaman/wip-librbd-async-refresh

librbd: image refresh code paths converted to async state machines

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

kv/LevelDBStore: log leveldb to ceph log

This greatly eases debugging, and avoids the unbounded growth of
the leveldb LOG file.

Signed-off-by: Sage Weil <sage@redhat.com>

kv/RocksDBStore: log to ceph log

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: refresh object map after rollback

An image refresh will no longer trigger an automatic object map
reload, therefore explicitly reload the object map after rollback.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: refresh image asynchronously from watch/notify path

The AIO path already ensures asynchronous image refreshes, but the
watch/notify path for maintenance ops was still using the sync refresh
path.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: the journal state machine is now asynchronous

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: adjust test cases to refactored librbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: all object map methods are now async

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: IO ops will now asynchronously refresh the image if needed

This avoids blocking the AIO path with the previous synchronous
refresh path.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

WorkQueue: PointerWQ now supports out-of-band processing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: flush all in-flight IO on block write and shut down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add shut down support to the AIO work queue

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: integrate new async image state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ImageWatcher should release lock upon request

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: implement image state tracker for open/close/refresh/etc

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: rename ImageRefresh to ImageState

This class handles more than just refreshing the image from
disk.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: object map lifespan now controlled by state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: decouple ImageWatcher from exclusive lock management

Use new ExclusiveLock state machine to handle all the proper
transitions between lock states.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: separate image refresh into async state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>