Doc: Updated ceph-deploy osd to choose underlying file system

Included paragraph about --fs-type and --bluestore

Signed-off-by: Ignacio Bravo ibravo@hotmail.com

Merge pull request #8640 from xiexingguo/xxg-wip-osdsideeffect

osd/ReplicatedPG: allow CMPXATTR ops to trigger side effects too

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8694 from wuxingyi/fixincomplete

osd/PG: PG is only likely to go active when min_size turns smaller.

Reviewed-by: Sage Weil <sage@redhat.com>

osd: honour result code passed in; tidy-up changesReviewed-by: Sage Weil <sage@redhat.com>

osd: honour result code passed in; tidy-up changes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8675 from roidayan/xio-build_deb_with_xio

xio: add MNop.h to dist tarball

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8678 from jack-changtao/pgbackend

osd/PGBackend: if min > max, PGBackend::objects_list_partial will list all objects of that pg

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8680 from ceph/rm-distribute

tools: remove installation for distribute. It is no longer maintained

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8681 from xiexingguo/xxg-wip-crushfix

crush: fix mishandled result of get_bucket() method

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8707 from liewegas/wip-kv-merge

kv: add merge operator support, fix test

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8688 from xiexingguo/xxg-wip-crushtooltip

tools/crushtool: add straw2 support for help message

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8671 from majianpeng/rados-objectsize-smaller-blocksize

tools/rados: fix "Floating point exception" if  objectsize smaller than  blocksize when do write-bench.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8502 from songbaisen/song20

crushtool: do not allow to compile crushmap with repeat ruleset.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8724 from aderumier/master

sysvinit: fix ceph init script

Reviewed-by: Sage Weil <sage@redhat.com>

fix ceph init script

commit 65963739cd6815b8008282c8f64cd64365662e60 have introduce a bug

test variables need to be quoted, or -n always return true, even if variables don't exist

Merge pull request #7942 from xiexingguo/xxg-wip-fixdirtyinc

mon: OSDMonitor: improve pool creation logic

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Xiaoxi Chen <xiaoxchen@ebay.com>

Merge pull request #7997 from songbaisen/song11

osdc: Reverse the probing vector in better way.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8384 from songbaisen/song18

crush: when take place the crush map should consider the rule is in …

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8591 from xiexingguo/xxg-wip-15492

test: generate temp pool name in a safer way

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8577 from runsisi/wip-fix-op-lock

osdc/Objecter: small cleanup for osd op completion lock

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8596 from xiexingguo/xxg-wip-blue

os/filestore: add sanity check for stat() syscall

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8710 from ceph/wip-rename-tests

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
  renamed the workunit and test to reflect reality and for better clarity

Merge pull request #8395 from renhwsky/renhw-wip-monmap

mon/monmap:  conditional optimization in build_from_host_list

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8449 from javacruft/ceph-osd-prestart

ceph-osd-prestart.sh: drop --setuser/--setgroup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8458 from jjhuo/clean_blue_configs

common/config_opts: remove invalid config options of bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8463 from majianpeng/bluestore-zero-truncate

os/bluestore/BlueStore: Don't forget assign nid when offset > size in _do_zero

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8706 from liewegas/wip-jewel

doc/release-notes: is, not will be, basis for LTS

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

kv/RocksDBStore: implement merge operator

Signed-off-by: Allen Samuels <allen.samuels@sandisk.com>

kv/KeyValueDB: add merge operator interface and test

Signed-off-by: Allen Samuels <allen.samuels@sandisk.com>
Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_keyvaluedb: clean up test dir

Just like ceph_test_objectstore

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: is, not will be, basis for LTS

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8705 from liewegas/wip-jewel

doc/release-notes: revise terminology

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc/release-notes: revise terminology

Avoid loaded 'production ready' term in favor of just 'stable'.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: fix memory leak in func 'RGWCreateBucket_ObjStore_S3::get_params' (#8670)

Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph_test_keyvaluedb: fixup unit test for keyvaluedb

Fixes: http://tracker.ceph.com/issues/15435
Signed-off-by: Allen Samuels <allen.samuels@sandisk.com>

Merge pull request #8527 from roidayan/wip-vstart_clean_prints

vstart: clean prints

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8549 from tchaikov/wip-kill-filestore-warning

pybind, FileStore: silence gcc warnings

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8551 from scienceluo/patch-1

doc: Fixes Different font of the title

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Yatin Kumbhare <yatinkumbhare@gmail.com>

bufferlist.cc fix typo in outputReviewed-by: Sage Weil <sage@redhat.com>

bufferlist.cc fix typo in output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8581 from emenguy/pgnum_pools_documentation

doc: adding pg_num to pools documentation

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #8595 from qhjindev/fix-LFNIndex

os: fix typo in LFNIndex.h

Merge pull request #8623 from rmechler/wip-15504

pybind: removed unneccesary parentheses from if statements

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8631 from stiopaa1/buffer_fwd

image_watcher/Notifier: remove header file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8644 from tchaikov/wip-doc-os-recommendations

doc/start/os-recommendations.rst: add Jewel requires

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8650 from scienceluo/wip-lkx-fixesdoc

doc: Fixes "branch -b" and "Qemu"

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #8674 from yatinkumbhare/doc-fix2

doc: fix broken link to monitor quorum

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #8630 from roidayan/xio-log_prefix

xio: add log prefix for log prints outside the XioMessenger class

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8701 from cernceph/wip-bsweight

ceph-osd-prestart.sh: compute correct defaultweight for bluestore

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8691 from flyd1005/master

cleanup: Fix typos, change prefered to preferred

Reviewed-by: Sage Weil <sage@redhat.com>

compute correct defaultweight for bluestore

The default crush weight should be the size of the block device, not
the tiny 10M xfs partition.

Backport: jewel
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

os/bluestore/BlueFS: Record the correctly perfcouter

os/bluestore/BlueFS: record the correctly perfcounter

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: Record the correctly perfcouter.

After "log_writer->append(bl)", the length of bl become zero.
So tune the order.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #8696 from cernceph/wip-util

osd: fix backwards min/max osd utilization

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8676 from runsisi/wip-fix-init-exlock

librbd: fix potential double free of SetSnapRequest instance

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: fix potential double free of SetSnapRequest instance

if image feature EXCLUSIVE_LOCK is not enabled we should not try to
initialize the exclusive lock, or we may end with two async Contexts
to finish the same SetSnapRequest instance

Fixes: http://tracker.ceph.com/issues/15571
Signed-off-by: runsisi <runsisi@zte.com.cn>

pybind/rados: silence "-Wstrict-prototypes" warnings

it silences warnings like:
```
warning:
/srv/autobuild-ceph/gitbuilder.git/build/build/src/pybind/rados/rados.cpp:11361:35:
comparison between signed and unsigned integer expressions
[-Wsign-compare]
__pyx_t_5 = ((__pyx_v_ret > __pyx_v_size) != 0);
^
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind: silence gcc warnings about '-Wstrict-prototypes'

this silences following warning

```
warning: cc1plus: command line option ‘-Wstrict-prototypes’ is valid for
C/ObjC but not for C++ [enabled by default]
```

* distutils pass "-DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes" to
  compiler by default if OPT environment variable is not set. so we opt to
  remove '-Wstrict-prototypes' as we are using c++ as the language of the
  generated bindinging.
* also use "env" to pass the environment variables to avoid unnecessary
  quote added by distutils, which just breaks the generated CLI command
  line.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8530 from wjwithagen/patch-6

ceph-helpers.sh: only use mon*pid files when killing MONs

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: config-h.in.cmake: avoid redefinition warnings

when compiling python bindings, the included "pyconfig.h" also defines
some of the macros which are duplicated with our versions in acconfig.h.
so do not define them if they are already defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/FileStore::OpSequencer: operator<<: silence gcc warning

it kills following warning:
```
/home/kefu/dev/ceph/src/os/filestore/FileStore.cc: In function
‘std::ostream& operator<<(std::ostream&, const
FileStore::OpSequencer&)’:
/home/kefu/dev/ceph/src/os/filestore/FileStore.cc:155:132: warning: the
compiler can assume that the address of ‘out’ will always evaluate to
‘true’ [-Waddress]
   assert(&out);
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8662 from majianpeng/nvmedevice-cleanup

os/bluestore/NVMEDevice: Remove the duplicated code.

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #8697 from vumrao/wip-vumrao-15568

man : ceph-deploy add config keyword in ceph-deploy

Reviewed-by: Alfredo Deza <adeza@redhat.com>

man : ceph-deploy add config keyword in ceph-deploy
      config push and pull command

Fixes: http://tracker.ceph.com/issues/15568
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

osd: fix backwards min/max osd utilization

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

pgbackend: if min > max ,PGBackend::objects_list_partial will list all objects of that pg

Signed-off-by: Tao Chang <changtao@hihuron.com>

PG: PG is only likely to go active when min_size turns smaller.

Signed-off-by: wuxingyi <wuxingyi@letv.com>

Fix typos, change prefered to preferred

tools/crushtool: add straw2 support for help message

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8539 from vuhuong/wip-xio-heavy-msgrs

Wip Xio heavy messengers and resources

Merge pull request #8649 from dzafman/wip-test-cot-fix

test: Fix ceph-objectstore-tool test to run manually from src non-cmake

common/config_opts: remove obsolete config options of bluestore

Those obsolete options are leftovers from newstore, and are not
used anymore.

Signed-off-by: Jianjian Huo <jianjian.huo@ssi.samsung.com>

Merge remote-tracking branch 'gh/jewel'

doc/release-notes: v10.2.0 jewel

Signed-off-by: Sage Weil <sage@redhat.com>

tools/rados: fix "Floating point exception" if  objectsize smaller than blocksize when do write-bench.

When do write-bench, if objectsize is smaller than blocksize. It will
casue "Floating point exception".
This because in bjBencher::write_bench:
>>if (data.op_size)
>>    writes_per_object = data.object_size / data.op_size;
  It make writes_per_object = 0;
......
>> name[i] = generate_object_name(i / writes_per_object);
  i/0 cause Floating point exception.

If objectsize is smaller than blocksize, make blocksize is eqaul
objectsize.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

crush: remove unnecessary sanity check

We have done sanity check with bucket_exists() above, so these
checks are redundant and less effcient.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: kill code we never used

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: use bucket_exists() for item sanity checks

This is because get_bucket() never returns 0 on error,
it always returns a non-zero value. So if you
assert(get_bucket(item)), you always get a postive answer.

Replace existence check for a specific item

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: fix get_rule_step() can't catch all errors

Because get_rule() may return -ENOENT on error, we shall
use the IS_ERR macro to catch that, as all the other callers does.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

crush: fix mishandle result of get_bucket() method

Get_bucket() is supposed to return a pointer to a specific bucket on success.
However, it never returns a null pointer if error occurs. It returns
-ENOENT instead. That's why most caller is misjudge the result code of get_bucket()
method.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

xio: add MNop.h to dist tarball

This file is needed when we want to build a package with xio messenger
but was not added to dist tarball.

Signed-off-by: Roi Dayan <roid@mellanox.com>

Merge pull request #8667 from SUSE/wip-15549

rpm: refrain from installing systemd unit files twice

Reviewed-by: Boris Ranto <branto@redhat.com>

doc: fix broken link to monitor quorum

Link pointing in Monitor Quorum section to
ceph configuration file was broken.
Fixing it to point to configuration section.

Signed-off-by: Yatin Kumbhare yatinkumbhare@gmail.com

install-deps: remove distribute requirement

Because it was meant for Ubuntu 12.04 and it is installing directly
from PyPI on newer releases that aren't being built for 12.04

Signed-off-by: Alfredo Deza <adeza@redhat.com>

tools: remove installation for distribute. It is no longer maintained and breaks builds

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #8643 from ceph/wip-udev

udev: remove 60-ceph-partuuid-workaround-rules

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

rpm: refrain from installing systemd unit files twice

These files are already installed by systemd/Makefile.am

Fixes: http://tracker.ceph.com/issues/15549
Signed-off-by: Nathan Cutler <ncutler@suse.com>

os/bluestore/NVMEDevice: Remove the duplicated code.

In spdk/pci.h spdk/nvme.h, it alread contain those code:
extern "C" {
 So remove the duplicated code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #8664 from theanalyst/doc/rgw/fix-multisite-zone

doc: multisite: fix json output for zone create

10.2.0

doc: mutlisite: fix json output for zone create

creation of ``us-west`` was wrongly showing ``us-east-2`` output

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

osd: kill PG_STATE_SCRUBQ state, which is never used

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: kill get_any_up_osd(), which is never used

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8658 from ceph/wip-cmake

cmake: Added CMAKE_CEPH_*_VIRTUALENV var to fix build

cmake: Added CMAKE_CEPH_*_VIRTUALENV var to fix build

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #8636 from ceph/wip-rgw-leak

rgw leaks fixes

Merge pull request #8653 from theanalyst/doc/fix-multisite-pools

doc: rgw multisite, add pools section & minor cosmetic improvements

rgw: drop rest op reference in error path

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

doc: rgw multisite, add pools section & minor cosmetic improvements

`Pool Configuration` was mentioned previously but didn't contain any
info, adding this section and minor cosmetic changes to a few headings
after looking at the html view.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

osd/ReplicatedPG: kill members we never used

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/filestore: improve error message of aio write failure

e.g. when we encountered an AIO error, the output is not
very human-readable, as below:

    -1> 2016-04-19 12:07:57.387805 7f192e320700 -1 journal aio to 53059584~1744896 wrote 18446744073709551611
     0> 2016-04-19 12:07:57.389205 7f192e320700 -1 os/FileJournal.cc: In function 'void FileJournal::write_finish_thread_entry()' th
read 7f192e320700 time 2016-04-19 12:07:57.387831

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>