msg/async: refactor NetHandler::set_priority() to sync with msg/simple

so it's sync'ed with Pipe::set_socket_options()

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/simple: use my addr when setting sock priority

actually we always fails to set ToS of listening
socket. 6f1037e22c2a304795895498cdc955e0ef80f8e3 reveals this issue.

Fixes: http://tracker.ceph.com/issues/19801
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14872 from tchaikov/wip-19770

qa/erasure-code: override min_size to 2

Reviewed-by: Sage Weil <sage@redhat.com>

qa/erasure-code: override min_size to 2

so isa(k=2,m=1) can survive with 1 down OSD.

Fixes: http://tracker.ceph.com/issues/19770
Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/rados: add optional setting of "min_size"

this setting only affects the newly created pool

Fixes: http://tracker.ceph.com/issues/19770
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14758 from xiexingguo/fix-can_mark_out-output

mon/OSDMonitor: fix output func name in can_mark_out

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14847 from smithfarm/wip-19797

tests: rados: move cephtool.yaml to new singleton/bluestore subsuite

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14849 from tchaikov/wip-19787

mon/OSDMonitor: do not alter the "created" epoch of a pg

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14161 from jcsp/wip-mgr-legacy-mds

mgr: fix metadata handling from old MDS daemons

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14706 from SUSE/wip-fix-19629

mgr: fix crash on set_config from python module with insufficient caps

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14855 from tchaikov/wip-increase-lec

mon/OSDMonitor: increase last_epoch_clean's lower bound if possible

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14859 from liewegas/wip-nonnull

librados: fix rados_pool_list when buf is null

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14860 from liupan1111/wip-fix-journal-sleep-performance

os/filestore: use new sleep strategy when io_submit gets EAGAIN.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14862 from tchaikov/wip-templatize-intarith

include/intarith: templatize ctz/clz/cbits helpers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14863 from tchaikov/wip-kill-clang-warnings

osd/OSDMap: should update input param if osd dne

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14864 from renhwztetecs/renhw-wip-mgr-key

mgr: add mgr allow * to client.admin

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14832 from dillaman/wip-rbd-mirror-deletion

rbd-mirror: image deletions should be handled by assigned instance

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14842 from cbodley/wip-rgw-admin-suite

qa/rgw: fix assertions in radosgw_admin task

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14826 from tchaikov/wip-prime-pg-temp

mon/OSDMonitor: skip prime_pg_temp if mapping is prior to osdmap

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #14818 from tchaikov/wip-rocksdb

rocksdb: sync with upstream

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14817 from TsaiJin/wip-set-boolean-variable-with-boolean-value

osd: ReplicatedBackend::prep_push() remove redundant variable assignments

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14835 from tchaikov/wip-19743

mgr: do shutdown using finisher so we can do it in the right order

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14838 from liewegas/wip-mgr-dirs

debian/ceph-base.dirs: create bootstrap-mgr dirs

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #14384 from hrchu/radosgw-admin-key

doc: rgw: Rewrite the key management

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14848 from dillaman/wip-19798

test: remove hard-coded image name from RBD metadata test

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13315 from dang/wip-build-no-nss

cmake: Allow tests to build without NSS

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14367 from mslovy/wip-fix-infinitloops

os/filestore: fix infinit loops in fiemap()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14391 from LiumxNL/wip-170407

mon/PGMonitor: fix wrongly report "pg stuck in inactive"

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14675 from ifed01/wip-bluestore-nolock

os/bluestore: eliminate some excessive stuff

Reviewed-by: Sage Weil <sage@redhat.com>

tests: rados: move cephtool.yaml to new singleton/bluestore subsuite

cephtool.yaml is bluestore-only, yet it was in singleton/ which runs against a
generalized objectstore matrix.

Fixes: http://tracker.ceph.com/issues/19797
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #14475 from liewegas/wip-message-compat-cruft

messages: remove compat cruft

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: move TransContext finalization out of osr_lock to avoid potential contention.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: get rid off Onode::flush_txn set.
It looks like flushing_count counter is enough.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

mgr: add mgr allow * to client.admin

Signed-off-by: huanwen ren ren.huanwen@zte.com.cn

common/hobject: use string::append() to avoid temp object

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/OSDMap: should update input param if osd dne

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/obj_bencher: silence divide by zero warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/test_async_driver: silence clang static analyzer warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

bluestore: use std::atomic<uint8_t> for min_alloc_size_order

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/intarith: templatize ctz/clz/cbits helpers

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/filestore: use new sleep strategy when io_submit gets EAGAIN.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

libcephfs: handle path==null to ceph_get_osd_crush_location

Signed-off-by: Sage Weil <sage@redhat.com>

librados: fix rados_pool_list when buf is null

This was the original intent, but buf==NULL wasn't documented
and only half-way implemented.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14840 from drunkard/master

doc: style fix for doc/cephfs/client-config-ref.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: typo fixes in doc/cephfs/client-config-ref.rst

fixed:
  Reference symbol, i.e. s/`/``/
  Hyperlink typo at ``client_trace``

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

Merge pull request #14302 from liewegas/wip-19452

osdc/Objecter: handle command target that goes down

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14850 from tchaikov/wip-cmake

cmake: add "container" to required boost components

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mon/OSDMonitor: increase last_epoch_clean's lower bound if possible

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14794 from joscollin/wip-warning-simplespin

test: fixing assert that creates warning: comparison between signed and unsigned integer expressions

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/rgw: fix assertions in radosgw_admin task

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mon/OSDMonitor: use emplace() to avoid creating temporary elements

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: do not alter the "created" epoch of a pg

Fixes: http://tracker.ceph.com/issues/19787
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add "container" to required boost components

it is used by our denc

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14843 from wjwithagen/wip-wjw-fix-pipe

src/msg/simple/Pipe.cc: Fix the inclusion of '}'

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: remove hard-coded image name from RBD metadata test

Fixes: http://tracker.ceph.com/issues/19798
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: provide image status updates without remotes

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: new state machine for preparing local image

This state machine will be invoked before the bootstrap state machine
and will be responsible for detecting if the local image is already
primary or if it needs to be deleted.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: remove duplicative journal helper functions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: forward initial pool image deletions to instance

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: deletions should be scheduled using global ids

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: pass pool watcher image ids by rvalue

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

src/msg/simple/Pipe.cc: Fix the inclusion of '}'

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14754 from trociny/wip-19128

rbd: import needs to sanity check auto-generated image name

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14592 from prallabh/wip-manifest

rgw: custom user data header

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14816 from cbodley/wip-rgw-test-multi-list-fix

test/rgw: fix for empty lists as default arguments

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14473 from ShiqiCooperation/master

os/bluestore: assert blob map returns success

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14491 from liewegas/wip-bluestore-deferred-rewrite

os/bluestore: rewrite deferred write handling

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14695 from dachary/wip-choosleaf-stable

crush: builder: legacy has chooseleaf_stable = 0

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14696 from dachary/wip-crush-workspace

crush: crush_init_workspace starts with struct crush_work

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14814 from neha-ojha/wip-osd-filestore-default

osd: switch filestore to default to rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14795 from tchaikov/wip-so-priority-cleanup

msg/async: return right away in NetHandler::set_priority() if not supported

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14802 from tchaikov/wip-mgr-cleanup

mgr: various cleanups

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14807 from liewegas/wip-jewel-x

ceph_test_rados_api_tier: tolerate ENOENT from 'pg scrub'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14717 from liewegas/wip-bluestore-remove-max-ops

os/bluestore: rename/fix throttle options

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14819 from tchaikov/wip-objecter-more-constness

osdc/Objecter: more constness

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14773 from viva64/sizeof_pointer

src/msg/async/AsyncConnect.cc: Use of sizeof() on a Pointer Type

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14782 from yuyuyu101/wip-rdma-perf

msg/async/rdma: add inqueue rx chunks perf counter

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14797 from wjwithagen/wip-wjw-freebsd-config-path

common/config: Add /usr/local/etc/ceph to default paths

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14801 from tchaikov/wip-crypto-cleanup

crypto: cleanup NSPR in main thread

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14805 from tchaikov/wip-mon-check-get

mon/OSDMonitor: check get()'s return value instead of bl's length

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14836 from idryomov/wip-krbd-wac-tests

qa: krbd write-after-checksum tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14803 from hrchu/rgwadmin-api-doc-usage-conf

doc: rgw: note rgw_enable_usage_log option in adminops guide

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

debian/ceph-base.dirs: create bootstrap-mgr dirs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14824 from renhwztetecs/renhw-wip-mgr-bootstrap

mgr: mkdir bootstrap-mgr

Reviewed-by: Sage Weil <sage@redhat.com>

qa: krbd write-after-checksum tests

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mgr: do shutdown using finisher so we can do it in the right order

otherwise we can shutdown a PyModules while it still being init'ed

Fixes: http://tracker.ceph.com/issues/19743
Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config_opts.h: Allow for /usr/local/etc/ceph to hold keyrings

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14831 from smithfarm/wip-stress-split

tests: fix broken links in upgrade/hammer-jewel-x/stress-split

rocksdb: sync with upstream

to avoid warning with glibc 2.25, see
https://github.com/facebook/rocksdb/pull/2208 and
https://github.com/ceph/rocksdb/pull/17

Signed-off-by: Kefu Chai <kchai@redhat.com>

build/ops: cmake: explicitly disable MSSE 4.2 if not supported

This became necessary when upstream unconditionally enabled MSSE 4.2
via https://github.com/facebook/rocksdb/commit/bc397325

Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14751 from trociny/wip-19716

test/librbd/test_notify.py: don't disable feature in slave

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests: fix broken links in upgrade/hammer-jewel-x/stress-split

Fixes: http://tracker.ceph.com/issues/19793
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #14820 from tchaikov/wip-deb-maint-fun

debian: package crypto plugin only on amd64

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mon/OSDMonitor: skip prime_pg_temp if mapping is prior to osdmap

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: mkdir bootstrap-mgr

ceph-deploy rely on bootstrap-mgr
as follows: https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/mgr.py#L37

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

src/osd/ReplicatedBackend.cc: remove redundant assignments which have been done in constructor function

Signed-off-by: Jin Cai <caijin.caij@alibaba-inc.com>

debian: package crypto plugins only on amd64

currently, only plugin based on isa-l is installed. archs other than
amd64 will not have this directory or the plugin(s) residing in it.
hence dh_install will fail when trying to copy nonexistence file/dir.

* debian/ceph-common.install: chmod +x, and only install crypto on amd64
  so dh_install can filter the install list using dh-exec
* debian/control: depends on dh-exec now. dh-exec v0.13 introduces support
  for filtering based on architecture. see dh-exec's changelog for more
  details. but trusty only offers dh-exec v0.12. so do not require ">=
  0.13) at this moment.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: os should depend on crypto_plugins not a certain plugin

Signed-off-by: Kefu Chai <kchai@redhat.com>

crypto: cleanup NSPR in main thread

quote from nspr's header file

```
 * Perform a graceful shutdown of NSPR.  PR_Cleanup() may be called by
 * the primordial thread near the end of the main() function.
```

this helps to silence some warnings from valgrind. but it does not hurt
in practice, because the process is about to die. and the freed memory
chunks are only allocated once in NSPR.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osdc/Objecter: more constness

Signed-off-by: Kefu Chai <kchai@redhat.com>