common/iso_8601.cc: Make return expression Clang compatible

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #15111 from Liuchang0812/wip-19885-mgr-help

ceph: introduce "tell x help" subcommand

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15289 from ifed01/wip-bluestore-fix-bmap-assert

os/bluestore: fix BitMapAllocator assert on out-of-bound hint value

Reviewed-by: Varada Kari <varada.kari@gmail.com>

Merge pull request #15288 from yuyuyu101/wip-20062

msg/async: remove false alert "assert"

Merge pull request #15214 from gregsfortytwo/wip-19931-snaptrim-pgref

osd: don't leak pgrefs or reservations in SnapTrimmer

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd: do not default-abort on leaked pg refs

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: shutdown our reserver_finisher earlier in the process

This finisher thread has a lot of callbacks which can hold PGRefs. Make
sure we drain them out before checking that all the PGs have finished
and have no outstanding references.

Moving this should be safe; we've already stopped the op thread et al
and the only things still running are the OSDService's objecter_finisher,
recovery_request_timer, and snap_sleep_timer (which has definitely been emptied
by the time we get here as it's synchronously cleared out on PG shutdown).

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15316 from linuxbox2/wip-limitcheck-fixes

rgw-admin: fix bucket limit check argparse, div(0)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw-admin: fix bucket limit check argparse, div(0)

1. normalize arg parsing for "bucket limit check"
1.1 s/buckets/bucket/

2. avoid dividing by num_shards when it is 0

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15304 from Linbing/modify-note-word-error

mon: Fixed typo in function comment blocks and in other comments

os/bluestore: fix BitMapAllocator assert on out-of-bound hint value

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #15256 from fangyuxiangGL/bucket-stats

rgw: display more info when using  radosgw-admin bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15221 from fangyuxiangGL/set-placement-rule

rgw: set placement rule properly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15287 from ktdreyer/doc-ceph-fs-settings

doc: describe CephFS max_file_size

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #15219 from fangyuxiangGL/extra_pool

rgw: use get_data_extra_pool() when get extra pool

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14341 from theanalyst/cleanup/rgw/str-find

rgw_common: use string::npos for the results of str.find

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14746 from wjwithagen/wip-wjw-reduce-noise-2

src/ceph.in: Use env(CEPH_DEV) to suppress noise from ceph

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15247 from liewegas/wip-upmap-save

osdmaptool: require --upmap-save before modifying input osdmap

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #15234 from trociny/wip-19832

librbd: potential read IO hang when image is flattened

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15239 from trociny/wip-19962

librbd: discard related IO should skip op if object non-existent

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15293 from linuxbox2/wip-rgwfile-buckattr

rgw_file: store bucket uxattrs on the bucket

Merge pull request #15290 from idryomov/wip-krbd-unmap-straw

qa/suites/krbd: unmap subsuite needs straw buckets

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: describe differences with "mds set" vs "fs set"

Thanks to John Spray <jspray@redhat.com> on ceph-users for this
information.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc: describe CephFS max_file_size

Add a description of max_file_size to the CephFS admin docs.

Thanks to John Spray <jspray@redhat.com> on ceph-users for this
information.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

./qa/workunits/ceph-helpers.sh: Do not trace kill_daemon

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

vstart:Suppress trivial logging info during devlopment

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

src/ceph.in: Use env(CEPH_DEV) to suppress noise from ceph

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Modify note error

Signed-off-by: linbing linbing@t2cloud.net

tool/ceph: support target mon in tell help subcommand

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

osd: Reset() the snaptrimmer on shutdown

We were failing to exit various wait states which held PGRefs. Error!

Fixes: http://tracker.ceph.com/issues/19931
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #14658 from chardan/jfw-wip-halflife_atomic_t-baragon

librados,libradosstriper,test: migrate atomic_t to std::atomic (baragon)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15227 from tchaikov/wip-20051

mon/MonClient: cancel pending commands on shutdown

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15249 from Alessandro-Barbieri/patch-2

yasm-wrapper: filter -pthread

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15268 from ifed01/wip-bluestore-false-assert

os/bluestore: fix false assert in IOContext::aio_wake

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15272 from tchaikov/wip-mon-cleanups

mon: cleanups

Reviewed-by: Sage Weil <sage@redhat.com>

tool/ceph: remove old mon support and verbose support

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

tool/ceph: validate target in tell subcommand

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

test/cephtool: convert FS_NAME to local variable

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

doc/tool: introduce "ceph tell x help" command in man

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

tool/ceph: create "tell x help" command

Fixes: http://tracker.ceph.com/issues/19885
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #15192 from badone/wip-tracker-19900-wait_auth_rotating-crash

mgr: Mark session connections down on shutdown

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15146 from tchaikov/wip-split-test-tiering

qa: split test_tiering into smaller pieces

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15156 from tchaikov/wip-ceph.in-assert-connected

ceph.in: assert(state==connected) before help_for_target()

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15224 from tchaikov/wip-denc-no-deep-copy-for-segmented-buffer

denc: add need_contiguous to denc_traits

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw_file: store bucket uxattrs on the bucket

This turns out to be the corollary of the earlier bug in removing
buckets w/uxattrs. Rather than conditionally put the attrs on a '/'
object, put them on the bucket--because that works, and will be
slightly more efficient when the check for the '/' object goes
away.

Fixes: http://tracker.ceph.com/issues/20082
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

osd: make sure we drop Backoff PGRefs on shutdown

Fixes: http://tracker.ceph.com/issues/19931
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: print out pg debug ref counts on acquire/drop

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: repair the PG_DEBUG_REFS build

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: use PGRef instead of boost::intrusive_ptr<PG> everywhere

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15062 from ukernel/wip-19912

qa/tasks/cephfs: use getattr to guarantee inode is in client cache

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15270 from ceph/wip-subman-cron-attr-tserlin

Set subman cron attributes in spec file

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #15253 from tchaikov/wip-ceph-daemon-termsize

pybind/ceph_daemon.py: fix Termsize.update

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #15279 from joscollin/wip-cleanup-rename-filejournal-object

test: Rename FileJournal object to distinguish

Set subman cron attributes in spec file

Fixes: http://tracker.ceph.com/issues/20074
Signed-off-by: Thomas Serlin <tserlin@redhat.com>

qa/suites/krbd: unmap subsuite needs straw buckets

The default tunables for luminous allow straw2 buckets, which aren't
supported by pre-single-major.yaml kernel.  The map is generated before
crush_tunables override takes effect, so set the bucket algorithm to
straw manually.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

msg/async: remove false alert "assert"

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #15200 from smithfarm/wip-rdma-devel

build/ops: rpm: move RDMA and python-prettytables build dependencies to distro-conditional section

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

test: Rename FileJournal object to distinguish

Renamed FileJournal object for easier searching and identifying with in test_filejournal.cc

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15258 from big-henry/wip-mds-misc-fix

mds: fix client ID truncation

Merge pull request #15235 from smithfarm/wip-doc-empowers

doc: fix factual inaccuracy in doc/architecture.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15275 from ktdreyer/wip-17396

docs: document "osd recovery max single start" setting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15276 from liewegas/wip-ext4-again

qa/suites/rados/singleton-nomsgr: fix syntax

Merge pull request #15252 from liewegas/wip-cleanup-tell

qa/tasks/ceph_manager: 'ceph $service tell ...' is obsolete

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15250 from neha-ojha/wip-19989

osd/OSDMap.cc: check if osd is out in subtree_type_is_down

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14944 from yuyuyu101/wip-msgr-throttle-ops

ceph_osd: remove client message cap limit

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14655 from chardan/jfw-wip-halflife_atomic_t-gojira

test: migrate atomic_t to std::atomic

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11966 from aclamk/wip-linear-zeros-crc

common: Improved CRC calculation for zero buffers

qa/suites/rados/singleton-nomsgr/health...: whitelist wrongly marked me down

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/singleton-nomsgr: fix syntax

This parsed out as

  tasks:
  - install: null
  - ceph:
      conf:
        osd: osd max object name len = 400 osd max object namespace len = 64
  - workunit:
      clients:
        all:
        - rados/test_health_warnings.sh

which is clearly not correct.

Signed-off-by: Sage Weil <sage@redhat.com>

docs: document "osd recovery max single start" setting

Fixes: http://tracker.ceph.com/issues/17396
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #15028 from pritha-srivastava/wip-rgw-bucket-policy-test

rgw: Adding code to create tenanted user for s3 bucket policy tests.

Merge pull request #15070 from ceph/wip-turn-off-fcgi-tserlin

rgw: Turn off fcgi as a frontend

mon/PGMonitor: assert(format_version == 1)

we have been PGMonitor's formart_version has been 1 since v0.67
(dumpling). so let's drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon,mgr: std::move(osd_stat) when possible

osd_stat_t could be a relatively large struct, so use std::move() when
possible

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15175 from cloudbau/update-rgw-docs

doc: Update some RGW documentation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mon/MonClient: cancel pending commands on shutdown

Fixes: http://tracker.ceph.com/issues/20051
Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonClient: drop unused param

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15213 from jcsp/wip-20040

mon: fix accesing pending_fsmap from peon

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #15130 from ukernel/wip-19946

mds: fix CDir::merge() for mds_debug_auth_pins

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15105 from ukernel/wip-19892

qa/cephfs: disable mds_bal_frag for TestStrays.test_purge_queue_op_rate

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15122 from batrick/test-fragment-error

qa: fix float parse error in test_fragment

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15106 from ukernel/wip-export-pin-cleanup

ceph: simplify CInode::maybe_export_pin()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #15129 from ukernel/wip-mds-standby-for-rank

mon/MDSMonitor: respect mds_standby_for_rank config

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #15131 from ukernel/wip-19955

mds: limit client writable range increment

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15198 from renhwztetecs/renhw-wip-client-inode

client/inode: fix the dump type of Inode::dump()

Reviewed-by: Yan, Zheng <zyan@redhat.com>

os/bluestore: fix false assertion

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: wrap IOContext::aio_wake with additional stuff to remove copy-paste for its use

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #15264 from xiexingguo/wip-kill-typo

os/bluestore/KernelDevice: fix comments

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15266 from ztczll/master

test:Check make_writeable() return value

test/:Check make_writeable return value

Signed-off-by: zhanglei <243290414@qq.com>

os/bluestore/KernelDevice: fix comments

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15206 from ifed01/wip-fix-rbd-bench

rbd/bench: fix write gaps when doing sequential writes with io-threads > 1

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: display more info when use bucket stats

display the zonegroup uuid and placement rule of bucket, when using radosgw-admin bucket stats

Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com

pybind/ceph_daemon.py: fix Termsize.update

it's a regression introduced by 84f9c12

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: Mark session connections down on shutdown

Also assert in OSD::ms_get_authorizer since it is dangerous to proceed
if we are shutting down.

Fixes: http://tracker.ceph.com/issues/19900
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

denc: add need_contiguous to denc_traits

so the decode() will not try to copy the bufferlist for a continous one
if the bufferlist is segmented *and* its length is greater than
CEPH_PAGE_SIZE (4K) if the decoded object does not "need_contiguous" per
its denc_traits<>.

copying a memory chunk could be expensive if the decoded bufferlist is
huge, so we should try to avoid this. this could happen when we read the
buffer from bluestore.

and drop the partial specialization for denc() which tries to differentiate
traits::featured and !traits::featured, it does not matter to
decode() if the type supports feature or not. the encode() does.

also use denc() in interval_set<>::decode(). unlike encode(), decode(),
encode_nohead() or decode_nohead(), denc() is not part of our legacy
dencoder, so i think it's fine and encouraged to use it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15238 from xiexingguo/wip-fix-blue-mkfs

os/bluestore: write "mkfs_done" into disk only if we pass fsck() tests

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15228 from chardan/jfw-wip-halflife_atomic_t-filestore

filestore: migrate atomic_t to std::atomic<>

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tasks/ceph_manager: 'ceph $service tell ...' is obsolete

This died forever ago; no need for the fallback here.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9307 from cxwshawn/wip-tc7-fix

crush: optimize header file dependency

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>