cmake: link radosgw with boost libraries

Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: delete excidentail reintroduced line.

- This target got reintroduced by accident after the creation
  of ceph-common in commit:
    https://github.com/ceph/ceph/commit/046b2bde6d93992c36d4943c4f7859993f034ca

  Detection was trigger by refering to libresolv, and results in a
  linking error.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

build/ops: deb: no rgw Beast frontend on s390x

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: rpm: no rgw Beast frontend on s390x

Since the Beast frontend uses boost::context which is not supported on
s390x.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

build/ops: rpm: reverse s390 bcond conditional block

I'm not a fan of "if NOT x - then - else" blocks.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: build boost::context and coroutine only with rgw Beast frontend

boost::context is currently (1.63) unsupported for s390x and anyway
it makes sense to conditionalize Boost components so they are only
built with the Ceph components that need them (like is already being
done for mgr).

Fixes: http://tracker.ceph.com/issues/20048
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Tim Serong <tserong@suse.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15345 from badone/wip-tracker-19900-wait_auth_rotating-crash-2

osd: Return early on shutdown

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: Return early on shutdown

eb5c02d was missing a return statement, rectify that.

Fixes: http://tracker.ceph.com/issues/19900
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #15336 from wjwithagen/wjw-wip-iso_8601

common/iso_8601.cc: Make return expression Clang compatible

Reviewed-By: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15055 from mogeb/wip-with-instrument-functions

cmake: Add -finstrument-functions flag to OSD code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15319 from liewegas/wip-upmap

mon,osd/OSDMap: a couple pg-upmap fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15292 from liewegas/wip-map-cache

osd: reduce map cache size

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15296 from liewegas/wip-fix-at-end

qa/tasks/repair_test: unset flags we set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15297 from liewegas/wip-19595

mgr/MgrStandby: prevent use-after-free on just-shut-down Mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14284 from tchaikov/wip-19439

qa/tasks/rebuild_mondb: grant "mgr:allow *" to client.admin

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15284 from majianpeng/bluestore-batch-throotle

os/bluestore: batch throttle

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15302 from xiexingguo/wip-fix-reg-leak

common/ceph_context: fix leak of registered commands on exit

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15320 from liewegas/wip-peon-mapping

mon/OSDMonitor: cancel mapping job from update_from_paxos

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/rebuild_mondb: update to address ceph-mgr changes

- revive ceph-mgr after updating the keyring cap
- grant "mgr:allow *" to client.admin
- minor refactors

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/iso_8601.cc: Make return expression Clang compatible

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #15111 from Liuchang0812/wip-19885-mgr-help

ceph: introduce "tell x help" subcommand

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15289 from ifed01/wip-bluestore-fix-bmap-assert

os/bluestore: fix BitMapAllocator assert on out-of-bound hint value

Reviewed-by: Varada Kari <varada.kari@gmail.com>

Merge pull request #15288 from yuyuyu101/wip-20062

msg/async: remove false alert "assert"

Merge pull request #15214 from gregsfortytwo/wip-19931-snaptrim-pgref

osd: don't leak pgrefs or reservations in SnapTrimmer

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd: do not default-abort on leaked pg refs

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: shutdown our reserver_finisher earlier in the process

This finisher thread has a lot of callbacks which can hold PGRefs. Make
sure we drain them out before checking that all the PGs have finished
and have no outstanding references.

Moving this should be safe; we've already stopped the op thread et al
and the only things still running are the OSDService's objecter_finisher,
recovery_request_timer, and snap_sleep_timer (which has definitely been emptied
by the time we get here as it's synchronously cleared out on PG shutdown).

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15316 from linuxbox2/wip-limitcheck-fixes

rgw-admin: fix bucket limit check argparse, div(0)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw-admin: fix bucket limit check argparse, div(0)

1. normalize arg parsing for "bucket limit check"
1.1 s/buckets/bucket/

2. avoid dividing by num_shards when it is 0

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

mon/OSDMonitor: cancel mapping job from update_from_paxos

On the leader we cancel the mapping job in encode_pending.  On a peon,
we don't cancel it at all!  It is surprising this didn't already cause
problems, but with the PGtempMap is pretty reliably crashes with a
largish map.

Fixes: http://tracker.ceph.com/issues/20067
Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: some better pg-upmap debug output

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: allow upmap records for deleted pools to be removed

Otherwise the offline osdmaptool --upmap-cleanup can't do its thing.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15304 from Linbing/modify-note-word-error

mon: Fixed typo in function comment blocks and in other comments

os/bluestore: fix BitMapAllocator assert on out-of-bound hint value

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #15256 from fangyuxiangGL/bucket-stats

rgw: display more info when using  radosgw-admin bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15221 from fangyuxiangGL/set-placement-rule

rgw: set placement rule properly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15287 from ktdreyer/doc-ceph-fs-settings

doc: describe CephFS max_file_size

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #15219 from fangyuxiangGL/extra_pool

rgw: use get_data_extra_pool() when get extra pool

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14341 from theanalyst/cleanup/rgw/str-find

rgw_common: use string::npos for the results of str.find

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14746 from wjwithagen/wip-wjw-reduce-noise-2

src/ceph.in: Use env(CEPH_DEV) to suppress noise from ceph

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15247 from liewegas/wip-upmap-save

osdmaptool: require --upmap-save before modifying input osdmap

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #15234 from trociny/wip-19832

librbd: potential read IO hang when image is flattened

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15239 from trociny/wip-19962

librbd: discard related IO should skip op if object non-existent

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15293 from linuxbox2/wip-rgwfile-buckattr

rgw_file: store bucket uxattrs on the bucket

Merge pull request #15290 from idryomov/wip-krbd-unmap-straw

qa/suites/krbd: unmap subsuite needs straw buckets

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: describe differences with "mds set" vs "fs set"

Thanks to John Spray <jspray@redhat.com> on ceph-users for this
information.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc: describe CephFS max_file_size

Add a description of max_file_size to the CephFS admin docs.

Thanks to John Spray <jspray@redhat.com> on ceph-users for this
information.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

os/bluestore/BlueStore: Only call once notify_all.

All txcs of deferred_running have the same osr, so only call once
notify_call.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

os/bluestore/BlueStore: avoid iterating to get ios/cost in _kv_sync_thread.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

os/bluestore/BlueStore: batch handle Throttle.put.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

./qa/workunits/ceph-helpers.sh: Do not trace kill_daemon

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

vstart:Suppress trivial logging info during devlopment

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

src/ceph.in: Use env(CEPH_DEV) to suppress noise from ceph

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Modify note error

Signed-off-by: linbing linbing@t2cloud.net

tool/ceph: support target mon in tell help subcommand

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

common/ceph_context: fix leak of registered commands on exit

By unregister them properly.
Also rearrange commands order to keep pace with register process.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: Reset() the snaptrimmer on shutdown

We were failing to exit various wait states which held PGRefs. Error!

Fixes: http://tracker.ceph.com/issues/19931
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #14658 from chardan/jfw-wip-halflife_atomic_t-baragon

librados,libradosstriper,test: migrate atomic_t to std::atomic (baragon)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15227 from tchaikov/wip-20051

mon/MonClient: cancel pending commands on shutdown

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15249 from Alessandro-Barbieri/patch-2

yasm-wrapper: filter -pthread

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15268 from ifed01/wip-bluestore-false-assert

os/bluestore: fix false assert in IOContext::aio_wake

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15272 from tchaikov/wip-mon-cleanups

mon: cleanups

Reviewed-by: Sage Weil <sage@redhat.com>

osd: reduce size of osdmap cache, messages

On large clusters, these large caches can be problematic (as maps get big).
We've seen good results with extremely small caches (10s of maps).  Make
a more modest reduction.

Signed-off-by: Sage Weil <sage@redhat.com>

test/osd/osd-config.sh: fix test to isolate cases

The third test (increasing osd_map_max_advance)
was triggering a warning from the 4th case (which
it didn't before).

Signed-off-by: Sage Weil <sage@redhat.com>

osd: fadvise hint WILL_NEED when reading encoded osdmaps

This way will ensure we cache data for recent osdmaps if we need to for
the benefit of laggy clients... even if (in bluestore's case)
bluestore_default_buffered_reads = false (it's true by default).  This
should mitigate any tail latency/work even if the osdmap cache size is too
small.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: instrument osdmap bl cache hits and misses

Add perfcounters so we can see whether we are missing osdmaps in the
cache.  This will let us tell whether, given a workload or environment,
our osdmap cache might be too small.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: do not bother with misdirected op diagnosis by default

We enable osd_debug_misdirected_ops in QA, but this is wasted effort on
a production cluster.  In particular, it means that a idle client that
sends an op to the wrong OSD based on an old map will require that OSD to
load that old map into memory to decide whether to print a warning... all
on the off-chance that the client is buggy.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: remove osd_enxio_on_misdirected_op option

There is no good reason anyone would want this turned on.

Introduced 923e7f5ce5ed437af15e178299a61029ff48e4a2 (post-kraken), but
backported to kraken and jewel (10.2.6).

Signed-off-by: Sage Weil <sage@redhat.com>

tool/ceph: remove old mon support and verbose support

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

tool/ceph: validate target in tell subcommand

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

test/cephtool: convert FS_NAME to local variable

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

doc/tool: introduce "ceph tell x help" command in man

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

tool/ceph: create "tell x help" command

Fixes: http://tracker.ceph.com/issues/19885
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #15192 from badone/wip-tracker-19900-wait_auth_rotating-crash

mgr: Mark session connections down on shutdown

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/MgrStandby: prevent use-after-free on just-shut-down Mgr

Fixes: http://tracker.ceph.com/issues/19595
Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/repair_test: unset flags we set

In particular, noscrub and nodeepscrub leave a health
warning, which prevents shutdown with at-end.yaml.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15146 from tchaikov/wip-split-test-tiering

qa: split test_tiering into smaller pieces

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15156 from tchaikov/wip-ceph.in-assert-connected

ceph.in: assert(state==connected) before help_for_target()

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15224 from tchaikov/wip-denc-no-deep-copy-for-segmented-buffer

denc: add need_contiguous to denc_traits

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw_file: store bucket uxattrs on the bucket

This turns out to be the corollary of the earlier bug in removing
buckets w/uxattrs. Rather than conditionally put the attrs on a '/'
object, put them on the bucket--because that works, and will be
slightly more efficient when the check for the '/' object goes
away.

Fixes: http://tracker.ceph.com/issues/20082
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

tracing: remove -pie if using function instrumentation

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

tracing: add documentation for function instrumentation

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

tracing: add -finstrument-functions for OSD

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

tracing: add cyg_profile tracepoint provider

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

cmake: configuration for OSD function instrumentation

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

osd: make sure we drop Backoff PGRefs on shutdown

Fixes: http://tracker.ceph.com/issues/19931
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: print out pg debug ref counts on acquire/drop

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: repair the PG_DEBUG_REFS build

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: use PGRef instead of boost::intrusive_ptr<PG> everywhere

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15062 from ukernel/wip-19912

qa/tasks/cephfs: use getattr to guarantee inode is in client cache

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15270 from ceph/wip-subman-cron-attr-tserlin

Set subman cron attributes in spec file

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #15253 from tchaikov/wip-ceph-daemon-termsize

pybind/ceph_daemon.py: fix Termsize.update

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #15279 from joscollin/wip-cleanup-rename-filejournal-object

test: Rename FileJournal object to distinguish

Set subman cron attributes in spec file

Fixes: http://tracker.ceph.com/issues/20074
Signed-off-by: Thomas Serlin <tserlin@redhat.com>

qa/suites/krbd: unmap subsuite needs straw buckets

The default tunables for luminous allow straw2 buckets, which aren't
supported by pre-single-major.yaml kernel.  The map is generated before
crush_tunables override takes effect, so set the bucket algorithm to
straw manually.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

msg/async: remove false alert "assert"

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #15200 from smithfarm/wip-rdma-devel

build/ops: rpm: move RDMA and python-prettytables build dependencies to distro-conditional section

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

test: Rename FileJournal object to distinguish

Renamed FileJournal object for easier searching and identifying with in test_filejournal.cc

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15258 from big-henry/wip-mds-misc-fix

mds: fix client ID truncation

Merge pull request #15235 from smithfarm/wip-doc-empowers

doc: fix factual inaccuracy in doc/architecture.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15275 from ktdreyer/wip-17396

docs: document "osd recovery max single start" setting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>