librbd,librados: do not include stdbool.h in C++ headers

stdbool.h is offered to provide C++ keywords for C source file.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19642 from xiexingguo/wip-pending-new-state

osd/OSDMap.h: toss osd out if it has no more pending states

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19901 from vico-research-and-consulting/master

mgr/zabbix: monitoring template improvements

Reviewed-by: Wido den Hollander <wido@widodh.nl>

Merge pull request #18833 from yuyuyu101/wip-async-perf

msg/async/AsyncConnection: combine multi alloc into one

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18906 from ovh/bp-buffer-crc-improvements

common/buffer: switch crc cache to single pair instead of map

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19547 from tchaikov/wip-22344

ceph.in: print all matched commands if arg missing

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19832 from shinobu-x/min_max_filestore

os: FileStore, Using stl min | max, MIN | MAX macros instead

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19874 from myoungwon/wip-fix-22641

osd: fix a valgrind issue (conditional jump depends on uninitialized value)

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/zabbix: monitoring template improvements

The changes need some review because i am currently not able to test this in a environment.
- added a host screen which displays all graphs in a convenient way
- replaced AVG by MAX Function in Graphs where maximum values are of
  high interest from operational view
- changed graph type to stacked where combined behavior is of interest
  (i.e. bandwidth and storage consumption)

Signed-off-by: Marc Schoechlin <ms@256bit.org>

Merge pull request #19615 from qrGitHub/wip-rgw-log-adjustment2

rgw: format logs in file rgw_lc.cc

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19740 from fangyuxiangGL/invalid-compress

rgw: keep compression type consistent between parts of s3 Multipart

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #19753 from qrGitHub/wip-rgw-fix-cephdencoder

rgw: format rgw_bucket_dir_header in ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19814 from qrGitHub/wip-rgw-optimize-function3

rgw: optimize function parse_raw_oid

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19817 from qrGitHub/wip-rgw-useless-line2

rgw: drop useless lines

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19882 from adamwg/awg/fix-config-docs

doc: Update config file search paths to reflect reality

Merge pull request #19441 from wumingqiao/wip-pg-on-flushed

osd/PG: handle flushed event directly

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19813 from xxhdx1985126/wip_xxh_low_list_size

common: compute SimpleLRU's size with contents.size() instead of lru.size()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19812 from shinobu-x/osd_get_next_up_osd_after

osd: Remove redundant local variable declaration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19805 from tchaikov/wip-perf-counters-cleanup

common/perf_counters: remove unused parameter

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19759 from ifed01/wip-ifed-throttle64

common/throttle: start using 64-bit values

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19529 from thmour/master

ceph-volume: Require lvm2, move to osd package

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19520 from ovh/bp-snap-trimq-visibility

osd, pg, mgr: make snap trim queue problems visible

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19374 from symptog/master

mgr/influx: Add InfluxDB SSL Option

Reviewed-by: Wido den Hollander <wido@42on.com>

Merge pull request #19875 from ceph/wip-rm22307

ceph-volume: adds success messages for lvm prepare/activate/create

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #19393 from tchaikov/wip-no-more-gcc-4.8

rgw,common,rbd: s/boost::regex/std::regex/

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #19622 from majianpeng/bluestore-check-obj-max-siz

os/bluestore: using macro OBJECT_MAX_SIZE to check osd_max_object_size.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19707 from shinobu-x/pick_address_a

common/pick_address.cc: Cleanup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19796 from LiumxNL/fix-ooo-caused-con-reset

osd: fix out of order caused by letting old msg from down osd be processed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19820 from yaozongyou/silence-maybe-uninitialized-warning

osd: silence maybe-uninitialized false positives

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19864 from rzarzynski/wip-bs-no-blobrefing-in-do_read

os/bluestore: avoid unneeded BlobRefing in _do_read().

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #19866 from liewegas/wip-bluestore-error-injection

os/bluestore: fix data read error injection in bluestore

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #19869 from liewegas/wip-log-desc

common/options: long description for log_stderr_prefix

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge PR #18424 into master

* refs/pull/18424/head:
mds: create snaprealm for quota enabled inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19572 into master

* refs/pull/19572/head:
client: add log when we got unexpected flush ack

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19672 into master

* refs/pull/19672/head:
ceph-fuse: ::rmdir() uses a deleted memory structure of dentry leads a core

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19693 into master

* refs/pull/19693/head:
mds: use session.get_client()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19696 into master

* refs/pull/19696/head:
client: avoid adding checked inode to the delayed_caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19702 into master

* refs/pull/19702/head:
client: avoid remove same cap twice

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19703 into master

* refs/pull/19703/head:
client: make mark_caps_clean and mark_caps_dirty as member function to Inode
client: fix dirty caps might never be flushed

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19723 into master

* refs/pull/19723/head:
mds: clean up log's function name for CDir.cc and Beacon.cc
mds: clean up log's function name for CDentry.cc
mds: clean up log's function name for CInode.cc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19730 into master

* refs/pull/19730/head:
mds: Using stl min | max, MIN | MAX macros instead

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19762 into master

* refs/pull/19762/head:
mds: fix dump last_sent

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19767 into master

* refs/pull/19767/head:
client: enhance log for _closed_mds_session

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: Update config file search paths to reflect reality

The config file paths listed in the docs don't match the config file
paths actually searched by ceph (see `src/common/config.cc`). Update the
docs to reflect the current reality.

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>

ceph-volume: adds success message to ceph-volume simple activate

This used to only show the message if systemd was not being used, now
it will show this always if the command is successful.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18788 from amitkumar50/cov-mon-1

mon: NULL check of logger before use

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19054 from cbodley/wip-librados-asio

librados: add async interfaces for use with Networking TS

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #19852 from yaozongyou/rgw-remove-unused-function

rgw: remove unused function get_system_obj_attrs
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16383 from mikulely/1-ban-change-swift-obj-placement

rgw: drop the unnecessary handling of Swift's X-Storage-Policy on objects

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #17683 from shashalu/fix-lc-xml

rgw: RGWPutLC return ERR_MALFORMED_XML  when missing <Rule> tag in lifecycle.xml

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #17873 from amitkumar50/cov-rgw-8

rgw: Returning when dst_ioctx.operate() returns error

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18861 from gaosibei/wip-fix-null-pointer

rgw: fix null pointer crush

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #19061 from Leeshine/wip-rgw-bug-21583

rgw: fix the bug of radowgw-admin zonegroup set requires realm

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #19854 from fangyuxiangGL/doc-sync-err

rgw: add radosgw-admin sync error trim to trim sync error log

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd: fix a vargrind issue

Fixes: http://tracker.ceph.com/issues/22641
Signed-off-by: Myoungwon Oh <omwmw@sk.com>

ceph-volume: add success message for ceph-volume lvm create

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: add success message for ceph-volume lvm activate

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: add success message for ceph-volume lvm prepare

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #19809 from liu-chunmei/fix_dpdk_mzname_setting_issue

osd: fix dpdk memzon mz_name setting issue

Reviewed-By: Kefu Chai <kchai@redhat.com>

librados: add async interfaces for use with Networking TS

Defines asynchronous librados operations that satisfy all of the
"Requirements on asynchronous operations" imposed by the C++ Networking
TS [1] in section 13.2.7. These operations are implemented in terms of
boost::asio, but the interfaces themselves are free of boost types -
this makes the transition to std::net trivial when it's available.

These interfaces conform to the Extensible Asynchronous Model [2] that
originated in boost::asio. This model allows the last 'handler' argument
to either be a callback that gets the result, a coroutine yield_context
that will suspend until completion, or a 'use_future' tag to request the
result in a std::future (see the unit tests for examples of each). The
'Extensible' part also enables further integration with new frameworks.

For now, only async_read(), async_write(), and the read/write variants
of async_operate() are provided.

[1] Working Draft, C++ Extensions for Networking
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4711.pdf
[2] "Library Foundations for Asynchronous Operations"
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19616 from ifed01/wip-ifed-revert-extents

osd: get rid off extent map in object_info

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19764 from ifed01/wip-ifed-storetest-cempty

objectstore/store_test: fix lack of flush prior to collection_empty()…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19841 from rzarzynski/wip-bs-dump-onode-by-ref

os/bluestore: _dump_onode() don't prolongate Onode anymore.

Reviewed-by: Sage Weil <sage@redhat.com>

common/options: long description for log_stderr_prefix

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: using macro OBJECT_MAX_SIZE to check object max size.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

os/bluestore: fix data read error injection in bluestore

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: avoid unneeded BlobRefing in _do_read().

The hottest place in the terms of CPU cycles according to `perf annotate`:
```
       |          intrusive_ptr(intrusive_ptr const & rhs): px( rhs.px )
  0,04 | 365:   mov    0x28(%rax),%r12
       |          {
       |              if( px != 0 ) intrusive_ptr_add_ref( px );
  0,41 |        test   %r12,%r12
       |      ↓ je     37c
       |      _ZNSt13__atomic_baseIiEppEv():
  0,04 |        lock   addl   $0x1,(%r12)
 19,75 |        mov    0xa8(%rsp),%rax
       |      _ZN9BlueStore8_do_readEPNS_10CollectionEN5boost13intrusive_ptrINS_5OnodeEEEmmRN4ceph6buffer4listEj():
       |          }
       |          BlobRef bptr = lp->blob;
       |          unsigned l_off = pos - lp->logical_offset
       | 37c:   mov    0xa0(%rsp),%rdx
       |        sub    0x18(%rax),%edx
       |          unsigned b_off = l_off + lp->blob_offset;
  0,08 |        mov    0x1c(%rax),%ecx
```

With the patch applied:

```
       |          T * operator->() const
       |          {
       |              BOOST_ASSERT( px != 0 );
       |              return px;
       |        mov    0x28(%rcx),%rax
       |        mov    0x8(%rax),%rax
       |      _ZN9BlueStore8_do_readEPNS_10CollectionEN5boost13intrusive_ptrINS
       |          unsigned b_len = std::min(left, lp->length - l_off);
       |
       |          ready_regions_t cache_res;
       |          interval_set<uint32_t> cache_interval;
       |          bptr->shared_blob->bc.read(
 17,78 |        lea    0x18(%rax),%rdi
       |      _ZNK5boost13intrusive_ptrIN9BlueStore10CollectionEEcvbEv():
       |      #if !defined( BOOST_NO_CXX11_EXPLICIT_CONVERSION_OPERATORS ) && !
       |          && !(defined(__SUNPRO_CC) && BOOST_WORKAROUND(__SUNPRO_CC, <=
       |
       |          explicit operator bool () const BOOST_NOEXCEPT
       |          {
       |              return px != 0;
  0,07 |        mov    0x8(%rax),%rax
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: do not build/install boost::regex

we now use std::regex now, since we've dropped the support of GCC 4.8

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon: s/boost::regex/std::regex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: s/boost::regex/std::regex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

common: s/boost::regex/std::regex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: s/boost::regex/std::regex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

Revert "rgw: Support certain archaic and antiquated distributions"

This reverts commit af8d9bdf07ab97b9d9387f68ba0fedccce803073.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/{rbd_nbd,rbd}: s/boost::regex/std::regex/

* src/tools/rbd/*: s/boost::regex/std::regex/
* */CMakeLists.txt: do not link against Boost_REGEX_LIBRARY
    anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: add radosgw-admin sync error trim to trim sync error log

Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com

Merge PR #19791 into master

* refs/pull/19791/head:
doc: misc fixes
doc: fix heading

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: misc fixes

misc fixes for best-practices.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19662 from tchaikov/wip-denc-cxx-17

denc: use constexpr-if to replace some SFINAE impls

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

rgw: remove unused function get_system_obj_attrs

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

denc: use constexpr-if to replace some SFINAE impls

more readable this way IMO.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #19807 into master

* refs/pull/19807/head:
mds: warn for mds_cache_memory_limit bug in doc

Reviewed-by: Zheng Yan <zyan@redhat.com>

mds: warn for mds_cache_memory_limit bug in doc

Fixes: http://tracker.ceph.com/issues/22599
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19657 from adamemerson/wip-hypermodern

The Day Has Come!

Update language standard version to C++17 and remove backported standard library features.

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #19823 from trociny/wip-ggate-list-xml

rbd-ggate: make list command produce valid xml format output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19846 from fangyuxiangGL/compress-info

rgw: make radosgw object stat RGW_ATTR_COMPRESSION dump readable

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19732 from shinobu-x/using_find_primarylogpg

osd: Making use of find to reduce computational complexity

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19765 from liewegas/wip-force-backfill-cleanups

qa: tolerate failure to force backfill

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19786 from dillaman/wip-osd-class-cap

osd: support class method whitelisting within caps

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

{common,osd,mds,rbd}: Remove unused captures

If we aren't referencing a variable, don't capture it. (This makes the
lambdas smaller, too, modulo Sufficiently Smart Compiler.)

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

dout: Remove VLA sanity check

This doesn't seem to be able to work for C++17 under both GCC and
Clang. Maybe there's a better way to do it. I might try routing
that argument through a constexpr function in the macro later.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Remove backport_std.h entirely

And use the experimental ostream joiner provided with the library.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Remove backported C++17 functions

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Remove backported typedefs

Since they're no longer needed.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

build: Bump language to C++17

Fix up all of the fallout from that.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

osd: fix dpdk memzon mz_name setting issue

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

objectstore/store_test: fix lack of flush prior to collection_empty() call

Fixes: http://tracker.ceph.com/issues/22409
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

qa/standalone/scrub/osd-scrub-repair.sh: remove extents flag from object_info_t

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

osd: get rid off extent map in object_info

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #19475 from ZVampirEM77/wip-em-copyobj-ret

rgw: add xml output header in RGWCopyObj_ObjStore_S3 response msg

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: scienceluo <luo.kexue@zte.com.cn>

Merge pull request #19824 from qrGitHub/wip-rgw-useless-line3

rgw: drop useless type conversion