client: fix cap revoke race

If caps are been revoking by the auth MDS, don't consider them as
issued even they are still issued by non-auth MDS. The non-auth
MDS should also be revoking/exporting these caps, the client just
hasn't received the cap revoke/export message.

The race I encountered is: When caps are exporting to new MDS, the
client receives cap import message and cap revoke message from the
new MDS, then receives cap export message from the old MDS. When
the client receives cap revoke message from the new MDS, the revoking
caps are still issued by the old MDS, so the client does nothing.
Later when the cap export message is received, the client removes
the caps issued by the old MDS. (Another way to fix the race is
calling ceph_check_caps() in handle_cap_export())

copid from kernel commit b1530f57

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

client: release revoking Fc after invalidate cache

this issue is likely revealed by commit 7bc1fa04
"client: avoid adding checked inode to the delayed_caps"

Fixes: http://tracker.ceph.com/issues/22652
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #19642 from xiexingguo/wip-pending-new-state

osd/OSDMap.h: toss osd out if it has no more pending states

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19901 from vico-research-and-consulting/master

mgr/zabbix: monitoring template improvements

Reviewed-by: Wido den Hollander <wido@widodh.nl>

Merge pull request #18833 from yuyuyu101/wip-async-perf

msg/async/AsyncConnection: combine multi alloc into one

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18906 from ovh/bp-buffer-crc-improvements

common/buffer: switch crc cache to single pair instead of map

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19547 from tchaikov/wip-22344

ceph.in: print all matched commands if arg missing

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19832 from shinobu-x/min_max_filestore

os: FileStore, Using stl min | max, MIN | MAX macros instead

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19874 from myoungwon/wip-fix-22641

osd: fix a valgrind issue (conditional jump depends on uninitialized value)

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/zabbix: monitoring template improvements

The changes need some review because i am currently not able to test this in a environment.
- added a host screen which displays all graphs in a convenient way
- replaced AVG by MAX Function in Graphs where maximum values are of
  high interest from operational view
- changed graph type to stacked where combined behavior is of interest
  (i.e. bandwidth and storage consumption)

Signed-off-by: Marc Schoechlin <ms@256bit.org>

Merge pull request #19615 from qrGitHub/wip-rgw-log-adjustment2

rgw: format logs in file rgw_lc.cc

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19740 from fangyuxiangGL/invalid-compress

rgw: keep compression type consistent between parts of s3 Multipart

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #19753 from qrGitHub/wip-rgw-fix-cephdencoder

rgw: format rgw_bucket_dir_header in ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19814 from qrGitHub/wip-rgw-optimize-function3

rgw: optimize function parse_raw_oid

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19817 from qrGitHub/wip-rgw-useless-line2

rgw: drop useless lines

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19882 from adamwg/awg/fix-config-docs

doc: Update config file search paths to reflect reality

Merge pull request #19441 from wumingqiao/wip-pg-on-flushed

osd/PG: handle flushed event directly

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19813 from xxhdx1985126/wip_xxh_low_list_size

common: compute SimpleLRU's size with contents.size() instead of lru.size()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19812 from shinobu-x/osd_get_next_up_osd_after

osd: Remove redundant local variable declaration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19805 from tchaikov/wip-perf-counters-cleanup

common/perf_counters: remove unused parameter

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19759 from ifed01/wip-ifed-throttle64

common/throttle: start using 64-bit values

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19529 from thmour/master

ceph-volume: Require lvm2, move to osd package

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19520 from ovh/bp-snap-trimq-visibility

osd, pg, mgr: make snap trim queue problems visible

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19374 from symptog/master

mgr/influx: Add InfluxDB SSL Option

Reviewed-by: Wido den Hollander <wido@42on.com>

Merge pull request #19875 from ceph/wip-rm22307

ceph-volume: adds success messages for lvm prepare/activate/create

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #19393 from tchaikov/wip-no-more-gcc-4.8

rgw,common,rbd: s/boost::regex/std::regex/

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #19622 from majianpeng/bluestore-check-obj-max-siz

os/bluestore: using macro OBJECT_MAX_SIZE to check osd_max_object_size.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19707 from shinobu-x/pick_address_a

common/pick_address.cc: Cleanup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19796 from LiumxNL/fix-ooo-caused-con-reset

osd: fix out of order caused by letting old msg from down osd be processed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19820 from yaozongyou/silence-maybe-uninitialized-warning

osd: silence maybe-uninitialized false positives

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19864 from rzarzynski/wip-bs-no-blobrefing-in-do_read

os/bluestore: avoid unneeded BlobRefing in _do_read().

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #19866 from liewegas/wip-bluestore-error-injection

os/bluestore: fix data read error injection in bluestore

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #19869 from liewegas/wip-log-desc

common/options: long description for log_stderr_prefix

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge PR #18424 into master

* refs/pull/18424/head:
mds: create snaprealm for quota enabled inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19572 into master

* refs/pull/19572/head:
client: add log when we got unexpected flush ack

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19672 into master

* refs/pull/19672/head:
ceph-fuse: ::rmdir() uses a deleted memory structure of dentry leads a core

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19693 into master

* refs/pull/19693/head:
mds: use session.get_client()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19696 into master

* refs/pull/19696/head:
client: avoid adding checked inode to the delayed_caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19702 into master

* refs/pull/19702/head:
client: avoid remove same cap twice

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19703 into master

* refs/pull/19703/head:
client: make mark_caps_clean and mark_caps_dirty as member function to Inode
client: fix dirty caps might never be flushed

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19723 into master

* refs/pull/19723/head:
mds: clean up log's function name for CDir.cc and Beacon.cc
mds: clean up log's function name for CDentry.cc
mds: clean up log's function name for CInode.cc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19730 into master

* refs/pull/19730/head:
mds: Using stl min | max, MIN | MAX macros instead

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19762 into master

* refs/pull/19762/head:
mds: fix dump last_sent

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19767 into master

* refs/pull/19767/head:
client: enhance log for _closed_mds_session

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: Update config file search paths to reflect reality

The config file paths listed in the docs don't match the config file
paths actually searched by ceph (see `src/common/config.cc`). Update the
docs to reflect the current reality.

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>

ceph-volume: adds success message to ceph-volume simple activate

This used to only show the message if systemd was not being used, now
it will show this always if the command is successful.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18788 from amitkumar50/cov-mon-1

mon: NULL check of logger before use

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19054 from cbodley/wip-librados-asio

librados: add async interfaces for use with Networking TS

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #19852 from yaozongyou/rgw-remove-unused-function

rgw: remove unused function get_system_obj_attrs
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16383 from mikulely/1-ban-change-swift-obj-placement

rgw: drop the unnecessary handling of Swift's X-Storage-Policy on objects

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #17683 from shashalu/fix-lc-xml

rgw: RGWPutLC return ERR_MALFORMED_XML  when missing <Rule> tag in lifecycle.xml

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #17873 from amitkumar50/cov-rgw-8

rgw: Returning when dst_ioctx.operate() returns error

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18861 from gaosibei/wip-fix-null-pointer

rgw: fix null pointer crush

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #19061 from Leeshine/wip-rgw-bug-21583

rgw: fix the bug of radowgw-admin zonegroup set requires realm

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #19854 from fangyuxiangGL/doc-sync-err

rgw: add radosgw-admin sync error trim to trim sync error log

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd: fix a vargrind issue

Fixes: http://tracker.ceph.com/issues/22641
Signed-off-by: Myoungwon Oh <omwmw@sk.com>

ceph-volume: add success message for ceph-volume lvm create

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: add success message for ceph-volume lvm activate

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: add success message for ceph-volume lvm prepare

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #19809 from liu-chunmei/fix_dpdk_mzname_setting_issue

osd: fix dpdk memzon mz_name setting issue

Reviewed-By: Kefu Chai <kchai@redhat.com>

librados: add async interfaces for use with Networking TS

Defines asynchronous librados operations that satisfy all of the
"Requirements on asynchronous operations" imposed by the C++ Networking
TS [1] in section 13.2.7. These operations are implemented in terms of
boost::asio, but the interfaces themselves are free of boost types -
this makes the transition to std::net trivial when it's available.

These interfaces conform to the Extensible Asynchronous Model [2] that
originated in boost::asio. This model allows the last 'handler' argument
to either be a callback that gets the result, a coroutine yield_context
that will suspend until completion, or a 'use_future' tag to request the
result in a std::future (see the unit tests for examples of each). The
'Extensible' part also enables further integration with new frameworks.

For now, only async_read(), async_write(), and the read/write variants
of async_operate() are provided.

[1] Working Draft, C++ Extensions for Networking
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4711.pdf
[2] "Library Foundations for Asynchronous Operations"
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19616 from ifed01/wip-ifed-revert-extents

osd: get rid off extent map in object_info

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19764 from ifed01/wip-ifed-storetest-cempty

objectstore/store_test: fix lack of flush prior to collection_empty()…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19841 from rzarzynski/wip-bs-dump-onode-by-ref

os/bluestore: _dump_onode() don't prolongate Onode anymore.

Reviewed-by: Sage Weil <sage@redhat.com>

common/options: long description for log_stderr_prefix

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: using macro OBJECT_MAX_SIZE to check object max size.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

os/bluestore: fix data read error injection in bluestore

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: avoid unneeded BlobRefing in _do_read().

The hottest place in the terms of CPU cycles according to `perf annotate`:
```
       |          intrusive_ptr(intrusive_ptr const & rhs): px( rhs.px )
  0,04 | 365:   mov    0x28(%rax),%r12
       |          {
       |              if( px != 0 ) intrusive_ptr_add_ref( px );
  0,41 |        test   %r12,%r12
       |      ↓ je     37c
       |      _ZNSt13__atomic_baseIiEppEv():
  0,04 |        lock   addl   $0x1,(%r12)
 19,75 |        mov    0xa8(%rsp),%rax
       |      _ZN9BlueStore8_do_readEPNS_10CollectionEN5boost13intrusive_ptrINS_5OnodeEEEmmRN4ceph6buffer4listEj():
       |          }
       |          BlobRef bptr = lp->blob;
       |          unsigned l_off = pos - lp->logical_offset
       | 37c:   mov    0xa0(%rsp),%rdx
       |        sub    0x18(%rax),%edx
       |          unsigned b_off = l_off + lp->blob_offset;
  0,08 |        mov    0x1c(%rax),%ecx
```

With the patch applied:

```
       |          T * operator->() const
       |          {
       |              BOOST_ASSERT( px != 0 );
       |              return px;
       |        mov    0x28(%rcx),%rax
       |        mov    0x8(%rax),%rax
       |      _ZN9BlueStore8_do_readEPNS_10CollectionEN5boost13intrusive_ptrINS
       |          unsigned b_len = std::min(left, lp->length - l_off);
       |
       |          ready_regions_t cache_res;
       |          interval_set<uint32_t> cache_interval;
       |          bptr->shared_blob->bc.read(
 17,78 |        lea    0x18(%rax),%rdi
       |      _ZNK5boost13intrusive_ptrIN9BlueStore10CollectionEEcvbEv():
       |      #if !defined( BOOST_NO_CXX11_EXPLICIT_CONVERSION_OPERATORS ) && !
       |          && !(defined(__SUNPRO_CC) && BOOST_WORKAROUND(__SUNPRO_CC, <=
       |
       |          explicit operator bool () const BOOST_NOEXCEPT
       |          {
       |              return px != 0;
  0,07 |        mov    0x8(%rax),%rax
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: do not build/install boost::regex

we now use std::regex now, since we've dropped the support of GCC 4.8

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon: s/boost::regex/std::regex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: s/boost::regex/std::regex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

common: s/boost::regex/std::regex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: s/boost::regex/std::regex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

Revert "rgw: Support certain archaic and antiquated distributions"

This reverts commit af8d9bdf07ab97b9d9387f68ba0fedccce803073.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/{rbd_nbd,rbd}: s/boost::regex/std::regex/

* src/tools/rbd/*: s/boost::regex/std::regex/
* */CMakeLists.txt: do not link against Boost_REGEX_LIBRARY
    anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: add radosgw-admin sync error trim to trim sync error log

Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com

Merge PR #19791 into master

* refs/pull/19791/head:
doc: misc fixes
doc: fix heading

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: misc fixes

misc fixes for best-practices.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19662 from tchaikov/wip-denc-cxx-17

denc: use constexpr-if to replace some SFINAE impls

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

rgw: remove unused function get_system_obj_attrs

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

denc: use constexpr-if to replace some SFINAE impls

more readable this way IMO.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #19807 into master

* refs/pull/19807/head:
mds: warn for mds_cache_memory_limit bug in doc

Reviewed-by: Zheng Yan <zyan@redhat.com>

mds: warn for mds_cache_memory_limit bug in doc

Fixes: http://tracker.ceph.com/issues/22599
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19657 from adamemerson/wip-hypermodern

The Day Has Come!

Update language standard version to C++17 and remove backported standard library features.

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #19823 from trociny/wip-ggate-list-xml

rbd-ggate: make list command produce valid xml format output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19846 from fangyuxiangGL/compress-info

rgw: make radosgw object stat RGW_ATTR_COMPRESSION dump readable

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19732 from shinobu-x/using_find_primarylogpg

osd: Making use of find to reduce computational complexity

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19765 from liewegas/wip-force-backfill-cleanups

qa: tolerate failure to force backfill

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19786 from dillaman/wip-osd-class-cap

osd: support class method whitelisting within caps

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

{common,osd,mds,rbd}: Remove unused captures

If we aren't referencing a variable, don't capture it. (This makes the
lambdas smaller, too, modulo Sufficiently Smart Compiler.)

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

dout: Remove VLA sanity check

This doesn't seem to be able to work for C++17 under both GCC and
Clang. Maybe there's a better way to do it. I might try routing
that argument through a constexpr function in the macro later.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Remove backport_std.h entirely

And use the experimental ostream joiner provided with the library.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Remove backported C++17 functions

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Remove backported typedefs

Since they're no longer needed.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

build: Bump language to C++17

Fix up all of the fallout from that.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

osd: fix dpdk memzon mz_name setting issue

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

objectstore/store_test: fix lack of flush prior to collection_empty() call

Fixes: http://tracker.ceph.com/issues/22409
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

qa/standalone/scrub/osd-scrub-repair.sh: remove extents flag from object_info_t

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

osd: get rid off extent map in object_info

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #19475 from ZVampirEM77/wip-em-copyobj-ret

rgw: add xml output header in RGWCopyObj_ObjStore_S3 response msg

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: scienceluo <luo.kexue@zte.com.cn>