Merge PR #22256 into master

* refs/pull/22256/head:
unittest_addrs: check parse() return value too
common/LogEntry: remove addrs from log entry operator<<
common/LogEntry: addr -> addrs
common/LogEntry: no need for LogEntryKey encode/decode
msg/msg_types: entity_addrvec_t ctors
msg/msg_types: add entity_addrvec_t::legacy_addr() helper
msg/msg_types: some comparators for entity_addrvec_t
msg/msg_types: entity_addrvec_t parse() and operator<<

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22261 into master

* refs/pull/22261/head:
qa/suites/rados: move valgrind test to singleton-flat

Merge PR #22263 into master

* refs/pull/22263/head:
doc/releases: mimic 13.2.0

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

unittest_addrs: check parse() return value too

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22210 from rjfd/wip-dashboard-controller-refactor

mgr/dashboard: controller infrastructure refactor and new features

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22159 from b-ranto/wip-dashboard-shutdown

mgr/dashboard: Fix dashboard shutdown/restart

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: hacking.rst: minor improvements and cleanup

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: hacking.rst: controller creation instructions update

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: test: controllers unit test

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: controllers: @Endpoint annotation implementation

With these changes we now have a single implementation for both the
BaseController and RESTController classes, with the respective
overrides.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21960 from socketpair/output

osd: write "debug dump_missing" output to stdout

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22267 from ZVampirEM77/wip-telegraf-typo-fix

doc: fix a typo in doc/mgr/telegraf.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix a typo in doc/mgr/telegraf.rst

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

common/LogEntry: remove addrs from log entry operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogEntry: addr -> addrs

We want to switch to an addrvec.  This requires multiple parts:

 - switch the Key type to use just the rank
 - separate entity_name_t rank
 - compat encoding
 - graylog field naming has changed (includes name)
 - syslog output formatting has changed (includes name)
 - LogEntry operator<< modified a bit

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogEntry: no need for LogEntryKey encode/decode

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t ctors

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addrvec_t::legacy_addr() helper

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: some comparators for entity_addrvec_t

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t parse() and operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22223 from wido/telemetry-bool

mgr/telemetry: Check if boolean is False or not present

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22252 from liewegas/wip-24308

mon/MDSMonitor: do not send redundant MDS health messages to cluster log

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/releases: mimic 13.2.0

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados: move valgrind test to singleton-flat

No distro facet (or anything else) since we require centos for this test.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22019 from majianpeng/cmake-fio

cmake: update fio version from 3.5 to 540e235dcd276e63c57.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #22257 into master

* refs/pull/22257/head:
mon/MonCommands: allow nautilus for require-osd-release

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22243 into master

* refs/pull/22243/head:
Revert "mon: no delay for single message MSG_ALIVE and MSG_PGTEMP"

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #22228 into master

* refs/pull/22228/head:
mon: add send_mon_message() helper
mon: change timecheck_* key to int
mon/MonClient: make connection check more robust

Reviewed-by: João Eduardo Luís <joao@suse.de>

mon/MonCommands: allow nautilus for require-osd-release

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22255 from neha-ojha/wip-double-reservation

PG: release reservations after backfill completes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PG: add custom_reaction Backfilled and release reservations after backfill

After backfill completes, we directly go to the Recovered state without
releasing reservations. The outstanding reservations cause double reservation
issues.

Creating a custom_reaction Backfilled, allows us to release reservations,
before transiting to the Recovered state.

Signed-off-by: Neha Ojha <nojha@redhat.com>

mon: add send_mon_message() helper

Avoid a zillion calls to monmap->get_inst()

Signed-off-by: Sage Weil <sage@redhat.com>

mon: change timecheck_* key to int

We don't need to use entity_inst_t here since we're only paying attention
to mons in the quorum.

This involves changing the message format.  To save some time/effort and
complexity, we only support time checks on all-nautilus.  If you are
mid-upgrade, the time sync checks are skipped.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: make connection check more robust

Do no rely on the address comparison; just check the Connection*
itself.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #21934 into master

* refs/pull/21934/head:
qa/standalone/mon/misc: fix features test
qa/workunits: osd require-osd-release mimic -> nautilus
test/cli/monmaptool/feature-set-unset: fix unknown ids
test/cli/monmaptool: fix test
osd/OSDMap: make NAUTILUS feature bit significant
mon: add initial NAUTILUS feature/compatset bits

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22037 into master

* refs/pull/22037/head:
filestore: Raise the priority of two counters

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #22133 into master

* refs/pull/22133/head:
osd/ReplicatedBackend: fix use-after-free on InProgressOp

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #22147 into master

* refs/pull/22147/head:
test: silence unused variable warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22187 from jdurgin/wip-pg-hard-limit

osd: increase default hard pg limit

Reviewed-by: Sage Weil <sage@redhat.com>

qa/standalone/mon/misc: fix features test

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits: osd require-osd-release mimic -> nautilus

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22233 from neha-ojha/wip-fix-2580436

qa/workunits/suites/pjd.sh: use correct dir name

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21381 from liewegas/wip-seastore

doc/dev/seastore.rst: initial draft notes

Merge PR #22230 into master

* refs/pull/22230/head:
test/cli-integration/rbd: fixed unit formatting
qa/workunits/rbd: fix unit formatting tests
librbd: clones should not inherit implicit feature bits from parent

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22250 from falcon78921/wip-doc-falcon78921-cloudstack

doc: updated reference link for creating new disk offerings in cloudstack

mon/MDSMonitor: do not send redundant MDS health messages to cluster log

Fixes: http://tracker.ceph.com/issues/24308
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22178 from aclamk/wip-bug-24225-fix

common/crc/aarch64: Added cpu feature pmull and make aarch64 specific…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22236 from tchaikov/wip-rocksdb

rocksdb: sync with upstream

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22185 from ceph/wip-yuri-powercycle

qa/tests:  added supported distro for powercycle suite

Reviewed-by: Sage Weil <sage@redhat.com>

doc: updated reference link for creating new disk offerings in cloudstack

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Revert "mon: no delay for single message MSG_ALIVE and MSG_PGTEMP"

This change doesn't looks right and causing twice as much proposal as we targeted to (limited by paxos_propose_interval).

Imaging we have a sequence of pg_temp/up_thru during a large recovery.

now =T
The 1st up_thru/pg_temp will go through fast path and trigger propose at T + paxos_min_wait, last_attempted_minwait_time = T.

now = T+ paxos_min_wait
The [2, K] up_thru will failed by (now - last_attempted_minwait_time > g_conf->paxos_propose_interval)
and go through PaxosService::should_propose, which will schedule the propose at) T+paxos_propose_interval

now= T+ paxos_propose_interval + paxos_min_wait
The K+1 up_thru/pg_temp comes, both (now - last_attempted_minwait_time > g_conf->paxos_propose_interval
and now - paxos->get_last_commit_time() > g_conf->paxos_min_wait satisfied, so we trigger another propose
in now+ paxos_min_wait = T+ paxos_propose_interval +paxos_min_wait.

clearly we made TWO proposal in each paxos_propose_interval.

This reverts commit ca652104fe91ac41c7c1788a2907178c36fbe6ef.

Signed-off-by: Xiaoxi CHEN <xiaoxchen@ebay.com>

Merge pull request #22204 from tchaikov/wip-cmake-silence-bjam

cmake: always turn off bjam debugging output

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22126 from cbodley/wip-qa-rgw-no-ec-cache

qa/rgw: disable testing on ec-cache pools

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22124 from cbodley/wip-rgw-test-zonegroup-checkpoint

test/rgw: fix for bucket checkpoints

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #22246 from tchaikov/wip-mkfs-EINVAL

qa: use $TESTDIR for testing mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

qa: use $TESTDIR for testing mkfs

as tmpfs does not support O_DIRECT, but bluefs use this flag for
accessing the underlying file.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/cli/monmaptool/feature-set-unset: fix unknown ids

Move 16 -> 64 so it is still unknown.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #22163 into wip-sage-testing-20180525.131829

* refs/pull/22163/head:
CodingStyle: update my_type_t sample
CodingStyle: allow #pragma once
CodingStyle: a few updates

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>

Merge pull request #22237 from a2batic/link

mgr/dashboard: Fixes documentation link- to open in new tab

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #22182 from tspmelo/wip-unit-test-pipe

mgr/dashboard: Add unit tests for all frontend pipes

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22206 from theanalyst/wip-doc-tag-fix

doc: rgw: fix tagging support status

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #22207 from tchaikov/wip-dpp-constexpr

dout: declare dpp using `decltype(auto)` instead of `auto`

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Fixes documentation link- to open in new tab
Adds 'target' attribute to open link in new tab.
Fixes : https://tracker.ceph.com/issues/24288

Signed-off-by: a2batic <murarkakanika@gmail.com>

rocksdb: sync with upstream

since
https://github.com/facebook/rocksdb/pull/3862 has been merged by
upstream.

Fixes: http://tracker.ceph.com/issues/23653
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #22026 into master

* refs/pull/22026/head:
mds: send cap export message when exporting non-auth caps to auth mds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22232 from batrick/cephfs-release-notes-mimic

doc/releases: update CephFS mimic notes

Reviewed-by: Sage Weil <sage@redhat.com>

qa/workunits/suites/pjd.sh: use correct dir name

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22195 from neha-ojha/wip-minor-fixes

qa/suites/rados: minor fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/releases: update CephFS mimic notes

Fixes: http://tracker.ceph.com/issues/23775
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22034 from mogeb/iostat-doc

doc: add documentation for iostat

Reviewed-by: Sage Weil <sage@redhat.com>

test/cli-integration/rbd: fixed unit formatting

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: fix unit formatting tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: clones should not inherit implicit feature bits from parent

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

CodingStyle: update my_type_t sample

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedBackend: fix use-after-free on InProgressOp

- op in flight to disk...
- on_change() clears the InProgressOp
- C_OSD_OnOpCommit calls op_commit() w/ bare pointer
- crash!

Fix by refcounting InProgressOp and clearing on_commit when it is
canceled.

Fixes: http://tracker.ceph.com/issues/24219
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22112 from szeestraten/fix_24199

common: fix typo in rados bench write JSON output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22093 from liewegas/wip-24162

msg/async/Event: clear time_events on shutdown

Reviewed-by: Hoamai Wang <haomai@xsky.com>

Merge pull request #21491 from majianpeng/kerneldevice

os/bluestore/KernelDevice: misc cleanup

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/telemetry: Check if boolean is False or not present

Now we disable the module if the boolean is True

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #22216 from votdev/fix_table_pagination_style

mgr/dashboard: Fix some datatable CSS issues

Approved-by: Ricardo Marques <rimarques@suse.com>

common/arch/arm: Improve CPU feature check.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

test/cli/monmaptool: fix test

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: make NAUTILUS feature bit significant

It's not actually significant to the encoding yet, but it will be, so put
this in place now.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: add initial NAUTILUS feature/compatset bits

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Fix some datatable CSS issues

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #22214 from joscollin/wip-ceph-fuse-typo

doc: Fix typo in ceph-fuse(8)

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: Fix typo in ceph-fuse

Fixes: https://github.com/ceph/ceph/pull/21616#pullrequestreview-122923127
Signed-off-by: Jos Collin <jcollin@redhat.com>

mgr/dashboard: Add unit tests for all frontend pipes

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22205 from tchaikov/wip-librgw_admin_user

deb,rpm: package librgw_admin_user.{h,so.*}

mgr/dashboard: tox: ignore .vscode directory from linting

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: controller infrastructure refactor

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #22181 from tchaikov/wip-24025

cmake/modules/BuildRocksDB.cmake: enable compressions for rocksdb

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: removed browsable api

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: restcontroller: removed proxy methods

Now the dispatcher will call directly the RESTController methods:
list, create, delete, set, etc...

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: task: fix bug in __eq__ method

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Remove unused imports

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Add documentation for method `endpoints`

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Code refactoring

- Remove definition and usage of `isfunction` function and replace it
  with Pythons built-in `callable.

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

dout: declare dpp using `decltype(auto)` instead of `auto`

this makes `pdpp` an alias of `dpp`, guess this ensure GCC that the
returned `sub` is a constant.

In file included from /home/kefu/dev/ceph/src/kv/LevelDBStore.h:25,
                 from /home/kefu/dev/ceph/src/kv/KeyValueDB.cc:6:
/home/kefu/dev/ceph/src/osd/osd_types.h: In lambda function:
/home/kefu/dev/ceph/src/common/dout.h:101:75: error: the value of ‘pdpp’
is not usable in a constant expression
     dout_impl(pdpp->get_cct(),
ceph::dout::need_dynamic(pdpp->get_subsys()), v) \
                                                                           ^
/home/kefu/dev/ceph/src/common/dout.h:81:58: note: in definition of
macro ‘dout_impl’
       return (cctX->_conf->subsys.template should_gather<sub, v>()); \
                                                          ^~~
/home/kefu/dev/ceph/src/osd/osd_types.h:2992:3: note: in expansion of
macro ‘ldpp_dout’
   ldpp_dout(dpp, 10) << "build_prior all_probe " << all_probe << dendl;
   ^~~~~~~~~
/home/kefu/dev/ceph/src/common/dout.h:100:12: note: ‘pdpp’ was not
declared ‘constexpr’
   if (auto pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for
'this' */ \
            ^~~~
/home/kefu/dev/ceph/src/common/dout.h:100:12: note: in definition of
macro ‘ldpp_dout’
   if (auto pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for
'this' */ \
            ^~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: rgw: fix tagging support status

As of mimic, policy and LC conditionals for tagging are merged, so let's drop
the unsupported status.

Fixes: http://tracker.ceph.com/issues/24164
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>