Merge pull request #22313 from b-ranto/wip-dashboard-fix-rate

dashboard: Return float if rate not available

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22320 from ukernel/wip-24343

mds: properly journal root inode's snaprealm

mds: properly journal root inode's snaprealm

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/24343

Merge pull request #21980 from Devp00l/wip-formatting-bug-24081

mgr/dashboard: Format small numbers correctly

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #22148 from ricardoasmarques/wip-duplicated-rbd

Prevent duplicated rows during async tasks

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22244 from tspmelo/wip-unit-test-services

mgr/dashboard: Add unit test to the frontend services

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #22278 from votdev/prettify_rgw

mgr/dashboard: Prettify RGW JS code

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #22287 from tchaikov/wip-hex-dec-in-logging

src: balance std::hex and std::dec manipulators

Reviewed-by: João Eduardo Luís <joao@suse.de>

dashboard: Return float if rate not available

This fixes a test case issue when some of the data might not be
available during the test, yet. The test expects the second value to be
JFloat but that fails because get_rates return an int in that case.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #22307 from ukernel/wip-22240-1

mds: fix leak of MDSCacheObject::waiting

mds: fix leak of MDSCacheObject::waiting

Fixes: http://tracker.ceph.com/issues/24289
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #22240 from ukernel/wip-24289

mds: fix some memory leak

Merge pull request #22305 from yehudasa/wip-rgw-mimic

docs: update rgw info for mimic

docs: update rgw info for mimic

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

src: balance std::hex and std::dec manipulators

we're caching the "t_os.streambuf", so the fmtflags persist across
log entries. to avoid this side effects, we should do the cleanup
before finishing each log entry.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22280 from hjwsm1989/wip-osdmaptool

osdmaptool: fix wrong test_map_pgs_dump_all output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22157 from ceph/wip-doc-clear-config

doc/mgr: mention how to clear config setting

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #22286 from ifed01/wip-ifed-fix-release-build

cmake: enable RTTI for both debug and release RocksDB builds.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22231 from linuxbox2/wip-get-irange

rgw: add configurable AWS-compat invalid range get behavior

Merge pull request #22203 from hydro-b/rbd-docu-fixes

doc: remove duplicate python packages

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22259 from liewegas/wip-24322

mon/OSDMonitor: no_reply on MOSDFailure messages

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: remove duplicate python packages, explicitly ensure python script installation in path /usr/bin.
Signed-off-by: Stefan Kooman <stefan@kooman.org>
Signed-off-by: Stefan Kooman <stefan@kooman.org>

mon/OSDMonitor: no_reply on MOSDFailure ops we eventually process

Failure ops get attached to the failure report.  Once we finally process
them, mark them no_reply so that the forwarding mon will know about it.
The other paths through prepare_failure() do no_reply on the messages
that don't get logged in the failure_info_t::reporters.

Fixes: http://tracker.ceph.com/issues/24322
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22279 from tspmelo/wip-improve-lint-rules

mgr/dashboard: Improve max-line-length tslint rule

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22281 from p-na/hacking-rst

mgr/dashboard: Improve `HACKING.rst`

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #22193 from liewegas/wip-mon-ranks

mon: make rank ordering explicit (not tied to mon address sort order)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22275 from tchaikov/wip-24321

qa: wait longer for osd to flush pg stats

Reviewed-by: Sage Weil <sage@redhat.com>

mds: fix some memory leak

Fixes: http://tracker.ceph.com/issues/24289
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

build/cmake: enable RTTI for both debug and release RocksDB builds.
Overwise ceph build in Release mode is failing.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

mon/Monitor: use legacy ranks during bootstrap for initial seed monmap

We get various fabricated monmaps with epoch 0 to get things started.  Use
the legacy rank order so that we have a well-defined ordering and mon
ranks do not collide.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonMap: use legacy rank order for all bootstrap monmaps

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #22256 into master

* refs/pull/22256/head:
unittest_addrs: check parse() return value too
common/LogEntry: remove addrs from log entry operator<<
common/LogEntry: addr -> addrs
common/LogEntry: no need for LogEntryKey encode/decode
msg/msg_types: entity_addrvec_t ctors
msg/msg_types: add entity_addrvec_t::legacy_addr() helper
msg/msg_types: some comparators for entity_addrvec_t
msg/msg_types: entity_addrvec_t parse() and operator<<

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Improve `HACKING.rst`

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge PR #22261 into master

* refs/pull/22261/head:
qa/suites/rados: move valgrind test to singleton-flat

Merge PR #22263 into master

* refs/pull/22263/head:
doc/releases: mimic 13.2.0

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

osdmaptool: fix wrong test_map_pgs_dump_all output

Signed-off-by: huangjun <huangjun@xsky.com>

mgr/dashboard: Improve max-line-length tslint rule

Prettier is not breaking strings if they are bigger
then the defined max line length.

With this rule we can make sure lines are not exceeding it,
with the exception of import/export lines.

Signed-off-by: Tiago Melo <tmelo@suse.com>

unittest_addrs: check parse() return value too

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Prettify RGW JS code.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #22210 from rjfd/wip-dashboard-controller-refactor

mgr/dashboard: controller infrastructure refactor and new features

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

qa: wait longer for osd to flush pg stats

pg sends pg-stats to mgr every 5 seconds, so we cannot check for the
number of pgs right after creating the pool, at that moment, the number
of pgs could be 0, that's why manger.wait_for_clean() returns right
away, and leaves us with 0 pgs: the pgs serving the pool are still being
created. that's why `manager.get_num_active_clean()` returns `0`
sometimes. so, we should force osd to flush their stats to mgr, and wait
until the pg stats converages.

Fixes: http://tracker.ceph.com/issues/24321
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22159 from b-ranto/wip-dashboard-shutdown

mgr/dashboard: Fix dashboard shutdown/restart

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: hacking.rst: minor improvements and cleanup

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: hacking.rst: controller creation instructions update

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: test: controllers unit test

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: controllers: @Endpoint annotation implementation

With these changes we now have a single implementation for both the
BaseController and RESTController classes, with the respective
overrides.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21960 from socketpair/output

osd: write "debug dump_missing" output to stdout

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22267 from ZVampirEM77/wip-telegraf-typo-fix

doc: fix a typo in doc/mgr/telegraf.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix a typo in doc/mgr/telegraf.rst

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

common/LogEntry: remove addrs from log entry operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogEntry: addr -> addrs

We want to switch to an addrvec.  This requires multiple parts:

 - switch the Key type to use just the rank
 - separate entity_name_t rank
 - compat encoding
 - graylog field naming has changed (includes name)
 - syslog output formatting has changed (includes name)
 - LogEntry operator<< modified a bit

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogEntry: no need for LogEntryKey encode/decode

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t ctors

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addrvec_t::legacy_addr() helper

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: some comparators for entity_addrvec_t

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t parse() and operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22223 from wido/telemetry-bool

mgr/telemetry: Check if boolean is False or not present

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22252 from liewegas/wip-24308

mon/MDSMonitor: do not send redundant MDS health messages to cluster log

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mon/MonMap: use c++17 structured bindings

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonmapMonitor: add 'mon set-rank <name> <rank>' command

Add a command to manipulate the mon ranks.  This lets you control which
mon is preferred as the primary, in case you care about such things.
Mostly it doesn't matter.  (Previously we preferred the mon with the
lowest-sorting IP address, which is pretty arbitrary.)

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonMap: separate rank ordering from entity_addr_t

We currently define the mon rank ordering based on the sort order of the
mon addresses.  Change that so that the rank order is explicitly encoded
in the ranks field of the MonMap.

If we load a legacy MonMap, calculate the legacy ordering.

If the monmap does not require nautilus features yet, force the legacy
ordering.  Once all mons are >= nautilus, we can reorder ranks.  Note that
the daemons and clients (MonClients) may see a different rank ordering.
That should be okay.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/releases: mimic 13.2.0

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados: move valgrind test to singleton-flat

No distro facet (or anything else) since we require centos for this test.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: no_reply on ignored MOSDFailure

If we are ignoring the message, tell the forwarding mon to discard it's
state.

Fixes: http://tracker.ceph.com/issues/24322
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22019 from majianpeng/cmake-fio

cmake: update fio version from 3.5 to 540e235dcd276e63c57.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #22257 into master

* refs/pull/22257/head:
mon/MonCommands: allow nautilus for require-osd-release

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/MonMap: simplify encode() flow

We only need legacy_mon_addr for the !MONENC case.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #22243 into master

* refs/pull/22243/head:
Revert "mon: no delay for single message MSG_ALIVE and MSG_PGTEMP"

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #22228 into master

* refs/pull/22228/head:
mon: add send_mon_message() helper
mon: change timecheck_* key to int
mon/MonClient: make connection check more robust

Reviewed-by: João Eduardo Luís <joao@suse.de>

mon/MonCommands: allow nautilus for require-osd-release

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22255 from neha-ojha/wip-double-reservation

PG: release reservations after backfill completes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PG: add custom_reaction Backfilled and release reservations after backfill

After backfill completes, we directly go to the Recovered state without
releasing reservations. The outstanding reservations cause double reservation
issues.

Creating a custom_reaction Backfilled, allows us to release reservations,
before transiting to the Recovered state.

Signed-off-by: Neha Ojha <nojha@redhat.com>

mon: add send_mon_message() helper

Avoid a zillion calls to monmap->get_inst()

Signed-off-by: Sage Weil <sage@redhat.com>

mon: change timecheck_* key to int

We don't need to use entity_inst_t here since we're only paying attention
to mons in the quorum.

This involves changing the message format.  To save some time/effort and
complexity, we only support time checks on all-nautilus.  If you are
mid-upgrade, the time sync checks are skipped.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: make connection check more robust

Do no rely on the address comparison; just check the Connection*
itself.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #21934 into master

* refs/pull/21934/head:
qa/standalone/mon/misc: fix features test
qa/workunits: osd require-osd-release mimic -> nautilus
test/cli/monmaptool/feature-set-unset: fix unknown ids
test/cli/monmaptool: fix test
osd/OSDMap: make NAUTILUS feature bit significant
mon: add initial NAUTILUS feature/compatset bits

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22037 into master

* refs/pull/22037/head:
filestore: Raise the priority of two counters

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #22133 into master

* refs/pull/22133/head:
osd/ReplicatedBackend: fix use-after-free on InProgressOp

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #22147 into master

* refs/pull/22147/head:
test: silence unused variable warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22187 from jdurgin/wip-pg-hard-limit

osd: increase default hard pg limit

Reviewed-by: Sage Weil <sage@redhat.com>

qa/standalone/mon/misc: fix features test

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: add configurable AWS-compat invalid range get behavior

If rgw_ignore_get_invalid_range is set, treat invalid range
restrictions as a request for the full object.  By default, retain
the RGW behavior to fail with ERANGE.

Fixes: http://tracker.ceph.com/issues/24317
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

qa/workunits: osd require-osd-release mimic -> nautilus

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22233 from neha-ojha/wip-fix-2580436

qa/workunits/suites/pjd.sh: use correct dir name

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21381 from liewegas/wip-seastore

doc/dev/seastore.rst: initial draft notes

Merge PR #22230 into master

* refs/pull/22230/head:
test/cli-integration/rbd: fixed unit formatting
qa/workunits/rbd: fix unit formatting tests
librbd: clones should not inherit implicit feature bits from parent

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22250 from falcon78921/wip-doc-falcon78921-cloudstack

doc: updated reference link for creating new disk offerings in cloudstack

mon/MDSMonitor: do not send redundant MDS health messages to cluster log

Fixes: http://tracker.ceph.com/issues/24308
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22178 from aclamk/wip-bug-24225-fix

common/crc/aarch64: Added cpu feature pmull and make aarch64 specific…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22236 from tchaikov/wip-rocksdb

rocksdb: sync with upstream

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22185 from ceph/wip-yuri-powercycle

qa/tests:  added supported distro for powercycle suite

Reviewed-by: Sage Weil <sage@redhat.com>

doc: updated reference link for creating new disk offerings in cloudstack

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Revert "mon: no delay for single message MSG_ALIVE and MSG_PGTEMP"

This change doesn't looks right and causing twice as much proposal as we targeted to (limited by paxos_propose_interval).

Imaging we have a sequence of pg_temp/up_thru during a large recovery.

now =T
The 1st up_thru/pg_temp will go through fast path and trigger propose at T + paxos_min_wait, last_attempted_minwait_time = T.

now = T+ paxos_min_wait
The [2, K] up_thru will failed by (now - last_attempted_minwait_time > g_conf->paxos_propose_interval)
and go through PaxosService::should_propose, which will schedule the propose at) T+paxos_propose_interval

now= T+ paxos_propose_interval + paxos_min_wait
The K+1 up_thru/pg_temp comes, both (now - last_attempted_minwait_time > g_conf->paxos_propose_interval
and now - paxos->get_last_commit_time() > g_conf->paxos_min_wait satisfied, so we trigger another propose
in now+ paxos_min_wait = T+ paxos_propose_interval +paxos_min_wait.

clearly we made TWO proposal in each paxos_propose_interval.

This reverts commit ca652104fe91ac41c7c1788a2907178c36fbe6ef.

Signed-off-by: Xiaoxi CHEN <xiaoxchen@ebay.com>

Merge pull request #22204 from tchaikov/wip-cmake-silence-bjam

cmake: always turn off bjam debugging output

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22126 from cbodley/wip-qa-rgw-no-ec-cache

qa/rgw: disable testing on ec-cache pools

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22124 from cbodley/wip-rgw-test-zonegroup-checkpoint

test/rgw: fix for bucket checkpoints

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #22246 from tchaikov/wip-mkfs-EINVAL

qa: use $TESTDIR for testing mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

qa: use $TESTDIR for testing mkfs

as tmpfs does not support O_DIRECT, but bluefs use this flag for
accessing the underlying file.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/cli/monmaptool/feature-set-unset: fix unknown ids

Move 16 -> 64 so it is still unknown.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #22163 into wip-sage-testing-20180525.131829

* refs/pull/22163/head:
CodingStyle: update my_type_t sample
CodingStyle: allow #pragma once
CodingStyle: a few updates

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>