doc/cephadm: Move FQDN chapter to host mgmt.rst

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/cephadm: Move SSH config from operations to host-mgmt.rst

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/cephadm: group host mgmt sections into one chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39692 from zdover23/wip-doc-dev-t8y-debugging-analyzing-rewrite-2021-Feb-25

doc/dev: t8y - rewrite "a. & d. a t8y job"

Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/dev: t8y - rewrite "a. & d. a t8y job"

This PR rewrites the "Analyzing and Debugging a
Teuthology Job" section of the "Debugging Tips"
page of the Teuthology User Guide in the Developer
Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39614 from sebastian-philipp/cephadm-again-test-ubuntu_2004_podman

qa/cephadm: Test again podman on ubuntu 20.04

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #38604 from cybozu/doc-fix-the-meaning-of-min_size

doc: fix the meaning of default_min_size of pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39684 from xxhdx1985126/wip-alienstore-parallelism-improvement

crimson/os/alienstore: improve alienstore's write parallelism

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39602 from tchaikov/wip-mgr-flux-ann

mgr: add type annotations to BaseMgrModule and influx

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #39371 from tchaikov/wip-mgr-rbd-support-ann

mgr/rbd_support: add type annotation and cleanups

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

crimson/os/alienstore: improve alienstore's write parallelism

replace the grand per store tp_mutex with a finer grained per-collection
lock for better concurrency

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #39666 from zdover23/wip-doc-dev-t8y-rerunning-tests-rewrite-2021-Feb-24

doc/dev: rewrite t8y "re-running tests"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39694 from zdover23/wip-doc-dev-t8y-debugging-triaging-rewrite-2021-Feb-25

doc/dev: rewrite t8y "triaging" section

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39695 from zdover23/wip-doc-dev-t8y-debugging-adding-sections-2021-Feb-26

doc/dev: add t8y debug section names

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39697 from zdover23/wip-doc-dev-t8y-debug-reporting-the-issue-rewrite-2021-Feb-26

doc/dev: rewrite t8y "reporting the issue"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39707 from petrutlucian94/overlapped

rbd: fix mingw 5.0.3 compatibility issue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: fix mingw 5.0.3 compatibility issue

GetOverlappedResultEx isn't avaialable when using mingw 5.0.3
(default on Ubuntu Bionic).

Since we don't really need the extra parameters, we'll switch
to GetOverlappedResult.

We're doing this mostly for testing purposes as the mingw 5.0.3
runtime is known to be broken.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #39649 from Daniel-Pivonka/fixorphaninitialdaemons

cephadm: fix bug in orphan-initial-daemons logic

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #39613 from sebastian-philipp/DaemonDescriptionStatus

mgr/orch: Add DaemonDescriptionStatus

Reviewed-by: Daniel-Pivonka <dpivonka@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #39672 from ceph/wip-159-notes

doc/releases/octopus.rst: 15.2.9 Release Notes

doc/dev: rewrite t8y "reporting the issue"

This commit rewrites the "Reporting the Issue"
section, reflecting my better understanding of the
various tools and parties in play. I also added a
TL;DR-style summary (or argument) paragraph at the
top of the section.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/dev: add t8y debug section names

This commit adds two section headings to the
Debugging section of the Teuthology Guide:

  1. Viewing Test Results
  2. Identifying Failed Jobs

The text as it stood before seemed to call out for
this explicit sectioning of material, so I've answered
that call.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/dev: rewrite t8y "triaging" section

This commit simplifes and clarifies the "Triaging
the Cause of Failure" section in the Teuthology
Guide in the Developer Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/dev: rewrite t8y "re-running tests"

This PR rewrites the "re-running tests" section
so that its elegance and readability are improved.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39598 from tchaikov/wip-crimson-op-sequencer

crimson/osd: refactor OperationRepeatSequencer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #39659 from tchaikov/wip-pybind-rados-cleanup

pybind/rados: remove rados_nobjects_list_next() from .pxi

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/{prometheus,telemetry}: appease mypy

update to adapt the type annotation of MgrModule.list_servers()

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/ceph_module: add type annotation to BaseMgrModule

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: use empty data points to signal shutting down

to appease mypy, which prefer a queue with elements of the same type.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: avoid casting variable

self.config is a dict with elements of different types, to appease mypy,
we would need to cast the indexed item to the expected type. another way
is to use f-string, so the variable are always printed using `__str__`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: use get_module_option() for typed option

get_module_option() cast the option to the type claimed in its defition,
also, the returned value are always validated before it is originally
set. so there is no need to repeat this in mgr module.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: use "N/A" for unknown hostname

in theory, there is chance that get_metadata() returns None, so let use
"N/A" in this case.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: add typing annotation

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: specify range of threads

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: specify the type of options

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/mgr_module correct typing of self_test

self_test() is allowed to return str

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: define commands uing CLICommand

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/influx: define options using Option

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38937 from myoungwon/fix-snap-refcount

src/test: fix to avoid fail notification when testing manifest refcount

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #39610 from tchaikov/wip-mgr-set-option

mgr: raise if set_module_option() fails

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #39320 from ifed01/wip-ifed-fix-huge-bluefs-write

os/bluestore: fix huge(>4GB) writes from RocksDB to BlueFS.

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>

Merge pull request #39600 from tchaikov/wip-mgr-zabbix-desc

mgr/zabbix: add desc to commands

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #39676 from liewegas/fix-kv

mon/KVMonitor: fix 'osd new' cross-service commit

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #38914 into master

* refs/pull/38914/head:
qa: bump osd heartbeat grace for ffsb workload

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #39498 into master

* refs/pull/39498/head:
test: use std::atomic<bool> instead of volatile for cb_done var

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #39183 into master

* refs/pull/39183/head:
mon: simplify fs 'services' view of 'ceph status'
mds: add volumes + status to data: section of 'ceph status'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

pybind/rados: remove rados_nobjects_list_next() from .pxi

this is a follow-up of da5d4c813ffdd391da54dcf5022763616bab4b21

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39668 from zdover23/wip-doc-dev-t8y-rewrite-naming-ceph-ci-branch-2021-Feb-25

doc/dev: t8y-naming-ceph-ci-branch rewrite

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39140 from sseshasa/wip-dmclock-profiles-improvements

osd: Refinements to mclock built-in profiles implementation.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #38921 from lixiaoy1/pwl_teuthology

qa: add tests for persistent writeback cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mon/KVMonitor: fix 'osd new' cross-service commit

When we converted ConfigKeyService to KVMonitor, we didn't correctly
change this to propose_pending(), which mean that the kv change wasn't
captured in the paxos transaction.

Fixes: bb7ebc41532aeb23cff2241ab07b3f01c2f57ddd
Fixes: https://tracker.ceph.com/issues/49460
Signed-off-by: Sage Weil <sage@newdream.net>

doc/releases/octopus.rst: 15.2.9 Release Notes

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge PR #39607 into master

* refs/pull/39607/head:
qa/cephfs: check for Ceph FS before mounting
qa/cephfs: remove parameter createfs from mount classes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #39609 into master

* refs/pull/39609/head:
doc/ceph-fuse: add some options to man page

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #39617 from tchaikov/wip-mgr-diskprediction-local-flake8

mgr/diskprediction_local: wrap line longer than 100 chars

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #39638 from Rethan/fix-description-throttle-val

common/Throttle: change description about throttle val

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39667 from tchaikov/wip-tools-rados-optional

tools/rados: assign to optional<> without deref'ing it

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #39562 from Daniel-Pivonka/cephadmversionauthregerror

cephadm: version command hide traceback when login is needed

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/dev: t8y-naming-ceph-ci-branch rewrite

This PR rewrites the "Naming the Ceph Branch"
section of the Teuthology docs, improving its
elegance and unburying the ledes.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39474 from sebastian-philipp/doc-cephadm-manual-mgr

doc/cephadm: troubleshooting: manually deploy MGR

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #39420 from sebastian-philipp/cephadm-enable-ubuntu-again

qa/suites/rados/cephadm: Add 20.04 podman:testing

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39421 from p-se/wip-pse-cephadm-port-in-use

cephadm: fix port_in_use when IPv6 is disabled

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39415 from jmolmo/paths_in_all_osd_components

cephadm: Allow to use paths in all <_devices> drivegroup sections

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39537 from adk3798/emsg-redeploy

mgr/cephadm: error if service action called with daemonless service

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39536 from liewegas/cephadm-drain-weight

mgr/cephadm: fix and improve osd draining

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

tools/rados: assign to optional<> without deref'ing it

this change addresses a regression introduced by
d333b35aa10bf03a8bc047994d5cf3fed019b49a

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39463 from trociny/wip-49115

librbd: use on-disk image name when storing mirror snapshot state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #39625 from trociny/wip-49418

rbd-mirror: reset update_status_task pointer in timer thread

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/cephfs: check for Ceph FS before mounting

If the given Ceph FS, or the default Ceph FS when no Ceph FS is given,
is absent, abort the execution with AsssertionError and an error
message.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #39622 from dillaman/wip-librbd-disable-qcow

librbd: permit disabling QCOW migration format support

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #38170 from rhcs-dashboard/feature-47375-select-rgw-daemon

mgr/dashboard: Select any object gateway on local cluster

Merge pull request #38940 from rhcs-dashboard/tenant-user

mgr/dashboard: Splitting tenant$user when creating rgw user

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #39322 from rzarzynski/wip-librados-nul-objname

librados, pybind, tests: allow to list objects with the NUL character in names

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #39570 from tchaikov/wip-getprogname

common: s/prctl/pthread_getname_np/ for better portability

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39427 from ktdreyer/rpm-whitespace

ceph.spec.in: remove trailing whitespace

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #39621 from kshtsk/wip-sle-luarocks

ceph.spec.in: use lua53 for openSUSE 15.2

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

crimson/osd: s/OperationRepeatSequencer/OpSequencers/

as pointed out by Radoslaw Zarzynski,

> this is actually a registry of sequencers.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: refactor OperationRepeatSequencer

* extract the OpSequencer out from OperationRepeatSequencer
* refactor OpSequencer so we don't need to track the ops using a map,
  only track the last op and last pg interval for better performance
  and smaller memory footprint.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: select any object gateway on local cluster.

Dashboard backend settings:
- Refactoring: now accepting more than 1 type of value.
- RGW_API_ACCESS_KEY & RGW_API_SECRET_KEY accept string (backward compatibility: legacy behavior) as well as dictionary of strings for connecting multiple daemons.
- Ease of use: deprecated: mgr/dashboard/RGW_API_USER_ID: not useful anymore (kept for backward compatibility).

UI/UX:
- Created context component (to be shown only on rgw-related routes) for selecting operating daemon.
- Daemon selector only shown if there is more than 1 daemon running on a local cluster (to reduce cognitive load).

Fixes: https://tracker.ceph.com/issues/47375
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

crimson/common: let the first id to be 1

it'd be simpler if we can use 0 as a sentry for sequencer. 0 would be
id of the last completed operation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39655 from cyx1231st/wip-seastore-onode-fix-cursor-cmp

crimson/onode-staged-tree: fix Cursor operator==()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/onode-staged-tree: add missing type in Cache::alloc_new_extent_by_type()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/onode-staged-tree: fix Cursor operator==

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #39653 from zdover23/wip-doc-dev-t8y-workflow-viewing-tests-results-2021-Feb-24

doc/dev: s/getting tests r.../getting test r...

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev: s/getting tests r.../getting test r...

s/getting tests results/getting test results/

This commit changes a title so that it is grammatically
correct. The adjective "test" now properly modifies the
noun "results". Another way of thinking of this is that
the complex noun "test results" has had a typo in it
corrected.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39564 from tchaikov/wip-doc-layout

doc: consolidate the layout settings using CSS

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39288 from cybozu/fix-the-procedure-of-recovery-mon-quorum-using-osd

doc: fix the procedure of recovery mon quorum using osd

Reviewed-by: Neha Ojha <nojha@redhat.com>

cephadm: fix bug in orphan-initial-daemons logic

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>

Merge pull request #39632 from jdurgin/wip-crontab

qa: adjust scheduled jobs to fit better in the lab

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #39615 from zdover23/wip-doc-dev-basic-workflow-intro-adding-ceph-ceph-2021-Feb-22

doc/dev: adding upstream repo name

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39606 from idryomov/wip-rbd-map-sanity-check

krbd: make sure the device node is accessible after the mapping

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #39596 into master

* refs/pull/39596/head:
qa/tasks/mgr/dashboard: add wear_level to devices schema

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

doc/dev: adding upstream repo name

This commit makes one sentence slightly less
ambiguous by designating the location from
which the "ceph fetch" command fetches objects.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39608 from tchaikov/wip-crimson-fix-sequencer

crimson/osd: drop pending ops when pg interval changes

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #39519 from petrutlucian94/minor_fixes

common, rbd: Minor Windows fixes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd: fix rbd-wnbd log messages

Previously, rbd-wnbd was called "rbd-nbd" as it was using NBD
to communicate with the driver.

The tool was then renamed to "rbd-wnbd" but some messages were
not updated.

This patch will simply replace "rbd-nbd" with "rbd-wnbd" in some
log messages.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

common: s/prctl/pthread_getname_np/ for better portability

prctl() is linux specific, let's use pthread_getname_np for thread names.

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: permit disabling QCOW migration format support

Downstream Red Hat products do not support the older QCOW format. This
will allow the support for the legacy QCOW format to be disabled for the
new RBD import-only migration support.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/Throttle: change description about throttle val

l_throttle_val indicates current used count, not available.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>

rbd: fix rbd-wnbd device status

The "rbd-wnbd show" command will always report the device status
as "inactive". This patch adds the missing check, similar to the
one used by the "list" command.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>