.github/labeler.yml: add cephadm

Signed-off-by: Neha Ojha <nojha@redhat.com>

.github/labeler.yml: add bluestore and more core rules

more to come

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #37939 from trociny/wip-rbd-nbd-wait-for-terminate

rbd-nbd: fixes and improvements for unmap/detach wait for process terminate

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38081 from wjwithagen/wjw-fix-test_mock_CryptoObjectDispatch.cc

test/librbd/crypto: Fixup include order to prevent implicit definitions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38105 from ofriedma/wip-barbican-qa-issue

qa/rgw: fix "cannot create secret" on barbican test

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #38072 from tchaikov/wip-github-labeler-crimson

github/labeler: add "crimson" to labeler.yml

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #38041 from tchaikov/wip-mon-trim-osdmap

mon/OSDMonitor: remove osd_epochs if osd is marked in or out

Reviewed-by: Joao Eduardo Luis <joao@suse.com>

Merge pull request #38103 from ybwang0211/link-to-balancer

doc: Fix the broken link to the "balancer"

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/rgw: fix "cannot create secret" on barbican test

fix "cannot create secret" on barbican test

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Fixes: https://tracker.ceph.com/issues/47799

doc:Fix the web link that jumps to the "balancer"

In "https://docs.ceph.com/en/latest/rados/operations/upmap/#upmap", the link to "balancer" was not set correctly, so we fixed it.

Signed-off-by: wangyingbin <wangyingbin@inspur.com>

Merge pull request #38007 from sseshasa/wip-qa-rados-perf-scheduler

qa/suite/rados: Introduce "scheduler" directory within rados perf suite.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #34842 into master

* refs/pull/34842/head:
qa/tasks/vstart_runner.py: disable the ns unsharing if using localhost

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #37746 into master

* refs/pull/37746/head:
client: add a dedicated thread for the Client tick
client: try to renew caps and flush old cap releases

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #37982 into master

* refs/pull/37982/head:
qa/cephfs: add code for when config is None in __init__

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge PR #38016 into master

* refs/pull/38016/head:
qa: switch to 'osdop_read' instead of 'op_r'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #38056 into master

* refs/pull/38056/head:
pybind/mgr/volumes: use = for equality comparision

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38077 from tchaikov/wip-doc-scrub-max

doc/rados/operation: add link to scrub_max_interval

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38038 from bk201/wip-48209

mgr/dashboard: fix cephadm e2e test failure on deleting OSDs

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #37908 from dzafman/wip-47930

test: Fix race in TEST_recovery_scrub test

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38043 from jhonxue/wip-fix-bluestore-bitmap-allocator

osd: fix bluestore bitmap allocator calculate wrong last_pos with hint

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #37963 from aclamk/wip-bluestore-no-debug-misc

os/bluestore: Removed bluestore_debug_misc option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #37999 from trociny/wip-48153

os/bluestore: fix "end reached" check in collection_list_legacy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #37630 from zjcmszh/aio_rmxattr

pybind/rados: add Ioctx::aio_rmxattr() and test aio_rmxattr()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37976 from batrick/msg-destructor

messages: use final specifier for destructor

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37992 from rishabh-d-dave/dashboard-qa-skipTest

qa/mgr: mgr_test_case: raise SkipTest instead of calling skipTest()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #37998 from tchaikov/wip-log-mon-cleanup

log,mon: use libfmt to format string and use range-based loop

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #38026 from wjwithagen/wjw-fix-test-mon-build_initial

test/mon: FreeBSD returns ENOENT on resolv error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

test/mon: Accept ENOENT as returned value on FreeBSD

FreeBSD returns ENOENT on resolv error

Tracker: https://tracker.ceph.com/issues/48183
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

doc/rados/operation: add link to scrub_max_interval

add the reference to the per-pool setting when explaing
PG_NOT_DEEP_SCRUBBED.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38082 from adamemerson/wip-anti-nihilism

Asyncening 0: The Yielding Anti-Nihilist

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge PR #38071 into master

* refs/pull/38071/head:
.github: add labeler for cephfs

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/librbd/crypto: Fixup include order to prevent implicit definitions

Clang complains a lot, part of it:
```
/home/jenkins/workspace/ceph-master-compile/src/test/librbd/crypto/test_mock_CryptoObjectDispatch.cc:35:8: error: explicit specialization of 'librbd::io::CopyupRequest<librbd::MockImageCtx>' after instantiation
struct CopyupRequest<librbd::MockImageCtx> {
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/jenkins/workspace/ceph-master-compile/src/librbd/io/ObjectRequest.cc:560:10: note: implicit instantiation first required here
  if (it == image_ctx->copyup_list.end()) {
         ^
/home/jenkins/workspace/ceph-master-compile/src/test/librbd/crypto/test_mock_CryptoObjectDispatch.cc:54:32: error: template specialization requires 'template<>'
        librbd::MockImageCtx>::s_instance = nullptr;
```

fixes: https://github.com/ceph/ceph/pull/37916
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #38067 from dillaman/wip-github-rbd-label

github: add 'rbd' auto-labelling

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38013 from drakonstein/master

rpm,deb: change sudoers file mode to 440

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #38073 from badone/wip-use-right-tunable-for-pg-scrub-doc

doc: Use correct tunable for PG_NOT_SCRUBBED

Reviewed-by: Kefu Chai <kchai@redhat.com>

github/labeler: add more rules to labeler.yml

mainly for the maintainers from the RADOS team.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37883 from diwilli/patch-1

cephadm: Set listen-addresses on alertmanager container

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

Merge pull request #38011 from mgfritch/cephadm-sudo-log-dir

cephadm: verify root privileges before log dir create

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

github/labeler: add "crimson" to labeler.yml

so crimson related changes can be labeled by the labler action.

Signed-off-by: Kefu Chai <kchai@redhat.com>

.github: add labeler for cephfs

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

github: add 'rbd' auto-labelling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: No null_yield in rgw_rest

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: No null_yield in rgw_role

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: no null_yield in rgw_tools

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: No null_yield in rgw_torrent

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: No null_yield in rgw_user

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: No null_yield in rgw_zone

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: No null_yield in RGWOp

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw/services: No null_yield

Null yield is fine at top levels where we know what kind of thread
we're running into, but do not belong in general use functions.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #38057 from zdover23/wip-doc-dev-guide-essentials-source-clean-update

doc/dev: update essentials.rst

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38036 from zdover23/wip-doc-dev-guide-basic-workflow-make-check-nov-2020

doc/dev: improve make check text

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #37977 from dillaman/wip-48145

librbd: update hidden global config when removing pool config override

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #37975 from dang/wip-dang-webdir-empty-object

RGW - Don't create empty obj in webdir

Merge pull request #38060 from rhcs-dashboard/wip-master-auto_labels

github: fix autolabel paths

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>

github: fix autolabel paths

Remove starting / from paths and add missing doc path from dashboard.

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #38055 from tchaikov/wip-doc-conf

doc/rados/configuration: document option types

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

doc/dev: update essentials.rst

This commit changes the noun "backup" to the
intended verb "back up" (this is one of my
technical documentation pet peeves and I couldn't
resist), and it also removes the verb "to nuke"
and replaces it with something less poetic but
more likely to be discoverable in a dictionary by
someone whose English is weak. It also adds a
missing colon.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

pybind/mgr/volumes: use = for equality comparision

should not rely on the internal implementation of Python to check the
equality of two strings.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration: document option types

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados: remove superfluous "::"

this change addresses a regression introduced by
0f2489406a8d4194c159c8777517d20b54e216b5

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: Use correct tunable for PG_NOT_SCRUBBED

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

doc/dev: improve make check text

This commit fixes the references to the
"make check" section in the "notes on
pr make check test" section and improves
the wording of that section so that it
reads a bit more clearly.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #38049 from rhcs-dashboard/wip-master-auto_labels

github: autolabel PRs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38023 from zdover23/wip-doc-dev-guide-essentials-nov-2020

doc/dev: update essentials.rst nov 2020

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev: update essentials.rst

This commit improves the grammar and structure
of essentials.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #37902 from pdvian/wip-clog-health-detail

mon: Log "ceph health detail" periodically in cluster log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

RGW - Don't create empty obj in webdir

Zipper objects shouldn't be empty, but should instead be null.  Fix this
case where parsing wasn't checking for empty object name, and the
website code that uses it.

Fixes: https://tracker.ceph.com/issues/48049
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

github: autolabel PRs

... starting with dashboard ones.

Requires https://github.com/marketplace/actions/labeler

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

osd: fix bluestore bitmap allocator calculated wrong last_pos with hint

Fixes: https://tracker.ceph.com/issues/48214
Signed-off-by: Xue Yantao <jhonxue@tencent.com>

mon/OSDMonitor: remove osd_epochs if osd is marked in or out

this is a cleanup. if we don't take the osd_epochs of osd marked out,
there is no need to keep those osd_epochs around anymore. so, in this
change, once the osd is marked in or out, we just drop its osd_epochs
right away.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38029 from kshtsk/wip-cephadm-bootstrap-remote

qa/task/cephadm: run cephadm only on bootstrap_remote

Merge pull request #36385 from ghyer/ghyer-dashboard

mgr/dashboard: Add 'Dirs' and 'Caps' for filesystems details

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38027 from s0nea/wip-dashboard-fix-doc-link

mgr/dashboard: fix HACKING.rst reference

Reviewed-by: aaSharma14 <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #37993 from votdev/issue_48051_ngx_pipe_function

mgr/dashboard: Use pipe instead of calling function within template

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #37770 from pritha-srivastava/wip-sts-47871

rgw/sts: fix for validating path in role arn in assume role api.

mgr/dashboard: Use pipe instead of calling function within template

This PR introduces the 'ngx-pipe-function' package to call functions in templates.

Fixes: https://tracker.ceph.com/issues/48051
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #37918 from dungdm93/avoiding-relative-paths

mgr/dashboards: avoiding relative paths '../../..'

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: nizamial09 <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: fix cephadm e2e test failure on deleting OSDs

- Adapt the OSD ID column change in this PR:
  https://github.com/ceph/ceph/pull/37506
- Turn off Cypress retries because it makes the failure log misleading.

Fixes: https://tracker.ceph.com/issues/48209
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

rpm,deb: change sudoers file mode to 440

change sudoers file mode to 440 to match recommended defaults.

From the sudoers man page.

> the default file mode is 0440 (read‐able by owner and group, writable
by none).
> The default mode may be changed via the “sudoers_mode” option to the
sudoers
>  Plugin line in the sudo.conf(5) file.

Fixes: https://tracker.ceph.com/issues/48169
Signed-off-by: David Turner <drakonstein@gmail.com>

Merge pull request #37922 from mgfritch/project-kubic

cephadm: install podman from the Kubic project

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38018 from tchaikov/wip-cmake-python

COPYING: note cmake/modules/FindPython* license and copyright

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38028 from tchaikov/wip-crimson-maybe-unused

crimson/os: mark unused variable [[maybe_unused]]

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #35865 from tchaikov/wip-crimson-recovery-qa

qa/suites: add crimson-rados suite

Reviewed-by: Samuel Just <sjust@redhat.com>

client: add a dedicated thread for the Client tick

The Client::timer will remain to be use by others like the Delegation.

Fixes: https://tracker.ceph.com/issues/47842
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #36759 from dzafman/wip-46264

mon: check for mismatched daemon versions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

client: try to renew caps and flush old cap releases

For multiple threads case, if there has more than one threads are
waiting the client_lock, which one to be woke up is not determined.
So in worst case the tick() may be stuck for a long time.

When the MDS daemons are out of resources it will recall them and
tell the client to release the caps, so the clients may surge in
receiving thousands of recall_state requests.

Fixes: https://tracker.ceph.com/issues/47842
Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/tasks/vstart_runner.py: disable the ns unsharing if using localhost

If the -l option is specified in :

$ MDS=3 MON=3 MGR=1 OSD=3 ../src/vstart.sh -n -l

The ceph cluster will use the localhost IP address as the mon host,
and then in the network namespace container the ceph-fuse daemon
couldn't connect to it.

Fixes: https://tracker.ceph.com/issues/45342
Signed-off-by: Xiubo Li <xiubli@redhat.com>

mon: Log "ceph health detail" periodically in cluster log

change mon_health_to_clog_interval from 1_hr -> 10_min to
log health summary or detail more frequently.

Fixes: https://tracker.ceph.com/issues/48042
Signed-off-by: Prashant Dhange <pdhange@redhat.com>

test: Changes based on code review comments

Signed-off-by: David Zafman <dzafman@redhat.com>

doc: Apply suggestions from code review

Co-authored-by: Neha Ojha <nehaojha88@gmail.com>
Co-authored-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>

osd test: Delay reporting until mon_warn_older_version_delay has passed

Move release notes description to 16.0.0 and update
Update documentation

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Temporary fix for alienstore which doesn't support asok commands yet

Signed-off-by: David Zafman <dzafman@redhat.com>

doc: Document health check DAEMON_OLD_VERSION

Signed-off-by: David Zafman <dzafman@redhat.com>

Update Release notes

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Simple test case for version health warning

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Use a config variable debug_version_for_testing

Signed-off-by: David Zafman <dzafman@redhat.com>

mon: Add mds version collector

Signed-off-by: David Zafman <dzafman@redhat.com>

mon: Add mgr version collector

Signed-off-by: David Zafman <dzafman@redhat.com>

mon: Add osd version collector

Signed-off-by: David Zafman <dzafman@redhat.com>

mon: Verify version consistency across daemons

Add a function called check_daemon_versions()
Made minor changes based on pull request comments
Changed the way versions are checked along with some other minor modifications
Made minor modification to the error message
Modifications to include the hostname in the error message

Fixes: https://tracker.ceph.com/issues/46264
Signed-off-by: Tyler <tyler_sheehan@student.uml.edu>
Remove time since health errors aren't preserved if not set every time
Rename health error to DAEMON_OLD_VERSION
Reoganize code so that we can get versions from other daemon types
Change to map of version (key), a list of daemons (value)
Get actual daemon names from get_versions() requests
Fix message output based on how many daemons and how many versions
Add a config to be able to turn off this features
Give HEALTH_ERR if more than one older version exists
Fix minor indentation problem with health detail

Signed-off-by: David Zafman <dzafman@redhat.com>