doc/mgr/diskprediction: add a configuration section

also link to monitoring, as diskprediction cannot function without
enabling monitoring.

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/devicehealth: drop the cloud model of disk prediction

as the service offered by https://www.diskprophet.com/ is not accessible
anymore, the cloud model of disk prediction is not available. so let's
drop the code handling "cloud" model.

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/dashboard: drop test for diskprediction_cloud

this module is removed, so no need to test its settings

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/mgr: drop commented code

the test for diskprediction_cloud is never enabled, and the used
cloud-based service is not reachable anymore. let's just remove the dead
code.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: stop referencing diskprediction_cloud

diskprediction_cloud is removed from master, so no need to have it in
the exclude list anymore

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/mgr/diskprediction.rst: remove references to diskprediction_cloud

since https://www.diskprophet.com is not accessible anymore, and this
feature is not available at the time of writing. let's drop the related
document.

Fixes: https://tracker.ceph.com/issues/46863
Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/diskprediction_cloud: remove mgr-diskprediction-cloud

the service offered by https://www.diskprophet.com is not accessible
anymore. before we have a replacement of it, let's drop
mgr-diskprediction-cloud. and remove its packaging.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34918 from bk201/wip-45397

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36674 from tchaikov/wip-doc-crimson

doc/dev/crimson: note about script/ceph-debug-docker.sh

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #36686 from sebastian-philipp/ceaphdm-upgrade-pytest

mgr/cephadm: Add tests/test_upgrade.py

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #36629 from yison/fix-mem-leak-nvmedevice

blk/spdk/NVMEDevice.cc: fix a memory leak issue

Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge PR #36515 into master

* refs/pull/36515/head:
libcephfs: ignore restoring the open files limit
msg: throw a system error when center.init fails

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36525 into master

* refs/pull/36525/head:
client: switch to use ceph_mutex_is_locked_by_me always

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36552 into master

* refs/pull/36552/head:
mds: fix kcephfs parse dirfrag's ndist is always 0

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #36617 into master

* refs/pull/36617/head:
mds: revert the decode version

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #36605 from rhcs-dashboard/fix-46920-master

mgr/dashboard/api: reduce amount of daemon logs

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #36312 from ishanrai05/bug-1

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/cephadm: also mock `ceph vesions`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: add test_upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm/tests: move with_service to fixtures.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

pybind/mgr: Also mock the config store in pytest

This will be used to dry-run mgr/cephadm/upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36476 from ceph/42072-landing-page

mgr/dashboard: Landing Page improvements

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #36669 from smithfarm/wip-nautilus-11

doc/releases: add Nautilus 14.2.11 to timeline

Reviewed-By: Kefu Chai <kchai@redhat.com>
Reviewed-By: Zac Dover <zac.dover@gmail.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #36508 from mgfritch/cephadm-nfs-rados-pool

mgr/cephadm: check existence of the RADOS pool

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

mgr/dashboard: clear Jest cache before testing

Jest seems to not detect removed files correctly. Let's clean the cache
before performing frontend unit tests in CI.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

mgr/dashboard: leverage features set from orchestrator

Orchestrator provides a list of supported functions. Using
this list to provide better UX and guard for unsupported
features.

- Backend: return 503 if required orchestrator features are
  missing, rather than calling non-implemented methods
  in orchestrator.
- Frontend:
  - Remove information modal when Orchestrator is not available.
  - Disable table action buttons if Orchestrator is not available
    or features are not supported.

Fixes: https://tracker.ceph.com/issues/45397
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #36228 from vosdev/doc-mgr-crash-typo

doc/mgr/crash: Add missing command in rm example

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36623 from cheese/master

common: document the current semantics of bl::append(std::istream)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36418 from changchengx/osd_map

osd/OSDMap: corret osd_info_t last_clean comment

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/dev/crimson: note about script/ceph-debug-docker.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36642 from tchaikov/wip-crimson-silence-warnings

crimson: silence warnings from clang++

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: do not capture unused variables

this change silences warnings like

src/crimson/osd/pg_backend.cc:232:34: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
        [&os, &osd_op, &extents, this](auto&& bl) -> read_errorator::future<> {

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: do not return a value for non-reachable branch

for silencing a warning like

btree/btree_lba_manager.cc:177:12: warning: binding dereferenced null pointer to reference has undefined behavior [-Wnull-dereference]
    return *static_cast<btree_range_pin_t*>(nullptr);
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ceph_abort_msg() is marked [[noreturn]], so we can just use it here
without worrying about complains from compiler.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/releases: add Nautilus 14.2.11 to timeline

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #36479 from dillaman/wip-46669

test/rbd-mirror: pool watcher registration error might result in race

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #36639 from tchaikov/wip-crimson-build-push-op

crimson/osd: clear data_included if fiemap fails

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #36505 from rzarzynski/wip-crimson-bump-up-seastar

crimson: bump up seastar version to freshest master

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36529 from tchaikov/wip-crimson-dts-9

ceph.spec.in: use devtoolset-9 for building crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

install-deps.sh: add tchaikov's copr repo for crimson build on el8

we need it before the updated version of gcc-toolset9 addressing
https://bugzilla.redhat.com/show_bug.cgi?id=1853900 is available in
RHEL8/CentOS8.

see https://copr.fedorainfracloud.org/coprs/tchaikov/gcc-toolset-9

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.spec.in: use gcc-toolset-9 for building crimson

since seastar dropped support from C++14, we have to move to a compiler
with a decent C++17 support.

in this change, gcc-toolset-9 is used for compiling ceph if seastar is
enabled. use version >= 9.2.1-2.2, because of
https://bugzilla.redhat.com/show_bug.cgi?id=1853900

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: clear data_included if read fails

* extract read_object_for_push_op() out of build_push_op().
  build_push_op() is composed of multiple steps. would be better
  if we can split and extract them into separated method for better
  readability
* clear push_op->data_included() if fiemap fails. before this change,
  a local copy of `copy_subset` is cleared if fiemap fails, but
  push_op->data_included is not updated in this case. we should
  clear the latter if no data is returned in push_op->data. so,
  in this change, push_op->data_included is cleared if fiemap
  fails. the same applies to other cases where no data is returned in
  push_op
* add comments to outline the steps in read_object_for_push_op().

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: print detailed info of object being recovered

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: remove stale comment

CyanStore supports readv() now, so remove this comment.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: do not use __func__ in a lambda

otherwise we would have logging messages like

operator(): returning obc ...

Signed-off-by: Kefu Chai <kchai@redhat.com>

blk/spdk/NVMEDevice.cc: fix a memory leak issue

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

Merge pull request #36618 from tspmelo/wip-backport-jq

script/ceph-backport.sh: Check if jq is installed

Merge pull request #36609 from tspmelo/wip-doc-browsers

mgr/dashboard: Update list of supported browsers

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36624 from lixiaoy1/fix_config

rbd/rwl: fix the config issue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36466 from lixiaoy1/rbd_image_info_br

rbd/status: get image cache state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36422 from orozery/object_dispatch_assert_version_write

librbd: support writing with version assertion on object dispatch

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36366 from runsisi/wip-fix-free

librbd: fix possible use after free error while shuting down exclusive lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36561 from lixiaoy1/fix_queue

rbd/rwl: fix the construction of ContextWQ

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Update list of supported browsers

Fixes: https://tracker.ceph.com/issues/46923
Signed-off-by: Tiago Melo <tmelo@suse.com>

rbd/rwl: fix the config issue

Include acconfig.h to the librbd/cache/Util.h.

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>

script/ceph-backport.sh: Check if jq is installed

If you tried to setup ceph-backport.sh without having 'jq' installed, you would
get a totally unrelated error message and it was hard to figure out the problem.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #36604 from tspmelo/wip-favicon-circle

mgr/dashboard: Fix favicon white circle

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36602 from tspmelo/wip-spell-check

mgr/dashboard: Fix spell errors

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #36585 from votdev/proper_format_portals

mgr/dashboard: Proper format iSCSI target portals

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #36566 from votdev/issue_46900_autocomplete

mgr/dashboard: Strange iSCSI discovery auth behavior

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #36567 from votdev/issue_46901_hide_limit_action_field

mgr/dashboard: Hide table action input field if limit=0

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

common: document the current semantics of bl::append(std::istream)

bl::append(std::istream) comes with a surprise implementation. People may think
it would just slurp the indentical binary data from the stream. Document the
trap to let people don't mis-use it.

Signed-off-by: Robin Lee <cheeselee@fedoraproject.org>

Merge pull request #36510 from athanatos/sjust/wip-tm-decref

crimson: remove extent from cache in TransactionManager::dec_ref

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore/transaction_manager: complete dec_ref

Previously, dec_ref didn't handle actually retiring the
extent from the cache.  dec_ref will now reach into the
cache and mark the extent retired if it exists either
in the cache or in the current transaction.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/lba_manager: clarify ref count operation return

Previously, we returned a refcount from inc_ref and dec_ref.  Now,
return the paddr as well for future code accounting for released
extents.

In addition, replumb btree_lba_manager to return an enoent error if
the mapping does not exist, and the resulting refcount, paddr
otherwise with a refcount of 0 indicating that the mapping has
been removed.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/cache: don't mark invalid extents clean

In the event that an extent is created and removed in the same
transaction (invalid extent in fresh_block_list), update
block specific metadata but don't add to cache and especially
do not mark clean.  LBAManager::complete_commit implementations
are meant to use skip based on state in the same way.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/cached_extent: add print_detail for logical extents

This way we can always see laddr in debug output.

Signed-off-by: Samuel Just <sjust@redhat.com>

mds: revert the decode version

Introduced in
https://github.com/ceph/ceph/commit/3fac3b1236c4918e9640e38fe7f5f59efc0a23b9,
the decode changes are reverted but the version number is not.

Fixes: https://tracker.ceph.com/issues/46926
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #36591 from idryomov/wip-snap-create-progress

qa: update krbd tests for snap create

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36434 from ricardoasmarques/fix-iscsi-logged-in-client-edit

mgr/dashboard: Unable to edit iSCSI logged-in client

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36564 from ceph/46899-log-info-from-unhandled-errors

mgr/dashboard: log useful information from internal server errors

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Fabrizio D'Angelo <fdangelo@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/dashboard: Landing Page improvements

Fixes: https://tracker.ceph.com/issues/42072
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

mgr/dashboard/api: reduce amount of daemon logs

Fixes: https://tracker.ceph.com/issues/46920
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: Fix favicon white circle

A white circle was displayed when the user logged out.

Fixes: https://tracker.ceph.com/issues/46919
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix spell errors

Fixes: https://tracker.ceph.com/issues/46917
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: fix error message for the confirmpassword

fixes: https://tracker.ceph.com/issues/46494

Signed-off-by: Ishan Rai <ishanrai05@gmail.com>

mgr/dashboard: remove password fields, if the login is using SSO

fixes: https://tracker.ceph.com/issues/46494

Signed-off-by: Ishan Rai <ishanrai05@gmail.com>

mgr/dashboard: Proper format iSCSI target portals

Proper format the 'Portals' and 'Images' columns.

Fixes: https://tracker.ceph.com/issues/46908
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #36532 from simon-rock/crash_on_restart

mds : move start_files_to_recover() to recovery_done

mds : move start_files_to_recover()  to recovery_done

the requests in the queue, named waiting_for_replay, may modify the state of the filelock,
resulting in the wrong lock state when repairing file (start_files_to_recover)

Fixes : https://tracker.ceph.com/issues/46906
Signed-off-by: Simon Gao <simon29rock@gmail.com>

Merge pull request #36119 from votdev/issue_44831_deploy_service

mgr/dashboard: Create Ceph services via Orchestrator by using ServiceSpec

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

qa: update krbd tests for snap create

"rbd snap create" now reports progress.  Pass --no-progress, as in
commit b5a5fea9e2b4 ("test/cli-integration/rbd: tweak after snap create
started to show progress").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #36569 from yuriw/wip-yuriw-crontab-master

qa/tests: use "-k distro" for all suites (except krbd)

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36570 from theanalyst/doc/releases/v14.2.11

doc: releases: notes for v14.2.11 nautilus

Reviewed-By: Neha Ojha <nojha@redhat.com>
Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Patrick Donelly <pdonell@redhat.com>

rbd/rwl: fix the construction of ContextWQ

Fix the build issue caused by the commit c11594a which changed the constructor
of ContextWQ.

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>

mgr/dashboard: log useful information from internal server errors

Fixes: https://tracker.ceph.com/issues/46899
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

rbd/status: get image cache state

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

mgr/dashboard: Strange iSCSI discovery auth behavior

Disable 'autocomplete' for the user and password form fields.

Fixes: https://tracker.ceph.com/issues/46900
Signed-off-by: Volker Theile <vtheile@suse.com>

doc: releases: notes for v14.2.11 nautilus

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #36550 from tspmelo/wip-npm-update

mgr/dashboard: Update npm packages

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #36565 from jan--f/c-v-py3-flake8

ceph-volume: run flake8 in python3

Merge PR #36351 into master

* refs/pull/36351/head:
qa/tasks/cephfs/nfs: Add tests for cluster config set and reset
doc/cephfs/nfs: Update the doc about 'reset' and 'set' config interfaces
mgr/volumes/nfs: Add interface for adding user defined configuration

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/tests: use "-k distro" for all suites (except krbd)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #36437 from yuriw/wip-yuriw-crontab-master

qa/tests: changes related to #36435 and #36436

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #36111 from vosdev/patch-2

doc/mgr/dashboard: Fix haproxy example to use ssl for backends

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Hide table action input field if limit=0

Fixes: https://tracker.ceph.com/issues/46901
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #36456 from sebastian-philipp/orch-type-py3

mgr/orch: _interface.py cleanup: PEP8 and type annotations

Reviewed-by: Joshua Schmid <jschmid@suse.de>

ceph-volume: run flake8 in pyhton3

Fixes: https://tracker.ceph.com/issues/46897
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #36431 from rishabh-d-dave/cv-add-tests-for-new-lvm-functions

ceph-volume: add tests for new functions that run LVM commands

Merge pull request #35937 from rishabh-d-dave/cv-rm-container-classes

ceph-volume: remove container classes from api/lvm.py

Merge pull request #36425 from majianpeng/rocksdb-remove-redundant-perfcount

kv/RocksdbStore: remove perfcounter l_rocksdb_txns/l_rocksdb_txns_sync

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>