ceph-volume: add mock dependency to tox.ini

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

 ceph-volume: python2 raises OSError on Popen with missing binary.

For compatibility alias FileNotFoundError to OSError.

Fixes: 72b79f71fa84fe0d56b2fd1745f5bb130380cc38
Fixes: https://tracker.ceph.com/issues/42967
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 12c5f89ddb51a1db9399e03591e1db6c429b9c9e)

ceph-volume: py2 compatibility for selinux test

Fixes: af30c7f956380beb3bc4021d60b9229a82bda4e2
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 114aa57df28107c2d14caacdfb3f0423b244075d)

ceph-volume: don't assume SELinux

Expect selinux related binaries to not be present.

Fixes: 33c8a64a54d9ea8962091caf8564cea3f603c5f5
Fixes: https://tracker.ceph.com/issues/42957
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 72b79f71fa84fe0d56b2fd1745f5bb130380cc38)

ceph-volume: fix test test_selinuxenabled_doesnt_exist

Popen does not open a shell by default, so A FileNotFoundError is
raised instead of returning a 127 return code.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit af30c7f956380beb3bc4021d60b9229a82bda4e2)

Merge PR #30521 into nautilus

* refs/pull/30521/head:
qa: have kclient tests use new mount.ceph functionality
doc: document that the kcephfs mount helper will search keyring files for secrets
mount.ceph: fork a child to get info from local configuration
mount.ceph: track mon string and path inside ceph_mount_info
mount.ceph: add name and secret to ceph_mount_info
mount.ceph: add ceph_mount_info structure
mount.ceph: clean up debugging output and error messages
mount.ceph: clean up return codes
mount.ceph: add comment explaining why we need to modprobe
mount.ceph: use bools for flags
common: have read_secret_from_file return negative error codes

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #31733 from epuertat/wip-42677-nautilus

nautilus: mgr/dashboard: Fix grafana dashboards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30229 from vumrao/wip-vumrao-bluefs-shared-alloc-with-log-level-change-nautilus

nautilus: os/bluestore: apply shared_alloc_size to shared device with log level change

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #31704 from smithfarm/wip-42858-nautilus

nautilus: mon/PGMap: fix incorrect pg_pool_sum when delete pool

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31676 from tchaikov/nautilus-42832

nautilus: ceph.in: do not preload ASan unless necessary

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31526 from ricardoasmarques/wip-42730-nautilus

nautilus: mgr/dashboard: tasks: only unblock controller thread after TaskManager thread

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30910 from rjfd/wip-42283-nautilus

nautilus: mgr/dashboard: internationalization support with AOT enabled

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #30685 from ifed01/wip-ifed-fast-fsck-nau

nautilus: os/bluestore: shallow fsck mode and legacy statfs auto repair.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #31742 from smithfarm/wip-42836-nautilus

nautilus: rgw: Silence warning: control reaches end of non-void function

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

rgw: drop getting list-type when get_data is false

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit ae56a918174cc6899bf7384004560ea57036b009)

rgw: Silence warning: control reaches end of non-void function

Build shows:
[ 53%] Building CXX object src/mds/CMakeFiles/mds.dir/JournalPointer.cc.o
ceph/src/rgw/rgw_rest_s3.cc: In member function ‘RGWOp* RGWHandler_REST_Bucket_S3::get_obj_op(bool)’:
ceph/src/rgw/rgw_rest_s3.cc:3588:5: warning: control reaches end of non-void function [-Wreturn-type]
   }   }
     ^
Fixes: 4ffc765c4c5debc665ade7769c4647c3a7278fd2
Fixes: http://tracker.ceph.com/issues/40747
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit abb2451dd5164e6b610589207b900a6464e21282)

Merge pull request #31522 from ricardoasmarques/wip-42729-nautilus

nautilus: mgr/dashboard: RBD tests must use pools with power-of-two pg_n…

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #31516 from rhcs-dashboard/42694-filter-non-pool-fields

nautilus: mgr/dashboard: do not show non-pool data in pool details

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31413 from smithfarm/wip-41980-nautilus

nautilus: mgr/dashboard: do not log tokens

Merge pull request #31375 from tspmelo/wip-42150-nautilus

nautilus: mgr/dashboard: Configuring an URL prefix does not work as expected

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31349 from rhcs-dashboard/42589-edit-image-after-data-received

nautilus: mgr/dashboard: edit/clone/copy rbd image after its data is received

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31263 from ricardoasmarques/wip-42295-nautilus

nautilus: mgr/dashboard: Provide the name of the object being deleted

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31160 from Exotelis/wip-42482-nautilus

nautilus: mgr/dashboard: Add transifex-i18ntool

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1
Fixes: https://tracker.ceph.com/issues/41399
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
(cherry picked from commit 96d48af3252f0ec2c1864383ad5eee993ecfeb4f)

os/bluestore/BlueFS: apply shared_alloc_size to shared device

Keep an alloc_size vector so that we have this value handy at all times.
Allow bluestore to fetch this value directly instead of looking at the
bluefs_* config options since this encapsulates things a bit better, and
also isn't vulnerable to the config setting changing at runtime.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e8b5a458c306dfe166b09239fc9f57c04fa47ead)

os/bluestore/BlueFS: fix device_migrate_to_* to handle varying alloc sizes

The previous implementation moved extents individually.  This caused
problems when moving an extent with a small alloc_size that wasn't
a multiple of the target device's alloc_size.

Instead, identify files with extents that need to be moved, and then read
the file in its entirety and rewrite it in its entirety.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 94269741959a4280225b6330dfb015f1799fd1f2)

os/bluestore: whitespace

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 99c56559d17fac4fde65ef38a82ac6cab090dafb)

os/bluestore: cleanup around allocator calls

Both stupid and bitmap allocator returs -ENOSPC if they're
unable to allocate any space. Existing callers aren't always
respect this - hence doing some cleanup.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit a1246da5705e2a699656c2561cebc5ed29628c90)

os/bluestore/BlueFS: add bluefs_shared_alloc_size

Add a separate config option that controls the alloc_size for the shared
device (BDEV_SLOW).

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c69c9539f628c61a1cf4cae6aadd39cb48a8b022)

os/bluestore/BlueStore.cc: start should be >= _get_ondisk_reserved()

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 27aa0db8e57bffd61225f2c4e73f596c7399d2b1)

mgr/dashboard: fix grafana dashboards

Fixes: https://tracker.ceph.com/issues/42542
Sort order was wrong for some dashboards,
fixed empty / buggy Top 3 clients IOPS by pool / Throughput - in Pools
Overall performance
fixed Avg utilization Multiple series found - in Host Overall
performance
Fixed invalid dimensions for plot - in OSD Overall performance

Signed-off-by: Radu Toader <radu.m.toader@gmail.com>
(cherry picked from commit 3beaf63761590fd35a72777fc56c0d5f4a6cad3b)

nautilus: mgr/dashboard: Fix data point alignment in MDS count… (#31535)

nautilus: mgr/dashboard: Fix data point alignment in MDS counters chart

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #31658 from smithfarm/wip-42731-nautilus

nautilus: mon/MonMap: encode (more) valid compat monmap when we have v2-only addrs

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #31604 from smithfarm/wip-42782-nautilus

nautilus: tests: test_librados_build.sh: grab from nautilus branch in nautilus

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31444 from dzafman/wip-41785

nautilus: core: Make dumping of reservation info congruent between scrub and recovery

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31261 from ceph/wip-nautilus-restful-node-items

nautilus: restful: Query nodes_by_id for items

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #31125 from smithfarm/wip-42439-nautilus

nautilus: build/ops: rpm: make librados2, libcephfs2 own (create) /etc/ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31011 from smithfarm/wip-42401-nautilus

nautilus: cmake: Allow cephfs and ceph-mds to be build when building on FreeBSD

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30740 from smithfarm/wip-41463-nautilus

nautilus: ceph-objectstore-tool: update-mon-db: do not fail if incmap is missing

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31682 from dzafman/wip-42432-nautilus

nautilus: mgr: Release GIL before calling OSDMap::calc_pg_upmaps()

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31077 from smithfarm/wip-42141-nautilus

nautilus: core: osd/PGLog: persist num_objects_missing for replicas when peering is done

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30900 from smithfarm/wip-42136-nautilus

nautilus: osd: Remove unused osdmap flags full, nearfull from output

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31136 from wjwithagen/wip-fixup-cephfs-nautilus

nautilus: test/{fs,cephfs}: Get libcephfs and cephfs to compile with FreeBSD

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #30951 from sidharthanup/mds-evict-duplicate-nautilus

nautilus: cephfs: mds: Fix duplicate client entries in eviction list

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #30043 from smithfarm/wip-41488-nautilus

nautilus: cephfs: client: return -eio when sync file which unsafe reqs have been dropped

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #29832 from pdvian/wip-41095-nautilus

nautilus: cephfs: qa: wait for MDS to come back after removing it

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #29811 from pdvian/wip-41093-nautilus

nautilus: cephfs: qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #29750 from pdvian/wip-41087-nautilus

nautilus: qa: sleep briefly after resetting kclient

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #31650 from uweigand/nautilus-backport-28013

nautilus: rgw: use explicit to_string() overload for boost::string_ref

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #31079 from smithfarm/wip-42155-nautilus

nautilus: cephfs: mds: fix infinite loop in Locker::file_update_finish

Reviewed-by: Yan, Zheng <zyan@redhat.com>

ceph.in: normalize BOOL values found by get_cmake_variables()

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 44cb44425ce19ee293a0e0f012229cf2389ac271)

os/bluestore: test conversion from global statfs

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 923becc88257a70604b313a3ea0419a1024c62d2)

os/bluestore: simplify per-pool-stat config options

The previous bluestore_no_per_pool_stats_tolerance had a lot of possible
values, not all of which make sense to users.  Replace with a single new
option, bluestore_fsck_error_on_no_per_pool_stats, which controls whether
the lack of per-pool stats is an error or a warning.  On repair, we will
unconditionally convert to per-pool stats.

This brings us in sync with the newer
bluestore_fsck_error_on_no_per_pool_omap.

Note that one part of the ceph_test_objectstore test is dropped since it
is no longer possible to create a store with legacy stats.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7e96024254d98591a5431623fb355a20f322b554)

 Conflicts:
PendingReleaseNotes
        - trivial
src/os/bluestore/BlueStore.cc
        - lacking of int_repaired method

os/bluestore: fix invalid stray shared blob detection in fsck.

This is a regression, caused by https://github.com/ceph/ceph/pull/30264

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 3f79b1976885880a188ed0a7f5e7d40dd554dc15)

os/bluestore: fix warning counting in fsck

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit d3308d2cc17b0a12480e5dce850d2bda84985add)

os/bluestore: remove duplicate logging

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit f1346a0ce1e9c35d44d016380c6a962d23288de4)

os/bluestore: introduce 'fix_on_mount' mode for no per pool tolerance.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 52af6cda58409d4e90363d1bd741485683f54395)

 Conflicts:
src/os/bluestore/BlueStore.cc
        - trivial

os/bluestore: parallelize quick-fix running.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit c432f2268db60eb701fd9b17618537513f5d4eea)

 Conflicts:
src/os/bluestore/BlueStore.cc
        - trivial

common/work_queue: make ThreadPool/WorkQueue more reusable.

Some tricks to be able to derive from ThreadPool.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit e72801074c9d464044be3e981728c99618797984)

os/bluestore: pass containers by ptr in FSCK_ObjectCtx

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 799d54125584c2f7cb42d110b53b1e33e7b9bacb)

 Conflicts:
src/os/bluestore/BlueStore.cc
        - lacking per-pool omap

os/bluestore: another split out for fsck logic blocks.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit c6ab317fff7e28a938de40644d714a51bdb73bf3)

os/bluestore: reorder blocks within fsck_check_objects.

The rationale is to to have better locality for shallow checks which can
later run in a multithreaded manner.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit d63fc8f86e0609ad03c9679a42b7dd3021e3111d)

 Conflicts:
src/os/bluestore/BlueStore.cc
        - lacking per-pool omap

os/bluestore: split out object checking from _fsck.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit d713d4b6eedb50a0923c21bd173509b3f011caed)

 Conflicts:
src/os/bluestore/BlueStore.cc
src/os/bluestore/BlueStore.h
        - lacking warnings and per-pool omap support

os/bluestore: fsck: int64_t for error count

Just to be on the safe side.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dee8f8cb7e906596693f8c6e22ff74f815169c87)

 Conflicts:
src/os/bluestore/BlueStore.cc
        - trivial

os/bluestore: introduce shallow (quick-fix) mode for bluestore fsck/repair.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ca2c4e5d6956a32a7ce83f28a01c825dde1ee117)

 Conflicts:
src/os/bluestore/BlueStore.cc
        - caused by lack of perpool omap support
src/os/bluestore/bluestore_tool.cc
        - lacking allocator fragmentation analysis commands

os/bluestore: do not collect onodes when doing fsck.

This both saves RAM and cpu cycles making fsck process faster.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 4b23bae6a2412ad752543d2aa5e0f7bf053306d2)

 Conflicts:
src/os/bluestore/BlueStore.cc
 std::shared_lock vs. RWLock::RLocker mismatch

mon/PGMap: fix incorrect pg_pool_sum when delete pool

We found the pools num diplayed by "ceph -s" is not the same with
"ceph osd lspools" after deleting a pool sometime. The result is
Mgr ClusterState::ingest_pgstats get the old pg_stat which pg is
not deleted in some osd before the pool deleted and add to
pending_inc.pool_statfs_updates. The deleted pool will be added to
pg_pool_sum unconsciously by PGMap::apply_incremental and which has
been deleted in OSDMap. This will also casue MON's Segmentation
fault.

Fixes: https://tracker.ceph.com/issues/42689
Fixes: https://tracker.ceph.com/issues/42592
Fixes: https://tracker.ceph.com/issues/41228
Fixes: https://tracker.ceph.com/issues/40011
Signed-off-by: luo rixin <luorixin@huawei.com>
(cherry picked from commit 446eca8defddda44fea7c789065afdb1a9d38dae)

Merge pull request #30144 from ifed01/wip-ifed-gc-blobs-nau

nautilus: os/bluestore: apply garbage collection against excessive blob count growth

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31397 from jan--f/wip-41460-nautilus

nautilus: os/bluestore/KernelDevice: fix RW_IO_MAX constant

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30774 from smithfarm/wip-42144-nautilus

nautilus: mgr/prometheus: Fix KeyError in get_mgr_status

Merge pull request #30624 from tchaikov/wip-nautilus-41983

nautilus: mgr: set hostname in DeviceState::set_metadata()

Merge pull request #30622 from smithfarm/wip-42083-nautilus

nautilus: pybind/rados: fix set_omap() crash on py3

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #30390 from pdvian/wip-41804-nautilus

nautilus: mgr: do not reset reported if a new metric is not collected

Merge pull request #30370 from pdvian/wip-41766-nautilus

nautilus: ceph.spec.in: reserve 2500MB per build job

Merge pull request #30360 from yuvalif/allow_gcc9_compilation

 nautilus: build/ops: cmake/BuildDPDK: ignore gcc8/9 warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #30261 from smithfarm/wip-41724-nautilus

nautilus: build/ops: fix build fail related to PYTHON_EXECUTABLE variable

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30215 from sebastian-philipp/nautilus-k8sevents

nautilus: mgr/k8sevents: Initial ceph -> k8s events integration

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>

Merge pull request #30016 from smithfarm/wip-41509-nautilus

nautilus: mgr/zabbix: encode string for Python 3 compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30006 from smithfarm/wip-40043-nautilus

nautilus: mgr/zabbix: Fix typo in key name for PGs in backfill_wait state

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29949 from smithfarm/wip-41282-nautilus

nautilus: bluestore: tool to check fragmentation

Merge pull request #30046 from smithfarm/wip-40270-nautilus

nautilus: build/ops: make patch build dependency explicit

Merge pull request #30089 from tchaikov/wip-nautilus-gcc-8

nautilus: build/ops: use gcc-8

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #30008 into nautilus

* refs/pull/30008/head:
mgr/pg_autoscaler: fix race with pool deletion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #30851 into nautilus

* refs/pull/30851/head:
doc/mgr/crash: document missing commands, options
qa/suites/rados/singleton/all/test-crash: whitelist RECENT_CRASH
qa/suites/rados/mgr/tasks/insights: whitelist RECENT_CRASH
qa/tasks/mgr/test_insights: crash module now rejects bad crash reports
mgr/crash: don't make these methods static
mgr/BaseMgrModule: handle unicode health detail strings
mgr/crash: verify timestamp is valid
qa/suites/mgr: whitelist RECENT_CRASH
mgr/crash: remove unused var
mgr/crash: remove unused import 'six'
qa/workunits/rados/test_crash: health check
mgr/crash: improve validation on post
mgr/crash: automatically prune old crashes after a year
mgr/crash: raise RECENT_CRASH warning for recent (new) crashes
mgr/crash: add 'crash ls-new'
mgr/crash: add option and serve infra
mgr/crash: keep copy of crashes in memory
mgr/pg_autoscaler: adjust style to match built-in tables
mgr/crash: make 'crash ls' a nice table with a NEW column
mgr/crash: nicely format 'crash info' output
mgr/crash: add 'crash archive <id>', 'crash archive-all' commands

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #30849 into nautilus

* refs/pull/30849/head:
mgr/dashboard: fix mgr module API tests
qa/tasks/mgr/dashboard/test_mgr_module: remove enable/disable test from MgrModuleTelemetryTest
qa/tasks/mgr/dashboard/test_mgr_module: sync w/ telemetry
mgr/dashboard/qa: add more fields to report
Merge branch 'nautilus' into wip-device-telemetry-nautilus
PendingReleaseNotes: fix typo
PendingReleaseNotes: remove kludge
mgr/telemetry: add stats about crush map
mgr/telemetry: add rgw metadata
mgr/telemetry: include fs size (files, bytes, snaps)
mds: report r{files,bytes,snaps} via perfcounters
mgr/telemetry: mds cache stats
mgr/telemetry: add some rbd metadata
mgr/telemetry: note whether osd cluster_network is in use
mgr/telemetry: add host counts
mgr/telemetry: add more pool metadata
mgr/telemetry: remove crush rule name
mgr/telemetry: include min_mon_release and msgr v1 vs v2 addr count
mgr/telemetry: add CephFS metadata
mgr/telemetry: include balancer info (active=true/false, mode)
mgr/telemetry: include per-pool pg_autoscale info
mgr/telemetry: dict.pop() errs on nonexistent key
mgr/telemetry: send device telemetry via per-host POST to device endpoint
mgr/telemetry: fix remote into crash do_ls()
mgr/telemetry: clear the event after being awaken by it
mgr/telemetry: bump content revision and add a release note
telemetry/server: add device report endpoint
mgr/telemetry: include device telemetry
mgr/telemetry: salt osd ids too
mgr/telemetry: obscure entity_name with a salt
mgr/telemetry: force re-opt-in if the report contents change
mgr/telemetry: less noise in the log
mgr/telemetry: wake up serve on config change
mgr/telemetry: track telemetry report revisions
qa/tasks/mgr/dashboard/test_mgr_module: adjust expected schema
mgr/telemetry: separate out cluster config vs running daemons
mgr/telemetry: include any config options that are customized
mgr/telemetry: specify license when opting in
doc/mgr/telemetry: update
mgr/telemetry: move contact info to an 'ident' channel
mgr/telemetry: accept channel list to 'telemetry show'
mgr/telemetry: always generate new report for 'telemetry show'
mgr/telemetry: add 'device' channel and call out to devicehealth module
mgr/telemetry: add telemetry channel 'device'
mgr/telemetry: add separate channels
mgr/devicehealth: pull out MAX_SAMPLES

Reviewed-by: Wido den Hollander <wido@42on.com>

Merge pull request #31605 from rhcs-dashboard/42795-fixtypo

nautilus: common: fix typo in rgw_user_max_buckets option long description.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31334 from LongDuncan/nautilus

nautilus: mgr/restful: requests api adds support multiple commands

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #31304 from smithfarm/wip-42545-nautilus

nautilus: qa/tasks/cbt: run stop-all.sh while shutting down

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31040 from smithfarm/wip-41495-nautilus

nautilus: tests: add debugging failed osd-release setting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #31019 from ifed01/wip-42041-nautilus

nautilus: core: os/bluestore: fix objectstore_blackhole read-after-write

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30983 from tchaikov/wip-nautilus-42363

nautilus: build/ops: python3-cephfs should provide python36-cephfs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #31097 from smithfarm/wip-42395-nautilus

nautilus: common/ceph_context: avoid unnecessary wait during service thread shutdown

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29592 from penglaiyxy/wip-simple-messenger

nautilus: msg/simple: reset in_seq_acked to zero when session is reset

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix mgr module API tests

Signed-off-by: alfonsomthd <almartin@redhat.com>
(cherry picked from commit 6e6711a1e217042390b379f98c0ec064ab4e2702)

qa/tasks/mgr/dashboard/test_mgr_module: remove enable/disable test from MgrModuleTelemetryTest

telemetry is always enabled since 2d62d71cd46d791294b3803e034c0995fef08b33

Fixes: https://tracker.ceph.com/issues/41186
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cc3ae85b05b759fd6ac955d2e798d5fe3a109294)

qa/tasks/mgr/dashboard/test_mgr_module: sync w/ telemetry

* use primitive types instead of `JLeaf(the_type)` as they are
  equivalent in this context
* remove fields which are added only if certain channels are
  activated.
* allow unknown fields, as we are including various stuff
  in the report, for instance, osdmap, usage, crash info, etc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3303116f5681f317dee53988e1de9d840d250884)

mgr/dashboard/qa: add more fields to report

to be in sync with #28847

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d6170e70b6d1d634be6af5540d85572a9c425d0c)

mgr: Improve balancer module status

Add balancer status fields so that slow optimizations can be detected

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f04c505b9dc128bdfbcca49d82f8aac4abf8b56f)

test: Test balancer module commands

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3a0e2c8ff1cce44bf44b139572f4d1bd9bbfb766)

mgr: Release GIL before calling OSDMap::calc_pg_upmaps()

Prevent optimize and execute commands from running with active balancer

Fixes: https://tracker.ceph.com/issues/42432
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e2a35e8c8e9f381a635ad0ca01326a2c512590a9)