ceph-volume: pass *-slots arguments to LV creation

Fixes: https://tracker.ceph.com/issues/44494
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 61f0c3af0b0f36a03748733b16de1b11048342e6)

use extent count for slots conversion instead of free count

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 4a81b4a2fbb4cafe45a8fe8cceeece636e51adb6)

ceph-volume: available_lvm: vg space takes precedence

This changes available_lvm to check for generic reasons only if no VGs
were found. A VG can contain a (mounted) lv, which triggers the
ro/locked test, despite the VG having space available.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c6d71eb0022935731bb9380098c975d9c5f27a86)

Merge pull request #34282 from votdev/issue_44809_select_tab

nautilus: mgr/dashboard: Tabs does not handle click events

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34424 from smithfarm/wip-44838-nautilus

nautilus: pybind/rbd: ensure image is open before permitting operations

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #34618 from neha-ojha/wip-45135-nautilus

nautilus: mon: disable min pg per osd warning

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34520 from smithfarm/wip-45060-nautilus

nautilus: qa/suites/rados/rest: don't pass empty dict as data arg

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #34326 from rzarzynski/wip-bug-24995-nautilus

nautilus: mgr: synchronize ClusterState's health and mon_status.

Reviewed-by: Neha Ojha <nojha@redhat.com>

mon: disable min pg per osd warning

Now that the pg_autoscaler is on by default, it is "normal" (and okay) to
have a small number of PGs in the cluster if the overall cluster usage is
also low.  This setting just results in a health warning out of the box
when you create a pool and haven't written any data yet.

Fixes: https://tracker.ceph.com/issues/41735
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1ac34a5ea3d1aca299b02e574b295dd4bf6167f4)

Merge pull request #34343 from Vicente-Cheng/wip-44480-nautilus

nautilus: mds: just delete MDSIOContextBase during shutdown

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #34308 from Vicente-Cheng/wip-44328-nautilus

nautilus: client: fix bad error handling in lseek SEEK_HOLE / SEEK_DATA

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34307 from Vicente-Cheng/wip-44337-nautilus

nautilus: mds: handle bad purge queue item encoding

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

mgr/dashboard: Update bootstrap

Fixes: http://tracker.ceph.com/issues/39326
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 07f28050dba61b851f149a2ed3b63cf5349c04cd)

nautilus: mgr/dashboard: Tabs does not handle click events
Fixes: https://tracker.ceph.com/issues/44809
Conflicts:
src/pybind/mgr/dashboard/frontend/angular.json
src/pybind/mgr/dashboard/frontend/package-lock.json
src/pybind/mgr/dashboard/frontend/package.json
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/iscsi-target-details/iscsi-target-details.component.spec.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/iscsi-target-details/iscsi-target-details.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/iscsi-target-form/iscsi-target-form.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/iscsi-target-image-settings-modal/iscsi-target-image-settings-modal.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/iscsi-target-iqn-settings-modal/iscsi-target-iqn-settings-modal.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/iscsi-target-list/iscsi-target-list.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-snapshot-form/rbd-snapshot-form.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-trash-list/rbd-trash-list.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/configuration/configuration-form/configuration-form.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/configuration/configuration.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.spec.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/logs/logs.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/osd/osd-list/osd-list.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/nfs/nfs-form-client/nfs-form-client.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/nfs/nfs-form/nfs-form.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/pool/pool-form/pool-form.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-user-details/rgw-user-details.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-user-form/rgw-user-form.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/auth/sso/sso-not-found/sso-not-found.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/auth/user-form/user-form.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/administration/administration.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/dashboard-help/dashboard-help.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/identity/identity.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/notifications/notifications.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/task-manager/task-manager.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/not-found/not-found.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/components/critical-confirmation-modal/critical-confirmation-modal.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/components/critical-confirmation-modal/critical-confirmation-modal.component.scss
src/pybind/mgr/dashboard/frontend/src/app/shared/components/grafana/grafana.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/components/loading-panel/loading-panel.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/components/select-badges/select-badges.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/components/submit-button/submit-button.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table-actions/table-actions.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table-actions/table-actions.component.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table-key-value/table-key-value.component.spec.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table/table.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table/table.component.spec.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/models/cd-notification.ts
src/pybind/mgr/dashboard/frontend/src/styles.scss
- all those files are not needed to fix the tab select issue.

Merge pull request #34388 from smithfarm/wip-44896-nautilus

nautilus: pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mds: handle bad purge queue item encoding

The bad encoding was introduced by commit a88f8d5eb4

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit ba06fcbe3e8345dc2c4c4c3dc3bcc18acf5ab076)
Conflicts:
        src/mds/PurgeQueue.cc: advance changed to +=
Fixes: https://tracker.ceph.com/issues/36635
Note: This commit from v13.2.3 fixes a bad backport in v13.2.2. It is
      also required in Octopus/Nautilus to handle upgrades.
(cherry picked from commit b73d1989bcbea227017607f8dd6e79633ec11f8f)
Conflicts:
            src/mds/PurgeQueue.cc: += changed to advance

client: fix error handling in lseek SEEK_HOLE/SEEK_DATA cases

The error handling here looks quite wrong. If the offset represents
an area outside the file, then we want to just return -ENXIO
immediately.

Fixes: https://tracker.ceph.com/issues/44021
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 2ca8db71364698e05c3a6ac9a5aa4da5421a35bc)

Conflicts:
src/client/Client.cc
  - minor modification for nautilus code flow

test: add new ceph_lseek test

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 339b09dc5921f81621f76faa55c56c31c16d4937)

mds: don't assert empty io context list when shutting down

Fixes: https://tracker.ceph.com/issues/44680
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 7f26bb7bb768a90177601915050b4f5020902943)

Revert "mds: make MDSIOContextBase delete itself when shutting down"

This reverts commit 73436961512bd87981244fa48212085faf7028c4.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 18118974294ee1286b4cbae2f5e6578a5a68f6d6)

Revert "mds: just delete MDSIOContextBase during shutdown"

This reverts commit 79f5052a1ddb61043de4e1cbec19ede2a6b4f53b.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 02658de9eeb700364af02f2c7710c3819e776fb2)

mds: just delete MDSIOContextBase during shutdown

We don't want the finisher called during shutdown.

Fixes: https://tracker.ceph.com/issues/44295
Fixes: 73436961512bd87981244fa48212085faf7028c4
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 79f5052a1ddb61043de4e1cbec19ede2a6b4f53b)

mds: add comment explaining unusual locking for MDSIOContextBase

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c2a079084c7653b6ed7e0912d4e737fe10a36836)

Merge pull request #34524 from smithfarm/wip-45054-nautilus

nautilus: doc/releases/nautilus: restart OSDs to make them bind to v2 addr

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/releases/nautilus: restart OSDs to make them bind to v2 addr

Fixes: https://tracker.ceph.com/issues/43896
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 4facd9daa524f1e2d77511ee6b1a18e108c07d31)

Conflicts:
doc/releases/nautilus.rst
- nautilus does not have 98ae20c2c874

qa/suites/rados/rest: Don't pass empty dict as data arg

Passing an empty 'args' dict as a data argument when calling
requests.get somehow confuses the transaction, causing it to fail. Pass
'None' instead.

Fixes: https://tracker.ceph.com/issues/43720
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit e20e6fc3295da836fb5f098d46d6190ab28195d5)

pybind/rbd: ensure image is open before permitting operations

Fixes: https://tracker.ceph.com/issues/44610
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 99287a1e552387f8eec505bffdffae7f4674cc8b)

Conflicts:
src/pybind/rbd/rbd.pyx
- no "snap_exists", "snap_get_name", "snap_get_id",
  "mirror_image_create_snapshot", "mirror_image_get_mode", "config_set",
  "config_get", "config_remove", "snap_get_mirror_namespace",  in nautilus
- nautilus "mirror_image_enable" does not take any argument

pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output

No lockers are obtained, ImageNotFound exception will be output,
but tht image is always exist.when lockers number is zero,
Should not output any exceptions。

Fixes: https://tracker.ceph.com/issues/44613
Signed-off-by: zhangdaolong <zhangdaolong@fiberhome.com>
(cherry picked from commit a183aac978dac69f996250324975073a78cb476b)

Merge remote-tracking branch 'origin/nautilus' into nautilus

Conflicts:
src/msg/async/ProtocolV2.cc - rename of outcoming_bl to outgoing_bl

14.2.9

Merge pull request #34473 from Vicente-Cheng/wip-44995-nautilus

nautilus: mgr/dashboard: Fix error in unit test caused by timezone

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34055 from callithea/wip-44668-nautilus

nautilus: qa: use small default pg count for CephFS pools

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34354 from smithfarm/wip-44711-nautilus

nautilus: mgr/DaemonServer: fix pg merge checks

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #34297 from vumrao/wip-vumrao-44818

nautilus: common/options: Disable bluefs_buffered_io by default again.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #33889 from ifed01/wip-ifed-flexible-bluefs-nautilus

nautilus: os/bluestore: more flexible DB volume space usage

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: fix tests in order to match pg num conventions

Update the tests test_ganesha and test_rbd_mirroring in order
to match the PG num conventions. It prevents the health warning
'POOL_PG_NUM_NOT_POWER_OF_TWO' from being shown.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit ee46e58e5dc779418492ef6b8c5e687f36ecb3f6)

test/store_test: test coverage for anti-spillover framework.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 57c6a79136a390c74a800d03bc595f3363718c2f)

 Conflicts:
src/os/bluestore/BlueFS.h
Future stuff tried to get into.

os/bluestore: introduce bluestore_volume_selection_reserved cfg option.

This is an alternative (and straightforward) way to specify barred space not allowed for 'use
some extra' BlueFS volume selector policy.
Disabled by default since it should depend on RocksDB settings and actual volume size.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 185e6a1adb2add91209d5f937c61c309528518f9)

os/bluestore: beautify RocksDBBlueFSVolumeSelector::dump output

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit fc84588536b4757bbe75065951a7215ffbcc0f60)

os/bluestore: introduce 'bluestore bluefs stats' admin socket command

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit f5568eb7209e43e6b916ea8450b2bd0c9484919a)

 Conflicts:
src/os/bluestore/BlueFS.cc
 Future stuff, legacy admin command iface
        src/os/bluestore/BlueStore.cc
 Need to use legacy bluefs_shared_bdev

os/bluestore: implement more intelligent DB volume space managment for BlueFS.

It allows excessive space usage for higher DB levels.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 5f72c376deb64562e5e88be2f22339135ac7372b)

 Conflicts:
src/common/legacy_config_opts.h
src/common/options.cc
Future options tried to get into.

rgw: reject control characters in response-header actions

S3 GetObject permits overriding response header values, but those inputs
need to be validated to insure only characters that are valid in an HTTP
header value are present.

Credit: Initial vulnerability discovery by William Bowling (@wcbowling)
Credit: Further vulnerability discovery by Robin H. Johnson <rjohnson@digitalocean.com>
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>

rgw: EPERM to ERR_INVALID_REQUEST

As per Robin's comments and S3 spec

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: reject unauthenticated response-header actions

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d8dd5e513c0c62bbd7d3044d7e2eddcd897bd400)

msg/async/crypto_onwire: fix endianness of nonce_t

As a AES-GCM IV, nonce_t is implicitly shared between server and
client.  Currently, if their endianness doesn't match, they are unable
to communicate in secure mode because each gets its own idea of what
the next nonce should be after the counter is incremented.

Several RFCs state that the nonce counter should be BE, but since we
use LE for everything on-disk and on-wire, make it LE.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV2: avoid AES-GCM nonce reuse vulnerabilities

The secure mode uses AES-128-GCM with 96-bit nonces consisting of a
32-bit counter followed by a 64-bit salt.  The counter is incremented
after processing each frame, the salt is fixed for the duration of
the session.  Both are initialized from the session key generated
during session negotiation, so the counter starts with essentially
a random value.  It is allowed to wrap, and, after 2**32 frames, it
repeats, resulting in nonce reuse (the actual sequence numbers that
the messenger works with are 64-bit, so the session continues on).

Because of how GCM works, this completely breaks both confidentiality
and integrity aspects of the secure mode.  A single nonce reuse reveals
the XOR of two plaintexts and almost completely reveals the subkey
used for producing authentication tags.  After a few nonces get used
twice, all confidentiality and integrity goes out the window and the
attacker can potentially encrypt-authenticate plaintext of their
choice.

We can't easily change the nonce format to extend the counter to
64 bits (and possibly XOR it with a longer salt).  Instead, just
remember the initial nonce and cut the session before it repeats,
forcing renegotiation.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Conflicts:
src/msg/async/ProtocolV2.cc [ context: commit 697aafa2aad2
  ("msg/async/ProtocolV2: remove unused parameter") not in
  nautilus ]
src/msg/async/ProtocolV2.h [ context: commit ed3ec4c01d17
  ("msg: Build target 'common' without using namespace in
  headers") not in nautilus ]

msg/async: rename outcoming_bl -> outgoing_bl in AsyncConnection.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 7997a3ea193344ea9782d2594cca295ac5bdf59d)

mgr/dashboard: Fix error in unit test caused by timezone

Unit test was only accepting positive timezones

Fixes: https://tracker.ceph.com/issues/42812
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit f5e88e1d113d5fbeeac2e52a2b2f9b03ace174fa)

Merge pull request #34373 from votdev/wip-44690-nautilus

nautilus: mgr/dashboard: list configured prometheus alerts

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34433 from badone/wip-44260-nautilus

nautilus: selinux: Allow ceph to setsched

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #34434 from badone/wip-nautilus-ceph-mgr-httpd-selinux-denial

nautilus: selinux: Allow ceph-mgr access to httpd dir

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #31842 from sseshasa/wip-42662-nautilus

nautilus: osd/OSDMap: Show health warning if a pool is configured with size 1

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34120 from tspmelo/wip-rbd-error

nautilus: mgr/dashboard: Fix error when listing RBD while deleting or moving

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34099 from shyukri/wip-44472-nautilus

nautilus: rgw: clear ent_list for each loop of bucket list

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33641 from tchaikov/nautilus-42477

nautilus: tools/rados: Unmask '-o' to restore original behaviour

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #33160 from shyukri/wip-43995-nautilus

nautilus: ceph_argparse: increment matchcnt on kwargs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34417 from smithfarm/wip-44952-nautilus

nautilus: mgr/dashboard: Repair broken grafana panels

Reviewed-by: Laura Paduano <lpaduano@suse.com>

selinux: Allow ceph to setsched

In several places, such as common/numa.cc we call sched_setaffinity
which requires this permission.

Fixes: https://tracker.ceph.com/issues/44196
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 43103e0207bfacf02f7f9533b36443d65f95d718)

selinux: Allow ceph-mgr access to httpd dir

ceph-mgr loads modules which require read access and this causes a
denial on el7.

Fixes: https://tracker.ceph.com/issues/44216
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

dashboard: Convert FQDN to hostname in grafana panels

The $ceph_hosts variable contained the FQDN for hosts
while the instance label created by ceph only has
the hostname.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit b7abaab5bd2b72f48f41b3d2a9732743afdcecfc)

dashboard: Resolve FQDN / hostname mismatch in hosts overview panel

In the AVG Disk Utilization panel, the result is calculated
by combining the output of node_disk_io_time_seconds_total
with the output of ceph_disk_occupation. However, the
first vector encodes the instance label with the full FQDN
while the ceph label only contains the hostname:port. In
order for these to match correctly, the domain name and port
has to be stripped from the labels.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 136d21e21dc3c05ec8c586a47eed1904ffbda578)

dashboard: Use exported_instance to identify OSDs

When moving to LVM-based ceph-volume setups, several
grafana dashboards stopped working. The problem is that
(device, instance) no longer results in unique labels
which causes errors like:

"many-to-many matching not allowed: matching labels must be unique on one side"

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 8b61b8d3d781b9d762098cf45b41b596a32509ab)

dashboard: AVG RAM Utilization panel always showed "N/A"

The references to `$osd_hosts` etc. were encoded as
`[[osd_hosts]]` in the PromQL expression divisor, and
the panel always displayed N/A as the result of the
query.

Replacing the `[[...]]` with `$...` makes the expression
work again.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 4444333243aaa81de1d7347bf0c57e039b657d43)

Merge pull request #34222 from ceph/wip-yuriw-clients-upgrades-nautilus-octopus

qa/tests: initial check in for client-upgrade-nautilus-octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: initial check in for client-upgrade-nautilus-octopus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34129 from rhcs-dashboard/nautilus-44666-fix-sso-certificate-file

nautilus: mgr/dashboard: fix error when enabling SSO with cert. file

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: show alert panel if prometheus/alertmanager is unconfigured

If the tabs under the "Monitoring" page aren't properly configured, a
notification is shown which explains the user which setting needs to be
enabled and also provides a link to the corresponding documentation.

Fixes: https://tracker.ceph.com/issues/42877
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 460f7bb3272c6536c9a5fc0919071d7c17e9aa5a)

Conflicts:
  src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/monitoring-list/monitoring-list.component.html
  src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/monitoring-list/monitoring-list.component.ts
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.ts

mgr/dashboard: extend dashboard documentation

by adding the previously added monitoring related features as well as
the newest feature addition. Extends the documentation where necessary
to describe the Prometheus' alert configuration.

Fixes: https://tracker.ceph.com/issues/42877
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 36421284c315baf7e79a8c0586ca98ac0126037e)

mgr/dashboard: move monitoring tabs to a single page

with a tab for 'active alerts', 'all alerts' and 'silences'. Due to
ambiguity with existing names, `AlertListComponent` has been renamed to
`ActiveAlertListComponent`. Introduces `MonitoringListComponent` as
first page for monitoring concerns, using path `/monitoring`.

Keeps the activated tab open, independent of the way that's used to go
back to the previous page, be it the cancel button or submit button or
the link on the breadcrumb. Also keeps the active tab open even when the
page is reloaded.

Fixes: https://tracker.ceph.com/issues/42877
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 855f214b29c8ed935c8f4ba0b8a8396692f946a1)

Conflicts:
  src/pybind/mgr/dashboard/frontend/e2e/cluster/alerts.e2e-spec.ts
  src/pybind/mgr/dashboard/frontend/e2e/cluster/alerts.po.ts
  src/pybind/mgr/dashboard/frontend/src/app/app-routing.module.ts
  src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/cluster.module.ts
  src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/active-alert-list/active-alert-list.component.spec.ts
  src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/active-alert-list/active-alert-list.component.ts
  src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/prometheus-tabs/prometheus-tabs.component.html
  src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/prometheus-tabs/prometheus-tabs.component.spec.ts
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.ts
  src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table-actions/table-actions.component.html

mgr/dashboard: Unify Tasks and Notifications into a sidebar

Fixes: https://tracker.ceph.com/issues/37402
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 7fdb3c1504a96a991041934c8abd6453d762857d)

Conflicts:
  src/pybind/mgr/dashboard/frontend/e2e/cluster/crush-map.po.ts
  src/pybind/mgr/dashboard/frontend/package-lock.json
  src/pybind/mgr/dashboard/frontend/package.json
  src/pybind/mgr/dashboard/frontend/src/app/app.component.html
  src/pybind/mgr/dashboard/frontend/src/app/app.module.ts
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/notifications/notifications.component.html
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/notifications/notifications.component.scss
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/notifications/notifications.component.ts
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/task-manager/task-manager.component.html
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/task-manager/task-manager.component.spec.ts
  src/pybind/mgr/dashboard/frontend/src/app/core/navigation/task-manager/task-manager.component.ts
  src/pybind/mgr/dashboard/frontend/src/app/shared/components/components.module.ts
  src/pybind/mgr/dashboard/frontend/src/styles/popover.scss

mgr/dashboard: refactor test of Prometheus alert service

Mocking the test the way it was removed the asynchronous nature of the
test. By using an Observable the test can stay asynchronous and be
tested as well.

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit b85b24b2d085fd775298625b546b9fd5abe4d741)

mgr/dashboard: list configured Prometheus alerts

Fixes: https://tracker.ceph.com/issues/42877
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 2a50e277d4ae764aa50f9b726f79c84f335ef15a)

Conflicts:
  src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/cluster.module.ts

mgr/dashboard: return Prometheus alert groups, too

Fixes: https://tracker.ceph.com/issues/42877
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 4917279f9f5ed321cada297706a96e7ce16f01eb)

mgr/DaemonServer: add explicit check that acting matches for merge

Add an explicit check that the PG acting for the source and target
match before merging.

Fixes: https://tracker.ceph.com/issues/44684
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1700d181580f9ffc9a808ee29daf661f3776a901)

mgr: synchronize ClusterState's health and mon_status.

Fixes: https://tracker.ceph.com/issues/24995
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit da362675da51de8b20a12e4cec479a77c7f5442b)

Merge pull request #34316 from tchaikov/nautilus-six

nautilus: pybind/mgr: use six==1.14.0

Reviewed-by: Laura Paduano <lpaduano@suse.com>

pybind/mgr: use six==1.14.0

this change is not cherry-picked from master. because in master, we
are using different ways to install six:

see
- 68527ab83a77d20b779759581ce0b8844991231b
- 1e07237d92da3bb5eba82b49f6276d17696e6985

because in teuthology we are using six.ensure_str, which was added in
six 1.12.0, see https://github.com/benjaminp/six/blob/1.12.0/CHANGES ,
we cannot continue using six 1.11.0, as a result, we need switch over to
six>1.12.0. since the latest stable version of six is now 1.14.0, let's
just use it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: Disable bluefs_buffered_io by default again.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 5574617b9f4cf950c108cb06cb6f9eb48396aa57)

Merge pull request #34162 from k0ste/wip-44735-nautilus

nautilus: mgr/prometheus: ceph_pg_* metrics contains last value instead of sum across all reported states

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32849 from smithfarm/wip-43469-nautilus

nautilus: osd/PeeringState: transit async_recovery_targets back into acting before backfilling

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

tools/rados: Unmask '-o' to restore original behaviour

0b369e1aff1 masked the original behaviour of '-o' which was to indicate
'outfile' as documented in the man page. Changing object-size to capital
o will restore the original behaviour.

Fixes: https://tracker.ceph.com/issues/42477
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit d2b806070de5f56474283035a7ffc91fe96afbab)

Conflicts:
PendingReleaseNotes: trivial resolution

Merge pull request #34116 from smithfarm/wip-43820-nautilus

nautilus: pybind/mgr/*: fix config_notify handling of default values

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33665 from rhcs-dashboard/wip-44375-nautilus

nautilus: mgr/dashboard: do not show RGW API keys if only read-only privileges

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34039 from rhcs-dashboard/wip-44639-nautilus

nautilus: mgr/dashboard: Pool read/write OPS shows too many decimal places

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #33141 from shyukri/wip-44046-nautilus

nautilus: mgr/telemetry: catch exception during requests.put

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #33024 from shyukri/wip-39230-nautilus

nautilus: mgr/dashboard: Updated existing E2E tests to match new format

Reviewed-by: Laura Paduano <lpaduano@suse.com>

osd/OSDMap: Show health warning if a pool is configured with size 1

Introduce a config option called 'mon_warn_on_pool_no_redundancy' that is
used to show a health warning if any pool in the ceph cluster is
configured with a size of 1. The user can mute/unmute the warning using
'ceph health mute/unmute POOL_NO_REDUNDANCY'.

Add standalone test to verify warning on setting pool size=1. Set the
associated warning to 'false' in ceph.conf.template under qa/tasks so
that existing tests do not break.

Fixes: https://tracker.ceph.com/issues/41666
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 33c647e8114b37404d8d62a08c85664cea709118)

 Conflicts:
PendingReleaseNotes
- Added release notes under 14.2.9
qa/standalone/mon/health-mute.sh
- Deleted the script as 'health mute/unmute' cmd is unavailable in nautilus
qa/tasks/ceph.conf.template
- Removed a flag not available in nautilus
src/common/options.cc
- Removed a flag not available in nautilus
src/osd/OSDMap.cc

Merge pull request #30704 from vshankar/wip-41508

nautilus: mds: display scrub status in ceph status

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

test: query using mds id, not rank

Fixes: http://tracker.ceph.com/issues/42835
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit fb4cd42cbcdaa4812838b846ae33ee26356ce733)

mgr: re-enable mds `scrub status` info in ceph status

MDS does not register with manager, therefore `task status`
(scrub status updates) were not getting displayed in ceph
status output.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit d6ad0a255a27c5e3da20c7ae03066a692a6d49bb)

 Conflicts:
src/mgr/DaemonServer.cc
src/mgr/DaemonServer.h

mgr/DaemonServer.{h,cc} deals with raw pointers while master uses ref_t<>
cast -- adjust to that. a minor conflict in the header and the metrics
templatization is not backported to nautilus. also, DaemonKey is a std::pair
in nautilus but a struct in master -- that requires a change in referencing
daemon type and name.

mon: filter out ceph normal ceph entity types when dumping service metadata

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit dfc056028e44cdbda84d7118e29120ae70edb35a)

mgr: filter out normal ceph services when processing service map

This is done is couple of places in ceph manager -- when culling
entries from service map and the other when dumping serice status.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 20139d12423fc6154b57499c576d1e4bb3f1eade)

 Conflicts:
src/mgr/DaemonServer.cc

Switch to non-initializer list in range-based for loop.

mgr: helper function to check if a service is a normal ceph service

This would be widely required since ceph metadata server entries are
maintained in service map (DaemonServer::pending_service_map). Such
normal ceph services would need to filtered when processing the service
map to avoid extraneous entries getting processed.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 79503fc16749ed0cfe8a89ea3b3c8c792d6b8809)

mds: establish session with mgr only after added to FSMap

Note that we now sub to the mgrmap after init because the MgrClient
connection to the mgr is driven by receipt of the MgrMap.

This is important so that the MDS does not have metadata with the mgr
when the mons are ignoring the MDS otherwise due to CompatSet
incompatibilities.

Fixes: https://tracker.ceph.com/issues/41538
Fixes: https://tracker.ceph.com/issues/42635
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e765f2d533440cfc4189f36fcaba24617a302e84)

 Conflicts:
src/mds/MDSDaemon.cc

Nautilus uses Lock/Unlock rather than lock/unlock in master.

mds: do not register as a service daemon

This commit undoes the service daemon registration for the MDS. It doesn't look
absolutely necessary and it causes the MDS to be listed twice in the `ceph
versions` output:

    $ ceph versions
        ...
        "mds": {
            "ceph version v15.0.0-6915-g0143b904676 (0143b9046763ea1801efa8358a0c033ec862cea9) octopus (dev)": 3
        },
        "mds": {
            "unknown": 3
        },
        "overall": {
            "ceph version v15.0.0-6915-g0143b904676 (0143b9046763ea1801efa8358a0c033ec862cea9) octopus (dev)": 10,
            "unknown": 3
        }

Fixing that requires looking for duplicates or ignoring MDSs in the
service daemons when the mon processes `ceph versions`. I have a feeling
that it wasn't actually designed to be used by the MDS this way however.
Additionally, the reason for "unknown" version is because the metadata
sent to the mgr does not include "ceph_version".

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 046137f819aae72f1423e3feb213f0e46c97c9ce)

mds: do not try to diagnose cause of MDSMap removal

The Monitors send an empty MDSMap to an MDS it is removing. The MDS
can't diagnose the cause. Instead suggest looking at the cluster/monitor
logs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c385178b0c04d09bc6774acf5bdb4537fa6803ad)

mds: fix handling of initial MDS states

Few things here:

- Make explicit the check for getting removed from the MDSMap. This was
  only done before by checking if MDS held a rank which does not check the
  case where a standby is removed from the FSMap.

- Use mds_info_t::dump to simplify various debug output.

- Add a few sanity asserts for invalid state transitions.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 26a08df2adc7495fc660113cd26c33d5debd3ee6)

 Conflicts:
src/mds/MDSDaemon.h

Nautilus still uses "const_ref" rather than "cref_t<>" in master.

mds: remove unnecessary const qualifier

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3355581a298963711ad058c6e47d21b7eab887a4)

mds: cleanup type decl and map iteration

This is a trivial refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bafa3b731ff93a356fcf60ebbaf0e5f299182992)

mds: define stream operator for mds_info_t

This also cleans up the output to be more readable/useful in debug
output.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 1c56632e88a126d0eac75235d9a9716833cba6b7)

 Conflicts:
PendingReleaseNotes

Fix minor conflict in PendingReleaseNotes.

mds/MDSRank: report state to mgr as mds id, not rank

Introduced by 625dffe65c0f8001b3b6ca6d0b12732a1a103849, which added
periodic scrub stats reporting.

Fixes: https://tracker.ceph.com/issues/42494
Fixes: https://tracker.ceph.com/issues/41525
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a1a220d137f7fa128d08f96d1eff83865388e5fc)