selinux: Allow getattr access to /proc/kcore

Required for an fstat call in BlkDev::get_devid

Fixes: https://tracker.ceph.com/issues/40743
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #34433 from badone/wip-44260-nautilus

nautilus: selinux: Allow ceph to setsched

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #34434 from badone/wip-nautilus-ceph-mgr-httpd-selinux-denial

nautilus: selinux: Allow ceph-mgr access to httpd dir

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #31842 from sseshasa/wip-42662-nautilus

nautilus: osd/OSDMap: Show health warning if a pool is configured with size 1

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34120 from tspmelo/wip-rbd-error

nautilus: mgr/dashboard: Fix error when listing RBD while deleting or moving

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34099 from shyukri/wip-44472-nautilus

nautilus: rgw: clear ent_list for each loop of bucket list

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33641 from tchaikov/nautilus-42477

nautilus: tools/rados: Unmask '-o' to restore original behaviour

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #33160 from shyukri/wip-43995-nautilus

nautilus: ceph_argparse: increment matchcnt on kwargs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34417 from smithfarm/wip-44952-nautilus

nautilus: mgr/dashboard: Repair broken grafana panels

Reviewed-by: Laura Paduano <lpaduano@suse.com>

selinux: Allow ceph to setsched

In several places, such as common/numa.cc we call sched_setaffinity
which requires this permission.

Fixes: https://tracker.ceph.com/issues/44196
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 43103e0207bfacf02f7f9533b36443d65f95d718)

selinux: Allow ceph-mgr access to httpd dir

ceph-mgr loads modules which require read access and this causes a
denial on el7.

Fixes: https://tracker.ceph.com/issues/44216
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

dashboard: Convert FQDN to hostname in grafana panels

The $ceph_hosts variable contained the FQDN for hosts
while the instance label created by ceph only has
the hostname.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit b7abaab5bd2b72f48f41b3d2a9732743afdcecfc)

dashboard: Resolve FQDN / hostname mismatch in hosts overview panel

In the AVG Disk Utilization panel, the result is calculated
by combining the output of node_disk_io_time_seconds_total
with the output of ceph_disk_occupation. However, the
first vector encodes the instance label with the full FQDN
while the ceph label only contains the hostname:port. In
order for these to match correctly, the domain name and port
has to be stripped from the labels.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 136d21e21dc3c05ec8c586a47eed1904ffbda578)

dashboard: Use exported_instance to identify OSDs

When moving to LVM-based ceph-volume setups, several
grafana dashboards stopped working. The problem is that
(device, instance) no longer results in unique labels
which causes errors like:

"many-to-many matching not allowed: matching labels must be unique on one side"

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 8b61b8d3d781b9d762098cf45b41b596a32509ab)

dashboard: AVG RAM Utilization panel always showed "N/A"

The references to `$osd_hosts` etc. were encoded as
`[[osd_hosts]]` in the PromQL expression divisor, and
the panel always displayed N/A as the result of the
query.

Replacing the `[[...]]` with `$...` makes the expression
work again.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 4444333243aaa81de1d7347bf0c57e039b657d43)

Merge pull request #34222 from ceph/wip-yuriw-clients-upgrades-nautilus-octopus

qa/tests: initial check in for client-upgrade-nautilus-octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: initial check in for client-upgrade-nautilus-octopus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34129 from rhcs-dashboard/nautilus-44666-fix-sso-certificate-file

nautilus: mgr/dashboard: fix error when enabling SSO with cert. file

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34316 from tchaikov/nautilus-six

nautilus: pybind/mgr: use six==1.14.0

Reviewed-by: Laura Paduano <lpaduano@suse.com>

pybind/mgr: use six==1.14.0

this change is not cherry-picked from master. because in master, we
are using different ways to install six:

see
- 68527ab83a77d20b779759581ce0b8844991231b
- 1e07237d92da3bb5eba82b49f6276d17696e6985

because in teuthology we are using six.ensure_str, which was added in
six 1.12.0, see https://github.com/benjaminp/six/blob/1.12.0/CHANGES ,
we cannot continue using six 1.11.0, as a result, we need switch over to
six>1.12.0. since the latest stable version of six is now 1.14.0, let's
just use it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34162 from k0ste/wip-44735-nautilus

nautilus: mgr/prometheus: ceph_pg_* metrics contains last value instead of sum across all reported states

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32849 from smithfarm/wip-43469-nautilus

nautilus: osd/PeeringState: transit async_recovery_targets back into acting before backfilling

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

tools/rados: Unmask '-o' to restore original behaviour

0b369e1aff1 masked the original behaviour of '-o' which was to indicate
'outfile' as documented in the man page. Changing object-size to capital
o will restore the original behaviour.

Fixes: https://tracker.ceph.com/issues/42477
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit d2b806070de5f56474283035a7ffc91fe96afbab)

Conflicts:
PendingReleaseNotes: trivial resolution

Merge pull request #34116 from smithfarm/wip-43820-nautilus

nautilus: pybind/mgr/*: fix config_notify handling of default values

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33665 from rhcs-dashboard/wip-44375-nautilus

nautilus: mgr/dashboard: do not show RGW API keys if only read-only privileges

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34039 from rhcs-dashboard/wip-44639-nautilus

nautilus: mgr/dashboard: Pool read/write OPS shows too many decimal places

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #33141 from shyukri/wip-44046-nautilus

nautilus: mgr/telemetry: catch exception during requests.put

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #33024 from shyukri/wip-39230-nautilus

nautilus: mgr/dashboard: Updated existing E2E tests to match new format

Reviewed-by: Laura Paduano <lpaduano@suse.com>

osd/OSDMap: Show health warning if a pool is configured with size 1

Introduce a config option called 'mon_warn_on_pool_no_redundancy' that is
used to show a health warning if any pool in the ceph cluster is
configured with a size of 1. The user can mute/unmute the warning using
'ceph health mute/unmute POOL_NO_REDUNDANCY'.

Add standalone test to verify warning on setting pool size=1. Set the
associated warning to 'false' in ceph.conf.template under qa/tasks so
that existing tests do not break.

Fixes: https://tracker.ceph.com/issues/41666
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 33c647e8114b37404d8d62a08c85664cea709118)

 Conflicts:
PendingReleaseNotes
- Added release notes under 14.2.9
qa/standalone/mon/health-mute.sh
- Deleted the script as 'health mute/unmute' cmd is unavailable in nautilus
qa/tasks/ceph.conf.template
- Removed a flag not available in nautilus
src/common/options.cc
- Removed a flag not available in nautilus
src/osd/OSDMap.cc

Merge pull request #30704 from vshankar/wip-41508

nautilus: mds: display scrub status in ceph status

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

test: query using mds id, not rank

Fixes: http://tracker.ceph.com/issues/42835
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit fb4cd42cbcdaa4812838b846ae33ee26356ce733)

mgr: re-enable mds `scrub status` info in ceph status

MDS does not register with manager, therefore `task status`
(scrub status updates) were not getting displayed in ceph
status output.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit d6ad0a255a27c5e3da20c7ae03066a692a6d49bb)

 Conflicts:
src/mgr/DaemonServer.cc
src/mgr/DaemonServer.h

mgr/DaemonServer.{h,cc} deals with raw pointers while master uses ref_t<>
cast -- adjust to that. a minor conflict in the header and the metrics
templatization is not backported to nautilus. also, DaemonKey is a std::pair
in nautilus but a struct in master -- that requires a change in referencing
daemon type and name.

mon: filter out ceph normal ceph entity types when dumping service metadata

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit dfc056028e44cdbda84d7118e29120ae70edb35a)

mgr: filter out normal ceph services when processing service map

This is done is couple of places in ceph manager -- when culling
entries from service map and the other when dumping serice status.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 20139d12423fc6154b57499c576d1e4bb3f1eade)

 Conflicts:
src/mgr/DaemonServer.cc

Switch to non-initializer list in range-based for loop.

mgr: helper function to check if a service is a normal ceph service

This would be widely required since ceph metadata server entries are
maintained in service map (DaemonServer::pending_service_map). Such
normal ceph services would need to filtered when processing the service
map to avoid extraneous entries getting processed.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 79503fc16749ed0cfe8a89ea3b3c8c792d6b8809)

mds: establish session with mgr only after added to FSMap

Note that we now sub to the mgrmap after init because the MgrClient
connection to the mgr is driven by receipt of the MgrMap.

This is important so that the MDS does not have metadata with the mgr
when the mons are ignoring the MDS otherwise due to CompatSet
incompatibilities.

Fixes: https://tracker.ceph.com/issues/41538
Fixes: https://tracker.ceph.com/issues/42635
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e765f2d533440cfc4189f36fcaba24617a302e84)

 Conflicts:
src/mds/MDSDaemon.cc

Nautilus uses Lock/Unlock rather than lock/unlock in master.

mds: do not register as a service daemon

This commit undoes the service daemon registration for the MDS. It doesn't look
absolutely necessary and it causes the MDS to be listed twice in the `ceph
versions` output:

    $ ceph versions
        ...
        "mds": {
            "ceph version v15.0.0-6915-g0143b904676 (0143b9046763ea1801efa8358a0c033ec862cea9) octopus (dev)": 3
        },
        "mds": {
            "unknown": 3
        },
        "overall": {
            "ceph version v15.0.0-6915-g0143b904676 (0143b9046763ea1801efa8358a0c033ec862cea9) octopus (dev)": 10,
            "unknown": 3
        }

Fixing that requires looking for duplicates or ignoring MDSs in the
service daemons when the mon processes `ceph versions`. I have a feeling
that it wasn't actually designed to be used by the MDS this way however.
Additionally, the reason for "unknown" version is because the metadata
sent to the mgr does not include "ceph_version".

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 046137f819aae72f1423e3feb213f0e46c97c9ce)

mds: do not try to diagnose cause of MDSMap removal

The Monitors send an empty MDSMap to an MDS it is removing. The MDS
can't diagnose the cause. Instead suggest looking at the cluster/monitor
logs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c385178b0c04d09bc6774acf5bdb4537fa6803ad)

mds: fix handling of initial MDS states

Few things here:

- Make explicit the check for getting removed from the MDSMap. This was
  only done before by checking if MDS held a rank which does not check the
  case where a standby is removed from the FSMap.

- Use mds_info_t::dump to simplify various debug output.

- Add a few sanity asserts for invalid state transitions.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 26a08df2adc7495fc660113cd26c33d5debd3ee6)

 Conflicts:
src/mds/MDSDaemon.h

Nautilus still uses "const_ref" rather than "cref_t<>" in master.

mds: remove unnecessary const qualifier

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3355581a298963711ad058c6e47d21b7eab887a4)

mds: cleanup type decl and map iteration

This is a trivial refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bafa3b731ff93a356fcf60ebbaf0e5f299182992)

mds: define stream operator for mds_info_t

This also cleans up the output to be more readable/useful in debug
output.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 1c56632e88a126d0eac75235d9a9716833cba6b7)

 Conflicts:
PendingReleaseNotes

Fix minor conflict in PendingReleaseNotes.

mds/MDSRank: report state to mgr as mds id, not rank

Introduced by 625dffe65c0f8001b3b6ca6d0b12732a1a103849, which added
periodic scrub stats reporting.

Fixes: https://tracker.ceph.com/issues/42494
Fixes: https://tracker.ceph.com/issues/41525
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a1a220d137f7fa128d08f96d1eff83865388e5fc)

mgr: missing lock release in DaemonServer::handle_report()

Fixes: http://tracker.ceph.com/issues/42169
Introduced-by: 5c25a01864
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit cda3eadbfbd3e3a8a057cc7ed042ba0a6d7fef11)

mgr/ServiceMap: use plain gid

* partially revert 5c25a018. which is not backward compatible.
* change `ServiceMap::get_daemon()` so it returns a
  `pair<Daemon*,bool>`.

git is an `optional<uint64_t>`, so we cannot dump it without checking.

Fixes: http://tracker.ceph.com/issues/41424
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3e65551d0ab4acafaf4012601987b3391b2411ed)

doc: document scrub summary in ceph status output

Fixes: http://tracker.ceph.com/issues/36370
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 05d17994a879995b56bda5d770a938d0aabaaed9)

test: extend scrub control test to validate mds task status

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 465a3adc6c31fd9b8359920ab47adf8e1f45d5f1)

mds: send scrub state changes to cluster log.

... also log new and completed scrubs.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 49520156414cb0667e8edd6ea4b31462e7cb7752)

mds: periodically sent mds scrub status to ceph manager

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 625dffe65c0f8001b3b6ca6d0b12732a1a103849)

 Conflicts:
src/mds/MDSRank.cc

mgr, mon: allow normal ceph services to register with manager

Additionally, introduce `task status` field in manager report
messages to forward status of executing tasks in daemons (e.g.,
status of executing scrubs in ceph metadata servers).

`task status` makes its way upto service map which is then used
to display the relevant information in ceph status.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5c25a018643b10aa78db8270cae1476f71d8f4f4)

 Conflicts:
src/messages/MMgrReport.h
src/mgr/DaemonServer.cc
src/mgr/ServiceMap.h

pybind/mgr/*: fix config_notify handling of default values

After 5ace82e65c72847fb875fc01c419937a26a59d70 was merged, I found
three more instances of the code being patched.

The commit message of 5ace82e65c72847fb875fc01c419937a26a59d70 was/is:

"The default values are handled by mgr_module.py's _get_module_option();
the or here means that we break any non-true (0, false, none) value and
override it with the default."

Fixes: 5ace82e65c72847fb875fc01c419937a26a59d70
Fixes: https://tracker.ceph.com/issues/43746
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 30585f44de6147c824ed2df1a477ad17f7b51fff)

Conflicts:
src/pybind/mgr/osd_support/module.py
- file does not exist in nautilus

Merge pull request #33684 from k0ste/wip-44367-nautilus

nautilus: doc/mgr/telemetry: added device channel details

doc/mgr/telemetry: added device channel details

Plus details about license agreement.

Fixes: https://tracker.ceph.com/issues/43648
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
(cherry picked from commit 66c41f3d0e516e217dc89ab20c53563ea10f97f7)

mgr: ceph_pg_* metrics contains last value instead of sum all of them

During evaluation of pool stats metrics contains last reported value instead of sum

Fixes: https://tracker.ceph.com/issues/44590
Signed-off-by: Jacek Suchenia <jacek.suchenia@gmail.com>
(cherry picked from commit ad4d790a35af483f3a240f247cda5754301a9199)

Merge pull request #34038 from rhcs-dashboard/wip-44334-nautilus

nautilus: mgr/dashboard: UI fixes

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #33764 from shyukri/wip-44327-nautilus

nautilus: mgr/dashboard: 'destroyed' view in CRUSH map viewer

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #33763 from shyukri/wip-43258-nautilus

nautilus: mgr/dashboard: Disable cache for static files

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by:  Tiago Melo <tmelo@suse.com>

Merge pull request #33739 from rhcs-dashboard/wip-44435-nautilus

nautilus: mgr/dashboard: remove 'config-opt: read' perm. from system roles.

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: fix error when enabling SSO with cert. file

Nautilus dedicated fix: added py2 compatibility code.
Also:
* Disabled security setting 'wantNameIdEncrypted': not all Identity Providers support this and we are already requiring encrypted assertions (which is the default).

Fixes: https://tracker.ceph.com/issues/44666
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

pybind/mgr/*: fix config_notify handling of default values

The default values are handled by mgr_module.py's _get_module_option();
the or here means that we break any non-true (0, false, none) value and
override it with the default.

https://tracker.ceph.com/issues/43746
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5ace82e65c72847fb875fc01c419937a26a59d70)

Conflicts:
src/pybind/mgr/cephadm/module.py
- nautilus has no "cephadm" module. It does have an "orchestrator_cli"
  module but it doesn't contain the code being patched
src/pybind/mgr/hello/module.py
- nautilus has a "hello" module, but it doesn't contain the code being
  patched

mgr/dashboard: Fix error when listing RBD while deleting or moving

This fix is based on a commit made to octopus, that fixed this problem in
octopus/master: 5627919c016785a27666b1992d336f8ff378a072.

Since the codebase is very different and a backport is not recommended or even
possible, I have created this commit with only the minimal code necessary.

Fixes: https://tracker.ceph.com/issues/42330
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #33738 from rhcs-dashboard/wip-44434-nautilus

nautilus: vstart.sh: set prometheus port for each mgr.

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #33530 from thmour/nautilus_test

nautilus: osd: fix racy accesses to OSD::osdmap.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph_argparse: increment matchcnt on kwargs

Currently when you pass a param in on the ceph cli as a kwarg
(--<param_name>) the matchcnt isn't incremented in the validate method
which is used to choose the right command signature.

This means currently things fail like:

  ceph orchestrator rgw rm --realm_name=default --zone_name=default
  no valid command found; 1 closest matches:
  orchestrator rgw rm <realm_name> <zone_name>
  Error EINVAL: invalid command

The '--realm_name' and '--zone_name' isn't counted to the matchcnt, so
'orchestrator rgw rm' isn't picked as the valid command.

This patch simply corrects this by incrementing matchcnt on the kwarg
validate path before calling shortcircuiting the loop.

Fixes: https://tracker.ceph.com/issues/43803
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit cb37c9ee609864a078edf38d98608bd8cc18cbd7)

Conflicts:

  test: exclude helper method from nosetest discovery

  On nautilus the assertion helper was recognized by nosetest as a test
  even though it doens't start with test_ prefix. Explicitely decorate it
  with @nottest

Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>

rgw: clear ent_list for each loop of bucket list

if ent_list is not cleared, the old element will be checked repeatedly
and will occupy more memory.
Fixes: http://tracker.ceph.com/issues/44394
Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
(cherry picked from commit f63bf47aa464c345c907c748dfdbbc5a239d8488)

Merge pull request #34064 from smithfarm/wip-44145-nautilus

nautilus: rgw: set correct storage class for append

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33913 from shyukri/wip-42324-nautilus

nautilus: qa: get rid of iterkeys for py3 compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33649 from k0ste/wip-44364-nautilus

nautilus: mgr/telemetry: fix and document proxy usage

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33421 from k0ste/wip-44218-nautilus

nautilus: common/blkdev: fix some problems with smart scraping

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33420 from k0ste/wip-44219-nautilus

nautilus: mgr/pg_autoscaler: fix division by zero

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #33409 from k0ste/wip-44199-nautilus

nautilus: mgr/telemetry: add 'last_upload' to status

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33355 from shyukri/wip-44141-nautilus

nautilus: rgw: sts: add all http args to req_info

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #33893 from rhcs-dashboard/wip-44200-nautilus

nautilus: mgr: Add get_rates_from_data to mgr_util.py

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #33275 from smithfarm/wip-44081-nautilus

nautilus: mon/PGMap: fix summary display of >32bit pg states

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33261 from smithfarm/wip-43997-nautilus

nautilus: global: ensure CEPH_ARGS is decoded before early arg processing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #33032 from shyukri/wip-42164-nautilus

nautilus: mgr/dashboard: REST API: OpenAPI docs require internet connection

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #32891 from shyukri/wip-42673-nautilus

nautilus: mgr/dashboard: Searchable objects for table

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34087 from jdurgin/wip-target-ratio-nautilus

nautilus: mgr/pg_autoscaler: treat target ratios as weights

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: Pool read/write OPS shows too many decimal places
Fixes: https://tracker.ceph.com/issues/39714
Signed-off-by: anurag <anurag@localhost.localdomain>
(cherry picked from commit 27a2bbb12614b7aba0561c027346d9b5427f2405)
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
Conflicts:
        src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table/table.component.spec.ts,
        src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table/table-key-value/table-key-value.component.spec.ts:
    added import of PipesModule to Angular unit tests

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: do not show API keys if user has only read-only privileges.

Fixes: https://tracker.ceph.com/issues/42475
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 38d419620508af75f7c164b56d07d1ecde9173b4)

Conflicts:
- src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-user-details/rgw-user-details.component.spec.ts
- src/pybind/mgr/dashboard/tests/test_rgw.py
Adapted (after cherry-pick) frontend & backend unit tests to nautilus code.

PendingReleaseNotes: mention target_size_ratio change

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 9118065c4c3803f8a2c26c28c69ffc9f14ff260a)

Merge pull request #34036 from ajarr/nautilus-volumes-cancel-clone

nautilus: fs subvolume clone cancel

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc: update autoscaler docs for target ratio

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 0a29cf7d6c9076c79416ef7868be3635921fd140)

mgr/pg_autoscaler: include effective target ratio in status

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 9758197afa856f3680d5b5d599c79353816e9927)

mgr/pg_autoscaler: add warning when target bytes and ratio are both set

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 772d7c1d3c1adf49821670da0f6cdb06e293dc0d)

mgr/pg_autoscaler: remove target ratio warning

Since the ratios are normalized, they cannot exceed 1.0 or overcommit
combined with target_bytes.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit d62c121ee3839a2fd4243a9b1c33aded1cc3db13)

qa: use ratios >1 in pg_autoscaler test

Also check for pg_num_target being set correctly, rather than pg_num,
so the test doesn't depend on merging/splitting speed.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 1ea8ce3480e96aa87b4d4f570022f9844bc54c17)

mgr/pg_autoscaler: treat target ratios as weights

Normalize across pools so that it's simpler to use - this way you
don't have to adjust every other pool when you add one.

Handle pools with target_bytes by taking their capacity off the top,
and dividing the rest into the pools with a target_ratio.

If both target bytes and ratio are specified, ignore bytes. This
matches the docs and makes accounting simpler.

Fixes: https://tracker.ceph.com/issues/43947
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 435c56a03b7424b23fe977a13c4704304189e829)

Merge pull request #33978 from yuvalif/wip-yuval-fix-44614

nautilus: rgw/notifications: fix topic action fail with "MethodNotAllowed"

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #33965 from smithfarm/wip-44466-nautilus

nautilus: rgw: get barbican secret key request maybe return error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33666 from yaarith/nautilus-fix-uuid-str-concat

nautilus: mgr/telemetry: fix UUID and STR concat

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33560 from liewegas/workaround-43914-nautilus

nautilus: qa/tasks/ceph_manager: increase CLI command timeout

Reviewed-by: Neha Ojha <nojha@redhat.com>

rgw: set correct storage class for append

fix https://tracker.ceph.com/issues/42444

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
(cherry picked from commit ef3ec2e56236fb031ddd4404121449c656a62481)

Conflicts:
src/rgw/rgw_putobj_processor.cc
- cur_manifest assignment looks different in nautilus

doc: document `clone cancel` command

Fixes: http://tracker.ceph.com/issues/44208
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 60105f6101d2370f34c2d758e1452acddf962075)

test: add `clone cancel` tests

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit ba4b9358e125eb44f621a4087f46d13636d73d20)

mgr/volumes: introduce "clone cancel" volume command

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit a427c7074ce92e6e832566e0afd43c8d7d8e2770)

mgr/volumes: allow canceling a single asynchronous job for a volume

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 89a16c7867415085dd20c06d87e97bfa7de34fb2)

mgr/volumes: helper for looking up a clone entry index

... this would be used when canceling a clone operation to
find the clone index for a given clone.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 2dc41ea2db423b1e8178d41ce2fef6ca0eeccf32)

mgr/volumes: periodically check if clone operations should be canceled

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 2f93b3beb421f6be4611ffab8d1fb7fbe3fb856c)

mgr/volumes: periodically check if copy operations should be canceled

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 9a0507acb62249e904d54c5ed11ef106f3239bd8)