rgw/notification: trigger notifications on changes from any user

any user authorized to make changes to a bucket may trigger
notifications defined on that bucket.
manual test procedure of the fix is described here:
https://gist.github.com/yuvalif/39c183aa0f74d286ecef7844268817df

Fixes: https://tracker.ceph.com/issues/48461
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 658993efc16498c2106108fe407f6d44040f349d)

Conflics:
src/test/rgw/rgw_multi/tests_ps.py
manual changes were made to the tests so they can pass. this does not affect teuthology runs

Merge pull request #39390 from kotreshhr/wip-mgr-backports-octopus

octopus: mgr/volume: subvolume auth_id management and few bug fixes

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #39836 from rhcs-dashboard/wip-49594-octopus

octopus: mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

qa: leave one standby available to avoid warning

Four file systems will use all MDS and generate this warning:

2020-11-02T03:48:33.407 INFO:teuthology.orchestra.run.smithi003.stdout:2020-11-02T03:24:21.817337+0000 mon.a (mon.0) 481 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY).

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 59451923d31f5e4f707aa6d22ececc8edd395ca9)

qa: Fix a few mgr/volume test cases

Recovering dirty auth metadata file might not retain the order,
fixed the comparison in 'test_recover_auth_metadata_during_authorize'
and 'test_recover_auth_metadata_during_deauthorize'.

Fixes: https://tracker.ceph.com/issues/49192
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 4952d2132ffd92594e749a652970d5d9415c721d)

qa/cephfs: add methods to read/write on CephFS mounts

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 3f0284f272231c3b62b0f3f201cbaaecfa405bcd)

Conflicts:
    qa/tasks/cephfs/mount.py: get_file and IP module is not present in
      octopus

ceph_volume_client: Fix failure of test_idempotency

With the test environment, 'args must be encodeable
 as a bytearray' error is seen for 'ceph_mds_command'.
Hence removed tuple and passed the JSON formatted string.

Fixes: https://tracker.ceph.com/issues/48830
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 1c6c172a9b665d6b769d67e301061dbd7b044472)

mgr/volumes: Evict clients based on auth-IDs and subvolume mounted

Add subvolume evict command which evicts the subvolume mounts
which are mounted using particular auth-ID.

Fixes: https://tracker.ceph.com/issues/44928
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 269adcc8b8ab0742ba741ed7c2b59ccfb17a63f9)

Conflicts:
    qa/tasks/cephfs/test_volumes.py: Few of the tests are re-organized,
           hence the conflicts. Resolved the same.

doc/mgr/volumes: Document 'fs subvolume authorized_list' cli

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 9f9f8adc47486432c746565ea5a1f204736632c1)

qa: Add tests for list auth-ids of a subvolume

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 1b98e63e75236ff1cd1c9cb1ead8eb965698d5b6)

Conflicts:
   qa/tasks/cephfs/test_volumes.py: Few of the tests are re-organized,
       hence the conflicts. Resolved the same.

mgr/volumes: Update the 'volumes' key to 'subvolumes' in auth metadata file

The older auth metadata files created by CephVolumeClient stores the
authorized subvolumes using the 'volumes' key as the notion of
'subvolumes' brought in by mgr/volumes. Hence, this would be tranparently
updated to 'subvolumes' and newer auth metadata files would store them
with 'subvolumes' key.

Also fails the deauthorize if the auth-id doesn't exist.

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 5f32eb15918657bcf1f73025fbb3adf0cc06317e)

mgr/volumes: Optionally authorize existing auth-ids

Optionally allow authorizing auth-ids not created by mgr plugin
via the option 'allow_existing_id'. This can help existing deployers
of manila to disallow/allow authorization of pre-created auth IDs
via a manila driver config that sets 'allow_existing_id' to False/True.

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 713270d1869e2370b674b1a4bc6f6a37023a5917)

mgr/volumes: Preserve existing caps while authorize/deauthorize auth-id

Authorize/Deauthorize used to overwrite the caps of auth-id which would
end up deleting existing caps. This patch fixes the same by retaining
the existing caps by appending or deleting the new caps as needed.

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 2dece3be081fe572455d6b634e38a663d1643dc8)

mgr/volumes: Disallow authorize existing auth_id

This patch disallow the mgr plugin to authorize the auth_id
which is not created via mgr plugin. Those auth_ids could be
created by other means for other use cases which should not be modified
via mgr plugin.

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit c02890404e47a5a95c5cc16b699306045d586c7f)

mgr/volumes: Add subvolume authorized_list command

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 1abec3d0ca8c4fa405cdbf56c55f44f37aca9ca8)

mgr/volumes: Add tenant_id option to subvolume authorize

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 39acfcc91c1b6a85f6fa96a5e894d81a7225f9dc)

mgr/volumes: Persist auth and subvolume metadata

1. Subvolume create and delete operations create and delete subvolume
   metadata file respectively.
2. Subvolume authorize creates the auth meta file and persists the
   required metadata on subvolume metadata file and auth metdata file
   on disk. Subvolume deauthorize clears the required metadata on
   both metadata files.

Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 04d876ced756ca86580bdff4ac116333dbb102e5)

mgr/volumes: Filter inherited snapshots while listing snapshots

Filter inherited snapshots resulted as part of a snapshot
at ancestor level while listing snapshots of a subvolume
and subvolumegroup

Also, fail the snapshot info on inherited snapshot.

Fixes: https://tracker.ceph.com/issues/48501
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit bd49b6409be79dba4a119c809983a05687242732)

Conflicts:
    qa/tasks/cephfs/test_volumes.py: Few of the tests are re-organized,
       hence the conflicts. Resolved the same.

doc/mgr/volumes: Document authorize/deauthorize cli commands

Fixes: https://tracker.ceph.com/issues/40401
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 1426c23ab50a4d64e1c4f1f61185117a270c8ec7)

qa/tasks/cephfs: test `fs subvolume authorize/deauthorize`

Fixes: https://tracker.ceph.com/issues/40401
Signed-off-by: Ramana Raja <rraja@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 7c98dc1ad35a2244cbf949be9782a3412886b305)

Conflicts:
    qa/tasks/cephfs/test_volumes.py: Few of the test cases are
re-organized, hence the conflicts. Resolved the same.

Signed-off-by: Kotresh HR <khiremat@redhat.com>

mgr/volumes: Allow/deny auth IDs access to FS subvolumes

... via the `ceph fs subvolume authorize/deauthorize` command.

Fixes: https://tracker.ceph.com/issues/40401
Signed-off-by: Ramana Raja <rraja@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 6c3b7547fbf3d987e715e9502359acd873374831)

mon/MonCap: allow 'profile mgr' to create/update/del auth IDs

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit d3aea5579778a73dc2418e7b137dc21717944c3e)

Merge pull request #39512 from rhcs-dashboard/wip-48862-octopus

octopus: mgr/dashboard: Monitoring alert badge includes suppressed alerts

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Fixes: https://tracker.ceph.com/issues/49574
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 4b96bb51e8f133badd7bc651bcb4dcd755b43d75)

 Conflicts:
src/pybind/mgr/dashboard/services/auth.py
        - Addressed conflicts.

Merge pull request #39715 from sebastian-philipp/octopus-backport-39069

octopus: cephadm: fix 'inspect' and 'pull'

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #39633 from donggyupark/fix-cephadm-openports

octopus: cephadm: Delete the unnecessary error line in open_ports

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39507 from k0ste/wip-49316-octopus

octopus: mgr/dashboard: prometheus alerting: add some leeway for package drops and errors

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #39325 from ifed01/wip-ifed-add-rocksdb-annex

octopus: os/bluestore: introduce bluestore_rocksdb_options_annex config parame…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39645 from rhcs-dashboard/wip-49388-octopus

octopus: mgr/dashboard: fix tooltip for Provisioned/Total Provisioned fields

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39527 from rhcs-dashboard/wip-49328-octopus

octopus: mgr/dashboard: avoid using document.write()

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39545 from aaSharma14/wip-48734-octopus

octopus: mgr/dashboard:minimize console log traces of Ceph backend API tests

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39819 from yuriw/wip-yuriw-client-upgrade-octopus-pacific-octopus

qa/tests: fixed branch entry

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: fixed branch entry

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #39220 from jan--f/wip-49093-octopus

Merge PR #39220 into octopus

* refs/pull/39220/head:
        ceph-volume: Fix usage of is_lv

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Dimitri Savineau <savineau.dimitri@gmail.com>

Merge pull request #39523 from jan--f/wip-49332-octopus

Merge PR #39523 into octopus

* refs/pull/39523/head:
        ceph-volume: pass --filter-for-batch from drive-group subcommand

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #39469 from jan--f/wip-49289-octopus

Merge PR #39469 into octopus

* refs/pull/39469/head:
        Update batch.py

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #39662 from pponnuvel/wip-49441-octopus

octopus: rgw: radosgw-admin: clarify error when email address already in use

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39577 from dillaman/wip-49252-octopus

octopus: librbd: correct incremental deep-copy object-map inconsistencies

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #39376 from dillaman/wip-48091-octopus

octopus: mgr/rbd_support: create mirror snapshots asynchronously

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #39103 from k0ste/wip-49022-octopus

octopus: mgr/dashboard: trigger alert if some nodes have a MTU different than the median value

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39027 from rhcs-dashboard/wip-48960-octopus

octopus: mgr/dashboard: Fix for incorrect validation in rgw user form

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #39306 from rhcs-dashboard/wip-48829-octopus

octopus: mgr/dashboard/monitoring: upgrade Grafana version due to CVE-2020-13379

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: p-se <NOT@FOUND>

Merge pull request #39271 from jan--f/wip-49141-octopus

octopus: ceph-volume: add some flexibility to bytes_to_extents

Merge pull request #39534 from ideepika/wip-backport-blocklist-ignorelist-qa

octopus: whitelist -> ignorelist for qa/* only

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39235 from k0ste/wip-49107-octopus

octopus: mgr/zabbix: format ceph.[{#POOL},percent_used as float

Reviewed-by: Kefu Chai <kchai@redhat.com>

cephadm: fix 'inspect' and 'pull'

With podman 2.2.1 I would get

ERRO[0000] Error printing inspect output: template: all inspect:1: function "json" not defined

Removing the 'json' portion of the query resolves it.

Fixes: https://tracker.ceph.com/issues/49522
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d31bed79411ca493ec48eeed4e9cbb7ad92295c3)

Conflicts:
src/cephadm/cephadm

cephadm: Delete the unnecessary error line in open_ports

In #39020, d9fbd7e is cherry picked from 70722a2. there is no bug in 70722a2,
but there is a bug in d9fbd7e. It seems that the unnecessary error line was added during cherry picking.
So error only occurs in octopus branch.
This commit directly fixes issue in octopus branch instead of cherry picking
since cherry picking from 70722a2 has already been applied to octopus branch.

This commit deletes the unnecessary error line added in d9fbd7e.
In d9fbd7e, the parameter verbose_on_failure was removed in call.
However, the unnecessary line that uses verbose_on_failure was
added in open_ports and so error occurs.

Fixes: https://tracker.ceph.com/issues/49467
Signed-off-by: Donggyu Park <donggyu_park@tmax.co.kr>

Merge pull request #39277 from smithfarm/wip-49131-octopus

octopus: rgw: fix trailing null in object names of multipart reuploads

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39583 from liewegas/pr-39561-octopus

octopus: qa: use normal build for valgrind

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw/radosgw-admin clarify error when email address already in use

The error message if you try and create an S3 user with an email
address that is already associated with another S3 account is very
confusing; this patch makes it much clearer

To reproduce:

radosgw-admin user create --uid=foo --display-name="Foo test" --email=bar@domain.invalid
radosgw-admin user create --uid=test --display-name="AN test" --email=bar@domain.invalid
could not create user: unable to parse parameters, user id mismatch, operation id: foo does not match: test

With this patch:

radosgw-admin user create --uid=test --display-name="AN test" --email=bar@domain.invalid
could not create user: unable to create user test because user id foo already exists with email bar@domain.invalid

Fixes: https://tracker.ceph.com/issues/49137
Fixes: https://tracker.ceph.com/issues/19411
Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
(cherry picked from commit 05318d6f71e45a42a46518a0ef17047dfab83990)

mgr/dashboard: fix tooltip for Provisioned/Total Provisioned fields

Fixes: https://tracker.ceph.com/issues/46619
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 2dd341d63b7dd7d6b8767b93151c0c1f243be6b6)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-details/rbd-details.component.html
      - Adopting the master branch changes.

15.2.9

qa/valgrind.supp: suppress free[] warnings

This is apparently not going to get fixed any time soon.

https://github.com/gperftools/gperftools/issues/792

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 28fe149ea276693b47cc4d600780971f723672e4)

qa/suites: do not use notcmalloc flavor

teuthology now knows how to run valgrind against a tcmalloc binary

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dc64ccf063f1ddf843fae4d2621ec25187f56135)

# Conflicts:
# qa/suites/rgw/sts/tasks/0-install.yaml
  - test not present in octopus
# qa/suites/rgw/verify/validater/valgrind.yaml

Merge pull request #39289 from kamoltat/wip-ksirivad-octopus-turn-off-backports

octopus: mgr/progress: introduce turn off/on feature

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39256 from ifed01/wip-ifed-fsck-spanning-blobs-octopus

octopus: os/bluestore: detect and fix "zombie" spanning blobs using fsck.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39253 from ifed01/wip-ifed-fix-huge-bluefs-reads-oct

octopus: os/bluestore: fix huge (>4GB) bluefs reads

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #39560 from neha-ojha/wip-revert-39248-octopus

octopus: Revert "mgr/pg_autoscaler: avoid scale-down until there is pressure"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>

rbd-mirror: don't prune older mirror snapshots when pruning incomplete snapshot

Since we normally prune in order, we need to ensure that we don't prune older
snapshots when we need to delete an incomplete mirror snapshot since the
older snapshot might be the only remaining mirror snapshot.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7ba9214ea5b73d0436af6c2896abf4836d741de9)

qa/workunits/rbd: show snapshot deltas during stress test failure

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f079116e87161b55acaa08c55bf8b8e79cee8670)

qa/suites/rbd: add snapshot-based mirroring stress test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 094bfeaf8efe1c4794a2b301314eddedfa5246f9)

librbd/deep_copy: added new migrating flag to object copy

The migration operation and the copyup state machine will set
this flag when attempting to perform a deep-copy due to a
live-migration.

This flag will prevent a possible race condition between the
start of the object deep-copy when migration was enabled and
the writing portion of the deep-copy when migration might
have completed via external means.

Fixes: https://tracker.ceph.com/issues/45694
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1baba64e213cb808804796575d3f7969cf37a3c6)

Conflicts:
src/librbd/deep_copy/ObjectCopyRequest.cc: trivial resolution

librbd/deep_copy: added bitwise flag parameter to object copy

This initial version subsumes the original "flatten" boolean flag.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e79f6b1c157e042f57b577bc510debb21e004ea7)

Conflicts:
src/librbd/deep_copy/ObjectCopyRequest.cc: trivial resolution
src/librbd/io/CopyupRequest.cc: trivial resolution
src/test/librbd/deep_copy/test_mock_ObjectCopyRequest.cc: trivial resolution

librbd/deep-copy: object-copy state machine must update object map

If there was no data to copy, the object-copy state machine was bypassing
the object-map update states and prematurely completing. Since the
object-map is default-initialized to all non-existent objects, this results
in incorrect state for OBJECT_EXISTS_CLEAN objects.

This commit was derived from ca0b9bfc28ef7287ca139ca9640c876223eda87b

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: deep-copy should update object-map before writing to object

For the original use-case of RBD mirroring it was (maybe) more
acceptable to write to the object before updating the object map
because an interrupted sync will be retried. However, when using
the deep-copy object copy state machine as part of copyup, it's
more likely that the object-map has the potential to become
out-of-sync with reality if it's updated after the object is
written.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e782b85bfda8ae6487c637af0059ab94fba332d6)

Conflicts:
src/librbd/deep_copy/ObjectCopyRequest.cc: trivial resolution
src/test/librbd/deep_copy/test_mock_ObjectCopyRequest.cc: trivial resolution

librbd/object_map: diff state machine should track object existence

The deep-copy snapshot-create state machine initializes the object-map
state to non-existent for all objects. There was an assumption that the
deep-copy object-copy state machine would always update the object map
but that was being skipped for clean objects as an optimization. This
change will support a future commit to run the object-copy state machine
for existing objects.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b81cd2460de748c71210520f8c819895f257f0c7)

Conflicts:
src/librbd/api/DiffIterate.cc: trivial resolution due to renames

test/librbd: print difference if deep-copy or migration test fails

It may appear to be useful to track the sporadic test failures
observed on jenkins, not reproducible locally.

Previously it was disabled because the output could be too
large. But after the hexdump was improved to skip repeating bytes
the output will hopefully be much smaller.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit bb77f740df749de1bba0e91b03c4eb23d5586e43)

Revert "PendingReleaseNotes: mgr/pg_autoscaler"

This reverts commit 1177acb6de32331c67d707032a2ecaeff6bf8365.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Revert "mgr/pg_autoscaler: avoid scale-down until there is pressure"

This reverts commit 49dba6edaa807f2c1047c40ce96a76fc5f409b82.

Reason: Needs to bake more given that it may cause pg budget issues
when multiple pools are created at the same time.

More details in https://tracker.ceph.com/issues/49364.

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados/thrash: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c0a358670a8cda2d9e319f0528d6cb9d74b6c605)

qa/suites/krbd/thrash: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 01fb7e7f7b1f5c6d2c8371560ccf295567c0eb07)

qa/suites/multimds: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit a8c7efab2d51a249e716dad710e2ca3d01482f16)

qa: log-whitelist -> log-ignorelist

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2ee9365d0bc220523632753f02c61eb5c2f4fbdd)

Merge pull request #39540 from tchaikov/octopus-pr-35352

octopus: qa/tasks/vstart_runner: do not teardown test_path if "create-cluster-only"

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard:minimize console log traces of Dashboard backend API tests

Changed log levels in some files to reduce console log traces

Fixes: https://tracker.ceph.com/issues/45822
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit f5dd0195cd45b6e00e43ef97e209e565e6452777)

qa/tasks/vstart_runner: do not teardown test_path if "create-cluster-only"

otherwise we could be removing a "None" directory when tearing down the cluster,
and have following failure:

Exception ignored in: <bound method LocalContext.__del__ of <__main__.LocalContext object at 0x7f99fd4a6cc0>>
Traceback (most recent call last):
  File "../qa/tasks/vstart_runner.py", line 1189, in __del__
    shutil.rmtree(self.teuthology_config['test_path'])
  File "/tmp/tmp.mmM2ugspuR/venv/lib/python3.6/shutil.py", line 477, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/tmp/tmp.mmM2ugspuR/venv/lib/python3.6/shutil.py", line 475, in rmtree
    orig_st = os.lstat(path)
TypeError: lstat: path should be string, bytes or os.PathLike, not NoneType

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 954e9a6fa67cce8e3eb8105ee858340b60b84b15)

Merge pull request #39532 from liewegas/pr-39496-octopus

octopus: mgr/cephadm: fix host refresh

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

mgr/cephadm: fix host refresh

Fixes: 01f60cf4e0a751c314120c02956d4ff941eb71b4
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9df5a6d73ed21b394c01afe6c9800b6e50737c90)

mgr/dashboard: avoid using document.write()

Fixes: https://tracker.ceph.com/issues/49220
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 36efe519b90b7508fb830f2c250e8b6956eaaf05)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/app.module.ts
     - Adopting the master branch changes.

ceph-volume: pass --filter-for-batch from drive-group subcommand

Otherwise the drive-group spec potentially tries to pull in the root
device (if it fits).

Fixes: https://tracker.ceph.com/issues/48631
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit a6a03d0f5535a053239ae9e49c53ac9208bb0738)

os/bluestore: introduce bluestore_rocksdb_options_annex config parameter.

This simplifies modifying a subset of rocksdb settings since it
eliminates the need to re-provide bluestore's rocksdb settings.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 711505e8f9effbf4f0ada3bfe8d25dbc7e86861e)

 Conflicts:
  (trivial) PendingReleaseNotes
  (trivial) src/common/options.cc
  (trivial) src/os/bluestore/BlueStore.cc

mgr/dashboard: alert badge includes suppressed alerts

On a cluster with alerting enabled, when alerts are triggered, even if they are silenced, the vertical navigation item (Cluster > Monitoring) displays the total number of alerts, including the ones suppressed.This PR intends to fix this issue.

Fixes: https://tracker.ceph.com/issues/48591
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit b4e32461f1f8430f223965c245974b2a2dbab3aa)

mgr/dashboard: prometheus alerting: add some leeway for package drops and errors (1%)

Fixes: https://tracker.ceph.com/issues/48201
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 9ac248b0c309b95d33bb0b1b64dc2e397bf8508e)

Merge pull request #39393 from kamoltat/wip-ksirivad-octopus-release-notes

octopus: PendingReleaseNotes: mgr/pg_autoscaler

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39230 from ifed01/wip-ifed-fix-pin-octopus

octopus: os/bluestore: fixing onode pinning and more

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #39004 from Vicente-Cheng/wip-48568-octopus

octopus: qa/tasks/cephfs/nfs: Check if host ip is in cluster info output

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Update batch.py

Correct a typo

Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
(cherry picked from commit aa5f48780e469b3ed90f251a01cfa3e2a4f7beb0)

PendingReleaseNotes: mgr/pg_autoscaler

Added details describing the changes that
occured in after merging #39248 into octopus
upstream.

Signed-off-by: Kamoltat <ksirivad@redhat.com>

mgr/rbd_support: create mirror snapshots asynchronously

To scale up with number of images.

Fixes: https://tracker.ceph.com/issues/47827
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 08151a191fd41427a751d966d523cacead5fe8fe)

Conflicts:
src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py: fixed close_image bug

mgr/rbd_support: attempt to unregister watcher on mgr exit

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 0bcaa261510f5bf2c69f2fe411ea3e2efe37ba60)

mgr/rbd_support: make mirror_snapshot_schedule rescan only updated pools

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 69259c8d3722830c13da1e33f61fb6a72151b803)

pybind/rbd: add async mirror image get mode and info methods

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 13ce488d7aaf60f85429c2e4c036362d61e5f763)

librbd: add missing declaration of rbd_aio_mirror_image_get_mode

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit fd5e642cd74c02e8ef00b7018baaa3093167a01f)

pybind/rbd: add image async open and close methods

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit b27db87c0391231b39ecd4b5aacf8ac24193c1cf)

pybind/rbd: async API for creating mirror snapshots

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d1571ed2743f97280a2f3a2a70fff59e0cafc0b9)

Conflicts:
src/pybind/rbd/rbd.pyx: trival resolution re: quiesce flags

librbd: async API for creating mirror snapshots

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d8468ec7e49d93860c3b8b06ff36bde539d10a94)

Conflicts:
src/librbd/api/Mirror.h/cc: no quiesce flags
src/test/rbd_mirror/test_ImageReplayer.cc: no quiesce flags

Merge pull request #39000 from Vicente-Cheng/wip-48521-octopus

octopus: cephfs: client: add ceph.{cluster_fsid/client_id} vxattrs suppport

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38949 from Vicente-Cheng/wip-48644-octopus

octopus: cephfs: client: ensure we take Fs caps when fetching directory link count from cached inode

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38947 from Vicente-Cheng/wip-48642-octopus

octopus: cephfs: client: set CEPH_STAT_RSTAT mask for dir in readdir_r_cb

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38612 from ShyamsundarR/wip-47158-octopus

octopus: mgr/volumes: Add a per subvolume trash

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>