qa/suites/rados/thrash-old-clients: only centos and 16.04

We don't have old client builds for the newer distros.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dd46ae1006e580b4f25d190732e7b4b400501955)

Merge pull request #22105 from dillaman/wip-rbd-mirror-policy-mimic

mimic: rbd-mirror: optionally support active/active replication

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22099 from liewegas/wip-pgmap-digest-decode-mimic

mon/PGMap: fix PGMapDigest decode

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22104 from tchaikov/mimic-24180

mimic: mon/LogMonitor: call no_reply() on ignored log message

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22051 from b-ranto/wip-mgr-restarts

mimic: Fix ceph-mgr restarts

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reveiwed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mon/LogMonitor: call no_reply() on ignored log message

If we're dropping it on the floor, we need to tell the mon that, so that
it can tell the forwarding mon to discard its state.

Fixes: https://tracker.ceph.com/issues/24180
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9661fa08ccc500587d35d7af8a905916167d8314)

Merge pull request #22101 from liewegas/wip-mimic-upgrades-mimic

mimic: qa/suites/upgrade/luminous-x fixes

qa/suites/thrashosds-health: whitelist REQUEST_SLOW

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 459b33e146d85cf4c03c07bea0718248a0403e0a)

qa/suites/upgrade/luminous-x: whitelist SLOW_OPS

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 59fe5dc5ce86f4f936a6be8a1f8014a184f5d33c)

qa/suites/upgrade/luminous-x: more cinder volumes

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5a5e74e89c1296b156a4a2a1fa9f54fa5c535fe2)

Merge pull request #22068 from yuriw/wip-yuriw-mimic-backport-21895-mimic

mimic: tests/qa: adding rados/.. dirs

cls/rbd: fixed last read key for interrupted mirror_image_map_list

Previously, the image map would only return a maximum of 64 mappings.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3cbe0cedcd3057066cee88e98ccf8a19dbdf52d2)

qa/suites/rbd: reduce runtime of rbd-mirror fsx test case

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 61e94e6905ca6bc6931782277dfd63478f281898)

qa/suites/rbd: added active/active rbd-mirror tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e2d5f8441aae08dccded6ec740181efc54bde19a)

qa/workunits/rbd: fixed rbd-mirror snapshot comparison loop

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 917f8a037cf2e9e8795fd4bedeffc8655b515bac)

qa/workunits/rbd: fixed missing error code on trap handler

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 02e32fe45a3c6c922feeb09fd1e6b95f90278674)

qa/workunits/rbd: skip rbd-mirror start/stop cases when thrashing daemons

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3086f29b64adef963ce9d9292cc5275b91d85952)

qa/workunits/rbd: rbd-mirror admin socket commands should retry

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a6a8c8a2abfe19b10f9315946eb48f0c1cf630f4)

qa/workunits/rbd: don't override rbd-mirror logs if running under teuthology

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1bb6d4fd89c6434de9441581f125eb8ef55c6d07)

qa/workunits/rbd: append rbd-mirror instance number for all-daemon asok command

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 15a197ed06ce80dd0a42d2467595908095cf648e)

test/rbd: fsx should utilize the default image features

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c1e111e9855047d69762d06bb9094f93c6c12139)

test/rbd_mirror: fix race condition with image map timer callbacks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2534e5e4558b624eb62639a1f620dfb42b21044c)

librbd: deep-copy should remove parent metadata if overlap updated

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit af4298464d6809969eb35382b3cbbb1ea0b3bd1e)

librbd: journal replay should no-op out-of-bounds IO

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9a1fcef5edbfdfc77293c4b60c9e038db49abf1f)

librbd: don't assert if parent pool doesn't exist

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 88436b58bca5b271403f583f6b95e1d30ea81f8f)

rbd-mirror: image create should lookup local parent snapshot id

A recent code change associated with a librbd cleanup incorrectly started
using the remote parent image's snapshot id.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 02afa2a5b41074c651e62a5475ed4828df982ed4)

rbd-mirror: do not propagate image map instance updates when shutting down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4525b7a597ccfa6e966243e24bd9e9662c994a41)

rbd-mirror: propagate image delete if leader was offline

In an active/active scenario, if the leader was offline while mirroring for
a remote image was disabled, the assigned replayer instance may not detect
the image removal.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b3acc569837985590fad2705ae2dcd7f148667a5)

rbd-mirror: avoid sending duplicate re-acquire messages upon leader startup

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a9d335d8399984b338d7cfe2b7d49e1f67940ebf)

rbd-mirror: reduce unnecessary logging for periodic start image replayer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d7cb5db188faf133904ec4532620bbd210cb60e5)

rbd-mirror: cancel image replayer status timer task at shut down

Also avoid attempting to send status using an invalid librados::IoCtx
handle due to a deleted pool.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e2a00887c1a2c08cab8566d930dede1dac186b9b)

rbd-mirror: policy map shouldn't default to first instance if dead

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 524f08cef697c94b331d52b2077c09a0b3ff9d2e)

rbd-mirror: prevent blacklisting of local instance after failover/back

If the leader role is manually released, upon failback the instance
will have removed its local instance object, preventing RPC
messaging.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ce974305893c4262b62547ce184d514578358218)

rbd-mirror: instantly remove instances without associated images

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit bce5328a328f3f9b98674d9e4611a2a40bdf3ddb)

rbd-mirror: instance replayer now respects manual stop command

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 06e1244ee53640edbbab0168d3050152d048d681)

rbd-mirror: support optionally enabling active/active mirroring

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 53b87b9d22b268e57c293e977aa8220bc2b1fddf)

mon/PGMap: fix PGMapDigest decode

The compat path wasn't clearing the map before filling in new entries.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 42bdb066bb0397dbb84a2ca60ec4640cdd0afdef)

Merge pull request #22004 from tchaikov/mimic-22420

mon,osd: dump "compression_algorithms" in "mon metadata"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22005 from batrick/i23768-mimic

mimic: MDSMonitor: clean up use of pending fsmap in uncommitted ops

Merge pull request #22013 from wido/mimic-telegraf

mimic: mgr/telegraf: Telegraf module for Ceph Mgr

Merge pull request #22041 from dzafman/wip-24045-mimic

mimic: osd: Don't evict even when preemption has restarted with smaller chunk

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22079 from LenzGr/mgr-dashboard-documentation-mimic

mimic: doc: Updated dashboard documentation (features, SSL config)

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: Updated dashboard documentation (features, SSL config)

Updated Ceph manager dashboard documentation: added SSL to the
feature overview, added new section about configuring SSL certificates.
Rearranged installation instructions for setting host name and port.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
(cherry picked from commit 32e0660556ee0566b340b29cb9dbd2fdb8c17bfd)

Merge pull request #22052 from tspmelo/wip-momentjs

mimic: mgr/dashboard: Fix moment.js deprecation warning

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #22070 from yuriw/wip-yuriw-mimic-backport-21932-mimic

mimic: tests/qa: Adding $ distro mix - rgw

Reviewed-by: Casey Bodley <cbodley@redhat.com>

tests/qa: Adding $ distro mix - rgw

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 7ce916394881a0e8db2a160f246af0d907aedb19)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

tests/qa: adding rados/.. dirs

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 9f2c485942746bdef532f762d7bc27b21533d218)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

tests/qa: trying $ distro mix

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 2618aa5325d49f52be6c9b78300dab6d5db4fdc4)

Merge pull request #21979 from votdev/bug_24097_mimic

mimic: mgr/dashboard: Navbar dropdown button does not respond for mobile browsers

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22032 from votdev/fix_deletion_dialog_mimic

mimic: mgr/dashboard: Deletion dialog falsely executes deletion when pressing 'Cancel'

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #22029 from votdev/fix_deletion_mimic

mimic: mgr/dashboard: Handle errors during deletion

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22048 from tchaikov/mimic-23653

mimic: cmake: do not check for aligned_alloc() anymore

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Fix moment.js deprecation warning

Fixes: https://tracker.ceph.com/issues/24115
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 89ae2577303a6b48b8822cddd3ec4fc255b6ff1f)

dashboard: Fix dashboard shutdown/restart

The dashboard serve method uses block which it should not as it can
cause several problems when shutting down (restarting) ceph-mgr. As a
result the restart of the ceph-mgr module will block until time-out if
the dashboard module is enabled. This fixes the problem by using a
simple simple shutdown_event instead of block() and stopping the server
afterwards.

Signed-off-by: Boris Ranto <branto@redhat.com>

prometheus: Fix prometheus shutdown/restart

The prometheus serve method uses block which it should not as it can
cause several problems when shutting down (restarting) ceph-mgr. As a
result the restart of the ceph-mgr module will block until time-out if
the prometheus module is enabled. This fixes the problem by using a
simple simple shutdown_event instead of block() and stopping the server
afterwards.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit a662caa79b8fc30e36df0da93f368b470d26564e)

cmake: do not check for aligned_alloc() anymore

this reverts 16f0a67d

Fixes: http://tracker.ceph.com/issues/23653
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4d646ef579ee2f820a1cb0a7b0bd99b9a6cf20ea)

Merge pull request #22038 from dillaman/wip-22787-mimic

mimic: librbd: deep copy optionally support flattening cloned image

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22039 from dillaman/wip-24141-mimic

mimic: rbd-mirror: potential deadlock when running asok 'flush' command

Reviewed-by: Mykola Golub <mgolub@suse.com>

osd: Clear part of cleaned_meta_map in case of a restarted smaller chunk

This can not happen at the primary because scrub_compare_maps() is only
called once per chunk start.

Preemption causes a smaller chunk from start to be processed again at
replicas.  We clear any of the previous chunk's information.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9e0ac797c602a088447679b04e14ec0cfaf9dd7b)

osd: Don't evict even when preemption has restarted with smaller chunk

Fixes: https://tracker.ceph.com/issues/24045
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 818b59fa95ee60e86991276f18c4dee405dc79b1)

rbd-mirror: make 'flush' asok command asynchronous

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 6fce4cabe7874efef780c7adce8479db8571afe7)

rbd-mirror: don't reregister asok hook in process entry thread

It might get stuck if the asok is busy executing a command.
Instead kick status update, which will call reregister on finish.

Fixes: http://tracker.ceph.com/issues/24141
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 43e50bd582ba2d1b66f55d7f8d2b7f030da99ee6)

rbd: deep copy option to copy clone parent data

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit e387f9195545522351306ce420bd758b888dc774)

librbd: deep copy optionally support flattening cloned image

Fixes: http://tracker.ceph.com/issues/22787
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 0d43d2faceca158cc9cf2177802b80dda2b554c9)

librbd/deep_copy: adjust in-memory overlap if set_size updated it on disk

so set_parent won't be called when overlap is changed due to image
size change.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit ca0893c4d9f54e3d80cf7ea90c8b4431d1f5a4f0)

librbd: hide parent object when deep copying empty child

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 3d884eaee89d39570f0dbe9bf1f25f7d27bcd907)

librbd: fix deep copy clone to destination with larger object size

When assembling a destination object from source objects we need to
read from parent if some (but not all) of assembles do not exist.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 947993a8d36f60bde05bc7b9a49c333dd51da4c5)

Conflicts:
src/test/librbd/deep_copy/test_mock_ImageCopyRequest.cc: trivial resolution

test/librbd: add deep copy clone tests

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 928540272f16911e3e0867db606ac5be40f49c27)

Merge pull request #21836 from tchaikov/mimic-23962

mimic: pybind/ceph_daemon: expand the order of magnitude of

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21862 from cbodley/wip-gcc-8-mimic

mimic: cmake,common,filestore: silence gcc-8 warnings/errors

Merge pull request #21958 from tchaikov/mimic-24000

mimic: osdc: self-managed snapshot helper should catch decode exception

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #21959 from tchaikov/mimic-23899

mimic: osd: clean up smart probe

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21974 from tchaikov/mimic-22302

mimic: selinux: Allow ceph to execute ldconfig

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

mgr/dashboard: Handle errors during deletion

- Take care that the deletion dialog gets notified about errors to stop the progress spinner and enable the delete button again.
- Prettify JS code.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit b2fa819fdfee93e6951666198720bce44c37b95d)

mgr/dashboard: Deletion dialog falsely executes deletion when pressing 'Cancel'

If a deletion fails and the 'Cancel' button is pressed to close the dialog, then the deletion process is triggered again.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 0473b02872f9dbc1993ec5d6686484c24986e85c)

Merge pull request #21977 from votdev/info_no_rgw_mimic

mimic: mgr/dashboard: Display notification if RGW is not configured

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/telegraf: Telegraf module for Ceph Mgr

Telegraf is a agent for collecting and reporting metrics.

It has multiple inputs and can send data to various outputs like
for example InfluxDB or ElasticSearch.

This module works by using the socket_listener of Telegraf and can
send data over UDP, TCP and a local Unix Socket.

Signed-off-by: Wido den Hollander <wido@42on.com>
(cherry picked from commit d15d510babe06a54a17a6949ed7c22c495b08b97)

Merge PR #21988 into mimic

* refs/pull/21988/head:
mds: handle imported session race
mds: check for session import race

Reviewed-by: Zheng Yan <zyan@redhat.com>

MDSMonitor: clean up use of pending fsmap in uncommitted ops

Gist of this commit is to eliminate use of PaxosFSMap::get_working_fsmap which
allowed looking at the pending FSMap depending if it's the leader. Instead,
pass the FSMap being worked on to helper functions.

Also, ensure that we are only looking at the committed (i.e. current) FSMap
in the preprocess_* service methods.

Fixes: http://tracker.ceph.com/issues/23768
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit dd68c2386cff7857c4466216db80cfdb4f0353b5)

MDSMonitor: refactor last_beacons to use mono_clock

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 1933bdf37098130342d0debacf4daa0a41011351)

mds: refactor MDSMap init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7be74fd004686dc12c3c6b8d2be5f83cb2f3dea8)

mds: refactor FSMap init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 344be27d696f8f7f9522f75c8aae55a2b62cdd7c)

mds: refactor Filesystem init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 91c01dc958c8bae29ee31a38e63187a341376ecc)

mds: move compat set methods to MDSMap

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit db3a24c94924dbe12796307704d3b12278c8c8a3)

mon,osd: dump "compression_algorithms" in "mon metadata"

Fixes: http://tracker.ceph.com/issues/22420
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1684ac34331f48e59c0485c3e18b1fe55b66c690)

compressor: use initializer_list for compression_algorithms

use a compile-type array instead of heap-allocated tree to associate
compression types with their names

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f9b76c62c0c0ddad77007893bbf62bcc2e6ee945)

Merge pull request #21978 from votdev/improve_error_panel_mimic

mimic: mgr/dashboard: Improve error panel

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mds: handle imported session race

Thanks to Yan Zheng for identifying and reviewing the fix.

Fixes: http://tracker.ceph.com/issues/24087
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b9c3751e9e375c679bada1fab228b52420cf7a4b)

mds: check for session import race

Credit to Yan Zheng for identifying the race condition [1].

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1572555#c7

Test-for: http://tracker.ceph.com/issues/24072

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit fbc25b44865f6c13c9a9c4710049f9e37169747b)

Merge pull request #21970 from wido/mgr-brag

mgr/telemetry: Add Ceph Telemetry module to send reports back to project

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21956 from tchaikov/mimic-pr-21942

mimic: qa/workunits/ceph-disk: --no-mon-config

mgr/dashboard: Navbar dropdown button does not respond for mobile-like browsers

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit b3175d9fa777ea4d7437f0c6d0f281b2b36d7560)

mgr/dashboard: Improve error panel

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit a8dae80dce973c441db3d4b914febd4b48725f1e)

mgr/dashboard: Display useful info if RGW is not configured

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 83441af93cae11e62286a99e65e470a5788e8429)

mgr/dashboard: Add ModuleStatusGuardService route guard

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit f49e192dd56fb471961cebe71583c16b823e0bed)

mgr/dashboard: Add info panel component

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit d1b179d757aac258d3ce0abc162ed8f6550a535a)

Merge pull request #21976 from ukernel/mimic-revert-23837

Revert "ceph-fuse: Delete inode's bufferhead was in Tx state would le…

Revert "ceph-fuse: Delete inode's bufferhead was in Tx state would lead a assert fail"

This reverts commit ede7817a36e7b10f01f906000ce3ed621c0fc358.
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

selinux: Allow ceph to block suspend

Ceph is now trying to block suspend in certain cases, we need to update
the policy accordingly.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit e3be372d854f9014ca711f14525ca9e510d34f1a)

selinux: Allow ceph to execute ldconfig

The ceph-volume testing showed that the ceph daemons can run ldconfig in
a corner case when they are forbidden access to some files. This patch
allows ceph to execute ldconfig in Enforcing mode.

Fixes: https://tracker.ceph.com/issues/22302
Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit fa5071b6d7182f54cd7b1ffe171a4b006f5255cb)