Merge pull request #34214 from neha-ojha/wip-34183-octopus

octopus: doc/releases/octopus.rst: fix min_mon_release string

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34207 from liewegas/bp-34188-octopus

octopus: doc: do not encourage readers to run multiple clusters on same HW

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #34237 from smithfarm/wip-sle-15.2-octopus

octopus: qa/distros: add SLE-15-SP2

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

doc: fix cephadm link

Fixes: 8fe460e7562b28e007179eeb533ef68a3e99b4c8
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7dd012252ce3be13d8d44a5c3829198fc72b5754)

qa/distros: add SLE-15-SP2

Ceph octopus is known to run on SLE-15-SP2 so add it to
qa/distros/all.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
(cherry picked from commit 3261e24636474a3e274c65b859e9325299470f11)

doc/releases/octopus.rst: fix min_mon_release string

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c1c6f8f8c191d206659116bbc04695999478b450)

Merge PR #34194 into octopus

* refs/pull/34194/head:
CMakeLists: 15.2.0
ceph_release: stable, not rc

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #33939 into octopus

* refs/pull/33939/head:
mgr/dashboard: NoRebalance flag is added to the Dashboard

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge PR #34163 into octopus

* refs/pull/34163/head:
mgr: ceph_pg_* metrics contains last value instead of sum all of them

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #34185 into octopus

* refs/pull/34185/head:
mgr/cephadm: DriveGroupSpec needs to know about unmanaged attribute
mgr/cephadm: DriveGroupSpec needs to support/ignore _unmanaged_

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge PR #34186 into octopus

* refs/pull/34186/head:
mgr/cephadm: use docker.io in default container image name
mgr/cephadm: fix 'upgrade start' message when specifying a version

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34190 into octopus

* refs/pull/34190/head:
mgr/cephadm: fix 'orch ps --refresh'

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34191 into octopus

* refs/pull/34191/head:
mgr/cephadm: bind grafana to all interfaces

Reviewed-by: Michael Fritch <mfritch@suse.com>

doc: do not encourage readers to run multiple clusters on same HW

This material was first added on December 22, 2012 by
46d0334456c14da2bc8e012f121c545b1b33b95f

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 8fe460e7562b28e007179eeb533ef68a3e99b4c8)

Merge pull request #34176 from theanalyst/backports/oct-distro-fix

octopus: doc: releases: add a note on debian packages & those built by the community

Reviewed-By: Sage Weil <sweil@redhat.com>

CMakeLists: 15.2.0

I'm not actually sure if this is used for much of anything...

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_release: stable, not rc

This should have happened before v15.2.0.. oops!

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: bind grafana to all interfaces

Fixes: https://tracker.ceph.com/issues/44608
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5f4fedbd5beeccc4792241bcfd2b81bdeb4c7d57)

mgr/cephadm: fix 'orch ps --refresh'

The use of host in the refresh loop was clobbering the argument value.

Fixes: https://tracker.ceph.com/issues/44513
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 764ce71bd202315fdf4403d9e1e8ae55ef849a90)

Merge PR #34154 into octopus

* refs/pull/34154/head:
mgr/cephadm: point alertmanager at all mgr/dashboard URLs

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34155 into octopus

* refs/pull/34155/head:
debian: add ceph-grafana-dashboards package
ceph.spec: put prometheus alerts in vendor-neutral location
mgr/cephadm: include prom alerts, if present in the container

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34157 into octopus

* refs/pull/34157/head:
Add alertmanager adopt

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34158 into octopus

* refs/pull/34158/head:
Add alertmanager sample

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34160 into octopus

* refs/pull/34160/head:
mgr/orch: allow list daemons by service_name

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34153 into octopus

* refs/pull/34153/head:
mgr/cephadm: Add example to run when debugging ssh failures

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34156 into octopus

* refs/pull/34156/head:
cephadm: init config and keyring with None
cephadm: allow config and keyring args for ceph-volume
qa/workunits/cephadm: create lvs before calling cephadm
qa/workunits/cephadm/test_cephadm.sh: move osd test to ceph-volume
qa/workunits/cephadm/test_cephadm.sh: fix mon.b failure

Reviewed-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: use docker.io in default container image name

Not all hosts have docker.io configured as a default registry.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d4f367a7ea96c97c7aa437f79a1c2dabb2c7caf8)

mgr/cephadm: fix 'upgrade start' message when specifying a version

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c6b0c28d748f7eec1455d00be15ab7f0b985ecf5)

mgr/cephadm: DriveGroupSpec needs to know about unmanaged attribute

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit f7bb0236aa601e5439875a97c2250be20956a4e7)

mgr/cephadm: DriveGroupSpec needs to support/ignore _unmanaged_

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit bda6a47aac0d5db6a5b07c50e7ac45c4682a9d0d)

cephadm: init config and keyring with None

and we should not assume that both `config` and `keying` are specified
when calling this method. because, for instance, `create_daemon_dirs()`
does handle the case where `config` and/or `keyring` is not specified.

this is a follow-up fix of 245d6a5cec9cc0f299613b8cc0415e494a4c3ac5

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 86de901d89b26920c1493e3867ddfafa9f9c95cd)

cephadm: allow config and keyring args for ceph-volume

easier to script via the cephadm tests

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit bac530f06a60d96259376cd19224bdbffce1bdc4)

doc: releases: add a note on debian packages & those built by the community

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 2cdca1ff6def0da2b6e6c67b276eaf964b02a010)

mgr: ceph_pg_* metrics contains last value instead of sum all of them

During evaluation of pool stats metrics contains last reported value instead of sum

Fixes: https://tracker.ceph.com/issues/44590
Signed-off-by: Jacek Suchenia <jacek.suchenia@gmail.com>
(cherry picked from commit ad4d790a35af483f3a240f247cda5754301a9199)

mgr/orch: allow list daemons by service_name

Services like rgw and mds are differentiated by service_name. For
example: mds.xyz vs. mds.abc. With current interface, we can't list all
daemons belonged to mds.xyz only. Add service_name as a new argument to
filter daemons by it.

Fixes: https://tracker.ceph.com/issues/44541
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 079c9fd7718ceb17660da61d46019c53012e0f0c)

Add alertmanager sample

Default contents of alertmanager.yml with additional peers key

Signed-off-by: Eric Jackson <swiftgist@gmail.com>
(cherry picked from commit 5cbe0c4f8fc9b7ca24a140643237e13df6851463)

Add alertmanager adopt

Signed-off-by: Eric Jackson <swiftgist@gmail.com>
(cherry picked from commit 354d9d2d58b3991c73eb0fb9b58fcfaf9436afba)

qa/workunits/cephadm: create lvs before calling cephadm

Fixes: https://tracker.ceph.com/issues/44682
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ed0db2445f0de91b2ce53fd133cc8381cb234d1e)

qa/workunits/cephadm/test_cephadm.sh: move osd test to ceph-volume

instead of via the orchestrator

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 26b7a3f8192abbd20563bd8280f17744b64f79e8)

qa/workunits/cephadm/test_cephadm.sh: fix mon.b failure

and is_available check to ensure quorum

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 90cb2feec3165817f5041a6bdc26aa0073a90e00)

debian: add ceph-grafana-dashboards package

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 01015f8ca320e2eab089741e5a8871a4d78c2fc2)

ceph.spec: put prometheus alerts in vendor-neutral location

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f1b516d1bf44f2236c22d5a36f0363f4768e4a6b)

mgr/cephadm: include prom alerts, if present in the container

The prometheus config already looks in/etc/prometheus/alerting/*, so drop
this file into position.

Note that the file is currently only present in SUSE builds, so that needs
to be fixed still.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b1463998bddb67973b59930cc73901143d0961f8)

mgr/cephadm: point alertmanager at all mgr/dashboard URLs

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 41d0966b72b52a78aa63cb8776b07d23668526d9)

mgr/cephadm: Add example to run when debugging ssh failures

```
$ ceph orch host add foobar
Error ENOENT: Failed to connect to foobar (foobar).  Check that the host is reachable and accepts connections using the cephadm SSH key
you may want to run:
> ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) rook@foobar
$  ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) rook@foobar
ssh: Could not resolve hostname foobar: Temporary failure in name resolution
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit c840dd2204e9a419c28875ace18d33ad6f87500f)

Merge pull request #34125 from smithfarm/wip-cbs-deprecate-octopus

octopus: ceph-backport.sh: add deprecation warning

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #34137 from theanalyst/octopus-changelog-1

octopus: doc: add changelog for octopus release

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc: add changelog for octopus release

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit dc7cf3fe8986de7b3e245ea04084eef51c7a0cdf)

Merge PR #34100 into octopus

* refs/pull/34100/head:
doc/cephadm: notes on status/stability
doc/cephadm: some troubleshooting tips

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34115 into octopus

* refs/pull/34115/head:
doc/releases/octopus: drop stray line
doc/releases/octopus: note about repository locations
doc/releases: include octopus in index
doc/install/get-packages: update package install instructions
doc/releases/octopus: final notes

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

doc/releases/octopus: drop stray line

Signed-off-by: Sage Weil <sage@redhat.com>

doc/releases/octopus: note about repository locations

Signed-off-by: Sage Weil <sage@redhat.com>

doc/releases: include octopus in index

Signed-off-by: Sage Weil <sage@redhat.com>

15.2.0

doc/install/get-packages: update package install instructions

- include cephadm method
- update to include new version URLs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #34119 into octopus

* refs/pull/34119/head:
doc: update octopus release notes for rgw

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge PR #34122 into octopus

* refs/pull/34122/head:
cephadm: default to v15 tag for octopus

Reviewed-by: Michael Fritch <mfritch@suse.com>

ceph-backport.sh: add deprecation warning

The script is not maintained in the stable versions of Ceph.

This commit is not a cherry-pick from master, because the script is maintained
in master. It is unmaintained only in the stable versions, like
"octopus".

Fixes: https://tracker.ceph.com/issues/42194
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(manual cherry pick of commit 365a93bd3fe62369d33d07993dd246e9b4dd63fd)

Conflicts:
    src/script/ceph-backport.sh

cephadm: default to v15 tag for octopus

Signed-off-by: Sage Weil <sage@redhat.com>

doc: update octopus release notes for rgw

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 984f3ff7e0fe10f1b54da4271a2a0879ffea0ddb)

doc/releases/octopus: final notes

Signed-off-by: Sage Weil <sage@redhat.com>

doc/cephadm: notes on status/stability

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #34042 into octopus

* refs/pull/34042/head:
mgr/rook: list rgw services
mgr/rook: tolerate timestamps that are None
mgr/orch: add 'subcluster' property to RGWSpec
mgr/rook: do not create radosgw pools
mgr/rook: refactor apply/add for rgw
mgr/cephadm: configure rgw_frontends for rgw service
mgr/orch: accept port and ssl flags to 'apply rgw'
python-common/ceph/deployment/service_spec: add ssl to RGWSpec
mgr/rook: fix 'orch ps' for osds

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #33959 into octopus

* refs/pull/33959/head:
qa: fix all the fsx.sh-invoking yaml files to install dependencies

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #34067 into octopus

* refs/pull/34067/head:
mgr/DaemonServer: add explicit check that acting matches for merge

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr/DaemonServer: add explicit check that acting matches for merge

Add an explicit check that the PG acting for the source and target
match before merging.

Fixes: https://tracker.ceph.com/issues/44684
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #34040 from dillaman/wip-44396-partial-fix

rbd-mirror: snapshot-based mirroring should use image sync throttler

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #34098 into octopus

* refs/pull/34098/head:
cephadm: relabel /etc/ganesha mount

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/rook: list rgw services

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: tolerate timestamps that are None

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: add 'subcluster' property to RGWSpec

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: do not create radosgw pools

First, we don't know how big they should be or what they should look like.
The caller should already know that, and/or radosgw can create the pools
itself.

This depends on https://github.com/rook/rook/pull/5058

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: refactor apply/add for rgw

A few caveats here:

- enforce that realm == zone, since that is all rook does at the moment.
- we force a (bad!) pool configuration, since rook requires that these
be present (instead of allowing radosgw or the caller to create the pools)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #34082 into octopus

* refs/pull/34082/head:
cephadm: switch grafana image to the ceph repo

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34068 into octopus

* refs/pull/34068/head:
mgr/cephadm: clean up client.crash.* container_image settings after upgrade

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

cephadm: relabel /etc/ganesha mount

Fixes: https://tracker.ceph.com/issues/44701
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #34046 into octopus

* refs/pull/34046/head:
qa/suites/rados/cephadm/upgrade: update starting version
mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34092 into octopus

* refs/pull/34092/head:
doc/releases/octopus.rst: add note about ec recovery below min_size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33719 from ukernel/wip-44416

mds: pass proper MutationImpl::LockOp to Locker::wrlock_start()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

rbd-mirror: leader watcher should not cancel get locker if locker is invalid

When a new leader acquires the lock, it will send out a lock acquired
notification along with periodic heartbeats. The get locker will attempt to
run immediately, but if a heartbeat arrives before it executes the heartbeat
will cancel the timer and reschedule it for the future. This process repeats
for each periodic heartbeat and the locker is never re-read from the OSD.

This is an issue only for namespace replayers due to the delayed fashion in
which the leader instance id is retrieved.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: snapshot sync request needs to check for interruption

If the sync request was locally canceled, we need to resume the paused
shut down logic instead of just notifying the image replayer state
machine of the change -- since it had already requested a shut down and
will not re-request it.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: request exclusive lock when moving to trash

Even if the image is in-use, moving it to the trash does not
remove any data. This also solves a race between snapshot-based
mirroring shutting down and being able to move a mirrored image
to the trash.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: basic integration with sync throttling

snapshot-based mirroring did not have any throttling to prevent
too many concurrent syncs from running. Since each sync might need
to iterate over every object of an image, that could potentially
put an extreme burden on the remote cluster.

A future PR will add a more intelligent throttle based on the actual
number of objects needed to be scanned.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: don't prematurely finish snapshot replay loop

The unlink step was being incorrectly skipped if a state machine
shut down was requested.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: pass InstanceWatcher to snapshot Replayer

The replayer will request image syncs via the instance watcher in the next
commit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/releases/octopus.rst: add note about ec recovery below min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>

mgr/cephadm: configure rgw_frontends for rgw service

This assumes the beast frontend.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: switch grafana image to the ceph repo

Pull the pre-built grafana image from the ceph org on
docker.io

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

Merge PR #34034 into octopus

* refs/pull/34034/head:
cephadm: make add-repo --release and --version independent

Reviewed-by: Sebastian Wagner <swagner@suse.com>

qa/suites/rados/cephadm/upgrade: update starting version

Need upgrade command to match!

Signed-off-by: Sage Weil <sage@redhat.com>

doc/cephadm: some troubleshooting tips

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33540 into octopus

* refs/pull/33540/head:
doc/orchestrator: update rgw creation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34023 into octopus

* refs/pull/34023/head:
mgr/test_orchestrator: fix service filtering when using dummy data
mgr/dashboard: fix adding/removing host errors

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge PR #34044 into octopus

* refs/pull/34044/head:
remove ceph_test_rados_watch_notify

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #34030 into octopus

* refs/pull/34030/head:
cephadm: env over last used
cephadm: fall back to default for infer_image
cephadm: remove outdated check
cephadm: consolidate default image logic
cephadm: only infer image for shell, run, inspect-image, pull, ceph-volume

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

doc/orchestrator: update rgw creation

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mgr/cephadm: clean up client.crash.* container_image settings after upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #34027 into octopus

* refs/pull/34027/head:
qa/workunits/cephadm/test_cephadm: mark services unmanaged for test
mgr/cephadm: do not reconfig unmanaged services
qa/workunits/cephadm/test_cephadm: output file for pub key

Reviewed-by: Sebastian Wagner <swagner@suse.com>

cephadm: make add-repo --release and --version independent

Specify either --release name (to get the latest) or --version x.y.z to
get a specific version.

Adapt to updated locations on download.ceph.com so that we don't need to
know the release name for a specific x.y.z release.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #34045 into octopus

* refs/pull/34045/head:
mgr/progress: fix duration strings

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>