Merge pull request #43655 from liewegas/auth-rotate

mon,auth,cephadm: support auth key rotation

Merge pull request #47967 from yangxiaolaing/cephadm_doc

doc/cephadm: Add creation method of OSD

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47521 from adk3798/iscsi-ip-list

mgr/cephadm: reconfig iscsi daemons if trusted_ip_list changes

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47640 from adk3798/no-cgroups

mgr/cephadm: make setting --cgroups=split configurable

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47936 from adk3798/tuned-profile-dashes

mgr/cephadm: fix tuned profiles getting removed if name has dashes

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47786 from rkachach/fix_issue_57192

mgr/cephadm: validating tuned profile specification

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #48008 from adk3798/docs-extra-container-mounts

doc/cephadm: document recommended syntax for mounting files with ECA

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

mgr/cephadm: validating tuned profile specification
fixes: https://tracker.ceph.com/issues/57192
fixes: https://tracker.ceph.com/issues/57191

Signed-off-by: Redouane Kachach <rkachach@redhat.com>

mgr/cephadm: reconfig iscsi daemons if trusted_ip_list changes

This is necessary as iscsi daemons expect the iscsi-gateway.cfg
which includes this list to match across the iscsi daemons

Fixes: https://tracker.ceph.com/issues/57078
Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #47898 from adk3798/no-sudo-true

mgr/cephadm: don't use "sudo" in commands if user is root

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

mgr/cephadm: fix tuned profiles getting removed if name has dashes

Previously, due to the way it was just splitting the file name on
dashes to try and get the profile name, any profile name with dashes
was not getting properly matched and would therefore get marked
stray and removed. This new strategy instead tries to match the actual
expected file name.

Fixes: https://tracker.ceph.com/issues/57405
Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #47469 from lxbsz/cleanup_C_MDC_EmptyImport

mds: remove the stale class C_MDC_EmptyImport code

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>

Merge pull request #48033 from joscollin/wip-kill-cephfs-top

stop.sh: kill cephfs-top

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48006 from rkachach/fix_issue_57422

mgr/cephadm: Adding --storage.tsdb.retention.size prometheus option

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47574 from rhcs-dashboard/osd-form-hostname-filter

mgr/dashboard: osd form preselect db/wal device filters

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

Merge pull request #48001 from aisakaki/wip-omap-inclusive-fix

crimson/seastore: fixs on omap list open/closed range

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #47968 from rhcs-dashboard/add-alert-silence-button

mgr/dashboard: Add a Silence button shortcut to alert notifications

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47305 from zdover23/wip-doc-2022-07-25-pr4600-cleanup

doc/monitoring: add min vers of apps in mon stack

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #47575 from zdover23/wip-doc-2022-08-13-rados-operations-pools-prompt-update

doc/rados: add prompts to pools.rst

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #48042 from SUSE/wip-fix-57497

ceph.spec.in: Use gcc11-c++ on openSUSE Leap 15.x

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47771 from linuxbox2/wip-librgw-frontends

librgw: add ability to conditionally export HTTP frontends/apis

rgw/main: if !nfs, register service map as "rgw"

This was accidentally changed to "nfs"--which breaks various
ceph orchestration assumptions.

Found by Ernesto Puerta <epuertat@redhat.com>

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #47854 from phlogistonjohn/jjm-fix-57268

qa/tasks/kubeadm: set up tigera resources via kubectl create

Reviewed-by: Adam King <adking@redhat.com>

doc/cephadm: document recommended syntax for mounting files with ECA

Mounting files with extra container args seems to not
work with some of the more intuitive syntaxes.
See https://tracker.ceph.com/issues/57338

Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #48007 from adk3798/fix-staggered-pacific-version

doc/cephadm: correct version staggered upgrade got in pacific

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #46344 from mjenglish/ceph-volume-multiple-actuator

ceph-volume/ceph_volume/util: ceph-volume multi-actuator drive attribute

Merge pull request #47970 from soumyakoduri/wip-skoduri-coverity

rgw/dbstore: Fix coverity issues #1510412, #1511420

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

mgr/cephadm: fix custom config unit test

this test needs to be updated to account for the
new keyring information being introduced

Signed-off-by: Adam King <adking@redhat.com>

mgr/cephadm: execute conf update when rotating secret

This is a fixup for: mgr/cephadm: add daemon 'rotate-key' action

The way we rotate mgr's secret require:

1) writing the new pending-key to mgr's file,
2) restarting the mgr via the `mgr fail` mon command.

Unfortunately, we might be doing the first step wrongly.
`_create_daemoan()` is a coroutine (Python's `async def`)
while we don't `wait` for it. IIUC the underlying doc
correctly, this has no effect, and thus the mgr is restarted
with old key.

  "Note that simply calling a coroutine will not schedule
  it to be executed"

See: https://docs.python.org/3/library/asyncio-task.html#id1

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mon/AuthMonitor: don't forget about caps on "auth get".

The issue came in 98b89120321059397798170f7ae2bf7c64e4f4b2.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/cephadm: add crash keyring to unit test

Signed-off-by: Adam King <adking@redhat.com>

auth: refactor KeyServer::get_used_pending_keys().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph-crash: ping cluster with key on startup

This is mostly pointless, *except* that after a key rotation it ensures
that the new key is used immediately (and the pending_key is committed by
the mon).

Signed-off-by: Sage Weil <sage@newdream.net>

doc/cephadm/operations: document 'ceph orch daemon rotate-key'

Signed-off-by: Sage Weil <sage@newdream.net>

doc/cephadm/operations: document daemon commands

Signed-off-by: Sage Weil <sage@newdream.net>

qa/suites/orch/cephadm/with-work: add key rotation test

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: add rotate support for rgw, crash, nfs, rbd-mirror, iscsi

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: add daemon 'rotate-key' action

Caveats:
- only works with osd, mds, mgr so far
- sometimes we have to restart the daemon

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm/service: factor out get_auth_entity()

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: log exceptions in daemon actions

Signed-off-by: Sage Weil <sage@newdream.net>

mon/AuthMonitor: no stderr output on success

These messages are distracting.

Signed-off-by: Sage Weil <sage@newdream.net>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd: add 'rotate-stored-key' asok command

This writes the key to the osd_key in the block device label.

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: deploy pending_key when possible

Also, leave out the caps.

Signed-off-by: Sage Weil <sage@newdream.net>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/cephadm: fix keyring when updating caps

Signed-off-by: Sage Weil <sage@newdream.net>

mon/AuthMonitor: include pending_key in 'auth get-or-create' result

Signed-off-by: Sage Weil <sage@newdream.net>

mon/MonClient: add 'rotate-key' asok command

Rotate the live auth key for a running daemon without restarting.

Signed-off-by: Sage Weil <sage@newdream.net>

common/admin_socket: pass inbl to sync call() method

Only the async_call got this before.

Signed-off-by: Sage Weil <sage@newdream.net>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

qa/workunits/mon/auth_key_rotation.sh: exercise pending key / rotation

Signed-off-by: Sage Weil <sage@newdream.net>

mon: automatically commit used pending_keys

Signed-off-by: Sage Weil <sage@newdream.net>

auth: keep track of used pending_keys

Signed-off-by: Sage Weil <sage@newdream.net>

mon/AuthMonitor: 'auth {get-or-create,clear,commit}-pending'

Add commands to create, clear, or commit pending_key.

Signed-off-by: Sage Weil <sage@newdream.net>

auth/cephx: authenticate with either key or pending_key

Signed-off-by: Sage Weil <sage@newdream.net>

auth: add PendingKey to EntityAuth

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Add a Silence button shortcut to alert notifications

Fixes: https://tracker.ceph.com/issues/57457
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #47976 from rhcs-dashboard/fix-duplicate-snapshot

mgr/dashboard: fix snapshot creation with duplicate name

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

rgw/dbstore: Fix coverity issues #1510412, #1511420

Fixes: https://tracker.ceph.com/issues/57322
Fixes: https://tracker.ceph.com/issues/57325
Signed-off-by: Soumya Koduri <skoduri@redhat.com>

ceph.spec.in: Use gcc11-c++ on openSUSE Leap 15.x

Fixes: 80949babab4aac86647ce9973f6b44adb92fae95
Fixes: https://tracker.ceph.com/issues/57497
Signed-off-by: Tim Serong <tserong@suse.com>

Merge pull request #48032 from rhcs-dashboard/openapi-check-fix

mgr/dashboard: fix openapi-check

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: docs gen tags sort

When generating tags the order of endpoints wasn't taken into account.
Two endpoints with the same url prefix, for example `/api/cluster/` and
`/api/cluster/user`, have different docs and the tags is generated from
a doc of one of these two, and since the order of these endpoints might
vary it is imperative to sort them to have a deterministic output.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #47770 from rzarzynski/wip-doc-dups-trimming-relnotes

PendingReleaseNotes: document online and offline trimming of PG Log's…

Reviewed-By: Neha Ojha <nojha@redhat.com>

rgwlib: ensure AppMain is destructed before CephContext

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: cause ratelimiter to destruct in AppMain::shutdown()

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

librgw: fix mismatched register/unregister sigterm handler

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main:  fix init order of rgw_main's init_storage()

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: move service_map_meta into AppMain

The RGWRealmReloader holds a reference on it.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: move signal handler actions out of shutdown()

AppMain::shutdown() had borrowed the unregister_async_signal_handler()
text from librgw, but of course, that's wrong.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlib: robustify non-default initialization

Don't rely on the conf-default value of rgw_nfs_frontends to
provide "rgw-nfs," as that's not optional when running under
librgw.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_main: remove unused tracepoint decls

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlib: rework linkage

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_a: build reorganization

Reorganize some dependencies shared by the librgw.so and radosgw
main units.  The librgw.so build no longer builds a main(...)
unit.

Build time should be slightly reduced, as several shared dependencies
have been moved into rgw_a.

There are some other cleanups, e.g., rgw_signal.{h,cc}.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: remove friends, see_also

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: various doc and comment corrections by doc writers

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: use AppMain in c++ main function

Consolidate standalone and library initialization into one
unit.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: AppMain

s/rgw::InitHelper/rgw::AppMain/;

Move program state from RGWLib into AppMain, preparing it to
manage program state for all RGW instances (e.g., main()).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

librgw: add ability to conditionally export HTTP frontends/apis

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #47700 from s0nea/wip-rgw-overview-labels

monitoring/ceph-mixin: add RGW host to label info

Reviewed-by: MrFreezeex <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

stop.sh: kill cephfs-top

Signed-off-by: Jos Collin <jcollin@redhat.com>

mgr/dashboard: osd form preselect db/wal device filters

If the hostname is selected for the primary devices, then we can
preselect the hostname filter for the db/wal devices because osds will
be deployed only on the hostname of the primary device. If preselected
it'll be clear that only this devices will be used to deploy.

Addition to this, usually ssd devices are used for db/wal devices. So I
am preselecting these too in the filters.

Fixes: https://tracker.ceph.com/issues/57118
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47784 from rhcs-dashboard/ingress-unmanaged

mgr/dashboard: fix unmanaged service creation

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #47959 from rhcs-dashboard/service-id-filter-undefined

mgr/dashboard: fix Cannot read properties of undefined (reading 'filter')

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

auth/Crypto: add clear()

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #48014 from idryomov/wip-57116-cont

test/{librbd, rgw}: increase delay between and number of bind attempts

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #48003 from dang/wip-dang-zipper-admin-api

RGW - Zipper - Allow per-store admin APIs

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #48009 from ljflores/wip-install-deps-focal-bionic

install-deps.sh: fix install-deps script for focal and bionic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

RGW - Zipper - Allow per-store admin APIs

Some of the existing admin APIs are rados-specific, and other stores
will want to add specific APIs.  Allow this by having a function to add
store-specific APIs.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

PendingReleaseNotes: document online and offline trimming of PG Log's dups

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/cephadm: Adding --storage.tsdb.retention.size prometheus option
fixes: https://tracker.ceph.com/issues/57422

Signed-off-by: Redouane Kachach <rkachach@redhat.com>

test/{librbd, rgw}: increase delay between and number of bind attempts

Commit aa7885f7cc41 ("test/{librbd, rgw}: retry when bind fail with
port 0") reduced the frequency of sporadic unit test failures caused
by EADDRINUSE a lot, but not entirely.

Currently, it yields a cumulative sleep of ~9 seconds.  Let's increase
that to 1 minute.

Fixes: https://tracker.ceph.com/issues/57116
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #47274 from faithuniterh/Golang-Notification-creation

examples/golang : Add put notification creation golang example code

reviewed-by: yuvalif

Merge pull request #46757 from nSedrickm/navigation-component-accessibility-update

mgr/dashboard: Improve accessibility for navigation components

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

install-deps.sh: fix install-deps script for focal and bionic

When run on focal and bionic, install-deps ends early
due to this extra debug message that was added to the
end of `ensure_decent_gcc_on_ubuntu`. The debug message
prints when the script is run in a jenkins environment.

When the script is not run in a jenkins environment, the
value returned there is "false" or "0", which acts as
an early return. This stops the script from completing.

We can remove this line, as `ensure_decent_gcc_on_ubuntu`
is only called for focal and bionic, and most of the jenkins
nodes are running jammy. Also, there is a debug message at the
beginning of the function that should suffice.

Fixes: https://tracker.ceph.com/issues/57466
Signed-off-by: Laura Flores <lflores@redhat.com>

doc/cephadm: correct version staggered upgrade got in pacific

This didn't end up making 16.2.10 so the first version
it will actually be available is 16.2.11

Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #47091 from faithuniterh/Golang-Topic-creation

Topic creation example and sorting Golang examples into different folders

reviewed-by: anthonyeleven, yuvalif

Merge pull request #47864 from yuvalif/wip-yuval-fix-53186

rgw/notifications: add region to bucket ARN

reviewed-by: mattbenjamin, nthonyeleven

Merge pull request #47924 from tchaikov/wip-57355

denc: support non-c++20 compliant C++ standard library

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47935 from ivancich/wip-simplify-bucket-listing-logging

rgw: simplify bucket listing logging

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #47741 from ljflores/wip-python3-fix

do_cmake.sh: build with python3.10 on ubuntu version >= 22.0

Merge pull request #47857 from nSedrickm/duplicate-id-accessibility-update

mgr/dashboard: remove duplicate ids from table column filters

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/dashboard: fix snapshot creation with duplicate name

Snapshot creation with same name on UI throwing 500 Internal Error, This PR intends to fix this issue.

Fixes: https://tracker.ceph.com/issues/57456
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #47981 from myoungwon/wip-rbm-cleanup2

crimson/os/seastore: RBM cleanups (mkfs, open, unittest)

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>