cephadm: compile cephadm

use cephadm.py when mounting via the `--dev_share_folder` option

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_cephadm: add `--no-pager`

avoid user interaction with the systemd pager

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_repos: add zypper

cephadm adds partial enablement for zypper repos

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_repos: compile cephadm

build the compiled cephadm binary during the workunit test

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_adoption: compile cephadm

build the compiled cephadm binary during the workunit test

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_adoption: remove python kludge

we are no longer testing against py2 as of 225cbd8d790602b30050e0e32ecffd400ec17731

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_cephadm: compile cephadm

build the compiled cephadm binary during the workunit test

Signed-off-by: Michael Fritch <mfritch@suse.com>

cpatch: compile the cephadm binary

Signed-off-by: Michael Fritch <mfritch@suse.com>

cstart.sh: compile cephadm

update cstart/ckill scripts to use the compiled cephadm binary

Signed-off-by: Michael Fritch <mfritch@suse.com>

stop.sh: compile cephadm

use the path to the compiled cephadm binary file

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: allow execution from a relative path

let the build script run from a relative path (e.g. build or qa dir)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: zipapp module requires Python35 or later

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: locate path to the python3 binary

fixes incorrect shebang in the compiled zipfile (`#!python3`)

Signed-off-by: Michael Fritch <mfritch@suse.com>

rpm, debian: use cmake to install the cephadm binary

Signed-off-by: Michael Fritch <mfritch@suse.com>

cmake: add cephadm to vstart target

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: Only compress cephadm on Python >= 3.7

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

qa/tasks: Drop downloading cephadm from the git source tree

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: compile cephadm

Compiling cephadm into an single binary offers the ability to split the
code from cephadm up into several files/modules. A side effect of having
a `cephadm.py` file (which, by renaming to cephadm.py has already become
a module) is that it can properly be imported and static type checking
inside `src/cephadm/tests/test_cephadm.py` is not an issue anymore.

Next steps would to split up cephadm into several modules and adapting
the test_cephadm.py import mechanism, as well as fixing the typing
issues, which are revealed by using a proper important statement.

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #47521 from adk3798/iscsi-ip-list

mgr/cephadm: reconfig iscsi daemons if trusted_ip_list changes

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47640 from adk3798/no-cgroups

mgr/cephadm: make setting --cgroups=split configurable

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47936 from adk3798/tuned-profile-dashes

mgr/cephadm: fix tuned profiles getting removed if name has dashes

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47786 from rkachach/fix_issue_57192

mgr/cephadm: validating tuned profile specification

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #48008 from adk3798/docs-extra-container-mounts

doc/cephadm: document recommended syntax for mounting files with ECA

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

mgr/cephadm: validating tuned profile specification
fixes: https://tracker.ceph.com/issues/57192
fixes: https://tracker.ceph.com/issues/57191

Signed-off-by: Redouane Kachach <rkachach@redhat.com>

mgr/cephadm: reconfig iscsi daemons if trusted_ip_list changes

This is necessary as iscsi daemons expect the iscsi-gateway.cfg
which includes this list to match across the iscsi daemons

Fixes: https://tracker.ceph.com/issues/57078
Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #47898 from adk3798/no-sudo-true

mgr/cephadm: don't use "sudo" in commands if user is root

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

mgr/cephadm: fix tuned profiles getting removed if name has dashes

Previously, due to the way it was just splitting the file name on
dashes to try and get the profile name, any profile name with dashes
was not getting properly matched and would therefore get marked
stray and removed. This new strategy instead tries to match the actual
expected file name.

Fixes: https://tracker.ceph.com/issues/57405
Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #47469 from lxbsz/cleanup_C_MDC_EmptyImport

mds: remove the stale class C_MDC_EmptyImport code

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>

Merge pull request #48033 from joscollin/wip-kill-cephfs-top

stop.sh: kill cephfs-top

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48006 from rkachach/fix_issue_57422

mgr/cephadm: Adding --storage.tsdb.retention.size prometheus option

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47574 from rhcs-dashboard/osd-form-hostname-filter

mgr/dashboard: osd form preselect db/wal device filters

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

Merge pull request #48001 from aisakaki/wip-omap-inclusive-fix

crimson/seastore: fixs on omap list open/closed range

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #47968 from rhcs-dashboard/add-alert-silence-button

mgr/dashboard: Add a Silence button shortcut to alert notifications

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47305 from zdover23/wip-doc-2022-07-25-pr4600-cleanup

doc/monitoring: add min vers of apps in mon stack

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #47575 from zdover23/wip-doc-2022-08-13-rados-operations-pools-prompt-update

doc/rados: add prompts to pools.rst

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #48042 from SUSE/wip-fix-57497

ceph.spec.in: Use gcc11-c++ on openSUSE Leap 15.x

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47771 from linuxbox2/wip-librgw-frontends

librgw: add ability to conditionally export HTTP frontends/apis

rgw/main: if !nfs, register service map as "rgw"

This was accidentally changed to "nfs"--which breaks various
ceph orchestration assumptions.

Found by Ernesto Puerta <epuertat@redhat.com>

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #47854 from phlogistonjohn/jjm-fix-57268

qa/tasks/kubeadm: set up tigera resources via kubectl create

Reviewed-by: Adam King <adking@redhat.com>

doc/cephadm: document recommended syntax for mounting files with ECA

Mounting files with extra container args seems to not
work with some of the more intuitive syntaxes.
See https://tracker.ceph.com/issues/57338

Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #48007 from adk3798/fix-staggered-pacific-version

doc/cephadm: correct version staggered upgrade got in pacific

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #46344 from mjenglish/ceph-volume-multiple-actuator

ceph-volume/ceph_volume/util: ceph-volume multi-actuator drive attribute

Merge pull request #47970 from soumyakoduri/wip-skoduri-coverity

rgw/dbstore: Fix coverity issues #1510412, #1511420

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

mgr/dashboard: Add a Silence button shortcut to alert notifications

Fixes: https://tracker.ceph.com/issues/57457
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #47976 from rhcs-dashboard/fix-duplicate-snapshot

mgr/dashboard: fix snapshot creation with duplicate name

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

rgw/dbstore: Fix coverity issues #1510412, #1511420

Fixes: https://tracker.ceph.com/issues/57322
Fixes: https://tracker.ceph.com/issues/57325
Signed-off-by: Soumya Koduri <skoduri@redhat.com>

ceph.spec.in: Use gcc11-c++ on openSUSE Leap 15.x

Fixes: 80949babab4aac86647ce9973f6b44adb92fae95
Fixes: https://tracker.ceph.com/issues/57497
Signed-off-by: Tim Serong <tserong@suse.com>

Merge pull request #48032 from rhcs-dashboard/openapi-check-fix

mgr/dashboard: fix openapi-check

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: docs gen tags sort

When generating tags the order of endpoints wasn't taken into account.
Two endpoints with the same url prefix, for example `/api/cluster/` and
`/api/cluster/user`, have different docs and the tags is generated from
a doc of one of these two, and since the order of these endpoints might
vary it is imperative to sort them to have a deterministic output.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #47770 from rzarzynski/wip-doc-dups-trimming-relnotes

PendingReleaseNotes: document online and offline trimming of PG Log's…

Reviewed-By: Neha Ojha <nojha@redhat.com>

rgwlib: ensure AppMain is destructed before CephContext

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: cause ratelimiter to destruct in AppMain::shutdown()

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

librgw: fix mismatched register/unregister sigterm handler

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main:  fix init order of rgw_main's init_storage()

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: move service_map_meta into AppMain

The RGWRealmReloader holds a reference on it.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: move signal handler actions out of shutdown()

AppMain::shutdown() had borrowed the unregister_async_signal_handler()
text from librgw, but of course, that's wrong.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlib: robustify non-default initialization

Don't rely on the conf-default value of rgw_nfs_frontends to
provide "rgw-nfs," as that's not optional when running under
librgw.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_main: remove unused tracepoint decls

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlib: rework linkage

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_a: build reorganization

Reorganize some dependencies shared by the librgw.so and radosgw
main units.  The librgw.so build no longer builds a main(...)
unit.

Build time should be slightly reduced, as several shared dependencies
have been moved into rgw_a.

There are some other cleanups, e.g., rgw_signal.{h,cc}.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: remove friends, see_also

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: various doc and comment corrections by doc writers

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: use AppMain in c++ main function

Consolidate standalone and library initialization into one
unit.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw/main: AppMain

s/rgw::InitHelper/rgw::AppMain/;

Move program state from RGWLib into AppMain, preparing it to
manage program state for all RGW instances (e.g., main()).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

librgw: add ability to conditionally export HTTP frontends/apis

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #47700 from s0nea/wip-rgw-overview-labels

monitoring/ceph-mixin: add RGW host to label info

Reviewed-by: MrFreezeex <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

stop.sh: kill cephfs-top

Signed-off-by: Jos Collin <jcollin@redhat.com>

mgr/dashboard: osd form preselect db/wal device filters

If the hostname is selected for the primary devices, then we can
preselect the hostname filter for the db/wal devices because osds will
be deployed only on the hostname of the primary device. If preselected
it'll be clear that only this devices will be used to deploy.

Addition to this, usually ssd devices are used for db/wal devices. So I
am preselecting these too in the filters.

Fixes: https://tracker.ceph.com/issues/57118
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47784 from rhcs-dashboard/ingress-unmanaged

mgr/dashboard: fix unmanaged service creation

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #47959 from rhcs-dashboard/service-id-filter-undefined

mgr/dashboard: fix Cannot read properties of undefined (reading 'filter')

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #48014 from idryomov/wip-57116-cont

test/{librbd, rgw}: increase delay between and number of bind attempts

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #48003 from dang/wip-dang-zipper-admin-api

RGW - Zipper - Allow per-store admin APIs

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #48009 from ljflores/wip-install-deps-focal-bionic

install-deps.sh: fix install-deps script for focal and bionic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

RGW - Zipper - Allow per-store admin APIs

Some of the existing admin APIs are rados-specific, and other stores
will want to add specific APIs.  Allow this by having a function to add
store-specific APIs.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

PendingReleaseNotes: document online and offline trimming of PG Log's dups

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/cephadm: Adding --storage.tsdb.retention.size prometheus option
fixes: https://tracker.ceph.com/issues/57422

Signed-off-by: Redouane Kachach <rkachach@redhat.com>

test/{librbd, rgw}: increase delay between and number of bind attempts

Commit aa7885f7cc41 ("test/{librbd, rgw}: retry when bind fail with
port 0") reduced the frequency of sporadic unit test failures caused
by EADDRINUSE a lot, but not entirely.

Currently, it yields a cumulative sleep of ~9 seconds.  Let's increase
that to 1 minute.

Fixes: https://tracker.ceph.com/issues/57116
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #47274 from faithuniterh/Golang-Notification-creation

examples/golang : Add put notification creation golang example code

reviewed-by: yuvalif

Merge pull request #46757 from nSedrickm/navigation-component-accessibility-update

mgr/dashboard: Improve accessibility for navigation components

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

install-deps.sh: fix install-deps script for focal and bionic

When run on focal and bionic, install-deps ends early
due to this extra debug message that was added to the
end of `ensure_decent_gcc_on_ubuntu`. The debug message
prints when the script is run in a jenkins environment.

When the script is not run in a jenkins environment, the
value returned there is "false" or "0", which acts as
an early return. This stops the script from completing.

We can remove this line, as `ensure_decent_gcc_on_ubuntu`
is only called for focal and bionic, and most of the jenkins
nodes are running jammy. Also, there is a debug message at the
beginning of the function that should suffice.

Fixes: https://tracker.ceph.com/issues/57466
Signed-off-by: Laura Flores <lflores@redhat.com>

doc/cephadm: correct version staggered upgrade got in pacific

This didn't end up making 16.2.10 so the first version
it will actually be available is 16.2.11

Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #47091 from faithuniterh/Golang-Topic-creation

Topic creation example and sorting Golang examples into different folders

reviewed-by: anthonyeleven, yuvalif

Merge pull request #47864 from yuvalif/wip-yuval-fix-53186

rgw/notifications: add region to bucket ARN

reviewed-by: mattbenjamin, nthonyeleven

Merge pull request #47924 from tchaikov/wip-57355

denc: support non-c++20 compliant C++ standard library

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47935 from ivancich/wip-simplify-bucket-listing-logging

rgw: simplify bucket listing logging

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #47741 from ljflores/wip-python3-fix

do_cmake.sh: build with python3.10 on ubuntu version >= 22.0

Merge pull request #47857 from nSedrickm/duplicate-id-accessibility-update

mgr/dashboard: remove duplicate ids from table column filters

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/dashboard: fix snapshot creation with duplicate name

Snapshot creation with same name on UI throwing 500 Internal Error, This PR intends to fix this issue.

Fixes: https://tracker.ceph.com/issues/57456
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #47981 from myoungwon/wip-rbm-cleanup2

crimson/os/seastore: RBM cleanups (mkfs, open, unittest)

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/seastore: lack of config.inclusive==true situation in omap_manager list test

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>

Merge pull request #47054 from nSedrickm/grafana-component-accessibility-update

mgr/dashboard: Improve level A accessibility for grafana component

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

crimson/os/seastore/rbm: use RBMDevice::size instead of TestMemory::size in TestMemory

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

crimson/os/seastore/rbm: force to make rbm superblock when mkfs is called

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

crimson/os/seastore/rbm: call device.open() before rbm.open() or rbm.mkfs()

This commit also deletes open_device() in rbn.open()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

test/crimson/seastore: fix randomblockmanager test

1. remove transaction manager initialization in rbm test
2. do not call check_bitmap_blocks directly and use close() and open()
because open() invokes it internally

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

crimson/os/seastore/omap: fix assert bug

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>

Merge pull request #47987 from idryomov/wip-52810

librbd: retry ENOENT in V2_REFRESH_PARENT as well

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #47783 from soumyakoduri/wip-skoduri-sync-flow

radosgw-admin: fix crash with 'sync flow' cmd without 'flow-type'

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #45883 from ifed01/wip-ifed-fix-bluefs-truncate

os/bluestore: fix no metadata update on truncate+fsync

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #47630 from sseshasa/wip-add-mclk-cpu-util-rel-note

PendingReleaseNotes: Note the fix for high CPU utilization during recovery

Reviewed-by: Neha Ojha <nojha@redhat.com>