Merge pull request #53721 from rishabh-d-dave/mon-mdsmon

mon/MDSMonitor: clean up MDSMonitor

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #53405 from rishabh-d-dave/ceph-auth-caps-val-caps

mon/AuthMonitor: make "ceph auth caps" print error messsages

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge PR #53099 into main

* refs/pull/53099/head:
script: update ceph-debug-docker for centos 9.stream

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge PR #53206 into main

* refs/pull/53206/head:
mds: use LogSegment dump for debugging

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52254 from vedanshbhartia/coverity_uninit

rgw: Add coverity uninitialized variable and initialize RGWBucketEntryMetadataObject

reviewed-by: yuvalif

Merge pull request #52328 from vedanshbhartia/coverity_1512267

rgw: fix potential null dereference in rgw_iam_policy.c: ParseState::do_string

reviewed-by: yuvalif

Merge pull request #52472 from vedanshbhartia/coverity_1510724

rgw: Remove unnecessary null check from valid_s3_bucket_name

reviwed-by: soumyakoduri , yuvalif

Merge pull request #52734 from vedanshbhartia/coverity_ostream

rgw: Restore ostream format state after changing it

reviwed-by: yuvalif

Merge pull request #52326 from yuvalif/wip-yuval-lua-reload

rgw/lua: support reloading lua packages on all RGWs

reviwed-by: dang, cbodle, anthonyeleven

Merge pull request #53890 from zdover23/wip-doc-2023-10-09-troubleshooting-troubleshooting-mon-4-of-x

doc/rados: edit troubleshooting-mon.rst (4 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #53892 from rishabh-d-dave/fix-test_cephfs.py

src/test/pybind: don't use decorator "with_setup"

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc/rados: edit troubleshooting-mon.rst (4 of x)

Edit doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/53875

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #53884 from leonid-s-usov/rgw-posix-test

test/rgw: don't compile POSIX test unless enabled

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge PR #53873 into main

* refs/pull/53873/head:
qa: typo fix when checking for perf counter - s/md_thresh_evicted/mdthresh_evicted
qa: lower mds_session_metadata_threshold for tests

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>

src/test/pybind: don't use decorator "with_setup"

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: typo fix when checking for perf counter - s/md_thresh_evicted/mdthresh_evicted

Signed-off-by: Venky Shankar <vshankar@redhat.com>

qa: lower mds_session_metadata_threshold for tests

... and increase the number of files that are created so as to
hit the threshold with a high probability.

Fixes: http://tracker.ceph.com/issues/62873
Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #53817 from cloudbehl/active-alert-filter

mgr/dashboard: Filter active alerts

Reviewed-by: Nizamudeen A <nia@redhat.com>

test/rgw: don't compile POSIX test unless enabled

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>

Merge pull request #53874 from zdover23/wip-doc-2023-10-07-rados-troubleshooting-community

doc/rados: edit troubleshooting/community.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #53875 from zdover23/wip-doc-2023-10-07-troubleshooting-troubleshooting-mon-3-of-x

doc/rados: edit troubleshooting-mon.rst (3 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: edit troubleshooting/community.rst

Edit doc/rados/troubleshooting/community.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/rados: edit troubleshooting-mon.rst (3 of x)

Edit doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/52827

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #53876 from zdover23/wip-doc-2023-10-08-architecture-rbd-sentence-repair

doc/architecture: repair RBD sentence

doc/architecture: repair RBD sentence

Improve an ambiguous sentence in doc/architecture.rst.

The problem presented by the original sentence is that the phrasal verb
"to provide with" is implicated in one of its possible readings.
Interpreted in that way, the sentence seems to express the incorrect
idea that RBD furnishes block devices with snapshotting and cloning, as
though snapshotting and cloning are being delivered to the block
devices. In fact, snapshotting and cloning are just features of RBD, and
are features that are described on this page:
https://docs.ceph.com/en/quincy/rbd/rbd-snapshot/.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #53790 from zdover23/wip-doc-2023-10-03-architecture-17-of-x

doc/architecture: edit "Peering and Sets"

doc/architecture: edit "Peering and Sets"

Edit the English in the section "Peering and Sets" in the file
doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge PR #53855 into main

* refs/pull/53855/head:
script: add option for debug build

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

script: add option for debug build

See: https://github.com/ceph/ceph-build/pull/2167

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #53417 from jrchyang/fix_mclock_scheduling_slow_main

osd: fix: slow scheduling when item_cost is large

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #52430 from vedanshbhartia/coverity_datarace

rgw: Add coverity annotations for missing mutex locks

reviewed-by: yuvalif, mkogan1

rgw/lua/doc: support reloading lua packages on all RGWs

without requiring a restart of the RGWs
test instructions:
https://gist.github.com/yuvalif/95b8ed9ea73ab4591c59644a050e01e2
also use capitalized "Lua" in logs/doc

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

mon/AuthMonitor: check if entity is absent before creating it

Although this code path is not used for creating entities yet, it is
better to fix the bug sooner than later. Method
AuthMonitor::_update_or_create_entity() must exit (with appropriate
error code) when entity to be created on the Ceph cluster is already
present.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

mds/MDSAuthCaps: re-word an error message for better clarity

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53834 from dparmar18/remove-egg-fragment-from-doc

doc: remove egg fragment from dev/developer_guide/running-tests-locally

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52296 from asm0deuz/bz2153448_ipv6

mgr/cephadm: ceph orch add fails when ipv6 address is surrounded by square brackets.

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #53621 from phlogistonjohn/jjm-cephadm-dtypes-common

cephadm: introduce Daemon Forms

Reviewed-by: Adam King <adking@redhat.com>

Merge PR #53836 into main

* refs/pull/53836/head:
PendingReleaseNotes: add a note about disallowing delegating inodes
qa: start testing mds_client_delegate_inos_pct config
mds: disable delegating inode ranges to clients

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

doc: remove egg fragment from dev/developer_guide/running-tests-locally

DEPRECATION: git+https://github.com/ceph/teuthology#egg=teuthology
[test] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce
this behaviour change. A possible replacement is to use the req @ url syntax,
and remove the egg fragment. Discussion can be found at
https://github.com/pypa/pip/issues/11617

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

src/MDSMonitor: make use of imported namespace symbols

Symbols imported into current namespace should be used directly; there
is no need to mention their parent namespace while using them. IOW, to use
"std::string" after it has been imported, just write "string" instead of
"std::string".

Signed-off-by: Rishabh Dave <ridave@redhat.com>

mon/MDSMonitor: remove unused namespace imports

Remove symbols imported into the current namespace since they are not being
used.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

rgw/lua: install lua packages in temp directory

and switch to it only once installation is done.
this is needed for cases where installation can happen
while RGW is running

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

rgw/lua/doc: fix luajson example

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #53700 from cloudbehl/cephfs-snapshot

mgr/dashboard: Fixed cephfs snapshot listing in directories

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53782 from sachinpunadikar/vstart-fix

vstart: exclude default route during cluster setup

Reviewed-by: Adam King <adking@redhat.com>

mgr/dashboard: Filter active alerts

- List alerts which are not suppressed

fixes: https://tracker.ceph.com/issues/62990

Signed-off-by: cloudbehl <cloudbehl@gmail.com>

Merge pull request #53819 from zdover23/wip-doc-2023-10-04-rados-troubleshooting-index

doc/rados: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

PendingReleaseNotes: add a note about disallowing delegating inodes

Signed-off-by: Venky Shankar <vshankar@redhat.com>

qa: start testing mds_client_delegate_inos_pct config

Signed-off-by: Venky Shankar <vshankar@redhat.com>

mds: disable delegating inode ranges to clients

Fixes: http://tracker.ceph.com/issues/63103
Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52735 from vedanshbhartia/coverity_minlong

rgw: Add coverity annotation for warning about tautological comparison

reviewed-by: thotz, yuvalif

Merge pull request #53788 from AliMasarweh/wip-alimasa-topic-stats-help

RGW: add the missing help print for command 'topic stats'

Reviewed-by: thotz <thottanjiffin@gmail.com>, Daniel Gryniewicz <dang@redhat.com>

Merge pull request #53789 from rhcs-dashboard/rgw-overview-fix

mgr/dashboard: fix rgw inventory card and broken shadows

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

vstart: exclude default route during cluster setup

"ip route list" may list default route, and that needs to be excluded
while doing cluster setup
Typical o/p of ip route list:
$ ip route list
default via 10.8.159.254 dev eno1 proto dhcp src 10.8.152.13 metric 100
10.8.152.0/21 dev eno1 proto kernel scope link src 10.8.152.13 metric 100

Signed-off-by: Sachin Punadikar <sachin.punadikar@ibm.com>

Merge pull request #52313 from linuxbox2/wip-fix-disable-dashboard

dashboard: regression, make install fails w/dashboard disabled

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

cephadm: convert cephadm agent to a daemon form

The cephadm agent is a bit special in that it will not be converted
to a ContainerDaemonForm (it is not containerized) but we still want
to have it registered as a DeamonForm so that the deamon_type can be
passed to create and have it resolve correctly.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: convert SNMPGateway to a ContainerDaemonForm

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: convert CustomContainer to a ContainerDaemonForm

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: convert NFSGanesha to a ContainerDaemonForm

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: add func to deploy any generic ContainerDaemonForm

While there are no ContainerDaemonForms implemented yet, add a function
that uses the ContainerDaemonForm methods to construct a deployment
for the container based daemons.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: add ContainerDaemonForm

Add a supplemental DaemonForm subclass that helps deploy container
based daemons in a standard fashion. Most of these methods are
optional and should have sensible defaults.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: move DeploymentType to deploy.py

The DeploymentType is used by a number of other classes and functions
and has no dependencies beyond enum and is safe to move.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: move firewalld related items to firewalld.py

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: remove direct daemon-class deps from firewall

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: move sysctl specific functions to sysctl.py

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: remove direct daemon-type deps from sysctl

Using the appropriate daemon form we can break the direct dependency
that the sysctl setup function has on particular classes and use
a generic interface.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: add test_daemon_form.py

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: introduce daemon forms to cephadm.py

Introduce the DeamonForm base class to cephadm.py and make various
daemon-type classes into fully fleged deamon form classes.

Some classes already had a semi-standard `init` classmethod for
instantiation. In these cases the new `create` classmethod is a thin
wrapper over the existing method. In cases where the class was not
already being instantiated a minimal set of methods are added.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: add daemon_form.py: bases and funcs for daemon forms

Create daemon_form.py containing the DaemonForm class and a few
subclasses and utility functions for working with DaemonForms.
In a future commit, DaemonForm will become the base class for
the current assortment of classes named after the daemon or
family of daemon they help manage.

A daemon form, think "form" as in "template" or "mold", assists
in setting up, creating, and managing daemons controlled with
cephadm. Because cephadm supports a variety of services the
DaemonForm is an abstract base class and the module also supports
additional ABCs that may be used by DaemonForms to implement
optional features.

The daemon forms that are expected to be used directly must be
registered using the provided decorator. This is an explicit extra
step so that common bases that inherit from DaemonForm can be
implemented. Plus explicit is better than implicit. :-)
All DeamonForm subclasses are expected to provide a small set
of standard methods so that the types can be chosen, instantiated,
and used a common manner.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: add some unit test coverage for deploying nfs, snmp

Signed-off-by: John Mulligan <jmulligan@redhat.com>

doc/rados: edit troubleshooting.rst

Edit doc/rads/troubleshooting.rst to remove some language that sounds
quite close to marketing language.

Signed-off-by: Zac Dover <zac.dover@proton.me>

dashboard: regression, make install fails w/dashboard disabled

https://tracker.ceph.com/issues/63100

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #53436 from rhcs-dashboard/fix-bootstrap-script

mgr/dashboard: fix bootstrap script for cephadm installation

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #45069 from ifed01/wip-ifed-alloc-stuff

os/bluestore: fixing some non-critical bugs for HybridAllocator

Reviewed-by: Adam Kupczyk <akupzyk@ibm.com>

Merge pull request #53642 from rhcs-dashboard/fix-62964-main

mgr/dashboard: Fix user/bucket count in rgw overview dashboard

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #53792 from zdover23/wip-doc-2023-10-03-rados-operations-control-2-of-x

doc/rados: edit ops/control.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

RGW: add the missing help print for command 'topic stats'

Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>

mon/AuthMonitor: revise a comment

Signed-off-by: Rishabh Dave <ridave@redhat.com>

mgr/dashboard: fix bootstrap script for cephadm installation

Fixes: https://tracker.ceph.com/issues/62827
Signed-off-by: avanthakkar <avanjohn@gmail.com>

Merge branch 'main' into wip-doc-2023-10-03-rados-operations-control-2-of-x

Signed-off-by: zdover23 <zac.dover@proton.me>

mgr/dashboard: fix rgw inventory card and broken shadows

Mess up of the dashboard landing page layout fixes PR

Fixes: http://tracker.ceph.com/issues/62961
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53791 from zdover23/wip-doc-2023-10-03-rados-operations-control-1-of-x

doc/rados: edit ops/control.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: edit ops/control.rst (2 of x)

Edit doc/rados/operations/control.rst (2 of x).

Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/rados: edit ops/control.rst (1 of x)

Edit doc/rados/operations/control.rst (1 of x).

Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge PR #53494 into main

* refs/pull/53494/head:
mds: fix stray CInodes' use-after-free bug when submit ELid entry

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #53332 from zhsgao/mds_overload_epochs

mds: add option mds_bal_overload_epochs

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #53746 from rhcs-dashboard/enhance-usage-bar

mgr/dashboard: rbd image hide usage bar when disk usage is not provided

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53307 from rhcs-dashboard/fix-62721-main

mgr/dashboard: Rgw Multi-site naming improvements

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52938 from batrick/mon-loop-refactor

mon: refactor loop variable names

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #52733 from batrick/i59813

osd/OSDMonitor: check svc is writeable before changing pending

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #53449 from ljflores/wip-tracker-62833

osd: fix read balancer logic to avoid redundant primary assignment

mgr/dashboard: fixed cephfs snapshot & Quota list

fixes: https://tracker.ceph.com/issues/63007

Signed-off-by: cloudbehl <cloudbehl@gmail.com>

Merge pull request #53671 from rhcs-dashboard/fix-protect-if-layering-enabled

mgr/dashboard: enable protect option if layering enabled

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Sunil Angadi <Sunil.Angadi@ibm.com>

Merge pull request #53673 from rhcs-dashboard/fix-62984-main

mgr/dashboard: show a message to restart the rgw daemons after moving from single-site to multi-site

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53703 from adk3798/cephadm-refactor-ssh

cephadm: start ssh.py in cephadmlib

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #53600 into main

* refs/pull/53600/head:
mds/FSMap: allow upgrades if no up mds

Reviewed-by: Milind Changire <mchangir@redhat.com>

mgr/dashboard: enable protect option if layering enabled

Fixes: https://tracker.ceph.com/issues/63076
Signed-off-by: avanthakkar <avanjohn@gmail.com>

mgr/dashboard: show a message to restart the rgw daemons after moving from single-site to multi-site

Fixes: https://tracker.ceph.com/issues/62984
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #53522 from rhcs-dashboard/landing-page-layout-fix

mgr/dashboard: fix the landing page layout issues

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: rbd image hide usage bar when disk usage is not provided

Fixes: https://tracker.ceph.com/issues/63037
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>