doc: fix titles

Merge pull request #44955 from aclamk/wip-vselector-check-fix

os/bluestore/bluefs: Fix improper vselector tracking in _flush_special()

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #45057 from zdover23/wip-doc-2022-02-17-os-recs-nautilus-and-luminous-table-removal

doc/start: remove L and N release-related tables

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #45049 from liu-chunmei/crimson-clang-warning-cleanup

crimson: eliminate clang build warning in crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

crimson: eliminate clang build warning in crimson

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

doc/start: remove L and N release-related tables

This PR removes the tables for Luminous and Nautilus,
in order to keep the documentation as current as possible.
This is one a series of PRs intended to update the OS
Recommendations page. The scope of this PR is confined to
the removal of these two tables. Other PRs will update the
information elsewhere on the OS Recommendations page.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #45045 from idryomov/wip-qemu-task-rbd-package

qa/tasks/qemu: make sure block-rbd.so is installed

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

qa/suites/rbd: make sure block-rbd.so is installed

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45021 from idryomov/wip-groupsnapns-operator-less

cls/rbd: GroupSnapshotNamespace comparator violates ordering rules

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #45020 from rhcs-dashboard/snmp-priv-protocol-fix

mgr/dashboard: change privacy protocol field from required to optional

Merge pull request #44659 from ifed01/wip-ifed-alloc-replay-boind-size

test/alloc_replay_test: be able to cap allocator space with capacity …

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

os/bluestore/bluefs: Fix vselector

Fix bluefs volume selector in device_migrate_to_existing.
Fix bluefs volume selector in _rewrite_log_and_layout_sync_LNF_LD.

Fixes: https://tracker.ceph.com/issues/54248
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #44997 from pcfe/pcfe-add-cos9-instructions

doc/cephadm: Add CentOS Stream install instructions

Reviewed-by: Adam King <adking@redhat.com>

doc/cephadm: Add CentOS Stream install instructions

Signed-off-by: Patrick C. F. Ernzer <pcfe@pcfe.net>.

Merge pull request #44988 from caisan/FixRGWLogStringError

Fix rgw data sync log info error

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>

mgr/dashboard: add validation for snmp v3 engine id

Fixes: https://tracker.ceph.com/issues/54270
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

qa/tasks/qemu: make sure block-rbd.so is installed

Fixes: https://tracker.ceph.com/issues/54286
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45032 from ceph/revert-45018

Revert "admin/doc-requirements: pin breathe to 4.32.0"

Merge pull request #44951 from votdev/issue_54215_expand_cluster

mgr/dashboard: "Please expand your cluster first" shouldn't be shown if cluster is already meaningfully running

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #43588 from lxbsz/fsize_support

mds: add truncate size handling support for fscrypt

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45011 from tchaikov/librados/cleanup

include/rados/librados.h: drop duplicated function declaration

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

admin/doc-requirements.txt: Not breathe=4.33

See https://github.com/michaeljones/breathe/issues/803

Signed-off-by: David Galloway <dgallowa@redhat.com>

Revert "admin/doc-requirements: pin breathe to 4.32.0"

This reverts commit d020d07be97c56db88fadd9921e7b3eff11c0802.

Fixed upstream https://github.com/michaeljones/breathe/issues/803#issuecomment-1039341857

Signed-off-by: David Galloway <dgalloway@redhat.com>

Merge pull request #44779 from rkachach/fix_issue_53010

mgr/cephadm: Adding logic to cleanup several ceph dirs (rm-cluster)

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #45014 from guits/fix-cephadm-ingress

cephadm/ingress: make frontend stats bind on localhost

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44794 from rzarzynski/wip-common-fix-numa-warning

common: fix compilation warnings in numa.cc

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #44781 from ljflores/wip-basic-channel-additions

mgr/telemetry: add `basic_pool_usage` and `basic_usage_by_class` collections to the telemetry module

Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>

Merge pull request #44746 from ljflores/wip-telemetry-down-osd-fix

mgr/telemetry: collect what we can from histograms, mempools, and heap stats

Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>

Merge pull request #44670 from rzarzynski/wip-common-dout-gcc12-ftbfs

common: fix FTBFS due to dout & need_dynamic on GCC-12

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #44983 from cbodley/wip-52673

rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation

Reviewed-by: Neha Ojha <nojha@redhat.com>

os/bluestore/bluefs: Fix improper vselector tracking in _flush_special()

Moves vselector size tracking outside _flush_special().
Function _compact_log_async...() updated sizes twice.
Problem could not be solved by making second modification of size just update,
as it will possibly disrupt vselector consistency check (_vselector_check()).
Feature to track vselector consistency relies on the fact that either log.lock or nodes.lock
are taken when the check is performed. Which is not true for _compact_log_async...().

Now _flush_special does not update vselector sizes by itself but leaves the update to
the caller.

Fixes: https://tracker.ceph.com/issues/54248
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #44917 from guits/fix_rook_smoke_nvme_loop

qa/orch: use hwe kernel in rook/smoke tests (ubuntu 20.04)

Reviewed-by: Neha Ojha nojha@redhat.com
Reviewed-by: Yuri Weinstein yweinste@redhat.com

Merge pull request #45018 from tchaikov/wip-doc-breathe

admin/doc-requirements: pin breathe to 4.32.0

mgr/dashboard: change privacy protocol field from required to optional

Fixes: https://tracker.ceph.com/issues/54270
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Privacy protocol field shouldn't be a required field.

cephadm/ingress: make frontend stat bind on localhost

The current configuration of keepalived makes it do
a curl on localhost:9999 in order to check the endpoint is alive.
Given the endpoint only binds on the vip addr, that doesn't work.

Fixes: https://tracker.ceph.com/issues/53807
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #44965 from adk3798/test_cli_timeout

qa/tasks/cephadm_cases: increase timeouts in test_cli.py

Reviewed-by: Michael Fritch mfritch@suse.com

cls/rbd: GroupSnapshotNamespace comparator violates ordering rules

For

  GroupSnapshotNamespace a(1, "group-1", "snap-2");
  GroupSnapshotNamespace b(1, "group-2", "snap-1");

both a < b and b < a evaluate to true.  This violates STL strict weak
ordering requirements which is a problem because GroupSnapshotNamespace
is used as a key in std::map (ictx->snap_ids at least), etc.

Fixes: https://tracker.ceph.com/issues/49792
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

admin/doc-requirements: pin breathe to 4.32.0

in breathe v4.33, it includes following commit

https://github.com/michaeljones/breathe/commit/2498a437234343503e0087cad793c6f333f1f781

which specfies the app config value of "graphviz_dot". this annoys
sphinx:

WARNING: while setting up extension breathe: node class 'graphviz' is already registered, its visitors will be overridden

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/application.py", line 245, in __init__
    self.setup_extension(extension)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/application.py", line 402, in setup_extension
    self.registry.load_extension(self, extname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/registry.py", line 430, in load_extension
    metadata = setup(app)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/breathe/__init__.py", line 14, in setup
    renderer_setup(app)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2613, in setup
    app.add_config_value("graphviz_dot", "dot", "html")
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/application.py", line 535, in add_config_value
    self.config.add(name, default, rebuild, types)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/config.py", line 282, in add
    raise ExtensionError(__('Config value %r already present') % name)
sphinx.errors.ExtensionError: Config value 'graphviz_dot' already present

Extension error:
Config value 'graphviz_dot' already present

this issue has been reported to upstream, see
https://github.com/michaeljones/breathe/issues/803

before it is fixed upstream, let's stick with 4.32.0
which is known to work.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

mgr/dashboard: "Please expand your cluster first" shouldn't be shown if cluster is already meaningfully running

This PR will assume that a cluster is already up and fully running. If this should not be the expected behaviour, deployment tools have to set 'INSTALLED' explicitly. Without this assumption it might happen that upgraded and fully running clusters, e.g. Octopus -> Pacific, will show the 'Expand Cluster' on first log in.

cephadm will take care that the bootstrap phase will write the necessary key to show the 'Expand cluster' page.

Fixes: https://tracker.ceph.com/issues/54215
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #44971 from joscollin/wip-doc-fix-name-id

doc: mention MDS name and ID format in ceph-mds.rst

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #39315 from shenhang/fetchkey

mds: fetch single dentry instead of complete dirfrag for lookup

Reviewed-by: Venky Shankar <vshankar@redhat.com>

include/rados/librados.h: drop duplicated function declaration

rados_mgr_command() has been declared in the very same header file
with exactly the same function signature, so there is no need to
repeat it.

in this change, the duplicated declaration of rados_mgr_command()
is dropped. the one with doxygen document is reserved.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44308 from vshankar/wip-document-mds-locker

doc: document mds locking operation

Merge pull request #44898 from taggelos/patch-1

doc/man/8: Fix spelling in user rename example

Merge pull request #44780 from sosodev/dump-blocked-ops-count

mds, osd: add dump_blocked_ops_count command

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44752 from NitzanMordhai/wip-nitzan-osd-status

pybind/mgr: ceph osd status crash with ZeroDivisionError

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #44588 from kamoltat/wip-ksirivad-disable-progress-by-default

pybind/mgr/progress: disable pg recovery event by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #44289 from Vicente-Cheng/fix-outgoing-bl-overflow

msg/async: fix outgoing_bl overflow and reset state_offset

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #44809 from cyx1231st/wip-crimson-cleanup-epm

crimson/os/seastore: allow EPM to make decisions on the general allocation path

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

doc: document mds locking operation

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44849 from rhcs-dashboard/bugfix-cephfs-invalid-dir-permission

mgr/dashboard: Directories Menu Can't Use on Ceph File System Dashboard

Fix rgw data sync log info error

Signed-off-by: caishan <caishan1993@foxmail.com>

Merge pull request #44970 from kotreshhr/fix-clone-uid-gid

mgr/volumes: Fix clone uid/gid mismatch

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc: mention MDS name and ID format in ceph-mds.rst

Signed-off-by: Jos Collin <jcollin@redhat.com>

crimson/os/seastore: misc log improvements

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: initialize segment_info_t correctly when open/empty

Previously, segment_info_t was not initialized for out-of-line segments
when open, including journal_segment_seq and the legacy out_of_line.

And the journal_segment_seq was not reset when empty.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: replace out_of_line by segment_type_t

Leverage segment_seq_t to identify the type of the segment.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: cleanup dependency between epm and journal

OOL segments won't contain deltas and don't need to replay. There is no
need to assign segment sequence from journal.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: cleanup dependency between epm and lba_manager

Move lba_manamger->update_mapping() from epm to transaction manager.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: update_mapping related cleanups

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: allow EPM to make decisions on the general extent allocation path

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: drop the unnecessary backend_type in every cached extent

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/epm: cleanup unnecessary cache dependency when mark extent ool

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: cleanup delayed paddr

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #44968 from cyx1231st/wip-crimson-cleanup-cleaner-1

crimson/os/seastore/segment_cleaner: minor cleanups

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44964 from yaarith/pending-release-notes-perf

PendingReleaseNotes: add a note about perf channel

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44963 from ljflores/wip-peering-state-docs

doc/dev: add section on using the gen_state_diagram.py script

Reviewed-by: Neha Ojha <nojha@redhat.com>

rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation

use the cluster defaults for pg_num_min

Fixes: https://tracker.ceph.com/issues/52673
Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/dev: add section on using the gen_state_diagram.py script

Commands sourced from Jianshen Liu's blog post on the Ceph peering state machine: https://jianshenliu.com/blog/development/ceph-peering-state-machine/

Also fixed the formatting of "PG info".

Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #44765 from guits/guits-fix-cve-cv

ceph-volume: honour osd_dmcrypt_key_size option

Merge pull request #44808 from rhcs-dashboard/support-snmp-service-creation

mgr/dashboard: support snmp-gateway service creation from UI

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

mgr/volumes: Fix clone uid/gid mismatch

This is the regression caused by commit 18b85c53a.
The 'set_attrs' function sets the uid/gid of the
group to the subvolume if uid/gid is not passed.
The attrs of the clone should match the source
snapshot. Hence, don't use the 'set_attrs'
function to set only the quota attrs for the
clone.

Fixes: https://tracker.ceph.com/issues/54066
Signed-off-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #44953 from aejv/spellingerror

purge_queue.py: fixed typo

Reviewed-by: Venky Shankar <vshankar@redhat.com>

crimson/os/seastore/segment_cleaner: mark get_next_gc_target() private

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/segment_cleaner: make init_mark_segment_closed() private

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/segment_cleaner: drop unused code

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net/socket: add a missing header

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

ceph-volume/activate: load the config from lv tag

When `ceph-volume lvm trigger` is called with an OSD where the tag
`ceph.cluster_name` is not 'ceph', it fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

qa/tasks/cephadm_cases: increase timeouts in test_cli.py

These seem to be failing sometimes but in my testing
sometimes these events are happening a few seconds after
we hit the timeout. Trying to see if this makes the tests
more consistent. No need to mark the test as failed
if we report something up in 34 seconds vs 25 especially
when cephadm works on a cyclic daemon refresh.

Signed-off-by: Adam King <adking@redhat.com>

ceph-volume/tests: use centos/stream8 images

Since recent move from CentOS 8 to CentOS Stream 8, let's do the same here.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-volume/tests: add tests in util/encryption.py

this adds some unit tests in order to cover `luks_format()` and `luks_open()`
in `util/encryption.py`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-volume: honour osd_dmcrypt_key_size option

ceph-volume doesn't honour osd_dmcrypt_key_size.
It means the default size is always applied.

It also changes the default value in `get_key_size_from_conf()`

From cryptsetup manpage:

> For XTS mode you can optionally set a key size of 512 bits with the -s option.

Using more than 512bits will end up with the following error message:

```
Key size in XTS mode must be 256 or 512 bits.
```

Fixes: https://tracker.ceph.com/issues/54006
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

PendingReleaseNotes: add a note about perf channel

Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>

Merge pull request #44956 from rhcs-dashboard/appli-fix

mgr/dashboard: change the readFile to readFileSync

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44939 from soumyakoduri/wip-skoduri-dbstore-tests

rgw/dbstore: Add dbstore-tests to `make check`

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: change the readFile to readFileSync

Apparently the readFile i added in #44934 is async and that's not what
we want. so changing it to the synchronous call that is readFileSync

Fixes: https://tracker.ceph.com/issues/54190
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge PR #42000 into master

* refs/pull/42000/head:
qa: update rhel kclient to setup container tools
qa: stop overriding distro for k-testing
qa: only use RHEL for workload testing
qa: convert fs:workload to use cephadm
qa: split fs begin task
qa/tasks/cephadm: setup CephManager when OSDs are provisioned
qa/tasks/cephadm: setup file system if MDS are provisioned

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

purge_queue.py: fixed typo

Signed-off-by: Judah van der Ster <jevanderster@gmail.com>

Merge pull request #44937 from idryomov/wip-rbd-help-positional-optional

rbd: mark optional positional arguments as such in help output

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #44492 from xijiacun/master

rgw: Erase old storage class attr when the object is rewrited using r…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard:Directories Menu Can't Use on Ceph File System Dashboard

Added exception handling to opendir() in cephfs.py for directories with no execute permission.

Fixes: https://tracker.ceph.com/issues/51611
Signed-off-by: Sarthak0702 <sarthak.0702@gmail.com>

mgr/dashboard: add snmp-gateway service e2e tests

Fixes: https://tracker.ceph.com/issues/54034
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #44934 from rhcs-dashboard/applitools-baseline-branch

mgr/dashboard: set appropriate baseline branch for applitools

Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

rgw/dbstore: Add dbstore-tests to `make check`

Include and run dbstore-tests as part of `make check` target

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

mds: add an option to decide whether prefetching entire dir or not.
Accessing one single dentry could be fastened by set this option to
false, when dir is not in the memory.
Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>

mds: avoid adding null dentry to bloom filter

Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>

mds: make open_ino(...) fetch dentries in batch

Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>

mds: don't add null auth dentry to bottom lru

null dentries are the key of not fetching complete dirfrag. don't trim
them too fast

Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>

mds: fetch single dentry instead of complete dirfrag for lookup

Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>

mds: remove CDir::take_sub_waiting()

migrator already does the job when importing dirfrag.

Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>