doc: mgr/dashboard: feature-toggles: Fix config

Add '*.inc.rst' to the list of excluded patterns from sphinx-build. This
allows for using '*.inc.rst' as includes, and avoids duplicates. The
benefit of keeping the trailing '.rst' extension is that most IDEs use
that to render reStructured Text files.

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: minor fixes

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: Update Hacking

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: Fix formatting

Fix front-end formatting as suggested by tspmelo

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

doc: mgr/dashboard: feature-toggles: Add doc

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: add py unit tests

Add python unit test for feature toggles plugin, and refactor other
files, and add a new type of function cache decorator based on LRU but
with a TTL.

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: add front-end

Add front-end behaviours to feature toggles:
- In navigation pane, drop-down menu items are displayed/hidden accordingly.
- In main dashboard page, info cards are displayed/hidded.
- Routes are also enabled/disabled. When disabled, they redirect to 404.

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: Add REST hook

Provide a hook for creating REST controllers and endpoints. Provide also
a REST controller for Feature Toggles plugin.

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: Add plugin

Add feature-toggles plugin. It allows to enable, disable and check
status of a feature. Features are disabled by making their
corresponding enpoints return HTTP 501 error (Not Implemented).

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: add plugin support

Provide plugin infrastructure and a minimal set of hooks. As
python-pluggy library is not yet available for all the distros that Ceph
is targeted at, a minimal implementation has been provided.

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: feature-toggles: add Option helper

A new helper class `Options` has been added to ease adding new
MODULE_OPTIONS.

Fixes: http://tracker.ceph.com/issues/37530
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #26290 from dzafman/wip-38027-38195

Wip 38027 38195: osd/osd-backfill-space.sh fails

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25918 from rjfd/wip-dashboard-nfs-backend

mgr/dashboard: NFS Ganesha management REST API

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26177 from liewegas/wip-stricter-osd-gate

osd: refuse to start if we're > N+2 from recorded require_osd_release

Reviewed-By: Neha Ojha <nojha@redhat.com>

Merge pull request #26181 from iliul/fix-pybind-mgr-pep8

pybind/mgr: PEP 8 code clean and fix typo

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26258 from liewegas/wip-disable-krb

build: disable kerberos for nautilus

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26259 from rzarzynski/wip-bl-freeze-alloc-unit

common: make CEPH_BUFFER_ALLOC_UNIT known at compile-time.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26094 from sebastian-philipp/orchestrator-cli-command

mgr/orchestrator: make use of @CLICommand

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #26178 from dzafman/wip-38070

mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_IN…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #26273 from yehudasa/wip-38174

vstart.sh: fix params generation for monmaptool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test: Ignore kill_daemons() error

Workaround for: http://tracker.ceph.com/issues/38195

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Fix kill_daemon() to check after last large sleep

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Increase timeouts in osd-backfill-space.sh because of failure seen

Fixes: http://tracker.ceph.com/issues/38027
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #26279 from idryomov/wip-krbd-bluestore

qa: objectstore snippets for krbd

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: ganesha: support no namespace

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #25995 from tspmelo/wip-iscsi-ui

mgr/dashboard: iSCSI management UI

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/orchestrator: fix broken `device ls`

* Added `InventoryDevice.from_ceph_volume_inventory`
* Improved documentation

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Add iSCSI Form UI

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add read-only UI for iSCSI

Signed-off-by: Ricardo Marques <rimarques@suse.com>
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: ganesha: notify ganesha daemons to reload configuration

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: ganesha: add LookupError when checking RGW connectivity

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Fix TableComponent when changing columns afterViewInit

Angular was throwing an ExpressionChangedAfterItHasBeenCheckedError if you
changed columns information right after the table was initiated.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Set CdModal max-height

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enhance CdValidators.validateIf method

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add generic Select component

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26202 from smithfarm/wip-fix-s390x-rpm-build

rpm: package crypto libraries for all archs

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/pybind/rados: add binding for rados_notify2

Add a new Ioctx.notify operation that we can call to send a rados
notification over an object.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mgr/orchestrator: make use of @CLICommand

Also Modified some command to allow multiple hosts
Also: Added more tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

qa: objectstore snippets for krbd

krbd was being tested with filestore, up until recently when the
default for osd_objectstore was changed to bluestore.  This broke
rbd_simple_big.yaml because bluestore_block_size defaults to 10G.
Pick up the sepia setting of 90G from bluestore-bitmap.yaml.

Run fsx subsuite with both filestore and bluestore.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #26106 from yehudasa/wip-rgw-es-fixes

rgw: multiple es related fixes and improvements

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

vstart.sh: fix params generation for monmaptool

Fixes: http://tracker.ceph.com/issues/38174
This broke in the msgr21 case, due to bash globbing in cases where
the generated string somehow matched names of certain files in local
directory. Also need to pass params to the prun function separately,
otherwise everything is quoted together.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #26111 into master

* refs/pull/26111/head:
mds: broadcast quota message to client when disable quota

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #26123 into master

* refs/pull/26123/head:
mds: refactor MDSContext inheritance

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #26023 from jmolmo/inventory_pb_updated

mgr/ansible: Replace Ansible playbook used to retrieve storage devices data

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

Merge pull request #26170 from ifed01/wip-ifed-fix-fsck-read

os/bluestore: deep fsck fails on inspecting very large onodes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #26223 from majianpeng/remove-unused-function

osd: remove unused function.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25278 from ZVampirEM77/wip-deleteobject-policy

rgw: fix obj can still be deleted even if deleteobject policy is set
rgw: cleanup for RGWDeleteObj::verify_permission(

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #26244 from idryomov/wip-krbd-alloc-size

krbd: alloc_size map option and tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rpm: package crypto libraries for all archs

Since 318a8e3c079c937d4e006a9eb4f47c5349648360 we are now building at least the
openssl crypto plugin on all architectures, not just x86_64.

This fixes the non-x86_64 builds for master.

Fixes: 318a8e3c079c937d4e006a9eb4f47c5349648360
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25309 from Devp00l/wip-prometheus-alerting

mgr/dashboard: Prometheus integration

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #26256 into master

* refs/pull/26256/head:
msg/async/ProtocolV2: respect policy.resetcheck on cookie==0

Reviewed-by: Ricardo Dias <rdias@suse.com>

common: make CEPH_BUFFER_ALLOC_UNIT known at compile-time.

It sounds extremely unlikely that any platform Ceph targets can have
PAGE_SIZE lower than 4k. However, to handle these corner cases,
`bufferlist` was rounding to `CEPH_PAGE_SIZE` known at run-time only.
As a result, compilers were enforced to generate pretty inefficient
code like in `ceph::buffer::list::refill_append_space` (self is 1,10%):

```
       │         min(const _Tp& __a, const _Tp& __b)
       │         {
       │           // concept requirements
       │           __glibcxx_function_requires(_LessThanComparableConcept<_Tp>)
       │           //return __b < __a ? __b : __a;
       │           if (__b < __a)
  5,44 │       mov    _DYNAMIC+0x14a0,%rax
       │     _Z11round_up_toImjENSt13make_unsignedINSt11common_typeIJT_T0_EE4typeEE4typeES2_S3_():
       │       mov    $0x1000,%ecx
       │     _ZN4ceph6buffer4list19refill_append_spaceEj():
       │         size_t alen = round_up_to(need, CEPH_BUFFER_ALLOC_UNIT) -
       │           sizeof(raw_combined);
       │         buffer::hangable_ptr& new_back = \
       │           hangable_ptr::create(raw_combined::create(alen, 0, get_mempool()));
       │       mov    %rbp,%rdi
       │     _Z11round_up_toImjENSt13make_unsignedINSt11common_typeIJT_T0_EE4typeEE4typeES2_S3_():
  6,21 │       cmpl   $0x1000,(%rax)
  0,78 │       cmovbe (%rax),%ecx
       │       mov    %r12,%rax
       │       xor    %edx,%edx
  0,78 │       mov    %ecx,%ecx
 45,59 │       div    %rcx
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

build: disable kerberos for nautilus

We can re-enable after we branch.  For N, though, there is no point in dragging
in the dependencies and implying there is something that works.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26240 from votdev/improve_get_module_option

mgr: Use Py_BuildValue to create the argument tuple

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26233 from liewegas/wip-bluestore-disable-csum

os/bluestore: add bluestore_ignore_data_csum option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #25077 from rzarzynski/wip-bl-kill_append_buffer

common: drop append_buffer from bufferlist. Use simple carriage instead

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26245 from liewegas/wip-38150

os/bluestore/KernelDevice: use flock(2) for block device lock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

msg/async/ProtocolV2: respect policy.resetcheck on cookie==0

It's possible the accepting end knows the cookie but the connecting end
does not if the fault happens before it receives server_ident.  If the
next connection attempt is in the other direction, we need to avoid a
reset or the queued messages at the first site will get lost.

Fixes: http://tracker.ceph.com/issues/38118
Signed-off-by: Sage Weil <sage@redhat.com>

qa: update krbd tests for zeroout

Discard no longer guarantees zeroing, use BLKZEROOUT and "fallocate -z"
instead (blkdiscard(8) in xenial doesn't support -z).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa: krbd discard with alloc_size vs zeroout tests

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #26179 from xiexingguo/wip-37968

crush: fix upmap overkill

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26222 from tchaikov/wip-crimson-heartbeat

crimson/osd: add heartbeat support

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: disable GCC-9 warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add heartbeat support

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: move "_preboot" log to where it belongs to

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: bind and listen

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: add Messenger::try_bind()

in case the server is just fine with any unused local port number, and it
will report it to monitor.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: s/client_msgr/public_msgr/

because this messenger also serves requests from librados clients.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: add pick_address.cc to crimson-common

crimson-osd needs pick_address() for initializing the listening
addresses.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/pick_address: add necessary includes

* add needed forward declaration
* add necessary #includes
* move `#include "common/config.h"` to .cc, as it's not part of
interface.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/pick_address: conditional compile for crimson

crimson can only update configuration async, so we cannot let
pick_addreses() call set_val_or_die() without chaning its function
signature.

for implementing crimson-osd, what we need is only a single
pick_addresses() variant, which returns entity_addrvec_t. so we just
enable it if WITH_SEASTAR is defined instead of adding a wrapper or
reimplement all pick_addresses() variants for crimson-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: add get_fsid() to mon::Client

we need it for creating some messages. `MOSDPing`, for instance.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: define OSDMapService interface

* define OSDMapService interface to decouple the consumers of
  this interface from its implementation -- the `OSD` class
* implement its methods in `OSD`

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add editor variable settings

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: remove unused method

Signed-off-by: Kefu Chai <kchai@redhat.com>

mds: refactor MDSContext inheritance

This commit greatly simplifies things.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26250 from falcon78921/wip-doc-rados-typo

doc: fixed typo in add-or-rm-mons.rst

Reviewed-by: Neha Ojha <nojha@redhat.com>

doc: fixed typo in add-or-rm-mons.rst

Redundant use of the word "monitor"

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

common: drop decltype(_buffers) from include/buffers.h.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

tests: introduce BufferList.ContiguousAppender.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: optimize bufferlist::contiguous_appender.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: introduce obtain_contiguous_space() to bufferlist.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

tests: introduce BufferList.InternalCarriage.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: introduce bl::_carriage to track writeable area.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: kill remainings of bufferlist's append_buffer.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: simplify the C-string-taking variant of bl::append().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

tests: introduce BufferListIterator.iterate_with_empties.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: bufferlist can contain 0-length bptrs.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: contiguous_appender doesn't use append_buffer anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: bl::appends don't use the append_buffer anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #25703 into master

* refs/pull/25703/head:
os/bluestore: option to disable bluestore warns on spillover.
mon: expose object store health warnings
osd: expose object store alerts
os/bluestore: indicate BlueFS spillover and lacking compressor alerts
os:enable ObjectStore::statfs() api to return OS alert list.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #25773 into master

* refs/pull/25773/head:
common: Use OpenSSL for cryptographic hashing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #26067 into master

* refs/pull/26067/head:
pybind/rados: fix application metadata list

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #26112 into master

* refs/pull/26112/head:
mgr: load modules in finisher to avoid potential lock cycles

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge PR #26164 into master

* refs/pull/26164/head:
mon/LogMonitor: lower logging level for check_sub

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore/KernelDevice: use flock(2) for block device lock

The fcntl locks fail due to the classic posix lock gotcha: if you close
*any* fd to the same inode from the process, the lock(s) go away.

Use flock(2) instead.  We have to be careful because we open the main
bluestore device via two KernelDevice instances: one for bluestore and
one for bluefs.  Add a no-lock flag so that the bluefs instance does not
try to lock and does not conflict with bluestore's.

Fixes: http://tracker.ceph.com/issues/38150
Signed-off-by: Sage Weil <sage@redhat.com>

rbd: recognize alloc_size map option

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #26224 from tchaikov/radek-lru-cleanup

common: drop the unused methods from SharedLRU.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26171 from sebastian-philipp/orchestrator-merge-create-add

mgr/orchestrator: Unify `osd create` and `osd add`

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #26151 from nathan-weinberg/cluster-tests

mgr/dashboard: Added additional breadcrumb and tab tests to Cluster menu

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>