mgr/dashboard: Explicitly type page variables

Explicitly gives a type to page variables.
Helps make better use of Typescript's static type checking.
Undoes mistake from #29070.

Fixes: https://tracker.ceph.com/issues/40961
Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Rafael Quintero <rquinter@redhat.com>

Merge pull request #29237 from adk3798/usertest

mgr/dashboard: RGW user E2E Tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #29192 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/lodash-4.17.13

build(deps): bump lodash from 4.17.11 to 4.17.13 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29198 from rhcs-dashboard/e2e-ci

mgr/dashboard: run e2e tests against prod build (jenkins job)

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29280 from rhcs-dashboard/bucket-tests

mgr/dashboard: RGW bucket creation when no placement target received

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #29264 from rjfd/wip-dashboard-fix-qa-rbd

mgr/dashboard: qa: fix RBD test when matching error strings

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28597 from markhpc/wip-bs-ind-cache

os/bluestore: Refactor Bluestore Caches

Merge PR #28847 into master

* refs/pull/28847/head:
doc/mgr/telemetry: update
mgr/telemetry: move contact info to an 'ident' channel
mgr/telemetry: accept channel list to 'telemetry show'
mgr/telemetry: always generate new report for 'telemetry show'
mgr/telemetry: add 'device' channel and call out to devicehealth module
mgr/telemetry: add telemetry channel 'device'
mgr/telemetry: add separate channels

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge PR #29130 into master

* refs/pull/29130/head:
os/bluestore: Reduce unnecessary wakeups.
os/bluestore: Reduce unnecessary wakeups.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #29221 into master

* refs/pull/29221/head:
mon/OSDMonitor: do not clobber pending xinfo during boot
osd/OSDMap: add is_dead() helper
mon/OSDMonitor: 'osd down ... --definitely-dead'
osd: send MOSDMarkMeDead when we are marked down
mon/OSDMonitor: implement MOSDMarkMeDead handling
mon/OSDMonitor: set dead_epoch for immediate failure (ECONNREFUSED)
osd/OSDMap: add dead_epoch to xinfo

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #29323 into master

* refs/pull/29323/head:
rocksdb: enable rocksdb_rmrange=true by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29300 into master

* refs/pull/29300/head:
doc: fix youtube channel url
doc: fix urls in posix.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28874 into master

* refs/pull/28874/head:
ceph-mds: dump all info of ceph_file_layout, InodeStoreBase, frag_info_t, inode_t

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29103 into master

* refs/pull/29103/head:
mgr/volumes: add `ceph fs subvolumegroup getpath` command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29166 into master

* refs/pull/29166/head:
mds: Reorganize class members in Capability header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #29149 into master

* refs/pull/29149/head:
cephfs-shell: Fix flake8 E302 blank line error
cephfs-shell: Fix flake8 indentation error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28860 into master

* refs/pull/28860/head:
mds: Move scalar type initializations of CDir data members to its header
mds: Reorganize class members in CDir header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

mon/OSDMonitor: do not clobber pending xinfo during boot

If we have a pending xinfo change and also process a boot message, we
should not throw out the pending xinfo change.

This triggers when you mark an osd down and it sends both a MOSDMarkMeDead
and a new MOSDBoot message in quick succession: the first message sets
dead_epoch but the boot message processing clobbers it.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: add is_dead() helper

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: 'osd down ... --definitely-dead'

Like 'osd down', but we also set dead_epoch.  Should only be used if we
know the process/host is already completely dead and unresponsive.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29036 from mikechristie/rbd-nbd-add-nl-resize

rbd-nbd: add nl resize

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #28955 into master

* refs/pull/28955/head:
doc: Update mount CephFS index

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29111 into master

* refs/pull/29111/head:
qa/mgr/progress: Update the test suite for progress module

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: send MOSDMarkMeDead when we are marked down

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: implement MOSDMarkMeDead handling

Allow updates to the dead_epoch in osd_xinfo, so that the OSD can inform
us that they know they are down (as of a particular epoch).

Signed-off-by: Sage Weil <sage@redhat.com>

doc/mgr/telemetry: update

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/telemetry: move contact info to an 'ident' channel

Signed-off-by: Sage Weil <sage@redhat.com>

rocksdb: enable rocksdb_rmrange=true by default

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29117 from tchaikov/wip-mgr-set_health_checks-unicode

mgr: check for unicode passed to "set_health_checks()"

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #29320 into master

* refs/pull/29320/head:
os/bluestore: remove unused arg to _get_deferred_op()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29282 from yuriw/wip-yuriw-upgrade-rwg-splits-master

qa/tests: added rgw into upgrade sequence to improve coverage - splits

os/bluestore: remove unused arg to _get_deferred_op()

Fixes: http://tracker.ceph.com/issues/40918
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: RGW bucket creation when no placement target received

Fixes: https://tracker.ceph.com/issues/40567
Signed-off-by: alfonsomthd <almartin@redhat.com>

mgr/dashboard: run e2e tests against prod build (jenkins job)

Signed-off-by: alfonsomthd <almartin@redhat.com>

doc: fix youtube channel url

Signed-off-by: Jos Collin <jcollin@redhat.com>

doc: fix urls in posix.rst

Signed-off-by: Jos Collin <jcollin@redhat.com>

doc: Update mount CephFS index

Updated based on the discussion:
https://github.com/ceph/ceph/pull/28900#discussion_r301812933

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #29296 from tchaikov/wip-remove-script/dep-report.sh

script: remove dep-report.sh

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/PyFormatter.h: add necessary #include

this header uses `std::string`, `std::string_view` and
`std::stringstream`, so it should include the corresponding headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: check for unicode passed to "set_health_checks()"

in python2, in addition to `str`, a string could also be unicode
string. there is chance that user passes a unicode string to
`set_health_checks()`, so let's check for unicode string also.

in python3, all strings are `unicode` string, and `PyString_Check()` is
defined as an alias of `PyUnicode_Check()`, so we are fine with python3.

the wrapper method of `PyString_ToString()` returns a pair of converted
string and a bool. if it fails to detect a string, an empty string and
false are returned.

Signed-off-by: Kefu Chai <kchai@redhat.com>

script: remove dep-report.sh

dep-report.sh depends on .Po and .Plo files under .deps/, which is
created by automake. but we've switched to CMake. so we cannot use
these depfile or temporary depfiles anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #29034 into master

* refs/pull/29034/head:
doc/mgr/crash: document missing commands, options
qa/suites/rados/singleton/all/test-crash: whitelist RECENT_CRASH
qa/suites/rados/mgr/tasks/insights: whitelist RECENT_CRASH
qa/tasks/mgr/test_insights: crash module now rejects bad crash reports
mgr/telemetry: fix remote into crash do_ls()
mgr/crash: don't make these methods static
mgr/BaseMgrModule: handle unicode health detail strings
mgr/crash: verify timestamp is valid
qa/suites/mgr: whitelist RECENT_CRASH
mgr/crash: remove unused var
mgr/crash: remove unused import 'six'
qa/workunits/rados/test_crash: health check
mgr/crash: improve validation on post
mgr/crash: automatically prune old crashes after a year
mgr/crash: raise RECENT_CRASH warning for recent (new) crashes
mgr/crash: add 'crash ls-new'
mgr/crash: add option and serve infra
mgr/crash: keep copy of crashes in memory
mgr/pg_autoscaler: adjust style to match built-in tables
mgr/crash: make 'crash ls' a nice table with a NEW column
mgr/crash: nicely format 'crash info' output
mgr/crash: add 'crash archive <id>', 'crash archive-all' commands

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26160 from linuxbox2/wip-rgw-lc-header

rgw: project and return lc expiration from GET/HEAD and PUT ops

qa/tests: added rgw into upgrade sequence to improve coverage - splits

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

doc/mgr/crash: document missing commands, options

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29234 from yuriw/wip-yuriw-upgrade-rwg-master

qa/tests: added rgw into upgrade sequence to improve coverage

Merge pull request #29263 from trociny/wip-rbd-external_callback

librbd: don't use complete_external_callback if ImageCtx destroyed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #29180 into master

* refs/pull/29180/head:
mon/FSCommand: set pg_num_min via 'fs new', not in mgr/volumes
mon/FSCommands: set pg_autoscale_factor on 'fs new', not via mgr/volumes
mon: set recovery_priority=5 on fs metadata pool

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27575 into master

* refs/pull/27575/head:
osd: avoid prep_object_replica_pushes() on clone object when head missing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29261 from jan--f/add-auth-to-ghub-api-calls

script/ceph-backport-sh: add access_token parameter to all ghub api c…

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge PR #29124 into master

* refs/pull/29124/head:
test/old/test*buffers.cc: remove tests against bufferlist
test/old/test_*seek_read: remove this test
test/old/test{crush,bucket}: remove stale tests
test/old/testmpi.cc: remove this test
test/old/testxattr.cc: remove this test
test/old/testtree.cc: remove test for BinaryTree

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: qa: fix RBD test when matching error strings

This commit fixes a problem caused by changes to the error
descriptions in the RBD python bindings introduced in:
a66b06cc764ba66cdabfe2856e9f1bcd738f0772

Signed-off-by: Ricardo Dias <rdias@suse.com>

script/ceph-backport-sh: add access_token parameter to all ghub api calls

Otherwise one can get a "API rate limit exceeded for $YOUR_IP." message
when working from company offices and such.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

librbd: don't use complete_external_callback if ImageCtx destroyed

complete_external_callback (which requires ImageCtx) is used to
prevent concurrent callbacks, and not needed in this particular case.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29148 from tchaikov/wip-cmake-python2

cmake: use python2 by default

Reviewed-by: Casey Bodley <cbodley@redhat.com>

os/BlueStore: Split Cache into Onode/Buffer caches

Signed-off-by: Mark Nelson <mnelson@redhat.com>

mds: Reorganize class members in Capability header

Fixes: https://tracker.ceph.com/issues/40866
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #28448 from shyukri/wip-25030

ceph-volume: util: Use proper param substition

Merge pull request #29073 from dillaman/wip-40790

mon/mgr: add 'rbd_support' to list of always-on mgr modules

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #28765 from myoungwon/fix-chunk-scrub-dedup-tool

src/tools/ceph-dedup-tool: Fix chunk scrub

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28832 from majianpeng/osd-only-empty-do-wakeup

osd/OSD: only wake up empty pqueue.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28778 from myoungwon/wip-chunk-flush-op

src/osd: add tier-flush op

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28983 from myoungwon/wip-dedup-io-test

src/test, qa/suites/rados/thrash: add dedup test

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29053 from dmick/master

ceph-crash: use open(..,'rb') to read bytes for Python3

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28054 from iotcg/starlingx

pybind: add verbose error message

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26724 from jecluis/dev/wip-mon-osd-info

mon: add `ceph osd info` to obtain info on osds rather than parsing `osd dump`

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28772 from neha-ojha/wip-40528-2

osd: add hdd, ssd and hybrid variants for osd_snap_trim_sleep

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

qa/tests: added rgw into upgrade sequence to improve coverage

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #29026 from markhpc/wip-rocksdb-v6.1.2

rocksdb: Updated to v6.1.2

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29083 from dillaman/wip-40802

doc/rbd: update krbd version support for RBD features

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard: RGW user E2E Tests

Tests creating, editing and deleting RGW users.
Tests providing invalid input to user creation and editing forms.

Fixes: https://tracker.ceph.com/issues/40364
Fixes: https://tracker.ceph.com/issues/40908
Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Rafael Quintero <rquinter@redhat.com>

rbd-nbd: Fix spacing in UpdateWatchCtx

UpdateWatchCtx used a mix of 2 spaces and 4 spaces. This fixes the
function to use 2 spaces which is used in the rest of the file.

Signed-off-by: Mike Christie <mchristi@redhat.com>

rbd-nbd: Add resize support

This adds resize support to rbd nbd when using the netlink interface.
It requires this linux kernel patch:

https://marc.info/?l=linux-block&m=155916096905474&w=2

which has been merged into the block tree for 5.4.

Signed-off-by: Mike Christie <mchristi@redhat.com>

do_cmake.sh: specify the supported py versions for different distros

there is chance that a distro offer a newer python3 version than the
supported one, so we need to avoid using the latest python3 version by
specifying the supported python3

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: use exact version of python if minor version is specified

`find_package(Python ${version}...)` tries to find the greater python version
which is greater than `${version}`, on fc30, at the time of writing, both
python3.8 and python3.7 are offered. but `python3-Cython` is packaged only
for python3.7. so if user installs python3.8, this will prevent user from
building Ceph. as Ceph will not be able to find Cython python module, as it
will try to run `python3.8 -m cython --version`, where python3.8 is the
greatest python version available in the system. but since cython module is
not available to python3.8, cmake will fail to find cython even if is available
to python3.7.

in this change, if user specifies a python version with minor version, we
will use the exact specified version instead of trying to use a version
greater than the specified one.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29152 from tchaikov/wip-cmake-crimson-test

test/crimson/: use 256M mem and 1 cpu core for each test

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mon/FSCommand: set pg_num_min via 'fs new', not in mgr/volumes

Signed-off-by: Sage Weil <sage@redhat.com>

mon/FSCommands: set pg_autoscale_factor on 'fs new', not via mgr/volumes

Signed-off-by: Sage Weil <sage@redhat.com>

mon: set recovery_priority=5 on fs metadata pool

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29114 into master

* refs/pull/29114/head:
mgr/dashboard: qa: whitelist client eviction warning

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge PR #29164 into master

* refs/pull/29164/head:
mgr: cleanup idle debug log at level 4.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29172 into master

* refs/pull/29172/head:
mgr/dashboard: Fix dashboard health test failure

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29169 into master

* refs/pull/29169/head:
qa/suites/rados/mgr/tasks/module_selftest: whitelist mgr client getting backlisted

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26690 from jecluis/dev/wip-vstart-fix-rgw

vstart.sh: run multiple rgws with different ids

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29195 from ricardoasmarques/fix-ceph-iscsi-required-version

doc: Update 'ceph-iscsi' min version

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/rbd: update krbd version support for RBD features

Fixes: http://tracker.ceph.com/issues/40802
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

vstart.sh: run multiple rgws with different ids

We should expect being able to run multiple rgws with different ids
using vstart, and yet that was not the case as they were sharing the
same name (and same key).

We now mimic each rgw running on a different host, with ids being their
ports; as such, we'll have 'client.rgw.X', 'client.rgw.X+1', and so
forth. These will match the ports the daemons are running on.

Signed-off-by: Joao Eduardo Luis <joao@suse.com>

Merge pull request #29132 from votdev/issue_40829

mgr/dashboard: RGW User quota validation is not working correctly

Reviewed-by: Ricardo Marques <rimarques@suse.com>

doc: Update 'ceph-iscsi' min version

Signed-off-by: Ricardo Marques <rimarques@suse.com>

ceph-mds: dump all info of ceph_file_layout, InodeStoreBase, frag_info_t, inode_t

Signed-off-by: simon gao <simon29rock@gmail.com>

Merge pull request #28986 from sebastian-philipp/orchestrator-verify-certificate

pybind/mgr: Remove code duplication

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #29184 from neha-ojha/wip-1731894

mon/OSDMonitor.cc: better error message about min_size

Merge pull request #29070 from adk3798/reformat

mgr/dashboard: Update formatting of e2e test files

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29042 from sebastian-philipp/orchestrator-doc-disable

mgr/orchestrator: Fix disabling the orchestrator

Reviewed-by: Tim Serong <tserong@suse.com>

build(deps): bump lodash in /src/pybind/mgr/dashboard/frontend

Bumps [lodash](https://github.com/lodash/lodash) from 4.17.11 to 4.17.13.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.11...4.17.13)

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #29058 from tchaikov/wip-ceph-conf

tool/ceph-conf: s/global_pre_init()/global_init()/

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29151 from tchaikov/wip-rgw-kill-gcc-warnings

rgw/rgw_crypt.cc: silence -Wsign-compare GCC warning

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #29155 from tchaikov/wip-msg-cleanup

msg/Message: remove unused local variables

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

mon/OSDMonitor.cc: better error message about min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29183 from gabriellasroman/patch-2

doc/dev/developer_guide: remove web address

Reviewed-by: Neha Ojha <nojha@redhat.com>