Merge pull request #29950 from smithfarm/wip-bri-deep-scan-of-pr-desc

script/backport-resolve-issue: handle tracker URLs better

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29815 from smanjara/wip-fix-bucket-link

rgw: fix missing tenant prefix in bucket name during bucket link

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

script/backport-resolve-issue: handle tracker URLs better

Before this commit, backport-resolve-issue would blindly grab the first tracker
URL it saw in the GitHub PR description. With this commit, it examines all the
tracker URLs it sees in the PR description and uses the first one that
corresponds to an issue in the Redmine Backport tracker.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: migrate e2e tests to async await (#29758)

mgr/dashboard: migrate e2e tests to async await

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29942 from tchaikov/wip-bump-seastar

seastar: pick up latest changes and cleanups

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

mgr/dashboard: fix protractor.conf.js

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: set sensible `implicitWaitTimeout`

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: fix issue in dashbboard test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: prepend await before expect statements are used

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: fix issue E2E image with unclickable element

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring E2E mirroring test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refatoring E2E page helper test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring E2E iscsi test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring Helper class to remove PageHelper instances

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring E2E dashboard test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: reformat of `protractor.conf.js`

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring of references of classes to point to classes directly

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: fix UnhandledPromiseRejectionWarning in RGW users

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E dashboards to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: improve overall E2E stability of navigation

The `navigateTo` method will check if a table exists on the table it has
been used to navigate to and if so, wait for the data of the table to
be loaded.

Fixes: https://tracker.ceph.com/issues/41349
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E user management tests

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E images to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E iscsi to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E mirroring to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E alerts to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E configuration to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E crushmap to async/await

Needed to adapt the backend to return the type of the tree nodes as the
previous selector did not work reliably.

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E hosts to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E logs to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E mgr modules to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E monitors to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E osds to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E filesystems to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E nfs to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E daemons to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E rgw-users to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E buckets to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E Helper to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: run feature toggle service outside of Angular zone

to prevent Proctator from hanging indefinitely when testing pages where
the feature toggles are retrieved.

Fixes: mgr/dashboard: Switch to async/await for E2E tests
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: replace E2E test `moveClick` with `clickCheckbox`

Ceph Dashboard replaces the original checkbox with a fancy replacement
using the pseudo selector "::before" inside the <label> tag. When the
fancy replacement checkbox is clicked, it actually triggers a click on
the label of the checkbox. `clickCheckbox` does the same.

Removed `moveClick` as it was used to perform an action that the user
would not be able to perform, namely clicking on the checkbox instead of
the checkboxes' label. It was also used for cases where it was not
necessary to be used for, hence the replacement `clickCheckbox` has been
introduced. It checks if the argument passed to it is of type `checkbox`
or `label`. If it is of type `label`, the label will be clicked. If it
is of type `checkbox`, the corresponding `label` is determined and
clicked. This replaces the old implementation with a solution that has
an easy to understand purpose. It also prevents every false application.

This method is designed to be used on all checkboxes of Ceph Dashboard.

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E pools to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: prepare PageHelper to migrate pools

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: disable selenium promise manager

and migrate `protractor.conf.js` to use async/await.

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

Merge pull request #29889 from smithfarm/wip-clay-typo-fwdport

doc/rados: Correcting some typos in the clay code documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

crimson/net/Socket: adapt to seastar v2 api

variadic futures is deprecated in seastar api v2, so let's use the new
variant.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net/ProtocolV2: include fmt/chrono.h if fmt version >= 6.0

* check libfmt version instead of the exisistence of chrono.h, as
  libfmt merged fmt/time.h into fmt/chrono.h since v 6.0, while in
  libfmt v5.3, fmt/chrono.h did not provide all the necessary bits
  for fmt::to_format(..., const tm&). see also
  https://github.com/fmtlib/fmt/commit/a939c75956d27996680ce498529da0d1668c4eac
  and https://github.com/fmtlib/fmt/blob/master/ChangeLog.rst
* use fmt API instead of libc API for getting `localtime()`, it's
  more C++ friendly than plain `localtime_r()`

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29858 from dillaman/wip-41409

test/librbd: set nbd timeout due to newer kernels defaulting it on

Reviewed-by: Mike Christie <mchristi@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29932 from smithfarm/wip-doc-layout-fwdport

doc: fix typo in doc/radosgw/layout.rst

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

seastar: pick up latest changes

* v2 api: remove variadic futures. let's be in sync with upstream.
* warn if the returned future is ignored. this helps us to catch
  bugs and to improve the readability of the code -- we need to
  explicitly ignore a future if we want to. and ideally, we should
  document the reason why a future is ignored.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #28819 into master

* refs/pull/28819/head:
doc/man/8/monmaptool: document the `--addv` argument.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29470 from liu-chunmei/ceph_seastar_cyan_store

crimson: futurized CyanStore's member functions and Collection

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/os/cyan_store: remove stale comment

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: add FuturizedCollection

we will need to talk to BlueStore, and currently `ceph::os::Collection`
only works with `CyanStore`, so a wrapper around `ceph::os::Collection`
is added. we can subclass it to adapt to CyanStore and to BlueStore.

the underlying concrete `FuturizedStore` classes are responsible to
cast `FuturizedCollection` to their own `Collection` type if they
want to access the interface not exposed by `FuturizedCollection`.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

crimson/os: s/try_emplace()/insert()/

avoid specifying the type of `map<>::value_type`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: always use Collection::get_cid()

prepare for the futurized Collection, which will be an abstract
interface allowing us to talk to BlueStore with one of its concrete
subclass.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/ObjectStore: fix typo in comments

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: lift continuation in PGMeta::load()

after making `open_collection()` return a future, we don't need to chain
the body of `PGMeta::load()` after `omap_get_values()`. instead we can
lift it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: add futurized interfaces to CyanStore

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

Merge pull request #29853 from dillaman/wip-41411

test/cli-integration/rbd: fixed missing image and snap ids

Reviewed-by: Mykola Golub <mgolub@suse.com>

doc: modify the wrong word "defails" to "details".

Signed-off-by: ypdai <self19900924@gmail.com>
(cherry picked from commit 8cefe3de7835ce136826faf595122cc210bf90af)

Merge pull request #29877 from yuvalif/pubsub_regression_issue2

rgw/pubsub: service reordering issue

Merge pull request #29797 from smithfarm/wip-backport-resolve-issue

script: add backport-resolve-issue

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29904 from smithfarm/wip-bci-resolve-parent

script/backport-create-issue: add --resolve-parent feature

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29912 from tchaikov/wip-wheels-only-for-make-check

install-deps.sh: only prepare wheels for "make check"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

script: add backport-resolve-issue

This script processes GitHub backport PRs, checking for proper cross-linking
with a Redmine Backport tracker issue and, if a PR is merged and properly
cross-linked, it can optionally resolve the tracker issue and correctly
populate the "Target version" field.

The script takes a single positional argument, which is optional. If the
argument is an integer, it is assumed to be a GitHub backport PR ID (e.g. "28549").
In this mode ("single PR mode") the script processes a single GitHub backport
PR and terminates.

If the argument is not an integer, or is missing, it is assumed to be a
commit (SHA1 or tag) to start from. If no positional argument is given, it
defaults to the tag "BRI", which might have been added by the last run of the
script. This mode is called "scan merge commits mode".

In both modes, the script scans a local git repo, which is assumed to be
in the current working directory. In single PR mode, the script will work
only if the PR's merge commit is present in the current branch of the local
git repo. In scan merge commits mode, the script starts from the given SHA1
or tag, taking each merge commit in turn and attempting to obtain the GitHub
PR number for each.

For each GitHub PR, the script interactively displays all relevant information
(NOTE: this includes displaying the GitHub PR and Redmine backport issue in
web browser tabs!) and prompts the user for her preferred disposition.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #29790 from hjwsm1989/fix-unittest-osdmap

tests/osd: fix typo in unittest_osdmap

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw/pubsub: service reordering issue

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>

install-deps.sh: only prepare wheels for "make check"

* move `for_make_check` to the beginning of script, as FreeBSD will also
  use this variable
* extract `preload_wheels_for_tox()` function out to improve readability
* call `preload_wheels_for_tox()` only if `for_make_check` is true

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/man/8/monmaptool: document the `--addv` argument.

Fixes: http://tracker.ceph.com/issues/40568
Signed-off-by: Luca Castoro <luca.castoro@brightcomputing.com>

Merge pull request #29090 from liewegas/wip-40792

mon/MonClient: ENXIO when sending command to down mon

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28177 from jecluis/wip-noreply-failures

mon: ensure prepare_failure() marks no_reply on op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

script/backport-create-issue: add --resolve-parent feature

When --resolve-parent is provided on the command line, the script
will check the status of backport issues for those parent issues
all of whose backport issues already exist. If all the backport
issues are in status "Resolved", the parent issue's status is set
to "Resolved" as well.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #29862 from tchaikov/wip-41416

rgw,bluestore: fixes to address failures from check-generated.sh

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29893 from xiexingguo/wip-inc-recovery-5

osd/osd_types: inc-recovery - add special handler for lost_revert

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

test/encoding/check-generated.sh: show diff if cmp fails

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/encoding/check-generated.sh: run loop in the same shell

set parent scope variables in the same shell.

foo | while read ....

in the above statement, `while read ...` is executed in a subshell. so
it cannot change the bash variables in its parent shell.

in this change, the output of `foo` is redirected to the stdin of `while
read` statement. so we can capture the test failures.

before this change, the test always succeed, even if there are failures.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/DecayCounter: do not decay when testing denc

check-generated.sh tries to verify the invariance of a type after
encoding and decoding. so we should keep the value of `DecayCounter` the
same when encoding/decoding/dumping.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: rgw_log_entry always initialize total_time

zero-initialize rgw_log_entry::total_time

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/ServiceMap: use plain gid

* partially revert 5c25a018. which is not backward compatible.
* change `ServiceMap::get_daemon()` so it returns a
  `pair<Daemon*,bool>`.

git is an `optional<uint64_t>`, so we cannot dump it without checking.

Fixes: https://tracker.ceph.com/issues/41424
Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: init pg_pool_t::pg_autoscale_mode in ctor

so its value is determined

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: use scoped enum for pg_pool_t::pg_autoscale_mode

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: use in-class initialization in pg_pool_t

less repeat this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: implement copy ctor and assignment for bluestore_blob_use_tracker_t

to address the `check-generated.sh` test failure.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: RGWObjManifest::obj_iterator::seek always update locations

we should update explicit loc and location even if
`ofs >= manifest->obj_size`. there is chance that we are updating an end
iterator whose ofs is equal to obj_size. before being updated, the end
iterator points to an implicit location, while after being updated, the
manifest could be using an explicit location, so we should update the
end iterator as well.

Fixes: https://tracker.ceph.com/issues/41416
Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: RGWObjManifest::obj_iterator: init member variables in-class

* remove the init() member function which is solely used by the
  constructors.
* use constructor delegation to minimize the repeating

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: update_iterator() in RGWObjManifest::set_explicit()

we should update the end iterator when updating RGWObjManifest with its
parts

Fixes: https://tracker.ceph.com/issues/41416
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29913 from tchaikov/wip-debian/control

debian/control: fix Build-Depends

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

debian/control: fix Build-Depends

it's a regression introduced by 5d6d770e

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #29867 into master

* refs/pull/29867/head:
doc: default values for mon_health_to_clog_* were flipped

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29822 from vumrao/wip-vumrao-41399

os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1.

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1
Fixes: https://tracker.ceph.com/issues/41399
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

Merge pull request #29903 from tchaikov/wip-pip-download

install-deps.sh: download wheel using 'pip wheel'

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

install-deps.sh: download wheel using 'pip wheel'

otherwise we will fail to install the build dependencies of
`lazy-object-proxy` from the wheelhouse. as `lazy-object-proxy` does not
add `setuptools_scm` in its `setup.py`, instead it lists
`setuptools_scm` in `setup.cfg` and `pyproject.toml` as a `build-system`
requires. but unfortunately, `pip download` only downloads the
install/run-time dependencies at this moment. and `lazy-object-proxy`
does not offer binary package for at least python2.7.

ideally, `pip download` should collects its dependencies like

Collecting setuptools_scm>=3.3.1 (from lazy-object-proxy->astroid<3,>=2.2.0->pylint->-r requirements-lint.txt (line 1))

so we need to use `pip wheel` do download build-time dependencies

see also https://github.com/pypa/pip/issues/6222

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29854 from tchaikov/wip-sloppy-cond

common/condition_variable_debug: do not assert() if sloppy

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #29866 from joscollin/wip-fix-stupid

os/bluestore: silence StupidAllocator reorder warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29890 from rhcs-dashboard/wip-signed-master

github: display phrase for signed-off check

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #29831 from tchaikov/wip-python-int

mgr/BaseMgrModule: use PyInt_Check() to compatible with py2

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

mgr/dashboard: Edit Pool Placement Group E2E Test (#29779)

mgr/dashboard: Edit Pool Placement Group E2E Test

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29707 from rafaelquint/dashdoc

mgr/dashboard: Extend "Writing End-to-End Tests" section (describe vs it)

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

github: display phrase for signed-off check

https://github.com/ceph/ceph-build/pull/1379 will introduce a
trigger-phrase for pr-commit check (DCO/signed-off).

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>