mgr/dashboard: Display the duration of each e2e test

Now we are able to see which tests are taking
too much time and may need improvements.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Increase the global value of jasmine timeout

Some tests are taking longer than the default jasmine timeout to finish
and are causing a failure.

This commit will increase the timeout to 5 minutes,
removing the need to increase it for each failing test.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #30106 from smithfarm/wip-more-cbs

script/ceph-backport.sh: make the script idempotent

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

mgr/dashboard: key-value-table doesn't render booleans (#30203)

 mgr/dashboard: key-value-table doesn't render booleans

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Fix error message not displayed in pool form (#30137)

mgr/dashboard: Fix error message not displayed in pool form

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #28936 from tchaikov/wip-unaligned-access

cmake: check for unaligned access

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28939 from trociny/wip-37529

rbd-mirror: add namespace support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30192 from dzafman/wip-41669

Make dumping of reservation info congruent between scrub and recovery

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30347 from neha-ojha/wip-add-backend

qa/suites/rados/perf: test min recommended osd_memory_target

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #30319 from zhangsw/rgw-fix-zone-rm

rgw: fix incorrect "radosgw-admin zonegroup rm" info.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

qa/suites/rados/perf: test min recommended osd_memory_target

Add bluestore-basic-min-osd-mem-target.yaml to objectstore.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #30346 into master

* refs/pull/30346/head:
script: raw_input was renamed to input in py3

Reviewed-by: Sage Weil <sage@redhat.com>

script: raw_input was renamed to input in py3

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30014 into master

* refs/pull/30014/head:
doc: update CephFS overview in introductory page
doc: filesystem to file system

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #30221 from cbodley/wip-rgw-dns-name-iends-with

rgw: fix dns name comparison for virtual hosting

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #30310 from rzarzynski/wip-crimson-op_effects

crimson/osd: extend OpsExecuter to carry about op effects.

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: E2E enhancements (#30017)

mgr/dashboard: E2E enhancements

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29606 from sseshasa/bluestoreCfgObserver

os/bluestore: Add config observer for osd memory specific options.

Merge pull request #29793 from sebastian-philipp/ansible-default-realm-zonegroup

mgr/ansible: Change default realm and zonegroup

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #30321 from tchaikov/wip-run-cbt.sh

script/run-cbt.sh: always use python3

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Update protractor dependencies

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enable waitforAngular in E2E

Convert waitForTableData into a method,
using it as a decorator was causing problems.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add protractor-fast-fail plugin

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add click call inside waitClickable

All waitClickable calls were being followed by a click.
This will remove the need for the click call.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix error in pool list component

Sometimes the pools list result comes without the tiers field.
This was causing an error in the code.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix random failure in images e2e test

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove duplicated method

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove helper.po.ts

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove E2E warning

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Extract delete method into page-helper

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Use suites in protractor

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Split user-mgmt e2e files

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Move some e2e into new "ui" folder

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve clickCheckbox with some waits

Signed-off-by: Tiago Melo <tmelo@suse.com>

script/ceph-backport.sh: help the user set up the script

This commit adds several options:

    --setup
    --setup-advice
    --usage-advice
    --troubleshooting-advice

It replaces the "comment block at the beginning of the script" which was lamely
masquerading as documentation.

It also implements heuristic deduction of GitHub remotes.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

script/ceph-backport.sh: check Backport ticket assignee

Don't work on tickets assigned to someone else.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Improve images e2e test (#30212)

mgr/dashboard: Improve images e2e test

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

mgr/dashboard: Wait for breadcrumb text is present in e2e tests (#30267)

mgr/dashboard: Wait for breadcrumb text is present in e2e tests

mgr/dashboard: Rendering datatable causes Angular runtime error (#30209)

mgr/dashboard: Rendering datatable causes Angular runtime error

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

script/run-cbt.sh: always use python3

run-cbt.sh will be used by the "ceph-perf-pull-requests" jenkins job,
which will prepare the runtime dependencies by installing python3
modules.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30318 from tchaikov/wip-run-make.sh

script/run-make.sh: do not pass cmake options twice

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: fix incorrect "radosgw-admin zonegroup rm" info.

Signed-off-by: zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

script/run-make.sh: do not pass cmake options twice

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30292 from Reperator/wip-doc-dashboard-fix-typo

doc: Fixes typo for ceph dashboard command.

Merge pull request #30281 from sebastian-philipp/doc-orchestrator-feature-update

doc/orchestrator: Sync status with reality

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #30231 from xiexingguo/wip-blue-balance

os/bluestore: add bluestore_bluefs_max_free; smooth space balancing a bit

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30315 from tchaikov/wip-crimson-perf-test

test/crimson: add acceptable section to tests

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: check for unaligned access

C and C++ allow us to cast a pointer with an alignment requirement to
one with a different alignment requirement. but if the destination
type's alignment is greater/stricter than the source type, the behavior
is undefined. on amd64, it's allowed. and on aarch64, GCC offers an
option named `-munaligned-access` to enable this behavior. so we cannot
take this as guaranteed.

in denc.h and messenger implementations, we just cast the pointers to
the type we want to access without memcpy(), which is, strictly
speaking, not quite standard compliant.

so, to be on the safe side, we detect the support of unaligned access at
compile time.

please note, this would fail the cross-compile. as the target platform
might not support unaligned access, and the produced executable does not
run on the build platform. so this test is skipped if the building
host's architecture is different than that of target host.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: extract checks into CephChecks.cmake

also use case chosen in the macro definition as per suggested by cmake
document. see
https://cmake.org/cmake/help/latest/command/macro.html#invocation

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #29809 into master

* refs/pull/29809/head:
common/config_proxy: hold lock while accessing mutable container

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

test/crimson: add acceptable section to tests

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30124 from rzarzynski/wip-crimson-conn-in-msg

crimson/osd: Message has non-null ref to SocketConnection now.

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30309 from neha-ojha/wip-41661

qa/suites/rados/perf/objectstore: do not symlink to qa/objectstore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30255 from xiexingguo/wip-bluestore-gift-ratio

os/bluestore: fix space balancing overflow

Reviewed-by: Sage Weil <sage@redhat.com>

crimson/osd: extend OpsExecuter to carry about op effects.

This commit brings a new infrastructural piece that will be
needed by e.g. the watch/notify mechanism.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #30308 into master

* refs/pull/30308/head:
doc/dev: add teuthology priority recommendations

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

osd: Replace active/pending scrub tracking for local/remote

This is similar to how recovery reservations are split between
local and remote.

It was the case that scrubs_pending was used for reservations at
the replicas as well as at the primary while requesting reservations
from the replicas.  There was no need for scrubs_pending to turn
into scrubs_active at the primary as nothing treated that value
as special.  scrubber.active = true when scrubbing is
actually going.

Now scurbber.local_reserved indicates scrubs_local incremented
Now scrubber.remote_reserved indicates scrubs_remote incremented

Fixes: https://tracker.ceph.com/issues/41669
Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Add dump_scrub_reservations to show scrub reserve tracking

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Rename dump_reservations to dump_recovery_reservations

Signed-off-by: David Zafman <dzafman@redhat.com>

qa/suites/rados/perf/objectstore: do not symlink to qa/objectstore

Removing the symlink since we do not want to run performance tests
on filestore-xfs.yaml.

Fixes: https://tracker.ceph.com/issues/41661
Signed-off-by: Neha Ojha <nojha@redhat.com>

doc/dev: add teuthology priority recommendations

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30269 from dabukalam/patch-1

doc: Update ceph-deploy docs from dumpling to nautilus

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: Update ceph-deploy docs from dumpling to nautilus

Signed-off-by: Danny Abukalam <danny@softiron.co.uk>

Merge pull request #30274 from alfredodeza/wip-rm41660

ceph-volume: fix stderr failure to decode/encode when redirected

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc: update CephFS overview in introductory page

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30287 from tchaikov/wip-run-cbt

script/run-cbt.sh: check option correctly

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #30290 from tchaikov/wip-doc-cbt

doc/dev/crimson.rst: document CBT testing

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc: filesystem to file system

"Filesystem" is not a word (although fairly common in use).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30289 from wjwithagen/wjw-fix-pylint-dashboard-progress.py

dashboard/services: fix lint error

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

dashboard/services: fix lint error

```
lint run-test: commands[0] | flake8
 many blank lines (3)
1     E303 too many blank lines (3)
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

doc: Fixes typo for ceph dashboard command.

Signed-off-by: Fabian Bonk <fabian.bonk@croit.io>

os/bluestore: smooth space balancing a bit

For workloads that need to handle tons of small objects,
the space of "slow" could become extremely fragmented,
which makes it is hard to transmit free space between
bluestore and bluefs.

Hard cap alloc size at 1GB at per request, so there is
1024 small allocations at most. Hence we don't block
write ops for long.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph-volume tests create a test file for checking unicode output

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests create a shell test for functional unicode

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests verify new logging fallback and encodings in terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume terminal remove unicode stream handler

This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: instantiate the new terminal logger in main()

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume create a logger for the terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge PR #30155 into master

* refs/pull/30155/head:
mgr/MgrClient: send MMgrCommand for octopus+ mgrs
mon/MgrMap: include mgr_features in mgrmap
mgr: accept MMgrCommand[Reply] messages (in addition to MCommand[Reply])

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30162 from cyx1231st/wip-seastar-msgr-test-failover-crimson

crimson/test: v2 failover tests with crimson FailoverTestPeer

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/dev/crimson.rst: document CBT testing

Signed-off-by: Kefu Chai <kchai@redhat.com>

script/run-cbt.sh: check option correctly

* s/source_dir/source-dir/: as the option is named "--source-dir", not
  "--source_dir"
* cd $build_dir, before launching/stopping vstart. as vstart.sh is
  supposed to run from the build directory

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30111 from tspmelo/wip-nfs-read

mgr/dashboard: Set RO as the default access_type for RGW NFS exports

Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: progress: support rbd_support module async tasks (#29424)

mgr/dashboard: progress: support rbd_support module async tasks

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #30095 from rjfd/wip-fix-progress-qa

qa/mgr/progress: fix timeout error when waiting for osd in event

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: Wait for breadcrumb text is present in e2e tests

When the cluster is busy it might take some time until the tested component has loaded it's data and is rendered in the browser. Waiting for the breadcrumb text to be present might help to fix errors in the e2e tests.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #30078 from rhcs-dashboard/fix-41602-master

mgr/dashboard: disable pylint's --py3k flag

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #30126 from dvanders/dvanders_spec

ceph.spec.in: reserve more memory per build job

Reviewed-By: Nathan Cutler <nculter@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30258 from tchaikov/wip-41721

test/cls_rbd/test_cls_rbd: update TestClsRbd.sparsify

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/orchestrator: Sync status with reality

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

os/bluestore: fix space balancing overflow

When gifting, bluestore should not gift more than it actually has.
same for bluefs when reclaming.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30273 from neha-ojha/wip-documenting-ceph

doc/start/documenting-ceph.rst: make better doc recommendations

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc/start/documenting-ceph.rst: make better doc recommendations

I do not think "Editors" is relevant anymore.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #30187 into master

* refs/pull/30187/head:
script/backport-resolve-issue: better error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume create a new tox.ini for shell-based tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #30226 from neha-ojha/wip-41385-2

osd/MissingLoc.cc: do not rely on missing_loc_sources only

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge PR #30196 into master

* refs/pull/30196/head:
qa/tasks/ceph: restart: stop osd, mark down, then start

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30268 from cbodley/wip-41719

cls/rgw: fix bilog trim tests in ceph_test_cls_rgw

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge PR #30220 into master

* refs/pull/30220/head:
test: Allow fractional milliseconds to make test possible

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix error message not displayed in pool form

Since the parent form-group was not assigned to any field and was never invalid,
the invalid-feedback element was never displayed.

Signed-off-by: Tiago Melo <tmelo@suse.com>