script: raw_input was renamed to input in py3

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30014 into master

* refs/pull/30014/head:
doc: update CephFS overview in introductory page
doc: filesystem to file system

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #30221 from cbodley/wip-rgw-dns-name-iends-with

rgw: fix dns name comparison for virtual hosting

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #30310 from rzarzynski/wip-crimson-op_effects

crimson/osd: extend OpsExecuter to carry about op effects.

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: E2E enhancements (#30017)

mgr/dashboard: E2E enhancements

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29606 from sseshasa/bluestoreCfgObserver

os/bluestore: Add config observer for osd memory specific options.

Merge pull request #29793 from sebastian-philipp/ansible-default-realm-zonegroup

mgr/ansible: Change default realm and zonegroup

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #30321 from tchaikov/wip-run-cbt.sh

script/run-cbt.sh: always use python3

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Update protractor dependencies

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enable waitforAngular in E2E

Convert waitForTableData into a method,
using it as a decorator was causing problems.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add protractor-fast-fail plugin

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add click call inside waitClickable

All waitClickable calls were being followed by a click.
This will remove the need for the click call.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix error in pool list component

Sometimes the pools list result comes without the tiers field.
This was causing an error in the code.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix random failure in images e2e test

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove duplicated method

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove helper.po.ts

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove E2E warning

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Extract delete method into page-helper

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Use suites in protractor

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Split user-mgmt e2e files

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Move some e2e into new "ui" folder

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve clickCheckbox with some waits

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve images e2e test (#30212)

mgr/dashboard: Improve images e2e test

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

mgr/dashboard: Wait for breadcrumb text is present in e2e tests (#30267)

mgr/dashboard: Wait for breadcrumb text is present in e2e tests

mgr/dashboard: Rendering datatable causes Angular runtime error (#30209)

mgr/dashboard: Rendering datatable causes Angular runtime error

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

script/run-cbt.sh: always use python3

run-cbt.sh will be used by the "ceph-perf-pull-requests" jenkins job,
which will prepare the runtime dependencies by installing python3
modules.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30318 from tchaikov/wip-run-make.sh

script/run-make.sh: do not pass cmake options twice

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

script/run-make.sh: do not pass cmake options twice

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30292 from Reperator/wip-doc-dashboard-fix-typo

doc: Fixes typo for ceph dashboard command.

Merge pull request #30281 from sebastian-philipp/doc-orchestrator-feature-update

doc/orchestrator: Sync status with reality

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #30231 from xiexingguo/wip-blue-balance

os/bluestore: add bluestore_bluefs_max_free; smooth space balancing a bit

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30315 from tchaikov/wip-crimson-perf-test

test/crimson: add acceptable section to tests

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #29809 into master

* refs/pull/29809/head:
common/config_proxy: hold lock while accessing mutable container

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

test/crimson: add acceptable section to tests

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30124 from rzarzynski/wip-crimson-conn-in-msg

crimson/osd: Message has non-null ref to SocketConnection now.

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30309 from neha-ojha/wip-41661

qa/suites/rados/perf/objectstore: do not symlink to qa/objectstore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30255 from xiexingguo/wip-bluestore-gift-ratio

os/bluestore: fix space balancing overflow

Reviewed-by: Sage Weil <sage@redhat.com>

crimson/osd: extend OpsExecuter to carry about op effects.

This commit brings a new infrastructural piece that will be
needed by e.g. the watch/notify mechanism.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #30308 into master

* refs/pull/30308/head:
doc/dev: add teuthology priority recommendations

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

qa/suites/rados/perf/objectstore: do not symlink to qa/objectstore

Removing the symlink since we do not want to run performance tests
on filestore-xfs.yaml.

Fixes: https://tracker.ceph.com/issues/41661
Signed-off-by: Neha Ojha <nojha@redhat.com>

doc/dev: add teuthology priority recommendations

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30269 from dabukalam/patch-1

doc: Update ceph-deploy docs from dumpling to nautilus

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: Update ceph-deploy docs from dumpling to nautilus

Signed-off-by: Danny Abukalam <danny@softiron.co.uk>

Merge pull request #30274 from alfredodeza/wip-rm41660

ceph-volume: fix stderr failure to decode/encode when redirected

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc: update CephFS overview in introductory page

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30287 from tchaikov/wip-run-cbt

script/run-cbt.sh: check option correctly

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #30290 from tchaikov/wip-doc-cbt

doc/dev/crimson.rst: document CBT testing

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc: filesystem to file system

"Filesystem" is not a word (although fairly common in use).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30289 from wjwithagen/wjw-fix-pylint-dashboard-progress.py

dashboard/services: fix lint error

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

dashboard/services: fix lint error

```
lint run-test: commands[0] | flake8
 many blank lines (3)
1     E303 too many blank lines (3)
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

doc: Fixes typo for ceph dashboard command.

Signed-off-by: Fabian Bonk <fabian.bonk@croit.io>

os/bluestore: smooth space balancing a bit

For workloads that need to handle tons of small objects,
the space of "slow" could become extremely fragmented,
which makes it is hard to transmit free space between
bluestore and bluefs.

Hard cap alloc size at 1GB at per request, so there is
1024 small allocations at most. Hence we don't block
write ops for long.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph-volume tests create a test file for checking unicode output

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests create a shell test for functional unicode

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests verify new logging fallback and encodings in terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume terminal remove unicode stream handler

This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: instantiate the new terminal logger in main()

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume create a logger for the terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge PR #30155 into master

* refs/pull/30155/head:
mgr/MgrClient: send MMgrCommand for octopus+ mgrs
mon/MgrMap: include mgr_features in mgrmap
mgr: accept MMgrCommand[Reply] messages (in addition to MCommand[Reply])

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30162 from cyx1231st/wip-seastar-msgr-test-failover-crimson

crimson/test: v2 failover tests with crimson FailoverTestPeer

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/dev/crimson.rst: document CBT testing

Signed-off-by: Kefu Chai <kchai@redhat.com>

script/run-cbt.sh: check option correctly

* s/source_dir/source-dir/: as the option is named "--source-dir", not
  "--source_dir"
* cd $build_dir, before launching/stopping vstart. as vstart.sh is
  supposed to run from the build directory

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30111 from tspmelo/wip-nfs-read

mgr/dashboard: Set RO as the default access_type for RGW NFS exports

Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: progress: support rbd_support module async tasks (#29424)

mgr/dashboard: progress: support rbd_support module async tasks

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #30095 from rjfd/wip-fix-progress-qa

qa/mgr/progress: fix timeout error when waiting for osd in event

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: Wait for breadcrumb text is present in e2e tests

When the cluster is busy it might take some time until the tested component has loaded it's data and is rendered in the browser. Waiting for the breadcrumb text to be present might help to fix errors in the e2e tests.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #30078 from rhcs-dashboard/fix-41602-master

mgr/dashboard: disable pylint's --py3k flag

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #30126 from dvanders/dvanders_spec

ceph.spec.in: reserve more memory per build job

Reviewed-By: Nathan Cutler <nculter@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30258 from tchaikov/wip-41721

test/cls_rbd/test_cls_rbd: update TestClsRbd.sparsify

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/orchestrator: Sync status with reality

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

os/bluestore: fix space balancing overflow

When gifting, bluestore should not gift more than it actually has.
same for bluefs when reclaming.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30273 from neha-ojha/wip-documenting-ceph

doc/start/documenting-ceph.rst: make better doc recommendations

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc/start/documenting-ceph.rst: make better doc recommendations

I do not think "Editors" is relevant anymore.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #30187 into master

* refs/pull/30187/head:
script/backport-resolve-issue: better error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume create a new tox.ini for shell-based tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #30226 from neha-ojha/wip-41385-2

osd/MissingLoc.cc: do not rely on missing_loc_sources only

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge PR #30196 into master

* refs/pull/30196/head:
qa/tasks/ceph: restart: stop osd, mark down, then start

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30268 from cbodley/wip-41719

cls/rgw: fix bilog trim tests in ceph_test_cls_rgw

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge PR #30220 into master

* refs/pull/30220/head:
test: Allow fractional milliseconds to make test possible

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

cls/rgw: fix bilog trim tests in ceph_test_cls_rgw

the bucket index prepare step no longer writes a bilog entry, so the
tests were expecting more entries

Fixes: https://tracker.ceph.com/issues/41719
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29127 from bk201/wip-40337

mgr/dashboard: orchestrator integration initial works

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

test/cls_rbd/test_cls_rbd: update TestClsRbd.sparsify

after #30061, extents returned by sparse read will always be
an empty map as long as the extents to be read is empty or the extent(s)
in it are empty, even of the objectstorage does not support sparse read.

Fixes: https://tracker.ceph.com/issues/41721
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30115 from alfredodeza/wip-rm41378-2

ceph-volume: pre-install python-apt and its variants before test runs

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #30142 from votdev/issue_41648

mgr/dashboard: Handle always-on Ceph Manager modules correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30079 from uweigand/issue-41605-fixes

Fix big-endian handling

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30087 from egggHang/ec_benchmark

test/erasure-code: add exception handling to k & m

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29892 from Yan-waller/wip-yanj-fixreed_sol_r6_op

osd: fix the missing default value m=2 of reed_sol_r6_op in profile

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30125 from majianpeng/msg-protocolv1-reduece-swap

msg/async/ProtocolV1: avoid unnecessary bufferlist::swap.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

PendingReleaseNotes: note about changes in `ceph osd erasure-code-profile set`

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

osd: add sanity check for invalid m/w of liber8tion

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

ec/jerasure: save the default m=2 of reed_sol_r6_op in profile

save the default value of m to profile so that it could be dumped
in command 'ceph osd erasure-code-profile get xxx', which is more
useful and friendly to user.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #30109 from rzarzynski/wip-bug-25098-bluefs_layout_t

os/bluestore: verify disk layout of BlueFS

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30197 from bk201/wip-41675

mgr/orchestrator: fix refs property of progresses

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Handle always-on Ceph Manager modules correctly

Currently always-on modules are not marked as enabled in the WebUI and can be disabled. This PR will fix that.
Note, this PR will NOT implement code that will prevent a developer from trying to disable an always-on module through the REST API. The Mgr Python extension will throw an adequate exception.

This PR will also do:
* Remove old code fragments from a previous Mgr Module management UI that is obsolete now.
* Cleanup code in BaseMgrModule code.

Fixes: https://tracker.ceph.com/issues/41648
Signed-off-by: Volker Theile <vtheile@suse.com>

os/bluestore: add bluestore_bluefs_max_free option

This prevents bluefs from trying to acquire more space than
it actually needs, e.g., due to batch deletion of user data.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30199 from IlsooByun/cmake_ptyhon_var

build/ops: fix build fail related to PYTHON_EXECUTABLE variable

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29986 into master

* refs/pull/29986/head:
mgr/pg_autoscaler: use 'stored' for pool_logical_used

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #30139 into master

* refs/pull/30139/head:
mgr/DaemonServer: skip adjusting pgp_num when merging is in-progress

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30061 from xiexingguo/wip-sparse-read

os, osd: readv

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30059 from xiexingguo/wip-clean-vs-repair

osd/PG: fix _finish_recovery vs repair race

Reviewed-by: Neha Ojha <nojha@redhat.com>