qa/rgw: crypt suite uses clusters/fixed-1.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: abstract out barbican under 2-kms/

Signed-off-by: Casey Bodley <cbodley@redhat.com>

tasks/barbican: Added task to deploy and configure barbican

Fixed keystone, rgw and s3tests tasks to work with it.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

Merge PR #30185 into master

* refs/pull/30185/head:
mgr/volumes: drop unnecessary size

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29037 from zhangsw/rgw-reshard-numshards

rgw: shard number must be non-negative when resharding the bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29623 from majianpeng/bluestore-combine-deferred_finisher-and-finisher

os/bluestore: Merge deferred_finisher and finisher.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #30342 from ifed01/wip-ifed-fix-blues-alerts

mon/pgmap: fix bluestore alerts output.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30372 from tchaikov/wip-crimson-single-pg-peering

crimson/osd: use single-pg peering ops

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #24794 into master

* refs/pull/24794/head:
mds: move BatchOp to separate translation unit
mds: use auto ref during iteration
mds: answering all pending getattr/lookups targeting the same inode in one go

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28702 into master

* refs/pull/28702/head:
qa: update json format from session listing
mds: recall caps from quiescent sessions
mds: use Session::dump method uniformly
mds: use auto to deduce iterator type
mds: simplify method definition
mds: remove useless debug message
mds: use const get_request_count
mds: use session_info_t socket inst for addr
mds: refactor session lookup
mds: add explicit trim flag
mds: alphabetize tracked config keys
common: provide method to get half-life
common: correct variable name

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #29005 into master

* refs/pull/29005/head:
client: nfs-ganesha with cephfs client, removing dir reports not empty

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #29594 into master

* refs/pull/29594/head:
mds: reject sessionless messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29752 into master

* refs/pull/29752/head:
mds: make MDSIOContextBase delete itself when shutting down

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30361 into master

* refs/pull/30361/head:
qa/tasks/mon_thrash: sync force requires some force flags

Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge pull request #30357 from zhangsw/rgw-cleanup-bilog-status

rgw: add missing bilog status to help info.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29081 from wintamute/patch-1

Documentation: Add missing ceph-volume lvm batch argument to ceph-volume.rst

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PR #30352 into master

* refs/pull/30352/head:
mon: disable min pg per osd warning

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30284 from Damenly/missing_cmath

tests: add missing header cmath to test/mon/test_mon_memory_target.cc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: Add missing ceph-volume lvm batch argument to ceph-volume.rst

The optional argument '--osds-per-device' is missing. Since this is quite helpful when creating NVMe based OSDs, it should be documented.

Signed-off-by: Andreas Krebs <wintamute@gmail.com>

mgr/dashboard: Increase the global value of jasmine timeout (#30359)

mgr/dashboard: Increase the global value of jasmine timeout

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #29761 from wjwithagen/wjw-fix-venv-python-version

test/venv: do not hardwire to py2.7 for tox tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix calculation of PG Status percentage (#30343)

mgr/dashboard: Fix calculation of PG Status percentage

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

crimson/osd: require_osd_release >= octopus to boot

we assume that crimson will GA after octopus+2, so no need to be
compatible with pre-octopus releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use single-pg peering ops

classic OSD started to use single-pg peering ops since ce05c172, and it
switched over to these ops since octopus. and it's assumed that
crimson-osd won't be GA until octopus+2 release, so it's not needed to
support pre-octopus releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: pass octopus to PeeringCtx's constructor

to accommodate the change of 77ecc327

classic OSD started to use single-pg peering ops since ce05c172, and it
switched over to these ops since octopus. and it's assumed that
crimson-osd won't be GA until octopus+2 release, so it's not needed to
support pre-octopus releases. hence "octopus" is passed to PeeringCtx's
constructor.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30285 from IlsooByun/slow_op_report

mgr: do not reset reported if a new metric is not collected

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa: update json format from session listing

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30112 from liewegas/wip-autoscaler-on

common: default pg_autoscale_mode=on for new pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

mds: recall caps from quiescent sessions

This introduces two new config options [1,2] that dictate when a session
is considered quiescent by the MDS. (Options are documented fully in
options.cc.) When a session is quiescent, the MDS will preemptively
recall caps to reduce the outstanding capabilities which optimizes for
reducing work during failover.

[1] mds_session_cache_liveness_magnitude
[2] mds_session_cache_liveness_decay_rate

Fixes: https://tracker.ceph.com/issues/22446
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use Session::dump method uniformly

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use auto to deduce iterator type

This prevents accidental type mismatches.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: simplify method definition

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: remove useless debug message

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use const get_request_count

So it can be used from a const dump method. Make the elist mutable
instead.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use session_info_t socket inst for addr

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: refactor session lookup

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: add explicit trim flag

So that the intent of the caller is clear to trim due to the cache being
too full.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: alphabetize tracked config keys

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

common: provide method to get half-life

Print the half-life in Formatter output as it is human understandable
and the value used when configuring the counters.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

common: correct variable name

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29574 into master

* refs/pull/29574/head:
mds: Reorganize class members in FSMapUser header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29883 into master

* refs/pull/29883/head:
mds: Reorganize class members in InoTable header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30090 into master

* refs/pull/30090/head:
mds: Reorganize class members in Anchor header

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #30143 into master

* refs/pull/30143/head:
mds: Reorganize class members in LocalLock header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30205 into master

* refs/pull/30205/head:
mds: Reorganize class members in LogEvent header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #30191 into master

* refs/pull/30191/head:
cls/hello: disable write returning data clobbering demo
messages/MOSDOpReply: assert no write result payloads
osd/PrimaryLogPG: assert write result == 0
osd/PrimaryLogPG: pass TMAPGET errors up
osd/PrimaryLogPG: set result=0 for successful writes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29792 from sebastian-philipp/orchestrator-api-replace

mgr/orchestrator: Document OSD replacement

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

common: default pg_autoscale_mode=on for new pools

Signed-off-by: Sage Weil <sage@redhat.com>

cls/hello: disable write returning data clobbering demo

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29820 into master

* refs/pull/29820/head:
osd/PeeringState: log_weirdness during peering
osd/PeeringState: send new message types
osd/PeeringState: give require_osd_release to BufferedRecoveryMessages
osd: add new notify, query, and info messages
osd/PeeringState: use send_info() for replica activation ack
osd/PeeringState: remove old info_map member
osd/PeeringState: send infos via message_map (not info_map)
osd/PeeringState: remove old query_map member
osd/PeeringState: send queries via message_map (not query_map)
osd/PeeringState: use return value from discover_all_missing
osd/PeeringState: give PeeringCtxWrapper a BufferedRecoveryMessages ref
osd/PeeringState: send notifies via message_map (not notify_list)
osd/PeeringState: add message_map to PeeringCtx/BufferedRecoveryMessages
osd: dispatch_context inside PG lock

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #30146 from ivancich/wip-fix-min-unordered-bkt-list

rgw: fix minimum of unordered bucket listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>

mgr/dashboard: Display the duration of each e2e test

Now we are able to see which tests are taking
too much time and may need improvements.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Increase the global value of jasmine timeout

Some tests are taking longer than the default jasmine timeout to finish
and are causing a failure.

This commit will increase the timeout to 5 minutes,
removing the need to increase it for each failing test.

Signed-off-by: Tiago Melo <tmelo@suse.com>

qa/tasks/mon_thrash: sync force requires some force flags

AFAICS this has been the case for basically forever.  Not sure why/how
the mon_thrash task hasn't had a problem with that?

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #30106 from smithfarm/wip-more-cbs

script/ceph-backport.sh: make the script idempotent

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

mgr/dashboard: key-value-table doesn't render booleans (#30203)

 mgr/dashboard: key-value-table doesn't render booleans

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Fix error message not displayed in pool form (#30137)

mgr/dashboard: Fix error message not displayed in pool form

Reviewed-by: Volker Theile <vtheile@suse.com>

rgw: add missing bilog status to help info.

Signed-off-by: zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

Merge pull request #28936 from tchaikov/wip-unaligned-access

cmake: check for unaligned access

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

mds: make MDSIOContextBase delete itself when shutting down

Fixes: https://tracker.ceph.com/issues/41346
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

Merge pull request #28939 from trociny/wip-37529

rbd-mirror: add namespace support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mon: disable min pg per osd warning

Now that the pg_autoscaler is on by default, it is "normal" (and okay) to
have a small number of PGs in the cluster if the overall cluster usage is
also low.  This setting just results in a health warning out of the box
when you create a pool and haven't written any data yet.

Fixes: https://tracker.ceph.com/issues/41735
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #30192 from dzafman/wip-41669

Make dumping of reservation info congruent between scrub and recovery

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30347 from neha-ojha/wip-add-backend

qa/suites/rados/perf: test min recommended osd_memory_target

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #30319 from zhangsw/rgw-fix-zone-rm

rgw: fix incorrect "radosgw-admin zonegroup rm" info.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

qa/suites/rados/perf: test min recommended osd_memory_target

Add bluestore-basic-min-osd-mem-target.yaml to objectstore.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #30346 into master

* refs/pull/30346/head:
script: raw_input was renamed to input in py3

Reviewed-by: Sage Weil <sage@redhat.com>

script: raw_input was renamed to input in py3

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30014 into master

* refs/pull/30014/head:
doc: update CephFS overview in introductory page
doc: filesystem to file system

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #30221 from cbodley/wip-rgw-dns-name-iends-with

rgw: fix dns name comparison for virtual hosting

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #30310 from rzarzynski/wip-crimson-op_effects

crimson/osd: extend OpsExecuter to carry about op effects.

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix calculation of PG Status percentage

We were reading a wrong value for the total of PGs.

Fixes: https://tracker.ceph.com/issues/41536
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: E2E enhancements (#30017)

mgr/dashboard: E2E enhancements

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mon/pgmap: fix bluestore alerts output.

Lost OSD count for most of them.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29606 from sseshasa/bluestoreCfgObserver

os/bluestore: Add config observer for osd memory specific options.

Merge pull request #29793 from sebastian-philipp/ansible-default-realm-zonegroup

mgr/ansible: Change default realm and zonegroup

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #30321 from tchaikov/wip-run-cbt.sh

script/run-cbt.sh: always use python3

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Update protractor dependencies

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enable waitforAngular in E2E

Convert waitForTableData into a method,
using it as a decorator was causing problems.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add protractor-fast-fail plugin

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add click call inside waitClickable

All waitClickable calls were being followed by a click.
This will remove the need for the click call.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix error in pool list component

Sometimes the pools list result comes without the tiers field.
This was causing an error in the code.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix random failure in images e2e test

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove duplicated method

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove helper.po.ts

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove E2E warning

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Extract delete method into page-helper

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Use suites in protractor

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Split user-mgmt e2e files

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Move some e2e into new "ui" folder

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve clickCheckbox with some waits

Signed-off-by: Tiago Melo <tmelo@suse.com>

script/ceph-backport.sh: help the user set up the script

This commit adds several options:

    --setup
    --setup-advice
    --usage-advice
    --troubleshooting-advice

It replaces the "comment block at the beginning of the script" which was lamely
masquerading as documentation.

It also implements heuristic deduction of GitHub remotes.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

script/ceph-backport.sh: check Backport ticket assignee

Don't work on tickets assigned to someone else.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Improve images e2e test (#30212)

mgr/dashboard: Improve images e2e test

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

mgr/dashboard: Wait for breadcrumb text is present in e2e tests (#30267)

mgr/dashboard: Wait for breadcrumb text is present in e2e tests

mgr/dashboard: Rendering datatable causes Angular runtime error (#30209)

mgr/dashboard: Rendering datatable causes Angular runtime error

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

script/run-cbt.sh: always use python3

run-cbt.sh will be used by the "ceph-perf-pull-requests" jenkins job,
which will prepare the runtime dependencies by installing python3
modules.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30318 from tchaikov/wip-run-make.sh

script/run-make.sh: do not pass cmake options twice

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: fix incorrect "radosgw-admin zonegroup rm" info.

Signed-off-by: zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

script/run-make.sh: do not pass cmake options twice

Signed-off-by: Kefu Chai <kchai@redhat.com>