rgw/test: enhanced java s3-tests change setting of JAVA_HOME

Under Centos 9 the Java 8 version is recognized by the substring
"java-1.8" rather than "java-8". So the grep has been modified to
accept either.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit a49d4446e4d84b28273b460b85a193011a9c4ed8)

Merge PR #68025 into main

* refs/pull/68025/head:
mailmap: Add Karthik U S

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #68029 from djgalloway/rook-e2e

ci: Fixes for ceph-rook-e2e job

Merge pull request #68208 from ivancich/wip-fix-s3tests-java-home

rgw: java s3-tests change setting of JAVA_HOME

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #68004 into main

* refs/pull/68004/head:
doc: add notable changes for v20.2.1
doc: add 20.2.1 Release Notes

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

doc: add notable changes for v20.2.1

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc: add 20.2.1 Release Notes

Fixes: https://tracker.ceph.com/issues/74838
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #68199 from rhcs-dashboard/fix-75860

mgr/dashboard: Fix tags in subvolume list and subvolume groups list

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>

Merge pull request #68217 from tchaikov/wip-cmake-crimson-gtest

test/crimson/seastore: add missing crimson::gtest to link libraries

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #68200 from tchaikov/crimson-plain-load-obc

crimson/osd: make load_obc(obc, md_ref) return void

Reviewed-by: Matan Breizman<mbreizma@redhat.com>

test/crimson/seastore: add missing crimson::gtest to link libraries

unittest-transaction-manager, unittest-omap-manager,
unittest-btree-lba-manager, and unittest-seastore all include
gtest_seastar.cc but were not explicitly linking against crimson::gtest.
This worked previously because gtest symbols were pulled in transitively,
but with gcc-toolset-13 and LTO the transitive dependency is no longer
satisfied, producing undefined reference errors for testing::Message,
testing::Test, testing::AssertionSuccess, etc.

Fix by linking these tests against crimson::gtest

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

Merge pull request #68009 from Hezko/support-no-pool-in-spec

mgr/cephadm: support no pool value in spec for nvmeof gateway

Merge pull request #68201 from ronen-fr/wip-rf-zbd-crimson

crimson/seastore: avoid using 'final' to mean 'override'

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #64249 from zhuwei127/missing-return

src/common: fix missing return

Reviewed-by: Kefu Chai <k.chai@proxmox.com>

crimson/osd: make load_obc(obc, md_ref) return void

load_obc() taking an already-resolved loaded_object_md_t::ref is
synchronous, because it just populates obc state, it does yield.
Returning an errorated future was unnecessary and caused a
-Wunused-result warning at its only call site:
ECRecoveryBackend::maybe_load_obc().

In this change, we change it to return void and deduplicate the OBC
population logic: the private async overload (taking future<md_ref>)
now validates ssc and returns object_corrupted on failure.

This silences the warning, and simpler this way. The async error
propagation is preserved.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

Merge pull request #68212 from ronen-fr/wip-rf-testobh-crimson

crimson/test: rm 'assert's in test_object_data_handlers.cc

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

crimson/seastore: avoid using 'final' to mean 'override'

Note: when using 'final', the compiler *does not*
check that the function is actually overriding a base
class function.

Using only 'override' wherever the class itself is
already marked 'final', 'override final' otherwise.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

crimson/seastore: use correct signature for start()

... in NVMeBlockDevice and ZBDSegmentManager

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #65425 from xxhdx1985126/wip-seastore-aggregate-read

crimson/os/seastore/object_data_handler: make sure continuous object data blocks are loaded with a single io request

Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/test: rm 'assert's in test_object_data_handlers.cc

As these 'assert's triggered 'unused variable' warnings.
Replaced with equivalent ASSERT_EQ() macros.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #67096 from kshtsk/wip-misc-cleanup

qa/standalone/misc: make less noise for asok calculation

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>

qa/standalone/misc: make less noise for asok calculation

The $() notation not only calls the function, it also creates subshell,
which is a separate process. Additionally in debug mode all the content
of the function get_asok_path is printed out in the logs each time
whenever it is called, for example:

    ...stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr:  get_asok_path
    ...stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path:  local name=
    ...stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path:  '[' -n '' ']'
    ...stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path:  get_asok_dir
    ...stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir:  '[' -n '' ']'
    ...stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir:  echo /tmp/ceph-asok.48301
    ...stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path:  echo '/tmp/ceph-asok.48301/$cluster-$name.asok'

Instead of calling get_asok_path each time we need to define osd.0,
etc. asok file path, we just predefine corresponding variables.
It not only avoids extra resource usage, but also removes a lot of
noise from the logs.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>

rgw: java s3-tests change setting of JAVA_HOME

Previously s3tests_java.py set JAVA_HOME using the `alternatives`
command. That had issues in that `alternatives` is not present on all
Ubuntu systems, and some installations of Java don't update
alternatives. So instead we look for a "java-8" jvm in /usr/lib/jvm/
and set JAVA_HOME to the first one we find.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

mgr/dashboard: Fix tags in subvolume list and subvolume groups list

Fixes: https://tracker.ceph.com/issues/75860
Signed-off-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #67852 from bluikko/wip-doc-start-improve

doc: Improve docs in start

Merge pull request #67824 from bluikko/wip-doc-rados-spelling2

doc/rados: Fix spelling errors (2 of 3)

Merge pull request #67813 from bluikko/wip-doc-rados-spelling

doc/rados: Fix spelling errors (1 of 3)

Merge pull request #67843 from rhcs-dashboard/fix-sc-duplicate-name-issue

mgr/dashboard : Restrict create storage class with existing name

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #68072 from syedali237/unit-tests

mgr/dashboard (test) : refactored tests and resolved TODOs

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #68146 from vumrao/vikhyat-governance-update-email-id

doc/governance: Update governance.rst to change the email id

Merge pull request #67679 from SHASHI9705/fix-notification-icon-alignment

dashboard: fix notification icon alignment with timestamp in notification panel

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #67934 from rhcs-dashboard/preserve-id-fix

mgr/dashboard: fix the missing injectors for child formgroup tpl

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #68129 from Aryan-coder06/dashboard-75762-notification-relative

mgr/dashboard: keep notification tests in the past

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #68173 from Annmool/wip-vstart-extblkdev-fix

vstart: disable extblkdev plugins for file-backed OSDs

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #68053 from rhcs-dashboard/dashboard-freeze-issue

mgr/dashboard: telemetry banner causes UI freeze on initial load

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Puja Shahu <pshahu@redhat.com>

test/crimson/seastore/test_object_data_handler: always initialize
the object's known_contents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #68168 from batrick/pr67961-followup

qa: undo supported-random-distros mishap

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

vstart: disable extblkdev plugins for file-backed OSDs

Signed-off-by: Annmool <aydv.267@gmail.com>

Merge pull request #66678 from bugwz/support_docker_build

Support build image with docker

Change format

Signed-off-by: bugwz <xbugwz@gmail.com>

Support build image with docker

Signed-off-by: bugwz <xbugwz@gmail.com>

qa: add back centos9.stream

When graph walks is complete, it will actually be possible link to
supported without repeating the links. The matrix implementation doesn't
support this.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Revert "qa/distros: link random distro to supported"

This reverts commit e00e327d5fa08e38b833e6b4fc32483b3ca2340d.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

qa: remove file from silent conflict

Added back by 2755bec58f6b659fa46c8d717c895ee4fa4270d0.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #67961 into main

* refs/pull/67961/head:
qa/distros: link random distro to supported
qa/distros: add .qa links
qa: remove special chars from supported-random-distro
qa: remove redundant single-container-host
qa/distros/container-hosts: remove centos8
qa: remove distros/podman
qa: switch from distros/podman to supported-container-hosts

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #67957 into main

* refs/pull/67957/head:
qa/distros: add centos 9 stream back to supported distros

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #68153 from gbregman/main

mgr/cephadm: don't try to map /etc/kmip if it doesn't exist

Merge pull request #68002 from ronen-fr/wip-rf-pgscrubber-cln26

osd/scrub: minor cleanups

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #68013 from badone/wip-tracker-75391-kernel_untar_build-rocky-failure

qa/workunits: Add updated kernel archive URL

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

mgr/cephadm: don't try to map /etc/kmip if it doesn't exist

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

mgr/dashboard : Restrict create storage class with existing name
fixes :
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

Merge pull request #68115 from bluikko/wip-doc-ceph-volume-spelling

doc/ceph-volume: Fix spelling etc errors

Merge pull request #68130 from tchaikov/wip-crimson-main-includes

crimson: clean ups in main_config_bootstrap_helpers.cc

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd: remove unnecessary named string for --smp value

The local variable `smp` is used only in the two immediately following
statements. Inline the fmt::format() call into emplace_back() and pass
reactor_num directly to the logger.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

crimson/osd: make early_config_t::to_ptr_vector private

The helper is an implementation detail of get_early_args() and
get_ceph_args(). Making it private prevents callers from inadvertently
holding the returned const char* pointers past the lifetime of the
input vector. Also fix the truncated doc-comment ("must not outlive in").

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

crimson/osd: remove duplicate include of main_config_bootstrap_helpers.h

The header was included twice in main_config_bootstrap_helpers.cc.
The second inclusion is redundant due to #pragma once.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

doc/governance: Update governance.rst to change the email id

Updating my redhat.com email id to ibm.com email id.

Signed-off-by: Vikhyat Umrao <vikhyat@ibm.com>

Merge pull request #67553 from JonBailey1993/subobject-linkage-fix

test: Fix gcc warnings about subobject-linkage in ceph_test_rados_io_sequence

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>
Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>

Merge pull request #67498 from jamiepryde/73246-jerasure-cauchy-init-mutex

erasure-code/jerasure: Fix thread safety in plugin init

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>

Merge pull request #67737 from phlogistonjohn/jjm-cephadm-choose-action

cephadm: Refactor _check_daemons to allow service subclasses to choose the next action

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #68120 from bluikko/wip-doc-hardware-monitoring-improvements

doc/hardware-monitoring: improve index.rst

Merge pull request #68124 from rhcs-dashboard/fix-subsystem-loading-page

mgr/dashboard: fix-subsystem-loading-page-issue

Merge pull request #67925 from shraddhaag/wip-shraddhaag-use-pgrecoverydelete

crimson/osd/pg_recovery: call MOSDPGRecoveryDelete instead of MOSDPGBackfillRemove

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: keep notification tests in the past

Signed-off-by: Aryan-coder06 <aryaniiitian06@gmail.com>

mgr/dashboard: fix-subsystem-loading-page-issue

Fixes: https://tracker.ceph.com/issues/75813
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

Merge pull request #64894 from ibm-s390-cloud/main-s390x-smcd

msg/async: SMC on s390x

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #68093 from tchaikov/wip-crimson-comments

crimson: remove redundant comments and improve comment so it's more accurate

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd/pg_recovery: call MOSDPGRecoveryDelete instead of MOSDPGBackfillRemove

This commit fixes the abort in Recovered::Recovered.

There is a race to acquire the OBC lock between backfill and
client delete for the same object.

When the lock is acquired first by the backfill, the object is
recovered first, and then deleted by the client delete request.
When recovering the object, the corresponding peer_missing entry
is cleared and we are able to transition to Recovered state
successfully.

When the lock is acquired first by client delete request, the
object is deleted. Then backfill tries to recover the object,
finds it deleted and exists early. The stale peer_missing
entry is not cleared. In Recovered::Recovered, needs_recovery()
sees this stale peer_missing entry and calls abort.

The issue is fixed by sending MOSDPGRecoveryDelete from the client
path to peers and waiting for MOSDPGRecoveryDeleteReply in
recover_object.

Fixes: https://tracker.ceph.com/issues/70501
Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>

doc/hardware-monitoring: improve index.rst

Try to improve language. Fix a typo.
Fix CLI example commands rendered as comments.
Fix capitalization and hyphenation.
Try to follow the usual style.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67955 from guits/issue-12737

orch/cephadm: fix osd.default creation

mgr/dashboard: telemetry banner causes UI freeze on initial load

Fixes: https://tracker.ceph.com/issues/75761
Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #67654 from bst2002git/bst2002cephfsdashboard-git-patch-1

mgr/dashboard: Grafana dashboard/Filesystem Overview: "matching labels must be unique on one side"

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #66776 from rhcs-dashboard/carbonize-osd-flags-modal

mgr/dashboard: carbonize-osd-flags-modal

Reviewed-by: pujaoshahu <pshahu@redhat.com>

doc/ceph-volume: Fix spelling etc errors

Low-hanging spelling, punctuation, and capitalization errors.
Ignore style and other more complex issues.
Use angle brackets consistently for value placeholders.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67929 from myoungwon/wip-fix74982

crimson/osd: add warning log for stray pgmeta-like objects

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #67740 from xxhdx1985126/wip-75333

crimson/common/interruptible_future: fix interruption leakage in green_get/get/yield/maybe_yield

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #67866 from xxhdx1985126/wip-75574

crimson/osd/pg_recovery: call on_global_recover on the "recover deleted" object when it's not missing on the primary too

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #68049 from rhcs-dashboard/fix-padding

mgr/dashboard: Fix padding of overview page

Reviewed-by: Nizamudeen A <nia@redhat.com>

This change introduces the shared memory communication (SMC-D) for the cluster network.
SMC-D is faster than ethernet in IBM Z LPARs and/or VMs (zVM or KVM).

Fixes: https://tracker.ceph.com/issues/66702
Signed-off-by: Aliaksei Makarau <aliaksei.makarau@ibm.com>

Merge pull request #67991 from rhcs-dashboard/nvmeof-gw

mgr/dashboard: Allow creating nvmeof gateway group via new flow

Reviewed-by: pujaoshahu <pshahu@redhat.com>

found duplicate series for the match group {fs_id="-1"}

when 1 MDS active and 2 MDS standby (on 3Node-Cluster)
found duplicate series for the match group {fs_id="-1"} on the right hand-side of the operation
many-to-many matching not allowed: matching labels must be unique on one side

Fixes: https://tracker.ceph.com/issues/75754
Signed-off-by: bst2002git <bst2002@ios.dhs.org>

Merge pull request #68100 from bluikko/wip-doc-sphix-warnings-202604

doc: Fix Sphinx warnings

Merge pull request #68110 from aainscow/tentacle-exclude

qa: Exclude missing ceph-osd-classic and ceph-osd-crimson from tentacle installs.

Reviewed-by: Shraddha Agrawal <shraagra@in.ibm.com>

Merge pull request #67766 from VallariAg/wip-nvmeof-auto-listener-cli

mgr/dashboard: Add 'network_mask' to nvmeof cli

qa: Exclude missing ceph-osd-classic and ceph-osd-crimson from tentacle installs.

The upgrade from tentacle builds were failing because the system was attempting
to install packages whose names were changed in Umbrella.

We use the same workaround previously used for squid.

Fixes: https://tracker.ceph.com/issues/75730
Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

Merge pull request #68104 from shraddhaag/wip-shraddhaag-fix-upgrades

qa/tasks/cephadm.py: only pass --objectstore when not bluestore

mgr/dashboard: Add 'network_mask' to nvmeof cli

This commit add the following to nvmeof cli:
0. Add new param `--network-mask` to 'subsystem add' cmd
   It's a list parameter so we can pass multiple netmask by
   `subsystem add --network-mask <subnet1> --network-mask <subnet2>`
1. Add new cli `subsystem add_network --network-mask <subnet>`
2. Add new cli `subsystem del_network --network-mask <subnet>`
3. Add column 'network_mask' to `subsystem list` output
4. Add column 'manual' to `listener list` output

Fixes: https://tracker.ceph.com/issues/75348
Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>

Merge pull request #67996 from guits/fix-cv-nvme-vendor

ceph-volume: fix vendor detection for nvme devices

Merge pull request #67624 from ujjawal-ibm/filter-devices

ceph-volume: skip cdrom devices in inverntory

Merge pull request #67891 from guits/tracker_75615

ceph-volume: fix inventory without /dev/vg/lv (slashed paths)

qa/tasks/cephadm.py: only pass --objectstore when not bluestore

This commit ensure that we only pass --objectstore argument to
cephadm's add/apply OSD command only when the value is not the
default value, bluestore.

This is done to ensure older ceph releases, like Squid and Tentacle
do not fail, as --objectstore argument was only added in Umbrella.

Fixes: https://tracker.ceph.com/issues/75731
Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>

crimson/os/seastore/object_data_handler: fiemap return laddr continuous
mappings as a single region

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

test/crimson/seastore/test_object_data_handler: add the UT case for
aggregate read

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/object_data_handler: read consecutive extents in
batches

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/transaction_manager: add read_pins

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cache: add "read_extents_maybe_partial"

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/device: add readv

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

doc: Fix Sphinx warnings

Fix indentation and section title underline.
Fixes six Sphinx warnings.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #68066 from rhcs-dashboard/ui-empty-subvol-info-fix

mgr/dashboard: fix subvolume group corruption from smb share form

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>

Merge pull request #66920 from kamoltat/wip-ksirivad-osd-df

src/osd/OSDMap.cc: Display SUM of PGs in ceph osd df