Merge pull request #66674 from ceph/GHSA-p433-fp4g-pc2c

.github/workflows: Fix GHSA-p433-fp4g-pc2c

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>

Fix GHSA-p433-fp4g-pc2c

https://github.com/ceph/ceph/security/advisories/GHSA-p433-fp4g-pc2c

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #66687 from cbodley/wip-doc-rgw-admin-account-args

doc/rgw: POST /admin/account doesn't require id/name

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

Merge pull request #66681 from bluikko/wip-doc-rados-ops-health-checks-fixes3

doc/rados: Fix minor issues, improvements in health-checks.rst (3 of x)

Merge pull request #66675 from bluikko/wip-doc-rados-ops-health-checks-fixes2

doc/rados: Fix minor issues and improvements in health-checks.rst (2 of x)

Merge pull request #66662 from bluikko/wip-doc-rados-ops-health-checks-fixes

doc/rados: Fix minor issues and improvements in health-checks.rst

Merge pull request #66677 from bluikko/wip-doc-cephadm-services-osd-typo

doc/cephadm: Fix formatting errors and improvements in osd.rst

Merge pull request #66656 from ceph/apt-mirror-main

install-deps: Replace apt-mirror

doc/rgw: POST /admin/account doesn't require id/name

the RGWOp_Account_Create admin op doesn't require id or name

Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/rados: Fix minor issues, improvements in health-checks.rst (3 of x)

Also changed in monitoring.rst:
- Add label for link.
- Linkify mention of health checks.

Fix "bluestore bluefs" admin-socket commands after #38437.

Use ":confval:" instead of simple inline literal text for config values.

Use a privileged prompt for CLI command examples requiring privileges.

Trim section title syntax underline.

Use admonition instead of emphasis on the whole paragraph.

Attempt to improve language in a few sentences.

Use double backticks consistently for literals.

Linkify, adding a label in the destination if necessary:
- Muting health checks
- BlueStore migration

Capitalize Ceph, BlueStore, RADOS.

Add paragraph linking to more info on the "ceph df" command.

Use DB or DB device instead of literal db.

Add unordered list formatting to what was being rendered as single paragraph.

Use ":program:" for linking to ceph-bluestore-tool.

Change few last double spaces after full stop to a single space.

Use code-block paragraphs instead of inline literals where appropriate.

Add missing word "is".

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/cephadm: Fix formatting errors and improvements in osd.rst

Use console code-block instead of literal text to avoid highlighting.

Promptify few last CLI commands that were missing prompts.

Remove lone single-item unordered list formatting.

Add missing full stops.

Linkify mentions of other sections.

Use title case consistently in section titles.

Use :confval: consistently instead of literal text.

Use double backticks consistently.

Use "YAML" and "JSON" instead of literal .yaml and .json.

Fix two occurrences of double backticks missing one backtick that lead
to broken rendering.

Use *italic* RST syntax instead of rendered underscores.

Try to improve language in simple sentences.

Remove unnecessary * characters that were rendered in CLI example
commands.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/rados: Fix minor issues and improvements in health-checks.rst (2 of x)

Use :confval: instead of simple inline literal text for config values.

Use RST **emphasis** instead of CAPS.

Use a privileged prompt for CLI command examples requiring privileges.

Capitalize Manager when talking about the service/dameon.

Use double backticks for literals instead of single backticks.

Remove spaces around slash in "and/or".

Add a paragraph referring OSD troubleshooting in OSD free space checks.

Modify flag lists to have a consistent style.

Reorder sentences to make more sense.

One small language improvement attempt.

Expand {a,b} glob used in text.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/rados: Fix minor issues and improvements in health-checks.rst

Use title case consistently in section titles.

Wrap long lines over 80 characters.

Use "Ceph Manager" instead of "ceph-mgr".
Use "Ceph Monitor" or "Monitor" consistently instead of "monitor".

Use :confval: instead of simple inline literal text for config values.

Linkify, adding a label in the destination if necessary:
- Muting health checks
- Mon election strategy

Fix a typo.

Use RST *italic* instead of _rendered underscores_.

Use a privileged prompt for CLI command examples requiring privileges.

Use path including fsid to mon data consistently.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #66643 from bluikko/wip-doc-mgr-telemetry-collection-desc

doc/mgr: Fix collection desc in telemetry.rst

install-deps: Replace apt-mirror

apt-mirror.front.sepia.ceph.com has happened to always work because we set up CNAMEs to gitbuilder.ceph.com.

That host is making its way to a new home upstate (literally and figuratively) so we'll get rid of the front subdomain since it's publicly accessible anyway and add TLS while we're at it.

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #66576 from rhcs-dashboard/carbonized-multisite-realm-token-modal

mgr/dashboard: carbonized-multisite-export-realm-token-modal

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #66510 from rhcs-dashboard/manager-module-table-fix

mgr/dashboard: Fix table width expansion on manager module dropdown selection #74089

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>

Merge pull request #65324 from rishabh-d-dave/inode_t-fields

mds: compare all fields of inode_t

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #65496 from rishabh-d-dave/rmtree-enhancements

pybind,cephfs: enhancements for rmtree()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #65805 from rishabh-d-dave/qa-returns-unbound-var

qa/cephfs: don't return undefined variable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

doc/mgr: Fix collection desc in telemetry.rst

Use correct desc taken from source file for basic channel in example CLI
output.

Also turn off syntax highlighting for the CLI outputs so that some
keywords do not get colored.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #66554 from NitzanMordhai/wip-nitzan-improve-corpus-import-script

script/gen-corpus: cleanup and improve readability and performance

Merge pull request #66232 from NitzanMordhai/wip-nitzan-add-rocky-test-rados-workunit

qa/workunits: add Rocky Linux support to librados tests

Merge pull request #66630 from rhcs-dashboard/copy2-clipboard-fix

mgr/dashboard: emit success and error on copy2cliboard

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #66631 from rhcs-dashboard/validator-js-bump

mgr/dashboard: bump validator.js to 13.15.23

Reviewed-by: Naman Munet <nmunet@redhat.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #66641 from datdenkikniet/document-msgr2-crc

doc/dev: document exact CRC parameters for crc32c used in msgr2

doc/dev: document exact CRC parameters for crc32c used in msgr2

The description of the CRC used by the msgr2 protocol as CRC32-C
can be ambiguous. Document the actual parameters that make up
the CRC that is used.

Signed-off-by: Johannes Cornelis Draaijer <jcdra1@gmail.com>

Merge pull request #66068 from timqn22/balancer-info-log-reduction

mgr/balancer: reduce info-level logs

Merge pull request #66129 from Tom-Sollers/fixing_blaum_roth_for_ec_profiles

Mon: Add new health warning for non prime w+1 in blaum-roth EC profiles

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
Reviewed-by: Anthony D'Atri <anthony.datri@ibm.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #66277 from NitzanMordhai/wip-nitzan-lazyomap-wait-active-clean-test

test/lazy-omap-stats: overload get_output() to accept both lvalue and…

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #66204 from apataki/73795-cephmon-cleanupmap

ceph-mon: Improve efficiency of upmap cleanup in osdmaps for large clusters

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #66542 from aainscow/ec_trunctate_bug

osd: Fix fast EC truncate to whole stripe

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
Reviewed-by: Radosław Zarzyński <Radoslaw.Adam.Zarzynski@ibm.com>

Merge pull request #66579 from phlogistonjohn/jjm-smb-doc-fixes

doc: some small smb mgr module doc improvements

Merge pull request #66632 from bluikko/wip-doc-very-minor-fixes

doc: Fix minor formatting, typo etc issues

Merge pull request #66322 from phlogistonjohn/jjm-qa-workunits-smb

smb: qa workunit for smb

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>

Merge pull request #66497 from yielde/add-destractor-to-sahred_mutex_debug

src/common: Add a destructor to shared_mutex_debug

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>

Merge pull request #66617 from imran-imtiaz/dashboard

mgr/dashboard: add API endpoint to delete consistency group

Merge pull request #66441 from hasan4791/fix-install-deps

install-deps.sh: build without crimson when it is disabled

Reviewed-by: Kefu Chai <k.chai@proxmox.com>

script/gen-corpus: cleanup and improve readability and performance

- gen-corpus cleanup missed removing the temporary directory.
- improve it a bit for readability
- import.sh script was slow, improve performance by using less forks and
  batch processing

Signed-off-by: Nitzan Mordechai <nmordech@ibm.com>

Merge pull request #66602 from hasan4791/issue-74183

erasure-code: Enable isa for ppc64le target.

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
Reviewed-by: Jamie Pryde <jamiepry@uk.ibm.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>

doc: Fix minor formatting, typo etc issues

Remove formatting syntax from inside literal text in
cephadm/services/rgw.rst.
Use quotation marks similarly to other placement examples with only
parameter value quoted and not the whole parameter in
cephadm/services/rgw.rst.
Capitalize "YAML" in cephadm/services/rgw.rst.

Remove double space in the middle of a sentence in
rados/operations/erasure-code.rst.

Use double backticks consistently for default values in
radosgw/frontends.rst.
Capitalize "I/O", stylize as "OpenSSL" in radosgw/frontends.rst.

Fix typo "and object" to "an object" in radosgw/s3/bucketops.rst.

Stylize as "CentOS" in start/os-recommendations.rst.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #66507 from rhcs-dashboard/74046-QAT-configuration-through-RGW-service-on-Dashbard

mgr/dashboard: Adding QAT Compression dropdown on RGW Service form

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: bump validator.js to 13.15.23

Fixes: https://github.com/advisories/GHSA-vghf-hv5q-vc2g
Fixes: https://tracker.ceph.com/issues/74214
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: emit success and error on copy2cliboard

This is needed since the notification service we have right now is
tightly coupled with the dashboard so toast won't show up in the
applications where this is being consumed. So emitting an output which
the application can use to show relavant toasts.

Fixes: https://tracker.ceph.com/issues/74213
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #66629 from tokuhirom/wip-doc-orometheus-typo

doc/monitoring: fix typo s/Orometheus/Prometheus/

Merge pull request #66620 from bluikko/doc-notifications-formatting-radosgw

doc/radosgw: Update formatting in notifications.rst

Fixes: https://tracker.ceph.com/issues/74162
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

doc/monitoring: fix typo s/Orometheus/Prometheus/

Signed-off-by: Tokuhiro Matsuno <tokuhirom@gmail.com>

Merge pull request #66581 from idryomov/wip-74168

librbd: fix ExclusiveLock::accept_request() when !is_state_locked()

Reviewed-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

Merge pull request #66605 from ceph/fixes#2414677

Fix the prometheus module crash

doc/radosgw: Update formatting in notifications.rst

Change parameter list descriptions to start with capital case after
colon as discussed in #64768.

Remove a stray comma and end in full stop. Remove article from what
seems to be a plural. Hyphenation in "single-shard topics".

Change several same external links to use a single link definition
instead of repeating the destination URL each time.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #66621 from bluikko/wip-doc-start-documenting-formatting

doc/start: Fix stray RST syntax in documenting-ceph.rst

Merge pull request #66341 from cbodley/wip-73943

qa/rgw/upgrade: exclude ceph-osd-classic/crimson on squid and tentacle

Reviewed-by: Oguzhan Ozmen <oozmen@bloomberg.net>

doc/start: Fix stray RST syntax in documenting-ceph.rst

Remove stray single colons before literal text blocks.
Perhaps leftovers moving from :: to prompt blocks.

Remove double backticks used inside literal text.

Fix URL missing // in a link example and use HTTPS in it.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

qa/rgw/upgrade: don't randomize distros

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw/upgrade: exclude ceph-osd-classic/crimson on squid and tentacle

split packages for ceph-osd-classic and ceph-osd-crimson were added on
main, but don't exist on squid and tentacle. exclude these packages from
their install tasks

Fixes: https://tracker.ceph.com/issues/73943
Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: add API endpoint to delete consistency group

Signed-off-by: Imran Imtiaz <imran.imtiaz@uk.ibm.com>
Fixes: https://tracker.ceph.com/issues/74201
Add a dashboard API endpoint to delete a consistency group.

mgr/dashboard: Adding QAT Compression dropdown on RGW Service form
Fixes:https://tracker.ceph.com/issues/74046
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>

Merge pull request #66233 from benhanokh/dedup_ver

rgw/dedup: Add support for RGW versions.

Merge pull request #66188 from harriscr/ch_wip_qa_cbt_task

qa: Use a more pythonic way to install CBT python requirements

Fix the prometheus module crash

fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2414677
Signed-off-by: Anmol Babu <Anmol.Babu@ibm.com>

src/common: Add a destructor to shared_mutex_debug

Signed-off-by: Hui Tong <th937013596@gmail.com>

erasure-code: Enable isa for ppc64le target.

Fixes: https://tracker.ceph.com/issues/74183
Signed-off-by: T K Chandra Hasan <t.k.chandra.hasan@ibm.com>

Merge pull request #66588 from rhcs-dashboard/task_74176

mgr/dashboard: where the alert page tag colors are not being applied

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #66547 from Hezko/fix-nvme-cert-name

mgr/dashboard: fix nvme certificate name

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #66394 from elliotcourant/fix/debian-post-install-home

deb/cephadm: Don't assume a home directory is configured

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #66134 from rzarzynski/wip-pc-humanreadable-prio

common/perf_counters: don't use magics for add_u64_counter{,_histogram}

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

deb/cephadm: Don't assume a home directory is configured

cephadm.postinst can fail if cephadm was originally installed using a
version that didn't configure a home directory for the user at all.
Newer versions do configure a home directory (as either `/home/cephadm`
or `/var/lib/cephadm`) so if that is configured then nothing needs to be
done. But if the user was created with no home directory then one needs
to be added for the configure step to succeed.

Fixes: https://tracker.ceph.com/issues/72083
commit 90bc0369243077c2aaf67f0de2bab5810b217f4e added home directories
for new cephadm users created, but didn't add home directories to
cephadm users that already existed.

Signed-off-by: Elliot Courant <me@elliotcourant.dev>

Merge pull request #66574 from anoopcs9/whitespace-for-smb-login-names

mgr/smb: Allow whitespace in share login names

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

doc/mgr: fix some wording issues

Be more smarticle about the article.
Also banish a stray.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

doc/mgr: prefer the word interface over device

As per the suggestion in https://github.com/ceph/ceph/pull/66579#discussion_r2606834482

Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/dashboard: where the alert page tag colors are not being applied

Fixes: https://tracker.ceph.com/issues/74176
Signed-off-by: pujashahu <pshahu@redhat.com>

Merge pull request #66516 from rhcs-dashboard/74078-host-labels-removed-fix

mgr/dashboard :  Fixed labels issue

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Fixes: https://tracker.ceph.com/issues/74089
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

Merge pull request #66528 from tchaikov/wip-debian-deps-c-v

debian/control: record python3-packaging dependency for ceph-volume

mgr/smb: Allow whitespace in share login names

Whitespaces are perfectly valid in Windows AD user or group names.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>

Merge pull request #66249 from thotz/adminops-restore-op

rgw/adminops: support for adding restore operation

Merge pull request #54435 from dparmar18/libcephfs-nonblocking-io-testcases

src/test: add libcephfs tests for async(nonblocking) calls

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #66404 from tchaikov/wip-bwc-with-more-branch-names

script: sanitize git branch names for OCI tag compliance

Reviewed-by: John Mulligan <jmulligan@redhat.com>

qa/tasks: disable ulmits+coverage wrapper scripts for smb workunit

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/orch/cephadm/smb: add new smb.workunit task (non-AD)

Add some instances of the new smb workunit wrapper to a couple of
non-AD tests that require the user name and password to be explicitly
provided.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/orch/cephadm/smb: add new smb.workunit tasks to some AD tests

These tests will automatically use a user account from the known
AD environment.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/tasks: add a workunit wrapper function to smb.py

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/workunits/smb: add test_simple_smb.py very simple smb tests

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/workunits/smb: add conftest.py for pytest fixtures, etc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/workunits/smb: add smbutil.py a testing utility module

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/workunits/smb: add a stub pytest config

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/workunits/smb: add a stub workunit script for smb testing

Signed-off-by: John Mulligan <jmulligan@redhat.com>

librbd: fix ExclusiveLock::accept_request() when !is_state_locked()

To accept an async request, two conditions must be met: a) exclusive
lock must be a firm STATE_LOCKED state and b) async requests shouldn't
be blocked or if they are blocked there should be an exception in place
for a given request_type.  If a) is met but b) isn't, ret_val is set
to m_request_blocked_ret_val, as expected -- the reason for denying
the request is that async requests are blocked.  However, if a) isn't
met, ret_val also gets set to m_request_blocked_ret_val.  This is wrong
because the reason for denying the request in this case isn't that
async requests are blocked (they may or may not be) but a much heavier
circumstance of exclusive lock being in a transient state or not held
at all.

In such scenarios, whether async requests are blocked or not isn't
relevant and ExclusiveLock::accept_request() behaving otherwise can
lead to bogus "duplicate lock owners detected" errors getting raised
during an attempt to handle any maintenance operation notification in
ImageWatcher::handle_operation_request().  This error isn't considered
retryable so the entire operation that needed the exclusive lock would
be spuriously failed with EINVAL.

Fixes: https://tracker.ceph.com/issues/74168
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

librbd: add ExclusiveLock::accept_request() overload

Make ret_val out parameter required for the existing method and
introduce an overload taking just request_type to compensate.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

doc/mgr: add examples to ceph smb cluster create and share create

Signed-off-by: John Mulligan <jmulligan@redhat.com>

doc/mgr: various fixes and improvements for smb mgr module doc

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #66533 from imran-imtiaz/dashboard

mgr/dashboard: add API endpoint for consistency group name update

mgr/dashboard :  Fixed labels issue
fixes : https://tracker.ceph.com/issues/74078
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

Merge pull request #66506 from Matan-B/wip-matanb-crimson-seastore-extent-race-v3

crimson/os/seastore/cache: Verify crc prior to complete_io

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

rgw/radosgw-admin.cc: use new apis for admin cli command

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

rgw/adminops: support for adding restore operation

Adding support for restore cli in admin ops, added existing clis list and statusw

Fixes: https://tracker.ceph.com/issues/70931
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

Merge pull request #65689 from VallariAg/wip-listener-mon-cmd

mon: Add command "nvme-gw listeners"

minor changes requested by reviewer

Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>

rgw/dedup: Add support for RGW versions.
Dedup code will scan all instances of versioned objects and include them
in the dedup process

Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>