erasure-code: Build the avx512 versions of functions in ISA-L

Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com>

Merge pull request #58961 from NitzanMordhai/wip-nitzan-dencoder-test-forward-incompat-fix

workunit/dencoder: dencoder test forward incompat fix

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #58594 from jamiepryde/isa-xor-raid

erasure-code/isa: Use isa/raid's xor_gen() instead of the region_xor(…

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #59239 from yuvalif/wip-yuval-67513

Reviewed-By: Casey Bodley <cbodley@ibm.com>
test/rgw/notification: use real ip address instead of localhost

based on that comment:
https://tracker.ceph.com/issues/67206#note-6
the address used by the endpoint is taken as the real IP address of the
host where the test script is running and not localhost.

we also changed the rabbitmq-server conf to allow "guest"
user to connect over non localhost address

Fixes: https://tracker.ceph.com/issues/67206
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #59200 from ifed01/wip-ifed-fix-store-test-col-ref

test/store_test: fix assertions due to unclosed collection refs.

Reviewd-by: Pere Diaz Bou <pere-altea@hotmail.com>

Merge pull request #59256 from zdover23/wip-doc-2024-08-17-cephfs-ceph-dokan-mount-point

doc/cephfs: s/mountpoint/mount point/

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #58995 from rhcs-dashboard/fix-66844-main

qa/mgr/dashboard: fix test race condition

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59212 from cyx1231st/wip-seastore-more-reports

crimson/os/seastore/cache: report lru usage/in/out with trans and extent type

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #59290 from anthonyeleven/mountpoint

doc: Harmonize 'mountpoint'

doc: Harmonize 'mountpoint'

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #59257 from zdover23/wip-doc-2024-08-17-cephfs-mount-point

doc/cephfs: s/mountpoint/mount point/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/cephfs: s/mountpoint/mount point/

Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).

cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486 in
which page 345 of The IBM Style Guide is referenced to back up this
change.

This commit alters only English-language text and example commands in
which the string "{mount point}" is meant to be replaced. No commands
meant for cutting-and-pasting have been altered in this commit.

Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/cephfs: s/mountpoint/mount point/

Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).

cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486
in which page 345 of The IBM Style Guide is referenced to back up this
change.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #58355 from batrick/ceph-backport-fetchhead

script/ceph-backport: robustness adjustments for local git repo quirks

Reviewed-by: Venky Shankar <vshankar@redhat.com>

test/rgw/notification: use real ip address instead of localhost

based on that comment:
https://tracker.ceph.com/issues/67206#note-6
the address used by the endpoint is taken as the real IP address of the
host where the test script is running and not localhost.

we also changed the rabbitmq-server conf to allow "guest"
user to connect over non localhost address

Fixes: https://tracker.ceph.com/issues/67206
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #59167 from zdover23/wip-doc-2024-08-12-cephfs-file-layouts

doc/cephfs: improve "layout fields" text

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #58896 into main

* refs/pull/58896/head:
client: flush the caps release in filesystem sync

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #59205 from xxhdx1985126/wip-seastore-find-pending-version

crimson/os/seastore/btree: fix minor corner case issue

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

client: flush the caps release in filesystem sync

We have hit a race between cap releases and cap revoke request
that will cause the check_caps() to miss sending a cap revoke ack
to MDS. And the client will depend on the cap release to release
that revoking caps, which could be delayed for some unknown reasons.

In Kclient we have figured out the RCA about race and we need
a way to explictly trigger this manually could help to get rid
of the caps revoke stuck issue.

Fixes: https://tracker.ceph.com/issues/67221
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #58415 from ljflores/wip-tracker-66809

qa/suites/upgrade: ignore PG_AVAILABILITY and MON_DOWN for quincy-x and reef-x upgrade suites

Merge pull request #59220 from ivoalmeida/carbon-datatable-cleanups

mgr/dashboard: carbon datatables impr and cleanups

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59218 from yuvalif/wip-yuval-67525

rgw/notifications: fixing radosgw-admin notification json

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #59118 from xxhdx1985126/wip-crimson-backfill-cancellation

crimson/osd/backfill_state: support backfill cancellation

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd/backfill_state: support backfilling cancellation

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/osd/pg_recovery: reset backfill_state when backfill finished

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #57966 from xxhdx1985126/wip-crimson-concurrent-recover-missing

crimson/osd/osd_operations: make the "recover_missing" phase concurrent

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #53151 from xxhdx1985126/wip-crimson-backfill-fixes

crimson/osd/backfill_state: fixes two corner cases in backfilling

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #58911 from yuvalif/wip-yuval-67229

test/cls_2pc_queue: prevent list+remove race between consumers

Reviewed-By: Casey Bodley <cbodley@ibm.com>

Merge pull request #59219 from yuvalif/wip-yuval-50610

doc/rgw/notification: persistent notification queue full behavior

Reviewed-By: Anthony D'Atri <anthony.datri@gmail.com>

crimson/os/seastore/cache: report lru usage/in/out with trans and extent type

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: cleanup periodical reporting

Consolidate time into a single place per SeaStore::Shard.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache/lru: renames

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: refine lru logics

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: move counter_by_extent_t definition

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/seastore_types: unify checks to the extent types

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #59225 from zdover23/wip-doc-2024-08-15-glossary-flapping-osd

doc/glossary: add "flapping OSD"

Merge pull request #59028 from cbodley/wip-67326

rgw/notify: visit() returns copy of owner string

Reviewed-by: Adam Emerson <aemerson@redhat.com>

doc/glossary: add "flapping OSD"

Add an entry for "Flapping OSD" to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #58448 from cbodley/wip-rgw-lc-async

cls/rgw: define lc ops in terms of ObjectOperation instead of IoCtx

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

mgr/dashboard: carbon datatables impr and cleanups

Fixes: https://tracker.ceph.com/issues/67544,
https://tracker.ceph.com/issues/67538,
https://tracker.ceph.com/issues/67542, https://tracker.ceph.com/issues/67545, https://tracker.ceph.com/issues/67546

Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

Merge pull request #58965 from linuxbox2/wip-lcgt-typo

rgwlc: fix typo in getlc (ObjectSizeGreaterThan)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/rgw/notification: persistent notification queue full behavior

Fixes: https://tracker.ceph.com/issues/50610
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #59169 from cbodley/wip-67464

rgw: revert account-related changes to get_iam_policy_from_attr()

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #57888 from liangmingyuanneo/wip-standalone-test-pg-repair

qa/standalone: bugfix for latecy repair after scrub

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #59210 from rhcs-dashboard/multi-cluster-overview-usage-template-fix

mgr/dashboard: fix multi-cluster usage bar error after carbon changes

rgw/notifications: fixing radosgw-admin notification json

Fixes: https://tracker.ceph.com/issues/67525
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #59168 from zdover23/wip-doc-2024-08-12-cephfs-cache-configuration

doc/cephfs: improve cache-configuration.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/os/seastore/cache: pass missing src to touch_extent()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: cleanup add_extent()

Move add_to_dirty() and touch_extent() out of add_extent(), this removes
duplicated calls to touch_extent() from the on_cache callback.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: cleanup remove_from_dirty()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: drop duplicated calls to touch_extent()

The extent is already PRESENT, which means it was already touched in
this transaction.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cached_extent: rename primary_ref_list

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

mgr/dashboard: fix multi-cluster usage bar error after carbon changes

fixes: https://tracker.ceph.com/issues/67536

Signed-off-by: Naman Munet <nmunet@redhat.com>

Merge pull request #59186 from rhcs-dashboard/replace-cluster-capacity-with-usage-bar

mgr/dashboard: replace individual cluster's capacity info with Usage bar in Multi-Cluster

Merge PR #59025 into main

* refs/pull/59025/head:
tools/rados: Fix extra NL in getxattr

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Gabriel Benhanokh <gbenhano@redhat.com>

Merge pull request #58485 from ivoalmeida/carbon-datatable

mgr/dashboard: replace ngx-datatable by carbon datatable

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

crimson/os/seastore/btree: fix minor corner case issue

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

qa/suites/upgrade: ignore MON_DOWN and PG_AVAILABILITY warnings in upgrade tests

Signed-off-by: Laura Flores <lflores@ibm.com>

Merge pull request #58140 from guits/cv-tpm2-support

ceph-volume: add TPM2 token enrollment support for encrypted OSDs

Merge pull request #58956 from ThomasLamprecht/ceph-volume-debian-dependency

debian pkg: record python3-packaging dependency for ceph-volume

qa/s3tests: configure tenant name for 's3 tenant' section

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: replace ngx-datatable by carbon

Fixes: https://tracker.ceph.com/issues/66965
* replaced ngx-datatable by carbon datatable
* created carbon themes for content and tables
* redesigned table actions to render as kebab menu options per data row
* keep only primary actions on datatable toolbar
* implemented carbon batch actions

Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

test/store_test: fix assertions due to unclosed collection refs.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #59112 from idryomov/wip-67353

qa: adjust expected io_opt in krbd_discard_granularity.t

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #54504 from aclamk/wip-aclamk-bs-refactor-write-path

os/bluestore: Recompression, part 2. New write path.

Merge pull request #59001 from cbodley/wip-rgw-async-inline-gc

rgw/rados: delete_objs_inline() uses rgw_rados_operate()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

mgr/dashboard: replace individual cluster's capacity info with Usage bar in Multi-Cluster

fixes: https://tracker.ceph.com/issues/67193

Signed-off-by: Naman Munet <nmunet@redhat.com>

Merge pull request #58410 from bill-scales/erasure_coding_design_document

doc: doc/dev/osd_interals/erasure_coding/enhancements.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Sam Just <sjust@redhat.com>

Merge pull request #59177 from anthonyeleven/aad-qat-followup

doc/radosgw: improve qat-accel.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>

doc/radosgw: improve qat-accel.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge PR #58532 into main

* refs/pull/58532/head:
qa: improve and add commit ID to comment for context

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/cephfs: improve cache-configuration.rst

Improve the text in the section about dealing with cache-pressure alerts
that was added in https://github.com/ceph/ceph/pull/59077. The changes
in this commit were suggested by Anthony D'Atri.

Co-authored-by: Patrick Donnelly <pdonnelly@redhat.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #55050 from rhcs-dashboard/nfs-monitoring

mgr/nfs: scrape nfs monitoring endpoint

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>

rgw: revert account-related changes to get_iam_policy_from_attr()

while bucket ARNs in iam policies don't include account names, policy
evaluation does need to differentiate between buckets in different
tenant namespaces

when requests pass bucket/object ARNs into
verify_bucket/object_permission(), those do include the bucket's tenant
name. to match against those ARNs, we also need to pass the requested
bucket's tenant name into get_iam_policy_from_attr()

Fixes: https://tracker.ceph.com/issues/67464
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54075 from aclamk/wip-aclamk-bs-nice-print

os/bluestore: Recompression, part 1. Nice debugs.

Merge pull request #59164 from toficly/patch-2

doc:update e-mail addresses governance

doc/cephfs: improve "layout fields" text

Improve "layout fields" text in doc/cephfs/file-layouts.rst, as suggesed
by Anthony D'Atri in these comments:

https://github.com/ceph/ceph/pull/59021#discussion_r1704108581
https://github.com/ceph/ceph/pull/59021#discussion_r1704112320

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

doc:update e-mail addresses governance

with the consent of my colleagues
- Dan van der Ster
- João Eduardo Luis
- Mark Nelson

changed their e-mail addresses to the current one

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>
Signed-off-by: Tobias Fischer <138117964+toficly@users.noreply.github.com>

ceph-volume: refactor device path handling for LVM lookups

This consolidates the conditional checks for device paths to
reduce redundancy and improve readability and adds logic to
handle both '/dev/mapper' and '/dev/dm-' paths uniformly by
introducing a utility function `get_lvm_mapper_path_from_dm()`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

ceph-volume: add TPM2 token enrollment support for encrypted OSDs

This adds the required changes to ceph-volume and cephadm in order to support
deploying tpm2 token enrolled encrypted OSDs.

Adding `--with-tpm` when deploying with `--dmcrypt` will enroll a tpm2
token to the corresponding LUKS2 devices.

Example of a osd service spec:

```
service_type: osd
service_id: tpm2_osds
placement:
  host_pattern: '*'
spec:
  data_devices:
    paths:
      - /dev/sdb
  encrypted: true
  tpm2: true
```

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59098 from athanatos/sjust/wip-67415-alienstore-mkfs-crash

crimson: clear AlienStore::coll_map in umount rather than in stop

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #59077 from zdover23/wip-doc-2024-08-07-cephfs-cache-configuration-cache-pressure

doc/cephfs: add cache pressure information

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58951 from rhcs-dashboard/ceph-users-doc-fix

mgr/dashboard: fix ceph-users api

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #58976 from athanatos/sjust/for-review/wip-nvmeofpaxosservice-cosmetic-fixes

mon: cosmetic fixes for NVMeofGwMon files

Reviewed-by: Alexander Indenbaum <aindenba@redhat.com>
Reviewed-by: Leonid Chernin <lechernin@gmail.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Aviv Caro <Aviv.Caro@ibm.com>

Merge pull request #57003 from linuxkidd/wip-pool-max-avail

PGMap: remove pool max_avail scale factor

Merge pull request #58573 from idryomov/wip-selinux-drop-ignore-agetty

qa: agetty entry in selinux allowlist is redundant now

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #58784 into main

* refs/pull/58784/head:
script/backport-create-issue: add debug messages for retry

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

qa: cover a custom object size in krbd_discard_granularity.t

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa: adjust expected io_opt in krbd_discard_granularity.t

With linux.git commit a00d4bfce7c6 ("rbd: increase io_opt again"),
io_opt is set to object set size.

Fixes: https://tracker.ceph.com/issues/67353
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #58605 from adk3798/upgrade-suite-upgrade-in-progress-checks

qa/upgrade: fix checks to make sure upgrade is still in progress

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: fix ceph-users api doc

Fixes: https://tracker.ceph.com/issues/67224
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/nfs: scrape nfs monitoring endpoint

Fixes: https://tracker.ceph.com/issues/62558
Signed-off-by: avanthakkar <avanjohn@gmail.com>

doc/cephfs: add cache pressure information

Add information to doc/cephfs/cache-configuration.rst about how to deal
with a message that reads "clients failing to respond to cache
pressure". This procedure explains how to slow the growth of the
recall_caps value so that it does not exceed the
mds_recall_warning_threshold.

The information in this commit was developed by Eugen Block. See
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/5ROH5CWKKOEIQMVXOVRT5OO7CWK2HPM3/#J65DFUPP4BY57MICPANXKI7KAXSZ5Z5P
and https://www.spinics.net/lists/ceph-users/msg73188.html.

Fixes: https://tracker.ceph.com/issues/57115
Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #59087 from kamoltat/wip-ksirivad-fix-stretch-device-class-docs

docs/rados/operations/stretch-mode: warn device class is not supported

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58736 from amathuria/wip-66922-amat

qa/rados/dashboard: Add PG_DEGRADED to ignorelist

Merge pull request #57037 from adk3798/cephadm-limit-shell-mounts

cephadm: limit mounts for shell and ceph-volume commands

Merge PR #58861 into main

* refs/pull/58861/head:
mds: only authpin on wrlock when not a locallock

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>

PGMap: remove pool max_avail scale factor

The scaling of max_avail by the ratio of non-degraded to total objects
count results in the reported max_avail increasing proportionally to the
number of OSDs marked `down` but not `out`.  This is counter intuitive
since OSDs going `down` should never result in more space being
available.

Removing the scale factor allows max_avail to remain unchanged until the
OSDs are marked `out`.

Signed-off-by: Michael J. Kidd <linuxkidd@gmail.com>

Merge pull request #59091 from zdover23/wip-doc-2024-08-08-README-md-alerts

doc/README.md - add "tip" alert styling

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

Merge pull request #59030 from ljflores/wip-fix-crontab-2

qa/crontab: force priority for main rados nightlies