os/bluestore: Fix collect_alerts

In some cases bdev->collect_alerts() was called twice on the same device.
It happened when DB and SLOW are the same device.
Now only called once.

It made no difference when we just set alert string,
but with EXTBLKDEV we append warning message.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

os/bluestore: extblkdev related health warnings

Replaced start error with health warnings:
EXTBLKDEV: plugin xxx not loaded
EXTBLKDEV: plugin xxx used on mkfs, but now uses plugin yyy

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

extblkdev/fcm: Replace errors with health warning

Now plugin does not assert or fail to load,
but instead raises following health warnings:
EXTBLKDEV: multivolume fcm will not work properly
EXTBLKDEV: failed accessing FCM utilization log
EXTBLKDEV: bdev_enable_discard not enabled - free space will leak

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

extblkdev/fcm: Cleanup

Add `override` to virtual interface functions.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

extblkdev: Fix preload

One execution path did not save preload_result value.
Fixed now.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

blk/extblkdev: Health warns from ExtBlkDev plugins

Expanded interface by collect_alerts.
Created category "EBD_PROBLEMS" for ExtBlkDev problems.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

blk: Expand collect_alerts to allow specialization

Previously we had BlockDevice::collect_alerts that had fixed
implementation.
Expanded BlockDevice::collect_alerts into virtual, so KernelDevice can
override it.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #68493 from rhcs-dashboard/76158-rgw-restart-stop-bug

mgr/dashboard : Fix services restart/stop issue

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #68036 from ifed01/wip-ifed-print-effective-vsel-extra

os/bluestore: print effective extra in 'bluefs stats' report

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #68177 from ifed01/wip-ifed-better-ebd-detect

blk,os/bluestore: do not call extblkdev::detect_device on every

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #68353 from Jayaprakash-ibm/wip-clean-discard-warn

mon: simplify discard queue health warning

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #68441 from rhcs-dashboard/mgr/dashboard/carbonize-multisite-wizard-realm-token

mgr/dashboard: mgr/dashboard: Carbonize Realm Name and Token block in Multi-site Replication Wizard

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #68537 from rhcs-dashboard/fix-76200

mgr/dashboard: [snap-visibility]Edit Client config option remains stuck in loading when nfs user is configured

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #68521 from rhcs-dashboard/fix-osd

mgr/dashboard: Fix osd modal spacings and footer

Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>

Merge pull request #67995 from rhcs-dashboard/rename-namespacesize

mgr/dashboard: Rename namespace block size field and reorder namespace form order

Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: [snap-visibility]Edit Client config option remains stuck in loading when nfs user is configured

Fixes: https://tracker.ceph.com/issues/76200
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

mgr/dashboard: mgr/dashboard: Rename namespace block size field and reorder namespace create form fields

Fixes: https://tracker.ceph.com/issues/75707
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

Merge pull request #68490 from rhcs-dashboard/fix-76163-main

mgr/dashboard: remove sync_from entry when sync_from_all is true

Reviewed-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard : Fix RGW restart/stop issue
fixes : https://tracker.ceph.com/issues/76158
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

mgr/dashboard: Fix osd modal spacings and footer

Fixes https://tracker.ceph.com/issues/76188

- aligned as per https://carbondesignsystem.com/components/modal/usage/#transactional-modal

Signed-off-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #68465 from rhcs-dashboard/IBMCEPH-10149-rgw-non-version-bucket

mgr/dashboard : fix-non-versioning-bucket-issue

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #68412 from cloudbehl/cephfsdashboardCPUMEM

mgt/dashboard: Add the CPU, Memory to CephFS dashboard

Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>

mgr/dashboard: remove sync_from entry when sync_from_all is true

Fixes: https://tracker.ceph.com/issues/76163
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #68411 from rhcs-dashboard/fix-76054-main

mgr/dashboard: Option to select archive option while Import Multi-site token

Reviewed-by: Naman Munet <nmunet@redhat.com>

Merge pull request #67869 from rhcs-dashboard/sync-policy-replication

mgr/dashboard: sync policy created for a bucket in Object >> Multi-site >> Sync-policy, is not reflecting under bucket's replication

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #68485 from kginonredhat/issue-76147-typo-spcified-in-comment

mgr/cephadm: fix typo in schedule.py comment (specified)

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #68464 from vshankar/wip-exlude-crimson-fs-upgrade

qa/cephfs: exclude ceph-osd-{classic,crimson} in fs:upgrade:featurefu…

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #67867 from vshankar/wip-75380

qa/cephfs: treat "implicit declaration of function" for blogbench workunit for newer gcc version

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #68179 from ifed01/wip-ifed-util-no-warn

common/util: silent "warning: ‘cgroup_limit’ may be used uninitialized"

Reviewed-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>

Merge pull request #68486 from rhcs-dashboard/fix-76156-main

mgr/dashboard: remove archive option from master zone creation

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #68440 from rhcs-dashboard/tearsheet-layout-fix

mgr/dashboard: tearsheet-layout-fix

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #68224 from rhcs-dashboard/fix/nvmeof-initiator-hostname-prevalidation

mgr/dashboard: NVMe-of Initiator Hostname Pre-validation

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #68160 from rhcs-dashboard/fix-delete-subsystem

mgr/dashboard: fix delete subsystem issue

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #67546 from kotreshhr/mirror-syncthreads-fair-scheduling

tools/cephfs_mirror: Distribute datasync threads evenly across snapshots

Reviewed-by Venky Shankar <vshankar@redhat.com>

Merge pull request #67710 from vshankar/wip-75083

qa/cephfs: make test_journal_migration immune to stdout changes

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #67878 from salieri11/igolikov-fix-subvolume-map-75343-new

mds: fix data race on subvolume_metrics_map in MetricsHandler

Reviewed-by: Venky Shankar <vshankar@redhat.com>

mgr/dashboard: mgr/dashboard: Carbonize Realm Name and Token block in Multi-site Replication Wizard

Fixes: https://tracker.ceph.com/issues/76085
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

Merge pull request #68175 from rhcs-dashboard/bugfix/nvmeof-namespace-subsystem-selection-order

mgr/dashboard: NVMeoF namespace create form should show subsystem selection first

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>
Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>

mgr/dashboard: Option to select archive option while Import Multi-site Token

Fixes: https://tracker.ceph.com/issues/76054
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

mgt/dashboard: Add the CPU, Memory to CephFS dashboard

Fixes: https://tracker.ceph.com/issues/76058
Signed-off-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #68377 from rhcs-dashboard/fix-namespace-block-size

mgr/dashboard: fix namespace block size in namespace form

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #68302 from rhcs-dashboard/bug-13708

mgr/dashboard : Unable to remove gateway node from gateway group

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

mgr/dashboard: remove archive option from master zone creation

Fixes: https://tracker.ceph.com/issues/76156
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard : fix-non-versioning-bucket-issue
fixes : https://tracker.ceph.com/issues/76109
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

mgr/cephadm: fix typo in schedule.py comment (specified)

Correct spelling in the ip_addrs allocation comment: "spcified" → "specified".

Fixes: https://tracker.ceph.com/issues/76147
Signed-off-by: Kobi Ginon <kginon@redhat.com>

Merge pull request #68033 from smanjara/wip-zg-rename

rgw/multisite: remove old zonegroup name on the remote zone after a rename

Merge pull request #68163 from ceph/wip-oozmen-75831

rgw/admin: enhance bucket list --marker to support versioned bucket pagination

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #67424 from BBoozmen/wip-oozmen-75031

rgw/beast: use strand executor for timeout timer to prevent concurrent socket access

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>

Merge pull request #67665 from tchaikov/wip-test-rgw-fix-warning

test/rgw: fix -Wstring-plus-char warning in test_d4n_policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #67189 from cbodley/wip-74231

rgw: fix 'bucket stats' when bucket index doesn't exist

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #67958 from cbodley/wip-qa-rgw-compress-encrypted

qa/rgw: crypt subsuite tests compress-encrypted

Reviewed-by: Matthew N. Heler <matthew.heler@hotmail.com>

Merge pull request #68393 from Hezko/traddr-compat

mgr/dashboard: introduce traddr for backward compatibility

Merge pull request #67841 from rhcs-dashboard/fix-pools-app-list-names

mgr/dashboard : Fix application names in pools form

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #68011 from mheler/wip-75650

rgw: always decompress in CopyObject when source is compressed

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #68324 from mheler/wip-multipart-cloud-transition

rgw: fix cloud tier multipart resume using part number 0

Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #67560 from mheler/wip-cloud-restore-etag-fix

rgw/cloud-restore: strip quotes from ETag on cloud tier fetch

Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #67966 from mheler/wip-75649

rgw: fix fixup_range cascade for compressed+encrypted objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #68466 from bluikko/wip-doc-fix-tentacle-releases

doc/releases: fix Tentacle data in releases.yml

qa/rgw: crypt subsuite tests compress-encrypted

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: rgw task can enable/disable zone features

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: don't duplicate 'user list' commands for default zone

some commands during setup expect the zone to exist already, so run
'radosgw-admin user list' to make sure a default zone/zonegroup are
created. avoid duplicating this in several subtasks by moving this to
its own subtask that runs when a realm is not configured

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #67863 from mheler/wip-bug-75569

rgw: stop crashing when an admin tries to assume a nonexistent role

rgw: fix cloud tier multipart resume starting at part number 0

When resuming a cloud tier multipart upload, the part-size
calculation was inside the fresh-init block and never executed.
cur_part, num_parts, and part_size stayed at 0, causing the
remote endpoint to reject part number 0 as invalid.

Move the part-size calculation out of the init block so it
runs for both fresh and resumed uploads.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>

rgw/cloud-restore: strip quotes from ETag on cloud tier fetch

When objects are restored from a cloud endpoint, the ETag value read
from the HTTP response includes surrounding double-quotes per RFC 7232.
RGW stores ETags unquoted internally, and dump_etag() adds its own
quotes when serving responses. The mismatch results in double-quoted
ETags like ""abc123-6"" on restored objects.

Strip the quotes from both the etag output parameter and the
RGW_ATTR_ETAG attribute after fetching from the cloud endpoint,
matching the unquoted format RGW uses everywhere else.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>

Merge pull request #68204 from abhishek593/bug-73437

rgw: fix uninitalized fields in pubsub topic creation

Merge pull request #67548 from mertsunacoglu/wip-rgw-lua-postauth

rgw/lua: Add new Lua postAuth hook that is being run after authorization

Merge pull request #68097 from cheese-cakee/wip-72088-sync-checkpoint-stdout

radosgw-admin: output bucket sync checkpoint to stdout

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #68279 from wuhongsong/fix/rgw-vault-content-type

rgw: add Content-Type header to Vault KMS requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #67479 from nbalacha/wip-nbalacha-75127

rgw: Lua Package management fixes

mgr/dashboard : Select replicated rule by default in pools form
fixes : https://tracker.ceph.com/issues/75632
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

Merge pull request #66778 from Kushal-deb/d3n-cephadm-automation

mgr/cephadm: Implement D3N L1 persistent datacache support for RGW

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>

mgr/dashboard : Fix application names in pools form
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

Merge pull request #66194 from ShwetaBhosale1/fix_issue_73774_nfs_tls_add_xprtsec

mgr/nfs: Cephadm support for NFS-Ganesha TLS configuration adding new option xprtsec

Reviewed-by: Adam King adking@redhat.com

Merge pull request #66763 from Shubhaj1810/fix-issue-2359181

mgr/orchestrator: sort host labels in ceph orch host ls output

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
Reviewed-by: Kushal Deb <Kushal.Deb@ibm.com>
Reviewed-by: Ujjawal Anand <Ujjawal.Anand@ibm.com>

Merge pull request #67828 from linuxbox2/wip-75540

rgw_file: always assign string_view of current common_prefix

Merge pull request #67722 from kchheda3/wip-add-transid-notification

rgw/notification: notification Event is not printing the correct  x_amz_request_id.

Merge pull request #67656 from kchheda3/wip-oidc-thumbprint-optional

rgw/oidc: make the thumprint field optional

Merge pull request #67616 from smanjara/wip-bucket-sync-run

rgw/multisite: 'bucket sync run' command during full sync does not take a bucket-wide lock

Merge pull request #60387 from thotz/rgw-placement-store-standard-storageclass

rgw/placement: display storage class in user info dump

Merge pull request #67219 from mertsunacoglu/wip-lua-bucket-metadata

RGW: Add Bucket metadata attribute for Lua scripts

Merge pull request #67365 from Kushal-deb/fix-test-nvmeof-sysctl

cephadm: reapply hugepages for nvmeof at service start

Reviewed-by: Adam King adking@redhat.com

Merge pull request #67999 from Shubhaj1810/nfs-ganesha-servicemap-fix

mgr/cephadm: align nodeid and add register_service for NFS Ganesha service visibility

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>

Merge pull request #68454 from idryomov/wip-76101

qa: fix setting rbd_sparse_read_threshold_bytes in test_migration_clone()

Reviewed-by: Miki Patel <miki.patel132@gmail.com>

mgr/cephadm: preserve in-flight D3N allocations during fresh deploy

Avoid pruning per-(service, host) D3N device assignments when
daemon details are not yet visible in the mgr cache.

This keeps initial daemon-to-device mappings stable on fresh
deployment so 1:1 allocation is preserved when free devices exist.

Signed-off-by: Kushal Deb <Kushal.Deb@ibm.com>

Merge pull request #67735 from bachmanity1/fix/cephadm-host-rm-keyerror

mgr/cephadm: fix KeyError when host is removed during serve loop

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

doc/releases: fix Tentacle data in releases.yml

Fixes the Ceph Releases doc that is now completely missing 20.2.0
and renders the Tentacle gantt bar starting at the time of 20.2.1.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67836 from rhcs-dashboard/74243-add-bottom-padding-to-dashboard

mgr/dashboard : Add bottom padding for dashboard screens

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>

Merge pull request #67952 from rhcs-dashboard/stretched-cluster-pool-size

mgr/dashboard : add stretch cluster validation for pools form

Reviewed-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: introduce traddr together with existing server_address for backward compatibilty

Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

mgr/nfs: Cephadm support for NFS-Ganesha TLS configuration adding new option xprtsec

Fixes: https://tracker.ceph.com/issues/73774
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>

Merge pull request #65641 from rkachach/fix_issue_69325

mgr/cephadm: Fix alertmanager TLS and global security handling

Reviewed-by: Adam King adking@redhat.com

Merge pull request #66121 from ShwetaBhosale1/fix_issue_73712_fix_nfs_client_commands_hung_issue

mgr/cephadm: increase default backend health check interval for NFS

Reviewed-by: Adam King adking@redhat.com
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #67598 from Kushal-deb/fix-zg-modify

Avoid calling zonegroup modify if realm/zonegroup/zone are not present

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #68061 from JoshuaGabriel/75744-cephadm-zap

mgr/cephadm: Fixes infinite loop when OSD zap fails during removal

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

mgr/dashboard: tearsheet-layout-fix

Fixes: https://tracker.ceph.com/issues/76084
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

Fix test_remote_executables

update ast.ButOr -> ast.BitOr

Signed-off-by: Kushal Deb <Kushal.Deb@ibm.com>

mgr/cephadm: Implement D3N L1 persistent datacache support for RGW

Add RGW D3N L1 persistent datacache support backed by host block devices.
Select devices deterministically per (service, host) with intra-service
sharing, forbid cross-service reuse, prepare/mount devices, and
bind-mount per-daemon cache directories into the container.

Signed-off-by: Kushal Deb <Kushal.Deb@ibm.com>

qa/cephfs: exclude ceph-osd-{classic,crimson} in fs:upgrade:featureful_client:upgraded_client

Signed-off-by: Venky Shankar <vshankar@redhat.com>

qa/cephfs: make test_journal_migration immune to stdout changes

Fixes: http://tracker.ceph.com/issues/75083
Signed-off-by: Venky Shankar <vshankar@redhat.com>

mgr/dashboard : add stretch cluster validation for pools form
fixes : https://tracker.ceph.com/issues/75667
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>