rgw: add new cap user-without-keys

This new cap removes S3 keys and Swift keys from
the response in the admin api command 'get user info'.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #49594 from Matan-B/wip-matanb-crimson-calc-subsets

crimson: Support Partial Object Recovery

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #55022 from yuvalif/wip-yuval-63901

rgw/kafka: make the connection idle timeout configurable

reviwed-by: cbodley

Merge pull request #54967 from yuvalif/wip-yuval-63859

rgw/notifications/lc: handle publish_commit() failures as warnings

reviewed-by: dang, thotz

crimson/osd: Keep track of modified_ranges

* `modifies_ranges` interval_set is added to osd_op_params_t
* keep track of modified_ranges while executing relevant ops
* Add `osd_op_params` parameter to `PGBackend::remove()`.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd/replicated_recovery_backend: handle_pull_response() - recalc_subsets()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

mson/osd/replicated_recovery_backend: Introduce prep_push_to_repli…
…ca()

Intelligently push an object to a replica.  make use of existing
clones/heads and dup data ranges where possible.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd/replicated_recovery_backend: seperate prep_push_to_replica and prep_push

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

rename prep_push to prep_push_to_replica

dismiss head_ctx

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd/replicated_recovery_backend: prepare_pull to support calc_clone_subsets

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd/object_metadata_helper: Introduce calc_*_subsets

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #55037 from ronen-fr/wip-rf-crush-dummy

test/crush: removed unused code

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #53319 from leonid-s-usov/vstart-localhost-for-real

vstart.sh: make sure that --localhost does only bind to 127.0.0.1 in all cases

Merge pull request #54883 from Svelar/memstore

os/memstore: deep copy on partial bufferptrs on setattrs

Merge pull request #54884 from Svelar/memory_is_zero_dev

inline_memory: optimized mem_is_zero for aarch64 by neon intrinsic

Merge pull request #55048 from zdover23/wip-doc-2024-01-03-radosgw-admin-add-remove-a-key

doc/radosgw: edit "Add/Remove a Key"

Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #54214 from athanatos/sjust/wip-crimson-scrub

crimson: add initial scrub support

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #55053 from zdover23/wip-doc-2023-01-04-release-note-cleanup-following-55049

doc/releases: edit reef.rst

Merge pull request #54994 from ceph/wip-lflores-upgrade-stale-bot

.github/workflows: upgrade stale bot

doc/radosgw: edit "Add/Remove a Key"

Edit the section "Add/Remove a Key" in doc/radosgw/admin.rst. Each
operation (e.g. "Adding an S3 key pair for a user", "Removing an S3 key
pair for a user") now has its own subsection. This increased granularity
should make it easier in the future to link to each of these specific
operations, if needed.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/releases: edit reef.rst

Make minor corrections to doc/releases/reef.rst. These corrections were
suggested by Anthony D'Atri in https://github.com/ceph/ceph/pull/55049.

Signed-off-by: Zac Dover <zac.dover@proton.me>

rgw/notifications/lc: handle publish_commit() failures as warnings

Fixes: https://tracker.ceph.com/issues/63859
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #53582 from rhcs-dashboard/chartjs-upgrade

mgr/dashboard: chartjs and ng2-charts version upgrade

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge PR #54825 into main

* refs/pull/54825/head:
mds: change priority of mds rss perf counter to useful

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #54607 from athanatos/sjust/wip-async-recovery-62338-release-note

PendingReleaseNotes: add release note for 62338

Reviewed-by: Neha Ojha <nojha@redhat.com>

PendingReleaseNotes: add release note for 62338

See https://tracker.ceph.com/issues/62338 and
2fc5486e.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #54024 from Matan-B/wip-matanb-snap_seq_inc

osd/osd_types: don't increment snap_seq on removal

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #55043 from zdover23/wip-doc-2024-01-02-rados-troubleshooting-log-and-debug-title

doc/rados: improve subsection title in log-and-debug.rst

mgr/dashboard: chartjs and ng2-charts version pugrade

Fixes:https://tracker.ceph.com/issues/63160
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

doc/rados: improve subsection title in log-and-debug.rst

Change the title of "Logging Settings" to "Logging and Debugging
Settings" in doc/rados/troubleshooting/log-and-debug.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54876 from ifed01/wip-ifed-fix-63795

test/store_test: fix deferred writing test cases

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
Reviewed-by: Pere Diaz Bou <pere-altea@hotmail.com>

Merge pull request #55038 from dparmar18/minor-doc-update-disaster-recovery-experts

doc/cephfs: Update disaster-recovery-experts.rst to mention Slack

doc/cephfs: Update disaster-recovery-experts.rst to mention Slack

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

rgw/kafka: make the connection idle and sleep timeouts and configurable

Fixes: https://tracker.ceph.com/issues/63901
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

test/crush: removed unused code

to fix a compiler warning re unused variable.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #55031 from zdover23/wip-doc-2024-01-01-mgr-nfs-mounting-windows

doc/mgr: document lack of MSWin NFS 4.x support

doc/mgr: document lack of MSWin NFS 4.x support

Document that no version of Microsoft Windows supports mounting an NFS
v4.x export natively, as of the time of this commit.

This commit has been made in response to an anonymous note left on
https://pad.ceph.com/p/Report_Documentation_Bugs.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #55030 from zdover23/wip-doc-2023-12-31-radosgw-admin-remove-user-subuser-rewrite

doc/radosgw: edit "remove a subuser"

doc/radosgw: edit "remove a subuser"

Edit the English language in the section "Remove a Subuser" in
doc/radosgw/admin.rst. This commit is made in response to Matt
Benjamin's request for improvement of this section
(https://github.com/ceph/ceph/pull/55028#discussion_r1438599833).

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #55023 from zdover23/wip-doc-2023-12-29-radosgw-admin-remove-user-subuser

doc/radosgw: edit sections

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/radosgw: edit sections

Edit the following sections in doc/radosgw/admin.rst:

* Remove a User
* Remove a Subuser

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54982 from ronen-fr/wip-rf-be-error

osd/scrub: unhandled scrub backend errors should cause an abort

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #54907 from ljflores/wip-balancer-incremental

osd: fix Incremental decode for new/old_pg_upmap_primary

Reviewed-by: Josh Salomon <josh.salomon@gmail.com>

Merge pull request #54627 from maydaycc/wip-63615

mgr: set MMgrBeacon high priority

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #54194 from ifed01/wip-ifed-refresh-perf-counters

os: introduce ObjectStore::refresh_perf_counters() method

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #54047 from lxbsz/wip-62081

osd: clean up the sparse-read related code

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #54644 from idryomov/wip-63607

librados: make querying pools for selfmanaged snaps reliable

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #55011 from zdover23/wip-doc-2023-12-27-radosgw-admin-create-a-subuser

doc/radosgw: admin.rst - edit "Create a Subuser"

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

Merge pull request #54998 from ronen-fr/wip-rf-lmap-event

osd/scrub: remove unused PGScrubGotLocalMap message

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>

Merge pull request #55016 from zdover23/wip-doc-2023-12-27-radosgw-user-info-suspend-enable

doc/radosgw: admin.rst - edit sections

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/radosgw: admin.rst - edit sections

Edit the following sections in doc/radosgw/admin.rst:

- Get User Info
- Modify User Info
- User Suspend/Enable (I split this into two sections)

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54835 from rhcs-dashboard/directories-fixes

mgr/dashboard: small fixes to directories view

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

doc/radosgw: admin.rst - edit "Create a Subuser"

Edit the section "Create a Subuser" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54663 from Matan-B/wip-matanb-cot-pgmeta-attrs

tools/ceph_objectstore_tool: action_on_all_objects_in_pg to skip pgmeta

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #54597 from sseshasa/wip-mclk-snaptrim-cost

osd: Tune snap trim item cost to reflect a PGs' average object size for mClock scheduler

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #54518 from NitzanMordhai/wip-nitzan-cleanomap-rados-optional

Tools/rados: Improve Error Messaging for Object Name Resolution

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>

Merge pull request #54498 from Matan-B/wip-matanb-send-inc-map-refactor

osd/OSD: rewrite send_incremental_map()

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #55003 from zdover23/wip-doc-2023-12-26-radosgw-admin-create-a-user

doc/radosgw: admin.rst - edit "Create a User"

Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

doc/radosgw: admin.rst - edit "Create a User"

Edit the section "Create a User" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54997 from zdover23/wip-doc-2023-12-24-radosgw-admin-main

doc/radosgw: edit admin.rst 1 of x

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/radosgw: edit admin.rst 1 of x

Edit doc/radosgw/admin.rst to the end of the section "User Management".
Correct grammar (mostly awkward prepositions) and tidy up weird clauses.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

osd/scrub: remove unused PGScrubGotLocalMap message

The relevant event (IntLocalMapDone) is processed directly within the
scrubber, instead of being queued to the OSD events queue.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: unhandled scrub backend errors should cause an abort

... as we do not have any mechanism to handle them.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

.github/workflows: upgrade stale bot

Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #54801 from WangWNico/balancer_project

Improve Ceph Balancer Status Output

Merge pull request #53191 from rzarzynski/wip-all-kickoff-s-2

kickoff v19 squid

Merge pull request #54989 from zdover23/wip-doc-2023-12-22-radosgw-compression-supported-plugins

doc/radosgw: list supported plugins-compression.rst

doc/radosgw: list supported plugins-compression.rst

List supported compression plugins in doc/radosgw/compression.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54970 from zdover23/wip-doc-2023-12-20-radosgw-compression-grammar

doc/radosgw: edit compression.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/radosgw: edit compression.rst

Improve the grammar and simplify the sentence structure of
doc/radosgw/compression.rst. This commit is made in anticipation of a
near-future commit that will list the compression algorithms available
to users of Ceph.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54918 from tchaikov/cmake-build-rockdb-with-cxxflags

cmake/modules/BuildRocksDB.cmake: inherit parent's CMAKE_CXX_FLAGS

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54936 from ronen-fr/wip-rf-pcnt3

osd/scrub: initial set of performance counters

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

osd/scrub: collect replica reservations performance counters

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

test/osd: log scrub performance counters in osd-scrub-test.sh

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: provide selected counters set to Scrubber & FSM

also - move most time keeping from the Scrubber to the FSM.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: add a basic set of performance counters

Add a labeled set of performance counters, with the labels selecting
one of four copies of the counters - one per each scrub level and
pool type combination.

Inside the Scrubber, the relevant set of counters is selected
when the scrub is initialized.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #54828 from ronen-fr/wip-rf-dec-fixes-1

osd/scrub: don't block high-priority scrubs on local resources

Reviewed-by: Samuel Just <sjust@redhat.com>-

Merge pull request #54023 from Matan-B/wip-matanb-objectstore-tool-superblock

tools/ceph_objectstore_tool: Support get/set/superblock

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #54195 from ifed01/wip-ifed-silent-warns

core: silent a couple of compiler warnings on uninitialized vars.

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #52871 from NitzanMordhai/wip-nitzan-ceph-dencoder-extend-osd-type-available

ceph-dencoder: OSD - Add missing types

ceph-dencoder: osd - Add missing types

Currently, ceph-dencoder lacks certain osd types, preventing us from accurately checking the ceph corpus for encode-decode mismatches.
This pull request aims to address this issue by adding the missing types to ceph-dencoder.

To successfully incorporate these types into ceph-dencoder, we need to introduce the necessary `dump` and `generate_test_instances`
functions that was missing in some types. These functions are essential for proper encode and decode of the added types.

This PR will enhance the functionality of ceph-dencoder by including the missing types, enabling a comprehensive analysis of encode-decode consistency.
With the addition of these types, we can ensure the robustness and correctness of the ceph corpus.

This update will significantly contribute to improving the overall reliability and accuracy of ceph-dencoder.
It allows for a more comprehensive assessment of the encode-decode behavior, leading to enhanced data integrity and stability within the ceph ecosystem.

Fixes: https://tracker.ceph.com/issues/61788
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>

Merge pull request #54916 from tchaikov/cmake-boost-workaround

cmake: only apply workaround for boost < 1.84

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #54919 from tchaikov/cmake-boost-1.84

cmake: add 1.82 and 1.83 to known versions

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #53591 from EdwardVitor/fix-makecheck-warning

kv: resolve three compilation warnings

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #53524 from sseshasa/wip-fix-osd-op-queue-rndmize-62171

osd: Apply randomly selected scheduler type across all OSD shards

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #53228 from rzarzynski/wip-bug-62645

 common/weighted_shuffle: don't feed std::discrete_distribution with all-zero weights

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #54953 from ljflores/wip-fix-18.2.1-doc

doc/releases: specify dashboard improvements

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/releases: specify dashboard improvements

...and address other nits

Signed-off-by: Laura Flores <lflores@ibm.com>

Merge pull request #54935 from yuvalif/wip-yuval-return-lua-devel

build/rgw/lua: return lua-devel runtime dependency

reviewed-by: cbodley

Merge pull request #54710 from kalaspuffar/replace_jwt_with_pure_python

mgr/dashboard: Simplify authentication protocol

Reviewed-by: Fabian-Gruenbichler <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #54897 from aclamk/wip-aclamk-bs-fix-esb-multichunk-alloc

os/bluestore: Fix assert() caused by ESB

Merge pull request #54609 from xxhdx1985126/wip-crimson-rollback-fixes

crimson/osd/pg_backend: fix rollback on whiteouts

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #54960 from zdover23/wip-doc-2023-12-19-install-clone-source-submodule-update

doc/install: update "update submodules"

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

doc/install: update "update submodules"

Remove misleading material that would give readers the wrong idea about
when stale submodules are present. This commit is made in response to
information given to me by Ilya Dryomov here: https://github.com/ceph/ceph/pull/54929#issuecomment-1859237986.

Signed-off-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: small fixes to directories view

Increased the precedence for Directories
Expand the node by default and select them as well

Fixes: https://tracker.ceph.com/issues/63754
Signed-off-by: Nizamudeen A <nia@redhat.com>

crimson/osd/pg_backend: whiteout objects if they will have clones when
rolling back

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #54917 from tchaikov/cmake-cephfs-sans-libcephfs

cmake: differentiate WITH_CEPHFS from WITH_LIBCEPHFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #54931 from athanatos/sjust/wip-crimson-2023-12-13-fixes

crimson: misc fixes

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #54506 from ceph/wip-yuriw-release-18.2.1-main

doc: Add Reef 18.2.1 release notes

Merge pull request #52495 from adamemerson/wip-neorados-learning-from-experience

neorados: Learning from experience

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>