Merge pull request #48597 from ronen-fr/wip-rf-osd-bitfields

osd: change two ObjectContext flags from bitfields to full variables

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #49624 from ronen-fr/wip-rf-unique-formatter

common: extend Formatter API with a unique_ptr-returning static function

Merge pull request #49537 from rzarzynski/wip-bug-57546

Revert "osd/PeeringState: proc_lease_ack break once found from OSD"

Merge pull request #48854 from ifed01/wip-ifed-small-chunk-bluefs

os/bluestore: enable 4K allocation unit for BlueFS

Merge pull request #49314 from SUSE/wip-tserong-qa-crash-chown

qa/workunits/rados/test_crash: chown crash files to ceph user

Merge pull request #48799 from DanWritesCode/add-exception-handling

mon: add exception handling to ceph health mute

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #48836 from ljflores/wip-safe-timer-crash

osd: shut down the MgrClient before osd_fast_shutdown

Merge pull request #49741 from adamemerson/wip-datalog-nonnull-yield-mostly

rgw: Remove (most) uses of null_yield in datalog

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #49826 from cbodley/wip-qa-rgw-s3tests-tox

qa/s3tests: run with tox instead of bootstrap+nose

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #49879 from ljflores/wip-16.2.11-doc-fix

doc/releases: fix PR list

doc/releases: fix PR list

Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #46535 from cbodley/wip-use-epel-arrow

rpm: use system arrow/utf8proc packages by default

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #49769 from alimaredia/wip-rgw-optional-yield-create-bucket

rgw: Thread optional_yield through create bucket op

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #48731 from alimaredia/wip-rgw-admin-api-secondary-zone-user-create-fix

rgw: set keys from from master zone on admin api user create

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rpm: use system arrow packages for fedora/centos9

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rpm: combine system_utf8proc option into system_arrow

libarrow-devel depends on utf8proc-devel, so we can't enable
system_arrow without system_utf8proc

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rpm: fix package names for arrow/parquet

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49839 from ceph/wip-yuriw-release-16.2.11-main

doc: 16.2.11 Release Notes

doc: 16.2.11 Release Notes

See https://tracker.ceph.com/issues/58257#note-2 for details

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #49432 from jmolmo/fix_hardcoded_ns

mgr/rook: Replace hardcoded namespace

Replace hardcoded namespace

Reviewed-by:
Travis Nielsen <tnielsen@redhat.com>
Nick Gentoli <gentoli@live.com>

Merge pull request #49819 from ivancich/wip-typing-policies-maxnum

rgw: minor clean-ups to user policy code

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #49873 from zdover23/wip-doc-2023-01-26-dev-developer-guide-slack

doc/dev: add Slack to Dev Guide essentials

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev: add Slack to Dev Guide essentials

Add Ceph's Slack to doc/developer_guide/essentials.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #49797 from cbodley/wip-58513

qa/openssl_keys: 'rm' ignores file not found

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #49313 from ifed01/wip-ifed-fix-cls-queue-list

cls/queue: use larger read chunks in queue_list_entries

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>

Merge pull request #49171 from guits/tracker_58137

ceph-volume: fix a bug in lsblk_all()

rgw: set keys from from master zone on admin api user modify

When users were being modified via the admin_api
to generate keys on a secondary zone, the keys
returned in the json output are not correct.

This commit ensures the proper keys are set to be
returned to the user via the json output.

fixes: https://tracker.ceph.com/issues/57724

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: set keys from from master zone on admin api user create

When users were being created via the admin_api
on a secondary zone the keys returned in the json
output are not correct.

This commit ensures the proper keys are set to be
returned to the user via the json output.

fixes: https://tracker.ceph.com/issues/57724

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: stop blocking for asio threads for create_bucket()

Asio thread for create_bucket operation are no
longer blocked because optional yield is passed
down in previous instances where asio threads for
op were blocked.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: add conditional backtrace logging after "WARNING: blocking librados call"

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #49818 from adk3798/iscsi-gw-rm-async

mgr/cephadm: call iscsi post_remove from serve loop

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #49802 from rkachach/fix_issue_58517

doc/cephadm: Adding doc for daemon_cache_timeout config option

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #49392 from ifed01/wip-ifed-fix-spillover-test

test/store_test: use original rocksdb settings to enable bluefs

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #48972 from Matan-B/wip-matanb-mon-dump-slow-ops

mon/MonCommands: Support dump_historic_slow_ops

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #48876 from Matan-B/wip-matanb-address-log

osd: Properly log OpRequest, ObjectContext and SnapSetContext

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #48835 from ifed01/wip-ifed-noerr-alloc-restore

os/bluestore: do not spoil log with errors from __restore_allocator

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Gabriel BenHanokh <benhanokh@gmail.com>

Merge pull request #47332 from NitzanMordhai/wip-nitzan-pglog-ec-getattr-error

Wip nitzan pglog ec getattr error

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48428 from zmc/cephadm-smoke-small

qa/suites: Add orch/cephadm/smoke-small suite

Reviewed-by: Adam King <adking@redhat.com>

doc/cephadm: Adding doc for daemon_cache_timeout config option
Fixes: https://tracker.ceph.com/issues/58517
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #49256 from yuvalif/wip-yuval-remove-pubsub-struct

rgw/pubsub: remove subscription data structures and cleanup

reviewed-by: cbodley

Merge pull request #49796 from batrick/first-damage-big-dir

tools/cephfs/first-damage: fix iteration of big dirfrags

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #49814 from ivancich/wip-clean-up-trailing-spaces

rgw: clean up header files

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

rgw: minor clean-ups to user policy and config options

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

rgw: clean up header files

Remove trailing spaces and use "#pragma once" rather than "#ifndef" to
assure a header files is only loaded once during compilation.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #49176 from cbodley/wip-58127

rgw: preserve RGWZoneGroup::master_zone on last zone deletion

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-By: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #49815 from adk3798/no-overwrite-logrotate

cephadm: don't overwrite cluster logrotate file

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #48552 from rkachach/fix_issue_57878

mgr/rgw: Enabling typing checks for rgw module

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #48880 from rkachach/fix_issue_58021

mgr/cephadm: fix haproxy monitoring endpoint

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #49813 from rkachach/fix_issue_58526

mgr/cephadm: Build service discovery end-point by using mgr-ip only

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #49827 from adk3798/remove-offline-host-ingress

mgr/cephadm: fix removing offline hosts with ingress daemons

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

rgw/notifications: add const to APIs when possible

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

rgw: Remove (most) uses of null_yield in datalog

The only ones remaining are in calls from top-level threads.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw/notifications: avoid unnecessary heap allocations

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

rgw/pubsub: remove subscription data structures

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #49760 from kalebskeithley/ceph-rgw-zipper

rgw: refactor selected files for better above- vs below-the-line

Merge pull request #48828 from yuvalif/wip-yuval-cython-fails-clang16

rgw/file: change function signature to match the one generated by cython

reviewed-by: cbodley, mattbenjamin

Merge pull request #49530 from yuvalif/wip-yuval-signed-comparisson

rgw: fix signed to unsigned comparisson warning

reviwed by: mattbenjamin

Merge pull request #49642 from cbodley/wip-58286

rgw/beast: fix interaction between keepalive and 100-continue

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

rgw: refactor selected files for better above- vs below-the-line

Move more files into driver/rados for better above- vs below-
the-line. Use #pragma once everywhere (versus fixing all the

Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>

Merge pull request #49820 from zdover23/wip-doc-2023-01-21-rados-config-pool-pg-config-ref

doc/rados: refine pool-pg-config-ref.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

Merge pull request #49778 from zdover23/wip-doc-2023-01-18-rados-config-ceph-conf-runtime-changes

doc/rados: refine ceph-conf.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: refine ceph-conf.rst

Correct grammar and usage in ceph-conf.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@gmail.com>

mgr/rgw: Adding typing checks for rgw
Fixes: https://tracker.ceph.com/issues/57878
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

mgr/cephadm: fix haproxy monitoring endpoint
Fixes: https://tracker.ceph.com/issues/58021
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

qa/s3tests: run with tox instead of bootstrap+nose

Fixes: https://tracker.ceph.com/issues/58512
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49735 from linuxkidd/wip-rgw-gap-list

rgw/rgw-gap-list: refactoring and adding more error checking

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #49315 from tengfWang/master

rgw: fix the problem of duplicate idx when bi list

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #47743 from cbodley/wip-rgw-replication-status

rgw: add replication status header to s3 GetObj response

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

qa/s3tests: add tox task before every s3tests task

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #48801 from adk3798/extra-entrypoint-args

mgr/cephadm: support for extra entrypoint args

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #48827 from rkachach/fix_issue_58001

mgr/cephadm: fix prometheus haproxy/ingress deps update

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #49556 from senax/haproxy

mgr/cephadm: increase ingress timeout values

Reviewed-by: Adam King <adking@redhat.com>

mgr/cephadm: fix removing offline hosts with ingress daemons

We need to convert the daemon type to service type here
or we'll try to access the "haproxy" or "keepalived" index
of the cephadm_services dictionary, which doesn't have those
keys (but does have an "ingress" key)

Fixes: https://tracker.ceph.com/issues/58537
Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #46209 from phlogistonjohn/jjm-format-nfs-mod

mgr/nfs: use object_format decorators to simplify response handling

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #49189 from pcuzner/improve-orch-host-ls

orchestrator: improvements to the orch host ls command

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #49823 from zdover23/wip-doc-2023-01-22-install-recommended-methods-rewrite

doc/install: refine index.rst

doc/install: refine index.rst

Refine English sentences in doc/install/index.rst. Remove adverbial
phrases of time that refer to Nautilus-era features as "new", since that
was four years ago.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/rados: refine pool-pg-config-ref.rst

Remove pleonasm.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49817 from ivancich/wip-flight-copyright-license-info

rgw: add copyright & licensing to flight integration source files

Reviewed-By: Adam Emerson <aemerson@redhat.com>

mgr/cephadm: call iscsi post_remove from server loop

If we call it directly during a synchronous command
from the command line it never returns.

Fixes: https://tracker.ceph.com/issues/58531
Signed-off-by: Adam King <adking@redhat.com>

rgw: add copyright & licensing to flight integration source files

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

cephadm: don't overwrite cluster logrotate file

To allow users to customize the file if they need to
akter the settings (e.g. to change the rotation interval
or rotate based on size)

Fixes: https://tracker.ceph.com/issues/58527
Signed-off-by: Adam King <adking@redhat.com>

mgr/cephadm: Build service discovery end-point by using mgr-ip only
Fixes: https://tracker.ceph.com/issues/58526
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #49791 from idryomov/wip-doc-exclusive-lock-transitions

doc/rbd/rbd-exclusive-locks: warn about automatic lock transitions

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #49784 from SUSE/wip-fix-58501

ceph.spec.in: Replace %usrmerged macro with regular version check

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #49764 from zdover23/wip-doc-2023-01-17-ceph-volume-lvm-encryption-refine

doc/ceph-volume: refine encryption.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rbd/rbd-exclusive-locks: warn about automatic lock transitions

A lot of people aren't aware of automatic lock transitions and
wrongfully assume that exclusive lock means that the image remains
locked for as long as the client is running.  Redo the explanation
and add a warning.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

doc/rbd/rbd-exclusive-locks: don't mention "profile rbd" requirement twice

It's (much better) described in the Blocklisting section.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rgw: fix the problem of duplicate idx when bi list

fix: https://tracker.ceph.com/issues/58213

Signed-off-by: wangtengfei <wangtengfei@inspur.com>

Merge pull request #49798 from adamemerson/wip-radosgw-cr-test-build-fix

build: Fix build of `radosgw-cr-test`

Reviewed-by: Casey Bodley <cbodley@redhat.com>

build: Fix build of `radosgw-cr-test`

Remove dependency on `libneorados_cls_fifo` removed in commit
`bf222906b01cbb9e8d55ae4685004a22cddcc138`.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw/rgw-gap-list: refactor / add more err checks

- rename function `log_out` to `log` to prevent mental confusion with
  'logging out' of the shell, vs sending a log message.

- Centralize error checking to provide more consistent output and ease
  of error check implementation

- Add check that supplied pool name(s) exist.

- Relocate the awk script creation to just before it's needed.

- Provide usage help for specifying multiple pools on the command line

- Add hostname to log output lines.

- add checks to ensure intermediate files are not empty before
  attempting comparison.

- switch to getopt to allow  more option flexibility, removing
  dependency on positional arguments which could cause parsing / user
  confusion.
  This is a **BREAKING change** due to change in command line syntax.

- add option for temp directory specification on the command line.

- set -m option to be presence based, vs needing 0|1 param

- fix typo in usage

Signed-off-by: Michael J. Kidd <linuxkidd@gmail.com>

qa/openssl_keys: 'rm' ignores file not found

on distros with openssl 3, the .srl files don't appear to be created.
don't fail if 'rm' can't find them

Fixes: https://tracker.ceph.com/issues/58513
Signed-off-by: Casey Bodley <cbodley@redhat.com>

tools/cephfs/first-damage: fix iteration of big dirfrags

We have to iterate in chunks otherwise the get_omap_vals API returns
partial results even if the max count is large.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #48898 from ceph/wip-rgw-multisite-reshard-reef

rgw: multisite stabilization for reef

Reviewed-By: Adam Emerson <aemerson@redhat.com>

Merge pull request #48848 from stefan-chivu/rbd-wnbd-live-resize

rbd: live resize support for rbd-wnbd

Reviewed-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/ceph-volume: refine encryption.rst

Improve the word choice and grammar of
doc/ceph-volume/lvm/encryption.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49280 from stefan-chivu/dokan-file-dir-access

dokan: Made file/dir access mode configurable

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #49759 from rhcs-dashboard/fix-pg-metrics

mgr/prometheus: export zero valued pg state metrics

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

ceph.spec.in: Replace %usrmerged macro with regular version check

%usrmerged is deprecated in favour of %if 0%{?suse_version} < 1550
(see https://en.opensuse.org/openSUSE:Usr_merge for details)

Fixes: https://tracker.ceph.com/issues/58501
Signed-off-by: Tim Serong <tserong@suse.com>