doc: deprecate blkin tracing

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55267 from athos-ribeiro/cmake-empty-submodules

cmake: quote variable to allow empty values

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #55398 from ifed01/wip-ifed-fix-ceph-vol-migrate-wal

ceph-volume: Fix migration from WAL to data with no DB

Merge pull request #55416 from ivoalmeida/snapshot-schedule-edit

mgr/dashboard: added edit functionality

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #55465 from neesingh-rh/wip-63991

doc: add description of metric fields for cephfs-top

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #55479 from ronen-fr/wip-rf-fifolegacy

rgw: mark maybe_unused variables

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55475 from zdover23/wip-doc-2024-02-07-radosgw-admin-confval

doc/radosgw: add confval directives

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #55149 from rkachach/fix_issue_63992

mgr/prometheus: fix orch check to prevent Prometheus from crashing

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #54623 from alimaredia/wip-rgw-op-metrics-section-rename

rgw: change section name of rgw_op counters

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/prometheus: fix orch check to prevent Prometheus from crashing
Fixes: https://tracker.ceph.com/issues/63992
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #55476 from zdover23/wip-doc-2024-02-07-cephadm-tshooting

doc/cephadm - edit t11ing.

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/radosgw: add confval directives

Add confval directives to the documentation of "quota cache" options.
This addresses a request made by Antony D'Atri in https://github.com/ceph/ceph/pull/55075/files#r1444006246.

Signed-off-by: Zac Dover <zac.dover@proton.me>

rgw: mark maybe_unused variables

Specifically, variables that are only used in debug builds.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #54596 from cbodley/wip-cmake-dencoder-modules

cmake: ceph-dencoder depends on ceph-dencoder-modules

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #55376 from guits/guits-cv-64248

ceph-volume: fix zap_partitions() in devices.lvm.zap

mgr/dashboard: snapshot schedule edit form

Fixes: https://tracker.ceph.com/issues/64331
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

doc/cephadm - edit t11ing.

Edit "Running Repeated Debugging Sessions" in
doc/cephadm/troubleshooting.rst. Improve grammar. Fix a misspelling.

Signed-off-by: Zac Dover <zac.dover@proton.me>

doc: add description of metric fields for cephfs-top

Fixes: https://tracker.ceph.com/issues/63991
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #55444 from Matan-B/wip-crimson-seastar-upgrade

Crimson: bump up seastar version

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #54280 from xxhdx1985126/wip-rbm-scattered-allocation

crimson/os/seastore/random_block_manager: allow allocating multiple extents for a single disk space allocation request

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>

ceph.spec.in: crimson add protobuf

Following https://github.com/scylladb/seastar/commit/eea4102091bb4d689e0b5a769512966b716ee8a2

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/crush/CrushLocation: fix compilation errors

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

CMakeLists.txt: fix Seastar expected dirs

```
  # create the directory so cmake won't complain when looking at the imported
  # target: Seastar exports this directory created at build-time
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

seastar: bump up seastar submodule version

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #55054 from pereman2/zns-remove

os/bluestore: remove zoned namespace support
It has never been finished and now its in the way of future improvements.

Merge pull request #55275 from qiuxinyidian/rgw-noti-dev

rgw: add topic owner user check when creating

reviewed-by: cbodley, kchheda3

Merge pull request #55447 from zdover23/wip-doc-2024-02-05-rados-operations-pools

doc/rados: update PG guidance

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

rgw: modify topic owner check when creating

add tests to cover topic policies
as well as behavior when no policies are defined

Fixes: https://tracker.ceph.com/issues/64124
Signed-off-by: Zhipeng Li <qiuxinyidian@gmail.com>

Merge pull request #54311 from gabriel-samfira/do-not-escape-slash

RGW: fix cloud-sync not being able to sync folders

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55252 from jzhu116-bloomberg/wip-21800

rgw/multisite: avoid writing multipart parts to the bucket index log

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55293 from jzhu116-bloomberg/wip-63178

rgw/lc: pass in flag as an argument when calling function complete(..)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54661 from lightmelodies/rgw-gc

rgw: optimize gc chain size calculation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55452 from nbalacha/wip-typo-fix

rbd-mirror: fix a typo in NotifyAckPayload::dump()

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #55406 from cbodley/wip-64255

rgw: SiteConfig::load() falls back to local zonegroup

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #55448 from guits/subclass-thread

node-proxy: refactor entrypoint

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #55280 from ronen-fr/wip-rf-extra-lg

osd: remove superfluous '>' in spg_t formatter

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #55414 from yuvalif/wip_yuval_url_decode_sns

rgw/rest: fix url decode of post params passed as attributes

reviewed-by: cbodley

rbd-mirror: fixed a typo in instance_watcher

Fixes a typo in NotifyAckPayload::dump().

Signed-off-by: N Balachandran <nibalach@redhat.com>

node-proxy: refactor entrypoint

This commit introduces a major refactor of the main
entrypoint.

- subclass threading.Thread:
  - Introduce a new class `BaseThread()` that is a
    `threading.Thread()` abstraction class in order
    to monitor the different threads.
  - `BaseSystem()` inherits from `BaseThread()`.
  - Handle `SIGTERM` signal in order to gracefully shutdown
    node-proxy (make threads exit gracefully, log out from RedFish API, etc.)

Additionally, this:
  - drops the class `Logger()` from util.py which
was not adding value. It is now replaced with a simple `get_logger()`
function.
  - changes the node-proxy API port from 8080 to 9456
    (8080 being widely used for frontend apps...)
  - changes the container entrypoint in order to use the
    `ceph-node-proxy` binary from the packaging

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

doc/rados: update PG guidance

Update the "Creating a Pool" section of doc/rados/operations/pools.rst
so that the documentation no longer insists that the user change the
values of "osd_pool_default_pg_num" and "osd_pool_default_pgp_num".

See also: https://github.com/ceph/ceph/pull/55419

Tracker: https://tracker.ceph.com/issues/64259

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

crimson/os/seastore/transaction_manager: allow allocating multiple
extents when rewriting data extents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/object_data_handler: allow alloc multiple extents
for a single request

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/transaction_manager: fix errorator mismatch

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

test/crimson/seastore: add unittest cases for scattered allocation

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

test/crimson/seastore: test avlallocator's scattered allocation

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/transaction_manager: TransactionManager::alloc_extent(s) may return enospc

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: add new interfaces to alloc multiple extents for a
single request

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #55089 from cbodley/wip-boost-mirrors

make-dist: update boost mirrors

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #55434 from anthonyeleven/rgwyamlin

src/common/options: Correct typo in rgw.yaml.in

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #55276 from Matan-B/wip-matanb-crimson-alienstore-cores

crimson: rework CPU pinning

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #55419 from zdover23/wip-doc-2024-02-02-rados-config-pool-pg-config-ref

doc/rados: update config for autoscaler

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

src/common/options: Correct typo in rgw.yaml.in

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

doc/rados: update config for autoscaler

Update doc/rados/configuration/pool-pg-config-ref.rst to account for the
behavior of autoscaler.

Previously, this file was last meaningfully altered in 2013, prior to
the invention of autoscaler. A recent confusion was brought to my
attention on the Ceph Slack whereby a user attempted to alter the
default values of a Quincy cluster, as suggested in this documentation.
That alteration caused Ceph to throw the error "Error ERANGE: 'pgp_num'
must be greater than 0 and lower or equal than 'pg_num', which in this
case is one" and a related "rgw_init_ioctx ERROR" reading in part
"Numerical result out of range". The user removed the
"osd_pool_default_pgp_num" configuration line from ceph.conf and the
cluster worked as expected. I presume that this is because the removal
of this configuration line allowed autoscaler to work as intended.

Fixes: https://tracker.ceph.com/issues/64259
Co-authored-by: David Orman <ormandj@corenode.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

rgw: SiteConfig::load() falls back to local zonegroup

allow radosgw-admin commands like 'user create' to operate on a new zone
that hasn't been committed to the period yet. this follows similar logic
in RGWSI_Zone::do_start()

Fixes: https://tracker.ceph.com/issues/64255
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54312 from dvanders/dvanders_mcltf_true

qa/tasks/cephadm: enable mon_cluster_log_to_file

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #54837 from imtzw/urlencode

rgw: fix list obj v1 nextmarker

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54867 from BBoozmen/oozmen_lc_subsystem

rgw/common: add rgw lifecycle specific debug log subsystem

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Jiffin Tony Thottan <jthottan@redhat.com>

Merge pull request #55162 from cfsnyder/wip-64014-cfsnyder-pacific

rgw: fix issue with concurrent versioned deletes leaving behind olh entries

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #55229 from tobias-urdin/http-error-keystone-issue-admin-token-request

rgw: fix http error check in keystone issue_admin_token_request

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55330 from baum/centos9-install-build-deps

🧹build dependencies: centos9

Merge PR #55233 into main

* refs/pull/55233/head:
qa: use centos 9.stream for cephfs stock kernel testing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53901 from cbodley/wip-qa-container-distros-s

qa/distros: bump container host distros from centos 8->9

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #55413 from petrutlucian94/flipped_tid

msg: update MOSDOp() to use ceph_tid_t instead of long

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

crimson: introduce crimson_seastar_cpu_cores

seastar::smp::count is deduced from the cpuset provided.

from seastar/src/core/reactor:
```
   if (smp_opts.smp) {
        smp::count = smp_opts.smp.get_value();
    } else {
        smp::count = cpu_set.size();
    }
```

Co-authored-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>

rgw/rest: fix url decode of post params passed as attributes

this is fixing a regression with the SNS policies
cause by: 4bdc5d18dd68b95c6ccd4c0e77a1bd04ad86dbb8

the changes to the test code is to accomodate different
boto3 versions

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

msg: update MOSDOp() to use ceph_tid_t instead of long

The MOSDOp constructor receives the the transaction ID as a long
instead of ceph_tid_t.

The issue is that "long" uses 32b on Windows instead of 64 bits,
so it flips after about 2 billion requests. At that point, the OSD
replies are dropped because of transaction ID mismatches.

We'll solve the issue by using the correct type for the transaction
id, specifically ceph_tid_t.

Fixes: https://tracker.ceph.com/issues/64291
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #55373 from ivoalmeida/snapshot-schedule-form-directory-typeahead

Snapshot schedule form directory typeahead

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #55393 from rkachach/fix_issue_64266

mgr/rook: adding deployment to ceph image built for rook e2e testing

vstarh.sh: Crimson - set crimson_alien_thread_cpu_cores

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/alienstore: cleanup N_CORES_FOR_SEASTAR constant

No CPU pinning if crimson_alien_thread_cpu_cores is not set.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

mgr/rook: adding deployment to ceph image built for rook e2e testing
Fixes: https://tracker.ceph.com/issues/64266
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

crimson/os/seastore/avlallocator: add the interface that allocate
multiple regions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

rgw: change topic label to lowercase

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: change section name of rgw_op counters

The rgw_op section of `counter dump/schema` becomes:
- rgw_op_global for the global op counters
- rgw_op_per_user for the user labeled counters
- rgw_op_per_bucket for the bucket labeled counters

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #47723 from mdw-at-linuxbox/wip-master-tempurl

rgw: swift: tempurl fixes for ceph

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55403 from cbodley/wip-doc-rgw-topic-attrs

doc/rgw: fix Attributes index in CreateTopic example

Merge pull request #54742 from guits/node-proxy

orch: implement hardware monitoring

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #55397 from jagombar/fixInvalidSnapId2

librbd: return ENOENT from Snapshot::get_timestamp for nonexistent snap_id

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rgw: fix Attributes index in CreateTopic example

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: added dir search to snap schdl form

Fixes: https://tracker.ceph.com/issues/64246
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

Merge pull request #54781 from rhcs-dashboard/nvmeof-integration

mgr/dashboard: implement APIs for nvmeof management

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #54900 from ivoalmeida/snapshot-schedule-create

added snap schedule form

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #55340 from ronen-fr/wip-rf-reserver2

osd/scrub: use an AsyncReserver to handle scrub reservations on the replica side

Reviewed-by: Samuel Just <sjust@redhat.com>-

Merge pull request #55198 from JoshSalomon/balancer-perf

osd: Read balancer for devices with different sizes (performance)

doc: add node-proxy documentation

This commit adds some documentation about the
'hardware inventory / monitoring' feature (node-proxy agent).

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #55396 from zdover23/wip-doc-2024-01-31-arch-monitors

doc/architecture: improve some paragraphs

Merge pull request #55383 from ceph/cpp-redis-cxx-standard

cpp_redis: update to a version with the CMAKE_CXX_STANDARD fixed

Reviewed-by: Ali Maredia <amaredia@redhat.com>

ceph-volume: Fix migration from WAL to data with no DB

Fixes: https://tracker.ceph.com/issues/64260
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>

ceph-volume: fix zap_partitions() in devices.lvm.zap

The current logic is unable to find the holders for the
partition being zapped.

This commit fixes this issue.

Fixes: https://tracker.ceph.com/issues/64248
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

librbd: return ENOENT from Snapshot::get_timestamp for nonexistent snap_id

Fixes: https://tracker.ceph.com/issues/47287
Signed-off-by: John Agombar <agombar@uk.ibm.com>

doc/architecture: improve some paragraphs

Improve paragraphs under the heading "The Ceph Storage Cluster". Remove
a sentence that was pleonastic in its context in the paragraph.

Signed-off-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: added snap schedule form

Fixes: https://tracker.ceph.com/issues/63827
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

Merge pull request #55305 from Matan-B/wip-matanb-crimson-do_delete_work

crimson/osd/pg: do_delete_work use claim_and_reset

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

cpp_redis: update to a version with the CMAKE_CXX_STANDARD fixed

This updates the cpp_redis submodule to a version
which contains https://github.com/ceph/cpp_redis/pull/5

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>

test: fix test_scrub_sched following recent scrub changes

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: restore 'dump_scrub_reservations' functionality

As ScrubResources is no longer involved in remote reservations, some
of the data listed by 'dump_scrub_reservations' is now collected by
OsdScrub itself (prior to this change, OsdScrub just forwarded the
request to ScrubResources).

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: control reservation queueing using a config option

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: directly manage remote reservations in the FSM

The FSM now interacts with the scrub_reserver directly.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

vstart: add nvmeof_gw to the vstart script

so that it can be later used by the dashboard to configure the nvmeof
through UI

and create rbd pool in UI

Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #55379 from ljflores/wip-revert-labler-changes