osd: Change description of 'dump_op_pq_state' command to be more generic

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

common: Introduce 'mclock' debug subsystem

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #44410 from tchaikov/wip-cmake-cleanup

cmake: detect linux/blk/zoned support

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44502 from yuvalif/fix-call-to-mrgw

rgw/multisite/test: pass zero (no) ssl port to mrgw.sh

Merge pull request #44495 from Matan-B/wip-matanb-crimson-flavor

qa/suites: Crimson flavor configuration fix

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: chunmei-liu chunmei.liu@intel.com

Merge pull request #44221 from BenoitKnecht/ceph-mon-device-metrics

mon: Abort device health when device not found

Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>

Merge pull request #43362 from ybwang0211/pubsub_settingng_spell

rgw:word spelling error in rgw/rgw_pubsub.h file

Merge pull request #42115 from sandrobonazzola/patch-2

doc/install: ensure to open ceph-mon firewall ports

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

rgw/multisite/test: passzero (no) ssl port to mrgw.sh

in multisite tests script.
this is to fix an issue from: 1f3fbf218faa4859f02917828dbdb48be41a579a

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #44301 from yuvalif/wip-yuval-cloudevents

rgw/notifications: add cloudevents support to HTTP endpoint

Merge pull request #44278 from fengchunsong/dpdk-tso

msg/async/dpdk: Add an option to enable/disable TSO

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44198 from wjwithagen/fix-bluestore-get-io-size

common/blkdev: add missing stubs for OSes not Linux

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44402 from inspur-wyq/wip-pybind-mgr

pybind/mgr: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #44055 from ifed01/wip-ifed-cosmetics-for-optimize-rm

test/store_test, os/bluestore: bunch of minor improvements/cleanups

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #43993 from SMIL-Infra/cmake-systemd-cleanup

cmake: cleanups about systemd and install

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #44423 from 5cs/fix-unshare-blob-test

test: fix bluestore unshare blob test case

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #44416 from yuvalif/wip-yuval-rgw-params-in-multisite

rgw/multisite/test: allow passing rgw parameter to multisite script

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #44020 from adk3798/host-ls-scale

mgr/orchestrator: add filtering and count option for orch host ls

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

rgw/notifications: add cloudevents support to HTTP endpoint

following the cloudevents HTTP spec:
https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md
and more specifically this aws-s3 spec:
https://github.com/cloudevents/spec/blob/main/cloudevents/adapters/aws-s3.md

Signed-off-by: yuval Lifshitz <ylifshit@redhat.com>

rgw/common: make "get" methods on RGWHTTPArgs const

Signed-off-by: yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #44412 from cyx1231st/wip-crimson-improve-log

crimson: leverage --default-log-level to configure the global log level

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44473 from johnbent/patch-1

README.md: Update README.md to add link to tracker.ceph.com

Merge pull request #43833 from melissa-kun-li/prevent-cephadm-stray-daemon-tcmurunner

mgr/cephadm: fix tcmu-runner cephadm_stray_daemon

qa/suites: Fix crimson flavor configuration

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge PR #44054 into master

* refs/pull/44054/head:
doc/rados/operations: document pg_num_max
mgr: set max of 32 pgs for .mgr pool
mgr/dashboard: expect pg_num_max property for pools
mon/OSDMonitor: add option --pg-num_max arg for create pool
mon/OSDMonitor: disallow setting pg_num < min or > max
mgr/pg_autoscaler: apply pg_num_max
mon: add pg_num_max pool property

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #44157 from CongMinYin/pwl-small-fixes

librbd/cache/pwl: code cleanup and some minor fixes

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

librbd/cache/pwl: merge multiple output "<<" calls

Follow the principle of fewer "<<" calls and higher performance.
Merge multiple output "<<" calls.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

Merge pull request #44475 from athanatos/sjust/wip-53783

crimson/os/seastore/cache: fix debug macro usage

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: fix debug macro usage

Introduced via conflict between 277e57 and 632916.

Fixes: https://tracker.ceph.com/issues/53783
Signed-off-by: Samuel Just <sjust@redhat.com>

README.md: Update README.md to add link to tracker.ceph.com

I searched the existing documentation for a link to the tracker and had a hard time finding it. Other folks like myself might like it so prominently displayed.
Added a link to https://tracker.ceph.com/projects/ceph as https://tracker.ceph.com/ is basically blank and not totally intuitive how to get to issues.

Signed-off-by: John Bent <john.bent@seagate.com>

Merge pull request #44429 from cyx1231st/wip-crimson-improve-log-2

crimson/os/seastore: differentiate loggers in seastore

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44420 from aisakaki/lru-cache

crimson/os/seastore/cache: add lru

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #44474 from ljflores/wip-doc-integration-tests

doc/.../testing_integration_tests: fix formatting of teuthology-suite option

Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/.../testing_integration_tests: fix formatting of teuthology-suite option

The `--subset` teuthology-suite option should be formatted.

Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #39192 from linuxbox2/wip-lc-notify

rgwlc:  optionally support notifications on object expiration

Merge pull request #44341 from sebastian-philipp/cleanup-snmp

mgr/cephadm: Cleanup snmp

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44000 from nmshelke/exception-handling-48473

mgr/stats: exception handling for ceph fs perf stats command

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44291 from vshankar/wip-ignore-invalid-client-op

mds: ignore unknown client op when tracking op latency

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44279 from jtlayton/wip-50235

cephfs-shell: add the ability to mount a named filesystem

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44158 from adk3798/agent-failed

mgr/cephadm: don't include agents in CEPHADM_FAILED_DAEMON

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44388 from rhcs-dashboard/02-host-inventory-fix

mgr/dashboard: fix orchestrator/02-hosts-inventory.e2e failure

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44193 from adk3798/agent-upgrade

mgr/cephadm: speed up upgrade when using agent

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44266 from adk3798/http-reconfig

mgr/cephadm: reconfig agents over http

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

python-common: move test_valid_snmp_gateway_spec from mgr/cephadm

We have to validate to_json() now as well, as we have spcial enums.
Otherwiese we might end up with !!python... representations.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

python-common: move test_invalid_snmp_gateway_spec from mgr/cephadm

Let's keep the tests in the same package where the class is defined.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: SNMP: don't write urls manually

this just broken for non-trivial urls. Don't be a bad example

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: SNMP: Don't write default values into the store

Enable us to chage defaults in the future

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: SNMP: use of python3 enums

Little reason to duplicate things ourselves

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44367 from sebastian-philipp/osd-count-fix-again

mgr/cephadm: Fix count for OSDs with OSD specs

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44374 from fmount/ingress_vip

mgr/cephadm: spec.virtual_ip  param should be used by the ingress daemon

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44442 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-simplebar-angular-2.3.6

mgr/dashboard: bump simplebar-angular from 2.3.0 to 2.3.6 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44361 from xxhdx1985126/wip-53612

crimson/os/seastore/cache: add lba nodes' logical begin addresses to extent_info_t

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

qa: test cases for ceph fs perf stats command

Fixes: https://tracker.ceph.com/issues/48473
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>

mgr/stats: exception handling for ceph fs perf stats command

Fixes: https://tracker.ceph.com/issues/48473
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>

Merge pull request #44458 from cyx1231st/wip-seastore-cleanup-with-empty-transactions

crimson/os/seastore: cleanup with empty transactions

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44268 from lxbsz/mds-fix1

mds: set hb to NULL after the memory released

Reviewed-by: Venky Shankar <vshankar@redhat.com>

crimson/os/seastore: cleanup with empty transactions

* Add logs to identify OSD operation that submits transaction;
* Mark empty transactions as debug log;
* Misc cleanup;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #44329 from ljflores/wip-telemetry-last-additions

mgr/telemetry: add heap stats and pool application to the perf channel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #43716 from kamoltat/wip-ksirivad-autoscale-global-flag

mgr/autoscaler: Introduce noautoscale flag

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

rgwlc: format event.eventName with to_event_string(...)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlc: add integration tests for lc triggered bucket notifications

Signed-off-by: yuval Lifshitz <ylifshit@redhat.com>

rgwlc: temporary logic to compensate for rgw::sal::Bucket::get_owner() always nullptr

(Acked by Daniel.)  Now with less use-after-free.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #44433 from sebastian-schmid/master

docs/start: fixes typo in hardware recommendation docs

Merge pull request #43580 from rhcs-dashboard/cephadm_box-docs

cephadm/box: DiD box documentation

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: sebastian-philipp <sewagner@redhat.com>

Merge pull request #43763 from rhcs-dashboard/cephadm_box

cephadm/box: Cephadm Docker in Docker dev box

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44449 from rhcs-dashboard/api-docs-e2e-test

mgr/dashboard: add test coverage for API docs (SwaggerUI)

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

docs/start: fixes typo and empty headline in hardware recommendation docs

Signed-off-by: Sebastian Schmid <mail@sebastian-schmid.de>

Merge pull request #44428 from rhcs-dashboard/e2e-improvement

mgr/dashboard: stabilizing the cephadm dashboard e2e

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #44453 from sebastian-philipp/mypy-cheroot-import-ssl-fix

mgr/dashboard: cheroot now ships type hints

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

notify: fix a use-after-move crash, unconstify res_id

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlc:  harmonize lifecycle notifications with the others

After working with notification configurations, it seemed to make
more sense to organize the different lifecycle notification sub-types
along the same lines as the official AWS ones.  So, there are now
ObjectExpirationCurrent and ObjectExpirationNoncurrent types, and
an ObjectExpiration wildcard.  Similarly, ObjectTransition
contains Current and Noncurrent sub-types.

Note that I have defined an ObjectExpirationAbortMPU notification
sub-type, but do not currently generate it--this is to avoid changing
the rgw::sal interface, but most likely we would support this in
future.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlc:  optionally support notifications on object expiration

Most of the work is to remove direct knowledge of req_state from
methods in rgw_notify.

I've chosen to create new notification types matching the different
expire actions (but not transition).  The new event types are not
nested under Delete.  Notifications are sent iff rgw_lc_notify is true
(default false).

Adjusted per comments in initial review, in particular, notification from
lifecycle is no longer conditional on a config setting, and constness
is restored.

Fixes: https://tracker.ceph.com/issues/49068
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

mgr/dashboard: stabilizing the cephadm dashboard e2e

Reordering the tests and adding some more tests to verify the cluster is
healthy before proceeding to do some complex tasks like maintenance and
drain host

Fixes: https://tracker.ceph.com/issues/53742
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: cheroot now ships type hints

Now we need to prevent mypy frm complainign about a duplicated import.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

systemd: Set PrivateDevices=false in ceph-mon@.service

The `ceph-mon` daemon needs access to block devices to check the health of the
disk that backs its DB store (#24151).

Fixes: https://tracker.ceph.com/issues/52416
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

mon: Abort device health when device not found

If `store->get_devname()` returns an empty device name, it means it couldn't
determine the device that backs the monitor DB store directory.

This can happen if `ceph-mon` runs with `PrivateDevices=yes` in systemd, or
within a container where the host `/dev` isn't exposed.

This commit makes sure we abort trying to get the device health at that point,
and return an appropriate error.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

Merge pull request #44067 from MrFreezeex/dirfrags-test

qa/suites/fs: add prefetch_dirfrags false to thrasher suite

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44255 from lxbsz/syncfs-44916

client: fix sync fs to force flush mdlog for all sessions

Reviewed-by: Venky Shankar <vshankar@redhat.com>

cephadm/box: remove unnecessary bash files

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #43618 from kotreshhr/recover-symlink

mds: Store symlink target in first data object

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44246 from yongseokoh/fix-prefetch-dirfrags

mds: split _prefetch_dirfrags to avoid heartbeat timeout during up:rejoin

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #43236 from mchangir/mgr/snap_schedule-fix-db-connection-concurrent-usage

mgr/snap_schedule: fix db connection concurrent usage

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc/rados/operations: document pg_num_max

Signed-off-by: Sage Weil <sage@newdream.net>

mgr: set max of 32 pgs for .mgr pool

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/dashboard: expect pg_num_max property for pools

Signed-off-by: Sage Weil <sage@newdream.net>

mon/OSDMonitor: add option --pg-num_max arg for create pool

Signed-off-by: Sage Weil <sage@newdream.net>

mon/OSDMonitor: disallow setting pg_num < min or > max

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/pg_autoscaler: apply pg_num_max

Signed-off-by: Sage Weil <sage@newdream.net>

mon: add pg_num_max pool property

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/dashboard: bump simplebar-angular

Bumps simplebar-angular from 2.3.0 to 2.3.6.

---
updated-dependencies:
- dependency-name: simplebar-angular
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #44407 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/swagger-ui-4.1.3

build(deps): bump swagger-ui from 4.1.2 to 4.1.3 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/dashboard: add test coverage for API docs (SwaggerUI)

Fixes: https://tracker.ceph.com/issues/53756
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #44354 from tchaikov/cmake-cleanup

cmake: use new CMP0127 policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43337 from ljflores/wip-bluestore-zero-detection

os/bluestore: detect unnecessary zeroes in bufferlist when data is written

Merge pull request #44187 from rhcs-dashboard/wip-configure_dependabot-master

mgr/dashboard: configure dependabot

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44130 from LittleFox94/wip-doc-rbd-default-object-size

doc: how to change default rbd object size

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mds: ignore unknown client op when tracking op latency

Server::handle_client_request() ignores unknown client operation
by returning -ENOTSUPP, however, Server::perf_gather_op_latency()
aborts on unknown client op, thereby causing -ENOTSUPP to never
reach the client. ceph_abort() seems unnecessary here.

Note, we could have invoked Server::perf_gather_op_latency()
when the return value to client is not -ENOTSUPP, however,
a valid client operation *might* just return -ENOTSUPP in
some cases.

@mchangir ran into this with his getvxattr op changes (PR #42001).

Signed-off-by: Venky Shankar <vshankar@redhat.com>

librbd/cache/pwl/: remove IO waited state

IO waited state variables including waited_lanes, waited_entries,
waited_buffers are only used to print logs and have no practical
effect.no_space = true is enough to indicate state.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

doc/rbd: change persistent write-back cache to persistent write log cache

The full name of PWL is persistent write log, and the full name of
RWL is replica write log. Change the title to make it consistent
with the name of the feature and better reflect its design.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

librbd/cache/pwl: clean code and update comments

some comments are outdated, some are no longger needed, update them.
remove redundant ";". remove redundant log. remove redundant "".

Signed-off-by: Yin Congmin <congmin.yin@intel.com>