test/librbd: fix group_info.name leaks in TestGroup.add_image

Introduced in commit c04e8c3374e0 ("test/librbd: add get_group test").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #44020 from adk3798/host-ls-scale

mgr/orchestrator: add filtering and count option for orch host ls

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44412 from cyx1231st/wip-crimson-improve-log

crimson: leverage --default-log-level to configure the global log level

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44473 from johnbent/patch-1

README.md: Update README.md to add link to tracker.ceph.com

Merge pull request #43833 from melissa-kun-li/prevent-cephadm-stray-daemon-tcmurunner

mgr/cephadm: fix tcmu-runner cephadm_stray_daemon

Merge PR #44054 into master

* refs/pull/44054/head:
doc/rados/operations: document pg_num_max
mgr: set max of 32 pgs for .mgr pool
mgr/dashboard: expect pg_num_max property for pools
mon/OSDMonitor: add option --pg-num_max arg for create pool
mon/OSDMonitor: disallow setting pg_num < min or > max
mgr/pg_autoscaler: apply pg_num_max
mon: add pg_num_max pool property

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #44157 from CongMinYin/pwl-small-fixes

librbd/cache/pwl: code cleanup and some minor fixes

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

librbd/cache/pwl: merge multiple output "<<" calls

Follow the principle of fewer "<<" calls and higher performance.
Merge multiple output "<<" calls.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

Merge pull request #44475 from athanatos/sjust/wip-53783

crimson/os/seastore/cache: fix debug macro usage

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: fix debug macro usage

Introduced via conflict between 277e57 and 632916.

Fixes: https://tracker.ceph.com/issues/53783
Signed-off-by: Samuel Just <sjust@redhat.com>

README.md: Update README.md to add link to tracker.ceph.com

I searched the existing documentation for a link to the tracker and had a hard time finding it. Other folks like myself might like it so prominently displayed.
Added a link to https://tracker.ceph.com/projects/ceph as https://tracker.ceph.com/ is basically blank and not totally intuitive how to get to issues.

Signed-off-by: John Bent <john.bent@seagate.com>

Merge pull request #44429 from cyx1231st/wip-crimson-improve-log-2

crimson/os/seastore: differentiate loggers in seastore

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44420 from aisakaki/lru-cache

crimson/os/seastore/cache: add lru

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #44474 from ljflores/wip-doc-integration-tests

doc/.../testing_integration_tests: fix formatting of teuthology-suite option

Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/.../testing_integration_tests: fix formatting of teuthology-suite option

The `--subset` teuthology-suite option should be formatted.

Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #39192 from linuxbox2/wip-lc-notify

rgwlc:  optionally support notifications on object expiration

Merge pull request #44341 from sebastian-philipp/cleanup-snmp

mgr/cephadm: Cleanup snmp

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44000 from nmshelke/exception-handling-48473

mgr/stats: exception handling for ceph fs perf stats command

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44291 from vshankar/wip-ignore-invalid-client-op

mds: ignore unknown client op when tracking op latency

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44279 from jtlayton/wip-50235

cephfs-shell: add the ability to mount a named filesystem

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44158 from adk3798/agent-failed

mgr/cephadm: don't include agents in CEPHADM_FAILED_DAEMON

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44388 from rhcs-dashboard/02-host-inventory-fix

mgr/dashboard: fix orchestrator/02-hosts-inventory.e2e failure

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44193 from adk3798/agent-upgrade

mgr/cephadm: speed up upgrade when using agent

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44266 from adk3798/http-reconfig

mgr/cephadm: reconfig agents over http

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

python-common: move test_valid_snmp_gateway_spec from mgr/cephadm

We have to validate to_json() now as well, as we have spcial enums.
Otherwiese we might end up with !!python... representations.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

python-common: move test_invalid_snmp_gateway_spec from mgr/cephadm

Let's keep the tests in the same package where the class is defined.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: SNMP: don't write urls manually

this just broken for non-trivial urls. Don't be a bad example

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: SNMP: Don't write default values into the store

Enable us to chage defaults in the future

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: SNMP: use of python3 enums

Little reason to duplicate things ourselves

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44367 from sebastian-philipp/osd-count-fix-again

mgr/cephadm: Fix count for OSDs with OSD specs

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44374 from fmount/ingress_vip

mgr/cephadm: spec.virtual_ip  param should be used by the ingress daemon

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44442 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-simplebar-angular-2.3.6

mgr/dashboard: bump simplebar-angular from 2.3.0 to 2.3.6 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44361 from xxhdx1985126/wip-53612

crimson/os/seastore/cache: add lba nodes' logical begin addresses to extent_info_t

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

qa: test cases for ceph fs perf stats command

Fixes: https://tracker.ceph.com/issues/48473
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>

mgr/stats: exception handling for ceph fs perf stats command

Fixes: https://tracker.ceph.com/issues/48473
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>

Merge pull request #44458 from cyx1231st/wip-seastore-cleanup-with-empty-transactions

crimson/os/seastore: cleanup with empty transactions

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44268 from lxbsz/mds-fix1

mds: set hb to NULL after the memory released

Reviewed-by: Venky Shankar <vshankar@redhat.com>

crimson/os/seastore: cleanup with empty transactions

* Add logs to identify OSD operation that submits transaction;
* Mark empty transactions as debug log;
* Misc cleanup;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #44329 from ljflores/wip-telemetry-last-additions

mgr/telemetry: add heap stats and pool application to the perf channel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #43716 from kamoltat/wip-ksirivad-autoscale-global-flag

mgr/autoscaler: Introduce noautoscale flag

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

rgwlc: format event.eventName with to_event_string(...)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlc: add integration tests for lc triggered bucket notifications

Signed-off-by: yuval Lifshitz <ylifshit@redhat.com>

rgwlc: temporary logic to compensate for rgw::sal::Bucket::get_owner() always nullptr

(Acked by Daniel.)  Now with less use-after-free.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #44433 from sebastian-schmid/master

docs/start: fixes typo in hardware recommendation docs

Merge pull request #43580 from rhcs-dashboard/cephadm_box-docs

cephadm/box: DiD box documentation

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: sebastian-philipp <sewagner@redhat.com>

Merge pull request #43763 from rhcs-dashboard/cephadm_box

cephadm/box: Cephadm Docker in Docker dev box

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44449 from rhcs-dashboard/api-docs-e2e-test

mgr/dashboard: add test coverage for API docs (SwaggerUI)

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

docs/start: fixes typo and empty headline in hardware recommendation docs

Signed-off-by: Sebastian Schmid <mail@sebastian-schmid.de>

Merge pull request #44428 from rhcs-dashboard/e2e-improvement

mgr/dashboard: stabilizing the cephadm dashboard e2e

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #44453 from sebastian-philipp/mypy-cheroot-import-ssl-fix

mgr/dashboard: cheroot now ships type hints

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

notify: fix a use-after-move crash, unconstify res_id

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlc:  harmonize lifecycle notifications with the others

After working with notification configurations, it seemed to make
more sense to organize the different lifecycle notification sub-types
along the same lines as the official AWS ones.  So, there are now
ObjectExpirationCurrent and ObjectExpirationNoncurrent types, and
an ObjectExpiration wildcard.  Similarly, ObjectTransition
contains Current and Noncurrent sub-types.

Note that I have defined an ObjectExpirationAbortMPU notification
sub-type, but do not currently generate it--this is to avoid changing
the rgw::sal interface, but most likely we would support this in
future.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgwlc:  optionally support notifications on object expiration

Most of the work is to remove direct knowledge of req_state from
methods in rgw_notify.

I've chosen to create new notification types matching the different
expire actions (but not transition).  The new event types are not
nested under Delete.  Notifications are sent iff rgw_lc_notify is true
(default false).

Adjusted per comments in initial review, in particular, notification from
lifecycle is no longer conditional on a config setting, and constness
is restored.

Fixes: https://tracker.ceph.com/issues/49068
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

mgr/dashboard: stabilizing the cephadm dashboard e2e

Reordering the tests and adding some more tests to verify the cluster is
healthy before proceeding to do some complex tasks like maintenance and
drain host

Fixes: https://tracker.ceph.com/issues/53742
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: cheroot now ships type hints

Now we need to prevent mypy frm complainign about a duplicated import.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44067 from MrFreezeex/dirfrags-test

qa/suites/fs: add prefetch_dirfrags false to thrasher suite

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44255 from lxbsz/syncfs-44916

client: fix sync fs to force flush mdlog for all sessions

Reviewed-by: Venky Shankar <vshankar@redhat.com>

cephadm/box: remove unnecessary bash files

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #43618 from kotreshhr/recover-symlink

mds: Store symlink target in first data object

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44246 from yongseokoh/fix-prefetch-dirfrags

mds: split _prefetch_dirfrags to avoid heartbeat timeout during up:rejoin

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #43236 from mchangir/mgr/snap_schedule-fix-db-connection-concurrent-usage

mgr/snap_schedule: fix db connection concurrent usage

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc/rados/operations: document pg_num_max

Signed-off-by: Sage Weil <sage@newdream.net>

mgr: set max of 32 pgs for .mgr pool

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/dashboard: expect pg_num_max property for pools

Signed-off-by: Sage Weil <sage@newdream.net>

mon/OSDMonitor: add option --pg-num_max arg for create pool

Signed-off-by: Sage Weil <sage@newdream.net>

mon/OSDMonitor: disallow setting pg_num < min or > max

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/pg_autoscaler: apply pg_num_max

Signed-off-by: Sage Weil <sage@newdream.net>

mon: add pg_num_max pool property

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/dashboard: bump simplebar-angular

Bumps simplebar-angular from 2.3.0 to 2.3.6.

---
updated-dependencies:
- dependency-name: simplebar-angular
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #44407 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/swagger-ui-4.1.3

build(deps): bump swagger-ui from 4.1.2 to 4.1.3 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/dashboard: add test coverage for API docs (SwaggerUI)

Fixes: https://tracker.ceph.com/issues/53756
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #44354 from tchaikov/cmake-cleanup

cmake: use new CMP0127 policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43337 from ljflores/wip-bluestore-zero-detection

os/bluestore: detect unnecessary zeroes in bufferlist when data is written

Merge pull request #44187 from rhcs-dashboard/wip-configure_dependabot-master

mgr/dashboard: configure dependabot

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44130 from LittleFox94/wip-doc-rbd-default-object-size

doc: how to change default rbd object size

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mds: ignore unknown client op when tracking op latency

Server::handle_client_request() ignores unknown client operation
by returning -ENOTSUPP, however, Server::perf_gather_op_latency()
aborts on unknown client op, thereby causing -ENOTSUPP to never
reach the client. ceph_abort() seems unnecessary here.

Note, we could have invoked Server::perf_gather_op_latency()
when the return value to client is not -ENOTSUPP, however,
a valid client operation *might* just return -ENOTSUPP in
some cases.

@mchangir ran into this with his getvxattr op changes (PR #42001).

Signed-off-by: Venky Shankar <vshankar@redhat.com>

librbd/cache/pwl/: remove IO waited state

IO waited state variables including waited_lanes, waited_entries,
waited_buffers are only used to print logs and have no practical
effect.no_space = true is enough to indicate state.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

doc/rbd: change persistent write-back cache to persistent write log cache

The full name of PWL is persistent write log, and the full name of
RWL is replica write log. Change the title to make it consistent
with the name of the feature and better reflect its design.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

librbd/cache/pwl: clean code and update comments

some comments are outdated, some are no longger needed, update them.
remove redundant ";". remove redundant log. remove redundant "".

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

mgr/dashboard: fix orchestrator/02-hosts-inventory.e2e failed.

I removed the `02-hosts-inventory.e2e` file because it is a duplicate
test of one of the test in the `01-hosts.e2e` file and fixed the error
from that file.

Also, in the inventory Identify test, we test for an element to be not
visible. According to the latest cypress docs, this should be not.exist
instead of not.visible since the cd-modal will not even be present in
the DOM

Fixes: https://tracker.ceph.com/issues/53499
Signed-off-by: Nizamudeen A <nia@redhat.com>

librbd/cache/pwl/rwl: add pmemobj_close() during error handle

After create/open pmem device, add pmemobj_close() during error handle.
Merge the same error handle to the end of function.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

crimson/os/seastore: differentiate loggers in seastore

To enable fine-grain logger control for easier bug triage.

See common/subsys.h for the new classifications.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: clean up logging

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: clean up LRU

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>

Merge pull request #44398 from rhcs-dashboard/e2e-script-improvement

mgr/dashboard: dashboard cephadm e2e improvement

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

test/crimson: leverage --default-log-level to configure the global log level

So we can have freedom to customize global/individual log levels during
manual tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

mgr/dashboard: dashboard cephadm e2e improvement

Fixes: https://tracker.ceph.com/issues/53742
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44403 from cyx1231st/wip-fix-crimson-args

crimson/osd: fix and improve args processing

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

crimson/os/seastore/cache: trim after adding to lru

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>

crimson/os/seastore/cache: add lru

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #44396 from cyx1231st/wip-seastore-fix-seastar-runner

crimson/test: fix SeastarRunner when app is not started

Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44274 from fengchunsong/dpdk-dfx

msg/async/dpdk: add commands to obtain the NIC status and statistics

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

msg/async/dpdk:add commands to obtain the NIC status and statistics

Commands are added to obtain the network adapter status and statistics
for debugging network adapter packet loss and mbuf insufficiency issues.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe <hanfengzhe@hisilicon.com>

Merge pull request #44276 from fengchunsong/dpdk-affinity

common/numa: Skip the DPDK thread when setting NUMA affinity

doc: add docs for rbd_default_order

Adds documentation how to change default rbd object size. With the
previous option `--order` it was easy to guess the config name for the
default value, with the current option name `--object-size` thats hard
to guess.

Also extends the documentation for rbd_default_order to include
* how object-size is derived from the configured value
* allowed range of the value

In the first version of this commit I also added min and max for this
parameter (12/25, matching the object size range in `man 8
rbd`/Striping/object-size), but this made some tests fail, since some
seem to set values outside this range (and probably are fine since
included for some time already). To have this a doc-change only, I
removed the range.

Signed-off-by: Mara Sophie Grosch <littlefox@lf-net.org>

Merge PR #44342 into master

* refs/pull/44342/head:
mds: trigger stray reintegration when loading dentry
qa: test that scrub causes reintegration

Reviewed-by: Xiubo Li <xiubli@redhat.com>

common/numa: Skip the DPDK thread when setting NUMA affinity

The CPU affinity of the DPDK thread has been set during DPDK initialization.
Do not modify the DPDK affinity when setting NUMA affinity.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe <hanfengzhe@hisilicon.com>

msg/async: refactory rename_thread for DPDKStack

The thread_name of the DPDK thread has been set during DPDK initialization.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe <hanfengzhe@hisilicon.com>

Merge pull request #44243 from ideepika/wip-deepika-iscsi

qa/rbd: update the cephadm required distro

Reviewed-by: Melissa Li <melissali@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

crimson/osd: print message when customize parameters

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>