]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
21 months agomon/AuthMonitor: check if entity is absent before creating it 53405/head
Rishabh Dave [Thu, 28 Sep 2023 17:34:51 +0000 (23:04 +0530)]
mon/AuthMonitor: check if entity is absent before creating it

Although this code path is not used for creating entities yet, it is
better to fix the bug sooner than later. Method
AuthMonitor::_update_or_create_entity() must exit (with appropriate
error code) when entity to be created on the Ceph cluster is already
present.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
21 months agomds/MDSAuthCaps: re-word an error message for better clarity
Rishabh Dave [Thu, 28 Sep 2023 14:29:24 +0000 (19:59 +0530)]
mds/MDSAuthCaps: re-word an error message for better clarity

Signed-off-by: Rishabh Dave <ridave@redhat.com>
21 months agomon/AuthMonitor: revise a comment
Rishabh Dave [Thu, 28 Sep 2023 14:26:01 +0000 (19:56 +0530)]
mon/AuthMonitor: revise a comment

Signed-off-by: Rishabh Dave <ridave@redhat.com>
21 months agomon/AuthMonitor: make "ceph auth caps" print error messsages...
Rishabh Dave [Mon, 11 Sep 2023 20:01:37 +0000 (01:31 +0530)]
mon/AuthMonitor: make "ceph auth caps" print error messsages...

generated by MDSAuthCaps::parse(). Specifically, in this case, this
command is supposed to print an error message when permission flag passed
for MDS capability is incorrect.

This needs the method AuthMonitor::_update_or_create_entity() to be
fixed. The issue is that stderr stream is not passed from the block under
"else if" clause of "ceph auth caps" command to MDSAuthCaps::_parse()
method. Let's fix this by adding a parameter for stderr stream to every
method in stack that is between this block and MDSAuthCaps::parse()
method.

qa.tasks.cephfs.test_admin.TestPermErrMsg contains tests that checks
whether or not all the Ceph commands that accept MDS capabilites print
the error message when permission flag in this capaibility is incorrect.
Add a test to this class to check if "ceph auth caps" also print an error
message when perm flag in MDS capability sting is incorrect.

Fixes: https://tracker.ceph.com/issues/63020
Signed-off-by: Rishabh Dave <ridave@redhat.com>
21 months agoMerge pull request #52524 from dparmar18/wip-62072
Rishabh Dave [Fri, 22 Sep 2023 14:45:04 +0000 (20:15 +0530)]
Merge pull request #52524 from dparmar18/wip-62072

tools/cephfs_mirror: only perform actions if init succeed

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge PR #53407 into main
Venky Shankar [Fri, 22 Sep 2023 14:41:49 +0000 (20:11 +0530)]
Merge PR #53407 into main

* refs/pull/53407/head:
Client: Fix nonblocking-io zero by read

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge PR #53279 into main
Venky Shankar [Fri, 22 Sep 2023 14:41:12 +0000 (20:11 +0530)]
Merge PR #53279 into main

* refs/pull/53279/head:
qa: descrease pgbench scale factor to 32 for postgresql database test

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
21 months agoMerge PR #52747 into main
Venky Shankar [Fri, 22 Sep 2023 14:40:42 +0000 (20:10 +0530)]
Merge PR #52747 into main

* refs/pull/52747/head:
cephfs-mirror: restart failed/blocklisted replayer instances
cephfs-mirror: do not run concurrent C_RestartMirroring context

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
21 months agoMerge PR #53418 into main
Venky Shankar [Fri, 22 Sep 2023 14:39:44 +0000 (20:09 +0530)]
Merge PR #53418 into main

* refs/pull/53418/head:
qa: use exisitng ignorelist override list for fs:mirror[-ha]

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
21 months agoMerge pull request #48027 from lxbsz/wip-57154
Rishabh Dave [Fri, 22 Sep 2023 14:38:10 +0000 (20:08 +0530)]
Merge pull request #48027 from lxbsz/wip-57154

mds/client: check the cephx mds auth access in client side

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
21 months agoMerge pull request #52576 from cfsnyder/wip-62075
Casey Bodley [Fri, 22 Sep 2023 14:33:25 +0000 (15:33 +0100)]
Merge pull request #52576 from cfsnyder/wip-62075

rgw: add radosgw-admin bucket check olh/unlinked commands

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #52042 from rishabh-d-dave/better-errmsg-for-perm
Rishabh Dave [Fri, 22 Sep 2023 14:32:40 +0000 (20:02 +0530)]
Merge pull request #52042 from rishabh-d-dave/better-errmsg-for-perm

MDSAuthCaps: print better error message for perm flag in MDS caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #47533 from rishabh-d-dave/ceph-fuse-cmd
Rishabh Dave [Fri, 22 Sep 2023 14:31:13 +0000 (20:01 +0530)]
Merge pull request #47533 from rishabh-d-dave/ceph-fuse-cmd

qa/cephfs: cleanup code for ceph-fuse command

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
21 months agoMerge pull request #52259 from cbodley/wip-61859
Casey Bodley [Fri, 22 Sep 2023 14:08:36 +0000 (15:08 +0100)]
Merge pull request #52259 from cbodley/wip-61859

qa/rgw: disable 'rgw lc debug interval' in most suites

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Soumya Koduri <skoduri@redhat.com>
21 months agoMerge pull request #53531 from ronen-fr/wip-rf-squeue2 53263/head
Ronen Friedman [Fri, 22 Sep 2023 13:46:38 +0000 (16:46 +0300)]
Merge pull request #53531 from ronen-fr/wip-rf-squeue2

osd/scrub: extract scrub initiation code out of the OSD

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53311 from idryomov/wip-62711
Ilya Dryomov [Fri, 22 Sep 2023 13:44:35 +0000 (15:44 +0200)]
Merge pull request #53311 from idryomov/wip-62711

qa/suites/{rbd,krbd}: disable POOL_APP_NOT_ENABLED health check

Reviewed-by: Ramana Raja <rraja@redhat.com>
21 months agorgw: radosgw-admin bucket check should only print index entries with --check-objects... 52576/head
Cory Snyder [Fri, 22 Sep 2023 08:35:16 +0000 (08:35 +0000)]
rgw: radosgw-admin bucket check should only print index entries with --check-objects flag

Printing all index entries can be very time consuming for large
buckets and the inability to switch this behavior off makes it
cumbersome to use the command for fixing bucket stats. This was
also preventing the command from outputting recalculated bucket
stats when the --fix flag wasn't specified.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
21 months agoMerge pull request #53528 from rishabh-d-dave/cephfs-qa-mdtest
Rishabh Dave [Fri, 22 Sep 2023 06:26:03 +0000 (11:56 +0530)]
Merge pull request #53528 from rishabh-d-dave/cephfs-qa-mdtest

qa/cephfs: fix build failure for mdtest project

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoqa/rgw: enable s3tests for 'storage classes' 52259/head
Casey Bodley [Thu, 21 Sep 2023 14:59:42 +0000 (10:59 -0400)]
qa/rgw: enable s3tests for 'storage classes'

other than the rgw/lifecycle subsuite, all of the other subsuites were just
adding storage classes to the rgw task, but not the s3tests task that
enables the storage class test cases

Signed-off-by: Casey Bodley <cbodley@redhat.com>
21 months agoqa/rgw: disable 'rgw lc debug interval' in most suites
Casey Bodley [Thu, 29 Jun 2023 14:54:03 +0000 (10:54 -0400)]
qa/rgw: disable 'rgw lc debug interval' in most suites

this has a performance cost, and the teuthology environment can
already be sluggish. if lifecycle expiration/transition testing
was limited to the rgw/lifecycle and rgw/cloud-transition suites,
the others would probably run significantly faster

Fixes: https://tracker.ceph.com/issues/61859
Signed-off-by: Casey Bodley <cbodley@redhat.com>
21 months agoqa/s3tests: use cloudtier_tests and lifecycle_tests to control filters
Casey Bodley [Thu, 29 Jun 2023 14:40:16 +0000 (10:40 -0400)]
qa/s3tests: use cloudtier_tests and lifecycle_tests to control filters

when `extra_attrs` was made additive, these tests were adding both
'not lifecycle_expiration' and 'lifecycle_expiration' to the command
line. use a yaml flag instead to control whether or not the s3tests task
adds the 'not' filter

Signed-off-by: Casey Bodley <cbodley@redhat.com>
21 months agoqa/rgw: remove duplicate overrides in rgw/lifecycle
Casey Bodley [Thu, 29 Jun 2023 14:44:02 +0000 (10:44 -0400)]
qa/rgw: remove duplicate overrides in rgw/lifecycle

Signed-off-by: Casey Bodley <cbodley@redhat.com>
21 months agoqa/suites/krbd: disable POOL_APP_NOT_ENABLED health check 53311/head
Ilya Dryomov [Fri, 15 Sep 2023 13:33:27 +0000 (15:33 +0200)]
qa/suites/krbd: disable POOL_APP_NOT_ENABLED health check

... same as for rbd suite.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
21 months agoqa/suites/rbd: drop POOL_APP_NOT_ENABLED from ignorelists
Ilya Dryomov [Fri, 15 Sep 2023 13:33:27 +0000 (15:33 +0200)]
qa/suites/rbd: drop POOL_APP_NOT_ENABLED from ignorelists

With "mon warn on pool no app = false" in the config, it's obviously
redundant.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
21 months agoqa/suites/rbd: disable POOL_APP_NOT_ENABLED health check
Ilya Dryomov [Fri, 15 Sep 2023 13:33:27 +0000 (15:33 +0200)]
qa/suites/rbd: disable POOL_APP_NOT_ENABLED health check

Commit 990806e635a1 ("mon, qa: issue pool application warning even
if pool is empty") made it impossible to create a pool without raising
a (bogus) health alert.  See [1] for details.

[1] https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/ZTDYC5HN677RR26EB4P6PORN6L2IFH4R/

Fixes: https://tracker.ceph.com/issues/62711
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
21 months agoMerge pull request #53448 from cbodley/wip-62378
Casey Bodley [Thu, 21 Sep 2023 21:15:37 +0000 (22:15 +0100)]
Merge pull request #53448 from cbodley/wip-62378

rgw/crypt: don't deref null manifest_bl

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
21 months agoMerge pull request #53045 from smanjara/wip-shilpa-revert-51772
Casey Bodley [Thu, 21 Sep 2023 19:59:59 +0000 (20:59 +0100)]
Merge pull request #53045 from smanjara/wip-shilpa-revert-51772

rgw/multisite: fixes assertion failure during realm reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agorgw: prevent another leftover bucket index olh entry scenario
Cory Snyder [Thu, 21 Sep 2023 19:27:51 +0000 (19:27 +0000)]
rgw: prevent another leftover bucket index olh entry scenario

If a call to bucket_index_link_olh or bucket_index_unlink_instance
fails, its associated pending xattr may have prevented the olh object
from being removed by another thread. We should do a best effort
cleanup attempt for this case by calling update_olh before returning
an error to the caller.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
22 months agoMerge pull request #53493 from zdover23/wip-doc-2023-09-18-architecture-7-of-x
zdover23 [Thu, 21 Sep 2023 17:41:21 +0000 (03:41 +1000)]
Merge pull request #53493 from zdover23/wip-doc-2023-09-18-architecture-7-of-x

doc/architecture: "Edit HA Auth" (one of several)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agoMerge pull request #53505 from cbodley/wip-62771
Casey Bodley [Thu, 21 Sep 2023 17:30:07 +0000 (18:30 +0100)]
Merge pull request #53505 from cbodley/wip-62771

rgw/sal: get_placement_target_names() returns void

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
22 months agoMerge pull request #52392 from yuvalif/wip-yuval-trace-name
Yuval Lifshitz [Thu, 21 Sep 2023 17:12:01 +0000 (20:12 +0300)]
Merge pull request #52392 from yuvalif/wip-yuval-trace-name

rgw: rename request traces and change for tags

reviewed-by: cbodley

22 months agoMerge PR #50503 into main
Patrick Donnelly [Thu, 21 Sep 2023 15:51:31 +0000 (11:51 -0400)]
Merge PR #50503 into main

* refs/pull/50503/head:
mon: do not change pending if strategy is unchanged
mon/MonmapMonitor: do not propose on error in prepare_update
mon/MonmapMonitor: wait for commit before reply
mon: use wait_for_commit to reply
mon: add context list for commit wait
mon: remove unused method
test/mon: add commit benchmark script
mon/MonClient: provide config to target specific rank

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
22 months agodoc/architecture: "Edit HA Auth" (one of several) 53493/head
Zac Dover [Sun, 17 Sep 2023 20:41:28 +0000 (06:41 +1000)]
doc/architecture: "Edit HA Auth" (one of several)

Edit "High Availability Authentication" in doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoosd/scrub: modify schedule_result_t to report error class 53531/head
Ronen Friedman [Thu, 21 Sep 2023 12:34:52 +0000 (07:34 -0500)]
osd/scrub: modify schedule_result_t to report error class

(which directly translates to the required followup action)
instead of reporting the exact failure. The specific of the failure
were never used by the scrub scheduler.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: scheduler refactoring - cleanups
Ronen Friedman [Thu, 21 Sep 2023 09:57:30 +0000 (04:57 -0500)]
osd/scrub: scheduler refactoring - cleanups

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoqa/cephfs: add "set -x" in mdtest.yaml 53528/head
Rishabh Dave [Thu, 21 Sep 2023 12:41:41 +0000 (18:11 +0530)]
qa/cephfs: add "set -x" in mdtest.yaml

Set the flag for printing the commands that will be executed so that
it's easier to go through teuthology.log

Signed-off-by: Rishabh Dave <ridave@redhat.com>
22 months agoqa/cephfs: fix build failure for mdtest project
Rishabh Dave [Wed, 20 Sep 2023 08:42:43 +0000 (14:12 +0530)]
qa/cephfs: fix build failure for mdtest project

To fix the mdtest job failure (which happens because building mdtest
project fails) do -

1. Use ior projects intead of mdtest project bcecause latter was merged
   into former. See:
   https://github.com/MDTEST-LANL/mdtest/blob/master/README.md

2. Purge mpich package and then install it again. This is a vital step
   that's needed to build ior project on Ubuntu 22.04.

Fixes: https://tracker.ceph.com/issues/61574
Signed-off-by: Rishabh Dave <ridave@redhat.com>
22 months agoMerge pull request #52863 from batrick/i62326
Adam King [Thu, 21 Sep 2023 12:14:54 +0000 (08:14 -0400)]
Merge pull request #52863 from batrick/i62326

pybind/mgr/cephadm/upgrade: stop disabling FSMap sanity checks

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoosd/scrub: scheduler: removing unused code
Ronen Friedman [Thu, 21 Sep 2023 09:59:11 +0000 (04:59 -0500)]
osd/scrub: scheduler: removing unused code

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: correct placement for some scheduler-related methods
Ronen Friedman [Tue, 19 Sep 2023 14:04:23 +0000 (09:04 -0500)]
osd/scrub: correct placement for some scheduler-related methods

Moving some member functions to their corresponding files.
Including ScrubQueue::dump_scrubs()
as it was moved in a previous commit,
and some ScrubJob code.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoMerge pull request #53501 from zhscn/wip-lba-backref-node-size
Yingxin [Thu, 21 Sep 2023 09:20:28 +0000 (17:20 +0800)]
Merge pull request #53501 from zhscn/wip-lba-backref-node-size

crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
22 months agoMerge pull request #52150 from paulreece42/wip-grafana-quorum-fix
Nizamudeen A [Thu, 21 Sep 2023 07:06:21 +0000 (12:36 +0530)]
Merge pull request #52150 from paulreece42/wip-grafana-quorum-fix

monitoring: grafana mons out of quorum should be count - sum

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
22 months agoMerge PR #53028 into main
Venky Shankar [Thu, 21 Sep 2023 00:41:54 +0000 (06:11 +0530)]
Merge PR #53028 into main

* refs/pull/53028/head:
Update MDSDaemon.cc
Update MDSRank.cc - Logoutput: Fix personal pronoun "I" to uppercase

Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge PR #53176 into main
Venky Shankar [Thu, 21 Sep 2023 00:33:42 +0000 (06:03 +0530)]
Merge PR #53176 into main

* refs/pull/53176/head:
doc: add note for removing (automatic) partitioning policy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
22 months agoMerge pull request #53467 from phlogistonjohn/jjm-cephadm-move-get_data_dir
Adam King [Wed, 20 Sep 2023 18:20:09 +0000 (14:20 -0400)]
Merge pull request #53467 from phlogistonjohn/jjm-cephadm-move-get_data_dir

cephadm: move get data dir function to daemonidentity method

Reviewed-by: Adam King <adking@redhat.com>
22 months agoMerge pull request #53415 from rkachach/fix_issue_62814
Adam King [Wed, 20 Sep 2023 18:19:02 +0000 (14:19 -0400)]
Merge pull request #53415 from rkachach/fix_issue_62814

cephadm: fix cephadm binary mount when --shared_ceph_folder is used

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
22 months agoMerge pull request #53298 from phlogistonjohn/jjm-logging-2
Adam King [Wed, 20 Sep 2023 18:17:03 +0000 (14:17 -0400)]
Merge pull request #53298 from phlogistonjohn/jjm-logging-2

cephadm: enhance logging behavior

Reviewed-by: Adam King <adking@redhat.com>
22 months agoMerge pull request #52251 from rkachach/fix_issue_61856
Adam King [Wed, 20 Sep 2023 18:14:23 +0000 (14:14 -0400)]
Merge pull request #52251 from rkachach/fix_issue_61856

mgr/cephadm: Adding sort-by support for ceph orch ps

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
22 months agoMerge pull request #52982 from batrick/backport-cp-simplify
Ilya Dryomov [Wed, 20 Sep 2023 17:58:40 +0000 (19:58 +0200)]
Merge pull request #52982 from batrick/backport-cp-simplify

script/ceph-backport: perform cherry-pick in single command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge PR #53145 into main
Patrick Donnelly [Wed, 20 Sep 2023 12:57:07 +0000 (08:57 -0400)]
Merge PR #53145 into main

* refs/pull/53145/head:
mds: log message when exiting due to asok command

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
22 months agoMerge PR #53149 into main
Patrick Donnelly [Wed, 20 Sep 2023 12:31:17 +0000 (08:31 -0400)]
Merge PR #53149 into main

* refs/pull/53149/head:
qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
22 months agodoc: add note for removing (automatic) partitioning policy 53176/head
Venky Shankar [Mon, 28 Aug 2023 10:42:57 +0000 (16:12 +0530)]
doc: add note for removing (automatic) partitioning policy

Signed-off-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge pull request #53309 from guits/bz2203397
Guillaume Abrioux [Wed, 20 Sep 2023 07:32:49 +0000 (09:32 +0200)]
Merge pull request #53309 from guits/bz2203397

ceph-volume: fix mpath device support

22 months agoosd/scrub: handle configuration changes in OsdScrub
Ronen Friedman [Tue, 19 Sep 2023 12:16:03 +0000 (07:16 -0500)]
osd/scrub: handle configuration changes in OsdScrub

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: move initiate_a_scrub() to OsdScrub
Ronen Friedman [Tue, 19 Sep 2023 11:55:25 +0000 (06:55 -0500)]
osd/scrub: move initiate_a_scrub() to OsdScrub

Scrub initiation is now fully owned by OsdScrub.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: add ScrubQueue::ready_to_scrub()
Ronen Friedman [Mon, 18 Sep 2023 16:31:37 +0000 (11:31 -0500)]
osd/scrub: add ScrubQueue::ready_to_scrub()

At this phase of the refactoring:
this is the main interface from the scrub scheduler in OsdScrub
to the ScrubQueue. The ScrubQueue provides the ordered list of
all targets (for now - PGs) that are ready for scrubbing.

Scrub initiation code is modified to use the new interface.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: move scrub_sleep_time() to OsdScrub
Ronen Friedman [Mon, 18 Sep 2023 15:11:21 +0000 (10:11 -0500)]
osd/scrub: move scrub_sleep_time() to OsdScrub

also scrub_time_permit().

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: extract restrictions_on_scrubbing()
Ronen Friedman [Mon, 18 Sep 2023 11:00:34 +0000 (06:00 -0500)]
osd/scrub: extract restrictions_on_scrubbing()

from ScrubQueue::select_pg_and_scrub().

Clearing the path to moving some ScrubQueue methods into
OscScrub. Starting here with the CPU load tracker.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: introduce OsdService::get_locked_pg()
Ronen Friedman [Sat, 16 Sep 2023 16:53:35 +0000 (11:53 -0500)]
osd/scrub: introduce OsdService::get_locked_pg()

which returns an RAII-wrapper around a locked PG.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: move OSD::sched_scrub() to OsdScrub
Ronen Friedman [Fri, 15 Sep 2023 14:03:09 +0000 (09:03 -0500)]
osd/scrub: move OSD::sched_scrub() to OsdScrub

... (as OsdScrub::initiate_scrub()).

The random backoff dice roller (scrub_random_backoff())
is moved as well.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: declare OsdScrub, an OSD subobject
Ronen Friedman [Thu, 14 Sep 2023 14:09:43 +0000 (09:09 -0500)]
osd/scrub: declare OsdScrub, an OSD subobject

for all OSD scrub things.

For now: OsdScrub is mostly a forwarder to the ScrubQueue object
(which it now owns).
The resource counters moved into a separate object within OsdScrub.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: unify dout macros across scrub scheduling code
Ronen Friedman [Wed, 13 Sep 2023 07:09:14 +0000 (02:09 -0500)]
osd/scrub: unify dout macros across scrub scheduling code

to facilitate easy migration of code fragments between
related classes.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: complete ScrubJob transition from within ScrubQueue
Ronen Friedman [Mon, 11 Sep 2023 06:56:33 +0000 (01:56 -0500)]
osd/scrub: complete ScrubJob transition from within ScrubQueue

ScrubJob is now in the Scrub namespace.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: set_reserving_now() signature modified
Ronen Friedman [Sun, 10 Sep 2023 19:44:33 +0000 (14:44 -0500)]
osd/scrub: set_reserving_now() signature modified

set_reserving_now() can now return a failure status, indicating
a race between two PGs to start scrubbing on the same OSD.

The scrubber FSM is modified to handle the failure.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: select_pg_and_scrub() moved into osd_scrub.cc
Ronen Friedman [Sun, 10 Sep 2023 18:50:35 +0000 (13:50 -0500)]
osd/scrub: select_pg_and_scrub() moved into osd_scrub.cc

No code changes.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: renaming & fmt support for restrictions structure
Ronen Friedman [Sun, 10 Sep 2023 18:32:53 +0000 (13:32 -0500)]
osd/scrub: renaming & fmt support for restrictions structure

Renaming ScrubPreconds, the collection of "environmental"
restrictions on possible scrubs, to OSDRestrictions.
Also - providing fmtlib support for that structure.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: introducing random_bool_with_probability()
Ronen Friedman [Sun, 10 Sep 2023 18:07:31 +0000 (13:07 -0500)]
osd/scrub: introducing random_bool_with_probability()

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: moving the resources counters code into a separate file
Ronen Friedman [Sun, 10 Sep 2023 17:55:59 +0000 (12:55 -0500)]
osd/scrub: moving the resources counters code into a separate file

No code changes in this commit.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: moving code as-is into osd_scrub.cc
Ronen Friedman [Sun, 10 Sep 2023 15:14:39 +0000 (10:14 -0500)]
osd/scrub: moving code as-is into osd_scrub.cc

Code from OSD.cc & osd_scrub_sched.cc moved as-is,
to be modified in followup commits.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agoosd/scrub: moving ScrubJob declaration as-is
Ronen Friedman [Sat, 9 Sep 2023 19:44:00 +0000 (14:44 -0500)]
osd/scrub: moving ScrubJob declaration as-is

from osd_scrub_sched.h into scrub_job.h.

A purely mechanical step. No code is changed.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
22 months agocrimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent 53501/head
Zhang Song [Tue, 19 Sep 2023 06:08:51 +0000 (14:08 +0800)]
crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
22 months agodoc/cephadm: document new cephadm logging destination settings 53298/head
John Mulligan [Wed, 6 Sep 2023 20:56:40 +0000 (16:56 -0400)]
doc/cephadm: document new cephadm logging destination settings

Add docs for setting the binary's log destination at cephadm bootstrap
or on a running cluster.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agodoc/cephadm: clarify what cephadm component writes to the cluster log channel
John Mulligan [Wed, 6 Sep 2023 20:18:37 +0000 (16:18 -0400)]
doc/cephadm: clarify what cephadm component writes to the cluster log channel

Clarify that the cephadm orchestrator module, a part of the ceph mgr,
logs to the cluster log channel. This prepares for adding a specific
section to cover logging for the cephadm "binary".

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: remember log destination used during bootstrap
John Mulligan [Wed, 6 Sep 2023 18:15:41 +0000 (14:15 -0400)]
cephadm: remember log destination used during bootstrap

Store the log destination(s) specified on the CLI for cephadm bootstrap
as the manager configuration, unless the configuration key is explicitly
set by the input config.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agomgr/cephadm: add a module option for controlling cephadm log dest
John Mulligan [Wed, 6 Sep 2023 17:39:06 +0000 (13:39 -0400)]
mgr/cephadm: add a module option for controlling cephadm log dest

Now that cephadm has multiple possible persistent logging destinations
we need a way to choose which one to use when the command is started by
the mgr. Add the option 'cephadm_log_destination' which can take one
of 'file', 'syslog', or 'file,syslog'. If left unset (empty string)
then the behavior is equivalent to 'file' and that is the same as
previous cephadm versions.

Fixes: https://tracker.ceph.com/issues/62233
Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: add cli option to enable logging to syslog
John Mulligan [Tue, 22 Aug 2023 19:11:35 +0000 (15:11 -0400)]
cephadm: add cli option to enable logging to syslog

Add the --log-dest option to cephadm. The --log-dest option can be
specified 0, 1 or more times. If unspecified, cephadm will log to
the default location, the log file. If specified one ore more times,
each instance will enable the named logging destination.
Example:

```
cephadm boostrap

cephadm --log-dest=syslog bootstrap

cephadm --log-dest=file bootstrap

cephadm --log-dest=syslog --log-dest=file bootstrap
```

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: add support for logging to syslog/journal
John Mulligan [Tue, 22 Aug 2023 19:11:16 +0000 (15:11 -0400)]
cephadm: add support for logging to syslog/journal

Add support to logging.py for persistent logging to syslog and thus to
journald. This is accomplished by switching logging handlers depending
on the log_dest attribute of the context. Setting this value is left
for a future patch.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agocephadm: move colored output support into logging.py
John Mulligan [Tue, 22 Aug 2023 16:42:14 +0000 (12:42 -0400)]
cephadm: move colored output support into logging.py

Rewrite cephadm's colored output support such that it abstracts away
the colorization into extra logging metadata. The new code will not
unconditionally put control characters into the log files. It will
only print the control chars if the stderr is a tty.
In theory this is probably more future proof as well, but it's only
got two callers so it is hard to say how useful it'll be.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
22 months agorgw/crypt: don't deref null manifest_bl 53448/head
Casey Bodley [Wed, 13 Sep 2023 20:30:03 +0000 (16:30 -0400)]
rgw/crypt: don't deref null manifest_bl

with dbstore, the manifest_bl pointer was null; check for null before
dereferencing for read_manifest_parts()

Fixes: https://tracker.ceph.com/issues/62378
Signed-off-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge pull request #51921 from kamoltat/wip-ksirivad-fix-54136
Kamoltat (Junior) Sirivadhna [Tue, 19 Sep 2023 18:12:35 +0000 (14:12 -0400)]
Merge pull request #51921 from kamoltat/wip-ksirivad-fix-54136

pybind/mgr/pg_autoscaler: Use bytes_used for actual_raw_used
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
22 months agoMerge pull request #53532 from zdover23/wip-doc-2023-09-19-man-ceph-monstore-tool
Ilya Dryomov [Tue, 19 Sep 2023 14:35:18 +0000 (16:35 +0200)]
Merge pull request #53532 from zdover23/wip-doc-2023-09-19-man-ceph-monstore-tool

doc/man: s/kvstore-tool/monstore-tool/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
22 months agodoc/man: s/kvstore-tool/monstore-tool/ 53532/head
Zac Dover [Tue, 19 Sep 2023 13:12:34 +0000 (23:12 +1000)]
doc/man: s/kvstore-tool/monstore-tool/

s/kvstore-tool/monstore-tool/ in accordance with Ilya's remark here:
https://github.com/ceph/ceph/pull/53450#discussion_r1329804085

Signed-off-by: Zac Dover <zac.dover@proton.me>
22 months agoMerge pull request #53411 from rhcs-dashboard/align-charts
Nizamudeen A [Tue, 19 Sep 2023 12:38:29 +0000 (18:08 +0530)]
Merge pull request #53411 from rhcs-dashboard/align-charts

mgr/dashboard: align charts of landing page

Reviewed-by: Nizamudeen A <nia@redhat.com>
22 months agoceph-volume: fix mpath device support 53309/head
Guillaume Abrioux [Wed, 6 Sep 2023 09:30:41 +0000 (09:30 +0000)]
ceph-volume: fix mpath device support

commit [1] broke mpath devices support in `disk.is_device()`

[1] https://github.com/ceph/ceph/commit/4fc6bc394dffaf3ad375ff29cbb0a3eb9e4dbefc

Fixes: https://tracker.ceph.com/issues/62722
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
22 months agoMerge pull request #53230 from myoungwon/fix-cbj-overflow-bug
Yingxin [Tue, 19 Sep 2023 08:03:34 +0000 (16:03 +0800)]
Merge pull request #53230 from myoungwon/fix-cbj-overflow-bug

crimson/os/seastore/cbj: fix a potential overflow bug on segment_seq

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
22 months agoMerge PR #52631 into main
Milind Changire [Tue, 19 Sep 2023 07:51:09 +0000 (13:21 +0530)]
Merge PR #52631 into main

* refs/pull/52631/head:
mds: add debug logs to monitor ceph.dir.subvolume management
mds: dump subvolume flag for inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge PR #52892 into main
Milind Changire [Tue, 19 Sep 2023 07:23:29 +0000 (12:53 +0530)]
Merge PR #52892 into main

* refs/pull/52892/head:
qa: add test to validate periodic checks by async threads
mgr/volumes: periodically check for async work

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
22 months agoMerge PR #52755 into main
Milind Changire [Tue, 19 Sep 2023 07:22:48 +0000 (12:52 +0530)]
Merge PR #52755 into main

* refs/pull/52755/head:
mds: adjust pre_segments_size for MDLog when trimming segments for standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
22 months agoMerge PR #52692 into main
Milind Changire [Tue, 19 Sep 2023 07:21:52 +0000 (12:51 +0530)]
Merge PR #52692 into main

* refs/pull/52692/head:
qa/tasks/cephfs: reset the client_inject_fixed_oldest_tid after test

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge pull request #53518 from likid0/patch-2
zdover23 [Mon, 18 Sep 2023 23:04:02 +0000 (09:04 +1000)]
Merge pull request #53518 from likid0/patch-2

doc/dev: Fix typos in cephfs-mirroring.rst and  deduplication.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agodoc/dev: Fix typos in files cephfs-mirroring.rst and deduplication.rst 53518/head
Daniel Parkes [Mon, 18 Sep 2023 21:03:28 +0000 (23:03 +0200)]
doc/dev: Fix typos in files cephfs-mirroring.rst and deduplication.rst

Typo Error in Doc cephfs-mirroring.rst , replace RAODS with RADOS
Typo Error in Doc deduplication.rst , replace RAODS with RADOS

Signed-off-by: Daniel Parkes <dparkes@redhat.com>
22 months agoMerge pull request #53502 from dang/wip-dang-cls-test
Casey Bodley [Mon, 18 Sep 2023 22:01:31 +0000 (23:01 +0100)]
Merge pull request #53502 from dang/wip-dang-cls-test

RGW - Fix cls test build on new gcc

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge pull request #53508 from dang/wip-dang-posix-cache
Daniel Gryniewicz [Mon, 18 Sep 2023 18:36:58 +0000 (14:36 -0400)]
Merge pull request #53508 from dang/wip-dang-posix-cache

RGW - Add wait backoff to posix bucket cache test

Reviewed-by Matt Benjamin <mbenjamin@redhat.com>

22 months agoRGW - Add wait backoff to posix bucket cache test 53508/head
Daniel Gryniewicz [Mon, 18 Sep 2023 15:47:54 +0000 (11:47 -0400)]
RGW - Add wait backoff to posix bucket cache test

The CI appears to be really slow, and even a second of wait for inotify
sometimes fails.  Add an exponential backoff wait of up to ~25 seconds
to hopefully make the test pass reliably.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
22 months agorgw/sal: get_placement_target_names() returns void 53505/head
Casey Bodley [Mon, 18 Sep 2023 15:15:02 +0000 (11:15 -0400)]
rgw/sal: get_placement_target_names() returns void

the function returned an integer error code, but two callers were
incorrectly testing the return value as a boolean

the function just returns placement ids that are in-memory, so none of
the drivers have a failure case; change the return value to void

Fixes: https://tracker.ceph.com/issues/62771
Signed-off-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge pull request #53478 from EdwardVitor/cuiming_chinamobile
Casey Bodley [Mon, 18 Sep 2023 13:46:55 +0000 (14:46 +0100)]
Merge pull request #53478 from EdwardVitor/cuiming_chinamobile

auth:rectify a cmake compilation warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoRGW - Fix cls test build on new gcc 53502/head
Daniel Gryniewicz [Mon, 18 Sep 2023 12:50:53 +0000 (08:50 -0400)]
RGW - Fix cls test build on new gcc

The new encoder types broke building the cls test on newer gcc (13+) due
to undefined encoder/decoder.  Add the file that defines those to the
test.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
22 months agoMerge PR #53175 into main
Patrick Donnelly [Mon, 18 Sep 2023 12:37:16 +0000 (08:37 -0400)]
Merge PR #53175 into main

* refs/pull/53175/head:
qa: increase the http postBuffer size and disable sslVerify

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoauth:rectify a cmake compilation warning 53478/head
cuiming_yewu [Fri, 15 Sep 2023 06:14:39 +0000 (14:14 +0800)]
auth:rectify a cmake compilation warning

rectify src/auth/cephx/CephxProtocol.cc 1 warning
with the variable 'ch' Used before initialized
auth/cephx/CephxProtocol.cc:595:57: warning: '*((void*)& ch +8)' may be used uninitialized in this function [-Wmaybe-uninitialized]
     msg.server_challenge_plus_one = ch.server_challenge + 1;
                                     ~~~~~~~~~~~~~~~~~~~~^~~

Signed-off-by: cuiming <cuiming_yewu@cmss.chinamobile.com>
22 months agoMerge pull request #53490 from zdover23/wip-doc-2023-09-17-architecture-6-of-x
Anthony D'Atri [Sun, 17 Sep 2023 14:40:26 +0000 (10:40 -0400)]
Merge pull request #53490 from zdover23/wip-doc-2023-09-17-architecture-6-of-x

doc/architecture: "Edit HA Auth" (one of several)