]>
git.apps.os.sepia.ceph.com Git - ceph-ci.git/log
John Mulligan [Tue, 30 Apr 2024 15:32:12 +0000 (11:32 -0400)]
mgr/smb: remove embedded join source and users and groups types
Drop the embedded versions of the join source and usrers and groups
configs from the cluster resource type. This means that potentially
sensitive information is *always* stored in a separate resource
from the less-sensitive general cluster config. Future changes can
then make security related choices about those specific resources
without worrying about having embedded variants inside the cluster.
For convenience I added a new "emtpy" users and groups source type that
can be used for testing. It defines *no* users and groups (an empty set
basically). This wouldn't be useful in a production scenario today as it
would deploy an smbd that you couldn't log into. But for unit testing
it's extremely handy, and maybe it can be used for integration testing
too.
This is a large change because lots of unit tests relied on embedded
join auths or users-and-groups. I tried to handle all the breaking
changes in one go for bisect-ability.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 30 Apr 2024 13:22:03 +0000 (09:22 -0400)]
mgr/smb: convert smb module cluster create to use linked resources
Convert the smb mgr module cli to use linked join auth and
users-and-groups resources when created on the `cluster create` command
line. These linked resources with have semi-ranodmized names and be
linked to the cluster that we're creating on the CLI.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 30 Apr 2024 13:21:48 +0000 (09:21 -0400)]
mgr/smb: add squash method to ResultGroup
The new squash method works similarly to `one` but permits multiple
resources to be reported on by moving them into the report for the
"parent" resource. This is meant for upcoming changes to `cluster
create` when it starts using linked join auths and linked
users-and-groups.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 30 Apr 2024 13:21:29 +0000 (09:21 -0400)]
mgr/smb: add handler tests for new linked_to_cluster field
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 30 Apr 2024 13:21:11 +0000 (09:21 -0400)]
mgr/smb: add rand_name function
This will be used later to create resources that are not directly named
by a user.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 30 Apr 2024 13:20:52 +0000 (09:20 -0400)]
mgr/smb: update handler to make use of linked_to_cluster fields
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 30 Apr 2024 13:19:56 +0000 (09:19 -0400)]
mgr/smb: add linked_to_cluster fields for JoinAuth and UsersAndGroups
The linked_to_cluster field will be used for resources that must be
life-cycled along side the cluster using them. This is mainly intended
for use with `cluster create` command line commands.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 29 Apr 2024 18:55:01 +0000 (14:55 -0400)]
mgr/smb: add staging area class to handler
This in-memory staging area for new resources being applied to the
config helps simplify validation.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 29 Apr 2024 18:17:12 +0000 (14:17 -0400)]
mgr/smb: extend key fetching support in internal store helper module
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 29 Apr 2024 18:14:37 +0000 (14:14 -0400)]
mgr/smb: split ConfigStore protocol listing methods out
Split the ConfigStore protocol into a new base protocol that covers the
methods needed to list keys in a store, called ConfigStoreListing.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 29 Apr 2024 14:53:32 +0000 (10:53 -0400)]
mgr/smb: move handler's resource check logic into functions
There's no need for these to be methods as they're largely
self-contained. Functions should be easier to unit-test independently in
the future too.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 29 Apr 2024 13:41:56 +0000 (09:41 -0400)]
mgr/smb: make handler's resource ordering a top-level function
There's no need for order_resources to be a method. Make order_resources
a proper function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Casey Bodley [Thu, 2 May 2024 14:58:30 +0000 (15:58 +0100)]
Merge pull request #57004 from kchheda3/wip-fix-retry
rgw/multisite-notification: retry storing bucket notification attrs for ECANCELED(ConcurrentModification) errors.
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@ibm.com>
Patrick Donnelly [Thu, 2 May 2024 14:39:57 +0000 (10:39 -0400)]
Merge PR #57207 into main
* refs/pull/57207/head:
doc/rados: s/cepgsqlite/cephsqlite/
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Adam King [Thu, 2 May 2024 12:04:30 +0000 (08:04 -0400)]
Merge pull request #57143 from adk3798/nvmeof-osd-profile-rbd
mgr/cephadm: set OSD cap for NVMEoF daemon to "profile rbd"
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Aashish Sharma [Thu, 2 May 2024 10:04:50 +0000 (15:34 +0530)]
Merge pull request #56575 from cloudbehl/ceph-cluster-json-update
monitoring/ceph-mixin: Add cluster variable to ceph-cluster.json
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Nizamudeen A [Thu, 2 May 2024 08:25:37 +0000 (13:55 +0530)]
Merge pull request #55495 from frittentheke/issue_64321
monitoring/ceph-mixin: Cleanup of variables, queries and tests (to fix showMultiCluster=True)
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Nizamudeen A [Thu, 2 May 2024 06:05:52 +0000 (11:35 +0530)]
Merge pull request #57151 from rhcs-dashboard/fix-65698-main
mgr/dashboard: exclude cloned-deleted RBD snaps
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Zac Dover [Wed, 1 May 2024 16:42:46 +0000 (02:42 +1000)]
doc/rados: s/cepgsqlite/cephsqlite/
As stated in the commit-message line, this corrects the typo "cepg" to
the correct string "ceph".
This typo was discovered by https://github.com/test-erik and this was
brought to our attention way back in
https://github.com/ceph/ceph/pull/50420.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Patrick Donnelly [Wed, 1 May 2024 15:21:09 +0000 (11:21 -0400)]
Merge PR #57183 into main
* refs/pull/57183/head:
script/ptl-tool: add switch to update existing qa ticket
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Patrick Donnelly [Wed, 1 May 2024 12:57:37 +0000 (08:57 -0400)]
Merge PR #57059 into main
* refs/pull/57059/head:
mds: do not try fragmenting or exporting a quiesced directory
mds: set/test ALL_LOCKED on fragment_dir request
mds: pass bypassfreezing to parent auth pin req
qa: add quiesce tests during fragmentation
qa: translate empty output from rank_tell to empty dict
qa: move reqid_tostr helper
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Matan Breizman [Wed, 1 May 2024 07:10:58 +0000 (10:10 +0300)]
Merge pull request #57102 from xxhdx1985126/wip-65672
crimson/osd/osdop_params:Unify OpsExecuter::user_modify and osd_op_params_t::user_modify
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Yuri Weinstein [Tue, 30 Apr 2024 23:32:33 +0000 (16:32 -0700)]
Merge pull request #56995 from athanatos/sjust/wip-65185-scrub-attr-error
osd: only call stat/getattrs once per object during deep-scrub
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Patrick Donnelly [Tue, 30 Apr 2024 20:46:06 +0000 (16:46 -0400)]
Merge PR #56997 into main
* refs/pull/56997/head:
pybind/mgr: disable sqlite3/python autocommit
qa/tasks/mgr: add tests for sqlite autocommit
qa/tasks/vstart_runner: run daemons in foreground
qa/tasks/vstart_runner: add missing poll method
qa/suites/rados/mgr: add cli/devicehealth tasks
qa: reorganize mgr unit tests
qa: use position-independent link
qa: add missing terminating newline
pybind/mgr: add killpoint for sqlite3 database setup
mgr: allow specifying module option level
mon/MgrMonitor: promote standby when unsetting down flag
mon/MgrMonitor: only drop active if exists
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Adam King [Tue, 30 Apr 2024 19:48:41 +0000 (15:48 -0400)]
Merge pull request #57144 from adk3798/stray-laundry
qa/cephadm: ignore stray daemon warning during rados_api_tests
Reviewed-by: Laura Flores <lflores@ibm.com>
Adam King [Tue, 30 Apr 2024 19:41:23 +0000 (15:41 -0400)]
Merge pull request #56961 from adk3798/agent-check-error-before-json
cephadm: have agent check for errors before json loading mgr response
Reviewed-by: <mfritch@suse.com>
Adam King [Tue, 30 Apr 2024 19:38:20 +0000 (15:38 -0400)]
Merge pull request #56933 from adk3798/service-size-cleanup
mgr/cephadm: cleanup service size logic block
Reviewed-by: Michael Fritch <mfritch@suse.com>
Adam King [Tue, 30 Apr 2024 19:36:27 +0000 (15:36 -0400)]
Merge pull request #56928 from adk3798/grafana-anon-access-dropped
python-common: handle "anonymous_access: false" in to_json of Grafana spec
Reviewed-by: Nizamudeen A <nia@redhat.com>
Adam King [Tue, 30 Apr 2024 19:32:39 +0000 (15:32 -0400)]
Merge pull request #56350 from phlogistonjohn/jjm-mgr-smb-module
mgr: add a new smb cluster & share module
Reviewed-by: Adam King <adking@redhat.com>
Adam King [Tue, 30 Apr 2024 19:30:30 +0000 (15:30 -0400)]
Merge pull request #56817 from guits/cephadm-check-apply-spec-file
cephadm: check if file exists when passing `--apply_spec`
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Adam King [Tue, 30 Apr 2024 19:23:59 +0000 (15:23 -0400)]
Merge pull request #56487 from adk3798/bootstrap-custom-alerts
cephadm: allow passing custom prometheus alerts to bootstrap
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Patrick Donnelly [Sat, 20 Apr 2024 19:57:22 +0000 (15:57 -0400)]
script/ptl-tool: add switch to update existing qa ticket
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Ernesto Puerta [Tue, 30 Apr 2024 08:51:23 +0000 (10:51 +0200)]
mgr/dashboard: exclude cloned-deleted RBD snaps
Fixes: https://tracker.ceph.com/issues/65698
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
Patrick Donnelly [Tue, 30 Apr 2024 16:22:41 +0000 (12:22 -0400)]
Merge PR #57044 into main
* refs/pull/57044/head:
qa: increase debugging for snap_schedule
Reviewed-by: Milind Changire <mchangir@redhat.com>
Patrick Donnelly [Tue, 30 Apr 2024 16:22:15 +0000 (12:22 -0400)]
Merge PR #57043 into main
* refs/pull/57043/head:
client: clear resend_mds only after sending request
Reviewed-by: Milind Changire <mchangir@redhat.com>
Patrick Donnelly [Tue, 30 Apr 2024 16:21:20 +0000 (12:21 -0400)]
Merge PR #57020 into main
* refs/pull/57020/head:
mds/cache: don't assume non-auth xlocks to be remote locks
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Patrick Donnelly [Tue, 30 Apr 2024 16:20:17 +0000 (12:20 -0400)]
Merge PR #56935 into main
* refs/pull/56935/head:
mds: encode flags for all inode types
qa: test file inode with F_QUIESCE_BLOCK is replicated
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Tue, 30 Apr 2024 16:19:31 +0000 (12:19 -0400)]
Merge PR #56934 into main
* refs/pull/56934/head:
mds: move drop_locks to directly after rdonly check
qa: test quiesce.block is replicated
qa: test that ceph.dir.subvolume is replicated properly
mds: add debug "lock path" command
qa: move reqid_tostr helper
qa: return run_shell process for waiters
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Tue, 30 Apr 2024 16:18:45 +0000 (12:18 -0400)]
Merge PR #56923 into main
* refs/pull/56923/head:
qa: extend rank 1 lockup for test_quiesce_authpin_wait
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Anthony D'Atri [Tue, 30 Apr 2024 14:10:17 +0000 (10:10 -0400)]
Merge pull request #57148 from zdover23/wip-doc-2024-04-30-cephadm-services-monitoring
doc/cephadm: add default monitor images
Zac Dover [Tue, 30 Apr 2024 12:24:10 +0000 (22:24 +1000)]
Merge pull request #57153 from rkhudov/doc-man-8-radosgw-admin_add-get-lifecycle-config-command
doc/man/8/radosgw-admin: add get lifecycle command
Reviewed-by: Zac Dover <zac.dover@proton.me>
Zac Dover [Tue, 30 Apr 2024 12:17:49 +0000 (22:17 +1000)]
Merge pull request #57154 from cityofships/main
doc/rados/operations: rephrase OSDs peering
Reviewed-by: Zac Dover <zac.dover@proton.me>
Ilya Dryomov [Tue, 30 Apr 2024 12:03:00 +0000 (14:03 +0200)]
Merge pull request #55978 from xxhdx1985126/wip-librbd-unittest-sparsify
test/librbd/test_internal: change to check for 4KB-aligned sparse read
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Piotr Parczewski [Tue, 30 Apr 2024 10:56:44 +0000 (12:56 +0200)]
doc/rados/operations: rephrase OSDs peering
Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
Leonid S. Usov [Tue, 30 Apr 2024 10:53:46 +0000 (13:53 +0300)]
Merge pull request #55823 from ceph/wip-lusov-quiescer
qa: cephfs quiesce thrasher
rkhudov [Tue, 30 Apr 2024 10:00:36 +0000 (11:00 +0100)]
doc/man/8/radosgw-admin: add get lifecycle command
Signed-off-by: rkhudov <rhudov@gmail.com>
Matan Breizman [Tue, 30 Apr 2024 09:29:59 +0000 (12:29 +0300)]
Merge pull request #56806 from xxhdx1985126/wip-65399
crimson/osd/pg_recovery: backoff if the recovery/backfill is deferred
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Matan Breizman [Tue, 30 Apr 2024 09:29:16 +0000 (12:29 +0300)]
Merge pull request #56998 from xxhdx1985126/wip-65568
crimson/osd/pg: only kick off snap trimming when PGs are clean
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Yingxin [Tue, 30 Apr 2024 08:40:50 +0000 (16:40 +0800)]
Merge pull request #57128 from cyx1231st/wip-fix-ut-seastore-timeout
crimson/os/seastore: misc cleanups and fixes
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Zac Dover [Tue, 30 Apr 2024 07:09:05 +0000 (17:09 +1000)]
doc/cephadm: add default monitor images
Add a list of default monitor images to the documentation. This commit
is made in response to a request from Eugen Block, and is made using the
information developed by Mr Block here:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/QGC66QIFBKRTPZAQMQEYFXOGZJ7RLWBN/.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Matan Breizman [Tue, 30 Apr 2024 06:50:45 +0000 (09:50 +0300)]
Merge pull request #56892 from xxhdx1985126/wip-crimson-urgent-recovery-version
crimson/osd/osd_operations/client_request: retrieve the correct version for objects to be recovered urgently
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Matan Breizman [Tue, 30 Apr 2024 06:45:57 +0000 (09:45 +0300)]
Merge pull request #56610 from Matan-B/wip-matanb-crimson-replicated-head-obc
crimson/osd/replicated_recovery_backend: Fix recovery obc usage
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Yingxin [Tue, 30 Apr 2024 01:52:17 +0000 (09:52 +0800)]
Merge pull request #57135 from xxhdx1985126/wip-seastore-checksum-assertion-fix
crimson/os/seastore/transaction_manager: remove incorrect assertions
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin [Tue, 30 Apr 2024 01:49:43 +0000 (09:49 +0800)]
Merge pull request #57129 from xxhdx1985126/wip-65585
crimson/os/seastore/transaction_manager: fix write pipeline phase leak
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2024 18:33:14 +0000 (14:33 -0400)]
mds: do not try fragmenting or exporting a quiesced directory
And handle inode becoming quiesced after op is created.
Fixes: https://tracker.ceph.com/issues/65603
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Mon, 29 Apr 2024 14:51:39 +0000 (10:51 -0400)]
mds: set/test ALL_LOCKED on fragment_dir request
This is an optimization to obviate repeated calls to acquire_locks.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2024 17:50:19 +0000 (13:50 -0400)]
mds: pass bypassfreezing to parent auth pin req
Credit to Leonid for first noticing this.
Fixes: https://tracker.ceph.com/issues/65603
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2024 19:49:42 +0000 (15:49 -0400)]
qa: add quiesce tests during fragmentation
Reproduces: https://tracker.ceph.com/issues/65603
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Leonid Usov [Sat, 23 Mar 2024 15:29:28 +0000 (08:29 -0700)]
qa/suites/fs: add quiescer to the fs suite
Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
Leonid Usov [Sat, 16 Mar 2024 15:42:11 +0000 (11:42 -0400)]
qa/tasks: the quiescer task and a waiter task to test it
Fixes: https://tracker.ceph.com/issues/63669
Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 24 Apr 2024 20:14:31 +0000 (16:14 -0400)]
qa: translate empty output from rank_tell to empty dict
Not all commands return JSON, like `dirfrag split`.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 16 Apr 2024 19:22:13 +0000 (15:22 -0400)]
qa: move reqid_tostr helper
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Wed, 17 Apr 2024 19:52:36 +0000 (15:52 -0400)]
pybind/mgr: disable sqlite3/python autocommit
SQLite3 and python's binding were both committing transactions at unintended
points. Turn it off and stop using executescript.
Fixes: https://tracker.ceph.com/issues/65494
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 19 Apr 2024 03:35:05 +0000 (23:35 -0400)]
qa/tasks/mgr: add tests for sqlite autocommit
That autocommit is properly turned off and that commits via context managers
work as expected.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Adam King [Mon, 29 Apr 2024 17:54:37 +0000 (13:54 -0400)]
qa/cephadm: ignore stray daemon warning during rados_api_tests
The "stray daemon" that is getting logged about in this test is
from "stray daemon laundry.pid70383 on host smithi027 not managed by cephadm".
It seems the rados_api_tests is creating some additional "laundry" entity
during these tests that gets reported as an actual daemon in the mgr,
but cephadm is unaware of it, resulting in the warning. Originally
we thought to maybe add "laundry" itself to the ignorelist, but
without an additional patch that added extra logging for debug
purposes (which can't be merged) the log statement found in
the logs due to this problem will not say what daemon it found
to be stray. There will just be a generic warning about a stray
daemon. In a real cluster, a user would then check "ceph health detail"
to find out what daemon is stray, but the log scraper can't do this
and just fails the test due to the presence of the warning.
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Mon, 29 Apr 2024 17:41:10 +0000 (13:41 -0400)]
mgr/cephadm: set OSD cap for NVMEoF daemon to "profile rbd"
Otherwise it is impossible to make rbd snapshots
Fixes: https://tracker.ceph.com/issues/65691
Signed-off-by: Adam King <adking@redhat.com>
Patrick Donnelly [Fri, 19 Apr 2024 03:34:16 +0000 (23:34 -0400)]
qa/tasks/vstart_runner: run daemons in foreground
This mirrors teuthology and makes it possible to check the exit status of a
daemon.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 19 Apr 2024 03:31:07 +0000 (23:31 -0400)]
qa/tasks/vstart_runner: add missing poll method
Otherwise you cannot use LocalDaemon.check_status.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 18 Apr 2024 20:58:56 +0000 (16:58 -0400)]
qa/suites/rados/mgr: add cli/devicehealth tasks
These should have been part of the commit adding the tests.
Fixes: 9ebcbdbed0fbaf37d60fbd8db321742ee5721d8c
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2024 14:39:10 +0000 (10:39 -0400)]
qa: reorganize mgr unit tests
Refactor common tasks and allow loading mgrmodules before unittests start.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2024 14:26:11 +0000 (10:26 -0400)]
qa: use position-independent link
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2024 14:22:00 +0000 (10:22 -0400)]
qa: add missing terminating newline
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Wed, 17 Apr 2024 19:44:30 +0000 (15:44 -0400)]
pybind/mgr: add killpoint for sqlite3 database setup
To catch unintentional autocommit by sqlite3.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 19 Apr 2024 18:31:09 +0000 (14:31 -0400)]
mgr: allow specifying module option level
Some are for development purposes and should be filtered out by the dashboard.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 19 Apr 2024 02:18:57 +0000 (22:18 -0400)]
mon/MgrMonitor: promote standby when unsetting down flag
In the same epoch, to simplify testing.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 18 Apr 2024 19:40:19 +0000 (15:40 -0400)]
mon/MgrMonitor: only drop active if exists
This avoids an assertion in ::drop_active.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Anthony D'Atri [Mon, 29 Apr 2024 12:24:37 +0000 (08:24 -0400)]
Merge pull request #57136 from zdover23/wip-doc-2024-04-29-radosgw-bucketpolicy
doc/radosgw: disambiguate version-added remarks
Zac Dover [Mon, 29 Apr 2024 10:23:13 +0000 (20:23 +1000)]
doc/radosgw: disambiguate version-added remarks
Rewrite two sentences so that they explain precisely what was added to
Ceph and when it was added to Ceph.
This commit is made in response to a request made by Dr Per Mildner on
https://pad.ceph.com/p/Report_Documentation_Bugs.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Xuehan Xu [Sun, 21 Apr 2024 04:41:29 +0000 (12:41 +0800)]
crimson/osd/osd_operation: yield the cpu before another round the
"with_throttle"
This is to avoid infinite recursive calling to "with_throttle_while"
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Xuehan Xu [Wed, 10 Apr 2024 06:51:47 +0000 (14:51 +0800)]
crimson/osd/pg_recovery: backoff if the recovery/backfill is deferred
Fixes: https://tracker.ceph.com/issues/65399
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Yingxin Cheng [Mon, 29 Apr 2024 03:39:44 +0000 (11:39 +0800)]
crimson/os/seastore: drop the impossible error eagain
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Mon, 29 Apr 2024 03:03:02 +0000 (11:03 +0800)]
crimson/os/seastore/extent_placement_manager: make reserve and block atomic
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Mon, 29 Apr 2024 02:46:11 +0000 (10:46 +0800)]
crimson/os/seastore: improve seastore_t logs
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Xuehan Xu [Mon, 29 Apr 2024 08:56:09 +0000 (16:56 +0800)]
crimson/os/seastore/transaction_manager: remove incorrect assertions
Indirect mappings always hold the checksums of the corresponding intermediate
mappings, which may have non-zero
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Yingxin Cheng [Fri, 26 Apr 2024 07:59:55 +0000 (15:59 +0800)]
crimson/os/seastore: add logs for background
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 26 Apr 2024 06:23:28 +0000 (14:23 +0800)]
test/crimson/gtest_seastar: add logs
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 26 Apr 2024 06:21:38 +0000 (14:21 +0800)]
test/crimson/gtest_seastar: cleanups
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin [Mon, 29 Apr 2024 08:44:36 +0000 (16:44 +0800)]
Merge pull request #55449 from xxhdx1985126/wip-seastore-extent-chksum
crimson/os/seastore: support extent checksum verification
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Xuehan Xu [Wed, 24 Apr 2024 09:00:53 +0000 (17:00 +0800)]
crimson/os/seastore/transaction_manager: fix write pipeline phase leak
At present, if a transaction gets interrupted right after it enters
WritePipeline::ReserveProjectedUsage and before any later continuations
get executed, WritePipeline::ReserveProjectedUsage will be locked
forever.
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Yingxin [Mon, 29 Apr 2024 06:50:01 +0000 (14:50 +0800)]
Merge pull request #57088 from xxhdx1985126/wip-seastore-update-onode-size
crimson/os/seastore: only update onode sizes when necessary
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Xuehan Xu [Mon, 29 Apr 2024 06:20:12 +0000 (14:20 +0800)]
Merge pull request #57118 from xxhdx1985126/wip-65673
test/ceph_crypto: define __has_feature if the compiler doesn't have it
Reviewed-by: Kefu Chai <kefu.chai@scylladb.com>
Zac Dover [Mon, 29 Apr 2024 02:37:53 +0000 (12:37 +1000)]
Merge pull request #57119 from zdover23/wip-doc-2024-04-28-cephfs-client-auth
doc/cephfs: edit front matter in client-auth.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Sun, 28 Apr 2024 13:18:50 +0000 (23:18 +1000)]
doc/cephfs: edit front matter in client-auth.rst
Clarify the front matter in doc/cephfs/client-auth.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
Kefu Chai [Sun, 28 Apr 2024 15:14:49 +0000 (23:14 +0800)]
Merge pull request #57112 from tchaikov/erasure-code-free
erasure-code/shec: use free() to release alloc()'ed memory chunk
Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>
Kefu Chai [Sun, 28 Apr 2024 12:27:03 +0000 (20:27 +0800)]
Merge pull request #57113 from tchaikov/erasure-code-free-crush-wrapper
test//erasure-code/TestErasureCodeLrc: free allocated CrushWrapper
Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>
Kefu Chai [Sun, 28 Apr 2024 12:26:21 +0000 (20:26 +0800)]
Merge pull request #57111 from tchaikov/erasure-code-nullptr
erasure-code/shec: replace 0 with nullptr when appropriate
Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>
Xuehan Xu [Fri, 26 Apr 2024 10:12:15 +0000 (18:12 +0800)]
crimson/osd/osdop_params:Unify OpsExecuter::user_modify and osd_op_params_t::user_modify
Before this change OpsExecuter::user_modify was maintained in OpsExecuter::do_write_op.
However, osd_op_params->user_modify was not updated when used in OpsExecuter::prepare_transaction
Fixes: https://tracker.ceph.com/issues/65672
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Xuehan Xu [Thu, 25 Apr 2024 05:56:25 +0000 (13:56 +0800)]
crimson/osd/pg: SnapTrimEvent to support interrupts
SnapTrimEvent operations are scheduled from `PG::on_active_actmap()`
using a `seastar::do_until` loop. This commit replaces the loop type
into an `interruptor::repeat` and SnapTrimEvent are now scheduled by
`start_operation_may_interrupt`.
Previously, `SnapTrimEvent::start` handled interruptions by returning
a `crimson::ct_error::eagain::make();`. Now, the errorator is directly
returned via the `snap_trim_event_ret_t` and interrupts the loop
described above.
As a result, interruptions originated by interval changes are now
supported by SnapTrimEvent.
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Xuehan Xu [Sun, 28 Apr 2024 03:11:57 +0000 (11:11 +0800)]
test/ceph_crypto: define __has_feature if the compiler doesn't have it
Refer to https://gcc.gnu.org/onlinedocs/cpp/_005f_005fhas_005ffeature.html
and https://clang.llvm.org/docs/LanguageExtensions.html#has-feature-and-has-extension
for further information
Fixes: https://tracker.ceph.com/issues/65673
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Kefu Chai [Sun, 28 Apr 2024 01:05:07 +0000 (09:05 +0800)]
test//erasure-code/TestErasureCodeLrc: free allocated CrushWrapper
in TestErasureCodeLrc.cc, we create a `CrushWrapper` without
destroying and freeing it. and ASan points this out:
```
Indirect leak of 72000 byte(s) in 1000 object(s) allocated from:
#0 0x559e00acc2bd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_erasure_code_lrc+0x1ed2bd) (BuildId:
1cd93d0f231006242239eb0d81a8d677e36aeba2 )
#1 0x559e00b2bb76 in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
#2 0x559e00b2bb00 in std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
#3 0x559e00b2bb00 in std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > > >::allocate(std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
#4 0x559e00b2b9d1 in std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_get_node() /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:561:16
#5 0x559e00b2aae4 in std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >* std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_create_node<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:611:23
#6 0x559e00b29520 in std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:2431:19
#7 0x559e00b255c2 in std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_map.h:501:15
#8 0x559e00b188da in CrushWrapper::set_item_name(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/crush/CrushWrapper.h:466:7
#9 0x7feb54a4dcc2 in CrushWrapper::insert_item(ceph::common::CephContext*, int, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/crush/CrushWrapper.cc:1119:5
#10 0x559e00ada922 in ErasureCodeTest_create_rule_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/TestErasureCodeLrc.cc:127:5
#11 0x559e00c0e0a6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
```
so, in this change, let's manage the lifecycle of the `CrushWrapper`
instance with a smart pointer, so that it is destroyed and free'd
properly, and this should silence the ASan warning.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>