Merge pull request #47893 from kotreshhr/ceph-mgr-finisher-block
mgr: Add one finisher thread per module
Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Samuel Just <sjust@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com> Reviewed-by: Brad Hubbard <bhubbard@redhat.com> Reviewed-by: Xiubo Li <xiubli@redhat.com>
Edit docs/rados/operations/health-checks.rst (2 of x). PR#50674, the PR
that immediately precedes this PR in the series of PRs that line-edit
health-checks.rst, wrongly identified this series as having five
sections. This has been rectified by using the "2 of x" formulation.
Mounts may create a Filesystem object which does not create the file
system. This causes self.id to be Null when we try to wait for daemons
to be up. Load it just in time!
Fixes: https://tracker.ceph.com/issues/59332 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Rishabh Dave [Fri, 24 Mar 2023 11:47:36 +0000 (17:17 +0530)]
qa/cephfs: simplify some code in test_multifs_auth.py
test_mutlifs_auth.TestMDSCaps._create_client() not only creates a client
but also generate caps strings for the client as per the parameter this
method receives and and then writes the keyring to all remote machines.
This creates confusion when reading code on test methods in TestMDSCaps.
Let's re-arrange this code such that this confusion is avoided.
Fixes: https://tracker.ceph.com/issues/58164 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Introducing module option in prometheus module to enable/disable support for exporting ceph daemons
perf counters as prometheus metrics, by default this option will be disabled. The use case for this option
is in case if ceph-exporter deployment failed for any reason then user still can have option to fetch metrics
from promethues exporter.
Ilya Dryomov [Tue, 28 Mar 2023 18:03:05 +0000 (20:03 +0200)]
librbd: avoid generating ESHUTDOWN in ManagedLock
EBLOCKLISTED has a very special meaning but happens to be an alias for
ESHUTDOWN. If the client gets blocklisted, we always want to propagate
EBLOCKLISTED error code since it's generated by the OSD.
For ManagedLock use case of indicating that an operation on the lock
raced with lock shut down, meaning that a higher level request can just
be restarted, ERESTART should do.
Ilya Dryomov [Tue, 28 Mar 2023 17:52:42 +0000 (19:52 +0200)]
librbd: fix recursive locking on owner_lock in ImageDispatch
needs_exclusive_lock() calls acquire_lock() with owner_lock held.
If lock acquisiton races with lock shut down, ManagedLock completes
ImageDispatch context directly and dispatch is retried immediately on
the same thread (due to DISPATCH_RESULT_RESTART). This results in
recursion into needs_exclusive_lock() and, barring locking issues, can
lead to unbounded stack growth if lock shut down takes its time.
In case multiple group policies are set to reflect for any sync pair
(<source-zone,source-bucket>, <dest-zone,dest-bucket>), the following
rules are applied in the order -
1) Even if one policy status is FORBIDDEN, the sync will be disabled
2) Atleast one policy should be ENABLED for the sync to be allowed.
Various cases tested are outlined here -
https://docs.google.com/document/d/19oBQA-bYxLBR4BnekA2DTwJJaTFvjAfrqAk9G3RGU0I/edit#heading=h.4qac9dpc76m
Ville Ojamo [Fri, 31 Mar 2023 14:11:59 +0000 (21:11 +0700)]
doc/cephfs: fix prompts in fs-volumes.rst
Modernize prompts in example commands in file
fs-volume.rst and use a root prompt correctly instead
of using a non-privileged prompt when privileges
are needed.
Indent all JSON in example command outputs with
four spaces consistently instead of a mix of four and
two spaces.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Ilya Dryomov [Thu, 30 Mar 2023 11:58:20 +0000 (13:58 +0200)]
librbd: clear Image::list_watchers() list before populating it
The "append to the passed list" behavior is confusing and not what the
corresponding C API (rbd_watchers_list) or other similar C++ APIs (e.g.
list_lockers) do.
Lucian Petrut [Thu, 30 Mar 2023 12:14:10 +0000 (12:14 +0000)]
include: move ALLPERMS definition to compat.h
The Windows CI job started to fail as some libcephfs tests that use
ALLPERMS have been moved [1] to a separate file which doesn't have
the ALLPERMS definition.
/ceph/src/test/libcephfs/suidsgid.cc:240:36: error: ‘ALLPERMS’ was
not declared in this scope
240 | ASSERT_EQ(stx.stx_mode & (mode_t)ALLPERMS, before_mode);
We'll move this definition to compat.h so that we won't have to
redefine it in each file that uses it.
Note that we're moving the Windows "fs_compat.h" include up,
ensuring that the constants used by ALLPERMS are defined.
lichaochao [Tue, 28 Mar 2023 03:17:26 +0000 (05:17 +0200)]
rgw: fix rgw cache invalidation after unregister_watch() error
When a metadata osd fails, an unregister_watch() error may occur,
resulting in an rgw cache invalidation.
By adding an unregister_done flag and when a register_watch() error ,
performing a reinit() operation again,
After the first reinit() failure, the register_watch() will be performed again
Redouane Kachach [Wed, 26 Oct 2022 09:33:38 +0000 (11:33 +0200)]
mgr/cephadm: Adding extra arguments support for RGW frontend Fixes: https://tracker.ceph.com/issues/57931 Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Venky Shankar [Fri, 31 Mar 2023 04:02:37 +0000 (09:32 +0530)]
Merge PR #49460 into main
* refs/pull/49460/head:
qa: fix issue with fn unable to fetch port and ip
qa: fix helper function _check_nfs_cluster_status()
qa: fix testcase 'test_cluster_set_user_config_with_non_existing_clusterid'
qa: fix cluster creation failure in test_nfs.py
qa: test export creation at filepath and symlink
qa: added test case test_nfs_export_with_invalid_path
mgr/nfs: disallow non-existent paths when creating export
mgr/nfs/tests: mock check_cephfs_path
mgr/nfs/utils: add helper func to check cephfs path
Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Rishabh Dave <ridave@redhat.com>
During send_acquire_lock, there's a case where
there's no watcher handle present and lock request is delayed.
If the client is blocklisted, the delayed request will not
continue and the call that requested lock will never complete.
The lock process will now propagate -EBLOCKLIST, to callback
instead of indefinitely delaying.
Fixes: https://tracker.ceph.com/issues/59115 Signed-off-by: Christopher Hoffman <choffman@redhat.com>
John Mulligan [Thu, 30 Mar 2023 20:49:27 +0000 (16:49 -0400)]
python-common: add a dedicated tox env to run mypy
IMO it's not a good practice to overload a tox rule with multiple
different test tools. It forces the tools to share the same virtualenvs
and makes it impossible to run the tools individually. A separate mypy
env also better matches the other tox.ini files in the ceph tree.
Since the new 'mypy' env is in the default env list it will continue
to get run automatically when no specific envs are selected.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 28 Mar 2023 20:42:41 +0000 (16:42 -0400)]
mypy: update pinned mypy version to 0.981
mypy version 0.981 fixes a bug where on newer python versions mypy
doesn't properly load pyi files with keyword only arguments.
As noted in the src/mypy-constrains.txt mypy version needs to be
manually bumped periodically, and ceph is overdue for an update too.
It's never been updated since the file was added in June 2021.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 30 Mar 2023 20:48:02 +0000 (16:48 -0400)]
python-common: fix variable name reuse to make mypy happy
The variables high and low were being used as both `str`s and regex
match objects. Rename the vars in the if block to avoid this problem.
This change makes this file pass mypy checking on mypy 0.981.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 28 Mar 2023 21:09:30 +0000 (17:09 -0400)]
mgr/dashboard: ignore type checking on mgr proxy object assignments
Add `# type: ignore` comments to two dashboard functions that attempt
to set manager properties. There appear to be two approaches to fixing
the problem:
1. The _MgrProxy object that the dashboard uses has a __getattr__ method
for pulling value from the underlying mgr object. It does not have a
__setattr__ method. This means the setting values on _MgrProxy do not
propogate down to the original mgr.
mypy detects the fact that the object doesn't have __setattr__ and
complains. One could add a __setattr__ to the proxy type and mypy
is satisfied.
2. We can just suppress the type check with the comment.
Because I have no idea why the _MgrProxy exists or why it's implemented
the way it is, I feel that 2 is simpler. It is easy enough to go back
later and clean up the comments rather than me investing a lot of time
to understand the dashboard's approach just to bump up the version of
mypy.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 28 Mar 2023 21:07:20 +0000 (17:07 -0400)]
mgr/dashboard: ignore type checking for exception handling module
Add a `# type: ignore` comment to the exception handling dashboard
module just like the instance to lines below. This module does not
already import typing, so I'm not going to add it.
This change is needed in order to run mypy 0.981.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
projects / ceph.git / log