Ville Ojamo [Fri, 31 Mar 2023 14:11:59 +0000 (21:11 +0700)]
doc/cephfs: fix prompts in fs-volumes.rst
Modernize prompts in example commands in file
fs-volume.rst and use a root prompt correctly instead
of using a non-privileged prompt when privileges
are needed.
Indent all JSON in example command outputs with
four spaces consistently instead of a mix of four and
two spaces.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Lucian Petrut [Thu, 30 Mar 2023 12:14:10 +0000 (12:14 +0000)]
include: move ALLPERMS definition to compat.h
The Windows CI job started to fail as some libcephfs tests that use
ALLPERMS have been moved [1] to a separate file which doesn't have
the ALLPERMS definition.
/ceph/src/test/libcephfs/suidsgid.cc:240:36: error: ‘ALLPERMS’ was
not declared in this scope
240 | ASSERT_EQ(stx.stx_mode & (mode_t)ALLPERMS, before_mode);
We'll move this definition to compat.h so that we won't have to
redefine it in each file that uses it.
Note that we're moving the Windows "fs_compat.h" include up,
ensuring that the constants used by ALLPERMS are defined.
Venky Shankar [Fri, 31 Mar 2023 04:02:37 +0000 (09:32 +0530)]
Merge PR #49460 into main
* refs/pull/49460/head:
qa: fix issue with fn unable to fetch port and ip
qa: fix helper function _check_nfs_cluster_status()
qa: fix testcase 'test_cluster_set_user_config_with_non_existing_clusterid'
qa: fix cluster creation failure in test_nfs.py
qa: test export creation at filepath and symlink
qa: added test case test_nfs_export_with_invalid_path
mgr/nfs: disallow non-existent paths when creating export
mgr/nfs/tests: mock check_cephfs_path
mgr/nfs/utils: add helper func to check cephfs path
Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Rishabh Dave <ridave@redhat.com>
John Mulligan [Thu, 30 Mar 2023 20:49:27 +0000 (16:49 -0400)]
python-common: add a dedicated tox env to run mypy
IMO it's not a good practice to overload a tox rule with multiple
different test tools. It forces the tools to share the same virtualenvs
and makes it impossible to run the tools individually. A separate mypy
env also better matches the other tox.ini files in the ceph tree.
Since the new 'mypy' env is in the default env list it will continue
to get run automatically when no specific envs are selected.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 28 Mar 2023 20:42:41 +0000 (16:42 -0400)]
mypy: update pinned mypy version to 0.981
mypy version 0.981 fixes a bug where on newer python versions mypy
doesn't properly load pyi files with keyword only arguments.
As noted in the src/mypy-constrains.txt mypy version needs to be
manually bumped periodically, and ceph is overdue for an update too.
It's never been updated since the file was added in June 2021.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 30 Mar 2023 20:48:02 +0000 (16:48 -0400)]
python-common: fix variable name reuse to make mypy happy
The variables high and low were being used as both `str`s and regex
match objects. Rename the vars in the if block to avoid this problem.
This change makes this file pass mypy checking on mypy 0.981.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 28 Mar 2023 21:09:30 +0000 (17:09 -0400)]
mgr/dashboard: ignore type checking on mgr proxy object assignments
Add `# type: ignore` comments to two dashboard functions that attempt
to set manager properties. There appear to be two approaches to fixing
the problem:
1. The _MgrProxy object that the dashboard uses has a __getattr__ method
for pulling value from the underlying mgr object. It does not have a
__setattr__ method. This means the setting values on _MgrProxy do not
propogate down to the original mgr.
mypy detects the fact that the object doesn't have __setattr__ and
complains. One could add a __setattr__ to the proxy type and mypy
is satisfied.
2. We can just suppress the type check with the comment.
Because I have no idea why the _MgrProxy exists or why it's implemented
the way it is, I feel that 2 is simpler. It is easy enough to go back
later and clean up the comments rather than me investing a lot of time
to understand the dashboard's approach just to bump up the version of
mypy.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 28 Mar 2023 21:07:20 +0000 (17:07 -0400)]
mgr/dashboard: ignore type checking for exception handling module
Add a `# type: ignore` comment to the exception handling dashboard
module just like the instance to lines below. This module does not
already import typing, so I'm not going to add it.
This change is needed in order to run mypy 0.981.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Laura Flores [Thu, 30 Mar 2023 15:38:14 +0000 (10:38 -0500)]
qa/crontab: check older builds on teuthology/nop when necessary
Today's scheduled run failed since the newest build of main
had failed. If we add `-n 10` to the command, this will
make it so we start at the newest build and backtrack
up to 10 older builds if necessary.
A higher number than that is not necessary, as the suite
failing to run will signal to us that more than the last
10 main builds are broken in Shaman.
2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner: info_output = json.loads(self._nfs_cmd('cluster', 'info', self.cluster_id))['test']['backend'][0]
2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:IndexError: list index out of range
dparmar18 [Tue, 21 Feb 2023 18:08:42 +0000 (23:38 +0530)]
qa: fix helper function _check_nfs_cluster_status()
Comment in the code says to wait for two minutes as cluster
creation takes time but actually it's waiting for thirteen
minutes, it's not required to wait this long, i think a minute
here is more than enough, also switched to using safe_while().
dparmar18 [Mon, 13 Feb 2023 14:32:06 +0000 (20:02 +0530)]
qa: fix cluster creation failure in test_nfs.py
Also adds a function _nfs_complete_cmd() that returns process obj so that stdout/stderr
can be used for evaluation(_nfs_cmd() uses raw_cluster_cmd() that returns just stdout
and it became difficult to time cluster creation errors in _test_create_cluster()).
It takes sometime to update the cluster data, therefore running the command set
(check nfs server status -> nfs cluster create test -> check cluster status) in
a loop (max six iteration with sleep of 5 secs at each iteration) fixes the issue.
Venky Shankar [Thu, 30 Mar 2023 10:43:48 +0000 (16:13 +0530)]
Merge PR #47649 into main
* refs/pull/47649/head:
mds: adjust MDSRank::command_tag_path invocation of enqueue_scrub()
doc/scrub: documented stray evaluation using recursive scrub
qa: added testcases
mds: make `scrub status` print flag `scrub_mdsdir`
mds: add scrub_mdsdir to ScrubHeader
mds: do not dump multiple JSON obj
mds: evaluate strays while performing scrub on root path
mds: remove inode from scrub_stack if being purged
mds: do not scrub inode if it is purging
Venky Shankar [Thu, 30 Mar 2023 09:18:26 +0000 (14:48 +0530)]
Merge PR #50053 into main
* refs/pull/50053/head:
libcephfs: move ClearSetuid to suidsgid.cc
libcephfs: add test cases for dropping the suid/sgid in write/truncate
libcephfs: add test cases for dropping the suid/sgid in fallocate
libcephfs: fix ClearSetuid incorrectly using SETATTR_MODE mask
client: switch to clear_suid_sgid for ftruncate
client: switch to clear_suid_sgid for _write()
mds/client: clear the suid/sgid in fallocate path
client: allow unprivileged users to clear suid/sgid
Patrick Donnelly [Wed, 29 Mar 2023 20:15:47 +0000 (16:15 -0400)]
Merge PR #49773 into main
* refs/pull/49773/head:
mds: add config to decide whether to mark dentry bad
qa: add missing scan_links step for data scan recovery
qa/tasks/cephfs: test damage to dentry's first is caught
qa/tasks/cephfs: use rank_asok and allow specifying rank
qa/tasks: allow specifying timeout command prefix to ceph
mds: provide test configs for creating first corruption
mds: catch damage to dentry's first field
mds: add debugging for pre_cow_old_inode
mds: cleanup code
Dhairya Parmar [Wed, 29 Mar 2023 17:50:50 +0000 (23:20 +0530)]
mgr/nfs/utils: add helper func to check cephfs path
this helper instantiates CephfsClient, however this was
initially planned in ExportMgr class in export.py but
due to make check failure where main python thread
experienced a dead lock which after several efforts
pointed at instantiation of CephfsClient in ExportMgr
was problematic, it was decided in order to achieve
singleton behavior, func has been added inside this
helper func that restricts instantiation using functool's
lru_cache.
projects / ceph.git / log