Kefu Chai [Sun, 2 Aug 2020 01:31:01 +0000 (09:31 +0800)]
mgr/cephadm: track dependency in tox.ini
* track dependency for running tox tests with tox.ini. the fix
test is not performed by "make check", so `install-deps.sh`
is not responsible for preparing the autopep8 wheels. hence
we can just put it in tox.ini
* remove "python_version >= '3'" as we do not support python2
Kefu Chai [Sat, 1 Aug 2020 16:28:36 +0000 (00:28 +0800)]
crimson/osd: send pg_stat with correct seq
* set the osd_stat in pg_stat message sent to mgr, otherwise the seq id
in the message would be always 0.
* return the seq id composed of up_epoch and osd_stat_seq when serving
"flush_pg_stats" tell command.
Patrick Donnelly [Fri, 31 Jul 2020 21:12:53 +0000 (14:12 -0700)]
Merge PR #35944 into master
* refs/pull/35944/head:
qa: defer cleaning the mountpoint's netnses and the bridge
qa/tasks/cephfs/mount.py: remove the stale netnses and bridge
qa/tasks/cephfs/mount.py: try to flush the stale ceph-brx dev info
qa/tasks/cephfs/mount.py: switch to run_shell_payload() helper
qa/tasks/cephfs/mount.py: clean up the none used code
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com> Reviewed-by: Rishabh Dave <ridave@redhat.com>
Patrick Donnelly [Fri, 31 Jul 2020 21:11:27 +0000 (14:11 -0700)]
Merge PR #35647 into master
* refs/pull/35647/head:
mgr/volumes: Add documentation regarding --retain-snapshots option
mgr/volumes: Avoid trashing retained subvolume on create errors
mgr/volumes: Add subvolume v2 test cases
mgr/volumes: Derive v2 from v1 to leverage common methods
mgr/volumes: Introduce v2 subvolumes
mgr/volumes: Use operation type during subvolume open
Reviewed-by: Ramana Raja <rraja@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com> Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Patrick Donnelly [Fri, 31 Jul 2020 21:08:41 +0000 (14:08 -0700)]
Merge PR #32120 into master
* refs/pull/32120/head:
client: periodically forward metrics to session MDSs
client: track cap hits and misses globally
client: minor cleanup to MetaSession class
Reviewed-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Jason Dillaman [Fri, 31 Jul 2020 14:31:07 +0000 (10:31 -0400)]
test/librbd: unquiesce notifications are handled lazily
The quiesce unit tests were expecting that the unquiesce notification
would be processed by the time the snapshot create command completed.
Since the RPC is immediately ACKed, there is a potential for a race
if the test doesn't wait for the unquiesce to process.
Fixes: https://tracker.ceph.com/issues/46679 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
* extract get_ragweed_branch() out of download() task, for better
readablity.
* use a loop for retry when the first clone fails
* drop the `raise ValueError()` clause as it never happens. we could use
an assert() here, but i don't think it is necessary anyway.
* use sh() instead of run() for better readablity.
* always set ragweed_repo. before this change this variable is
unbounded if `force-branch` is set.
MOSDScrub2 is sent from mgr for serving "ceph pg
{scrub|deep-scrub|repair}' commands when it's talking to a mimic and newer OSD.
ceph task checks if all pgs are scrubbed by looking at the `last_scrub_stamp` fields
in the `ceph pg dump` output. and it request the not-yet-scrubbed pgs a
deep scrub to ensure they are scrub before timeout.
in this change, crimson handles MOSDScrub2 by starting a remote peering
request, and the underlying peering_state will notify the corresponding
PG to start scrub. to get the test pass, a minimal implmentation is
added to update the scrub timestamp to `now` upon request of
peering_state.
we will need to add the correct scrubbing support later. but this is
enough for passing the thrasher test and for preparing for more tests
which uses the "ceph" task.
FreeBSD/Clang/lib complains:
/home/jenkins/workspace/ceph-master/src/test/librados_test_stub/TestRadosClient.cc:295:55: error: reference to '_1' is ambiguous
&TestRadosClient::finish_aio_completion, this, c, _1));
^
/usr/local/include/boost/bind/placeholders.hpp:46:38: note: candidate found by name lookup is 'boost::placeholders::_1'
BOOST_STATIC_CONSTEXPR boost::arg<1> _1;
^
/usr/include/c++/v1/functional:2448:46: note: candidate found by name lookup is 'std::__1::placeholders::_1'
/* _LIBCPP_INLINE_VAR */ constexpr __ph<1> _1{};
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
Xiubo Li [Wed, 29 Jul 2020 06:58:31 +0000 (14:58 +0800)]
client: clean up the unnecessary client_lock for _conf->client_trace
There is no need to make the "cct->_conf->client_trace" under the
client_lock, it is for the "ceph-syn" and it almost won't change,
even it will but it still won't cause any race.
For the "inode->ino" and "inode->snapid", since they won't change
once initialized when creating the inode, and the inode here will
be protected by the reference count, so no need the client_lock too.
Fixes: https://tracker.ceph.com/issues/46768 Signed-off-by: Xiubo Li <xiubli@redhat.com>
The 'mon_allow_pool_delete' option is set to 'True'
in 'setUp' of 'TestVolumes' and is cleared of in
corresponding 'tearDown' function. Hence, any pool
deletion in parent classes such as 'CephFSTestCase'
would fail. This patch fixes the same by setting
the config 'mon_allow_pool_delete' option in the
'CephFSTestCase'.
monclient: use "is_connected" check when scheduling tick
When schedule_tick is called for the first time (in init) we are
still not in hunting mode so mon_client_ping_interval and
mon_client_log_interval were used to schedule the tick.
Matthew Oliver [Thu, 30 Jul 2020 03:18:48 +0000 (13:18 +1000)]
cephadm: add unwrap_ipv6 helper method
When we pass in a mon-ip that is ipv6 we want it wrapped, this is so it
can be properly inserted into the mon_addrv address.
But there are times we need to unwrap it to test it's a valid ipv6
address.
This patch adds a helper method `unwrap_ipv6` which takes a string and
returns it either unwrapped or as it is, so it's harmless to other types
of ips.
This allows us to check a wrapped ipv6 base_ip with the networks on the
host.
Patrick Donnelly [Tue, 14 Jul 2020 02:50:00 +0000 (19:50 -0700)]
script/ptl-tool: page through github response
This fixes the script to go through the response pages from GitHub.
Previously it would only look at the first page and potentially miss
some reviews/comments.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
mgr/volumes: Avoid trashing retained subvolume on create errors
On any create or create_clone errors the entire subvolume was being
removed. This should be conditional and remove only the incarnation
if the subvolume was in the retained state.
Following support is added,
- Ability to retain snapshots on subvolume deletion
- Modify directory where snapshot is created to the subvolume
- "features" supported to subvolume info output, specifically ability
for a subvolume to retain snashots
- Current state of subvolume in info output
- Auto upgrade to v2 from eligible v1 subvolumes
- Adjust other functions as needed to support the changes
mgr/volumes: Use operation type during subvolume open
Subvolume open currently takes in 2 optional parameters to
denote desired state and type. This enables the open to
allow the operation to suceed based on the (type, state)
tuple.
Instead, pass an operation type to be performed on a subvolume
during open, and decide internal to a subvolume version if the
operation is allowed based on its state and type.
Also modifies the state machine code, to be more amenable to
modifications and improves redability.
Xiubo Li [Mon, 6 Jul 2020 12:27:34 +0000 (08:27 -0400)]
qa: defer cleaning the mountpoint's netnses and the bridge
The netnses maybe created/deleted many times in the whole test cases,
we can defer cleaning them untile the last mountpoint is unmounted
or when the test is exiting.
Fixes: https://tracker.ceph.com/issues/46282 Signed-off-by: Xiubo Li <xiubli@redhat.com>
Patrick Donnelly [Wed, 29 Jul 2020 18:05:02 +0000 (11:05 -0700)]
Merge PR #24068 into master
* refs/pull/24068/head:
mds: rename {CDir,Migrator}::cache to mdcache
mds: make MDSCacheObject::is_ambiguous_auth() virtual
mds: make sure rename old inode's parent dirfrag is projected.
mds: track projected inode/fnode in Mutation
mds: use smart pointer to manager CDir::fnode
mds: use smart pointer to manage CInode::{inode,xattrs,old_inodes}
osdc/Filer: make layout pointer const
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
projects / ceph.git / log