mon/MDSMonitor: deal with subscription after being expired

check expiration condition before taking further action

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

Merge PR #37899 into master

* refs/pull/37899/head:
qa: avoid using ceph.dir.entries for empty check
qa: do not test dir size equals rbytes
qa: disable check for value kclient fails to get
qa: leave one standby available to avoid warning
qa: do not check access timestamps
qa: ignore skip errors for kclient
qa: add more clients for test_volume_client
qa: accept client returning pool id for layout
qa: ignore expected warning about client missing features
qa: do not fail on skip
qa: fix mount directory
qa: rename distro link to distros
qa: accept eio for mount failure
qa: unmount volumes before removal
qa: some sessionmap tests require fuse
qa: remove obsolete test
qa: do not fail on test skip
qa: use more OSDs for workload tests
qa: use 4 clients for volumes testing
qa: relocate tasks
qa: rename workload suite
qa: relocate new tasks from fs:basic_functional
qa: remove kcephfs suite
qa: move kcephfs:mixed-clients to fs
qa: remove dead kcephfs:cephfs suite
qa: merge snap tests for kcephfs/fs
qa: add new fs:shell sub-suite
qa: remove task-less kcephfs:recovery suite
qa: add new fs:volumes sub-suite
qa: merge fs:basic_functional and kcephfs:recovery
qa: move kcephfs:thrash to fs:thrash
qa: reorganize thrash tasks
qa: move multifs/failover tests
qa: run norstats workunit for klient
qa: merge workload tests from kcephfs with fs
qa: use consistent syntax
qa: remove inline testing
qa: move log-rotate config to common yaml
qa: remove fuse default permissions config
qa: fix whitespace

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32394 from zhangsw/fix-appendobj-invalidposition

rgw: using strict_strtoll for append position. If not, any invalid value will be decoded as zero which is not correct.

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #37703 from theanalyst/rgw/sal/init-quota-fixes

rgw: op: check for bucket existance as well in init_quota

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #37160 from linuxbox2/wip-rgwlc-tagfilter

rgwlc: fix clause counting in LCFilter_S3::decode_xml()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #37066 from smanjara/wip-metadata-reshard

rgw: don't allow overwrites to BucketInfo's layout field when sync fetches bucket metadata

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37892 from IlsooByun/fix_bucket_sync_err

rgw: keep syncstopped flag when copying bucket shard headers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #37867 from zdover23/wip-doc-cephadm-install-oct-2020

doc/cephadm: improve install guide

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #37944 from tchaikov/wip-crimson-recovery-cleanup

crimson/osd: extract read_omap_for_push_op()

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/osd: extract read_omap_for_push_op()

extract read_omap_for_push_op() out of build_push_op() for better
readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: correct the function names in logging messages

it was a leftover of fbfc53c7081ea0b99fbc519d7fac9f5637cf5425

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #37881 into master

* refs/pull/37881/head:
doc/mgr/orchestrator: Update about nfs add in rook
mgr/rook: Refactor nfs add/apply methods
mgr/rook: Udpate to mypy py3 type style in _create(), _update() and apply()
mgr/rook: Remove unused 'current' argument from update_func method

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

qa: avoid using ceph.dir.entries for empty check

This avoids a bug [1] in the kernel client.

[1] https://tracker.ceph.com/issues/48104

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #37825 into master

* refs/pull/37825/head:
doc: document MDS cache configuration
test: add DecayCounter test

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

qa: do not test dir size equals rbytes

Avoiding this failure:

2020-11-03T03:11:59.980 INFO:tasks.cephfs_test_runner:test_subvolume_resize_fail_quota_lt_used_size_no_shrink (tasks.cephfs.test_volumes.TestSubvolumes) ... FAIL
2020-11-03T03:11:59.981 INFO:tasks.cephfs_test_runner:
2020-11-03T03:11:59.981 INFO:tasks.cephfs_test_runner:======================================================================
2020-11-03T03:11:59.981 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_resize_fail_quota_lt_used_size_no_shrink (tasks.cephfs.test_volumes.TestSubvolumes)
2020-11-03T03:11:59.982 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-11-03T03:11:59.982 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2020-11-03T03:11:59.982 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201102.231317/qa/tasks/cephfs/test_volumes.py", line 1310, in test_subvolume_resize_fail_quota_lt_used_size_no_shrink
2020-11-03T03:11:59.982 INFO:tasks.cephfs_test_runner:    self.assertEqual(usedsize, susedsize)
2020-11-03T03:11:59.982 INFO:tasks.cephfs_test_runner:AssertionError: 8912896 != 1

From: /ceph/teuthology-archive/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585383/teuthology.log

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: disable check for value kclient fails to get

See-also: https://tracker.ceph.com/issues/48075
Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: leave one standby available to avoid warning

Four file systems will use all MDS and generate this warning:

2020-11-02T03:48:33.407 INFO:teuthology.orchestra.run.smithi003.stdout:2020-11-02T03:24:21.817337+0000 mon.a (mon.0) 481 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: do not check access timestamps

kclient does not update these like ceph-fuse.

This avoids failures like:

2020-11-01T08:05:53.115 INFO:tasks.cephfs_test_runner:test_subvolume_clone_in_progress_getpath (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) ... FAIL
2020-11-01T08:05:53.115 INFO:tasks.cephfs_test_runner:
2020-11-01T08:05:53.116 INFO:tasks.cephfs_test_runner:======================================================================
2020-11-01T08:05:53.116 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_clone_in_progress_getpath (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones)
2020-11-01T08:05:53.116 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-11-01T08:05:53.116 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2020-11-01T08:05:53.117 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_batrick_ceph_cephfs-qa-reorg/qa/tasks/cephfs/test_volumes.py", line 2295, in test_subvolume_clone_in_progress_getpath
2020-11-01T08:05:53.117 INFO:tasks.cephfs_test_runner:    self._verify_clone(subvolume, snapshot, clone)
2020-11-01T08:05:53.117 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_batrick_ceph_cephfs-qa-reorg/qa/tasks/cephfs/test_volumes.py", line 154, in _verify_clone
2020-11-01T08:05:53.117 INFO:tasks.cephfs_test_runner:    self._verify_clone_attrs(path1, path2)
2020-11-01T08:05:53.117 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_batrick_ceph_cephfs-qa-reorg/qa/tasks/cephfs/test_volumes.py", line 107, in _verify_clone_attrs
2020-11-01T08:05:53.118 INFO:tasks.cephfs_test_runner:    self.assertEqual(sval, cval)
2020-11-01T08:05:53.118 INFO:tasks.cephfs_test_runner:AssertionError: 1604217951 != 1604217941

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: ignore skip errors for kclient

To avoid this failure:

2020-11-01T07:21:35.117 INFO:tasks.cephfs_test_runner:test_volume_without_namespace_isolation (tasks.cephfs.test_volume_client.TestVolumeClient) ... ok
2020-11-01T07:21:35.118 INFO:tasks.cephfs_test_runner:
2020-11-01T07:21:35.118 INFO:tasks.cephfs_test_runner:======================================================================
2020-11-01T07:21:35.119 INFO:tasks.cephfs_test_runner:FAIL: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient)
2020-11-01T07:21:35.119 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-11-01T07:21:35.119 INFO:tasks.cephfs_test_runner:Requires FUSE client to inject client metadata
2020-11-01T07:21:35.119 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-11-01T07:21:35.119 INFO:tasks.cephfs_test_runner:Ran 18 tests in 732.749s
2020-11-01T07:21:35.120 INFO:tasks.cephfs_test_runner:
2020-11-01T07:21:35.120 INFO:tasks.cephfs_test_runner:FAILED (failures=1)

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add more clients for test_volume_client

It requires 4.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: accept client returning pool id for layout

Resolving this failure:

2020-10-30T22:52:25.898 INFO:tasks.cephfs_test_runner:test_subvolume_group_create_with_desired_data_pool_layout (tasks.cephfs.test_volumes.TestSubvolumeGroups) ... FAIL
2020-10-30T22:52:25.898 INFO:tasks.cephfs_test_runner:
2020-10-30T22:52:25.899 INFO:tasks.cephfs_test_runner:======================================================================
2020-10-30T22:52:25.899 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_group_create_with_desired_data_pool_layout (tasks.cephfs.test_volumes.TestSubvolumeGroups)
2020-10-30T22:52:25.899 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T22:52:25.899 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2020-10-30T22:52:25.900 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_batrick_ceph_cephfs-qa-reorg/qa/tasks/cephfs/test_volumes.py", line 568, in test_subvolume_group_create_with_desired_data_pool_layout
2020-10-30T22:52:25.900 INFO:tasks.cephfs_test_runner:    self.assertEqual(desired_pool, new_pool)
2020-10-30T22:52:25.900 INFO:tasks.cephfs_test_runner:AssertionError: '34' != 'new_pool'
2020-10-30T22:52:25.900 INFO:tasks.cephfs_test_runner:- 34
2020-10-30T22:52:25.900 INFO:tasks.cephfs_test_runner:+ new_pool

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: ignore expected warning about client missing features

2020-10-30T22:54:14.842 INFO:teuthology.orchestra.run.smithi168.stdout:2020-10-30T22:43:34.067194+0000 mds.a (mds.0) 1 : cluster [WRN] evicting session smithi168:0 (8757), missing required features '0x0000000000000400'

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: do not fail on skip

Avoiding these failures:

2020-10-30T22:49:00.248 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) ...
2020-10-30T22:49:00.248 INFO:tasks.cephfs_test_runner:======================================================================
2020-10-30T22:49:00.248 INFO:tasks.cephfs_test_runner:FAIL: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)
2020-10-30T22:49:00.249 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T22:49:00.249 INFO:tasks.cephfs_test_runner:Require FUSE client to inject client release failure
2020-10-30T22:49:00.249 INFO:tasks.cephfs_test_runner:======================================================================
2020-10-30T22:49:00.250 INFO:tasks.cephfs_test_runner:FAIL: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits)
2020-10-30T22:49:00.250 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T22:49:00.251 INFO:tasks.cephfs_test_runner:Require FUSE client to inject client release failure
2020-10-30T22:49:00.251 INFO:tasks.cephfs_test_runner:======================================================================
2020-10-30T22:49:00.252 INFO:tasks.cephfs_test_runner:FAIL: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits)
2020-10-30T22:49:00.252 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T22:49:00.252 INFO:tasks.cephfs_test_runner:Require FUSE client to inject client release failure
2020-10-30T22:49:00.252 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T22:49:00.252 INFO:tasks.cephfs_test_runner:Ran 8 tests in 368.825s
2020-10-30T22:49:00.253 INFO:tasks.cephfs_test_runner:
2020-10-30T22:49:00.253 INFO:tasks.cephfs_test_runner:FAILED (failures=3)
2020-10-30T22:49:00.253 INFO:tasks.cephfs_test_runner:
2020-10-30T22:49:00.253 INFO:tasks.cephfs_test_runner:======================================================================
2020-10-30T22:49:00.253 INFO:tasks.cephfs_test_runner:FAIL: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)
2020-10-30T22:49:00.254 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T22:49:00.254 INFO:tasks.cephfs_test_runner:Require FUSE client to inject client release failure
2020-10-30T22:49:00.254 INFO:tasks.cephfs_test_runner:======================================================================
2020-10-30T22:49:00.254 INFO:tasks.cephfs_test_runner:FAIL: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits)
2020-10-30T22:49:00.255 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T22:49:00.255 INFO:tasks.cephfs_test_runner:Require FUSE client to inject client release failure
2020-10-30T22:49:00.255 INFO:tasks.cephfs_test_runner:======================================================================
2020-10-30T22:49:00.255 INFO:tasks.cephfs_test_runner:FAIL: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits)
2020-10-30T22:49:00.255 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T22:49:00.256 INFO:tasks.cephfs_test_runner:Require FUSE client to inject client release failure

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: fix mount directory

Otherwise we get this error:

2020-10-30T03:57:44.256 INFO:teuthology.run_tasks:Running task exec...
2020-10-30T03:57:44.265 INFO:teuthology.task.exec:Executing custom commands...
2020-10-30T03:57:44.266 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi095.front.sepia.ceph.com
2020-10-30T03:57:44.266 INFO:teuthology.orchestra.run.smithi095:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'cd $TESTDIR/mnt.* && dd if=/dev/zero of=./foo count=100'
2020-10-30T03:57:44.380 INFO:teuthology.orchestra.run.smithi095.stderr:bash: line 0: cd: too many arguments

From: /ceph/teuthology-archive/pdonnell-2020-10-30_02:26:51-fs-master-distro-basic-smithi/5573179/teuthology.log

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: rename distro link to distros

This needs to become any overrides present in "mount". Otherwise kclient
distro overrides have no effect.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: accept eio for mount failure

Example:

2020-10-30T03:37:33.752 INFO:tasks.cephfs_test_runner:======================================================================
2020-10-30T03:37:33.752 INFO:tasks.cephfs_test_runner:FAIL: test_mount_mon_and_osd_caps_present_mds_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)
2020-10-30T03:37:33.752 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-10-30T03:37:33.753 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2020-10-30T03:37:33.753 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_batrick_ceph_cephfs-qa-reorg/qa/tasks/cephfs/test_multifs_auth.py", line 311, in test_mount_mon_and_osd_caps_present_mds_caps_absent
2020-10-30T03:37:33.753 INFO:tasks.cephfs_test_runner:    self.check_that_mount_failed_for_right_reason(retval[2])
2020-10-30T03:37:33.753 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_batrick_ceph_cephfs-qa-reorg/qa/tasks/cephfs/test_multifs_auth.py", line 269, in check_that_mount_failed_for_right_reason
2020-10-30T03:37:33.753 INFO:tasks.cephfs_test_runner:    raise AssertionError('can\'t find expected set of words in the '
2020-10-30T03:37:33.754 INFO:tasks.cephfs_test_runner:AssertionError: can't find expected set of words in the stderr
2020-10-30T03:37:33.754 INFO:tasks.cephfs_test_runner:self.errmsgs - ('permission denied', 'no mds server is up or the cluster is laggy', 'no such file or directory')
2020-10-30T03:37:33.754 INFO:tasks.cephfs_test_runner:stderr - mount error 5 = input/output error

From: /ceph/teuthology-archive/pdonnell-2020-10-30_02:26:51-fs-master-distro-basic-smithi/5573109/teuthology.log

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: unmount volumes before removal

To avoid potential failures/hangs in umount.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: some sessionmap tests require fuse

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove obsolete test

Since the work in [1], the kclient can be killed with network namespaces
without powercycling.

[1] https://tracker.ceph.com/issues/44044

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: do not fail on test skip

test_client_config only works with ceph-fuse.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use more OSDs for workload tests

Running out of space / slow requests are less likely this way.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use 4 clients for volumes testing

The test_volume_client.py tests require 4 clients.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: relocate tasks

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: rename workload suite

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: relocate new tasks from fs:basic_functional

These were not tested with kcephfs before, let's see if there's any
bugs!

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove kcephfs suite

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: move kcephfs:mixed-clients to fs

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove dead kcephfs:cephfs suite

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: merge snap tests for kcephfs/fs

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add new fs:shell sub-suite

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove task-less kcephfs:recovery suite

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add new fs:volumes sub-suite

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: merge fs:basic_functional and kcephfs:recovery

These functional tests can be run with both clients.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: move kcephfs:thrash to fs:thrash

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: reorganize thrash tasks

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: move multifs/failover tests

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: run norstats workunit for klient

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: merge workload tests from kcephfs with fs

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use consistent syntax

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove inline testing

inline feature is now deprecated and TBR.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: move log-rotate config to common yaml

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove fuse default permissions config

It is the default.

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: fix whitespace

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #37923 into master

* refs/pull/37923/head:
client: increment file position on _read_sync near eof

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #37737 from votdev/issue_47926_orch_status

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

doc/cephadm: improve install guide

- Repair broken lists.
- Add bash prompt directives.
- Add code-block directives.
- Add command markup.
- Improve English grammar.
- Fix prompt indentation.

Fixes: https://tracker.ceph.com/issues/48023
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #37757 from votdev/issue_47884_display_time

mgr/dashboard: displaying time in human-readable format

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #37827 from dillaman/wip-librbd-migration-2

librbd: added API and CLI support for source-spec import migrations

Reviewed-by: Mykola Golub <mgolub@suse.com>

doc/mgr/orchestrator: Update about nfs add in rook

"nfs add" is no longer supported in rook. Instead "apply nfs" should be used to
deploy nfs-ganesha daemons.

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/rook: Refactor nfs add/apply methods

Fixes: https://tracker.ceph.com/issues/47923
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/rook: Udpate to mypy py3 type style in _create(), _update() and apply()

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/rook: Remove unused 'current' argument from update_func method

Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #37829 from liu-chunmei/seastore-extentmap-fix

crimson/seastore: add Extentmap node type in cache.cc

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/man: add new RBD migration prepare source-spec optionals

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/suite/rbd: simple migration import test case

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/tasks: support explicit disk configuration for qemu task

The 'disks' key will now be treated as a dictionary where all previous
global settings can be individually applied. Additionally, a disk can be
pre-created and provided for use by qemu.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: add new 'migration prepare --import-only' CLI optional

The '--import-only' optional can be combined with either a
'--source-spec' or '--source-spec-path' optional to define
the source for the read-only import.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #37872 from theanalyst/doc/releases/v14.2.13

doc: releases: v14.2.13 release notes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

client: increment file position on _read_sync near eof

Fixes: https://tracker.ceph.com/issues/48076
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/seastore: add replay test for extent map tree

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

Merge pull request #37287 from pcuzner/dashboard-goals

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #37878 into master

* refs/pull/37878/head:
doc/mgr/orchestrator: Update about "{mds, rgw} add" status in rook

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge PR #37732 into master

* refs/pull/37732/head:
osdc: restart read on truncate/discard

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: releases/nautilus: add more changelog entries & fix date

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #37898 from lxbsz/nbd_reexpandmeta

rbd-nbd: reexpand the conf meta in child process

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #37547 from ifed01/wip-ifed-fix-bmap-alloc

os/bluestore: fix segfault on out-of-bound offset provided to  claim_…

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #37879 from tchaikov/wip-seastar-v6

crimson: use Seastar API v6

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #37894 from dillaman/wip-48035

librbd: global config overrides do not apply to in-use images

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #37907 from athanatos/sjust/wip-fix-init-weak

crimson/os/seastore/.../lba_btree_node_impl: hold reference to *this during lookup

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore/.../lba_btree_node_impl: hold reference to *this during lookup

4f2f4f modified TransactionManager::mount() to use a weak_transaction
while calling init_cached_extents.  This is fine, but lookup() needs
to hold a reference to *this until the child lookup completes in order
to ensure residence in the lba pinning set.

Signed-off-by: Samuel Just <sjust@redhat.com>

doc/dev/dashboard: typo fix

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

doc/dev/dashboard: link designs to dev guide

Add a feature design section to the dashboard dev
guide, linking to the individual docs under dov/dev/dashboard

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

doc/dev/dashboard: Document key UI design principles

Document which defines the high level design principles
for the Ceph Dashboard UI

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

Merge pull request #37590 from pcuzner/prometheus-slow-ops

mgr/prometheus: Add SLOW_OPS healthcheck as a metric

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr: add doc to describe relationship to prometheus

The healthcheck for slow_ops is used by the mgr/prometheus
module so it's important the alert generated matches the
format expected in mgr/prometheus.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

mgr/prometheus: Add healthcheck metric for SLOW_OPS

SLOW_OPS is triggered by op tracker, and generates a health
alert but healthchecks do not create metrics for prometheus to
use as alert triggers. This change adds SLOW_OPS metric, and
provides a simple means to extend to other relevant health
checks in the future

If the extract of the value from the health check message fails
we log an error and remove the metric from the metric set. In
addition the metric description has changed to better reflect
the scenarios where SLOW_OPS can be triggered.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

rbd: show migration source spec in image status

When using the non-legacy mode image migration, the 'rbd status'
command will now show the source-spec for the migration if the
source pool id is invalid.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: ensure the help printer doesn't print past the end of the line

When long command names and long optional names are combined,
it's possible for the help text to be printed beyond the 80
character limit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: replace direct reads from ImageRequest with ImageDispatchSpec

Read requests will always need to go via the image dispatch layer to
ensure migrations are properly handled.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd/migration: allow native v2 images to be opened by name

When in import-only mode, allow the source image to be opened by
name in case it's a V2 image format w/o its image id specified in
the JSON.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd/migration: close image if failed to open source

To remain consistent between the native which will automatically close
the ImageCtx if it fails to open the image, the raw format should also
close the ImageCtx.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd/migration: initialize default source image layout

For backing source formats that don't care about the low-level
RBD layout details, provide safe default values for the layout
to ensure that striping logic will still function.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fix potential recursive lock of ImageCtx::image_lock

If the "list_snaps" IO call returns immediately, it can result
in an attempted recursive lock starting from CopyupRequest.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: migration execute should allow "EXECUTING" state

If the "execute" action was interrupted, we need to be able to
restart it.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: read image size when opening migration raw format

The migration prepare operation will need to know the size of
the source image so that it can create the destination image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd/migration: fix issue with prepare-import and native-format

When preparing an import, the native format can not expect to have
the child image opened since it hasn't been created yet.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: added new 'migration_prepare_import' API methods

These related methods accept a JSON-encoded source-spec that
describes how to read from the source image. The migration is
a one-way operation, so children/groups/etc won't be updated
to point to the new destination image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: store migration progress on both source+destination images

When migrating from an import-only source-spec like an external
file, it will be important to also store the progress on the
destination image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: switch to migration source-spec builder

Remove all the hard-coded references to the NativeFormat
handler and allow the RawFormat+FileStream handlers to be
instantiated.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: switch migration native format to use source-spec JSON object

Remove the dependency on the MigrationInfo struct and allow the native
format to directly pull parameters from the JSON-encoded object
structure.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>