Kefu Chai [Wed, 3 Jun 2020 01:39:26 +0000 (09:39 +0800)]
qa/tasks/vstart_runner: do not teardown test_path if "create-cluster-only"
otherwise we could be removing a "None" directory when tearing down the cluster,
and have following failure:
Exception ignored in: <bound method LocalContext.__del__ of <__main__.LocalContext object at 0x7f99fd4a6cc0>>
Traceback (most recent call last):
File "../qa/tasks/vstart_runner.py", line 1189, in __del__
shutil.rmtree(self.teuthology_config['test_path'])
File "/tmp/tmp.mmM2ugspuR/venv/lib/python3.6/shutil.py", line 477, in rmtree
onerror(os.lstat, path, sys.exc_info())
File "/tmp/tmp.mmM2ugspuR/venv/lib/python3.6/shutil.py", line 475, in rmtree
orig_st = os.lstat(path)
TypeError: lstat: path should be string, bytes or os.PathLike, not NoneType
* refs/pull/34719/head:
ceph-fuse: compatible with libfuse3.5 or higher
cmake: to get the header and library from specified path
libfuse: check the libfuse version from the pkconfig/fuse{3}.pc file
Reviewed-by: Zheng Yan <zyan@redhat.com> Reviewed-by: Kefu Chai <kchai@redhat.com>
Samuel Just [Tue, 12 May 2020 04:02:07 +0000 (21:02 -0700)]
crimson: distinguish record and block relative paddrs
Blocks get read independently of the surrounding record,
so paddr's embedded directly in a block need to refer
to other blocks within the same record by a block_relative
addr relative to the block's own offset. By contrast,
deltas to existing blocks need to use record_relative
addrs relative to the first block of the record.
This patch distinguishes the two kinds of relative paddr
(mainly for debugging purposes) and adapts cache, journal,
etc to use the appropriate types.
* refs/pull/26004/head:
mds: forward mds metrics to ceph manager w/ quering interfaces
mds: track per session client metrics
mds: record metrics from all MDSs in MDS rank 0
mds: non-rank based interface for sending message to an mds
mds: inter-mds ping-pong message and type
mgr: introduce query/report types for ceph metadata server
mds: new intra-mds message type for forwarding aggregated metrics
client: new message type for providing client side metrics
Reviewed-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/34838/head:
vstart_runner: don't use namespaces by default
qa/cephfs: run nsenter commands with superuser privileges
qa/cephfs: look for mountpoint in cmdline file
Reviewed-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
This follows b162541ac21e965a304ee6ffe604c43f22fa96c4.
The balancer was turned on by default in d4fbaf7, as a result of which we might see
PG_AVAILABILITY health warnings when pg-upmap-items are applied.
Kefu Chai [Sat, 30 May 2020 04:51:14 +0000 (12:51 +0800)]
rgw/reshard: use defined variable
use the defined reference for more concise code, this silences the
warning like:
```
../src/rgw/rgw_reshard.cc:530:15: warning: unused variable ‘bucket’ [-Wunused-variable]
530 | rgw_bucket& bucket = bucket_info.bucket;
| ^~~~~~
```
also move `ret` close to where it is used for the first time.
* refs/pull/34782/head:
qa/tasks/cephfs/mount.py: remove netns name parsing in mountpoint setter
qa/tasks/vstart_runner.py: add kwargs parameter to ignore the ones it does not understand
Reviewed-by: Rishabh Dave <ridave@redhat.com> Reviewed-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Lenz Grimmer [Tue, 2 Jun 2020 13:11:20 +0000 (15:11 +0200)]
Merge pull request #35249 from rhcs-dashboard/wip-45705-master
mgr/dashboard: add API team to CODEOWNERS
Reviewed-by: Alfonso Martínez <almartin@redhat.com> Reviewed-by: Laura Paduano <lpaduano@suse.com> Reviewed-by: Stephan Müller <smueller@suse.com> Reviewed-by: Tatjana Dehler <tdehler@suse.com> Reviewed-by: Volker Theile <vtheile@suse.com>
Rishabh Dave [Wed, 29 Apr 2020 18:10:16 +0000 (23:40 +0530)]
qa/cephfs: run nsenter commands with superuser privileges
And add a method that sets self.fuse_daemon.subproc.pid to the PID of
the process that doesn't have sudo in its arguments. For example, when
"sudo ceph-fuse /mnt/cephfs" is run on the shell, it launches process
with arguments "ceph-fuse /mnt/cephfs". The added method gets PID of
latter/child process and sets that as the fuse daemon's PID. Not doing
so kills the former/parent process but the not the child process.
Also, since we are around cleanup this method a bit.
Fixes: https://tracker.ceph.com/issues/45339 Signed-off-by: Rishabh Dave <ridave@redhat.com>
Kefu Chai [Sun, 31 May 2020 00:47:34 +0000 (08:47 +0800)]
qa/suites/rgw/tempest: update unsupported tests of tempest
after rerunning tempest with lastest radosgw, remove the supported
tests from the blacklist, and add the ones which are not supported
yet. now we can pass 123 tests in total.
also enable discoverity for better testing coverage, since it's
supported now.
Kefu Chai [Sun, 31 May 2020 00:38:00 +0000 (08:38 +0800)]
qa/tasks/keystone: use "keystone-manage bootstrap"
* qa/tasks/keystone.py:
instead of prefilling keystone manually, use "keystone-manage bootstrap"
instead. it helps to setup the admin user, a "Default" domain with
"default" id, and wire them up with the expected role and a "admin" project,
etc. as id of the admin domain is known to be "default", we can just use it
in our tests without querying openstack for the id of "Default"
domain. this is very handy.
* qa/suites/rgw/tempest/tasks/rgw_tempest.yaml:
use "Default" for domain name. as "Default" is the name of the domain
created by bootstrap, while "default" is its id.
* qa/suites/rgw/crypt/2-kms/barbican.yaml:
remove settings to bootstrap keystone
Kefu Chai [Thu, 28 May 2020 16:51:39 +0000 (00:51 +0800)]
qa/suites/rgw/tempest: use the latest tempest supporting py3.5
in case we need to use ubuntu xenial for testing, xenial only had python
3.5 packaged. and tempest 23.0 was the last version which supports
python3.5 and python2.7.
also do not replace link in tox.ini, as it is reachable.
to address the issues of
- pallets/markupsafe#116
- pypa/setuptools#2017
MarkupSafe is installed by
https://opendev.org/openstack/requirements/raw/branch/stable/pike/upper-constraints.txt
Kefu Chai [Mon, 25 May 2020 07:52:04 +0000 (15:52 +0800)]
qa/suites/rgw/tempest: bump up keystone to 17.0.0
* also generate a sample conf file following the document at
https://github.com/openstack/keystone/tree/17.0.0.0rc2/etc
* use "projects" instead of "tenants" to match the terminology used by
openstack identify API 3.0.
* test API 3.0 instead of API 2.0, by changing
`rgw_keystone_api_version` from "2" to "3"
* explicitly specify a domain "default" for project to be created,
otherwise a POST request will fail with:
```
{"error":{"code":400,"message":"You have tried to create a resource using the admin token. As this token is not within a domain you must explicitly include a domain for this resource to belong
to.","title":"Bad Request"}}
````
* create "default" domain, and use it, othewise a GET request fails
like:
```
2020-05-28T11:17:28.751 INFO:teuthology.orchestra.run.smithi092.stderr:http://smithi092.front.sepia.ceph.com:35357 "GET /v3/domains/default HTTP/1.1" 404 87
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP: [404] Content-Length: 87 Content-Type: application/json Date: Thu, 28 May 2020 11:17:28 GMT Server: WSGIServer/0.2
CPython/3.6.9 Vary: X-Auth-Token x-openstack-request-id: req-bc33796f-2bc3-411c-a7fb-1208918e0dbd
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP BODY: {"error":{"code":404,"message":"Could not find domain: default.","title":"Not Found"}}
```
* add user to "default" domain when creating it.
* use "type" as the positional argument, per
https://docs.openstack.org/keystone/pike/admin/cli-keystone-manage-services.html
otherwise we will have failures like:
```
2020-05-28T13:38:24.867 INFO:teuthology.orchestra.run.smithi198.stderr:openstack service create: error: unrecognized arguments: --type keystone
```
* update `create_endpoint()` to use the V3 API,
see
https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/endpoint.html
Kefu Chai [Thu, 28 May 2020 15:14:35 +0000 (23:14 +0800)]
qa/tasks/keystone.py: support multiple positional args
it's required when creating endpoint, see
https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/endpoint.html,
where we need to pass <service>, <interface>, and <url>
* refs/pull/34672/head:
qa/tasks/cephfs: Enable multiple exports tests
mgr/nfs: Instead of 'auth del' use 'auth rm'
qa/tasks/cephfs: Don't enable cephadm in TestNFS
qa/tasks/cephfs: Add tests for nfs exports
mgr/volumes/nfs: Fix idempotency of cluster and export commands
mgr/volumes/nfs: Fix incorrect read only access_type value
mgr/fs/nfs: Use check_mon_command() instead of mon_command()
qa/cephfs: Add tests for nfs
mgr/volumes/nfs: Remove type option from export create interface
vstart: Instead of CACHEINODE use MDCACHE
mgr/volumes: Rearrange nfs export interface
mgr/volumes/nfs: Delete common config object on cluster deletion
mgr/volumes/nfs: Delete all exports on cluster deletion
mgr/volumes: Make nfs create export interface idempotent
vstart: Add watch url for conf-nfs object
mgr/volumes/nfs: Delete user on removing export
mgr/volumes: Create user for given path and fs
vstart: Ensure cephadm and NFS does not conflict
vstart: Update details about ganesha packages
mgr/volumes/nfs: Add delete cephfs export command
mgr/volumes/nfs: Add RADOS notify for common config object
mgr/volumes/nfs: Pass cluster_id directly to NFSCluster {create, update, delete} methods
mgr/volumes: Add nfs cluster delete interface
mgr/volumes: Add nfs cluster update interface
vstart: Enable test_orchestrator in start_ganesha()
mgr/volumes: Add placement option to create nfs cluster interface
mgr/volumes/nfs: Change common ganesha config object name to 'conf-nfs.ganesha-<cluster_id>'
mgr/volumes/nfs: Call orch nfs apply
Reviewed-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: Sebastian Wagner <swagner@suse.com> Reviewed-by: Ramana Raja <rraja@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Rishabh Dave [Tue, 26 May 2020 11:10:29 +0000 (16:40 +0530)]
qa/cephfs: look for mountpoint in cmdline file
Look for self.mountpoint in the contents of /proc/<pid>/cmdline file
when finding asok file for the client so that vstart_runner.py won't end
up picking asok file for a client not created in current run.
This usually never happens so far because PID of newly created processes
is higher than that of previously created processes and list of asok
files returned by "glob.glob(asok_path)" in find_socket() is in
descending order of PIDs.
mds: forward mds metrics to ceph manager w/ quering interfaces
`MetricAggregator` sets up manager callback to forward metrics
data to ceph-mgr. Also, add querying interfaces for adding and
removing user queries and a simple interface to fetch MDS perf
metrics.
Venky Shankar [Mon, 4 Feb 2019 13:19:58 +0000 (18:49 +0530)]
mds: track per session client metrics
Every MDS maintains a view of metrics that are forwarded to
it by clients. This is updated when clients forward metrics
via MClientMetrics message type. Periodically, each MDS
forwards its collected metrics to MDS rank 0 (which maintains
an aggregated view of metrics of all clients on all ranks).
Venky Shankar [Mon, 5 Aug 2019 09:32:28 +0000 (05:32 -0400)]
mds: record metrics from all MDSs in MDS rank 0
`MetricAggregator` class aggregates metrics from all active ranks
and places metrics appropriately as defined by user queries.
This is implemented as a separate dispatcher since metric update
messages from active MDSs are frequent so as to avoid messages
getting stuck in MDSRank queue (suggested by Patrick).
Ilya Dryomov [Sat, 30 May 2020 17:03:20 +0000 (19:03 +0200)]
librbd: make rbd_read_from_replica_policy actually work
Commit d92be8b5856d ("librbd: Improved support for balanced and
localized reads") introduced read_flags, but used low-level Objecter
flags (CEPH_OSD_FLAG_*). This is wrong because librbd sits on top of
librados, not Objecter. "balance" ends up being translated into
IGNORE_REDIRECT and "localize" being ignored.
Wencong Wan [Tue, 19 May 2020 18:23:01 +0000 (02:23 +0800)]
librbd: support duration controllable continuous burst
Modify the design of token bucket to save more tokens than burst value,
so as to support continuous burst. The desired burst duration can be
controlled by rbd_qos_XYZ_burst_seconds.
Fixes: https://tracker.ceph.com/issues/45313 Signed-off-by: wencong wan <wanwc@chinatelecom.cn>
projects / ceph.git / log