Kefu Chai [Sat, 30 May 2020 04:51:14 +0000 (12:51 +0800)]
rgw/reshard: use defined variable
use the defined reference for more concise code, this silences the
warning like:
```
../src/rgw/rgw_reshard.cc:530:15: warning: unused variable ‘bucket’ [-Wunused-variable]
530 | rgw_bucket& bucket = bucket_info.bucket;
| ^~~~~~
```
also move `ret` close to where it is used for the first time.
* refs/pull/34782/head:
qa/tasks/cephfs/mount.py: remove netns name parsing in mountpoint setter
qa/tasks/vstart_runner.py: add kwargs parameter to ignore the ones it does not understand
Reviewed-by: Rishabh Dave <ridave@redhat.com> Reviewed-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Lenz Grimmer [Tue, 2 Jun 2020 13:11:20 +0000 (15:11 +0200)]
Merge pull request #35249 from rhcs-dashboard/wip-45705-master
mgr/dashboard: add API team to CODEOWNERS
Reviewed-by: Alfonso Martínez <almartin@redhat.com> Reviewed-by: Laura Paduano <lpaduano@suse.com> Reviewed-by: Stephan Müller <smueller@suse.com> Reviewed-by: Tatjana Dehler <tdehler@suse.com> Reviewed-by: Volker Theile <vtheile@suse.com>
Kefu Chai [Sun, 31 May 2020 00:47:34 +0000 (08:47 +0800)]
qa/suites/rgw/tempest: update unsupported tests of tempest
after rerunning tempest with lastest radosgw, remove the supported
tests from the blacklist, and add the ones which are not supported
yet. now we can pass 123 tests in total.
also enable discoverity for better testing coverage, since it's
supported now.
Kefu Chai [Sun, 31 May 2020 00:38:00 +0000 (08:38 +0800)]
qa/tasks/keystone: use "keystone-manage bootstrap"
* qa/tasks/keystone.py:
instead of prefilling keystone manually, use "keystone-manage bootstrap"
instead. it helps to setup the admin user, a "Default" domain with
"default" id, and wire them up with the expected role and a "admin" project,
etc. as id of the admin domain is known to be "default", we can just use it
in our tests without querying openstack for the id of "Default"
domain. this is very handy.
* qa/suites/rgw/tempest/tasks/rgw_tempest.yaml:
use "Default" for domain name. as "Default" is the name of the domain
created by bootstrap, while "default" is its id.
* qa/suites/rgw/crypt/2-kms/barbican.yaml:
remove settings to bootstrap keystone
Kefu Chai [Thu, 28 May 2020 16:51:39 +0000 (00:51 +0800)]
qa/suites/rgw/tempest: use the latest tempest supporting py3.5
in case we need to use ubuntu xenial for testing, xenial only had python
3.5 packaged. and tempest 23.0 was the last version which supports
python3.5 and python2.7.
also do not replace link in tox.ini, as it is reachable.
to address the issues of
- pallets/markupsafe#116
- pypa/setuptools#2017
MarkupSafe is installed by
https://opendev.org/openstack/requirements/raw/branch/stable/pike/upper-constraints.txt
Kefu Chai [Mon, 25 May 2020 07:52:04 +0000 (15:52 +0800)]
qa/suites/rgw/tempest: bump up keystone to 17.0.0
* also generate a sample conf file following the document at
https://github.com/openstack/keystone/tree/17.0.0.0rc2/etc
* use "projects" instead of "tenants" to match the terminology used by
openstack identify API 3.0.
* test API 3.0 instead of API 2.0, by changing
`rgw_keystone_api_version` from "2" to "3"
* explicitly specify a domain "default" for project to be created,
otherwise a POST request will fail with:
```
{"error":{"code":400,"message":"You have tried to create a resource using the admin token. As this token is not within a domain you must explicitly include a domain for this resource to belong
to.","title":"Bad Request"}}
````
* create "default" domain, and use it, othewise a GET request fails
like:
```
2020-05-28T11:17:28.751 INFO:teuthology.orchestra.run.smithi092.stderr:http://smithi092.front.sepia.ceph.com:35357 "GET /v3/domains/default HTTP/1.1" 404 87
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP: [404] Content-Length: 87 Content-Type: application/json Date: Thu, 28 May 2020 11:17:28 GMT Server: WSGIServer/0.2
CPython/3.6.9 Vary: X-Auth-Token x-openstack-request-id: req-bc33796f-2bc3-411c-a7fb-1208918e0dbd
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP BODY: {"error":{"code":404,"message":"Could not find domain: default.","title":"Not Found"}}
```
* add user to "default" domain when creating it.
* use "type" as the positional argument, per
https://docs.openstack.org/keystone/pike/admin/cli-keystone-manage-services.html
otherwise we will have failures like:
```
2020-05-28T13:38:24.867 INFO:teuthology.orchestra.run.smithi198.stderr:openstack service create: error: unrecognized arguments: --type keystone
```
* update `create_endpoint()` to use the V3 API,
see
https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/endpoint.html
Kefu Chai [Thu, 28 May 2020 15:14:35 +0000 (23:14 +0800)]
qa/tasks/keystone.py: support multiple positional args
it's required when creating endpoint, see
https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/endpoint.html,
where we need to pass <service>, <interface>, and <url>
* refs/pull/34672/head:
qa/tasks/cephfs: Enable multiple exports tests
mgr/nfs: Instead of 'auth del' use 'auth rm'
qa/tasks/cephfs: Don't enable cephadm in TestNFS
qa/tasks/cephfs: Add tests for nfs exports
mgr/volumes/nfs: Fix idempotency of cluster and export commands
mgr/volumes/nfs: Fix incorrect read only access_type value
mgr/fs/nfs: Use check_mon_command() instead of mon_command()
qa/cephfs: Add tests for nfs
mgr/volumes/nfs: Remove type option from export create interface
vstart: Instead of CACHEINODE use MDCACHE
mgr/volumes: Rearrange nfs export interface
mgr/volumes/nfs: Delete common config object on cluster deletion
mgr/volumes/nfs: Delete all exports on cluster deletion
mgr/volumes: Make nfs create export interface idempotent
vstart: Add watch url for conf-nfs object
mgr/volumes/nfs: Delete user on removing export
mgr/volumes: Create user for given path and fs
vstart: Ensure cephadm and NFS does not conflict
vstart: Update details about ganesha packages
mgr/volumes/nfs: Add delete cephfs export command
mgr/volumes/nfs: Add RADOS notify for common config object
mgr/volumes/nfs: Pass cluster_id directly to NFSCluster {create, update, delete} methods
mgr/volumes: Add nfs cluster delete interface
mgr/volumes: Add nfs cluster update interface
vstart: Enable test_orchestrator in start_ganesha()
mgr/volumes: Add placement option to create nfs cluster interface
mgr/volumes/nfs: Change common ganesha config object name to 'conf-nfs.ganesha-<cluster_id>'
mgr/volumes/nfs: Call orch nfs apply
Reviewed-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: Sebastian Wagner <swagner@suse.com> Reviewed-by: Ramana Raja <rraja@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Ilya Dryomov [Sat, 30 May 2020 17:03:20 +0000 (19:03 +0200)]
librbd: make rbd_read_from_replica_policy actually work
Commit d92be8b5856d ("librbd: Improved support for balanced and
localized reads") introduced read_flags, but used low-level Objecter
flags (CEPH_OSD_FLAG_*). This is wrong because librbd sits on top of
librados, not Objecter. "balance" ends up being translated into
IGNORE_REDIRECT and "localize" being ignored.
Wencong Wan [Tue, 19 May 2020 18:23:01 +0000 (02:23 +0800)]
librbd: support duration controllable continuous burst
Modify the design of token bucket to save more tokens than burst value,
so as to support continuous burst. The desired burst duration can be
controlled by rbd_qos_XYZ_burst_seconds.
Fixes: https://tracker.ceph.com/issues/45313 Signed-off-by: wencong wan <wanwc@chinatelecom.cn>
Jason Dillaman [Thu, 28 May 2020 20:38:40 +0000 (16:38 -0400)]
librbd: Watcher should not attempt to re-watch after detecting blacklisting
Currently, the Watcher state machine will spin as fast as it can sending
re-watch requests to the OSD and then retrying after it fails with the
EBLACKLISTED error. Treat a blacklisting similarly to how removal of the
object is treated: stop attempting to re-watch.
Fixes: https://tracker.ceph.com/issues/45715 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
The balancer was turned on by default in d4fbaf7ea959fd945857abd327271a97fb1da631, as a result of which we might see
PG_AVAILABILITY health warnings when pg-upmap-items are applied.
The balancer was turned on by default in d4fbaf7ea959fd945857abd327271a97fb1da631, as a result of which we might see
PG_AVAILABILITY health warnings when pg-upmap-items are applied.
Matthew Oliver [Thu, 28 May 2020 01:59:19 +0000 (11:59 +1000)]
cephadm: Detect stale and then recreate connections
Currently we make and cache connections to nodes during a check_host.
If a cached connection is disconnect from the other end the remoto
connection object doesn't track this, so further checks to the host
fail.
I have pushed up a PR[0] to remoto to add a `has_connection` method to
their `BaseConnection` class, which we now use in this patch to check to
see if the connection is stale. If it is it is then recreated.
There is some monkey patching happening so we can add the required
`has_connection` to remoto in this patch which we can remove as soon as
the other PR have landed and a new version of remoto is released.
[0] https://github.com/alfredodeza/remoto/pull/56 Fixes: https://tracker.ceph.com/issues/45627 Fixes: https://tracker.ceph.com/issues/45032 Signed-off-by: Matthew Oliver <moliver@suse.com>
Changcheng Liu [Tue, 21 Apr 2020 05:38:53 +0000 (13:38 +0800)]
src: extract backend driver from bluestore to access device
1. Both bluestore or other component e.g. rbd could use the same
driver to access storage device. It's better to use one library
to integrate the driver.
2. os and crimson-alienstore are static libraries. Link blk library into
them.
Main changes are below:
1. move backend driver into src/blk
src/$ mkdir -p blk/{aio,kernel,spdk,pmem,zns}
src/$ mv os/bluestore/{BlockDevice.h,BlockDevice.cc} blk
src/$ mv os/bluestore/{ceph_aio.h} aio/aio.h
src/$ mv os/bluestore/{aio.cc} aio/
src/$ mv os/bluestore/{KernelDevice.h,KernelDevice.cc} kernel/
src/$ mv os/bluestore/{ceph_io_uring.h} kernel/io_uring.h
src/$ mv os/bluestore/{io_uring.cc} kernel_drv/
src/$ mv os/bluestore/{NVMEDevice.h,NVMEDevice.cc} spdk/
src/$ mv os/bluestore/{PMEMDevice.h,PMEMDevice.cc} pmem/
src/$ mv os/bluestore/{HMSMRDevice.h,HMSMRDevice.cc} zns/
2. customize macro name in header file to remove bluestore specific text
3. adjust header file patch in source code
4. create cmake rule blk/CMakeLists.txt to build blk
5. modify src/CMakeLists.txt to integrate blk
6. modify other CMakeLists.txt to adapt to new file structure.
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com> Co-authored-by: Kefu Chai <kefu@redhat.com>
projects / ceph.git / log