]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Yuval Lifshitz [Wed, 17 Apr 2019 08:02:51 +0000 (11:02 +0300)]
rgw/pubsub: fix "no module named rgw_multi.zone_ps"
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
dfa501cfb77e72fe3e4cb2454aea99916705b812 )
Yuval Lifshitz [Tue, 16 Apr 2019 11:44:57 +0000 (14:44 +0300)]
rgw/pubsub: fix "no module named rgw_multi.tests"
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
b03a56fe930be0ff7bad4e375e22a8e2ca0fed76 )
Yuval Lifshitz [Mon, 15 Apr 2019 11:19:45 +0000 (14:19 +0300)]
rgw/pubsub: actually adding the ps tests
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
8362ecc7dfd8056567b74fdaaa52809082f15421 )
Yuval Lifshitz [Mon, 15 Apr 2019 06:01:03 +0000 (09:01 +0300)]
rgw/pubsub: fix popping the wrong key
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
2d7c07bbeb9887d76b89eded7458ac7af8068026 )
Yuval Lifshitz [Thu, 18 Apr 2019 08:21:26 +0000 (11:21 +0300)]
rgw/pubsub: fix topic arn. tenant support to multisite tests
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
be64adf916e0f318b22b57cedd02c2b0b2a503f2 )
Yuval Lifshitz [Thu, 11 Apr 2019 17:16:52 +0000 (20:16 +0300)]
rgw/pubsub: handle subscription conf errors better
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
f4e3745bcb9f75814b4052e2b651c25a39c2f17b )
Yuval Lifshitz [Thu, 11 Apr 2019 09:23:17 +0000 (12:23 +0300)]
rgw/pubsub: more info on notification deletion compatibility
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit
2cf1872bdf63e9093884b6d3bbc20b9a42963d53 )
Yuval Lifshitz [Wed, 10 Apr 2019 16:32:02 +0000 (19:32 +0300)]
rgw/pubsub: fix comments from PR #27493
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit
cb256dd92ac39baddce252acb634b587653da4f3 )
Yuval Lifshitz [Wed, 10 Apr 2019 12:38:35 +0000 (15:38 +0300)]
rgw/pubsub: clarify pubsub zone configuration
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit
89ddc407b48959831450638f14929f553511d240 )
Yuval Lifshitz [Wed, 3 Apr 2019 11:27:49 +0000 (14:27 +0300)]
rgw/pubsub: wrong link in S3 doc
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
98e6a0434609d7b29d61166276419eca1d552ad0 )
Yuval Lifshitz [Wed, 3 Apr 2019 10:29:47 +0000 (13:29 +0300)]
rgw/pubsub: fix documentation link errors
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
dc904249a00aacecf9a06a26223b5b2d5c46ace2 )
Yuval Lifshitz [Tue, 2 Apr 2019 16:27:21 +0000 (19:27 +0300)]
rgw/pubsub: fix comments from PR #27091
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
a0b43fa26c9ba1bebe6553c84fd98fad3a3d28ad )
Yuval Lifshitz [Tue, 2 Apr 2019 11:46:27 +0000 (14:46 +0300)]
rgw/pubsub: add more S3 compatibility documentation
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
e1e39d627ec9a044f9f7428400dd379350fc4bbb )
Yuval Lifshitz [Mon, 1 Apr 2019 09:28:11 +0000 (12:28 +0300)]
rgw/pubsub: test and doc bucket deletion impact on notifications
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
cb6e894937098999e4eb7d2c13c2f180329f2432 )
Yuval Lifshitz [Sun, 31 Mar 2019 19:52:27 +0000 (22:52 +0300)]
rgw/pubsub: avoid static creation of amqp manager
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
376a3df0d12b544a0378a1467fcf4b2c32fd8afa )
Yuval Lifshitz [Sun, 31 Mar 2019 08:34:05 +0000 (11:34 +0300)]
rgw/pubsub: add s3-compatible API documentation
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
9fd0f873188532c7552f087f65c5b279eb1fc3ca )
Yuval Lifshitz [Wed, 27 Mar 2019 15:47:18 +0000 (17:47 +0200)]
rgw/pubsub: allow for endpoint definition via topics
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
8ac119fdd50b6306e4b338e3f0b4731818c6e6c8 )
Yuval Lifshitz [Wed, 20 Mar 2019 16:43:36 +0000 (18:43 +0200)]
rgw/pubsub: implement S3 compatible get/delete APIs
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
0f8671b3a13af2713fed51f15c83cca8ef19dd19 )
Yuval Lifshitz [Wed, 13 Mar 2019 17:54:16 +0000 (19:54 +0200)]
rgw: pubsub support s3 records. refactor ARN
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit
2bd353233112ead32181b94a2af1b04e3fa9e1de )
Yuval Lifshitz [Wed, 27 Feb 2019 17:13:08 +0000 (19:13 +0200)]
rgw/pubsub: initial version of S3 compliant API
(cherry picked from commit
2e94f4d1cecb5164bc646fb6b088dd197f878263 )
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
Nathan Cutler [Wed, 18 Sep 2019 12:51:25 +0000 (14:51 +0200)]
Merge pull request #29487 from rhcs-dashboard/nautilus-run-dashboard-api-tests
nautilus: mgr/dashboard: run-backend-api-tests.sh improvements
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Alfredo Deza [Tue, 17 Sep 2019 14:00:31 +0000 (10:00 -0400)]
Merge tag 'v14.2.4' into nautilus
v14.2.4
This was required because the release was done with ceph-volume changes
only, and omitting the other code changes that were untested.
Alfonso Martínez [Thu, 22 Aug 2019 13:33:02 +0000 (15:33 +0200)]
mgr/dashboard: run-backend-api-tests.sh CI improvements
As there is now a jenkins job to run this script
(see https://github.com/ceph/ceph-build/pull/1351),
this refactoring adapt the script to be run in a jenkins job as well as locally.
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit
7563f8b6d661886b7b3c7a259b70433a8962f55a )
Kefu Chai [Wed, 31 Jul 2019 08:20:33 +0000 (16:20 +0800)]
pybind/mgr: install setuptools >= 12
otherwise we have
```
Running setup.py egg_info for package apache-libcloud
libcloud/utils/py3.py:76: UserWarning: Missing backports.ssl_match_hostname package
warnings.warn("Missing backports.ssl_match_hostname package")
Traceback (most recent call last):
...
File "build/bdist.linux-x86_64/egg/setuptools_scm/version.py", line 66, in _warn_if_setuptools_outdated
...
setuptools_scm.version.SetuptoolsOutdatedWarning: your setuptools is too old (<12)
...
Command python setup.py egg_info failed with error code 1 in /tmp/tmp.i95Fg82Ea4/venv/build/apache-libcloud
```
and more important thing is, setuptools >= 12 allows us to install the
python modules in venv, without uninstalling existing modules if they
are also installed in by system packages.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
2094c2f2fd6ac0f2028fcaf22f2b51a306e60172 )
Alfonso Martínez [Tue, 9 Jul 2019 07:03:09 +0000 (09:03 +0200)]
mgr/dashboard: set python binary for teuthology venv
* Added env. var. to set python binary when creating teuthology venv.
* venv/bin/activate: run after setting PATH (as it does the last PATH export).
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit
522cbe8a731cf01a48dd57b6169c66e4965ba272 )
Alfredo Deza [Fri, 13 Sep 2019 18:07:41 +0000 (14:07 -0400)]
14.2.4
Alfredo Deza [Fri, 13 Sep 2019 18:07:41 +0000 (14:07 -0400)]
14.2.4
Alfredo Deza [Mon, 9 Sep 2019 22:55:13 +0000 (18:55 -0400)]
ceph-volume tests create a test file for checking unicode output
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Mon, 9 Sep 2019 22:54:51 +0000 (18:54 -0400)]
ceph-volume tests create a shell test for functional unicode
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Mon, 9 Sep 2019 19:20:13 +0000 (15:20 -0400)]
ceph-volume tests verify new logging fallback and encodings in terminal
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Mon, 9 Sep 2019 17:44:17 +0000 (13:44 -0400)]
ceph-volume create a logger for the terminal
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Mon, 9 Sep 2019 17:44:36 +0000 (13:44 -0400)]
ceph-volume: instantiate the new terminal logger in main()
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Mon, 9 Sep 2019 17:45:08 +0000 (13:45 -0400)]
ceph-volume terminal remove unicode stream handler
This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Thu, 12 Sep 2019 13:20:35 +0000 (09:20 -0400)]
Merge pull request #29769 from alfredodeza/nautilus-bz-
1738379
nautilus ceph-volume: use the OSD identifier when reporting success
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Alfredo Deza [Thu, 12 Sep 2019 10:55:21 +0000 (06:55 -0400)]
Merge pull request #30294 from alfredodeza/nautilus-rm41378-2
nautilus ceph-volume: pre-install python-apt and its variants before test runs
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Jan Fajerski [Thu, 12 Sep 2019 10:05:55 +0000 (12:05 +0200)]
Merge pull request #30307 from alfredodeza/nautilus-rm40664
nautilus ceph-volume zap always skips block.db, leaves them around
Alfredo Deza [Wed, 11 Sep 2019 11:35:48 +0000 (07:35 -0400)]
Merge pull request #30300 from alfredodeza/nautilus-rm41660
nautilus: ceph-volume: fix stderr failure to decode/encode when redirected
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Alfredo Deza [Thu, 11 Jul 2019 15:56:07 +0000 (11:56 -0400)]
ceph-volume tests: verify that wipefs tries several times
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
845e62cb4ba2636325304db2c2d65a185ace25cf )
Alfredo Deza [Thu, 11 Jul 2019 13:34:10 +0000 (09:34 -0400)]
ceph-volume lvm.zap: retry wipefs several times to prevent race condition failures
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
b7b8b106fa485d3b8345b583fbc7f74e30cbc03e )
Alfredo Deza [Mon, 9 Sep 2019 22:55:13 +0000 (18:55 -0400)]
ceph-volume tests create a test file for checking unicode output
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
79196354aca3b3d30c7de1e2ac8b5a3c06ee8330 )
Alfredo Deza [Mon, 9 Sep 2019 22:54:51 +0000 (18:54 -0400)]
ceph-volume tests create a shell test for functional unicode
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
595e492783283359b613d199a75f3e7495bfb9ed )
Alfredo Deza [Mon, 9 Sep 2019 19:20:13 +0000 (15:20 -0400)]
ceph-volume tests verify new logging fallback and encodings in terminal
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
ee18ebc078b2d380b88f3726737184f34345e2f3 )
Alfredo Deza [Mon, 9 Sep 2019 17:45:08 +0000 (13:45 -0400)]
ceph-volume terminal remove unicode stream handler
This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
fe66e01f6a4b19bb15695537713a629a951b28ce )
Alfredo Deza [Mon, 9 Sep 2019 17:44:36 +0000 (13:44 -0400)]
ceph-volume: instantiate the new terminal logger in main()
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
bfb8422e262dcce49c50610a3977bfe1edc2a435 )
Alfredo Deza [Mon, 9 Sep 2019 17:44:17 +0000 (13:44 -0400)]
ceph-volume create a logger for the terminal
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
95b16b516c0736839fe50f12ecbeb7cd62c67335 )
Alfredo Deza [Mon, 9 Sep 2019 17:40:06 +0000 (13:40 -0400)]
ceph-volume create a new tox.ini for shell-based tests
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
4612ab3ae62a3e8e1cc9e51489feab0c938296d3 )
Alfredo Deza [Tue, 3 Sep 2019 14:00:14 +0000 (10:00 -0400)]
ceph-volume tests pre-instrall python-apt to prevent auto-install failing later
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
40cac7b9510f7904c2f9dfb1294d809e04bdcf7e )
Kefu Chai [Tue, 10 Sep 2019 12:36:24 +0000 (20:36 +0800)]
Merge pull request #30283 from tchaikov/wip-nautilus-c++17
nautilus: cmake: enforce C++17 instead of relying on cmake-compile-features
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Kefu Chai [Tue, 10 Sep 2019 06:56:59 +0000 (14:56 +0800)]
cmake: enforce C++17 instead of relying on cmake-compile-features
this change is a follow-up of #30114
in FindBoost.cmake, we try to deduce the C++ flags by checking the C++
features offered by different C++ standard like C++98, C++11 and C++14
against the ones required by different boost component. then choose the
C++ dialect based on the `CXX_STANDARD` and `COMPILE_FEATURES`.
in the case of nautilus, since cmake 3.5.1 does not offer
CXX_STANDARD=17 support by then, we have to add `-std=c++17` to
`CMAKE_CXX_FLAGS` manually. but unfortunately, the
`cmake-compile-features` machinary does not check `CMAKE_CXX_FLAGS` for
the default C++ standard required by the project. so it falls back to
the minimal C++ standard required by the target.
since Boost::fiber and Boost::context share the same C++ feature set
dependencies which can be fulfilled by C++11, and by default,
`CXX_EXTENSIONS` is enabled, `--std=gnu++11` is used as we are using gcc
in our building hosts. so we have CXX_FLAGS like:
-std=c++17 ... -std=gnu++11
when compiling targets linked against Boost::context. and it seems the
last one takes effect. that's why some targets failed to compile if
the makefiles were generated with cmake 3.5.1.
in this change, `INTERFACE_COMPILE_FEATURES <features>` is replaced with
`CXX_STANDARD 17`. this renders this setting a no-op, as CMake 3.5.1
does not translate this setting to any CXX_FLAGS. but this does not
hurt, as we use C++17 project-wide, and C++17 is able to offer all
compile features required by Boost::context.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: this change is not cherry-picked from master, because in
master, we use cmake 3.10 and up which offers the support of
`CXX_STANDARD=17`, hence cmake is able to bump up the C++ standard
to C++17 instead of falling back to C++11.
Nathan Cutler [Thu, 5 Sep 2019 13:07:54 +0000 (15:07 +0200)]
Merge pull request #30114 from tchaikov/wip-nautilus-CXX_FLAGS
nautilus: seastar,dmclock: use CXX_FLAGS from parent project
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
David Galloway [Wed, 4 Sep 2019 00:35:16 +0000 (20:35 -0400)]
Merge branch 'nautilus' of github.com:ceph/ceph into nautilus
Kefu Chai [Tue, 3 Sep 2019 16:53:24 +0000 (00:53 +0800)]
dmclock: pick up change to use specified C++ settings if any
see https://github.com/ceph/dmclock/pull/73
so dmclock won't pass `-std=c++11` even if the parent project already
have `-std=c++17` in CXX_FLAGS.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
fb83b331de9636e5de3d756e490748d5bfb9c4dc )
Conflicts: dmclock
this change is not cherry-picked from master, because master is using
a different dmclock version. so in this change, only the minimum fix
is included.
Kefu Chai [Tue, 3 Sep 2019 16:47:54 +0000 (00:47 +0800)]
seastar: pick up change in fmt submodule
see https://github.com/ceph/fmt/pull/1
so fmt won't fallback to C++11 if `CMAKE_CXX_STANDARD` is not set
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change is not cherry-picked from master, because
- the seastar submodule in nautilus is not in sync with that in
master
- master is not using fmt submodule included by seastar anymore
- master is using cmake 3.10, so there is no need to address
this issue in master on fmt front.
Alfredo Deza [Tue, 3 Sep 2019 14:58:55 +0000 (10:58 -0400)]
Merge pull request #30093 from jan--f/wip-41614-nautilus
nautilus: ceph-volume: lvm list is O(n^2)
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Jenkins Build Slave User [Tue, 3 Sep 2019 13:19:58 +0000 (13:19 +0000)]
14.2.3
Rishabh Dave [Mon, 19 Aug 2019 13:33:21 +0000 (19:03 +0530)]
ceph-volume: test number of times LVs list was created
Add a test that cofirm that the LVs list was created only once when
calling devices.lvm.listing.list()
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit
20d6264538dab444c0ef13f40c6c21ec3d2057f8 )
Rishabh Dave [Wed, 14 Aug 2019 15:57:38 +0000 (21:27 +0530)]
ceph-volume: reuse list of LVs
Avoid creating of list of LVs in different methods of the same module
and resue them.
Fixes: https://tracker.ceph.com/issues/37490
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit
ab3a58e78124a984d162b2f63992d01c5e59e2a7 )
Yuri Weinstein [Thu, 29 Aug 2019 14:43:24 +0000 (07:43 -0700)]
Merge pull request #29945 from trociny/wip-41475-nautilus
nautilus: pybind/mgr/rbd_support: ignore missing support for RBD namespaces
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Thu, 29 Aug 2019 14:42:53 +0000 (07:42 -0700)]
Merge pull request #29975 from theanalyst/nautilus-beast-endpoint-fix
nautilus: rgw: asio: check the remote endpoint before processing requests
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Wed, 28 Aug 2019 19:03:15 +0000 (12:03 -0700)]
Merge pull request #29977 from yuriw/wip-yuriw-41513-nautilus
qa/tests: adding mgr.x into the restart/upgrade sequence before monitors
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Wed, 28 Aug 2019 18:49:59 +0000 (11:49 -0700)]
qa/tests: adding mgr.x into the restart/upgrade sequence before monitors
Fixes: https://tracker.ceph.com/issues/41513
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Abhishek Lekshmanan [Wed, 7 Aug 2019 15:09:32 +0000 (17:09 +0200)]
rgw: asio: check the remote endpoint before processing requests
`socket.remote_endpoint()` can throw exceptions corresponding to errors in the
`getpeername` syscall, make sure these are handled.
Fixes: CVE-2019-10222, https://tracker.ceph.com/issues/40018
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit
caa653196856ecdf50519a9a33195d5c4e3372af )
Conflicts:
src/rgw/rgw_asio_frontend.cc
conflicts due to missing yield ctx in nautilus
Jason Dillaman [Wed, 31 Jul 2019 18:19:28 +0000 (14:19 -0400)]
pybind/mgr/rbd_support: ignore missing OSD support for RBD namespaces
During upgrades, the OSDs might not yet have been upgraded yet and
therefore might be missing namespace support.
Fixes: https://tracker.ceph.com/issues/41029
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
10c0c806819741012e1c440c38fac17682125d9a )
Jason Dillaman [Wed, 31 Jul 2019 18:13:54 +0000 (14:13 -0400)]
pybind/rbd: new OperationNotSupported exception
This will help catch errors where the backing OSDs do not support
a given RBD feature.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
97c1a120b7e7d934dd5d2456fd1fb4c1d0e891ec )
Yuri Weinstein [Tue, 27 Aug 2019 18:38:58 +0000 (11:38 -0700)]
Merge pull request #29918 from badone/wip-nautilus-tracker-41518-grafana-server
nautilus: qa/ceph-ansible: Disable dashboard
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Alfredo Deza [Tue, 27 Aug 2019 11:48:27 +0000 (07:48 -0400)]
Merge pull request #29899 from alfredodeza/nautilus-rm41378
nautilus ceph-volume tests set the noninteractive flag for Debian
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Brad Hubbard [Tue, 27 Aug 2019 03:50:47 +0000 (13:50 +1000)]
nautilus: qa/ceph-ansible: Disable dashboard
Recently a dashboard playbook was added which will fail without a
grafana server so disabling the dashboard
Fixes: https://tracker.ceph.com/issues/41521
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit
131f5a145adf21d402e39ac2c151664eb8ccec76 )
Alfredo Deza [Wed, 21 Aug 2019 18:15:32 +0000 (14:15 -0400)]
ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
89231c9a60a46ba0691f787efbfe5ab35f05c679 )
Yuri Weinstein [Thu, 22 Aug 2019 16:39:37 +0000 (09:39 -0700)]
Merge pull request #29805 from yuriw/wip-yuriw-41384-nautilus
qa/tests: changed running rbd tests test_librbd_python.sh from…
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Wed, 21 Aug 2019 19:38:00 +0000 (12:38 -0700)]
qa/tests: changed running rbd tests test_librbd_python.sh from tag: v14.2.2
Fixes: https://tracker.ceph.com/issues/41384
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Wed, 21 Aug 2019 19:14:08 +0000 (12:14 -0700)]
Merge pull request #29801 from smithfarm/wip-41263-nautilus
nautilus: rgw_file: advance_mtime() takes RGWFileHandle::mutex unconditionally
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Matt Benjamin [Thu, 8 Aug 2019 20:21:24 +0000 (16:21 -0400)]
rgw_file: dont deadlock in advance_mtime()
Fixes: https://tracker.ceph.com/issues/41173
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
0d3338c2d2271bbf6f4c7774ad175f7ee42d07f6 )
Yuri Weinstein [Wed, 21 Aug 2019 15:25:37 +0000 (08:25 -0700)]
Merge pull request #28862 from liewegas/wip-bluefs-extents-nautilus
nautilus: os/bluestore/bluefs_types: consolidate contiguous extents
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Kefu Chai [Wed, 21 Aug 2019 07:23:12 +0000 (15:23 +0800)]
Merge pull request #29191 from mynaramana/patch-1
doc/rados: Correcting some typos in the clay code documentation
Reviewed-by: Kefu Chai <kchai@redhat.com>
Myna V [Fri, 16 Aug 2019 06:28:44 +0000 (11:58 +0530)]
doc/rados: Correcting some typos in the clay code documentation
Signed-off-by: Myna <mynaramana@gmail.com>
Yuri Weinstein [Tue, 20 Aug 2019 21:43:23 +0000 (14:43 -0700)]
Merge pull request #29551 from pdvian/wip-40882-nautilus
nautilus: rbd: Reduce log level for cls/journal and cls/rbd expected errors
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Tue, 20 Aug 2019 21:42:40 +0000 (14:42 -0700)]
Merge pull request #29722 from dillaman/wip-39499-nautilus
nautilus: librbd: snapshot object maps can go inconsistent during copyup
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Tue, 20 Aug 2019 21:42:14 +0000 (14:42 -0700)]
Merge pull request #29723 from dillaman/wip-40511-nautilus
nautilus: librbd: improve journal performance to match expected degredation
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Tue, 20 Aug 2019 21:41:39 +0000 (14:41 -0700)]
Merge pull request #29725 from dillaman/wip-41078-nautilus
nautilus: mgr/rbd_support: support scheduling long-running background operations
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Alfredo Deza [Tue, 20 Aug 2019 12:04:56 +0000 (08:04 -0400)]
ceph-volume devices.lvm zap use the identifier to report success
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
d0d5750c1ba508152c1d5111f2003a039a27145f )
Alfredo Deza [Tue, 20 Aug 2019 12:04:26 +0000 (08:04 -0400)]
ceph-volume tests check success message when zapping
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
88c7429a60fdab6222d14132bea93b9bf773c720 )
Yuri Weinstein [Tue, 20 Aug 2019 16:14:12 +0000 (09:14 -0700)]
Merge pull request #29745 from liewegas/wip-bluestore-no-cgroup-nautilus
nautilus: os/bluestore: do not set osd_memory_target default from cgroup limit
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Sage Weil [Mon, 1 Jul 2019 14:33:22 +0000 (09:33 -0500)]
os/bluestore/bluefs_types: consolidate contiguous extents
If we allocate a new extent that is contiguous with the last extent,
just extend it. This avoids having long vectors of continguous extents
when a single large extent would suffice--especially with log files.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
4f3ce9bc201c20d10d739d42e3f7018408f110a6 )
Sage Weil [Fri, 9 Aug 2019 17:25:59 +0000 (12:25 -0500)]
os/bluestore: do not set osd_memory_target default from cgroup limit
On the aarch64 box I'm testing, this gives us a value of
7378697629483768832 , which is not what we want.
I think we are better off relying on this limit being explicitly set via
environment variables (POD_* by kuberentes/rook) or via the command line.
This partially reverts
5c6b533697814af8acfd9e731a2599b2294687ef , but not
all of it, since we wan to keep the option itself, as it is now used by
common/config.cc when dealing with the POD_MEMORY_LIMIT env var.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
9346d3c3bcde7d0a68e21a554eb7fc836df78ba9 )
Yuri Weinstein [Mon, 19 Aug 2019 16:14:00 +0000 (09:14 -0700)]
Merge pull request #29650 from pdvian/wip-40945-nautilus
nautilus: mgr/dashboard: RGW User quota validation is not working correctly
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Yuri Weinstein [Mon, 19 Aug 2019 16:13:01 +0000 (09:13 -0700)]
Merge pull request #29678 from pdvian/wip-40948-nautilus
nautilus: core: Better default value for osd_snap_trim_sleep
Reviewed-by: Neha Ojha <nojha@redhat.com>
Jason Dillaman [Thu, 13 Jun 2019 19:22:54 +0000 (15:22 -0400)]
librbd: tweaks to improve throughput for journaled IO
Increase the per-object append flush bytes to 1MiB to mimic the 16MiB
in-memory writeback cache. Finally, default enable the new journal
writethrough until flush option.
Performance results comparing no journal, pre-change, and post-change:
no journal original modified
4K 40310.84 7223.73 27427.27
32K 21530.37 3256.39 10284.05
256K 3159.67 564.73 1627.16
4M 223.03 48.03 91.91
Fixes: http://tracker.ceph.com/issues/40072
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
5f3b043896762fa97eb35648f8639004c45f5978 )
Jason Dillaman [Wed, 12 Jun 2019 17:36:24 +0000 (13:36 -0400)]
librbd: new rbd_journal_object_writethrough_until_flush option
When set to true, the journal will not attempt to batch appends until
after it receives the the first flush request from the user.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
172ad649fb8a2410bdc078a5752b2ca09dc03ac2 )
Conflicts:
src/librbd/io/ImageRequest.cc: trivial resolution
Jason Dillaman [Thu, 13 Jun 2019 00:06:11 +0000 (20:06 -0400)]
journal: support dynamically updating recorder flush options
Default to disabling writeback-style append flushes unless overridden
by a call to 'set_append_batch_options'.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
c0322a13c83f590067a120212620ebba15fc8661 )
Jason Dillaman [Thu, 13 Jun 2019 14:21:33 +0000 (10:21 -0400)]
journal: fix broken append batching implementation
The original flush_interval/bytes/age batching options no longer
actually do anything. Integrate these better with the max in-flight
AIO handling.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
a4e160e410ab3743515338510cb1e369a641152f )
Jason Dillaman [Thu, 13 Jun 2019 13:45:32 +0000 (09:45 -0400)]
journal: improve logging on recorder append path
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
84e48ddbf58d4028c4195d9c80634f895f62f2be )
Mykola Golub [Thu, 13 Jun 2019 11:05:17 +0000 (12:05 +0100)]
journal: wait for in flight advance sets on stopping recorder
Before object overflow detection optimization (
83461c42b ) the
overflow was detected when trying to send data, so when an object
set was advanced we always had some data and flush always
completed later. But now we detect overflow on client side, it
may happen when buffer is empty and flush may complete
when "advance object set" is still in flight.
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit
340726ec89c4a2b57905aa1be328d55c96758fee )
Mykola Golub [Fri, 24 May 2019 06:07:51 +0000 (07:07 +0100)]
journal: optimize object overflow detection
Previously to detect overflow we were sending journal append
requests until -EOVERFLOW is returned by osd. This means that we
had at least one waste (rejected) request per object set (though
there may be more if the number of in-flight appends is not
limited).
We can easily predict when the osd will start to return
-EOVERFLOW and avoid such additional requests.
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit
83461c42b03cf7d407ac3c5d02043cca1e908015 )
Casey Bodley [Mon, 19 Aug 2019 14:38:06 +0000 (10:38 -0400)]
Merge pull request #27684 from liewegas/wip-rgw-pgs-nautilus
nautilus: rgw: remove rgw_num_rados_handles; set autoscale parameters or rgw metadata pools
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 19 Aug 2019 14:37:47 +0000 (07:37 -0700)]
Merge pull request #29478 from xiaoxichen/wip-41002
nautilus: cephfs: client: unlink dentry for inode with llref=0
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Nathan Cutler [Mon, 19 Aug 2019 14:08:36 +0000 (16:08 +0200)]
Merge pull request #29724 from dillaman/wip-40888-nautilus
nautilus: doc/rbd: initial live-migration documentation
Reviewed-by: Mykola Golub <mgolub@suse.com>
Sage Weil [Fri, 5 Apr 2019 22:37:57 +0000 (17:37 -0500)]
rgw: pass mostly_omap flag when opening/creating pools
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
af30e3a512dd1e5c51639d1f823b99ed9b43fb50 )
Sage Weil [Thu, 4 Apr 2019 11:41:22 +0000 (06:41 -0500)]
rgw/rgw_rados: pass mostly_omap flag when opening/creating pools
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
c4b830069372302ee93687d417d04ad39f600c4e )
Abhishek Lekshmanan [Mon, 19 Aug 2019 10:50:37 +0000 (12:50 +0200)]
rgw: move rgw_init_ioctx() to rgw_tools.cc
move rgw_init_ioctx() out of rgw_rados.cc so that it doesn't need to be
duplicated by services/svc_rados.cc
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
1cf90269057ed18636879edef28b6e038352e335 )
Conflicts:
src/rgw/rgw_gc.cc
include conflicts with scope_guard, added rgw_tools include as well which
introduces rgw_init_ioctx
Adam C. Emerson [Wed, 30 Jan 2019 19:26:33 +0000 (14:26 -0500)]
rgw: Get rid of num_rados_handles infrastructure in RGWSI_RADOS
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit
5666b4c41a70bac75e26b44a2cec1bd895d8375d )