]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agocmake: use upstream repo for fio 42934/head
Kefu Chai [Thu, 26 Aug 2021 15:57:00 +0000 (23:57 +0800)]
cmake: use upstream repo for fio

this change partially reverts 10baab3fc8293b8c30ca90a4acd76f70d011f1b5,
but since the fix for C++ build is not included by any tag or branche so
far. let's just use the sha1 for now.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #42858 from josephsawaya/wip-mgr-rook-osd-removal
Joseph Sawaya [Thu, 26 Aug 2021 15:12:59 +0000 (11:12 -0400)]
Merge pull request #42858 from josephsawaya/wip-mgr-rook-osd-removal

mgr/rook: implementing OSD removal in the rook orchestrator

3 years agoMerge pull request #42869 from cbodley/wip-49955
Casey Bodley [Thu, 26 Aug 2021 14:32:23 +0000 (10:32 -0400)]
Merge pull request #42869 from cbodley/wip-49955

qa/rgw: move mixed-version multisite tests out of upgrade suite

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #42922 from dang/wip-dang-zipper-cleanup
Daniel Gryniewicz [Thu, 26 Aug 2021 13:45:17 +0000 (09:45 -0400)]
Merge pull request #42922 from dang/wip-dang-zipper-cleanup

RGW Zipper Cleanups

Reviewed-by: Soumya Kadouri <skoduri@redhat.com>
3 years agoMerge pull request #42893 from jan--f/snap-sched-fix-multi-retention
Venky Shankar [Thu, 26 Aug 2021 12:32:34 +0000 (18:02 +0530)]
Merge pull request #42893 from jan--f/snap-sched-fix-multi-retention

snap-schedule: count retained snapshots per retention policy

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #42895 from ifed01/wip-ifed-improve-alloc-replay
Igor Fedotov [Thu, 26 Aug 2021 11:08:40 +0000 (14:08 +0300)]
Merge pull request #42895 from ifed01/wip-ifed-improve-alloc-replay

test/allocator_replay_test: implement "try_alloc" command.

This tool has just limited manual  application when troubleshooting allocator's issues hence omitting QA run...

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #42916 from cyx1231st/wip-seastar-tree-metrics2
Samuel Just [Thu, 26 Aug 2021 01:50:41 +0000 (18:50 -0700)]
Merge pull request #42916 from cyx1231st/wip-seastar-tree-metrics2

crimson/seastore: improve metrics about onode and lba trees

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #42917 from rzarzynski/wip-crimson-watch-list
Samuel Just [Wed, 25 Aug 2021 19:23:54 +0000 (12:23 -0700)]
Merge pull request #42917 from rzarzynski/wip-crimson-watch-list

crimson/osd: implement CEPH_OSD_OP_LIST_WATCHERS.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoRGW Zipper - chunk/stripe size are RadosStore only 42922/head
Daniel Gryniewicz [Wed, 25 Aug 2021 17:02:12 +0000 (13:02 -0400)]
RGW Zipper - chunk/stripe size are RadosStore only

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #42886 from sebastian-philipp/prometheus-url-missing-slash
Yuri Weinstein [Wed, 25 Aug 2021 16:07:29 +0000 (09:07 -0700)]
Merge pull request #42886 from sebastian-philipp/prometheus-url-missing-slash

mgr/{prometheus,restful}: Fix url generation again

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agoMerge pull request #42921 from tchaikov/wip-crimson-cleanup
Kefu Chai [Wed, 25 Aug 2021 16:01:39 +0000 (00:01 +0800)]
Merge pull request #42921 from tchaikov/wip-crimson-cleanup

crimson/os: use structured binding in loop

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/os: use structured binding in loop 42921/head
Kefu Chai [Wed, 25 Aug 2021 14:25:54 +0000 (22:25 +0800)]
crimson/os: use structured binding in loop

also avoid using `map[key] = val` for setting an item in map, as, if
the key does not exist in map, `map[key]` would have to create a value
using its default ctor, and then call the `operator=(bufferlist&&)` to
set it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agomgr/rook: add better error handling to remove 42858/head
Joseph Sawaya [Wed, 25 Aug 2021 13:54:17 +0000 (09:54 -0400)]
mgr/rook: add better error handling to remove

This commit adds better error handling to the remove method
in the DefaultRemover by hiding stack traces from the user
and displaying meaningful error messages.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
3 years agomgr/rook: use label selector to fetch pods and jobs in remover
Joseph Sawaya [Tue, 24 Aug 2021 16:01:39 +0000 (12:01 -0400)]
mgr/rook: use label selector to fetch pods and jobs in remover

This commit adds a label_selector to the fetching of the osd pods and osd prepare
jobs in the DefaultRemover.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
3 years agoMerge pull request #42915 from AmnonHanuhov/wip-fix_omap_set_values
Amnon Hanuhov [Wed, 25 Aug 2021 11:34:18 +0000 (14:34 +0300)]
Merge pull request #42915 from AmnonHanuhov/wip-fix_omap_set_values

crimson/os: Use operator[] in CyanStore when inserting values to omap

3 years agocrimson/osd: implement CEPH_OSD_OP_LIST_WATCHERS. 42917/head
Radoslaw Zarzynski [Wed, 25 Aug 2021 10:36:00 +0000 (10:36 +0000)]
crimson/osd: implement CEPH_OSD_OP_LIST_WATCHERS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #42712 from idryomov/wip-malignant-noexcept
Ilya Dryomov [Wed, 25 Aug 2021 09:28:46 +0000 (11:28 +0200)]
Merge pull request #42712 from idryomov/wip-malignant-noexcept

common: drop noexcept on thread entry points

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agoMerge pull request #42090 from cypherean/feedback
Alfonso Martínez [Wed, 25 Aug 2021 08:52:39 +0000 (10:52 +0200)]
Merge pull request #42090 from cypherean/feedback

mgr/dashboard: report ceph tracker bug/feature through CLI/API

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #41918 from SudhanAruna/feature_parse
Juan Miguel Olmo [Wed, 25 Aug 2021 08:14:46 +0000 (10:14 +0200)]
Merge pull request #41918 from SudhanAruna/feature_parse

behave_tests: Implemented basic behave test scenario's for cephadm, ceph shell and OSD commands

3 years agocrimson/seastore: improve metrics about onode and lba trees 42916/head
Yingxin Cheng [Mon, 23 Aug 2021 07:31:46 +0000 (15:31 +0800)]
crimson/seastore: improve metrics about onode and lba trees

Absolute value:
* tree_depth
By transaction source:
* tree_inserts_committed
* tree_inserts_invalidated
* tree_erases_committed
* tree_erase_invalidated

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os: Use operator[] in CyanStore when inserting values to object omap 42915/head
Amnon Hanuhov [Wed, 25 Aug 2021 06:22:30 +0000 (09:22 +0300)]
crimson/os: Use operator[] in CyanStore when inserting values to object omap

std::map::insert() doesn't update the value associated with an existing
key, therefore we should use operator[] instead

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
3 years agoMerge pull request #42885 from cyx1231st/wip-seastar-tree-metrics
Samuel Just [Wed, 25 Aug 2021 02:40:29 +0000 (19:40 -0700)]
Merge pull request #42885 from cyx1231st/wip-seastar-tree-metrics

crimson/seastore: add metrics about onode and lba trees

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #42907 from rzarzynski/wip-crimson-smphack
Samuel Just [Wed, 25 Aug 2021 02:38:25 +0000 (19:38 -0700)]
Merge pull request #42907 from rzarzynski/wip-crimson-smphack

 crimson/osd: implicitly append '--smp 1' when invoked without it

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agocrimson/osd: implicitly append '--smp 1' when invoked without it 42907/head
Radoslaw Zarzynski [Tue, 24 Aug 2021 14:02:36 +0000 (14:02 +0000)]
crimson/osd: implicitly append '--smp 1' when invoked without it

This commit is basically a hack supposed to fulfil the obligation
of crimson being a drop-in replacement for the classical OSD
we already made by packaging it under `/usr/bin/ceph-osd`.
The discussion whether the interface-exactness should be continued
or not is out of scope of the commit; it's supposed just to handle
the issue unveiled by the Rook integration effort: `crimson-osd`
is unable to `--mkfs` because Seastar, if not restricted by passing
`--smp N`, considers all CPU cores available in the system when
allocating resources. This leads to the following error:

```
ERROR 2021-08-24 14:17:32,105 [shard 5] seastar - Could not setup Async I/O: Resource temporarily unavailable. The most common cause is not enough request capacity in /proc/sys/fs/aio-max-nr. Try increasing that number or reducing the amount of logical CPUs available for your application
```

This hack will need to be dropped when integrating multi-reactor
support in crimson.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #42910 from zdover23/doc-2021-08-25-doc-dev-config-verb-agreement...
Neha Ojha [Tue, 24 Aug 2021 23:27:32 +0000 (16:27 -0700)]
Merge pull request #42910 from zdover23/doc-2021-08-25-doc-dev-config-verb-agreement-typo

doc/dev/config: fix verb disagreement

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #42713 from ronen-fr/wip-ronenf-explicit-config
Yuri Weinstein [Tue, 24 Aug 2021 23:26:43 +0000 (16:26 -0700)]
Merge pull request #42713 from ronen-fr/wip-ronenf-explicit-config

common: removing the explicit attribute from a public copy constructor

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agodoc/dev/config: fix verb disagreement 42910/head
Zac Dover [Tue, 24 Aug 2021 19:27:16 +0000 (05:27 +1000)]
doc/dev/config: fix verb disagreement

This PR changes the following sentence:

   The other is to register an observer
   that will be called every time the
   relevant configuration values changes.

This sentence has been rewritten so that the
verb agrees with its subject, like this:

   The other is to register an observer
   that will be called every time the
   relevant configuration values change.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agomgr/dashboard: report ceph tracker bug/feature through CLI/API 42090/head
cypherean [Tue, 24 Aug 2021 09:55:42 +0000 (15:25 +0530)]
mgr/dashboard: report ceph tracker bug/feature through CLI/API

Fixes: https://tracker.ceph.com/issues/44851
Signed-off-by: Shreya Sharma <shreyasharma.ss305@gmail.com>
3 years agoMerge pull request #42806 from zdover23/wip-doc-2021-08-17-rbd-nomad-3-of-x
Ilya Dryomov [Tue, 24 Aug 2021 15:41:57 +0000 (17:41 +0200)]
Merge pull request #42806 from zdover23/wip-doc-2021-08-17-rbd-nomad-3-of-x

doc: rewrite configure ceph-csi to "conf. nomad"

Reviewed-by: Etienne Menguy <etienne.menguy@croit.io>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #42845 from tchaikov/wip-51483
Kefu Chai [Tue, 24 Aug 2021 15:32:39 +0000 (23:32 +0800)]
Merge pull request #42845 from tchaikov/wip-51483

common/options: validate see-also

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agodoc: rewrite configure ceph-csi to "conf. nomad" 42806/head
Zac Dover [Tue, 17 Aug 2021 05:07:26 +0000 (15:07 +1000)]
doc: rewrite configure ceph-csi to "conf. nomad"

This PR rewrites the sections
     - Configure Ceph-CSI
     - Configure Nomad

in the rbd-nomad.rst Chapter of the RBD
Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agoMerge pull request #42857 from dsavineau/cephadm_tcmalloc_ceph_daemons_only
Sebastian Wagner [Tue, 24 Aug 2021 14:55:02 +0000 (16:55 +0200)]
Merge pull request #42857 from dsavineau/cephadm_tcmalloc_ceph_daemons_only

cephadm: Set tcmalloc env var for Ceph daemons

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42833 from oleeander/cephadm_openntpd
Sebastian Wagner [Tue, 24 Aug 2021 14:54:50 +0000 (16:54 +0200)]
Merge pull request #42833 from oleeander/cephadm_openntpd

cephadm: check for openntpd.service as time sync service

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42824 from pcuzner/cephadm-add-ports
Sebastian Wagner [Tue, 24 Aug 2021 14:54:34 +0000 (16:54 +0200)]
Merge pull request #42824 from pcuzner/cephadm-add-ports

cephadm:Add listening ports to gather-facts output

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
3 years agoMerge pull request #42699 from sebastian-philipp/cephadm-duplicate-ips
Sebastian Wagner [Tue, 24 Aug 2021 14:54:14 +0000 (16:54 +0200)]
Merge pull request #42699 from sebastian-philipp/cephadm-duplicate-ips

cephadm: list-networks: Avoid duplicated IPs

Reviewed-by: Francesco Pantano <fpantano@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sage@newdream.net>
3 years agoMerge pull request #42405 from Daniel-Pivonka/cephadm_rm_offline_host
Sebastian Wagner [Tue, 24 Aug 2021 14:52:28 +0000 (16:52 +0200)]
Merge pull request #42405 from Daniel-Pivonka/cephadm_rm_offline_host

mgr/cephadm: add ability to remove offline host

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42805 from zdover23/wip-doc-2021-08-17-rbd-nomad-2-of-x
Ilya Dryomov [Tue, 24 Aug 2021 14:32:42 +0000 (16:32 +0200)]
Merge pull request #42805 from zdover23/wip-doc-2021-08-17-rbd-nomad-2-of-x

doc: improve "Create a Pool" in rbd-nomad.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agocmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/ 42845/head
Kefu Chai [Thu, 19 Aug 2021 10:01:03 +0000 (18:01 +0800)]
cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/

as FindPython3.cmake only sets Python3_EXECUTABLE for us, we should
stick with Python3_EXECUTABLE instead of Python_EXECUTABLE.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agocommon/options: validate see-also
Kefu Chai [Thu, 19 Aug 2021 09:24:32 +0000 (17:24 +0800)]
common/options: validate see-also

y2c.py is like a compiler which translates .yaml to .cc and .h files,
it does not have access to all .yaml files. to validate the dangling
see-also issue, we need to do this with a "linker".

in this change, validate-options.py is introduced to check if any of
option name included by the see-also property is valid.

Fixes: https://tracker.ceph.com/issues/51483
Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agodoc: improve "Create a Pool" in rbd-nomad.rst 42805/head
Zac Dover [Tue, 17 Aug 2021 04:44:52 +0000 (14:44 +1000)]
doc: improve "Create a Pool" in rbd-nomad.rst

This PR improves the English in the "Create
a Pool" section of the "RBD & Nomad Integration"
chapter of the RBD Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agoMerge pull request #42826 from tchaikov/wip-test-fair-mutex
Kefu Chai [Tue, 24 Aug 2021 14:04:28 +0000 (22:04 +0800)]
Merge pull request #42826 from tchaikov/wip-test-fair-mutex

test/common/test_fair_mutex: increase NR_ROUNDS from 256 to 512

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #42881 from wjwithagen/wjw-fix-vstart-mgr
Kefu Chai [Tue, 24 Aug 2021 13:51:34 +0000 (21:51 +0800)]
Merge pull request #42881 from wjwithagen/wjw-fix-vstart-mgr

src/vstart.sh: put modules option for MGR in the [mgr] block.

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #42882 from cyx1231st/wip-seastar-cleanup
Kefu Chai [Tue, 24 Aug 2021 13:49:42 +0000 (21:49 +0800)]
Merge pull request #42882 from cyx1231st/wip-seastar-cleanup

crimson/common: minor cleanups to futures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #42802 from Rethan/fix-rbd-bench-unit
Ilya Dryomov [Tue, 24 Aug 2021 11:19:03 +0000 (13:19 +0200)]
Merge pull request #42802 from Rethan/fix-rbd-bench-unit

rbd: avoid overflow of ios and clarify io-size limit for bench

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agosnap-schedule: add test for multiple retention set calculation 42893/head
Jan Fajerski [Mon, 23 Aug 2021 13:37:57 +0000 (15:37 +0200)]
snap-schedule: add test for multiple retention set calculation

Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
3 years agosnap-schedule: count retained snapshots per retention policy
Jan Fajerski [Mon, 23 Aug 2021 13:15:21 +0000 (15:15 +0200)]
snap-schedule: count retained snapshots per retention policy

Before this commit the length of the already-retained snapshots
terminated the loop. This fails when muliple retention policies are set.

Fixes: https://tracker.ceph.com/issues/52388
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
3 years agobehave_test: Implemented basic bahave test scenario's 41918/head
Arunagirinadan Sudharshan [Sun, 13 Jun 2021 17:05:40 +0000 (22:35 +0530)]
behave_test: Implemented basic bahave test scenario's

Fixes: https://tracker.ceph.com/issues/52371
This commit includes the basic implementation of behave test scenario's
(for cephadm, ceph shell and OSD commands) and python implementations for
interacting with kcli and behave test cases. The test scenarios can be executed
using behave command. The files are created under src/test/behave_tests directory.

Signed-off-by: Arunagirinadan Sudharshan <sudhan.aruna.97@gmail.com>
3 years agoMerge pull request #42841 from xxhdx1985126/wip-52305
Samuel Just [Tue, 24 Aug 2021 03:12:03 +0000 (20:12 -0700)]
Merge pull request #42841 from xxhdx1985126/wip-52305

crimson/common: keep ref count of crimson::interruptible::interrupt_cond

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agorbd: avoid overflow of ios and clarify io-size limit for bench 42802/head
haoyixing [Mon, 16 Aug 2021 10:55:24 +0000 (18:55 +0800)]
rbd: avoid overflow of ios and clarify io-size limit for bench

When doing rbd bench, we record done ios to print progress, current it's unsigned.
Suppose we do a bench of io-size 512B and io-total 4T, that means a total number of
8G ios which causes an overflow.
And we don't support io-size greater than 4G, so change help message.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>
3 years agocrimson/interruptible_future: remove unused code 42882/head
Yingxin Cheng [Mon, 23 Aug 2021 01:50:33 +0000 (09:50 +0800)]
crimson/interruptible_future: remove unused code

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/common: remove unused futurize::apply()
Yingxin Cheng [Mon, 23 Aug 2021 01:49:58 +0000 (09:49 +0800)]
crimson/common: remove unused futurize::apply()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge PR #42371 into master
Patrick Donnelly [Tue, 24 Aug 2021 00:02:31 +0000 (20:02 -0400)]
Merge PR #42371 into master

* refs/pull/42371/head:
mgr/volumes: Fix a race during clone cancel
mgr/volumes: Fail subvolume removal if it's in progress

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #42884 from ideepika/wip-remove-unused-submodules
Neha Ojha [Mon, 23 Aug 2021 23:48:17 +0000 (16:48 -0700)]
Merge pull request #42884 from ideepika/wip-remove-unused-submodules

.gitmodules: remove thrift submodule

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #41335 from noonedeadpunk/doc/stretch_mon
Gregory Farnum [Mon, 23 Aug 2021 23:13:54 +0000 (16:13 -0700)]
Merge pull request #41335 from noonedeadpunk/doc/stretch_mon

doc/rados/operations/stretch-mode: Adjust description of mon.e placement

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
3 years agoMerge pull request #42053 from TRYTOBE8TME/wip-rgw-dpp-4
Ali Maredia [Mon, 23 Aug 2021 19:34:41 +0000 (15:34 -0400)]
Merge pull request #42053 from TRYTOBE8TME/wip-rgw-dpp-4

rgw: DPP addition 5

Reviewed-by: Ali Maredia <amaredia@redhat.com>
3 years agoMerge pull request #42439 from ifed01/wip-ifed-more-errors-shared-blob-repair
Igor Fedotov [Mon, 23 Aug 2021 15:03:41 +0000 (18:03 +0300)]
Merge pull request #42439 from ifed01/wip-ifed-more-errors-shared-blob-repair

os/bluestore: fix additional errors during missed shared blob repair.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agotest/allocator_replay_test: implement "try_alloc" command. 42895/head
Igor Fedotov [Mon, 23 Aug 2021 14:46:10 +0000 (17:46 +0300)]
test/allocator_replay_test: implement "try_alloc" command.

This permits trying multiple allocations of chosen size on allocator
being at specific state (which is determined by free-dump report replay)

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
3 years agoMerge pull request #41569 from MrFreezeex/rbd-mirror-snap-perf-dump
Yuri Weinstein [Mon, 23 Aug 2021 14:29:40 +0000 (07:29 -0700)]
Merge pull request #41569 from MrFreezeex/rbd-mirror-snap-perf-dump

rbd-mirror: add perf counters to snapshot replayer

3 years agoMerge pull request #42871 from tchaikov/wip-cmake-exclude-grafonnet-lib
Kefu Chai [Mon, 23 Aug 2021 14:24:53 +0000 (22:24 +0800)]
Merge pull request #42871 from tchaikov/wip-cmake-exclude-grafonnet-lib

cmake: exclude "grafonnet-lib" target from "all"

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
3 years agoMerge pull request #42434 from zenomri/wip-omri-rgw-tracing
Yuval Lifshitz [Mon, 23 Aug 2021 14:01:54 +0000 (17:01 +0300)]
Merge pull request #42434 from zenomri/wip-omri-rgw-tracing

[WIP] -  RGW tracing implementation

3 years agoMerge pull request #41380 from Devp00l/wip-rgw-multisite-doc
Stephan Müller [Mon, 23 Aug 2021 13:52:57 +0000 (15:52 +0200)]
Merge pull request #41380 from Devp00l/wip-rgw-multisite-doc

doc: Improve RGW multisite documentation

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Alex Settle <asettle@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42864 from aaryanporwal/vrt-doc
Alfonso Martínez [Mon, 23 Aug 2021 12:59:00 +0000 (14:59 +0200)]
Merge pull request #42864 from aaryanporwal/vrt-doc

doc/dev: mgr/dashboard: Visual regression tests for ceph dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agosrc/vstart.sh: put modules option for MGR in the [mgr] block. 42881/head
Willem Jan Withagen [Sun, 22 Aug 2021 20:08:46 +0000 (22:08 +0200)]
src/vstart.sh: put modules option for MGR in the [mgr] block.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
3 years agoMerge pull request #42739 from Matan-B/lua-install-packages-fix
Yuval Lifshitz [Mon, 23 Aug 2021 12:46:54 +0000 (15:46 +0300)]
Merge pull request #42739 from Matan-B/lua-install-packages-fix

rgw: fix install packages function

3 years agoMerge pull request #42865 from rhcs-dashboard/fix-rgw-bucket-list-api
Alfonso Martínez [Mon, 23 Aug 2021 12:43:56 +0000 (14:43 +0200)]
Merge pull request #42865 from rhcs-dashboard/fix-rgw-bucket-list-api

mgr/dashboard: stats=false not working when listing buckets

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #42051 from melissa-kun-li/asyncssh
Sebastian Wagner [Mon, 23 Aug 2021 12:29:01 +0000 (14:29 +0200)]
Merge pull request #42051 from melissa-kun-li/asyncssh

mgr/cephadm: replace execnet and remoto with asyncssh

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42804 from zdover23/wip-doc-2021-08-17-rbd-nomad-1-of-x
Ilya Dryomov [Mon, 23 Aug 2021 11:46:31 +0000 (13:46 +0200)]
Merge pull request #42804 from zdover23/wip-doc-2021-08-17-rbd-nomad-1-of-x

doc: improve RBD "Block Devices and Nomad"

Reviewed-by: Etienne Menguy <etienne.menguy@croit.io>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #42879 from tchaikov/wip-seastar
Kefu Chai [Mon, 23 Aug 2021 11:10:37 +0000 (19:10 +0800)]
Merge pull request #42879 from tchaikov/wip-seastar

seastar: pick up change silencing gcc warning

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agomgr/dashboard: stats=false not working when listing buckets 42865/head
Avan Thakkar [Fri, 20 Aug 2021 09:07:17 +0000 (14:37 +0530)]
mgr/dashboard: stats=false not working when listing buckets

Fixes: https://tracker.ceph.com/issues/51154
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
3 years agocrimson/common: all interruptible future invocations should have already had interrup... 42841/head
Xuehan Xu [Mon, 23 Aug 2021 05:59:13 +0000 (13:59 +0800)]
crimson/common: all interruptible future invocations should have already had interrupt condition in place

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/common: correct comment error about interruptible future
Xuehan Xu [Mon, 23 Aug 2021 05:58:32 +0000 (13:58 +0800)]
crimson/common: correct comment error about interruptible future

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agotest/crimson: add interruptible async unit test
Xuehan Xu [Thu, 19 Aug 2021 08:29:38 +0000 (16:29 +0800)]
test/crimson: add interruptible async unit test

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/common: keep ref count of crimson::interruptible::interrupt_cond
Xuehan Xu [Thu, 19 Aug 2021 05:33:36 +0000 (13:33 +0800)]
crimson/common: keep ref count of crimson::interruptible::interrupt_cond

Currently, interrupt conditionss are transfered between inner and outer continuation
chains via a tls interrupt_cond variable. This simple strategy leads to problem when it
comes to mixing normal future/continuation procedures and seastar::thread. When seastar::async()
is called, the reactor can directly invoke the passed functor and lead to two different
scenarios:

1.if a seastar::get/yield() inside the passed lambda, the interrupt_cond should be erased at the
end of the continuation execution when it is yielded back;
2.otherwise, the interrupt_cond should be not erased.

There can be so many possible sequences of yielding of several different fibers that we can hardly
judge at the end of the continuation execution whether there was a yielding during the current
execution, which means we can't be able to know whether the tls interrupt_cond should be erased.
There could be other scenarios where the current strategy fails. To end this kind of issues
once and for all, we involve the ref counting mechinary.

Fixes: https://tracker.ceph.com/issues/52305
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocephadm: Add test_command_list_networks 42699/head
Sebastian Wagner [Mon, 23 Aug 2021 10:07:30 +0000 (12:07 +0200)]
cephadm: Add test_command_list_networks

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: test_networks: Move tests from test_cephadm
Sebastian Wagner [Mon, 23 Aug 2021 09:27:48 +0000 (11:27 +0200)]
cephadm: test_networks: Move tests from test_cephadm

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: list-networks: Avoid duplicated IPs
Sebastian Wagner [Fri, 6 Aug 2021 09:41:39 +0000 (11:41 +0200)]
cephadm: list-networks: Avoid duplicated IPs

Fixes: https://tracker.ceph.com/issues/52083
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: Fix TestCephVolume.test_noop
Sebastian Wagner [Mon, 23 Aug 2021 09:19:16 +0000 (11:19 +0200)]
cephadm: Fix TestCephVolume.test_noop

Fixes `PermissionError: [Errno 13] Permission denied: '/var/lib/ceph/760b...5a5fbb'`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agomgr/{prometheus,restful}: Fix url generation again 42886/head
Sebastian Wagner [Mon, 23 Aug 2021 08:33:37 +0000 (10:33 +0200)]
mgr/{prometheus,restful}: Fix url generation again

Fixes

```
======================================================================
ERROR: test_standby (tasks.mgr.test_prometheus.TestPrometheus)
----------------------------------------------------------------------
urllib3.exceptions.LocationParseError: Failed to parse: http://172.21.15.71:7789metrics

```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agodoc/dev: mgr/dashboard: Visual regression tests for ceph dashboard 42864/head
Aaryan Porwal [Sat, 21 Aug 2021 15:26:14 +0000 (20:56 +0530)]
doc/dev: mgr/dashboard: Visual regression tests for ceph dashboard

Fixes: https://tracker.ceph.com/issues/52370
Signed-off-by: Aaryan Porwal <aaryanporwal2233@gmail.com>
3 years agoMerge pull request #41405 from ideepika/wip-rbd-update-feature-test
Mykola Golub [Mon, 23 Aug 2021 08:33:41 +0000 (11:33 +0300)]
Merge pull request #41405 from ideepika/wip-rbd-update-feature-test

test/librbd: add unit tests for rbd update features

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agocrimson/onode-staged-tree: suppress warnings about unused variable 42885/head
Yingxin Cheng [Mon, 23 Aug 2021 08:19:27 +0000 (16:19 +0800)]
crimson/onode-staged-tree: suppress warnings about unused variable

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/seastore: add metrics about onode and lba trees
Yingxin Cheng [Mon, 23 Aug 2021 07:31:46 +0000 (15:31 +0800)]
crimson/seastore: add metrics about onode and lba trees

* tree_depth
* tree_inserts_committed
* tree_inserts_invalidated
* tree_erases_committed
* tree_erase_invalidated

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/common: trace TLS crimson::interruptible::interrupt_cond<InterruptCond> modif...
Xuehan Xu [Thu, 19 Aug 2021 03:50:44 +0000 (11:50 +0800)]
crimson/common: trace TLS crimson::interruptible::interrupt_cond<InterruptCond> modifications

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/errorator: remove unecessary template param
Yingxin Cheng [Mon, 23 Aug 2021 01:46:31 +0000 (09:46 +0800)]
crimson/errorator: remove unecessary template param

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #42532 from ivancich/wip-multisite-conflict-detection
J. Eric Ivancich [Sun, 22 Aug 2021 22:34:16 +0000 (18:34 -0400)]
Merge pull request #42532 from ivancich/wip-multisite-conflict-detection

rgw-multisite: metadata conflict not computed correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agorgw-multisite: metadata conflict not computed correctly 42532/head
J. Eric Ivancich [Wed, 28 Jul 2021 18:07:09 +0000 (14:07 -0400)]
rgw-multisite: metadata conflict not computed correctly

The former logic with a conditional based on `++i == 0` would never
execute. So this uses a boolean to differentiate the first from other
iterations and tries to clarify the code through commenting and an
explicit declaration. Additionally a warning is eliminated by
initializing a variable.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agoMerge pull request #42792 from tchaikov/wip-librbd-cleanups
Ilya Dryomov [Sun, 22 Aug 2021 19:01:15 +0000 (21:01 +0200)]
Merge pull request #42792 from tchaikov/wip-librbd-cleanups

librbd: trade a map<> for a plain array

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoseastar: pick up change silencing gcc warning 42879/head
Kefu Chai [Sun, 22 Aug 2021 14:15:56 +0000 (22:15 +0800)]
seastar: pick up change silencing gcc warning

silences compiler warning from src/net/tls.cc

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #42855 from josephsawaya/rook-label
Sebastian Wagner [Sat, 21 Aug 2021 08:55:45 +0000 (10:55 +0200)]
Merge pull request #42855 from josephsawaya/rook-label

.github/labeler: add rook label to PRs related to Rook

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years ago.gitmodules: remove thrift submodule 42884/head
Deepika Upadhyay [Sat, 21 Aug 2021 06:40:19 +0000 (12:10 +0530)]
.gitmodules: remove thrift submodule

with 80e82686ebafe36fca6dfd21cb32e63ced94d5cd we now use thrift as a
distro based dependency, hence we no longer need it as a submodule.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agomgr/cephadm: replace instances of remoto/execnet with asyncssh 42051/head
Melissa [Wed, 21 Jul 2021 03:46:41 +0000 (23:46 -0400)]
mgr/cephadm: replace instances of remoto/execnet with asyncssh

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: rewrite test_etc_ceph to apply to asyncssh
Melissa [Wed, 21 Jul 2021 03:36:20 +0000 (23:36 -0400)]
mgr/cephadm: rewrite test_etc_ceph to apply to asyncssh

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: remove test_stale_connections
Melissa [Wed, 21 Jul 2021 03:33:37 +0000 (23:33 -0400)]
mgr/cephadm: remove test_stale_connections

remove test_stale_connections because it applies to remoto

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: add test_offline, edit with_cephadm_module
Melissa [Wed, 21 Jul 2021 03:28:04 +0000 (23:28 -0400)]
mgr/cephadm: add test_offline, edit with_cephadm_module

add `event_loop` and `tkey` object to with_cephadm_module, and create MockEventLoopThread in fixtures.py to test async functions of ssh.py.
rewrite test_offline to be compatible with asyncssh

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: simplify _deploy_cephadm_binary, remove _deploy_cephadm_binary_conn
Melissa [Wed, 21 Jul 2021 03:19:33 +0000 (23:19 -0400)]
mgr/cephadm: simplify _deploy_cephadm_binary, remove _deploy_cephadm_binary_conn

_deploy_cephadm_binary can be simplified to call write_remote_file in ssh.py, and _deploy_cephadm_binary_conn is no longer needed

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: use _remote_connection (ssh.py), _execute_command, _check_execute_comman...
Melissa [Wed, 21 Jul 2021 03:12:21 +0000 (23:12 -0400)]
mgr/cephadm: use _remote_connection (ssh.py), _execute_command, _check_execute_command in _run_cephadm

remove _get_connection from module.py and _remote_connection in serve.py, replacing with _remote_connection in ssh.py.
also, replace remoto.process.check with _execute_command and _check_execute_command in ssh.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: remove remotes.py, replace old _write_remote_file in serve.py with write...
Melissa [Wed, 21 Jul 2021 02:43:38 +0000 (22:43 -0400)]
mgr/cephadm: remove remotes.py, replace old _write_remote_file in serve.py with write_remote_file in ssh.py

remove remotes.py because it is specific to execnet/remoto.
_write_remote_file in ssh.py now fulfills the function of write_file in remotes.py and the old _write_remote_file in serve.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: remove _executable_path from module.py
Melissa [Wed, 21 Jul 2021 01:50:33 +0000 (21:50 -0400)]
mgr/cephadm: remove _executable_path from module.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: make synchronous wrapper functions of the async functions in ssh.py
Melissa [Tue, 20 Jul 2021 23:11:29 +0000 (19:11 -0400)]
mgr/cephadm: make synchronous wrapper functions of the async functions in ssh.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
3 years agomgr/cephadm: create thread to start event loop for ssh.py, and return results of...
Melissa [Tue, 20 Jul 2021 23:02:57 +0000 (19:02 -0400)]
mgr/cephadm: create thread to start event loop for ssh.py, and return results of the async functions with get_result

The EventLoopThread class starts a thread and an event loop which runs forever. Coroutines are scheduled on the event loop by the `get_result` method which uses `run_coroutine_threadsafe` to return a concurrent.futures.Future, and ultimately the result with .result()

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>