ceph.git
3 years agoceph-volume: remove ceph_volume.sys_info as it's no longer used wip-lru-cache 37398/head
Andrew Schoen [Thu, 24 Sep 2020 21:07:11 +0000 (16:07 -0500)]
ceph-volume: remove ceph_volume.sys_info as it's no longer used

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
3 years agoceph-volume: only require backports.functools_lru_cache for < py3.2
Andrew Schoen [Thu, 24 Sep 2020 16:53:18 +0000 (11:53 -0500)]
ceph-volume: only require backports.functools_lru_cache for < py3.2

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
3 years agoceph-volume: adds backports.functools_lru_cache to install_requires
Andrew Schoen [Wed, 23 Sep 2020 17:12:00 +0000 (12:12 -0500)]
ceph-volume: adds backports.functools_lru_cache to install_requires

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
3 years agoceph-volume: remove usage of sys_info.devices from util.device
Andrew Schoen [Wed, 23 Sep 2020 17:01:57 +0000 (12:01 -0500)]
ceph-volume: remove usage of sys_info.devices from util.device

This is no longer needed now that disk.get_devices is cached.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
3 years agoceph-volume: cache util.disk.get_devices
Andrew Schoen [Wed, 23 Sep 2020 16:56:37 +0000 (11:56 -0500)]
ceph-volume: cache util.disk.get_devices

This uses backports.functools-lru-cache to provide
lru_cache to python2.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
3 years agoMerge pull request #34740 from jan--f/c-v-refactor-batch-use-create
Jan Fajerski [Wed, 30 Sep 2020 13:37:52 +0000 (15:37 +0200)]
Merge pull request #34740 from jan--f/c-v-refactor-batch-use-create

ceph-volume: major batch refactor

3 years agoMerge pull request #37014 from lixiaoy1/rwl_plugin
Jason Dillaman [Wed, 30 Sep 2020 13:33:03 +0000 (09:33 -0400)]
Merge pull request #37014 from lixiaoy1/rwl_plugin

rbd/cache: update RWL as ImageDispatch

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
3 years agoMerge pull request #37030 from Amrita42/wip-doc-Amy-radosgw
Yuval Lifshitz [Wed, 30 Sep 2020 10:51:52 +0000 (13:51 +0300)]
Merge pull request #37030 from Amrita42/wip-doc-Amy-radosgw

doc: Bucket notifications in the radosgateway-the usage of 'if not'

3 years agoMerge pull request #37312 from votdev/issue_47578_clipboard_firefox
Lenz Grimmer [Wed, 30 Sep 2020 09:49:41 +0000 (11:49 +0200)]
Merge pull request #37312 from votdev/issue_47578_clipboard_firefox

mgr/dashboard: Copy to clipboard does not work in Firefox

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
3 years agoMerge pull request #37411 from p-se/fix-cephadm-custom-template-escaping
Joshua Schmid [Wed, 30 Sep 2020 07:57:58 +0000 (09:57 +0200)]
Merge pull request #37411 from p-se/fix-cephadm-custom-template-escaping

mgr/cephadm: do not escape HTML for custom templates

3 years agoMerge pull request #36622 from dmick/wip-46665
Joshua Schmid [Wed, 30 Sep 2020 07:57:27 +0000 (09:57 +0200)]
Merge pull request #36622 from dmick/wip-46665

mgr/cephadm: continue trying all hosts for service deployment

3 years agorbd/cache: update RWL as ImageDispatch 37014/head
lixiaoy1 [Mon, 21 Sep 2020 15:05:43 +0000 (11:05 -0400)]
rbd/cache: update RWL as ImageDispatch

This is a preparation work to make ReplicatedWriteLog as a plugin.

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
3 years agorbd/io: split IO check
lixiaoy1 [Mon, 21 Sep 2020 15:03:09 +0000 (11:03 -0400)]
rbd/io: split IO check

Move IO check to interfaces in Utils.

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
3 years agoMerge pull request #37112 from sebastian-philipp/cephadm-services-types-mandatory
Joshua Schmid [Wed, 30 Sep 2020 07:56:20 +0000 (09:56 +0200)]
Merge pull request #37112 from sebastian-philipp/cephadm-services-types-mandatory

mgr/cephadm: Make type annotations for cephadm.services mandatory

3 years agoMerge pull request #37397 from adk3798/mirror
Joshua Schmid [Wed, 30 Sep 2020 07:56:12 +0000 (09:56 +0200)]
Merge pull request #37397 from adk3798/mirror

mgr/cephadm: get rbd-mirror daemon-id when checking for strays

3 years agoMerge pull request #37432 from sebastian-philipp/cephadm-upgrade-state-none
Joshua Schmid [Wed, 30 Sep 2020 07:55:57 +0000 (09:55 +0200)]
Merge pull request #37432 from sebastian-philipp/cephadm-upgrade-state-none

mgr/cephadm: upgrade_state can be literally "null"

3 years agoceph-volume batch: fix very_fast_allocation plan and add tests 34740/head
Jan Fajerski [Fri, 25 Sep 2020 09:35:19 +0000 (11:35 +0200)]
ceph-volume batch: fix very_fast_allocation plan and add tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: batch: call the right prepare method
Jan Fajerski [Wed, 16 Sep 2020 13:43:00 +0000 (15:43 +0200)]
ceph-volume: batch: call the right prepare method

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agopython-common drive-groups: allow strings for wal and db size
Jan Fajerski [Mon, 14 Sep 2020 09:32:38 +0000 (11:32 +0200)]
python-common drive-groups: allow strings for wal and db size

This enables user to pass sizes like "10G", which batch now understands.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume inventory: add option to filter unwanted devices
Jan Fajerski [Fri, 11 Sep 2020 14:35:00 +0000 (16:35 +0200)]
ceph-volume inventory: add option to filter unwanted devices

Some device we never want to pass to the batch subcommand. For now this
includes devices that have a partition or are mounted on the machine.
One goal is to filter the root device, so it is not included on a batch
command and thus would contribute to its implicit sizing calculation.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: address review comments
Jan Fajerski [Fri, 11 Sep 2020 08:36:43 +0000 (10:36 +0200)]
ceph-volume: address review comments

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoMerge pull request #36392 from Devp00l/wip-46375
Lenz Grimmer [Tue, 29 Sep 2020 11:08:17 +0000 (13:08 +0200)]
Merge pull request #36392 from Devp00l/wip-46375

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
3 years agomgr/cephadm: upgrade_state can be literally "null" 37432/head
Sebastian Wagner [Mon, 28 Sep 2020 13:09:16 +0000 (15:09 +0200)]
mgr/cephadm: upgrade_state can be literally "null"

Fixes: https://tracker.ceph.com/issues/47580
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
3 years agomgr/dashboard: Speed improvements of pool details 36392/head
Stephan Müller [Thu, 10 Sep 2020 12:28:47 +0000 (14:28 +0200)]
mgr/dashboard: Speed improvements of pool details

Now all variables that are shown in a listing that trigger a
more complex render cycle will only be updated if pool properties have
changed and not only the time series data, which isn't shown anymore,
as it can be seen graphically enhanced in the pool listing.

The pool detail component now uses the `onPush` change detection
strategy, to only call `ngOnChanges` if one of the input variables
have changed.

The function that only updates variables if they have changed is
now available through a helper class in order to provide this
useful functionality through out the dashboard.

Fixes: https://tracker.ceph.com/issues/46375
Signed-off-by: Stephan Müller <smueller@suse.com>
3 years agoMerge pull request #37286 from dillaman/wip-46822
Mykola Golub [Tue, 29 Sep 2020 06:18:55 +0000 (09:18 +0300)]
Merge pull request #37286 from dillaman/wip-46822

osdc/ObjectCacher: overwrite might cause stray read request callbacks

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #37074 from dillaman/wip-librbd-list-snaps-api
Mykola Golub [Tue, 29 Sep 2020 05:45:13 +0000 (08:45 +0300)]
Merge pull request #37074 from dillaman/wip-librbd-list-snaps-api

librbd: generic internal API for computing snapshot deltas

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #37281 from tchaikov/wip-fmt
Kefu Chai [Tue, 29 Sep 2020 00:14:34 +0000 (08:14 +0800)]
Merge pull request #37281 from tchaikov/wip-fmt

rpm: add fmt-devel as build deps

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #37417 from Songweibin/wip-rbd-metadata-update
Jason Dillaman [Mon, 28 Sep 2020 23:44:23 +0000 (19:44 -0400)]
Merge pull request #37417 from Songweibin/wip-rbd-metadata-update

librbd: should not permit update image metadata via snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
3 years agoMerge pull request #37429 from trociny/wip-47665
Jason Dillaman [Mon, 28 Sep 2020 23:43:53 +0000 (19:43 -0400)]
Merge pull request #37429 from trociny/wip-47665

rbd-nbd: don't ignore namespace when unmapping by image spec

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
3 years agoMerge pull request #37399 from smithfarm/wip-relax-backporting-rule
Nathan Cutler [Mon, 28 Sep 2020 18:56:08 +0000 (20:56 +0200)]
Merge pull request #37399 from smithfarm/wip-relax-backporting-rule

doc/SubmittingPatches-backports.rst: clarify where to explain

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge PR #37359 into master
Patrick Donnelly [Mon, 28 Sep 2020 18:49:27 +0000 (11:49 -0700)]
Merge PR #37359 into master

* refs/pull/37359/head:
mon: test before adding/removing required_client_features

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge PR #36912 into master
Patrick Donnelly [Mon, 28 Sep 2020 18:47:54 +0000 (11:47 -0700)]
Merge PR #36912 into master

* refs/pull/36912/head:
mds: defer encoding and storing the inode backtrace
mds: defer encoding and storing the CDir dentries
mds: add error handler with lock support

Reviewed-by: Zheng Yan <zyan@redhat.com>
3 years agoMerge PR #36730 into master
Patrick Donnelly [Mon, 28 Sep 2020 18:45:44 +0000 (11:45 -0700)]
Merge PR #36730 into master

* refs/pull/36730/head:
client: switch to unique_lock to avoid use the client_lock directly
client: unlock client_lock when copying data to bufferlist
client: do more check for the client_lock
client: simplify the iterating related code to make it more readable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #37424 from ronen-fr/rf_neo_comp
Jason Dillaman [Mon, 28 Sep 2020 15:50:52 +0000 (11:50 -0400)]
Merge pull request #37424 from ronen-fr/rf_neo_comp

neorados: removing an unused captured variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #36016 from rhcs-dashboard/wip-45863-master
Lenz Grimmer [Mon, 28 Sep 2020 14:48:25 +0000 (16:48 +0200)]
Merge pull request #36016 from rhcs-dashboard/wip-45863-master

mgr/dashboard/api: generate static API documentation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Courtney Caldwell <ccaldwel@redhat.com>
Reviewed-by: Karen Norteman <knortema@redhat.com>
3 years agoMerge pull request #37387 from rhcs-dashboard/expose-always-on
Lenz Grimmer [Mon, 28 Sep 2020 13:35:44 +0000 (15:35 +0200)]
Merge pull request #37387 from rhcs-dashboard/expose-always-on

mgr/dashboard: cluster > manager modules

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
3 years agomgr/dashboard/api: generate offline API docs 36016/head
Courtney Caldwell [Tue, 8 Sep 2020 15:06:33 +0000 (11:06 -0400)]
mgr/dashboard/api: generate offline API docs

Generate Sphinx documentation from OpenAPI YAML spec:
- Fixed Docs controller doc generation
- Now dashboard Python doesn't fails if dashboard/frontend/dist doesn't exist
- OpenAPI added to @ceph/api CODEOWNERS
- Renamed Ceph-Dashboard API to Ceph REST or RESTful API.
- OpenAPI Docs: fixed decorators and docstrings.
- Sphinx Docs: updated dashboard and dev ones.

Co-authored-by: Ernesto Puerta <epuertat@redhat.com>
Fixes: https://tracker.ceph.com/issues/45863
Signed-off-by: Courtney Caldwell <ccaldwel@redhat.com>
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
3 years agoMerge pull request #37328 from badone/wip-npm-cache-config-changes
Kefu Chai [Mon, 28 Sep 2020 01:35:36 +0000 (09:35 +0800)]
Merge pull request #37328 from badone/wip-npm-cache-config-changes

cmake: Don't try to access users' home directory

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agocmake: Don't try to access users' home directory 37328/head
Brad Hubbard [Tue, 22 Sep 2020 21:59:58 +0000 (07:59 +1000)]
cmake: Don't try to access users' home directory

Don't store .npmrc or cache directory in a user's home directory.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
3 years agorbd-nbd: don't ignore namespace when unmapping by image spec 37429/head
Mykola Golub [Sun, 27 Sep 2020 16:59:49 +0000 (17:59 +0100)]
rbd-nbd: don't ignore namespace when unmapping by image spec

Fixes: https://tracker.ceph.com/issues/47665
Signed-off-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #37282 from neha-ojha/wip-46323
Kefu Chai [Sun, 27 Sep 2020 16:17:49 +0000 (00:17 +0800)]
Merge pull request #37282 from neha-ojha/wip-46323

qa/*/thrash_cache_writeback_proxy_none.yaml: remove cache-mode proxy

Reviewed-by: Nathan Cutler <ncutler@suse.com>
3 years agoMerge pull request #37077 from wjwithagen/wjw-fix-cls_fifo.cc-__lea64
Kefu Chai [Sun, 27 Sep 2020 16:15:59 +0000 (00:15 +0800)]
Merge pull request #37077 from wjwithagen/wjw-fix-cls_fifo.cc-__lea64

cls: replace __le{32,64} types by their Ceph counterparts

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #37149 from ifed01/wip-ifed-fix-alloc-cmd-multi-reg
Kefu Chai [Sun, 27 Sep 2020 16:13:46 +0000 (00:13 +0800)]
Merge pull request #37149 from ifed01/wip-ifed-fix-alloc-cmd-multi-reg

os/bluestore: provide a different name for fallback allocator

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
3 years agoMerge pull request #37150 from pdvian/wip-crush-failure-domain
Kefu Chai [Sun, 27 Sep 2020 16:12:44 +0000 (00:12 +0800)]
Merge pull request #37150 from pdvian/wip-crush-failure-domain

mon: validate crush-failure-domain

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #37263 from ifed01/wip-ifed-more-structs-to-mempool
Kefu Chai [Sun, 27 Sep 2020 16:11:31 +0000 (00:11 +0800)]
Merge pull request #37263 from ifed01/wip-ifed-more-structs-to-mempool

os/bluestore: add missing structs to bluestore_cache_other mempool

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
3 years agoMerge pull request #37153 from tchaikov/wip-46124
Kefu Chai [Sun, 27 Sep 2020 16:10:26 +0000 (00:10 +0800)]
Merge pull request #37153 from tchaikov/wip-46124

blk/kernel: use open file description lock if available

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Niklas Hambüchen <mail@nh2.me>
3 years agoMerge pull request #37329 from changchengx/add_conn
Kefu Chai [Sun, 27 Sep 2020 16:05:40 +0000 (00:05 +0800)]
Merge pull request #37329 from changchengx/add_conn

mon: set MonClient::_add_conn return type to void

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #37321 from changchengx/mon_make
Kefu Chai [Sun, 27 Sep 2020 02:49:22 +0000 (10:49 +0800)]
Merge pull request #37321 from changchengx/mon_make

cmake: remove undefined source file lists

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agocmake: remove undefined source file lists 37321/head
Changcheng Liu [Wed, 23 Sep 2020 01:33:33 +0000 (09:33 +0800)]
cmake: remove undefined source file lists

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
3 years agolibrbd: should not permit update image metadata via snapshot 37417/head
songweibin [Thu, 24 Sep 2020 11:48:28 +0000 (19:48 +0800)]
librbd: should not permit update image metadata via snapshot

Fixes: https://github.com/ceph/ceph/pull/37121
Signed-off-by: songweibin <song.weibin@zte.com.cn>
3 years agoneorados: removing an unused captured variable 37424/head
Ronen Friedman [Sat, 26 Sep 2020 08:41:17 +0000 (11:41 +0300)]
neorados: removing an unused captured variable

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
3 years agoMerge pull request #37388 from johscheuer/add-missing-python-dependency
Kefu Chai [Sat, 26 Sep 2020 05:25:54 +0000 (13:25 +0800)]
Merge pull request #37388 from johscheuer/add-missing-python-dependency

debian: Add missing Python dependency for ceph-mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #37380 from changchengx/ceph_port
Kefu Chai [Sat, 26 Sep 2020 05:02:35 +0000 (13:02 +0800)]
Merge pull request #37380 from changchengx/ceph_port

msgr: remove the macro not referenced anywhere

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #36779 from athanatos/sjust/wip-dirty-writeout
Kefu Chai [Sat, 26 Sep 2020 04:52:42 +0000 (12:52 +0800)]
Merge pull request #36779 from athanatos/sjust/wip-dirty-writeout

crimson/seastore: add dirty extent writeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agotest/crimson/seastore/test_transaction_manager: add random overwrite test 36779/head
Samuel Just [Thu, 20 Aug 2020 21:17:31 +0000 (14:17 -0700)]
test/crimson/seastore/test_transaction_manager: add random overwrite test

Also improves replay() by recycling all structures except for the
segment_manager.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore: add segment_cleaner
Samuel Just [Tue, 18 Aug 2020 00:20:07 +0000 (17:20 -0700)]
crimson/os/seastore: add segment_cleaner

Adds SegmentCleaner component to manage rewriting
dirty segments.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/lba_manager: add rewrite_extent
Samuel Just [Tue, 11 Aug 2020 17:47:28 +0000 (10:47 -0700)]
crimson/os/seastore/lba_manager: add rewrite_extent

Adds support for writing out an extent to a new location and updating
lba mappings.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/journal: add more debugging for replay
Samuel Just [Fri, 21 Aug 2020 21:38:32 +0000 (14:38 -0700)]
crimson/os/seastore/journal: add more debugging for replay

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/journal: initialize current_journal_segment in find_replay_segments
Samuel Just [Fri, 21 Aug 2020 21:14:56 +0000 (14:14 -0700)]
crimson/os/seastore/journal: initialize current_journal_segment in find_replay_segments

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/journal: return next write location from open_for_write
Samuel Just [Thu, 20 Aug 2020 20:54:38 +0000 (13:54 -0700)]
crimson/os/seastore/journal: return next write location from open_for_write

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/cache: replay_delta handle obsolete extents properly
Samuel Just [Fri, 21 Aug 2020 21:32:55 +0000 (14:32 -0700)]
crimson/os/seastore/cache: replay_delta handle obsolete extents properly

If we see a delta at verison v (v != 0) and we haven't seen version v-1,
we know that the extent is obsolete and can be ignored (presuming that
journal trimming is behaving).

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/journal: pass journal_seq with each replay delta
Samuel Just [Thu, 20 Aug 2020 21:15:08 +0000 (14:15 -0700)]
crimson/os/seastore/journal: pass journal_seq with each replay delta

We'll need this to track where extents became dirty during replay.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore: maintain dirty_from value on dirty CachedExtents
Samuel Just [Thu, 20 Aug 2020 21:34:55 +0000 (14:34 -0700)]
crimson/os/seastore: maintain dirty_from value on dirty CachedExtents

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agolibrbd: list-snaps needs to writeout fancy-striped extents 37074/head
Jason Dillaman [Tue, 22 Sep 2020 19:24:38 +0000 (15:24 -0400)]
librbd: list-snaps needs to writeout fancy-striped extents

If the delta is being generated for a fancy-stiped extent such that
the provided object extents are not within the actual snapshot deltas,
we still need to indicate that the object exists (as a whiteout) to
prevent the possibility of an incorrect read request to the parent.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
3 years agomgr/cephadm: get rbd-mirror daemon-id when checking for strays 37397/head
Adam King [Wed, 23 Sep 2020 16:52:37 +0000 (12:52 -0400)]
mgr/cephadm: get rbd-mirror daemon-id when checking for strays

Currently, list_servers() gets the rbd-mirror service-id instead
of the daemon-id so the daemon is marked as stray. This PR uses
that service-id to find the daemon-id and uses that to check if
the daemon is stray.

Fixes: https://tracker.ceph.com/issues/47639
Signed-off-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #37400 from cbodley/wip-rgw-DatalogTrimImplCR-logging
Casey Bodley [Fri, 25 Sep 2020 14:52:55 +0000 (10:52 -0400)]
Merge pull request #37400 from cbodley/wip-rgw-DatalogTrimImplCR-logging

rgw: raise debug level of DatalogTrimImplCR message

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agoMerge pull request #36482 from cbodley/wip-rgw-bucket-sync-checkpoint
Casey Bodley [Fri, 25 Sep 2020 14:42:29 +0000 (10:42 -0400)]
Merge pull request #36482 from cbodley/wip-rgw-bucket-sync-checkpoint

rgw: add 'bucket sync checkpoint' command to radosgw-admin

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agomgr/cephadm: do not escape HTML for custom templates 37411/head
Patrick Seidensal [Fri, 25 Sep 2020 12:05:17 +0000 (14:05 +0200)]
mgr/cephadm: do not escape HTML for custom templates

Fixes: https://tracker.ceph.com/issues/47648
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
3 years agoceph-volume: batch: fix size retrieval for lvs
Jan Fajerski [Thu, 10 Sep 2020 14:45:34 +0000 (16:45 +0200)]
ceph-volume: batch: fix size retrieval for lvs

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: include encryption in batch report
Jan Fajerski [Wed, 9 Sep 2020 11:04:14 +0000 (13:04 +0200)]
ceph-volume: include encryption in batch report

Fixes: https://tracker.ceph.com/issues/44783
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume lvm batch: use namedtuple instead of tuple
Jan Fajerski [Wed, 9 Sep 2020 07:41:15 +0000 (09:41 +0200)]
ceph-volume lvm batch: use namedtuple instead of tuple

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: address review comments, mostly tidying, clarification
Jan Fajerski [Tue, 8 Sep 2020 14:53:53 +0000 (16:53 +0200)]
ceph-volume: address review comments, mostly tidying, clarification

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoidempotency must result in the same outcome
Jan Fajerski [Tue, 8 Sep 2020 12:11:15 +0000 (14:11 +0200)]
idempotency must result in the same outcome

...not should

Co-authored-by: Joshua Schmid <jschmid@suse.de>
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: batch test should pass --journal-devices with filestore
Jan Fajerski [Mon, 7 Sep 2020 12:54:40 +0000 (14:54 +0200)]
ceph-volume: batch test should pass --journal-devices with filestore

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: make --journal optional, add --journal-slots
Jan Fajerski [Mon, 7 Sep 2020 12:54:01 +0000 (14:54 +0200)]
ceph-volume: make --journal optional, add --journal-slots

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agodoc: update ceph-volume lvm batch docs
Jan Fajerski [Mon, 29 Jun 2020 15:42:26 +0000 (17:42 +0200)]
doc: update ceph-volume lvm batch docs

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agopy-common/deployment: drop workarounds for old batch
Jan Fajerski [Fri, 26 Jun 2020 13:29:59 +0000 (15:29 +0200)]
py-common/deployment: drop workarounds for old batch

This commit drops two workarounds in drive groups for the old batch
behaviour. OSDs on LVs no longer need to be created individually by
create (related to functional testing) and ceph-volume batch now filters
unavailable drives itself.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume batch: add deprecation warning for auto behaviour
Jan Fajerski [Fri, 26 Jun 2020 11:34:01 +0000 (13:34 +0200)]
ceph-volume batch: add deprecation warning for auto behaviour

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume batch: add ceph.conf mocking to pass tests
Jan Fajerski [Tue, 23 Jun 2020 14:58:46 +0000 (16:58 +0200)]
ceph-volume batch: add ceph.conf mocking to pass tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume batch: use disk.Size for size args
Jan Fajerski [Fri, 19 Jun 2020 10:58:17 +0000 (12:58 +0200)]
ceph-volume batch: use disk.Size for size args

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume batch: Fix osd_ids passing and improve plan formatting
Jan Fajerski [Fri, 19 Jun 2020 09:22:28 +0000 (11:22 +0200)]
ceph-volume batch: Fix osd_ids passing and improve plan formatting

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume batch: track rel_size in percent, more tests
Jan Fajerski [Tue, 9 Jun 2020 14:40:46 +0000 (16:40 +0200)]
ceph-volume batch: track rel_size in percent, more tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume batch: improve backwards compatibility
Jan Fajerski [Wed, 29 Apr 2020 05:47:18 +0000 (07:47 +0200)]
ceph-volume batch: improve backwards compatibility

This restores legacy batch behavior and also adds some initial test and
adjusts existing tests to changes.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: batch - enable legacy auto behaviour
Jan Fajerski [Mon, 27 Apr 2020 10:26:20 +0000 (12:26 +0200)]
ceph-volume: batch - enable legacy auto behaviour

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: batch - major refactor
Jan Fajerski [Mon, 27 Apr 2020 09:47:04 +0000 (11:47 +0200)]
ceph-volume: batch - major refactor

This completely refactors the batch code in order to make use of the
create/prepare code path for creating OSDs instead of having a second
code path doing this. This not only eases the maintenance burden but
also adds various features and fixes bugs. This subcommand can now
handle LVs, replace OSDs, reuse VGs and has a better notion of
idempotency.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: Device - available_lvm if 10 extents are free.
Jan Fajerski [Mon, 27 Apr 2020 09:35:51 +0000 (11:35 +0200)]
ceph-volume: Device - available_lvm if 10 extents are free.

This changes the available_lvm notion to only require 10 free extents
instead of 5GB.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: Device - add vg_free property
Jan Fajerski [Mon, 27 Apr 2020 09:34:19 +0000 (11:34 +0200)]
ceph-volume: Device - add vg_free property

This new property returns the free space in any VGs present. If no VGs
are on the device we project how much space a VG will have.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: prepare/create - size args as Size class
Jan Fajerski [Mon, 27 Apr 2020 09:27:08 +0000 (11:27 +0200)]
ceph-volume: prepare/create - size args as Size class

This add the disk.Size class as all size related argument types. We
often create this class form args like this anyway and it enables users
to pass not only bytes but also strings like 50G.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: disk.Size - add cast to bool
Jan Fajerski [Mon, 27 Apr 2020 09:45:26 +0000 (11:45 +0200)]
ceph-volume: disk.Size - add cast to bool

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: api/lvm - add VolumeGroup.free_percent property
Jan Fajerski [Mon, 27 Apr 2020 09:21:37 +0000 (11:21 +0200)]
ceph-volume: api/lvm - add VolumeGroup.free_percent property

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: util.device - add vg_free_percent property
Jan Fajerski [Mon, 27 Apr 2020 09:44:56 +0000 (11:44 +0200)]
ceph-volume: util.device -  add vg_free_percent property

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: api/lvm - query LV units in bytes
Jan Fajerski [Mon, 27 Apr 2020 09:44:20 +0000 (11:44 +0200)]
ceph-volume: api/lvm - query LV units in bytes

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: lvm/common - refactor common arg specification
Jan Fajerski [Tue, 14 Apr 2020 13:34:30 +0000 (15:34 +0200)]
ceph-volume: lvm/common - refactor common arg specification

This makes it easier to create valid Namespace objects/arg lists when
programmatically calling create/prepare.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoceph-volume: batch: fix argument help message
Jan Fajerski [Wed, 26 Feb 2020 14:36:53 +0000 (15:36 +0100)]
ceph-volume: batch: fix argument help message

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
3 years agoMerge pull request #37311 from mgariepy/fix-47502
Jan Fajerski [Fri, 25 Sep 2020 09:45:16 +0000 (11:45 +0200)]
Merge pull request #37311 from mgariepy/fix-47502

[ceph-volume]: remove unneeded call to get_devices()

3 years agoclient: switch to unique_lock to avoid use the client_lock directly 36730/head
Xiubo Li [Thu, 20 Aug 2020 05:12:51 +0000 (01:12 -0400)]
client: switch to unique_lock to avoid use the client_lock directly

Fixes: https://tracker.ceph.com/issues/47039
Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoclient: unlock client_lock when copying data to bufferlist
Xiubo Li [Wed, 23 Sep 2020 01:06:05 +0000 (09:06 +0800)]
client: unlock client_lock when copying data to bufferlist

It's no need to hold the lock when copying the data, which may
take a long time.

Fixes: https://tracker.ceph.com/issues/47047
Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoclient: do more check for the client_lock
Xiubo Li [Thu, 20 Aug 2020 05:12:51 +0000 (01:12 -0400)]
client: do more check for the client_lock

Fixes: https://tracker.ceph.com/issues/47039
Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoclient: simplify the iterating related code to make it more readable
Xiubo Li [Wed, 23 Sep 2020 01:09:56 +0000 (09:09 +0800)]
client: simplify the iterating related code to make it more readable

Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge PR #36685 into master
Patrick Donnelly [Fri, 25 Sep 2020 00:22:32 +0000 (17:22 -0700)]
Merge PR #36685 into master

* refs/pull/36685/head:
mds: allow quota for root directory

Reviewed-by: Zheng Yan <zyan@redhat.com>