]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agoMerge pull request #43014 from dsavineau/ceph_volume_lvm_active_id_fsid_mandatory
Guillaume Abrioux [Thu, 9 Sep 2021 14:13:26 +0000 (16:13 +0200)]
Merge pull request #43014 from dsavineau/ceph_volume_lvm_active_id_fsid_mandatory

ceph-volume: fix lvm activate arguments

3 years agoMerge pull request #43079 from petrutlucian94/hv_addr_doc
Ilya Dryomov [Thu, 9 Sep 2021 14:12:15 +0000 (16:12 +0200)]
Merge pull request #43079 from petrutlucian94/hv_addr_doc

doc/rbd: describe Hyper-V disk addressing limitations

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43086 from idryomov/wip-rbd-validate-pool-async
Ilya Dryomov [Thu, 9 Sep 2021 13:22:43 +0000 (15:22 +0200)]
Merge pull request #43086 from idryomov/wip-rbd-validate-pool-async

librbd: fix pool validation lockup

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #43107 from SUSE/wip-fix-52553
Mykola Golub [Thu, 9 Sep 2021 10:10:44 +0000 (13:10 +0300)]
Merge pull request #43107 from SUSE/wip-fix-52553

pybind/rados: don't close watch in dealloc if already closed

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #43042 from dsavineau/ceph_volume_lvm_migrate_args
Guillaume Abrioux [Thu, 9 Sep 2021 08:42:26 +0000 (10:42 +0200)]
Merge pull request #43042 from dsavineau/ceph_volume_lvm_migrate_args

ceph-volume: fix lvm migrate without args

3 years agopybind/rados: don't close watch in dealloc if already closed 43107/head
Tim Serong [Thu, 9 Sep 2021 07:08:24 +0000 (17:08 +1000)]
pybind/rados: don't close watch in dealloc if already closed

Fixes: https://tracker.ceph.com/issues/52553
Signed-off-by: Tim Serong <tserong@suse.com>
3 years agoMerge pull request #41803 from xxhdx1985126/wip-extent-placement-manager-2
Samuel Just [Thu, 9 Sep 2021 02:08:26 +0000 (19:08 -0700)]
Merge pull request #41803 from xxhdx1985126/wip-extent-placement-manager-2

crimson/os/seastore: add extent placement manager

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43105 from liu-chunmei/clean-alien-in-store-nbd
Samuel Just [Thu, 9 Sep 2021 01:18:46 +0000 (18:18 -0700)]
Merge pull request #43105 from liu-chunmei/clean-alien-in-store-nbd

crimson/tools: clean alien in store-nbd

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agocrimson/tools: clean alien in store-nbd 43105/head
chunmei-liu [Wed, 8 Sep 2021 23:22:02 +0000 (16:22 -0700)]
crimson/tools: clean alien in store-nbd

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agoMerge pull request #42604 from sseshasa/wip-skip-osd-benchmark
Neha Ojha [Wed, 8 Sep 2021 18:03:09 +0000 (11:03 -0700)]
Merge pull request #42604 from sseshasa/wip-skip-osd-benchmark

osd: Add config option to skip running the osd benchmark during init and update documentation.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43052 from dsavineau/ceph_volume_lvm_deactive_remove_all_arg
Guillaume Abrioux [Wed, 8 Sep 2021 15:35:33 +0000 (17:35 +0200)]
Merge pull request #43052 from dsavineau/ceph_volume_lvm_deactive_remove_all_arg

ceph-volume: remove --all ref from deactivate help

3 years agoMerge pull request #43045 from sebastian-philipp/qa-tox-import-yaml
Sebastian Wagner [Wed, 8 Sep 2021 15:10:13 +0000 (17:10 +0200)]
Merge pull request #43045 from sebastian-philipp/qa-tox-import-yaml

qa: tox.ini: verify yaml syntax

Reviewed-by: Sage Weil <sage@newdream.net>
3 years agoceph-volume: fix lvm migrate without args 43042/head
Dimitri Savineau [Fri, 3 Sep 2021 14:45:18 +0000 (10:45 -0400)]
ceph-volume: fix lvm migrate without args

When running the `lvm migrate` subcommand without any args then the
ceph-volume command fails with a stack trace.

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 59, in newfunc
    return f(*a, **kw)
  File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 151, in main
    terminal.dispatch(self.mapper, subcommand_args)
  File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
    instance.main()
  File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/main.py", line 46, in main
    terminal.dispatch(self.mapper, self.argv)
  File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
    instance.main()
  File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/migrate.py", line 520, in main
    self.migrate_osd()
  File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 16, in is_root
    return func(*a, **kw)
  File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/migrate.py", line 403, in migrate_osd
    if self.args.osd_id:
AttributeError: 'Migrate' object has no attribute 'args'

That's because we're exiting the parse_argv function but we continue to
execute the migrate_osd function. We should instead exit from the main function.

This update the parsing argument to have the same code than new-db and
new-wal classes.
Now the parsing is done in the make_parser function but the argv testing is
done in the main function allowing to exit the program and displaying the
help message when no arguments are provided.

Fixes: https://tracker.ceph.com/issues/51811
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
3 years agoMerge pull request #42859 from mgfritch/cephadm-check-host-no-engine
Sebastian Wagner [Wed, 8 Sep 2021 13:40:03 +0000 (15:40 +0200)]
Merge pull request #42859 from mgfritch/cephadm-check-host-no-engine

mgr/cephadm: show unhandled exceptions during host add

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #42989 from sebastian-philipp/post-remove-osd-key-missing
Sebastian Wagner [Wed, 8 Sep 2021 13:39:38 +0000 (15:39 +0200)]
Merge pull request #42989 from sebastian-philipp/post-remove-osd-key-missing

mgr/cephadm: Add OSDService.post_remove()

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sage Weil <sage@newdream.net>
3 years agoMerge pull request #43010 from mgfritch/cephadm-log-thread-ident
Sebastian Wagner [Wed, 8 Sep 2021 13:39:07 +0000 (15:39 +0200)]
Merge pull request #43010 from mgfritch/cephadm-log-thread-ident

cephadm: add thread ident to log messages

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43048 from dsavineau/ceph_volume_lvm_migrate_no_systemd
Guillaume Abrioux [Wed, 8 Sep 2021 11:34:41 +0000 (13:34 +0200)]
Merge pull request #43048 from dsavineau/ceph_volume_lvm_migrate_no_systemd

ceph-volume: support no_systemd with lvm migrate

3 years agoMerge pull request #43051 from dsavineau/ceph_volume_lvm_migrate_docs
Guillaume Abrioux [Wed, 8 Sep 2021 11:27:04 +0000 (13:27 +0200)]
Merge pull request #43051 from dsavineau/ceph_volume_lvm_migrate_docs

doc/ceph-volume: add lvm migrate/new-db/new-wal

3 years agoMerge pull request #43050 from dsavineau/fix_raw_list_with_lsblk
Guillaume Abrioux [Wed, 8 Sep 2021 11:12:29 +0000 (13:12 +0200)]
Merge pull request #43050 from dsavineau/fix_raw_list_with_lsblk

ceph-volume: fix raw list with logical partition

3 years agolibrbd: drop ValidatePoolRequest::m_op_work_queue 43086/head
Ilya Dryomov [Sat, 4 Sep 2021 11:49:09 +0000 (13:49 +0200)]
librbd: drop ValidatePoolRequest::m_op_work_queue

It is unused now.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agolibrbd: fix pool validation lockup
Ilya Dryomov [Sat, 4 Sep 2021 10:59:07 +0000 (12:59 +0200)]
librbd: fix pool validation lockup

Concurrent rbd_pool_init() or rbd_create() operations on an unvalidated
(uninitialized) pool trigger a lockup in ValidatePoolRequest state
machine caused by blocking selfmanaged_snap_{create,remove}() calls.
There are two reactor threads by default (librados_thread_count) but we
effectively need N + 1 reactor threads for N concurrent pool validation
requests, especially for small N.

Switch to aio_selfmanaged_snap_{create,remove}().  At the time this
code was initially written, these aio variants weren't available.  The
workqueue offload introduced later worked prior to the move to asio in
pacific.

Fixes: https://tracker.ceph.com/issues/52537
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43006 from CongMinYin/fix-assert-in-handle_flushed_sync_point
Ilya Dryomov [Wed, 8 Sep 2021 09:15:18 +0000 (11:15 +0200)]
Merge pull request #43006 from CongMinYin/fix-assert-in-handle_flushed_sync_point

librbd/cache/pwl: don't clear next_sync_point_entry prematurely

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43074 from wangshuaich/rbd-children-crash-fix
Ilya Dryomov [Wed, 8 Sep 2021 08:50:20 +0000 (10:50 +0200)]
Merge pull request #43074 from wangshuaich/rbd-children-crash-fix

librbd: fix use-after-free on ictx in list_descendants()

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agodoc/rbd: describe Hyper-V disk addressing limitations 43079/head
Lucian Petrut [Tue, 7 Sep 2021 14:42:39 +0000 (14:42 +0000)]
doc/rbd: describe Hyper-V disk addressing limitations

Hyper-V identifies passthrough VM disks by number instead of SCSI ID, although
the disk number can change across host reboots. This means that the VMs can end
up using incorrect disks after rebooting the host, which is an important
security concern. This issue also affects iSCSI and Fibre Channel disks.

We're going to document this Hyper-V limitation along with possible
workarounds.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
3 years agocrimson/os/seastore: better debug outputs 41803/head
Xuehan Xu [Fri, 3 Sep 2021 15:19:28 +0000 (23:19 +0800)]
crimson/os/seastore: better debug outputs

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: allow getting empty delta from omap extents
Xuehan Xu [Fri, 3 Sep 2021 15:17:12 +0000 (23:17 +0800)]
crimson/os/seastore: allow getting empty delta from omap extents

there are chances that a transaction gets invalidated after some of
its extents have their delta prepared.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: add ExtentIndex::clear_and_dispose
Xuehan Xu [Tue, 31 Aug 2021 03:26:31 +0000 (11:26 +0800)]
crimson/os/seastore: add ExtentIndex::clear_and_dispose

When a transaction is interrupted and needs to repeat, its reset_preserve_handle() method
is called to clear various extent set and list. The problem is the clear operation call
ExtentIndex::clear() instead of ExtentIndex::erase(), which would leave CachedExtent::parent_index
still pointing to the write_set/delayed_set while the CachedExtent is no longer linked to
those sets. This would make CachedExtent::~CachedExtent() to try to erase itself from
CachedExtent::parent_index even it's not linked, which would cause failures in the successive
operations

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/store_nbd: involve the extent placement manager and the scanner
Xuehan Xu [Tue, 10 Aug 2021 04:47:12 +0000 (12:47 +0800)]
crimson/store_nbd: involve the extent placement manager and the scanner

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: make fresh extents go though the extent placement manager
Xuehan Xu [Tue, 27 Jul 2021 15:00:56 +0000 (23:00 +0800)]
crimson/os/seastore: make fresh extents go though the extent placement manager

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: deal with transaction conflicts when doing mkfs
Xuehan Xu [Fri, 13 Aug 2021 13:57:48 +0000 (21:57 +0800)]
crimson/os/seastore: deal with transaction conflicts when doing mkfs

Cleaner transactions can conflict with mkfs transactions, retry mkfs transactions
when that happens

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: set journal_tail_target during replay
Xuehan Xu [Fri, 13 Aug 2021 13:57:27 +0000 (21:57 +0800)]
crimson/os/seastore: set journal_tail_target during replay

This is a bug fix, otherwise if crimson-osd boot up multiple times without
filling up more than one segment, segments may be used up and can't be
reclaimed as they would have the same journal tail

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: set root block to clean at the initialization phase
Xuehan Xu [Fri, 13 Aug 2021 13:51:10 +0000 (21:51 +0800)]
crimson/os/seastore: set root block to clean at the initialization phase

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: exclude ool extents from the judgement is_in_journal
Xuehan Xu [Tue, 27 Jul 2021 13:00:31 +0000 (21:00 +0800)]
crimson/os/seastore: exclude ool extents from the judgement is_in_journal

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: use extent placement manager to rewrite extents
Xuehan Xu [Sun, 25 Jul 2021 12:25:48 +0000 (20:25 +0800)]
crimson/os/seastore: use extent placement manager to rewrite extents

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: refactor extent rewrite procedure
Xuehan Xu [Fri, 23 Jul 2021 12:39:49 +0000 (20:39 +0800)]
crimson/os/seastore: refactor extent rewrite procedure

Move logical extents' rewrite from LBManager to TransactionManager

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson: add tracing mode for crimson-osd
Xuehan Xu [Mon, 14 Jun 2021 11:49:14 +0000 (19:49 +0800)]
crimson: add tracing mode for crimson-osd

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: add Scanner to scan extents
Xuehan Xu [Mon, 19 Jul 2021 07:38:12 +0000 (15:38 +0800)]
crimson/os/seastore: add Scanner to scan extents

As there will be two kinds of segments to be scanned, those created by the journal
and those created by the extent placement manager. We need a common module to scan
extents of both of these two kinds of segments

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: add extent placement manager
Xuehan Xu [Thu, 10 Jun 2021 06:21:39 +0000 (14:21 +0800)]
crimson/os/seastore: add extent placement manager

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/common: a new condition variable
Xuehan Xu [Wed, 1 Sep 2021 07:33:02 +0000 (15:33 +0800)]
crimson/common: a new condition variable

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: extract record related code into seastore_types.{h,cc}
Xuehan Xu [Wed, 21 Jul 2021 08:48:50 +0000 (16:48 +0800)]
crimson/os/seastore: extract record related code into seastore_types.{h,cc}

Extent Placement Manager will be using record too.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: move JournalSegmentProvider to SegmentProvider
Xuehan Xu [Tue, 18 May 2021 07:27:05 +0000 (15:27 +0800)]
crimson/os/seastore: move JournalSegmentProvider to SegmentProvider

This provider will serve not only for the journal, but also the extent placement
manager from now on.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agoMerge pull request #43084 from athanatos/sjust/wip-lba-pin
Samuel Just [Wed, 8 Sep 2021 02:38:32 +0000 (19:38 -0700)]
Merge pull request #43084 from athanatos/sjust/wip-lba-pin

crimson/seastore: lba pin debugging and fix

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agoMerge pull request #43083 from athanatos/sjust/wip-52532
Samuel Just [Wed, 8 Sep 2021 02:38:09 +0000 (19:38 -0700)]
Merge pull request #43083 from athanatos/sjust/wip-52532

crimson/os/seastore/.../lba_btree: fix handle_split internal nodes

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agoMerge pull request #43066 from rzarzynski/wip-crimson-fatal-reraise
Samuel Just [Wed, 8 Sep 2021 01:25:55 +0000 (18:25 -0700)]
Merge pull request #43066 from rzarzynski/wip-crimson-fatal-reraise

 crimson/common: explicitly reraise handled signal in FatalSignal

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/transaction_manager: add logging for invalid pin 43084/head
Samuel Just [Thu, 2 Sep 2021 23:13:46 +0000 (16:13 -0700)]
crimson/os/seastore/transaction_manager: add logging for invalid pin

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/transaction_manager: pin may not match unless paddr does
Samuel Just [Thu, 2 Sep 2021 23:13:12 +0000 (16:13 -0700)]
crimson/os/seastore/transaction_manager: pin may not match unless paddr does

The same laddr might be reallocated with a different size.

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/lba_manager: improve add_pin debugging
Samuel Just [Tue, 31 Aug 2021 04:14:07 +0000 (21:14 -0700)]
crimson/os/seastore/lba_manager: improve add_pin debugging

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/.../lba_btree: fix handle_split internal nodes 43083/head
Samuel Just [Tue, 7 Sep 2021 22:59:08 +0000 (22:59 +0000)]
crimson/os/seastore/.../lba_btree: fix handle_split internal nodes

Internal node pointers aren't actually allowed to point to end() -- that's
specific to the leaf pointer.

Fixes: https://tracker.ceph.com/issues/52532
Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/tools/store_nbd/fs_driver: fix FuturizedStore::create() call
Samuel Just [Tue, 7 Sep 2021 23:29:44 +0000 (16:29 -0700)]
crimson/tools/store_nbd/fs_driver: fix FuturizedStore::create() call

Introduced: 8ec97762c4809c25d3b38ac15b942dfd710e3d29
Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/os/seastore/.../lba_btree: clarify handle_split, find_insertion comments
Samuel Just [Tue, 7 Sep 2021 23:01:23 +0000 (16:01 -0700)]
crimson/os/seastore/.../lba_btree: clarify handle_split, find_insertion comments

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agolibrbd: report correct error for ictx->state->close() 43074/head
Ilya Dryomov [Tue, 7 Sep 2021 19:01:51 +0000 (21:01 +0200)]
librbd: report correct error for ictx->state->close()

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agolibrbd: fix use-after-free on ictx in list_descendants()
Wang ShuaiChao [Tue, 7 Sep 2021 08:43:11 +0000 (16:43 +0800)]
librbd: fix use-after-free on ictx in list_descendants()

Ictx is deleted when "ictx->state->open()" and "ictx->state->close()"
fail, and then "lderr(ictx->cct)" crashes.

Fixes: https://tracker.ceph.com/issues/52522
Signed-off-by: Wang ShuaiChao <wangshuaich@chinatelecom.cn>
3 years agoMerge pull request #43070 from krunerge/fix-children-list-log-text
Ilya Dryomov [Tue, 7 Sep 2021 16:15:14 +0000 (18:15 +0200)]
Merge pull request #43070 from krunerge/fix-children-list-log-text

cls/rbd: fix log text for children list

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43067 from tchaikov/wip-crimson-alien
Kefu Chai [Tue, 7 Sep 2021 15:36:37 +0000 (23:36 +0800)]
Merge pull request #43067 from tchaikov/wip-crimson-alien

crimson/osd: use reactor::alien to send message to alien

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agoMerge pull request #42933 from p-se/mgr-prom-disable-cache
Kefu Chai [Tue, 7 Sep 2021 13:54:53 +0000 (21:54 +0800)]
Merge pull request #42933 from p-se/mgr-prom-disable-cache

mgr/prometheus: offer ability to disable cache

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agocrimson/osd: use reactor::alien to send message to alien 43067/head
Kefu Chai [Mon, 6 Sep 2021 16:31:27 +0000 (00:31 +0800)]
crimson/osd: use reactor::alien to send message to alien

simpler this way.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoseastar: pick up change to add reactor::alien()
Kefu Chai [Mon, 6 Sep 2021 16:21:26 +0000 (00:21 +0800)]
seastar: pick up change to add reactor::alien()

so we don't need to pass the reference to alien all the way down to
AlienStore.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoceph-volume: test the lsblk command in list.py 43050/head
Guillaume Abrioux [Tue, 7 Sep 2021 13:38:33 +0000 (15:38 +0200)]
ceph-volume: test the lsblk command in list.py

Let's test we use the expected args when we build the `lsblk` command
to list the devices present on the node.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoceph-volume: fix raw list with logical partition
Dimitri Savineau [Fri, 3 Sep 2021 18:24:48 +0000 (14:24 -0400)]
ceph-volume: fix raw list with logical partition

This is a regression introduced by 9212420, when the host is using a
logical partition then lsblk reports that partition as a child from the
physical device.
That logical partition is prefixed by the `└─` character.

This leads the `raw list` subcommand to show the lsblk error on the stderr.

```
$ ceph-volume raw list
{}
 stderr: lsblk: `-/dev/sda1: not a block device
```

The lsblk command output looks like:

```
$ lsblk --paths --output=NAME --noheadings
/dev/sda
└─/dev/sda1
/dev/sdb
/dev/sdc
/dev/sdd
```

Using the `--list` option with lsblk solves the issue.

```
$ lsblk --list --paths --output=NAME --noheadings
/dev/sda
/dev/sda1
/dev/sdb
/dev/sdc
/dev/sdd
```

Fixes: https://tracker.ceph.com/issues/52504
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
3 years agoMerge PR #43017 into master
Patrick Donnelly [Tue, 7 Sep 2021 12:56:36 +0000 (08:56 -0400)]
Merge PR #43017 into master

* refs/pull/43017/head:
doc: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43009 from fengjiankui121/pvs_repair
Guillaume Abrioux [Tue, 7 Sep 2021 11:23:16 +0000 (13:23 +0200)]
Merge pull request #43009 from fengjiankui121/pvs_repair

ceph-volume: pvs --noheadings replace pvs --no-heading

3 years agoqa: tox.ini: verify yaml syntax 43045/head
Sebastian Wagner [Fri, 3 Sep 2021 16:06:34 +0000 (18:06 +0200)]
qa: tox.ini: verify yaml syntax

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocls/rbd: fix log text for children list 43070/head
krunerge [Tue, 7 Sep 2021 02:13:28 +0000 (10:13 +0800)]
cls/rbd: fix log text for children list

Signed-off-by: krunerge <krunerge@tencent.com>
3 years agoMerge PR #42529 into master
Patrick Donnelly [Mon, 6 Sep 2021 18:00:41 +0000 (14:00 -0400)]
Merge PR #42529 into master

* refs/pull/42529/head:
qa: verify rank 0 does not fail during journal repair tests
qa: avoid stopping/restarting mds in journal repair tests

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43065 from tchaikov/wip-doc-build
Kefu Chai [Mon, 6 Sep 2021 15:16:36 +0000 (23:16 +0800)]
Merge pull request #43065 from tchaikov/wip-doc-build

admin/doc-requirements: use funcparserlib from github

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42919 from sebastian-philipp/cephadm-async-close-conn
Kefu Chai [Mon, 6 Sep 2021 14:53:25 +0000 (22:53 +0800)]
Merge pull request #42919 from sebastian-philipp/cephadm-async-close-conn

mgr/cephadm: Also make ssh._reset_con async

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Melissa Li <li.melissa.kun@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #42992 from ifed01/wip-ifed-fix-migrate
Kefu Chai [Mon, 6 Sep 2021 14:50:20 +0000 (22:50 +0800)]
Merge pull request #42992 from ifed01/wip-ifed-fix-migrate

os/bluestore: fix bluefs migrate command

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agocrimson/common: explicitly reraise handled signal in FatalSignal. 43066/head
Radoslaw Zarzynski [Mon, 6 Sep 2021 14:39:19 +0000 (14:39 +0000)]
crimson/common: explicitly reraise handled signal in FatalSignal.

Over the current approach where we just reset the handler to
default and allow CPU to re-execute the segfaulting instruction,
the explicit `::reraise()` is:
1. immune to a race condition if muliple threads run into
   troubles the same time;
2. easier to understand and similar to the classic OSD.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/common: s/sig/signum/ in FatalSignal for better readability.
Radoslaw Zarzynski [Mon, 6 Sep 2021 14:37:07 +0000 (14:37 +0000)]
crimson/common: s/sig/signum/ in FatalSignal for better readability.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #42965 from ronen-fr/wip-ronenf-split-child
Kefu Chai [Mon, 6 Sep 2021 14:45:01 +0000 (22:45 +0800)]
Merge pull request #42965 from ronen-fr/wip-ronenf-split-child

osd: adding missing data to register_and_wake_split_child() logs

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #42953 from ronen-fr/wip-ronenf-handled-objs
Kefu Chai [Mon, 6 Sep 2021 14:39:21 +0000 (22:39 +0800)]
Merge pull request #42953 from ronen-fr/wip-ronenf-handled-objs

osd/scrub: stats_of_handled_objects(): reduce irrelevant logs

Reviewed-by: Amnon Hanuhov <ahanukov@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #43061 from sebastian-philipp/cephadm-no-asyncssh
Sebastian Wagner [Mon, 6 Sep 2021 14:32:52 +0000 (16:32 +0200)]
Merge pull request #43061 from sebastian-philipp/cephadm-no-asyncssh

pybind/mgr: tox.ini add nooptional testenv

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoadmin/doc-requirements: use funcparserlib from github 43065/head
Kefu Chai [Mon, 6 Sep 2021 14:30:02 +0000 (22:30 +0800)]
admin/doc-requirements: use funcparserlib from github

funcparserlib is pulled in as a dependency by blockdiag. the latest version of
funcparserlib available on pypi is v0.3.6 which is not compatible with
Python3.8.

in this change, funcparserlib is installed from github instead to
address the build failure like:

  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/41855/lib/python3.8/site-packages/sphinxcontrib/seqdiag.py", line 26, in <module>
    import seqdiag.utils.rst.nodes
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/41855/lib/python3.8/site-packages/seqdiag/utils/rst/nodes.py", line 16, in <module>
    from blockdiag.utils.rst import nodes
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/41855/lib/python3.8/site-packages/blockdiag/utils/rst/nodes.py", line 21, in <module>
    import blockdiag.builder
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/41855/lib/python3.8/site-packages/blockdiag/builder.py", line 16, in <module>
    from blockdiag import parser
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/41855/lib/python3.8/site-packages/blockdiag/parser.py", line 43, in <module>
    from funcparserlib.parser import (a, finished, forward_decl, many, maybe, skip,
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/41855/lib/python3.8/site-packages/funcparserlib/parser.py", line 123
    except NoParseError, e:
                       ^
SyntaxError: invalid syntax

once https://github.com/vlasovskikh/funcparserlib/issues/65 is
addressed, we should drop this change.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agomgr/prometheus: offer ability to disable cache 42933/head
Patrick Seidensal [Thu, 26 Aug 2021 13:35:49 +0000 (15:35 +0200)]
mgr/prometheus: offer ability to disable cache

Fixes: https://tracker.ceph.com/issues/52414
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
3 years agolibrbd/cache/pwl: don't clear next_sync_point_entry prematurely 43006/head
Yin Congmin [Wed, 1 Sep 2021 15:56:18 +0000 (15:56 +0000)]
librbd/cache/pwl: don't clear next_sync_point_entry prematurely

In SyncPointLogOperation::clear_earlier_sync_point(),
sync_point->log_entry->next_sync_point_entry was prematurely set to
nullptr in clear_earlier_sync_point(). It is in write op stage, but
next_sync_point_entry is used in writeback stage in
handle_flushed_sync_point().

handle_flushed_sync_point() may pass a nullptr
cause assert in m_work_queue.The solution is to move the statement
that set next_sync_point_entry to nullptr after it is used.

Fixes: https://tracker.ceph.com/issues/52465
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
3 years agoMerge pull request #43034 from rubenk/fix-unmapped-bytes-help-text
Kefu Chai [Mon, 6 Sep 2021 14:10:20 +0000 (22:10 +0800)]
Merge pull request #43034 from rubenk/fix-unmapped-bytes-help-text

common/PriorityCache: fix help text for unmapped_bytes metric

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agopybind/mgr: tox.ini remove duplicated `tox` env 43061/head
Sebastian Wagner [Mon, 6 Sep 2021 10:26:21 +0000 (12:26 +0200)]
pybind/mgr: tox.ini remove duplicated `tox` env

as this is the same as `py3`. Avoid executing it twice

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agopybind/mgr: tox.ini add nooptional testenv
Sebastian Wagner [Mon, 6 Sep 2021 10:21:58 +0000 (12:21 +0200)]
pybind/mgr: tox.ini add nooptional testenv

To verify cephadm properly works as expected without asyncssh installed

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agopybind/mgr: tox.ini: norecursedirs instead explicitly list modules
Sebastian Wagner [Mon, 6 Sep 2021 09:58:49 +0000 (11:58 +0200)]
pybind/mgr: tox.ini: norecursedirs instead explicitly list modules

Simplify things by not listing all modules individually.

Also: Fixed two new doctests
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #43047 from idryomov/wip-rbd-mirror-peer-profile-doc
Ilya Dryomov [Mon, 6 Sep 2021 08:07:41 +0000 (10:07 +0200)]
Merge pull request #43047 from idryomov/wip-rbd-mirror-peer-profile-doc

doc: adjust client.rbd-mirror-peer mon cap and add details about site names

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #42840 from TRYTOBE8TME/wip-rgw-kafka-timeout
Yuval Lifshitz [Sun, 5 Sep 2021 07:58:10 +0000 (10:58 +0300)]
Merge pull request #42840 from TRYTOBE8TME/wip-rgw-kafka-timeout

Wip rgw kafka timeout

3 years agoMerge pull request #42875 from pleiadesian/feature-lua
Yuval Lifshitz [Sun, 5 Sep 2021 07:57:15 +0000 (10:57 +0300)]
Merge pull request #42875 from pleiadesian/feature-lua

examples: add elasticsearch and prometheus examples

3 years agodoc: fix typos 43017/head
wangxinyu [Thu, 2 Sep 2021 05:08:21 +0000 (13:08 +0800)]
doc: fix typos

Signed-off-by: wangxinyu <wangxinyu@inspur.com>
3 years agoMerge pull request #42870 from ktdreyer/cmake-thread-libs-init
Kefu Chai [Sat, 4 Sep 2021 14:02:19 +0000 (22:02 +0800)]
Merge pull request #42870 from ktdreyer/cmake-thread-libs-init

cmake: link Threads::Threads instead of CMAKE_THREAD_LIBS_INIT

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #42960 from wjwithagen/wjw-fix-cephfs-lock
Kefu Chai [Sat, 4 Sep 2021 02:47:50 +0000 (10:47 +0800)]
Merge pull request #42960 from wjwithagen/wjw-fix-cephfs-lock

tools/cephfs_mirror: fix lock declaratie/locking

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #42962 from tchaikov/wip-cmake-ppc
Kefu Chai [Sat, 4 Sep 2021 02:46:28 +0000 (10:46 +0800)]
Merge pull request #42962 from tchaikov/wip-cmake-ppc

arch,cmake: compile ppc.c on all powerpc machines

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoceph-volume: remove --all ref from deactivate help 43052/head
Dimitri Savineau [Fri, 3 Sep 2021 23:06:42 +0000 (19:06 -0400)]
ceph-volume: remove --all ref from deactivate help

Until the `--all` feature is implemented.

This was partially removed in c13901f but not from the help command.

Fixes: https://tracker.ceph.com/issues/50109
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
3 years agoMerge pull request #43043 from tchaikov/crimson/os/cleanup
Kefu Chai [Fri, 3 Sep 2021 22:57:33 +0000 (06:57 +0800)]
Merge pull request #43043 from tchaikov/crimson/os/cleanup

crimson/os: use structured binding in loop

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agodoc/ceph-volume: add lvm migrate/new-db/new-wal 43051/head
Dimitri Savineau [Fri, 3 Sep 2021 19:44:39 +0000 (15:44 -0400)]
doc/ceph-volume: add lvm migrate/new-db/new-wal

The inital PR was implementing those new commands only adds the man page
and not the online documentation.

Fixes: https://tracker.ceph.com/issues/51814
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
3 years agoceph-volume: support no_systemd with lvm migrate 43048/head
Dimitri Savineau [Fri, 3 Sep 2021 16:47:25 +0000 (12:47 -0400)]
ceph-volume: support no_systemd with lvm migrate

The `ceph-volume lvm migrate/new-db/new-wal` commands don't support
running on non systemd systems or within containers.
Like other ceph-volume commands (lvm activate/batch/zap or raw activate)
we also need to be able to use the --no-systemd flag.

Fixes: https://tracker.ceph.com/issues/51854
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
3 years agodoc: adjust client.rbd-mirror-peer mon cap 43047/head
Ilya Dryomov [Fri, 3 Sep 2021 14:45:26 +0000 (16:45 +0200)]
doc: adjust client.rbd-mirror-peer mon cap

On top of "profile rbd" permissions, "profile rbd-mirror-peer" also
allows getting rbd/mirror and setting rbd/mirror/peer/* config keys.
This is what "rbd mirror pool peer bootstrap create" does.

Fixes: https://tracker.ceph.com/issues/50970
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agodoc: add some details about rbd-mirror site names
Ilya Dryomov [Fri, 3 Sep 2021 16:12:40 +0000 (18:12 +0200)]
doc: add some details about rbd-mirror site names

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #43035 from zdover23/wip-doc-2021-09-03-rados-bootstrap-options
Josh Durgin [Fri, 3 Sep 2021 15:27:49 +0000 (08:27 -0700)]
Merge pull request #43035 from zdover23/wip-doc-2021-09-03-rados-bootstrap-options

doc: rados/ceph-conf.rst bootstrap options

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge PR #43012 into master
Patrick Donnelly [Fri, 3 Sep 2021 15:20:52 +0000 (11:20 -0400)]
Merge PR #43012 into master

* refs/pull/43012/head:
script: ceph-debug-docker.sh deals with the centos-repos -> centos-linux-repos transition.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43040 from sebastian-philipp/qa-deadsymlinks
Sebastian Wagner [Fri, 3 Sep 2021 15:17:03 +0000 (17:17 +0200)]
Merge pull request #43040 from sebastian-philipp/qa-deadsymlinks

qa/tox.ini: check for dead symlinks

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@newdream.net>
3 years agoMerge pull request #42836 from ivancich/wip-fix-bi-test-error
J. Eric Ivancich [Fri, 3 Sep 2021 15:09:28 +0000 (11:09 -0400)]
Merge pull request #42836 from ivancich/wip-fix-bi-test-error

rgw: fix bucket index list test error

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agocrimson/os: use structured binding in loop 43043/head
Kefu Chai [Fri, 3 Sep 2021 14:52:06 +0000 (22:52 +0800)]
crimson/os: use structured binding in loop

also avoid using `map[key] = val` for setting an item in map, as, if
he key does not exist in map, `map[key]` would have to create a value
using its default ctor, and then call the `operator=(bufferlist&&)` to
set it.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge PR #43032 into master
Patrick Donnelly [Fri, 3 Sep 2021 14:27:31 +0000 (10:27 -0400)]
Merge PR #43032 into master

* refs/pull/43032/head:
backport-create-issue: set backport priority

Reviewed-by: Nathan Cutler <ncutler@suse.com>
3 years agoMerge pull request #43039 from sebastian-philipp/rm-kubic
Sebastian Wagner [Fri, 3 Sep 2021 09:44:44 +0000 (11:44 +0200)]
Merge pull request #43039 from sebastian-philipp/rm-kubic

qa/distros: Remove stale kubic distros

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43021 from sebastian-philipp/rhel_8.3_container_tools_rhel8.yaml
Sebastian Wagner [Fri, 3 Sep 2021 09:44:19 +0000 (11:44 +0200)]
Merge pull request #43021 from sebastian-philipp/rhel_8.3_container_tools_rhel8.yaml

qa/distros/podman: Add rhel_8.3_container_tools_rhel8.yaml

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>