ceph.git
56 min agoMerge pull request #37388 from johscheuer/add-missing-python-dependency master
Kefu Chai [Sat, 26 Sep 2020 05:25:54 +0000 (13:25 +0800)]
Merge pull request #37388 from johscheuer/add-missing-python-dependency

debian: Add missing Python dependency for ceph-mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>
79 min agoMerge pull request #37380 from changchengx/ceph_port
Kefu Chai [Sat, 26 Sep 2020 05:02:35 +0000 (13:02 +0800)]
Merge pull request #37380 from changchengx/ceph_port

msgr: remove the macro not referenced anywhere

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
89 min agoMerge pull request #36779 from athanatos/sjust/wip-dirty-writeout
Kefu Chai [Sat, 26 Sep 2020 04:52:42 +0000 (12:52 +0800)]
Merge pull request #36779 from athanatos/sjust/wip-dirty-writeout

crimson/seastore: add dirty extent writeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
10 hours agotest/crimson/seastore/test_transaction_manager: add random overwrite test 36779/head
Samuel Just [Thu, 20 Aug 2020 21:17:31 +0000 (14:17 -0700)]
test/crimson/seastore/test_transaction_manager: add random overwrite test

Also improves replay() by recycling all structures except for the
segment_manager.

Signed-off-by: Samuel Just <sjust@redhat.com>
10 hours agocrimson/os/seastore: add segment_cleaner
Samuel Just [Tue, 18 Aug 2020 00:20:07 +0000 (17:20 -0700)]
crimson/os/seastore: add segment_cleaner

Adds SegmentCleaner component to manage rewriting
dirty segments.

Signed-off-by: Samuel Just <sjust@redhat.com>
10 hours agocrimson/os/seastore/lba_manager: add rewrite_extent
Samuel Just [Tue, 11 Aug 2020 17:47:28 +0000 (10:47 -0700)]
crimson/os/seastore/lba_manager: add rewrite_extent

Adds support for writing out an extent to a new location and updating
lba mappings.

Signed-off-by: Samuel Just <sjust@redhat.com>
10 hours agocrimson/os/seastore/journal: add more debugging for replay
Samuel Just [Fri, 21 Aug 2020 21:38:32 +0000 (14:38 -0700)]
crimson/os/seastore/journal: add more debugging for replay

Signed-off-by: Samuel Just <sjust@redhat.com>
10 hours agocrimson/os/seastore/journal: initialize current_journal_segment in find_replay_segments
Samuel Just [Fri, 21 Aug 2020 21:14:56 +0000 (14:14 -0700)]
crimson/os/seastore/journal: initialize current_journal_segment in find_replay_segments

Signed-off-by: Samuel Just <sjust@redhat.com>
10 hours agocrimson/os/seastore/journal: return next write location from open_for_write
Samuel Just [Thu, 20 Aug 2020 20:54:38 +0000 (13:54 -0700)]
crimson/os/seastore/journal: return next write location from open_for_write

Signed-off-by: Samuel Just <sjust@redhat.com>
10 hours agocrimson/os/seastore/cache: replay_delta handle obsolete extents properly
Samuel Just [Fri, 21 Aug 2020 21:32:55 +0000 (14:32 -0700)]
crimson/os/seastore/cache: replay_delta handle obsolete extents properly

If we see a delta at verison v (v != 0) and we haven't seen version v-1,
we know that the extent is obsolete and can be ignored (presuming that
journal trimming is behaving).

Signed-off-by: Samuel Just <sjust@redhat.com>
10 hours agocrimson/os/seastore/journal: pass journal_seq with each replay delta
Samuel Just [Thu, 20 Aug 2020 21:15:08 +0000 (14:15 -0700)]
crimson/os/seastore/journal: pass journal_seq with each replay delta

We'll need this to track where extents became dirty during replay.

Signed-off-by: Samuel Just <sjust@redhat.com>
10 hours agocrimson/os/seastore: maintain dirty_from value on dirty CachedExtents
Samuel Just [Thu, 20 Aug 2020 21:34:55 +0000 (14:34 -0700)]
crimson/os/seastore: maintain dirty_from value on dirty CachedExtents

Signed-off-by: Samuel Just <sjust@redhat.com>
15 hours agoMerge pull request #37400 from cbodley/wip-rgw-DatalogTrimImplCR-logging
Casey Bodley [Fri, 25 Sep 2020 14:52:55 +0000 (10:52 -0400)]
Merge pull request #37400 from cbodley/wip-rgw-DatalogTrimImplCR-logging

rgw: raise debug level of DatalogTrimImplCR message

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
15 hours agoMerge pull request #36482 from cbodley/wip-rgw-bucket-sync-checkpoint
Casey Bodley [Fri, 25 Sep 2020 14:42:29 +0000 (10:42 -0400)]
Merge pull request #36482 from cbodley/wip-rgw-bucket-sync-checkpoint

rgw: add 'bucket sync checkpoint' command to radosgw-admin

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
20 hours agoMerge pull request #37311 from mgariepy/fix-47502
Jan Fajerski [Fri, 25 Sep 2020 09:45:16 +0000 (11:45 +0200)]
Merge pull request #37311 from mgariepy/fix-47502

[ceph-volume]: remove unneeded call to get_devices()

29 hours agoMerge PR #36685 into master
Patrick Donnelly [Fri, 25 Sep 2020 00:22:32 +0000 (17:22 -0700)]
Merge PR #36685 into master

* refs/pull/36685/head:
mds: allow quota for root directory

Reviewed-by: Zheng Yan <zyan@redhat.com>
30 hours agoMerge PR #37009 into master
Patrick Donnelly [Fri, 25 Sep 2020 00:21:36 +0000 (17:21 -0700)]
Merge PR #37009 into master

* refs/pull/37009/head:
client: remove the unecessary client_lock for objector->read/write()
client: fix the comment for RWRef

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
32 hours agoMerge PR #37266 into master
Patrick Donnelly [Thu, 24 Sep 2020 21:37:08 +0000 (14:37 -0700)]
Merge PR #37266 into master

* refs/pull/37266/head:
qa: ignore logrotate state rename error

Reviewed-by: Rishabh Dave <ridave@redhat.com>
33 hours agoceph-volume: remove unneeded call to get_devices() 37311/head
Marc Gariepy [Tue, 22 Sep 2020 13:01:33 +0000 (09:01 -0400)]
ceph-volume: remove unneeded call to get_devices()

there is no need to probe the device to generate the argparse help
message.

also removing the test on the function as it's not there anymore.

Signed-off-by: Marc Gariepy <gariepy.marc@gmail.com>
Fixes: https://tracker.ceph.com/issues/47502

35 hours agoMerge pull request #37374 from wjwithagen/wjw-fix-crypto-alloca
Jason Dillaman [Thu, 24 Sep 2020 18:45:55 +0000 (14:45 -0400)]
Merge pull request #37374 from wjwithagen/wjw-fix-crypto-alloca

librbd: Use stdlib.h instead of alloca.h

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
36 hours agorgw: raise debug level of DatalogTrimImplCR message 37400/head
Casey Bodley [Thu, 24 Sep 2020 18:02:30 +0000 (14:02 -0400)]
rgw: raise debug level of DatalogTrimImplCR message

the message was printed at level 0 even on success

Signed-off-by: Casey Bodley <cbodley@redhat.com>
37 hours agolibrbd: Use stdlib.h instead of alloca.h 37374/head
Willem Jan Withagen [Wed, 23 Sep 2020 15:49:08 +0000 (17:49 +0200)]
librbd: Use stdlib.h instead of alloca.h

src/librbd/crypto/BlockCrypto.cc:5:10: fatal error: 'alloca.h' file not found
         ^~~~~~~~~~
1 error generated.

Including <stdlib.h> is enough

fixes: https://tracker.ceph.com/issues/47614
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
38 hours agoMerge pull request #37385 from idryomov/wip-krbd-read-only-override
Ilya Dryomov [Thu, 24 Sep 2020 16:20:48 +0000 (18:20 +0200)]
Merge pull request #37385 from idryomov/wip-krbd-read-only-override

rbd: make common options override krbd-specific options

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
39 hours agoMerge pull request #37384 from tchaikov/wip-crimson-doc
Kefu Chai [Thu, 24 Sep 2020 14:30:12 +0000 (22:30 +0800)]
Merge pull request #37384 from tchaikov/wip-crimson-doc

doc/dev/crimson: add instructions on how to build crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
44 hours agodebian: Add missing Python dependency for ceph-mgr 37388/head
Johannes M. Scheuermann [Thu, 24 Sep 2020 10:10:13 +0000 (12:10 +0200)]
debian: Add missing Python dependency for ceph-mgr

The file mgr/volumes/fs/operations/pin_util.py imports distutils.util for using
strtobool and thus the python package is required.

Signed-off-by: Johannes M. Scheuermann <joh.scheuer@gmail.com>
45 hours agorbd: make common options override krbd-specific options 37385/head
Ilya Dryomov [Thu, 24 Sep 2020 08:50:24 +0000 (10:50 +0200)]
rbd: make common options override krbd-specific options

ceph-csi has added support for passing custom map and unmap options via
mapOptions and unmapOptions storage class parameters.  However, it also
uses --read-only for implementing ROX (ReadOnlyMany) PVs.  If the user
supplies "mapOptions: rw", they will get around the intended read-only
restriction (at least on the block device).

ceph-csi could be patched to use "-o ro", but it actually makes sense
for common options to win over device type-specific equivalents.

Fixes: https://tracker.ceph.com/issues/47625
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
46 hours agoMerge pull request #37236 from jschmid1/osd_rm_doc_fix
Kefu Chai [Thu, 24 Sep 2020 07:51:05 +0000 (15:51 +0800)]
Merge pull request #37236 from jschmid1/osd_rm_doc_fix

docs: remove docs should say osd_id instead of svc_id

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
46 hours agodoc/dev/crimson: add instructions on how to build crimson 37384/head
Kefu Chai [Thu, 24 Sep 2020 06:39:41 +0000 (14:39 +0800)]
doc/dev/crimson: add instructions on how to build crimson

and add a link to Scylla's document on debugging with GDB

Signed-off-by: Kefu Chai <kchai@redhat.com>
46 hours agoMerge pull request #37275 from bk201/wip-47494
Lenz Grimmer [Thu, 24 Sep 2020 07:27:18 +0000 (09:27 +0200)]
Merge pull request #37275 from bk201/wip-47494

mgr/dashboard: display devices' health information within a tabset

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2 days agoMerge pull request #37305 from zdover23/wip-doc-3-dev-guide-basic-workflow-date-updat...
zdover23 [Thu, 24 Sep 2020 05:11:50 +0000 (15:11 +1000)]
Merge pull request #37305 from zdover23/wip-doc-3-dev-guide-basic-workflow-date-update-sep-2020

doc/dev: updated date from 2016

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 days agomsgr: remove the macro not referenced anywhere 37380/head
Changcheng Liu [Thu, 24 Sep 2020 01:25:38 +0000 (09:25 +0800)]
msgr: remove the macro not referenced anywhere

The default port range should be between:
  [ms_bind_port_min, ms_bind_port_max]

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2 days agodoc/dev: added the other four tests 37305/head
Zac Dover [Wed, 23 Sep 2020 23:49:36 +0000 (09:49 +1000)]
doc/dev: added the other four tests

This PR adds a list of tests that are performed
when a PR hits Github.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 days agocrimson/os/seastore/journal: return addr and seq from submit_record
Samuel Just [Sat, 18 Jul 2020 00:29:44 +0000 (17:29 -0700)]
crimson/os/seastore/journal: return addr and seq from submit_record

Reworks journal_seq_t to be a monotonically increasing, but non-dense
value from the current segment_seq_t and segment offset.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/journal: add interface for trimming the journal
Samuel Just [Fri, 7 Aug 2020 22:08:18 +0000 (15:08 -0700)]
crimson/os/seastore/journal: add interface for trimming the journal

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore: rework pending extents
Samuel Just [Thu, 23 Jul 2020 00:23:53 +0000 (17:23 -0700)]
crimson/os/seastore: rework pending extents

Previously, we simply added the old extent to the retired set and
treated the new instance seperately.  Instead, let the new pending
instance reference the old instance via a prior_instance reference.
This has a few advantages:
- During commit, we swap cache hooks from the prior instance to
  avoid traversal
- Extents in the retired set are only those extents that are no
  longer mapped/valid.
- btree range pins can be swapped without a traversal
- position in the dirty list can be maintained

That last will be particularly important as the dirty list will
therefore be ordered from the extent with the oldest initial
delta forward.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore: factor out LogicalCachedExtent::on_delta_write
Samuel Just [Wed, 22 Jul 2020 19:23:36 +0000 (12:23 -0700)]
crimson/os/seastore: factor out LogicalCachedExtent::on_delta_write

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/journal: rename journal_replay_lb to journal_tail
Samuel Just [Fri, 7 Aug 2020 19:44:34 +0000 (12:44 -0700)]
crimson/os/seastore/journal: rename journal_replay_lb to journal_tail

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/lba_manager/btree: factor out common LBANode helpers
Samuel Just [Wed, 22 Jul 2020 23:32:12 +0000 (16:32 -0700)]
crimson/os/seastore/lba_manager/btree: factor out common LBANode helpers

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore: remove CachedExtent move constructor
Samuel Just [Wed, 12 Aug 2020 18:10:56 +0000 (11:10 -0700)]
crimson/os/seastore: remove CachedExtent move constructor

I'm not sure why I added this in the first place, there aren't really
any scenarios where invoking it wouldn't be a bug.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/.../btree_lba_node_impl: mutate_mapping adjust addr for extent...
Samuel Just [Fri, 21 Aug 2020 21:53:36 +0000 (14:53 -0700)]
crimson/os/seastore/.../btree_lba_node_impl: mutate_mapping adjust addr for extent state

Addresses must always be adjusted depending on whether the extent will
be written as a block or as a delta.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore: improve misc debugging
Samuel Just [Tue, 18 Aug 2020 03:10:21 +0000 (20:10 -0700)]
crimson/os/seastore: improve misc debugging

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/seastore_types: permit paddr_t::operator- to operate on block...
Samuel Just [Thu, 20 Aug 2020 21:25:52 +0000 (14:25 -0700)]
crimson/os/seastore/seastore_types: permit paddr_t::operator- to operate on block relative addrs

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/segment_manager: use smaller segments for tests
Samuel Just [Thu, 20 Aug 2020 21:18:22 +0000 (14:18 -0700)]
crimson/os/seastore/segment_manager: use smaller segments for tests

This way, we'll roll segments more frequently.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/lba_manager/btree/btree_lba_manager: manage pin in init
Samuel Just [Thu, 20 Aug 2020 21:24:02 +0000 (14:24 -0700)]
crimson/os/seastore/lba_manager/btree/btree_lba_manager: manage pin in init

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/lba_manager: drop from cache directly during replay if dead
Samuel Just [Thu, 20 Aug 2020 21:23:14 +0000 (14:23 -0700)]
crimson/os/seastore/lba_manager: drop from cache directly during replay if dead

Dead extents can't go through the normal retire_extent path as they don't
have valid pins or other elements of a real extent.  Instead, simply drop
them on the floor directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/.../lba_btree_node_impl: fix find_hole
Samuel Just [Fri, 21 Aug 2020 19:57:39 +0000 (12:57 -0700)]
crimson/os/seastore/.../lba_btree_node_impl: fix find_hole

find_hole shouldn't check entries prior to min.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore: initialize crc *after* read
Samuel Just [Fri, 21 Aug 2020 20:15:21 +0000 (13:15 -0700)]
crimson/os/seastore: initialize crc *after* read

Previously, all extents ended up initialized to the same crc value
upon read -- the one for an empty buffer.  This bug wasn't detected
because an extent was always evicted and reread following the first
write and subsequently kept in cache once initially written.

Also, add crc to operator<< for extents.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/transaction_manager: only add pin if not already pending
Samuel Just [Thu, 20 Aug 2020 19:38:20 +0000 (12:38 -0700)]
crimson/os/seastore/transaction_manager: only add pin if not already pending

If it's already pending, cache.duplicate_for_write is a noop and the extent
will already have a pin.  Also, add some debugging.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/cache: let root exist in the dirty list
Samuel Just [Fri, 21 Aug 2020 05:12:05 +0000 (22:12 -0700)]
crimson/os/seastore/cache: let root exist in the dirty list

Most handling remains special, but this way the dirty list we
work on will include root directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore: clarify crc initial state
Samuel Just [Fri, 21 Aug 2020 21:24:26 +0000 (14:24 -0700)]
crimson/os/seastore: clarify crc initial state

Otherwise, different users may use different values complicating
debugging.  We can re-add this in the future if it becomes
useful.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/os/seastore/transaction: let get_extent signal retired extents
Samuel Just [Fri, 21 Aug 2020 21:28:50 +0000 (14:28 -0700)]
crimson/os/seastore/transaction: let get_extent signal retired extents

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agocrimson/common/fixed_kv_node_layout: fix lower_bound
Samuel Just [Fri, 21 Aug 2020 23:06:55 +0000 (16:06 -0700)]
crimson/common/fixed_kv_node_layout: fix lower_bound

This went undetected because all existing users ensure that
the key does not already exist.

Signed-off-by: Samuel Just <sjust@redhat.com>
2 days agoMerge pull request #37186 from guits/guits-cephadm-filter-dangling-images
Michael Fritch [Wed, 23 Sep 2020 21:30:43 +0000 (15:30 -0600)]
Merge pull request #37186 from guits/guits-cephadm-filter-dangling-images

cephadm: filter dangling images in get_last_local_ceph_image()

2 days agoMerge pull request #37133 from jschmid1/ptrace_priv
Michael Fritch [Wed, 23 Sep 2020 21:29:30 +0000 (15:29 -0600)]
Merge pull request #37133 from jschmid1/ptrace_priv

cephadm: capadd and privileged are mutex with podman >= 2.0

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2 days agoMerge PR #37310 into master
Patrick Donnelly [Wed, 23 Sep 2020 15:05:47 +0000 (08:05 -0700)]
Merge PR #37310 into master

* refs/pull/37310/head:
client: refresh rstat for quota check of rename

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 days agoMerge pull request #37041 from bk201/wip-47356
Lenz Grimmer [Wed, 23 Sep 2020 15:01:00 +0000 (17:01 +0200)]
Merge pull request #37041 from bk201/wip-47356

mgr/dashboard: fix security scopes of some NFS-Ganesha endpoints

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2 days agoMerge pull request #37244 from ceph/47543-bucket-list-perf
Lenz Grimmer [Wed, 23 Sep 2020 14:26:09 +0000 (16:26 +0200)]
Merge pull request #37244 from ceph/47543-bucket-list-perf

mgr/dashboard: fix perf. issue when listing large amounts of buckets

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2 days agoMerge pull request #37242 from dang/wip-dang-47520
Daniel Gryniewicz [Wed, 23 Sep 2020 11:57:27 +0000 (07:57 -0400)]
Merge pull request #37242 from dang/wip-dang-47520

Fix missing req_state for librgw

2 days agoMerge pull request #37330 from idryomov/wip-backport-create-issue-key
Ilya Dryomov [Wed, 23 Sep 2020 11:39:59 +0000 (13:39 +0200)]
Merge pull request #37330 from idryomov/wip-backport-create-issue-key

doc: SubmittingPatches-backports.rst: backport scripts read key from file

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2 days agoMerge pull request #37136 from p-se/cd-add-per-rbd-graphs
Lenz Grimmer [Wed, 23 Sep 2020 10:59:06 +0000 (12:59 +0200)]
Merge pull request #37136 from p-se/cd-add-per-rbd-graphs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2 days agoSubmittingPatches-backports.rst: backport scripts read key from file 37330/head
Ilya Dryomov [Wed, 23 Sep 2020 09:37:17 +0000 (11:37 +0200)]
SubmittingPatches-backports.rst: backport scripts read key from file

The --key option was removed in commit 1b5f37bbcd4a
("backport-create-issue: read redmine key from file").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 days agoMerge pull request #37192 from rhcs-dashboard/fix-46898-master
Lenz Grimmer [Wed, 23 Sep 2020 09:01:06 +0000 (11:01 +0200)]
Merge pull request #37192 from rhcs-dashboard/fix-46898-master

mgr/dashboard/api: move/create OSD histogram in separate endpoint

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2 days agoMerge pull request #37303 from tchaikov/wip-crimson-osd
Kefu Chai [Wed, 23 Sep 2020 08:49:12 +0000 (16:49 +0800)]
Merge pull request #37303 from tchaikov/wip-crimson-osd

crimson: do not set_value() twice and fix Message leak

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 days agocrimson/mon: move handle_auth_done() to "v1 and v2" section 37303/head
Kefu Chai [Wed, 23 Sep 2020 07:56:23 +0000 (15:56 +0800)]
crimson/mon: move handle_auth_done() to "v1 and v2" section

crimson::mon::Connection::handle_auth_done() is shared by both v1 and v2
authentication implementation, so move it to "v1 and v2" section

Signed-off-by: Kefu Chai <kchai@redhat.com>
2 days agoMerge pull request #37326 from tchaikov/wip-47324
Kefu Chai [Wed, 23 Sep 2020 07:11:04 +0000 (15:11 +0800)]
Merge pull request #37326 from tchaikov/wip-47324

test/crimson: better error message when bind fails

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 days agoMerge pull request #37323 from tchaikov/wip-47593
Kefu Chai [Wed, 23 Sep 2020 06:24:48 +0000 (14:24 +0800)]
Merge pull request #37323 from tchaikov/wip-47593

crimson/osd: add_recovery(oid) before recover_object(oid)

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 days agotest/crimson: better error message when bind fails 37326/head
Kefu Chai [Wed, 23 Sep 2020 06:18:52 +0000 (14:18 +0800)]
test/crimson: better error message when bind fails

See also: https://tracker.ceph.com/issues/47324

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agocrimson/osd: more constness 37323/head
Kefu Chai [Wed, 23 Sep 2020 04:24:36 +0000 (12:24 +0800)]
crimson/osd: more constness

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agocrimson/osd: add_recovery(oid) before recover_object(oid)
Kefu Chai [Wed, 23 Sep 2020 04:14:36 +0000 (12:14 +0800)]
crimson/osd: add_recovery(oid) before recover_object(oid)

in 7736bddc534e5df2daf704c4bbc92a0927b0db4d, we assumed that the object
to be recovered did not exist in `recovering` before
`recover_object(oid)` was called. but this turns out not true. so, in
this change, `add_object(oid)` is called before `recover_object(oid)`
gets called.

Fixes: https://tracker.ceph.com/issues/47593
Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agocrimson/osd: use recoverying.at() when appropriate
Kefu Chai [Wed, 23 Sep 2020 04:11:43 +0000 (12:11 +0800)]
crimson/osd: use recoverying.at() when appropriate

this helps to avoid the confusion of "where the recovery is added" and
"are we adding a new instance of recovery here".

should call add_recovery() explicitly when we need to add a new recovery
instance.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agocrimson/osd: add RecoveryBackend:add_recovering()
Kefu Chai [Wed, 23 Sep 2020 04:07:32 +0000 (12:07 +0800)]
crimson/osd: add RecoveryBackend:add_recovering()

before this change, get_recovery() can also be used for adding a
recovery instance to `recovering`. this behavior is error-prone and
confusing.

after this change, add_recovery() is used in the place where we
want to add a new instance of recovery instance.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agoMerge pull request #37290 from tchaikov/wip-run-make-check
Kefu Chai [Wed, 23 Sep 2020 01:36:45 +0000 (09:36 +0800)]
Merge pull request #37290 from tchaikov/wip-run-make-check

run-make-check.sh: fix syntax error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 days agoMerge pull request #37106 from sebastian-philipp/cephadm-overwritable-templates
Sebastian Wagner [Tue, 22 Sep 2020 23:34:57 +0000 (01:34 +0200)]
Merge pull request #37106 from sebastian-philipp/cephadm-overwritable-templates

mgr/cephadm: make j2 teamplates overwritable

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
3 days agoMerge pull request #37121 from Songweibin/wip-rbd-metadata
Jason Dillaman [Tue, 22 Sep 2020 23:14:24 +0000 (19:14 -0400)]
Merge pull request #37121 from Songweibin/wip-rbd-metadata

librbd: async version of metadata_set/remove

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
3 days agoMerge pull request #37083 from orozery/librbd-data-cryptor
Jason Dillaman [Tue, 22 Sep 2020 23:14:07 +0000 (19:14 -0400)]
Merge pull request #37083 from orozery/librbd-data-cryptor

librbd: add data encryptor/decryptor using openssl

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
3 days agoMerge pull request #37110 from CongMinYin/qa_immutable_object_cache_check
Jason Dillaman [Tue, 22 Sep 2020 23:12:42 +0000 (19:12 -0400)]
Merge pull request #37110 from CongMinYin/qa_immutable_object_cache_check

qa: check immutable object cache content

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
3 days agoMerge pull request #37237 from MahatiC/wip-fix-rwl-path
Jason Dillaman [Tue, 22 Sep 2020 23:11:47 +0000 (19:11 -0400)]
Merge pull request #37237 from MahatiC/wip-fix-rwl-path

librbd/cache: Fix pwl_path read issue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
3 days agoMerge pull request #37206 from bk201/wip-47387
Michael Fritch [Tue, 22 Sep 2020 22:37:09 +0000 (16:37 -0600)]
Merge pull request #37206 from bk201/wip-47387

mgr/rook: fix listing daemons

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
3 days agoclient: refresh rstat for quota check of rename 37310/head
Yan, Zheng [Tue, 22 Sep 2020 12:31:08 +0000 (20:31 +0800)]
client: refresh rstat for quota check of rename

Fixes: https://tracker.ceph.com/issues/47224
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
3 days agoMerge pull request #37043 from s0nea/wip-dashboard-47331
Lenz Grimmer [Tue, 22 Sep 2020 11:55:47 +0000 (13:55 +0200)]
Merge pull request #37043 from s0nea/wip-dashboard-47331

mgr/dashboard: log in non-admin users successfully if the telemetry notification is shown

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
3 days agoMerge pull request #37293 from changchengx/osd_comment
Kefu Chai [Tue, 22 Sep 2020 09:46:09 +0000 (17:46 +0800)]
Merge pull request #37293 from changchengx/osd_comment

osd: fix wrong comment about setting auth_client for messenger

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 days agocrimson/osd: do not add ref when constructing MessageRef
Kefu Chai [Tue, 22 Sep 2020 09:03:17 +0000 (17:03 +0800)]
crimson/osd: do not add ref when constructing MessageRef

crimson::osd::PG::send_cluster_message() accepts a `Message*`
pointer, and then hand it over to `shard_services.send_to_osd()`,
which expects a `Ref<Message>`. so the raw pointer is used to
construct an `intrusive_ptr<Message>`, which increment the
refcount of that Message instance by one. but that Message
was owned by nobody before that, so we end up with an
`intrusive_ptr<Message>` of 2 refcount, and only a single
owner. hence the memory leak.

in this change, instructs the constructor to not add the refcount.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agocrimson/mon: do not set_value() again
Kefu Chai [Tue, 22 Sep 2020 08:06:17 +0000 (16:06 +0800)]
crimson/mon: do not set_value() again

once the continuation consuming the stored value of the associated
future, we cannot set_value() again. otherwise, ASan complains that we
are accessing the memory on heap after it is freed.

in this change, std::optional<> is used for holding
promise<auth_result_t>, once the promise is fulfilled, `auth_done` is
reset to prevent another call of `set_value()` or `set_exception()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agodoc/dev: updated date from 2016
Zac Dover [Tue, 22 Sep 2020 09:10:15 +0000 (19:10 +1000)]
doc/dev: updated date from 2016

This updates a date from 2016 to 2020,
so that readers can be confident that the
procedure that they're reading has been recently
tested.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 days agocrimson/mon: only finish auth when auth is completed
Kefu Chai [Tue, 22 Sep 2020 06:13:52 +0000 (14:13 +0800)]
crimson/mon: only finish auth when auth is completed

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agocrimson/mon: s/AuthResult/auth_result_t/
Kefu Chai [Tue, 22 Sep 2020 06:09:40 +0000 (14:09 +0800)]
crimson/mon: s/AuthResult/auth_result_t/

auth_result_t is merely an enum, so make it like so.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agocrimson/mon: extract mon::Client::_finish_auth()
Kefu Chai [Tue, 22 Sep 2020 06:08:00 +0000 (14:08 +0800)]
crimson/mon: extract mon::Client::_finish_auth()

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 days agoosd: correct comment about setting auth_client for messenger 37293/head
Changcheng Liu [Tue, 22 Sep 2020 05:11:57 +0000 (13:11 +0800)]
osd: correct comment about setting auth_client for messenger

client_messenger's auth_client has not been set until call
monc->init();

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
3 days agoMerge pull request #37152 from rhcs-dashboard/bottom-links-component
Lenz Grimmer [Tue, 22 Sep 2020 07:23:32 +0000 (09:23 +0200)]
Merge pull request #37152 from rhcs-dashboard/bottom-links-component

mgr/dashboard: Creating a new Login Page Legal Links Component

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
3 days agomgr/dashboard: display devices' health information within a tabset 37275/head
Kiefer Chang [Thu, 17 Sep 2020 12:58:47 +0000 (20:58 +0800)]
mgr/dashboard: display devices' health information within a tabset

Wrap all devices' health information within a tabset
instead of displaying them from top to bottom.

Add more guard in the HTML template to prevent referencing undefined
variables.

Fixes: https://tracker.ceph.com/issues/47494
Fixes: https://tracker.ceph.com/issues/43177

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
4 days agoMerge pull request #37291 from badone/wip-docker-test-helper-fixes
Kefu Chai [Tue, 22 Sep 2020 06:17:23 +0000 (14:17 +0800)]
Merge pull request #37291 from badone/wip-docker-test-helper-fixes

test/docker-test: helper fixes and support more distro releases

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 days agotest: Add support for fedora 32, 33 and Ubuntu 20.04 37291/head
Brad Hubbard [Tue, 22 Sep 2020 01:29:06 +0000 (11:29 +1000)]
test: Add support for fedora 32, 33 and Ubuntu 20.04

Enable creation and use of these OS images in docker-test.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
4 days agotests: Make sure install-deps is run noninteractively
Brad Hubbard [Tue, 22 Sep 2020 02:51:28 +0000 (12:51 +1000)]
tests: Make sure install-deps is run noninteractively

This gets past things like tzconfig stopping for user input.
Remove redundant install of python-virtualenv.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
4 days agoMerge pull request #37200 from tchaikov/wip-crimson-with-lock
Kefu Chai [Tue, 22 Sep 2020 02:53:48 +0000 (10:53 +0800)]
Merge pull request #37200 from tchaikov/wip-crimson-with-lock

crimson/common: add helpers for seastar::with_lock()

Reviewed-by: Samuel Just <sjust@redhat.com>
4 days agorun-make-check.sh: fix syntax error 37290/head
Kefu Chai [Tue, 22 Sep 2020 02:17:56 +0000 (10:17 +0800)]
run-make-check.sh: fix syntax error

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 days agoMerge pull request #37211 from tchaikov/wip-zbd
Kefu Chai [Tue, 22 Sep 2020 01:42:09 +0000 (09:42 +0800)]
Merge pull request #37211 from tchaikov/wip-zbd

rpm,install-deps,run-make-check: enable WITH_ZBD build on demand

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 days agotest/centos-8: Install git before running install-deps
Brad Hubbard [Tue, 22 Sep 2020 00:30:01 +0000 (10:30 +1000)]
test/centos-8: Install git before running install-deps

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
4 days agotest/docker-test: Fix permissions issue when using podman
Brad Hubbard [Tue, 22 Sep 2020 00:28:09 +0000 (10:28 +1000)]
test/docker-test: Fix permissions issue when using podman

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
4 days agoMerge PR #36776 into master
Patrick Donnelly [Mon, 21 Sep 2020 22:26:22 +0000 (15:26 -0700)]
Merge PR #36776 into master

* refs/pull/36776/head:
systemd: Support Graceful Reboot for AIO Node

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
4 days agoMerge PR #37227 into master
Patrick Donnelly [Mon, 21 Sep 2020 21:04:06 +0000 (14:04 -0700)]
Merge PR #37227 into master

* refs/pull/37227/head:
qa/cephfs: don't pass args to destroy() in recreate()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>