]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoqa: get pool holding file data from file layout 38776/head
Patrick Donnelly [Tue, 5 Jan 2021 21:43:26 +0000 (13:43 -0800)]
qa: get pool holding file data from file layout

Otherwise it always looks at the default data pool. For ec pools, this
may not be where the file data is.

Fixes: https://tracker.ceph.com/issues/48756
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #38742 into master
Patrick Donnelly [Tue, 5 Jan 2021 16:38:41 +0000 (08:38 -0800)]
Merge PR #38742 into master

* refs/pull/38742/head:
doc/cephfs/nfs: Add more details about path and pseudo path
doc/cephfs/nfs: Add note about how to fetch port details with rook
doc/cephfs/nfs: Add warning note about management of dashboard created exports

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
4 years agoMerge pull request #38735 from changchengx/pwl_load_existing_cache
Jason Dillaman [Tue, 5 Jan 2021 14:33:49 +0000 (09:33 -0500)]
Merge pull request #38735 from changchengx/pwl_load_existing_cache

librbd/cache/pwl: correct update previous_sync_point_entry

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #38520 from lixiaoy1/wip-doc-pwl
Jason Dillaman [Tue, 5 Jan 2021 14:12:20 +0000 (09:12 -0500)]
Merge pull request #38520 from lixiaoy1/wip-doc-pwl

doc: add RBD persistent write-back cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agodoc: add RBD persistent write-back cache 38520/head
lixiaoy1 [Tue, 17 Nov 2020 14:47:42 +0000 (09:47 -0500)]
doc: add RBD persistent write-back cache

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
4 years agodoc/cephfs/nfs: Add more details about path and pseudo path 38742/head
Varsha Rao [Thu, 31 Dec 2020 11:30:29 +0000 (17:00 +0530)]
doc/cephfs/nfs: Add more details about path and pseudo path

Fixes: https://tracker.ceph.com/issues/48731
Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/cephfs/nfs: Add note about how to fetch port details with rook
Varsha Rao [Thu, 31 Dec 2020 10:54:34 +0000 (16:24 +0530)]
doc/cephfs/nfs: Add note about how to fetch port details with rook

Fixes: https://tracker.ceph.com/issues/48731
Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/cephfs/nfs: Add warning note about management of dashboard created exports
Varsha Rao [Thu, 31 Dec 2020 10:45:09 +0000 (16:15 +0530)]
doc/cephfs/nfs: Add warning note about management of dashboard created exports

Fixes: https://tracker.ceph.com/issues/48731
Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agoMerge PR #32776 into master
Jan Fajerski [Tue, 5 Jan 2021 11:50:37 +0000 (12:50 +0100)]
Merge PR #32776 into master

* refs/pull/32776/head:
win32*.sh: fetch WNBD dependency
win32*.sh: bump win32 boost dependency
cmake: skip osd, Lua and neorados on Windows
win32*.sh: improve lib handling
win32*.sh: update openssl location
win32*.sh: cleanup dependency build dir when targetting Windows
win32*.sh: Strip binaries individually
win32*.sh: Cleanup zip archive creation
cmake,win32*.sh: Add OS=(ubuntu|suse) flag, for distro settings
win32*.sh: Use nproc for CPU count when targeting win32
test: Include compat.h to find uid_t/gid_t/gidlist when cross compiling
win32*.sh: improve win32 dependency check
win32*.sh: isolate network operations when building win32 dependencies
cmake: drop Python dependency when targeting Windows
win32*.sh: cleanup Windows build script
win32*.sh: fix patch command
win32*.sh: update Windows build scripts and readme

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #38699 from rosinL/boost-paralllel
Kefu Chai [Tue, 5 Jan 2021 11:06:57 +0000 (19:06 +0800)]
Merge pull request #38699 from rosinL/boost-paralllel

do_cmake: add a default value to BOOST_J

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38745 from zdover23/wip-doc-jan-2021-dev-guide-basic-workflow...
zdover23 [Tue, 5 Jan 2021 08:34:19 +0000 (18:34 +1000)]
Merge pull request #38745 from zdover23/wip-doc-jan-2021-dev-guide-basic-workflow-reorg

doc/dev: restructure basic-workflow.rst

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #38749 from tchaikov/wip-cmake-check-nasm
Kefu Chai [Tue, 5 Jan 2021 02:32:48 +0000 (10:32 +0800)]
Merge pull request #38749 from tchaikov/wip-cmake-check-nasm

cmake: only display the result of checking nasm once

Reviewed-by: Changcheng Liu <changcheng.liu@aliyun.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #38703 into master
Patrick Donnelly [Mon, 4 Jan 2021 17:49:00 +0000 (09:49 -0800)]
Merge PR #38703 into master

* refs/pull/38703/head:
test: move snap delete operation to non-root directory
client: optionally check client permission on rmsnap()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #38758 from tchaikov/wip-crimson-cleanup
Kefu Chai [Mon, 4 Jan 2021 17:39:14 +0000 (01:39 +0800)]
Merge pull request #38758 from tchaikov/wip-crimson-cleanup

crimson/os: do not let omap_get_values() return input_output_error

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #38757 from rosinL/wip-remove-unused-options
Kefu Chai [Mon, 4 Jan 2021 15:32:18 +0000 (23:32 +0800)]
Merge pull request #38757 from rosinL/wip-remove-unused-options

common/options: remove ununsed options

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: do not let omap_get_values() return input_output_error 38758/head
Kefu Chai [Mon, 4 Jan 2021 15:25:10 +0000 (23:25 +0800)]
crimson/os: do not let omap_get_values() return input_output_error

BlueStore only returns 0 or -ENOENT. so no need to handle the cases
where the return value is neither of them.

but Seastore might return EIO, so keep the interface unchanged.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38616 from ofriedma/wip-ofriedma-dist-cache
ofriedma [Mon, 4 Jan 2021 15:11:37 +0000 (17:11 +0200)]
Merge pull request #38616 from ofriedma/wip-ofriedma-dist-cache

rgw: distribute cache for exclusive put

4 years agoMerge pull request #38663 from tchaikov/wip-crimson-recovery-cleanups
Kefu Chai [Mon, 4 Jan 2021 14:54:10 +0000 (22:54 +0800)]
Merge pull request #38663 from tchaikov/wip-crimson-recovery-cleanups

crimson/osd: fixes and cleanups

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocommon/options: remove ununsed options 38757/head
luo rixin [Mon, 4 Jan 2021 12:40:54 +0000 (20:40 +0800)]
common/options: remove ununsed options

Signed-off-by: luo rixin <luorixin@huawei.com>
4 years agoMerge pull request #38676 from ronen-fr/wip-48690
Kefu Chai [Mon, 4 Jan 2021 11:04:45 +0000 (19:04 +0800)]
Merge pull request #38676 from ronen-fr/wip-48690

qa/standalone/scrub/osd-scrub-snaps.sh: fix Python print syntax

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38603 from smithfarm/wip-48604
Nathan Cutler [Mon, 4 Jan 2021 09:58:54 +0000 (10:58 +0100)]
Merge pull request #38603 from smithfarm/wip-48604

rpm: require smartmontools on SUSE

Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agocmake: only display the result of checking nasm once 38749/head
Kefu Chai [Sun, 3 Jan 2021 17:15:54 +0000 (01:15 +0800)]
cmake: only display the result of checking nasm once

if nasm is able to emit AVX512 instructions, we can assume that it's
able to generate AVX2 instructions as well. so no need to print "Found
nasm" multiple times.

and by chaining mutual exclusive branches with "elseif", we can have
better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: drop duplicated variable
Kefu Chai [Sun, 3 Jan 2021 17:28:23 +0000 (01:28 +0800)]
cmake: drop duplicated variable

not_arch_x32 replicates ${_support_x64}. and a variable starts with
"not" is just difficult to parse. so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38694 from dillaman/wip-migration-import
Mykola Golub [Sun, 3 Jan 2021 09:07:20 +0000 (11:07 +0200)]
Merge pull request #38694 from dillaman/wip-migration-import

librbd/migration: tweaks and initial set of documentation

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agodoc/rbd: incorporate external data source support for live-migration 38694/head
Jason Dillaman [Mon, 21 Dec 2020 19:40:13 +0000 (14:40 -0500)]
doc/rbd: incorporate external data source support for live-migration

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agolibrbd/migration: fixed missing return statements on stream error path
Jason Dillaman [Tue, 22 Dec 2020 18:56:59 +0000 (13:56 -0500)]
librbd/migration: fixed missing return statements on stream error path

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agolibrbd/migration: optionally pull S3 keys from MON config store
Jason Dillaman [Tue, 22 Dec 2020 18:36:57 +0000 (13:36 -0500)]
librbd/migration: optionally pull S3 keys from MON config store

This allows the S3 keys to be better protected since the MON can
be configured to restrict access to keys by user and the results
can be encrypted in transit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agolibrbd: helper utility to retrieve config from the MON config store
Jason Dillaman [Tue, 22 Dec 2020 18:34:51 +0000 (13:34 -0500)]
librbd: helper utility to retrieve config from the MON config store

A special "config://" URI prefix can be used to denote configuration
settings that should be (securely) pulled from the MON config store.
This will be first used in a follow-up commit to support storing
the S3 access and secret keys in the MON config store.

This "config://" syntax is already in-use by RGW and ceph-iscsi for
pulling secrets when deployed via cephadm.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agolibrbd/migration: require snapshot when importing from native source
Jason Dillaman [Tue, 22 Dec 2020 14:46:04 +0000 (09:46 -0500)]
librbd/migration: require snapshot when importing from native source

Since we cannot mark the source image read-only when running in import-only
migration mode, we should require the user to provide a snapshot to ensure
that data cannot change while the migration is running.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #30027 from majianpeng/optimize-mutex-contention
Kefu Chai [Fri, 1 Jan 2021 04:12:43 +0000 (12:12 +0800)]
Merge pull request #30027 from majianpeng/optimize-mutex-contention

os/bluestore: optimize mutex contention

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #36088 from rhcs-dashboard/fix-45822-master
Kefu Chai [Fri, 1 Jan 2021 04:10:25 +0000 (12:10 +0800)]
Merge pull request #36088 from rhcs-dashboard/fix-45822-master

mgr/dashboard:minimize console log traces of Ceph backend API test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #38651 from ifed01/wip-ifed-new-omap-naming
Kefu Chai [Fri, 1 Jan 2021 04:06:39 +0000 (12:06 +0800)]
Merge pull request #38651 from ifed01/wip-ifed-new-omap-naming

os/bluestore: introduce new "per-pg" naming scheme for omap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
4 years agoMerge pull request #38680 from ifed01/wip-ifed-rocksdb-options-annex
Kefu Chai [Fri, 1 Jan 2021 04:01:55 +0000 (12:01 +0800)]
Merge pull request #38680 from ifed01/wip-ifed-rocksdb-options-annex

os/bluestore: introduce bluestore_rocksdb_options_annex config parame…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
4 years agoMerge pull request #38738 from ronen-fr/wip-ronenf-rawformat
Jason Dillaman [Thu, 31 Dec 2020 13:00:37 +0000 (08:00 -0500)]
Merge pull request #38738 from ronen-fr/wip-ronenf-rawformat

librbd/migration: removing an unused capture

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #38709 from wuhongsong/aios_be_truncated
Kefu Chai [Thu, 31 Dec 2020 11:41:37 +0000 (19:41 +0800)]
Merge pull request #38709 from wuhongsong/aios_be_truncated

blk: add upper bound of bluestore_deferred_batch_ops* options

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/dev: restructure basic-workflow.rst 38745/head
Zac Dover [Wed, 30 Dec 2020 10:20:41 +0000 (20:20 +1000)]
doc/dev: restructure basic-workflow.rst

- Rewrote a sentence about forking.
- s/exactly how to proceed/how to proceed/
- Changed titles to present participles
- Rewrote "Fixing the Bug Locally" to improve its elegance.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #38655 from dillaman/wip-47828
Mykola Golub [Wed, 30 Dec 2020 14:15:47 +0000 (16:15 +0200)]
Merge pull request #38655 from dillaman/wip-47828

librbd: ensure stripe unit is at least 512 bytes

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agolibrbd/migration: removing an unused capture 38738/head
Ronen Friedman [Wed, 30 Dec 2020 06:35:00 +0000 (08:35 +0200)]
librbd/migration: removing an unused capture

...silencing a compiler warning.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
4 years agolibrbd/cache/pwl: correct update previous_sync_point_entry 38735/head
Changcheng Liu [Wed, 30 Dec 2020 05:30:41 +0000 (13:30 +0800)]
librbd/cache/pwl: correct update previous_sync_point_entry

Always update previous_sync_point_entry value whatever its old
value.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agoMerge pull request #38733 from ukernel/wip-yanzheng-email
Kefu Chai [Wed, 30 Dec 2020 03:23:57 +0000 (11:23 +0800)]
Merge pull request #38733 from ukernel/wip-yanzheng-email

mailmap: update affiliation for Yan, Zheng

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38237 from myoungwon/wip-fix-clone-size-mismatch
Kefu Chai [Wed, 30 Dec 2020 03:19:51 +0000 (11:19 +0800)]
Merge pull request #38237 from myoungwon/wip-fix-clone-size-mismatch

osd: fix clone size mismatch when deduped object is evicted

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agomailmap: update affiliation for Yan, Zheng 38733/head
Yan, Zheng [Wed, 30 Dec 2020 02:37:20 +0000 (10:37 +0800)]
mailmap: update affiliation for Yan, Zheng

Signed-off-by: "Yan, Zheng" <ukernel@gmail.com>
4 years agoMerge pull request #37704 from Huber-ming/log_error
Kefu Chai [Tue, 29 Dec 2020 16:31:02 +0000 (00:31 +0800)]
Merge pull request #37704 from Huber-ming/log_error

radosgw-admin: fix error message when datalog list

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoblk: add upper bound of bluestore_deferred_batch_ops* options 38709/head
hzwuhongsong [Tue, 29 Dec 2020 06:24:01 +0000 (14:24 +0800)]
blk: add upper bound of bluestore_deferred_batch_ops* options

so the number of pending io does not overflow when being passed to submit_batch().

Signed-off-by: hzwuhongsong <hzwuhongsong@corp.netease.com>
4 years agoMerge pull request #38728 from xxhdx1985126/wip-crimson-osd_op-no-splice
Kefu Chai [Tue, 29 Dec 2020 07:52:41 +0000 (15:52 +0800)]
Merge pull request #38728 from xxhdx1985126/wip-crimson-osd_op-no-splice

crimson/osd: do not splice osd_op indata

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: make sure PGBackend's modification operations not change osd_op state 38728/head
Xuehan Xu [Tue, 29 Dec 2020 07:19:52 +0000 (15:19 +0800)]
crimson/osd: make sure PGBackend's modification operations not change osd_op state

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/osd: do not splice osd_op indata
Xuehan Xu [Tue, 29 Dec 2020 04:06:43 +0000 (12:06 +0800)]
crimson/osd: do not splice osd_op indata

As each osd_op's indata holds only its own data, there's no need to drop their it.
On the other hand, indata is still needed when osd_op needs to be redone in case of
acting set change.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #38633 from rhcs-dashboard/fix-logs-copy-button
Kiefer Chang [Mon, 28 Dec 2020 13:12:14 +0000 (21:12 +0800)]
Merge pull request #38633 from rhcs-dashboard/fix-logs-copy-button

mgr/dashboard: Fix for copy2Clipboard failing to copy on the first time

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
4 years agomgr/dashboard: Fix for copy2Clipboard failing to copy on the first time. 38633/head
Avan Thakkar [Thu, 17 Dec 2020 09:44:46 +0000 (15:14 +0530)]
mgr/dashboard: Fix for copy2Clipboard failing to copy on the first time.

Fixes: https://tracker.ceph.com/issues/48601
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Fix for the copy2Clipboard button to work on the first click itself.

4 years agoMerge pull request #37267 from TheJJ/fix-up-acting-docs
Kefu Chai [Mon, 28 Dec 2020 06:38:12 +0000 (14:38 +0800)]
Merge pull request #37267 from TheJJ/fix-up-acting-docs

doc: fix up and acting set explanation

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc: fix and improve the explainations of up and acting osd sets 37267/head
Jonas Jelten [Sun, 20 Sep 2020 21:05:33 +0000 (23:05 +0200)]
doc: fix and improve the explainations of up and acting osd sets

acting = where clients requests are processed
up = where pgs were planned to be by crush+upmap

Fixes: https://tracker.ceph.com/issues/48718
Signed-off-by: Jonas Jelten <jj@sft.lol>
4 years agomailmap: update mail for Jonas Jelten
Jonas Jelten [Sun, 20 Sep 2020 21:07:54 +0000 (23:07 +0200)]
mailmap: update mail for Jonas Jelten

Signed-off-by: Jonas Jelten <jj@sft.lol>
4 years agoMerge pull request #38719 from fengchunsong/master
Kefu Chai [Sat, 26 Dec 2020 17:11:53 +0000 (01:11 +0800)]
Merge pull request #38719 from fengchunsong/master

test/perf_local: Add div32,prefetch,memory barrier bench for arm64

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38568 from batrick/i48594
Sebastian Wagner [Sat, 26 Dec 2020 12:08:33 +0000 (13:08 +0100)]
Merge pull request #38568 from batrick/i48594

pybind/mgr/cephadm: limit rgw osd caps

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #38644 from pcuzner/fix-missing-py3-error
Sebastian Wagner [Sat, 26 Dec 2020 12:04:16 +0000 (13:04 +0100)]
Merge pull request #38644 from pcuzner/fix-missing-py3-error

mgr/cephadm: correct connection failure message

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #38654 from Daniel-Pivonka/osddryrunbug
Sebastian Wagner [Sat, 26 Dec 2020 12:03:07 +0000 (13:03 +0100)]
Merge pull request #38654 from Daniel-Pivonka/osddryrunbug

mgr/cephadm:fix bug in orch apply osd --dry-run where empty table printed

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agotest/perf_local: Add div32,prefetch,memory barrier bench for arm64 38719/head
Chunsong Feng [Sat, 26 Dec 2020 09:06:06 +0000 (17:06 +0800)]
test/perf_local: Add div32,prefetch,memory barrier bench for arm64

arm64 support div32,prefetch,memory barrier, so add them to benchmark.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
4 years agoMerge pull request #38717 from swinds24/fix_compile_error
Kefu Chai [Sat, 26 Dec 2020 01:58:15 +0000 (09:58 +0800)]
Merge pull request #38717 from swinds24/fix_compile_error

kv/LevelDBStore: fix incorrect ceph_abort_msg() usage

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agokv/LevelDBStore: fix incorrect ceph_abort_msg() usage 38717/head
Jerry Lee [Fri, 25 Dec 2020 09:00:22 +0000 (17:00 +0800)]
kv/LevelDBStore: fix incorrect ceph_abort_msg() usage

Compile error is encountered when HAVE_LEVELDB_FILTER_POLICY is not
defined.

/ceph/src/kv/LevelDBStore.cc: In member function 'int LevelDBStore::load_leveldb_options(bool, leveldb::Options&)':
/ceph/src/kv/LevelDBStore.cc:96:22: error: invalid initialization of reference of type 'const string& {aka const std::__cxx11::basic_string<char>&}' from expression of type 'bool'ceph_abort_msg(0 == "bloom size set but installed leveldb doesn't support bloom filters");
                    ~~^~~~
/ceph/src/include/ceph_assert.h:90:69: note: in definition of macro 'ceph_abort_msg'
   ::ceph::__ceph_abort( __FILE__, __LINE__, __CEPH_ASSERT_FUNCTION, msg)
                                                                     ^~~
/ceph/src/include/ceph_assert.h:62:19: note: in passing argument 4 of 'void ceph::__ceph_abort(const char*, int, const char*, const string&)'
 [[noreturn]] void __ceph_abort(const char *file, int line, const char *func,
                   ^~~~~~~~~~~~
src/kv/CMakeFiles/kv.dir/build.make:130: recipe for target 'src/kv/CMakeFiles/kv.dir/LevelDBStore.cc.o' failed

Signed-off-by: Jerry Lee <leisurelysw24@gmail.com>
4 years agoMerge pull request #38702 from mkogan1/wip-fix-vstart-o
Kefu Chai [Thu, 24 Dec 2020 11:50:50 +0000 (19:50 +0800)]
Merge pull request #38702 from mkogan1/wip-fix-vstart-o

vstart: fix -o ceph.conf options processing

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38711 from Rethan/fix-osdmap-message-max-bytes-comments
Kefu Chai [Thu, 24 Dec 2020 11:47:07 +0000 (19:47 +0800)]
Merge pull request #38711 from Rethan/fix-osdmap-message-max-bytes-comments

option: update comments of option osd_map_message_max_bytes

4 years agooption: update comments of option osd_map_message_max_bytes 38711/head
haoyixing [Thu, 24 Dec 2020 10:22:30 +0000 (18:22 +0800)]
option: update comments of option osd_map_message_max_bytes

Option osd_map_message_max_bytes indicates the max bytes of MOSDMap message, not maps count
as osd_map_message_max.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>
4 years agoMerge pull request #38700 from tchaikov/wip-seastar-msgr-test-timeout
Kefu Chai [Thu, 24 Dec 2020 06:23:32 +0000 (14:23 +0800)]
Merge pull request #38700 from tchaikov/wip-seastar-msgr-test-timeout

test/crimson: add abort timer to messenger test

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #38576 from myoungwon/wip-fix-seg-fault-manifest-count
Kefu Chai [Thu, 24 Dec 2020 06:10:03 +0000 (14:10 +0800)]
Merge pull request #38576 from myoungwon/wip-fix-seg-fault-manifest-count

osd : prevent accessing deleted reference

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #38662 from Songweibin/wip-121918
Kefu Chai [Thu, 24 Dec 2020 06:08:48 +0000 (14:08 +0800)]
Merge pull request #38662 from Songweibin/wip-121918

osd/ReplicatedBackend: handle tailing zeros of recovering objects nicer

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #38665 from tchaikov/wip-arm32-cleanups
Kefu Chai [Thu, 24 Dec 2020 06:06:19 +0000 (14:06 +0800)]
Merge pull request #38665 from tchaikov/wip-arm32-cleanups

erasure-code, test, pybind: various arm32 related cleanups.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #38677 from ilanddev/mgr-deadlock-fix
Kefu Chai [Thu, 24 Dec 2020 06:02:48 +0000 (14:02 +0800)]
Merge pull request #38677 from ilanddev/mgr-deadlock-fix

mgr/ActivePyModules.cc: always release GIL before attempting to acquire a lock

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38678 from tchaikov/wip-denc
Kefu Chai [Thu, 24 Dec 2020 06:00:42 +0000 (14:00 +0800)]
Merge pull request #38678 from tchaikov/wip-denc

include/denc: use pair<const K,V> in range-based for loop

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
4 years agoMerge PR #37428 into master
Patrick Donnelly [Thu, 24 Dec 2020 04:35:39 +0000 (20:35 -0800)]
Merge PR #37428 into master

* refs/pull/37428/head:
mds: move duplicated inner loop out
mds: try to replicate hot dir to restarted MDS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
4 years agoMerge PR #36597 into master
Patrick Donnelly [Thu, 24 Dec 2020 04:34:25 +0000 (20:34 -0800)]
Merge PR #36597 into master

* refs/pull/36597/head:
client: make the send_to_auth display of mds_requests more accurate

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #38037 into master
Patrick Donnelly [Thu, 24 Dec 2020 04:33:47 +0000 (20:33 -0800)]
Merge PR #38037 into master

* refs/pull/38037/head:
qa: test_readahead add kernel client support

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #38631 into master
Patrick Donnelly [Thu, 24 Dec 2020 04:33:05 +0000 (20:33 -0800)]
Merge PR #38631 into master

* refs/pull/38631/head:
MDSLogContext: fix typo in MDSLogContextBase and add more comments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #38672 into master
Patrick Donnelly [Thu, 24 Dec 2020 04:32:34 +0000 (20:32 -0800)]
Merge PR #38672 into master

* refs/pull/38672/head:
mds: increase the l_mds_openino_backtrace_fetch when fetching the backtrace

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #38692 into master
Patrick Donnelly [Thu, 24 Dec 2020 04:30:35 +0000 (20:30 -0800)]
Merge PR #38692 into master

* refs/pull/38692/head:
pybind/cephfs: pass command as size-1 array of strings

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #38668 into master
Patrick Donnelly [Thu, 24 Dec 2020 04:20:09 +0000 (20:20 -0800)]
Merge PR #38668 into master

* refs/pull/38668/head:
osdc: Fix return type of map waiting functions
monc: Use post rather than defer
common/async: Hold lock in constructor of blocked_result
osdc: Don't keep wrapping completion in wait_for_latest_osdmap
neorados: Replace unsafe uses of dispatch with post

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoos/bluestore: upgrade omap naming scheme to be per-pg 38651/head
Igor Fedotov [Fri, 25 Sep 2020 18:34:43 +0000 (21:34 +0300)]
os/bluestore: upgrade omap naming scheme to be per-pg

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #38707 from ceph/wip-yuriw-octopus-x-4-master
Yuri Weinstein [Wed, 23 Dec 2020 21:09:51 +0000 (13:09 -0800)]
Merge pull request #38707 from ceph/wip-yuriw-octopus-x-4-master

qa/tests: initial check in for octopus-x/parallel tests (try 2)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoqa/tasks/cephadm: include first mgr host when applying mgrs 38707/head
Josh Durgin [Wed, 23 Dec 2020 00:25:48 +0000 (19:25 -0500)]
qa/tasks/cephadm: include first mgr host when applying mgrs

This prevents the first mgr from being shut down due to lack of
appropriate placements.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agorgw: distribute cache for exclusive put 38616/head
Or Friedmann [Wed, 16 Dec 2020 16:09:24 +0000 (18:09 +0200)]
rgw: distribute cache for exclusive put

Distribute cache for exclusive put

Fixes: https://tracker.ceph.com/issues/48632
Thanks to Casey Bodley for finding the solution

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
4 years agotest: move snap delete operation to non-root directory 38703/head
Venky Shankar [Wed, 23 Dec 2020 05:14:31 +0000 (00:14 -0500)]
test: move snap delete operation to non-root directory

In teuthology, root directory in a Ceph Filesystem is owned
by root (0:0) and has sticky-bit enabled. Snapshots created
on the directory carry its UID/GID and mode.

This causes tests that remove snapshots on root directory to
fail. Sticky bit on a directory gives permission to delete a
file in that directory only if you own the file.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
4 years agoclient: optionally check client permission on rmsnap()
Venky Shankar [Wed, 23 Dec 2020 11:34:22 +0000 (06:34 -0500)]
client: optionally check client permission on rmsnap()

By default, Client::rmsnap() skips checking client permissions.
This was ok till now since it was not a user facing API (via
libcephfs).

With PR #37721 ("mds: store custom metadata on snapshot creation"),
rmsnap can be invoked via ceph_rmsnap() libcephfs call. Therefore,
the usual client side permission checks should be performed before
calling out to the MDS.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
4 years agovstart: fix -o ceph.conf options processing 38702/head
Mark Kogan [Wed, 23 Dec 2020 11:54:32 +0000 (13:54 +0200)]
vstart: fix -o ceph.conf options processing

options passed to vstart.sh via the -o param and bunched on the same
line without line termination between them
hence not being parsed.

ex:
```
cat ./ceph.conf
...
debug_ms=0 debug_objecter=0 bluestore_debug_enforce_settings=ssd
bluestore_block_size=214748364800
...
```

Fixes: https://tracker.ceph.com/issues/48708
Signed-off-by: Mark Kogan <mkogan@redhat.com>
4 years agotest/crimson: add abort timer to messenger test 38700/head
Kefu Chai [Wed, 23 Dec 2020 09:21:35 +0000 (17:21 +0800)]
test/crimson: add abort timer to messenger test

Use a 120s timer to force abort the messenger test which should normally
succeeds in 60s, in order to:
* Fail the make check to print logs about where it is blocking;
* Free the listening IP address for the next make check;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodo_cmake: add a default value to BOOST_J 38699/head
luo rixin [Wed, 23 Dec 2020 08:37:12 +0000 (16:37 +0800)]
do_cmake: add a default value to BOOST_J

Signed-off-by: luo rixin <luorixin@huawei.com>
4 years agomgr/dashboard:minimize console log traces of Dashboard backend API tests 36088/head
Aashish Sharma [Tue, 14 Jul 2020 04:58:57 +0000 (10:28 +0530)]
mgr/dashboard:minimize console log traces of Dashboard backend API tests

Changed log levels in some files to reduce console log traces

Fixes: https://tracker.ceph.com/issues/45822
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
4 years agoqa/tests: add light octopus-x/copy (cephadm based)
Yuri Weinstein [Fri, 18 Dec 2020 19:30:29 +0000 (11:30 -0800)]
qa/tests: add light octopus-x/copy (cephadm based)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agoMerge PR #37721 into master
Patrick Donnelly [Tue, 22 Dec 2020 20:58:11 +0000 (12:58 -0800)]
Merge PR #37721 into master

* refs/pull/37721/head:
test: add tests for new snapshot APIs
pybind/cephfs: python bindings for new snapshot APIs
client: Snapshot cephfs APIs
mds: include snapshot metadata in trace reply to client
mds: include snap metadata on-disk on snapshot creation
mds: include metadata in SnapInfo structure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #38468 from IlsooByun/fix_lc_mp_iter
Harish Munjulur [Tue, 22 Dec 2020 19:15:05 +0000 (11:15 -0800)]
Merge pull request #38468 from IlsooByun/fix_lc_mp_iter

rgw: lc: fix multipart iteration problem

4 years agoMerge PR #38611 into master
Patrick Donnelly [Tue, 22 Dec 2020 18:25:54 +0000 (10:25 -0800)]
Merge PR #38611 into master

* refs/pull/38611/head:
qa/cephfs/nfs: Add test for readonly export

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agopybind/cephfs: pass command as size-1 array of strings 38692/head
Patrick Donnelly [Tue, 22 Dec 2020 17:28:58 +0000 (09:28 -0800)]
pybind/cephfs: pass command as size-1 array of strings

Fixes: https://tracker.ceph.com/issues/48701
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoos/bluestore: introduce bluestore_rocksdb_options_annex config parameter. 38680/head
Igor Fedotov [Mon, 21 Dec 2020 21:01:52 +0000 (00:01 +0300)]
os/bluestore: introduce bluestore_rocksdb_options_annex config parameter.

This simplifies modifying a subset of rocksdb settings since it
eliminates the need to re-provide bluestore's rocksdb settings.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #38187 from zdover23/wip-doc-dev-unit-tests-5-of-7
zdover23 [Tue, 22 Dec 2020 16:07:59 +0000 (02:07 +1000)]
Merge pull request #38187 from zdover23/wip-doc-dev-unit-tests-5-of-7

doc/dev: update unit-tests tox section

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #38414 from pcuzner/fix-drivegroup-example
Sebastian Wagner [Tue, 22 Dec 2020 12:30:54 +0000 (13:30 +0100)]
Merge pull request #38414 from pcuzner/fix-drivegroup-example

doc: Update drivegroup spec examples

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agodoc/dev: update unit-tests tox section 38187/head
Zac Dover [Thu, 19 Nov 2020 14:18:51 +0000 (00:18 +1000)]
doc/dev: update unit-tests tox section

This commit
- alters the structure of the
  unit-test section of the Developer Guide
  that discusses the tox tool, which is
  used to run certain unit tests
- corrects some grammar
- updates the list of locations of tox.ini
  files
- This also incorporates N. Cutler's suggestions.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #38621 from anthonyeleven/anthonyeleven/bluestore-doc-cleanup
zdover23 [Tue, 22 Dec 2020 12:05:17 +0000 (22:05 +1000)]
Merge pull request #38621 from anthonyeleven/anthonyeleven/bluestore-doc-cleanup

doc/rados/configuration: clean up BlueStore config and enhance DB sizing

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoqa/cephfs/nfs: Add test for readonly export 38611/head
Varsha Rao [Tue, 15 Dec 2020 10:53:03 +0000 (10:53 +0000)]
qa/cephfs/nfs: Add test for readonly export

Fixes: https://tracker.ceph.com/issues/48622
Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agoMerge pull request #38499 from aclamk/wip-bs-onode-prune-deleted
Igor Fedotov [Tue, 22 Dec 2020 08:42:38 +0000 (11:42 +0300)]
Merge pull request #38499 from aclamk/wip-bs-onode-prune-deleted

os/bluestore: Prune deleted onodes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #38650 from dmick/wip-dmick-docker-mirror
Sebastian Wagner [Tue, 22 Dec 2020 00:08:18 +0000 (01:08 +0100)]
Merge pull request #38650 from dmick/wip-dmick-docker-mirror

qa/distros: configure podman to pull from docker-mirror

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoinclude/denc: use pair<const K,V> in range-based for loop 38678/head
Kefu Chai [Mon, 21 Dec 2020 17:07:37 +0000 (01:07 +0800)]
include/denc: use pair<const K,V> in range-based for loop

map<K,V>::value_type is pair<const K, V>, so if we use range-based for
loop when iterating through a map, we should use pair<const K,V> instead
of pair<K,V>, the latter also compiles, but it might create a temporary
object of pair<K,V> from pair<const K,V>. GCC-11 complains at seeing
this:

../src/include/denc.h:1002:21: warning: loop variable ‘e’ of type ‘const T&’ {aka ‘const std::pair<OSDPerfMetricQuery, OSDPerfMetricReport>&’} binds to a tem\
porary constructed from type ‘const std::pair<const OSDPerfMetricQuery, OSDPerfMetricReport>’ [-Wrange-loop-constru
ct]
 1002 |       for (const T& e : s) {
      |                     ^

this change

* use the value_type of container in `maplike_details<Container>`,
  so we can avoid the overhead of creating temporay objects when
  encoding a map
* define denc_traits for std::pair<const A, B> as well, so the elements
  of a map can be encoded using denc facility

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #38666 into master
Patrick Donnelly [Mon, 21 Dec 2020 17:06:00 +0000 (09:06 -0800)]
Merge PR #38666 into master

* refs/pull/38666/head:
doc: Fix mds capabilities for manual deployment

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>