]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agodoc: add RBD persistent write-back cache 38520/head
lixiaoy1 [Tue, 17 Nov 2020 14:47:42 +0000 (09:47 -0500)]
doc: add RBD persistent write-back cache

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
4 years agodoc: rename RBD persistent cache
lixiaoy1 [Wed, 16 Dec 2020 09:47:51 +0000 (04:47 -0500)]
doc: rename RBD persistent cache

Rename peristent cache to persistent read-only cache.

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
4 years agoMerge pull request #36626 from lixiaoy1/rbd_image_discard
Jason Dillaman [Sun, 1 Nov 2020 13:42:01 +0000 (08:42 -0500)]
Merge pull request #36626 from lixiaoy1/rbd_image_discard

librbd/rwl: discard cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #37870 from xxhdx1985126/wip-crimson-recovery-bug-fixes
Kefu Chai [Sun, 1 Nov 2020 13:23:59 +0000 (21:23 +0800)]
Merge pull request #37870 from xxhdx1985126/wip-crimson-recovery-bug-fixes

crimson: recovery-related bug fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37910 from tchaikov/wip-crimson-osd-send-pg-temp
Kefu Chai [Sun, 1 Nov 2020 13:21:42 +0000 (21:21 +0800)]
Merge pull request #37910 from tchaikov/wip-crimson-osd-send-pg-temp

crimson/osd: merge pg_temp_wanted before sending them

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #37656 from yuvalif/allow_lua_to_set_metadata
Yuval Lifshitz [Sun, 1 Nov 2020 10:03:26 +0000 (12:03 +0200)]
Merge pull request #37656 from yuvalif/allow_lua_to_set_metadata

rgw/lua: allow setting metadata via lua

4 years agocrimson/osd: merge pg_temp_wanted before sending them 37910/head
Kefu Chai [Sun, 1 Nov 2020 03:23:13 +0000 (11:23 +0800)]
crimson/osd: merge pg_temp_wanted before sending them

there is chance that new pg_temp_wanted is added when we are sending
them in ShardServices::send_pg_temp(), but the pg temp are already
collected into the messages before seastar::parallel_for_each(). so,
we cannot move all elements from pg_temp_wanted to pg_temp_pending after
sending the messages, that'd also move the unsent pg temps to
pg_temp_pending as well. so we have to do this before do the async call,

this issue was root caused by Xuehan Xu <xxhdx1985126@gmail.com>.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: drop ShardServices::_sent_pg_temp()
Kefu Chai [Sun, 1 Nov 2020 03:14:48 +0000 (11:14 +0800)]
crimson/osd: drop ShardServices::_sent_pg_temp()

there is no need to define a wrapper for moving pg_temp_wanted to
pg_temp_pending. and it hurts the readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: report statfs data 37870/head
Xuehan Xu [Sun, 1 Nov 2020 03:30:38 +0000 (11:30 +0800)]
crimson/osd: report statfs data

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/osd: rename heartbeat_timer to tick_timer
Xuehan Xu [Sun, 1 Nov 2020 03:29:57 +0000 (11:29 +0800)]
crimson/osd: rename heartbeat_timer to tick_timer

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/common: make Gated handle gate_close_exception
Xuehan Xu [Sun, 1 Nov 2020 03:27:52 +0000 (11:27 +0800)]
crimson/common: make Gated handle gate_close_exception

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/osd: do not check HAVE_STDLIB_MAP_SPLICING
Kefu Chai [Sun, 1 Nov 2020 03:08:24 +0000 (11:08 +0800)]
crimson/osd: do not check HAVE_STDLIB_MAP_SPLICING

seastar always uses GCC-9 and up, so there is no need to check for
the existence of map::merge() before using it anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: clean up pending_txn when actingset is changed
Xuehan Xu [Tue, 20 Oct 2020 06:52:41 +0000 (14:52 +0800)]
crimson/osd: clean up pending_txn when actingset is changed

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/osd: fix pg info statistics error during recovery
Xuehan Xu [Tue, 20 Oct 2020 06:51:30 +0000 (14:51 +0800)]
crimson/osd: fix pg info statistics error during recovery

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/osd: make I/O reqs wait for object recovery if it's degraded or backfilling
Xuehan Xu [Tue, 20 Oct 2020 10:21:04 +0000 (18:21 +0800)]
crimson/osd: make I/O reqs wait for object recovery if it's degraded or backfilling

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #37327 from kamoltat/wip-mgr-progress-global-event 37427/head
Kefu Chai [Sat, 31 Oct 2020 03:43:27 +0000 (11:43 +0800)]
Merge pull request #37327 from kamoltat/wip-mgr-progress-global-event

mgr/progress: Global Recovery Event in ceph -s

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #37598 from ronen-fr/scrubstore-fix-rf
Kefu Chai [Sat, 31 Oct 2020 03:42:35 +0000 (11:42 +0800)]
Merge pull request #37598 from ronen-fr/scrubstore-fix-rf

osd: fix signatures of get_store_errors() and friends

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37751 from changchengx/nasm_avx512
Kefu Chai [Sat, 31 Oct 2020 03:41:19 +0000 (11:41 +0800)]
Merge pull request #37751 from changchengx/nasm_avx512

common: use nasm to build isa-l and isa-l_crypto to prepare for AVX512

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37773 from sunnyku/wip-f33-pyfix
Josh Durgin [Fri, 30 Oct 2020 15:15:59 +0000 (08:15 -0700)]
Merge pull request #37773 from sunnyku/wip-f33-pyfix

do_cmake.sh: use python-3.9 with fedora version 33

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #37789 from trociny/wip-47827
Jason Dillaman [Fri, 30 Oct 2020 13:40:10 +0000 (09:40 -0400)]
Merge pull request #37789 from trociny/wip-47827

mgr/rbd_support: create mirror snapshots asynchronously

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agorgw/lua: use lua exceptions instead of ceph_asserts 37656/head
Yuval Lifshitz [Wed, 28 Oct 2020 16:10:20 +0000 (18:10 +0200)]
rgw/lua: use lua exceptions instead of ceph_asserts

also use lua assert() in uni tests instead of print()
fix bug when accessing user id in an ACL grant

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
4 years agoMerge pull request #37482 from rhcs-dashboard/datatable_click_prevent
Lenz Grimmer [Fri, 30 Oct 2020 08:55:47 +0000 (09:55 +0100)]
Merge pull request #37482 from rhcs-dashboard/datatable_click_prevent

mgr/dashboard: Prevent table items from getting selected while expanding

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #35769 from Codom/master
Lenz Grimmer [Fri, 30 Oct 2020 08:50:05 +0000 (09:50 +0100)]
Merge pull request #35769 from Codom/master

mgr/dashboard: Added Versioning to the REST API

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agorgw/lua: allow setting metadata via lua
Yuval Lifshitz [Tue, 13 Oct 2020 16:54:18 +0000 (19:54 +0300)]
rgw/lua: allow setting metadata via lua

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
4 years agoMerge pull request #37788 from tchaikov/wip-zbd
Kefu Chai [Fri, 30 Oct 2020 03:17:56 +0000 (11:17 +0800)]
Merge pull request #37788 from tchaikov/wip-zbd

rpm,cmake: s/WITH_LIBZBD/WITH_ZBD/ and enable ZBD on demand

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37888 from tchaikov/wip-crimson-gtest
Kefu Chai [Fri, 30 Oct 2020 02:25:49 +0000 (10:25 +0800)]
Merge pull request #37888 from tchaikov/wip-crimson-gtest

crimson/gtest_seastar: do not keep a copy of argv

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
4 years agoMerge pull request #37891 from tchaikov/wip-cmake-alienstore
Kefu Chai [Fri, 30 Oct 2020 02:25:06 +0000 (10:25 +0800)]
Merge pull request #37891 from tchaikov/wip-cmake-alienstore

cmake: build bluestore-tp before crimson-alienstore

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
4 years agoMerge pull request #37893 from tchaikov/wip-wunused-variable
Kefu Chai [Fri, 30 Oct 2020 02:24:11 +0000 (10:24 +0800)]
Merge pull request #37893 from tchaikov/wip-wunused-variable

osd/osd_types: mark unused variable [[maybe_unused]]

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37864 from trociny/wip-48020
Jason Dillaman [Fri, 30 Oct 2020 00:58:44 +0000 (20:58 -0400)]
Merge pull request #37864 from trociny/wip-48020

mgr/rbd_support: store global schedule without localized prefix

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #37880 from trociny/wip-rbd-finisher-cancel
Jason Dillaman [Fri, 30 Oct 2020 00:58:13 +0000 (20:58 -0400)]
Merge pull request #37880 from trociny/wip-rbd-finisher-cancel

librbd: relax requirements on finisher canceled callback

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agomgr/rbd_support: create mirror snapshots asynchronously 37789/head
Mykola Golub [Wed, 21 Oct 2020 17:07:16 +0000 (18:07 +0100)]
mgr/rbd_support: create mirror snapshots asynchronously

To scale up with number of images.

Fixes: https://tracker.ceph.com/issues/47827
Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agopybind/rbd: add async mirror image get mode and info methods
Mykola Golub [Mon, 26 Oct 2020 11:09:28 +0000 (11:09 +0000)]
pybind/rbd: add async mirror image get mode and info methods

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agolibrbd: add missing declaration of rbd_aio_mirror_image_get_mode
Mykola Golub [Mon, 26 Oct 2020 11:06:09 +0000 (11:06 +0000)]
librbd: add missing declaration of rbd_aio_mirror_image_get_mode

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agopybind/rbd: add image async open and close methods
Mykola Golub [Fri, 23 Oct 2020 10:38:21 +0000 (11:38 +0100)]
pybind/rbd: add image async open and close methods

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #37886 from tchaikov/wip-osd-scheduler-move
Samuel Just [Thu, 29 Oct 2020 17:56:44 +0000 (10:56 -0700)]
Merge pull request #37886 from tchaikov/wip-osd-scheduler-move

osd/scheduler: rely on copy ellision to move return val

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
4 years agoMerge pull request #34866 from nickjanus/issue-45193
Casey Bodley [Thu, 29 Oct 2020 16:00:52 +0000 (12:00 -0400)]
Merge pull request #34866 from nickjanus/issue-45193

rgw-admin: fixes BucketInfo for missing buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37657 from batrick/rgw-lua-fixes
Yuval Lifshitz [Thu, 29 Oct 2020 15:56:46 +0000 (17:56 +0200)]
Merge pull request #37657 from batrick/rgw-lua-fixes

rgw: lua refactor

4 years agorbd/tool: add image-cache invalidate 36626/head
lixiaoy1 [Thu, 20 Aug 2020 16:23:44 +0000 (12:23 -0400)]
rbd/tool: add image-cache invalidate

Add an rbd command image-cache invalidate to discard RWL cache.

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
4 years agolibrbd/internal: update invalidate_cache
lixiaoy1 [Fri, 11 Sep 2020 14:26:39 +0000 (10:26 -0400)]
librbd/internal: update invalidate_cache

Except invalidating object cache, invalidate image writeback cache.

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
4 years agolibrbd/cache: discard writeback cache
lixiaoy1 [Fri, 11 Sep 2020 14:24:56 +0000 (10:24 -0400)]
librbd/cache: discard writeback cache

Add an interface in ImageCache to discard writeback cache.

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
4 years agopybind/rbd: async API for creating mirror snapshots
Mykola Golub [Wed, 21 Oct 2020 12:38:31 +0000 (13:38 +0100)]
pybind/rbd: async API for creating mirror snapshots

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agolibrbd: async API for creating mirror snapshots
Mykola Golub [Mon, 12 Oct 2020 09:25:46 +0000 (10:25 +0100)]
librbd: async API for creating mirror snapshots

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #37884 from dillaman/wip-48032
Mykola Golub [Thu, 29 Oct 2020 13:48:35 +0000 (15:48 +0200)]
Merge pull request #37884 from dillaman/wip-48032

qa/workunits/rbd: fix permission issue when removing mirror peer

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #37890 from tchaikov/wip-crimson-os-cleanup
Kefu Chai [Thu, 29 Oct 2020 11:59:08 +0000 (19:59 +0800)]
Merge pull request #37890 from tchaikov/wip-crimson-os-cleanup

crimson/os: fix couple issues reported by clang

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
4 years agoMerge pull request #36739 from pritha-srivastava/wip-rgw-sts-expire-session-token
Matt Benjamin [Thu, 29 Oct 2020 11:58:10 +0000 (07:58 -0400)]
Merge pull request #36739 from pritha-srivastava/wip-rgw-sts-expire-session-token

rgw/sts: adding code for "aws:TokenIssueTime" to be used

4 years agoMerge pull request #35612 from pritha-srivastava/wip-sts-role-min-duration
Matt Benjamin [Thu, 29 Oct 2020 11:57:50 +0000 (07:57 -0400)]
Merge pull request #35612 from pritha-srivastava/wip-sts-role-min-duration

rgw/sts: Adding a configurable rgw_sts_min_session_duration

4 years agoosd/scheduler: rely on copy ellision to move return val 37886/head
Kefu Chai [Mon, 26 Oct 2020 04:49:20 +0000 (12:49 +0800)]
osd/scheduler: rely on copy ellision to move return val

C++14 enforces copy ellision in this case.

also silences warning of

src/osd/scheduler/mClockScheduler.cc:122:21: warning: redundant move in return statement [-Wredundant-move]
  122 |     return std::move(ret);
      |            ~~~~~~~~~^~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoosd/osd_types: mark unused variable [[maybe_unused]] 37893/head
Kefu Chai [Thu, 29 Oct 2020 10:35:24 +0000 (18:35 +0800)]
osd/osd_types: mark unused variable [[maybe_unused]]

silences warning like:

rc/osd/osd_types.h:5482:18: warning: unused variable '_' [-Wunused-variable]
     auto [iter, _] = ref_delta.try_emplace(hoid, 0);
                  ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: build bluestore-tp before crimson-alienstore 37891/head
Kefu Chai [Thu, 29 Oct 2020 09:18:06 +0000 (17:18 +0800)]
cmake: build bluestore-tp before crimson-alienstore

to address the FTBFS of

src/os/bluestore/BlueStore.cc:52:10: fatal error: tracing/bluestore.h: No such file or directory
   52 | #include "tracing/bluestore.h"
      |          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.

tracing/bluestore.h is created by the "bluestore-tp" target, there
is a good chance that we build crimson when WITH_LTTNG is enabled,
let's ensure that tracing/bluestore.h is ready before compiling
alienstore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: capture local bindings using capture variable list 37890/head
Kefu Chai [Thu, 29 Oct 2020 08:24:45 +0000 (16:24 +0800)]
crimson/os: capture local bindings using capture variable list

structured binding does not define variables, and the implicit
capture-by-copy `=` does not help in this case, we have to define a
capture with an initializer to create a *variable* enclosed by
the closure explicitly.

this address the error like:

src/crimson/os/seastore/segment_cleaner.cc:315:5: error: reference to local binding 'addr' declared in enclosing lambda expression
                  addr);
                  ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: do not move temporay variable
Kefu Chai [Thu, 29 Oct 2020 08:21:06 +0000 (16:21 +0800)]
crimson/os: do not move temporay variable

copy ellision ensures that copy ctor is not called in this case

silences warning like:

mson/os/seastore/lba_manager/btree/lba_btree_node_impl.cc:262:5: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
    std::move(begin()),
    ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: do not capture unused variable
Kefu Chai [Thu, 29 Oct 2020 08:20:12 +0000 (16:20 +0800)]
crimson/os: do not capture unused variable

silences warning like:

btree/extentmap_btree_node.h:100:8: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
      [this, ec, len] (auto &extents) {
       ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37887 from tchaikov/wip-seastar
Kefu Chai [Thu, 29 Oct 2020 07:21:41 +0000 (15:21 +0800)]
Merge pull request #37887 from tchaikov/wip-seastar

seastar: pick up changes to colocate libc allocator and seastar alloc…

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/gtest_seastar: consume argv before passing them to seastar 37888/head
Kefu Chai [Thu, 29 Oct 2020 06:59:21 +0000 (14:59 +0800)]
crimson/gtest_seastar: consume argv before passing them to seastar

so we can pass gtest options to the test without annoying seastar app.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/gtest_seastar: do not keep a copy of argv
Kefu Chai [Thu, 29 Oct 2020 05:03:40 +0000 (13:03 +0800)]
crimson/gtest_seastar: do not keep a copy of argv

argv is always available in the whole life cycle of the application, so
there is no need to keep a copy of it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoseastar: pick up changes to colocate libc allocator and seastar allocator 37887/head
Kefu Chai [Wed, 28 Oct 2020 09:27:00 +0000 (17:27 +0800)]
seastar: pick up changes to colocate libc allocator and seastar allocator

this allows us to use seastar's builtin allocator along with the libc
allocator used by alien threads.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: remove yasm since it's been replaced by nasm 37751/head
Changcheng Liu [Wed, 28 Oct 2020 07:26:41 +0000 (15:26 +0800)]
cmake: remove yasm since it's been replaced by nasm

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocmake: use nasm to replace yasm to build isa-l
Changcheng Liu [Wed, 28 Oct 2020 07:11:10 +0000 (15:11 +0800)]
cmake: use nasm to replace yasm to build isa-l

1. use nasm to replace yasm to build isa-l
2. use the defined MACRO for nasm to replace all the defined MACRO for
   yasm

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agoinstall-deps.sh: install nasm for isa-l library
Changcheng Liu [Thu, 22 Oct 2020 06:41:34 +0000 (14:41 +0800)]
install-deps.sh: install nasm for isa-l library

nasm support build isa-l:AVX512 algorithm implementation while yasm
doens't support it. Install nasm assembler to build isa-l
refer to: https://github.com/yasm/yasm/issues/101

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocommon: use isa-l's crc32 to implement fast crc32 on IA arch
Changcheng Liu [Wed, 28 Oct 2020 06:46:21 +0000 (14:46 +0800)]
common: use isa-l's crc32 to implement fast crc32 on IA arch

1. Accoring to the commit history, crc32c_intel_fast_asm.s is copied from
isa-l's crc32_iscsi_00.asm. Let's use isa-l's crc32 to replace the old
one.
2. For nasm, it doesn't recongized "rip". Change all the address to be
rip relative by adding "default rel".

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocmake: wrapper nasm as assembler to deal with some options
Changcheng Liu [Thu, 22 Oct 2020 05:05:17 +0000 (13:05 +0800)]
cmake: wrapper nasm as assembler to deal with some options

1. unify some options, such as "-I -isystem"
2. ignore some known options, such as "-W*"
3. ignore some unknown options, such as '-fPIC"

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocmake: check nasm and define related MACRO
Changcheng Liu [Thu, 22 Oct 2020 05:05:17 +0000 (13:05 +0800)]
cmake: check nasm and define related MACRO

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocmake: define cmake macro to check nasm assembler
Changcheng Liu [Thu, 22 Oct 2020 05:11:05 +0000 (13:11 +0800)]
cmake: define cmake macro to check nasm assembler

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocmake: set HAVE_LIBZBD before creating "acconfig.h" 37788/head
Kefu Chai [Thu, 29 Oct 2020 03:17:32 +0000 (11:17 +0800)]
cmake: set HAVE_LIBZBD before creating "acconfig.h"

`acconfig.h` is generated using

configure_file(
  ${CMAKE_SOURCE_DIR}/src/include/config-h.in.cmake
  ${CMAKE_BINARY_DIR}/include/acconfig.h
)

in `config-h.in.cmake`, the cmake variable of `HAVE_LIBZBD` is checked.
so we need to ensure that this variable is visible from this
`configure_file()` statement.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agorpm: pass -DWITH_ZBD=ON to cmake if zbd is enabled
Kefu Chai [Mon, 26 Oct 2020 05:49:17 +0000 (13:49 +0800)]
rpm: pass -DWITH_ZBD=ON to cmake if zbd is enabled

this is a leftover of fd1b02a341e3afba32776e35ec7c4bbe883f9b06

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: s/WITH_LIBZBD/WITH_LIBZBD/
Kefu Chai [Mon, 26 Oct 2020 05:47:01 +0000 (13:47 +0800)]
cmake: s/WITH_LIBZBD/WITH_LIBZBD/

fix the regression introduced by
d53638630631cc6596a1238228332e7579318415

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37777 from chenerqi/fix-modified-dentries-count
Yan, Zheng [Thu, 29 Oct 2020 03:08:10 +0000 (11:08 +0800)]
Merge pull request #37777 from chenerqi/fix-modified-dentries-count

mds: fix count error of modified dentries

4 years agocmake: extract common in one place for further check
Changcheng Liu [Thu, 22 Oct 2020 04:51:39 +0000 (12:51 +0800)]
cmake: extract common in one place for further check

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocmake: check whether yasm support AVX512
Changcheng Liu [Wed, 28 Oct 2020 01:38:29 +0000 (09:38 +0800)]
cmake: check whether yasm support AVX512

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocommon: rename YASM MACRO to hint support SIMD
Changcheng Liu [Thu, 22 Oct 2020 02:42:29 +0000 (10:42 +0800)]
common: rename YASM MACRO to hint support SIMD

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocommon: rename variable name to improve readability
Changcheng Liu [Wed, 28 Oct 2020 00:59:09 +0000 (08:59 +0800)]
common: rename variable name to improve readability

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocmake: refine yasm support x86_64 and AVX2 info
Changcheng Liu [Wed, 28 Oct 2020 00:54:32 +0000 (08:54 +0800)]
cmake: refine yasm support x86_64 and AVX2 info

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agocmake: more quiet when checking for yasm
Changcheng Liu [Thu, 22 Oct 2020 01:32:32 +0000 (09:32 +0800)]
cmake: more quiet when checking for yasm

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agoyasm-wrapper: refine yasm option usage strategy in yasm-wrapper
Changcheng Liu [Wed, 21 Oct 2020 07:03:15 +0000 (15:03 +0800)]
yasm-wrapper: refine yasm option usage strategy in yasm-wrapper

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agoyasm-wrapper: remove redundant option
Changcheng Liu [Wed, 21 Oct 2020 09:06:58 +0000 (17:06 +0800)]
yasm-wrapper: remove redundant option

"-fPIC" is covered by "-f"

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agoyasm-wrapper: remove not existed options
Changcheng Liu [Wed, 21 Oct 2020 06:41:13 +0000 (14:41 +0800)]
yasm-wrapper: remove not existed options

commit "0cf627b4" has removed lowmem_builder condition.
ggc-min* not exist now.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agoMerge pull request #37636 from MahatiC/wip-init-ssd
Jason Dillaman [Wed, 28 Oct 2020 23:03:05 +0000 (19:03 -0400)]
Merge pull request #37636 from MahatiC/wip-init-ssd

librbd/cache: init functionality for SSD Cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #37680 from orozery/object-dispatch-aligned-crypto
Jason Dillaman [Wed, 28 Oct 2020 23:02:47 +0000 (19:02 -0400)]
Merge pull request #37680 from orozery/object-dispatch-aligned-crypto

librbd: crypto alignment support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #37871 from cyx1231st/wip-seastore-misc-fixes
Samuel Just [Wed, 28 Oct 2020 20:16:55 +0000 (13:16 -0700)]
Merge pull request #37871 from cyx1231st/wip-seastore-misc-fixes

crimson/seastore: misc fixes in transaction manager

crimson/seastore: misc fixes in transaction manager

4 years agoMerge pull request #37877 from dang/wip-dang-cond-writeback
Daniel Gryniewicz [Wed, 28 Oct 2020 18:27:06 +0000 (14:27 -0400)]
Merge pull request #37877 from dang/wip-dang-cond-writeback

rgw: Fix broken merge in #37774

4 years agoqa/workunits/rbd: fix permission issue when removing mirror peer 37884/head
Jason Dillaman [Wed, 28 Oct 2020 18:06:57 +0000 (14:06 -0400)]
qa/workunits/rbd: fix permission issue when removing mirror peer

Fixes: https://tracker.ceph.com/issues/48032
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agoRGW - Fix broken merge in #37774 37877/head
Daniel Gryniewicz [Wed, 28 Oct 2020 13:01:06 +0000 (09:01 -0400)]
RGW - Fix broken merge in #37774

I messed up the merge in #37774 and somehow forgot to push a change.
This resulted it the flag being set on the wrong branch.  This fixes
that.

RHBZ#1845501

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #37865 from neha-ojha/wip-mon-fixes
Kefu Chai [Wed, 28 Oct 2020 15:45:38 +0000 (23:45 +0800)]
Merge pull request #37865 from neha-ojha/wip-mon-fixes

qa: miscellaneous mon test fixes

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoosd: fix signatures of get_store_errors() and friends 37598/head
Ronen Friedman [Thu, 8 Oct 2020 12:35:35 +0000 (15:35 +0300)]
osd: fix signatures of get_store_errors() and friends

The 'store' parameter was ignored by ScrubStore-related get_*_errors()
functions, and is now removed.

The functions are now marked 'const'. That required tagging the caching
member Store::backend as 'mutable'. While 'mutable' is an 'eyesore',
here is one of the rare cases where its use is justified. Following
https://isocpp.org/wiki/faq/const-correctness:
"When methods change the physical but not logical state, the method should
generally be marked as const since it really is an inspector-method."
(The text then continues and specifically prescribes 'mutable' for these situations.)

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
4 years agoMerge pull request #36764 from ishanrai05/guide
Lenz Grimmer [Wed, 28 Oct 2020 14:47:32 +0000 (15:47 +0100)]
Merge pull request #36764 from ishanrai05/guide

mgr/dashboard: Style guide to give a the UI an overall look and feel

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agomgr/dashboard: Added Versioning to the REST API 35769/head
Christopher Odom [Mon, 22 Jun 2020 15:23:16 +0000 (11:23 -0400)]
mgr/dashboard: Added Versioning to the REST API

Versioning is handled by the RESTContoller decorators.
It works by adding a version attribute to the endpoint object,
which will be checked by the _request_wrapper against the requested
version before the controller method is dispatched.

This commit also updates all of the testing to support
version vendor mime types, as well as adding an http
interceptor to add versioned mime types to all frontend
requests.

Fixes: https://tracker.ceph.com/issues/40909
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
4 years agolibrbd: complete with -ECANCELED queued task requests instead of deleting 37880/head
Mykola Golub [Wed, 28 Oct 2020 13:36:09 +0000 (13:36 +0000)]
librbd: complete with -ECANCELED queued task requests instead of deleting

This is cleanup just to make it consistent with other cases.

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agolibrbd: relax requirements on finisher canceled callback
Mykola Golub [Wed, 28 Oct 2020 13:13:02 +0000 (13:13 +0000)]
librbd: relax requirements on finisher canceled callback

The finisher timer is started with safe_callbacks = false, and
cancel_event may fail.

When canceling a task it is safe to just ignore the cancel_event
result and proceed, because the returned false value means the
callback is in TaskFinisher::complete already but before
acquiring the lock, so when it eventually acquires the lock it
will just find out the task is already deleted and return.

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agomgr/rbd_support: backward compatibility when loading global schedule 37864/head
Mykola Golub [Wed, 28 Oct 2020 09:20:27 +0000 (09:20 +0000)]
mgr/rbd_support: backward compatibility when loading global schedule

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agomgr/rbd_support: store global schedule without localized prefix
Mykola Golub [Tue, 27 Oct 2020 18:00:52 +0000 (18:00 +0000)]
mgr/rbd_support: store global schedule without localized prefix

so it is still used after mgr failover

Fixes: https://tracker.ceph.com/issues/48020
Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge PR #37808 into master
Jan Fajerski [Wed, 28 Oct 2020 08:21:06 +0000 (09:21 +0100)]
Merge PR #37808 into master

* refs/pull/37808/head:
ceph-volume: consume mount opt in simple activate

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
4 years agocrimson/os/seastore/transaction_manager: add missing final keyword 37871/head
Yingxin Cheng [Wed, 28 Oct 2020 07:51:03 +0000 (15:51 +0800)]
crimson/os/seastore/transaction_manager: add missing final keyword

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/os/seastore/transaction_manager: use weak transaction when initialize cached...
Yingxin Cheng [Wed, 28 Oct 2020 07:45:20 +0000 (15:45 +0800)]
crimson/os/seastore/transaction_manager: use weak transaction when initialize cached extents

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #37761 from tchaikov/wip-ceph.spec-with-seastar
Kefu Chai [Wed, 28 Oct 2020 06:43:49 +0000 (14:43 +0800)]
Merge pull request #37761 from tchaikov/wip-ceph.spec-with-seastar

ceph.spec.in: install gcc-toolset on RHEL only

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agocrimson/osd: adjust peer info before mutating object
Xuehan Xu [Tue, 20 Oct 2020 10:20:43 +0000 (18:20 +0800)]
crimson/osd: adjust peer info before mutating object

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agolibrbd/cache: init functionality for SSD Cache 37636/head
Mahati Chamarthy [Thu, 8 Oct 2020 10:55:00 +0000 (16:25 +0530)]
librbd/cache: init functionality for SSD Cache

Adds build option and implements init functionality for SSD cache

Signed-off-by: Lisa Li <xiaoyan.li@intel.com>
Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
4 years agomds: fix count error of modified dentries 37777/head
Erqi Chen [Sat, 24 Oct 2020 06:50:49 +0000 (14:50 +0800)]
mds: fix count error of modified dentries

CDir::_omap_commit counts modified dentries, for fragmenting
new dir modified dentries include head and snap items.

Fixes: https://tracker.ceph.com/issues/47981
Signed-off-by: Erqi Chen <chenerqi@gmail.com>
4 years agoMerge pull request #37826 from badone/wip-make-dist-ng-cli-noprompt
Brad Hubbard [Tue, 27 Oct 2020 21:32:41 +0000 (07:32 +1000)]
Merge pull request #37826 from badone/wip-make-dist-ng-cli-noprompt

make-dist: Don't prompt while running 'npm ci'

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoqa/tasks/ceph_manager.py: remove redundant quorum status logging 37865/head
Neha Ojha [Tue, 27 Oct 2020 20:55:28 +0000 (20:55 +0000)]
qa/tasks/ceph_manager.py: remove redundant quorum status logging

2020-10-21T03:42:45.985 INFO:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status
2020-10-21T03:42:58.574 INFO:teuthology.orchestra.run.smithi114.stdout:{"election_epoch":1650,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"807c36f1-9e85-4fa3-81fc-95915ab50584","modified":"2020-10-21T00:34:48.421341Z","created":"2020-10-21T00:34:48.421341Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":3,"disallowed_leaders":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.114:6790","nonce":0}]},"addr":"172.21.15.114:6790/0","public_addr":"172.21.15.114:6790/0","priority":0,"weight":0}]}}
2020-10-21T03:42:58.589 INFO:tasks.mon_thrash.ceph_manager:quorum_status is {"election_epoch":1650,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"807c36f1-9e85-4fa3-81fc-95915ab50584","modified":"2020-10-21T00:34:48.421341Z","created":"2020-10-21T00:34:48.421341Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":3,"disallowed_leaders":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.114:6790","nonce":0}]},"addr":"172.21.15.114:6790/0","public_addr":"172.21.15.114:6790/0","priority":0,"weight":0}]}}

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoqa/suites/rados/dashboard: ignore MON_DOWN
Neha Ojha [Tue, 27 Oct 2020 20:44:15 +0000 (20:44 +0000)]
qa/suites/rados/dashboard: ignore MON_DOWN

This warning is expected during:

2020-10-27T16:29:40.549+0000 7fa1f0927700 10 mon.c@2(peon) e1 handle_auth_request haven't formed initial quorum, EBUSY

and is cleared after:

2020-10-27T16:29:42.597+0000 7fe7b3ada700  7 mon.a@1(peon).log v7 update_from_paxos applying incremental log 7 2020-10-27T16:29:42.319274+0000 mon.b (mon.0) 76 : cluster [INF] mon.b is new leader, mons b,a,c in quorum (ranks 0,1,2)
2020-10-27T16:29:42.597+0000 7fe7b3ada700  7 mon.a@1(peon).log v7 update_from_paxos applying incremental log 7 2020-10-27T16:29:42.326711+0000 mon.b (mon.0) 82 : cluster [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum b,a)

Signed-off-by: Neha Ojha <nojha@redhat.com>