]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agodocs/start: fixes typo and empty headline in hardware recommendation docs 44433/head
Sebastian Schmid [Sat, 1 Jan 2022 16:33:10 +0000 (17:33 +0100)]
docs/start: fixes typo and empty headline in hardware recommendation docs

Signed-off-by: Sebastian Schmid <mail@sebastian-schmid.de>
3 years agoMerge pull request #44398 from rhcs-dashboard/e2e-script-improvement
Nizamudeen A [Thu, 30 Dec 2021 05:55:58 +0000 (11:25 +0530)]
Merge pull request #44398 from rhcs-dashboard/e2e-script-improvement

mgr/dashboard: dashboard cephadm e2e improvement

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
3 years agomgr/dashboard: dashboard cephadm e2e improvement 44398/head
Nizamudeen A [Fri, 24 Dec 2021 05:39:37 +0000 (11:09 +0530)]
mgr/dashboard: dashboard cephadm e2e improvement

Fixes: https://tracker.ceph.com/issues/53742
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #44403 from cyx1231st/wip-fix-crimson-args
Yingxin [Wed, 29 Dec 2021 05:01:50 +0000 (13:01 +0800)]
Merge pull request #44403 from cyx1231st/wip-fix-crimson-args

crimson/osd: fix and improve args processing

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44396 from cyx1231st/wip-seastore-fix-seastar-runner
Kefu Chai [Tue, 28 Dec 2021 16:31:03 +0000 (00:31 +0800)]
Merge pull request #44396 from cyx1231st/wip-seastore-fix-seastar-runner

crimson/test: fix SeastarRunner when app is not started

Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44274 from fengchunsong/dpdk-dfx
Kefu Chai [Tue, 28 Dec 2021 11:55:41 +0000 (19:55 +0800)]
Merge pull request #44274 from fengchunsong/dpdk-dfx

msg/async/dpdk: add commands to obtain the NIC status and statistics

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agomsg/async/dpdk:add commands to obtain the NIC status and statistics 44274/head
Chunsong Feng [Fri, 10 Dec 2021 08:59:41 +0000 (08:59 +0000)]
msg/async/dpdk:add commands to obtain the NIC status and statistics

Commands are added to obtain the network adapter status and statistics
for debugging network adapter packet loss and mbuf insufficiency issues.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe <hanfengzhe@hisilicon.com>
3 years agoMerge pull request #44276 from fengchunsong/dpdk-affinity
fengchunsong [Mon, 27 Dec 2021 23:53:19 +0000 (07:53 +0800)]
Merge pull request #44276 from fengchunsong/dpdk-affinity

common/numa: Skip the DPDK thread when setting NUMA affinity

3 years agoMerge PR #44342 into master
Patrick Donnelly [Mon, 27 Dec 2021 17:55:30 +0000 (12:55 -0500)]
Merge PR #44342 into master

* refs/pull/44342/head:
mds: trigger stray reintegration when loading dentry
qa: test that scrub causes reintegration

Reviewed-by: Xiubo Li <xiubli@redhat.com>
3 years agocommon/numa: Skip the DPDK thread when setting NUMA affinity 44276/head
Chunsong Feng [Mon, 27 Dec 2021 13:03:21 +0000 (13:03 +0000)]
common/numa: Skip the DPDK thread when setting NUMA affinity

The CPU affinity of the DPDK thread has been set during DPDK initialization.
Do not modify the DPDK affinity when setting NUMA affinity.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe <hanfengzhe@hisilicon.com>
3 years agomsg/async: refactory rename_thread for DPDKStack
Chunsong Feng [Fri, 10 Dec 2021 10:18:36 +0000 (10:18 +0000)]
msg/async: refactory rename_thread for DPDKStack

The thread_name of the DPDK thread has been set during DPDK initialization.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe <hanfengzhe@hisilicon.com>
3 years agoMerge pull request #44243 from ideepika/wip-deepika-iscsi
Ilya Dryomov [Mon, 27 Dec 2021 11:29:07 +0000 (12:29 +0100)]
Merge pull request #44243 from ideepika/wip-deepika-iscsi

qa/rbd: update the cephadm required distro

Reviewed-by: Melissa Li <melissali@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agocrimson/osd: print message when customize parameters 44403/head
Yingxin Cheng [Fri, 24 Dec 2021 08:16:04 +0000 (16:16 +0800)]
crimson/osd: print message when customize parameters

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/osd: allow seastar application to print its helps
Yingxin Cheng [Fri, 24 Dec 2021 08:14:29 +0000 (16:14 +0800)]
crimson/osd: allow seastar application to print its helps

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/osd: --help-seastar does no longer belong to unknown_args
Yingxin Cheng [Fri, 24 Dec 2021 08:12:13 +0000 (16:12 +0800)]
crimson/osd: --help-seastar does no longer belong to unknown_args

Now that app_template handles --help-seastar, and prints out all
app-level usages if --help is included in command line options. There is
no need to add a dedicated --help-seastar by ourselves and translate it
to --help.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #44411 from tchaikov/wip-crimson-doc
Kefu Chai [Mon, 27 Dec 2021 09:14:58 +0000 (17:14 +0800)]
Merge pull request #44411 from tchaikov/wip-crimson-doc

doc: remove references to SEASTAR_DEFAULT_ALLOCATOR

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #44333 from rzarzynski/wip-crimson-fix-recovery-discarding
Kefu Chai [Sun, 26 Dec 2021 13:46:41 +0000 (21:46 +0800)]
Merge pull request #44333 from rzarzynski/wip-crimson-fix-recovery-discarding

crimson/osd: implement op discarding for pglog-based recovery.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agoMerge pull request #44275 from fengchunsong/dpdk-nic
Kefu Chai [Sun, 26 Dec 2021 12:41:33 +0000 (20:41 +0800)]
Merge pull request #44275 from fengchunsong/dpdk-nic

msg/async/dpdk: add NIC whitelist configuration

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44285 from fengchunsong/dpdk-rte-init
Kefu Chai [Sun, 26 Dec 2021 12:04:40 +0000 (20:04 +0800)]
Merge pull request #44285 from fengchunsong/dpdk-rte-init

msg/async/dpdk: handle DPDK initialization failure properly

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agomsg/async/dpdk:add the handling of DPDK initialization failure 44285/head
Chunsong Feng [Sat, 11 Dec 2021 01:11:18 +0000 (01:11 +0000)]
msg/async/dpdk:add the handling of DPDK initialization failure

If rte_eal_init returns with failure,the waiting msgr-worker thread is
woken up for exception handling.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe <hanfengzhe@hisilicon.com>
3 years agodoc: remove references to SEASTAR_DEFAULT_ALLOCATOR 44411/head
Kefu Chai [Sun, 26 Dec 2021 05:11:06 +0000 (13:11 +0800)]
doc: remove references to SEASTAR_DEFAULT_ALLOCATOR

now that https://github.com/scylladb/seastar/commit/067681b70172ab4a6676431b623e8bd0caab6dc8
has been merged upstream, and our src/seastar submodule includes
this commit. there is no need to worry that the seastar's lockless
allocator does not work with seastar::alien anymore.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agocrimson/test: fix SeastarRunner when app is not started 44396/head
Yingxin Cheng [Fri, 24 Dec 2021 03:05:30 +0000 (11:05 +0800)]
crimson/test: fix SeastarRunner when app is not started

Support the case when the SeastarRunner isn't able to start the app,
for example, when start with --help.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agomsg/async/dpdk: add NIC whitelist configuration 44275/head
Chunsong Feng [Fri, 10 Dec 2021 02:27:58 +0000 (02:27 +0000)]
msg/async/dpdk: add NIC whitelist configuration

Each DPDK process uses some exclusive network adapters.The network adapters to
be used are specified in the whitelist to prevent data sharing between multiple
DPDK process.The following is an example:
1)Configure a single NIC.
-a 0000:7d:010 or --allow=0000:7d:010
2)Configure the Bond Network Adapter
ms_dpdk_port_options=--allow=0000:7d:01.0 --allow=0000:7d:02.6
--vdev=net_bonding0,mode=2,slave=0000:7d:01.0,slave=0000:7d:02.6

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe <hanfengzhe@hisilicon.com>
3 years agoMerge pull request #44404 from inspur-wyq/wip-doc-mgr-zabbix
Kefu Chai [Sat, 25 Dec 2021 08:44:07 +0000 (16:44 +0800)]
Merge pull request #44404 from inspur-wyq/wip-doc-mgr-zabbix

doc/mgr/zabbix.rst: fix typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agodoc/mgr/zabbix.rst: fix typos 44404/head
wangyunqing [Fri, 24 Dec 2021 09:10:11 +0000 (17:10 +0800)]
doc/mgr/zabbix.rst: fix typos

Signed-off-by: wangyunqing <wangyunqing@inspur.com>
3 years agoMerge pull request #44378 from cyx1231st/wip-seastore-onode-cleanup
Yingxin [Fri, 24 Dec 2021 03:41:02 +0000 (11:41 +0800)]
Merge pull request #44378 from cyx1231st/wip-seastore-onode-cleanup

crimson/os/seastore: fix potential leak for onodes to live across transactions

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agoMerge pull request #44311 from ifed01/wip-ifed-cleanup-onode-pin
Yuri Weinstein [Fri, 24 Dec 2021 00:44:43 +0000 (16:44 -0800)]
Merge pull request #44311 from ifed01/wip-ifed-cleanup-onode-pin

os/bluestore: get rid of fake onode nref increment for pinned entry

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #44303 from 5cs/fix-new-state-weight
Yuri Weinstein [Fri, 24 Dec 2021 00:44:06 +0000 (16:44 -0800)]
Merge pull request #44303 from 5cs/fix-new-state-weight

mon/OSDMonitor: fix incorrect op between osd state and weight

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
3 years agoMerge pull request #44391 from anthonyeleven/anthonyeleven/correct-quay-url
Neha Ojha [Thu, 23 Dec 2021 23:34:22 +0000 (15:34 -0800)]
Merge pull request #44391 from anthonyeleven/anthonyeleven/correct-quay-url

doc/install: Update quay.io URL for ceph-ci builds

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agodoc/install: Update quay.io URL for ceph-ci builds 44391/head
Anthony D'Atri [Thu, 23 Dec 2021 20:11:25 +0000 (12:11 -0800)]
doc/install: Update quay.io URL for ceph-ci builds

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
3 years agoMerge pull request #44352 from SMIL-Infra/journald
Yuri Weinstein [Thu, 23 Dec 2021 22:02:39 +0000 (14:02 -0800)]
Merge pull request #44352 from SMIL-Infra/journald

common: fix fmt::format_to deprecated warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42896 from ifed01/wip-ifed-bluefs-improve-log
Yuri Weinstein [Thu, 23 Dec 2021 22:02:16 +0000 (14:02 -0800)]
Merge pull request #42896 from ifed01/wip-ifed-bluefs-improve-log

os/bluestore: dump alloc unit size on bluefs allocation failure.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #44353 from tchaikov/wip-str-map
Yuri Weinstein [Thu, 23 Dec 2021 22:00:25 +0000 (14:00 -0800)]
Merge pull request #44353 from tchaikov/wip-str-map

common/str_map: reimplement get_str_list() using for_each_pair

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44330 from rzarzynski/wip-osd-drop-includes
Yuri Weinstein [Thu, 23 Dec 2021 21:59:36 +0000 (13:59 -0800)]
Merge pull request #44330 from rzarzynski/wip-osd-drop-includes

osd: drop unnecessary recovery / scrub includes from OSD.cc.

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44364 from rhcs-dashboard/e2e-fixups
Ernesto Puerta [Thu, 23 Dec 2021 16:23:18 +0000 (17:23 +0100)]
Merge pull request #44364 from rhcs-dashboard/e2e-fixups

mgr/dashboard: fix timeout error in dashboard cephadm e2e job

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge PR #44322 into master
Patrick Donnelly [Thu, 23 Dec 2021 13:52:10 +0000 (08:52 -0500)]
Merge PR #44322 into master

* refs/pull/44322/head:
mds: skip directory size checks for reintegration
qa: test reintegration with directory limits

Reviewed-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge pull request #44375 from wjwithagen/wjw-fix-missing-utility
Kefu Chai [Thu, 23 Dec 2021 13:22:37 +0000 (21:22 +0800)]
Merge pull request #44375 from wjwithagen/wjw-fix-missing-utility

common: add missing #include <utility>

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agomgr/dashboard: fix timeout error in dashboard cephadm e2e job 44364/head
Nizamudeen A [Mon, 20 Dec 2021 09:14:29 +0000 (14:44 +0530)]
mgr/dashboard: fix timeout error in dashboard cephadm e2e job

1. Fix the timeout error happening in the dashboard e2e job
2. Take care of the flaky force maintenance check

Most of the time our test is getting timed out while searching for an item
in the table. Its because `.clear().type()` is not clearing the content
in the search field sometimes and that creates a wrong data to be
entered into the search field and it starts searching based on this
wrong name. To avoid this I am explicitly clearing the search area
before typing.

Fixes: https://tracker.ceph.com/issues/53672
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #44365 from tchaikov/wip-seastar
Kefu Chai [Thu, 23 Dec 2021 02:31:32 +0000 (10:31 +0800)]
Merge pull request #44365 from tchaikov/wip-seastar

seastar: pick up upstream change which includes cryptopp fix

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
3 years agoMerge PR #44324 into master
Patrick Donnelly [Wed, 22 Dec 2021 19:08:26 +0000 (14:08 -0500)]
Merge PR #44324 into master

* refs/pull/44324/head:
qa: set quincy require-osd-release to avoid health warning

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge PR #44313 into master
Patrick Donnelly [Wed, 22 Dec 2021 19:05:03 +0000 (14:05 -0500)]
Merge PR #44313 into master

* refs/pull/44313/head:
mds: support '~mds{rank number}' for dump tree

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge PR #43881 into master
Patrick Donnelly [Wed, 22 Dec 2021 19:03:37 +0000 (14:03 -0500)]
Merge PR #43881 into master

* refs/pull/43881/head:
osdc: add set_error in BufferHead, when split set_error to right

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #44372 from DimitriPapadopoulos/codespell_examples
Neha Ojha [Wed, 22 Dec 2021 18:48:00 +0000 (10:48 -0800)]
Merge pull request #44372 from DimitriPapadopoulos/codespell_examples

examples: typos found by codespell

Reviewed-by: Laura Flores <lflores@redhat.com>
3 years agoMerge pull request #44241 from kamoltat/wip-ksirivad-pool-bulk-flag
Neha Ojha [Wed, 22 Dec 2021 18:06:28 +0000 (10:06 -0800)]
Merge pull request #44241 from kamoltat/wip-ksirivad-pool-bulk-flag

mon: osd pool create <pool-name> with --bulk flag

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #44385 from trociny/wip-53677
Neha Ojha [Wed, 22 Dec 2021 17:23:22 +0000 (09:23 -0800)]
Merge pull request #44385 from trociny/wip-53677

qa/tasks: improve backfill_toofull test

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44360 from SMIL-Infra/fix-ios-frontend
Ernesto Puerta [Wed, 22 Dec 2021 16:09:42 +0000 (17:09 +0100)]
Merge pull request #44360 from SMIL-Infra/fix-ios-frontend

mgr/dashboard: fix white screen on Safari

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: huww98 <NOT@FOUND>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #43598 from ideepika/wip-opentelemetry
Josh Durgin [Wed, 22 Dec 2021 15:25:13 +0000 (07:25 -0800)]
Merge pull request #43598 from ideepika/wip-opentelemetry

migrate from using opentracing-cpp to opentelemetry-cpp static as distributed tracing API

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #44334 from aclamk/wip-aclamk-better-bluestore-perf
Ernesto Puerta [Wed, 22 Dec 2021 12:04:14 +0000 (13:04 +0100)]
Merge pull request #44334 from aclamk/wip-aclamk-better-bluestore-perf

os/bluestore: Better readability of perf output

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
3 years agoexamples: typos found by codespell 44372/head
Dimitri Papadopoulos [Wed, 8 Dec 2021 11:43:27 +0000 (12:43 +0100)]
examples: typos found by codespell

Signed-off-by: Dimitri Papadopoulos <dpo.ceph@mail.com>
3 years agoMerge pull request #44317 from aclamk/aclamk-fix-bluefs-import
Neha Ojha [Tue, 21 Dec 2021 19:57:01 +0000 (11:57 -0800)]
Merge pull request #44317 from aclamk/aclamk-fix-bluefs-import

Fix ceph-bluestore-tool bluefs-import command

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
3 years agoMerge pull request #44328 from yaarith/gather_crashinfo-indexerror
Yuri Weinstein [Tue, 21 Dec 2021 18:04:54 +0000 (10:04 -0800)]
Merge pull request #44328 from yaarith/gather_crashinfo-indexerror

mgr/telemetry: check that backtrace is not empty when redacting

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
3 years agoMerge pull request #44327 from yaarith/gather_device_report-indexerror
Yuri Weinstein [Tue, 21 Dec 2021 18:04:25 +0000 (10:04 -0800)]
Merge pull request #44327 from yaarith/gather_device_report-indexerror

mgr/telemetry: catch also IndexError in gather_device_report()

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
3 years agoMerge pull request #44314 from locallocal/master
Yuri Weinstein [Tue, 21 Dec 2021 18:03:31 +0000 (10:03 -0800)]
Merge pull request #44314 from locallocal/master

osd: remove unused variable legacy_infos_oid in the function PG::peek_map_epoch

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43686 from myoungwon/wip-ceph-dedup-tool-make-dedup-obj
Yuri Weinstein [Tue, 21 Dec 2021 18:03:10 +0000 (10:03 -0800)]
Merge pull request #43686 from myoungwon/wip-ceph-dedup-tool-make-dedup-obj

tool: add  chunk/object-dedup command

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoqa/tasks: improve backfill_toofull test 44385/head
Mykola Golub [Tue, 21 Dec 2021 18:00:28 +0000 (20:00 +0200)]
qa/tasks: improve backfill_toofull test

1) Write more data to the pool so we operate with larger ratios.
2) Round up ratios when truncating.

Fixes: https://tracker.ceph.com/issues/53677
Signed-off-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44190 from rhcs-dashboard/grafana-regex
Ernesto Puerta [Tue, 21 Dec 2021 16:58:17 +0000 (17:58 +0100)]
Merge pull request #44190 from rhcs-dashboard/grafana-regex

monitoring/grafana: improve grafana unit tests variable substitution

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44362 from rhcs-dashboard/kcli-e2e-improvements
Ernesto Puerta [Tue, 21 Dec 2021 16:53:41 +0000 (17:53 +0100)]
Merge pull request #44362 from rhcs-dashboard/kcli-e2e-improvements

mgr/dashboard: cephadm e2e job: improvements

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #43838 from rhcs-dashboard/fix-53128-master
Ernesto Puerta [Tue, 21 Dec 2021 16:33:21 +0000 (17:33 +0100)]
Merge pull request #43838 from rhcs-dashboard/fix-53128-master

mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agocrimson/os/seastore: fix potential leak for onodes to live across transactions 44378/head
Yingxin Cheng [Tue, 21 Dec 2021 14:32:34 +0000 (22:32 +0800)]
crimson/os/seastore: fix potential leak for onodes to live across transactions

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #43942 from rhcs-dashboard/host-drain-frontend
Ernesto Puerta [Tue, 21 Dec 2021 12:33:51 +0000 (13:33 +0100)]
Merge pull request #43942 from rhcs-dashboard/host-drain-frontend

mgr/dashboard: Implement drain host functionality in dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agomgr/dashboard: Implement drain host functionality in dashboard 43942/head
Nizamudeen A [Tue, 16 Nov 2021 11:00:02 +0000 (16:30 +0530)]
mgr/dashboard: Implement drain host functionality in dashboard

Fixes: https://tracker.ceph.com/issues/51587
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agocommon: Fix missing utility include 44375/head
Willem Jan Withagen [Tue, 21 Dec 2021 09:10:17 +0000 (10:10 +0100)]
common: Fix missing utility include

See:
    https://en.cppreference.com/w/cpp/utility/move

Detected on FreeBSD/Clang/libc++:
/home/jenkins/workspace/ceph-master-compile/src/common/deleter.h:111:43: error: no member named 'move' in namespace 'std'
  impl(deleter next) : refs(1), next(std::move(next)) {}

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
3 years agoAdded ReleasesNotes and documentation 44241/head
Kamoltat [Tue, 21 Dec 2021 07:56:37 +0000 (07:56 +0000)]
Added ReleasesNotes and documentation

Add Release Notes and remove any `profile`
related stuff in the autoscaler documentation
and replace it with `bulk` flag.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
3 years agoMerge pull request #42750 from aclamk/wip-aclamk-bluefs-log-incremental
Neha Ojha [Tue, 21 Dec 2021 01:15:13 +0000 (17:15 -0800)]
Merge pull request #42750 from aclamk/wip-aclamk-bluefs-log-incremental

os/bluestore: incremental update mode for bluefs log

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
3 years agoMerge pull request #44368 from ceph/fix-api-tests
Ernesto Puerta [Mon, 20 Dec 2021 22:43:18 +0000 (23:43 +0100)]
Merge pull request #44368 from ceph/fix-api-tests

mgr/dashboard: run-backend-api-tests.sh: Older setuptools

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
3 years agoos/bluestore/bluefs: Add tracking of bluefs log in noop replay mode 42750/head
Adam Kupczyk [Wed, 15 Dec 2021 09:59:55 +0000 (09:59 +0000)]
os/bluestore/bluefs: Add tracking of bluefs log in noop replay mode

Keep updating bluefs log when printing content of bluefs replay log.
Without this modification we only have initial content of log.
Log can be printed by 'ceph-bluestore-tool bluefs-log-dump'.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoos/bluestore/bluefs: Sync BlueFS log with its allocation delta
Adam Kupczyk [Wed, 24 Nov 2021 17:55:05 +0000 (18:55 +0100)]
os/bluestore/bluefs: Sync BlueFS log with its allocation delta

BlueFS log is the only file that we can append to.
When we append to file we must take into consideration previously commited allocations,
otherwise update will be miscalculated.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agotest/objectstore/bluefs_test: Add test for continuation of previous BlueFS log
Adam Kupczyk [Wed, 24 Nov 2021 17:52:35 +0000 (18:52 +0100)]
test/objectstore/bluefs_test: Add test for continuation of previous BlueFS log

Added test that verifies that in update mode we properly pick up delta.
BlueFS log is the only file that can be appended to, but it is done in very indirect way.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoos/bluefs: allow incremental file metadata updates in bluefs log
Igor Fedotov [Fri, 30 Jul 2021 11:02:26 +0000 (14:02 +0300)]
os/bluefs: allow incremental file metadata updates in bluefs log

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoqa/workunits/cephtool: modified test.sh
Kamoltat [Wed, 15 Dec 2021 05:26:29 +0000 (05:26 +0000)]
qa/workunits/cephtool: modified test.sh

Included bulk flag in the test script
such it tests the `get` and `set` of
the flag

Signed-off-by: Kamoltat <ksirivad@redhat.com>
3 years agoqa/workunits/mon/pg_autoscaler: modified test script
Kamoltat [Tue, 14 Dec 2021 21:32:10 +0000 (21:32 +0000)]
qa/workunits/mon/pg_autoscaler: modified test script

Modified test scrtipt to include `bulk` and
remove all `profile` options.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
3 years agopg_autoscaler/test: Modified unit-test for bulk flag
Kamoltat [Mon, 13 Dec 2021 15:30:17 +0000 (15:30 +0000)]
pg_autoscaler/test: Modified unit-test for bulk flag

Modified the unit-test cases to account for bulk
flag and remove any `profile` related things.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
3 years agomon: osd pool create <pool-name> with --bulk flag
Kamoltat [Tue, 7 Dec 2021 21:15:36 +0000 (21:15 +0000)]
mon: osd pool create <pool-name> with --bulk flag

Creating the pool with `--bulk` will allow
the pg_autoscaler to use the `scale-down`
mode on.

Creating pool:

`ceph osd pool create <pool-name> --bulk`

Get var:

`ceph osd pool get <pool-name> bulk`

Set var:

`ceph osd pool set <pool-name> bulk=true/false/1/0`

Removed `autoscale_profile` and incorporate bulk flag
into calculating `final_pg_target` for each pool.

bin/ceph osd pool autoscale-status no longer has
`PROFILE` column but has `BULK` instead.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
3 years agoMerge PR #44315 into master
Patrick Donnelly [Mon, 20 Dec 2021 19:41:43 +0000 (14:41 -0500)]
Merge PR #44315 into master

* refs/pull/44315/head:
doc/cephfs: mds default cache memory limit is now 4GB

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
3 years agopybind/mgr/dashboard/run-backend-api-tests: Older setuptools 44368/head
David Galloway [Mon, 20 Dec 2021 15:44:23 +0000 (10:44 -0500)]
pybind/mgr/dashboard/run-backend-api-tests: Older setuptools

https://github.com/pypa/setuptools/issues/2353

Signed-off-by: David Galloway <dgallowa@redhat.com>
3 years agoMerge pull request #42416 from galsalomon66/wip-test-albin-error-status
Matt Benjamin [Mon, 20 Dec 2021 15:50:44 +0000 (10:50 -0500)]
Merge pull request #42416 from galsalomon66/wip-test-albin-error-status

RGW/s3select : new-s3select-responds presto-alignments output-serialization

3 years agoseastar: pick up upstream change which includes cryptopp fix 44365/head
Kefu Chai [Mon, 20 Dec 2021 12:05:06 +0000 (20:05 +0800)]
seastar: pick up upstream change which includes cryptopp fix

update seastar submodule to upstream master, so we don't need to
use our own fork.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoos/bluestore: get rid of fake onode nref increment for pinned entry 44311/head
Igor Fedotov [Tue, 14 Dec 2021 14:56:37 +0000 (17:56 +0300)]
os/bluestore: get rid of fake onode nref increment for pinned entry

Looks like this isn't necessary any more after fixing
https://tracker.ceph.com/issues/53002

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
3 years agomgr/dashboard: cephadm e2e job: improvements 44362/head
Alfonso Martínez [Mon, 20 Dec 2021 08:12:37 +0000 (09:12 +0100)]
mgr/dashboard: cephadm e2e job: improvements

- Avoid jenkins env. error: Input file /workdir/./src/pybind/mgr/dashboard/ci/cephadm/ceph_cluster.yml not found.Leaving...
- More meaningful log output.

Fixes: https://tracker.ceph.com/issues/53666
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
3 years agomgr/dashboard: Cluster Expansion - Review Section: fixes and improvements 43838/head
Aashish Sharma [Mon, 8 Nov 2021 07:31:02 +0000 (13:01 +0530)]
mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements

Ensure "Storage capacity" keeps the "Description : Value" approach ("Number of devices: X" and "Raw Capacity: Y" in different lines).Correct issue with "host by services" host count

Fixes: https://tracker.ceph.com/issues/53128
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
3 years agomgr/dashboard: fix white screen on Safari 44360/head
胡玮文 [Mon, 20 Dec 2021 04:54:23 +0000 (12:54 +0800)]
mgr/dashboard: fix white screen on Safari

Safari do not support lookbehind in regular expression.

Fixes: https://tracker.ceph.com/issues/53665
Signed-off-by: 胡玮文 <huww98@outlook.com>
3 years agoMerge pull request #44350 from Huber-ming/fix_annotation_misspelling
Kefu Chai [Sun, 19 Dec 2021 07:53:45 +0000 (15:53 +0800)]
Merge pull request #44350 from Huber-ming/fix_annotation_misspelling

rgw: fix misspelling of 'effort'

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agocommon: fix fmt::format_to deprecated warning 44352/head
胡玮文 [Sun, 19 Dec 2021 06:47:20 +0000 (14:47 +0800)]
common: fix fmt::format_to deprecated warning

Signed-off-by: 胡玮文 <huww98@outlook.com>
3 years agomgr: move DaemonState methods into .cc 44353/head
Kefu Chai [Sun, 19 Dec 2021 06:48:07 +0000 (14:48 +0800)]
mgr: move DaemonState methods into .cc

for faster compilation, and for better readability.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agomgr: refactor DaemonState::set_metadata()
Kefu Chai [Sun, 19 Dec 2021 06:43:35 +0000 (14:43 +0800)]
mgr: refactor DaemonState::set_metadata()

restructure DaemonState::set_metadata() using for_each_pair() for better
readability.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agocommon/str_map: reimplement get_str_list() using for_each_pair
Kefu Chai [Sun, 19 Dec 2021 04:25:50 +0000 (12:25 +0800)]
common/str_map: reimplement get_str_list() using for_each_pair

to avoid creating a temporary list<string> and then dropping it
on the floor after iterating through it for collecting the kv
pairs in it.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44277 from fengchunsong/dpdk-rte-exit
Kefu Chai [Sat, 18 Dec 2021 14:19:50 +0000 (22:19 +0800)]
Merge pull request #44277 from fengchunsong/dpdk-rte-exit

msg/async/dpdk: replacing rte_exit with ceph_assert to avoid exit fai…

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agousing fmt for message formatting 42416/head
galsalomon66 [Thu, 2 Dec 2021 11:11:00 +0000 (13:11 +0200)]
using fmt for message formatting

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years agofix comments; enum-class; typo; documentation
galsalomon66 [Thu, 2 Dec 2021 10:08:31 +0000 (12:08 +0200)]
fix comments; enum-class; typo; documentation

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years agotypo
galsalomon66 [Sat, 27 Nov 2021 19:53:55 +0000 (21:53 +0200)]
typo

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years agomodification for error-handling and boto3
galsalomon66 [Sat, 27 Nov 2021 17:44:40 +0000 (19:44 +0200)]
modification for error-handling and boto3

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years agoadding/modifying documentation for output-serialization, s3select-response, syntax...
galsalomon66 [Fri, 26 Nov 2021 08:22:51 +0000 (10:22 +0200)]
adding/modifying documentation for output-serialization, s3select-response, syntax support, error-handling

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years agos3select: point to s3select/master(after merging pr #90)
galsalomon66 [Wed, 24 Nov 2021 09:45:27 +0000 (11:45 +0200)]
s3select: point to s3select/master(after merging pr #90)

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years agoin the case Presto is push-down query, it also send output-serialization parameter...
galsalomon66 [Wed, 17 Nov 2021 17:55:48 +0000 (19:55 +0200)]
in the case Presto is push-down query, it also send output-serialization parameter('&#10;'), that should be handle as '\n'

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years agos3tests.yaml points to ceph-master
galsalomon66 [Mon, 15 Nov 2021 17:29:30 +0000 (19:29 +0200)]
s3tests.yaml points to ceph-master

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years ago RGW: Implement continuation, progress, stats, end s3select response
gal salomon [Fri, 7 May 2021 21:29:13 +0000 (00:29 +0300)]
RGW: Implement continuation, progress, stats, end s3select response

    RGW/S3select: Implement output-serializationi. user may request different CSV defintions
    for output (field delimiter, row delimiter, quote handling.

    RGW/S3select: Implement presto-alignments. presto-application sends
    queries with table-alias,case insensitive, and with no-semicolon at the
    end of statement.

    RGW/s3select: zero object size issue

Signed-off-by: Albin Antony <aantony@redhat.com>
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
3 years agorgw: fix misspelling of 'effort' 44350/head
Huber-ming [Sat, 18 Dec 2021 06:43:34 +0000 (14:43 +0800)]
rgw: fix misspelling of 'effort'

Signed-off-by: Huber-ming <zhangsm01@inspur.com>
3 years agoMerge pull request #43139 from josephsawaya/rook-orch-qa
Joseph Sawaya [Fri, 17 Dec 2021 17:50:18 +0000 (12:50 -0500)]
Merge pull request #43139 from josephsawaya/rook-orch-qa

qa/tasks/rook: test reapplication of drive groups stored in mgr

3 years agoMerge PR #44337 into master
Sage Weil [Fri, 17 Dec 2021 14:59:48 +0000 (09:59 -0500)]
Merge PR #44337 into master

* refs/pull/44337/head:
mon: prevent new sessions during shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43872 from Huber-ming/reshard
Casey Bodley [Fri, 17 Dec 2021 14:30:42 +0000 (09:30 -0500)]
Merge pull request #43872 from Huber-ming/reshard

rgw: init bucket index only if putting bucket instance info succeeds

Reviewed-by: Casey Bodley <cbodley@redhat.com>