]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agocrimson/os/seastore/omap: remove unnecessary extent loads when clearing. 47290/head
Xinyu Huang [Tue, 26 Jul 2022 23:53:41 +0000 (23:53 +0000)]
crimson/os/seastore/omap: remove unnecessary extent loads when clearing.

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
3 years agoMerge pull request #46911 from ifed01/wip-ifed-fix-mempool-cache-other
Igor Fedotov [Mon, 18 Jul 2022 09:06:46 +0000 (12:06 +0300)]
Merge pull request #46911 from ifed01/wip-ifed-fix-mempool-cache-other

os/bluestore: fix AU accounting in bluestore_cache_other mempool.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #47106 from idryomov/wip-56561
Ilya Dryomov [Mon, 18 Jul 2022 08:08:20 +0000 (10:08 +0200)]
Merge pull request #47106 from idryomov/wip-56561

rbd: don't default empty pool name unless namespace is specified

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #46741 from Matan-B/wip-matanb-docker-debug
Matan [Mon, 18 Jul 2022 07:40:06 +0000 (10:40 +0300)]
Merge pull request #46741 from Matan-B/wip-matanb-docker-debug

script: CentOS 8 EOL, use archived mirror in ceph-debug-docker.sh

Reviewed-by: Nitzan Mordechai nmordech@redhat.com
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #46375 from ceph/cv-loop-devs
Guillaume Abrioux [Mon, 18 Jul 2022 07:24:24 +0000 (09:24 +0200)]
Merge pull request #46375 from ceph/cv-loop-devs

ceph-volume: Optionally consume loop devices

3 years agoMerge pull request #47113 from zhscn/fix-split-test
Yingxin [Mon, 18 Jul 2022 02:42:29 +0000 (10:42 +0800)]
Merge pull request #47113 from zhscn/fix-split-test

crimson/os/seastore: fix bugs in test_map_existing_extent_concurrent

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #47025 from adamemerson/wip-55765
Adam C. Emerson [Sun, 17 Jul 2022 02:17:05 +0000 (22:17 -0400)]
Merge pull request #47025 from adamemerson/wip-55765

rgw: Guard against malformed bucket URLs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #46897 from rkachach/fix_issue_55808
Adam King [Sat, 16 Jul 2022 22:43:11 +0000 (18:43 -0400)]
Merge pull request #46897 from rkachach/fix_issue_55808

mgr/cephadm: check for events key before accessing it

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
3 years agoMerge pull request #47011 from s0nea/wip-prevent-alert-redirects
Nizamudeen A [Sat, 16 Jul 2022 13:56:26 +0000 (19:26 +0530)]
Merge pull request #47011 from s0nea/wip-prevent-alert-redirects

mgr/dashboard: prevent alert redirect

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #44217 from CongMinYin/fix-pwl-recovery-test
Ilya Dryomov [Sat, 16 Jul 2022 09:34:55 +0000 (11:34 +0200)]
Merge pull request #44217 from CongMinYin/fix-pwl-recovery-test

qa/suites/rbd/pwl-cache: ensure recovery is actually tested

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agorbd: drop unused default_empty_pool_name argument 47106/head
Ilya Dryomov [Thu, 14 Jul 2022 12:42:45 +0000 (14:42 +0200)]
rbd: drop unused default_empty_pool_name argument

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agorbd: don't default empty pool name unless namespace is specified
Ilya Dryomov [Thu, 14 Jul 2022 12:19:06 +0000 (14:19 +0200)]
rbd: don't default empty pool name unless namespace is specified

Commit 96f05a7956b3 ("rbd: delay determination of default pool name")
broke "rbd perf image iostat" and "rbd perf image iotop" GLOBAL_POOL_KEY
support (the ability to blend all rbd pools together into a single
view).

Fixes: https://tracker.ceph.com/issues/56561
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoqa/tasks: rename persistent write log cache trash task 44217/head
Ilya Dryomov [Sat, 16 Jul 2022 06:54:38 +0000 (08:54 +0200)]
qa/tasks: rename persistent write log cache trash task

It doesn't really thrash anything, just repeatedly restarts the
workload on top of a dirty cache file.  rbd_pwl_cache_recovery is
more on point and gets covered by existing CODEOWNERS.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #47109 from zdover23/wip-doc-2022-07-15-start-hw-recs-cleanup-1
zdover23 [Sat, 16 Jul 2022 02:28:43 +0000 (12:28 +1000)]
Merge pull request #47109 from zdover23/wip-doc-2022-07-15-start-hw-recs-cleanup-1

doc/start: update hardware recs

Reviewed-by: Anthony D'Atri
3 years agoMerge pull request #46908 from mlausch/snapshot_key_conversion
Neha Ojha [Fri, 15 Jul 2022 20:50:47 +0000 (13:50 -0700)]
Merge pull request #46908 from mlausch/snapshot_key_conversion

osd/SnapMapper: fix legacy key conversion in snapmapper class

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agorgw: Guard against malformed bucket URLs 47025/head
Adam C. Emerson [Fri, 8 Jul 2022 18:58:16 +0000 (14:58 -0400)]
rgw: Guard against malformed bucket URLs

Misplaced colons can result in radosgw thinking is has a bucket URL
but with no bucket name, leading to a crash later on.

Fixes: https://tracker.ceph.com/issues/55765
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
3 years agorgw: Fix `rgw::sal::Bucket::empty` static method signatures
Adam C. Emerson [Mon, 11 Jul 2022 15:52:09 +0000 (11:52 -0400)]
rgw: Fix `rgw::sal::Bucket::empty` static method signatures

`unique_ptr` overload should take by reference.

Both should be const.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
3 years agoMerge pull request #47052 from neha-ojha/wip-cot-label
Neha Ojha [Fri, 15 Jul 2022 18:36:18 +0000 (11:36 -0700)]
Merge pull request #47052 from neha-ojha/wip-cot-label

.github/labeler.yml: add core label to some tools

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #47115 from ceph/fix-mib
Ilya Dryomov [Fri, 15 Jul 2022 16:48:47 +0000 (18:48 +0200)]
Merge pull request #47115 from ceph/fix-mib

ceph.spec.in: fix path for mib file and properly mark in %files

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #47035 from idryomov/wip-56516
Ilya Dryomov [Fri, 15 Jul 2022 13:46:24 +0000 (15:46 +0200)]
Merge pull request #47035 from idryomov/wip-56516

rbd-mirror: remove bogus completed_non_primary_snapshots_exist check

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoceph.spec.in: fix path for mib file and properly mark in %files 47115/head
Justin Caratzas [Thu, 14 Jul 2022 22:45:49 +0000 (18:45 -0400)]
ceph.spec.in: fix path for mib file and properly mark in %files

Fixes typos introduced in https://github.com/ceph/ceph/pull/46918

Signed-off-by: Justin Caratzas <jcaratza@redhat.com>
3 years agoMerge pull request #47028 from xxhdx1985126/wip-seastore-backref-cache-refactor
Yingxin [Fri, 15 Jul 2022 07:16:42 +0000 (15:16 +0800)]
Merge pull request #47028 from xxhdx1985126/wip-seastore-backref-cache-refactor

crimson/os/seastore: simplify backref cache

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore: fix bugs in test_map_existing_extent_concurrent 47113/head
Zhang Song [Fri, 15 Jul 2022 06:40:46 +0000 (14:40 +0800)]
crimson/os/seastore: fix bugs in test_map_existing_extent_concurrent

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
3 years agocrimson/os/seastore: simplify backref cache 47028/head
Xuehan Xu [Thu, 7 Jul 2022 08:05:20 +0000 (16:05 +0800)]
crimson/os/seastore: simplify backref cache

Currently, the following transaction exec sequence would lead to
loss of backref:

1. Trans `A` merge a alloc backref for extent `X`
2. Trans `B` add a release backref for extent `X` to backref cache,
   during which it finds an in-cache alloc backref for extent `X` and
   decide not to add the release backref to cache
3. Trans `A` commit

In the above sequece, the release backref for extent `X` is lost.

This is a regression introduced when we try to optimize the backref cache.

This commit fix the issue by caching inflight backrefs in a multiset,
alloc/release ops that happen on the same paddr are queued in the order of
their happening. When doing gc, all those backrefs are merged.

Fixes: https://tracker.ceph.com/issues/56519
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agodoc/start: update hardware recs 47109/head
Zac Dover [Thu, 14 Jul 2022 19:29:11 +0000 (05:29 +1000)]
doc/start: update hardware recs

This PR picks up the parts of
https://github.com/ceph/ceph/pull/44466
that were not merged back in January, when that
pull request was raised.

Matters added here:
* improved organzation of matter
* emphasis of IOPs per core over cores per OSD

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agoMerge pull request #46546 from mgfritch/vstart-stop-mds
Adam King [Thu, 14 Jul 2022 18:02:32 +0000 (14:02 -0400)]
Merge pull request #46546 from mgfritch/vstart-stop-mds

src/stop.sh: stop existing ceph-mds daemons

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
3 years agoMerge pull request #46014 from vrushch/rbd_form_fix
Pere Diaz Bou [Thu, 14 Jul 2022 16:20:24 +0000 (18:20 +0200)]
Merge pull request #46014 from vrushch/rbd_form_fix

mgr/dashboard: rbd striping setting pre-population and pop-over

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #47066 from adk3798/osd-removal-docs-update
Adam King [Thu, 14 Jul 2022 15:07:08 +0000 (11:07 -0400)]
Merge pull request #47066 from adk3798/osd-removal-docs-update

doc/cephadm: add note about OSDs being recreated to OSD removal section

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
3 years agoMerge pull request #47062 from guits/update-latest-stable-default
Adam King [Thu, 14 Jul 2022 15:03:39 +0000 (11:03 -0400)]
Merge pull request #47062 from guits/update-latest-stable-default

cephadm: update LATEST_STABLE_RELEASE

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #46260 from CongMinYin/wip-enable-ndctl
Ilya Dryomov [Thu, 14 Jul 2022 13:21:29 +0000 (15:21 +0200)]
Merge pull request #46260 from CongMinYin/wip-enable-ndctl

cmake: enable ndctl when building PMDK for WITH_BLUESTORE_PMEM

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #46644 from rhcs-dashboard/rbd-list-pagination
Nizamudeen A [Thu, 14 Jul 2022 11:34:31 +0000 (17:04 +0530)]
Merge pull request #46644 from rhcs-dashboard/rbd-list-pagination

mgr/dashboard: rbd image pagination

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #45981 from rhcs-dashboard/box-no-more-ultron
Nizamudeen A [Thu, 14 Jul 2022 11:13:49 +0000 (16:43 +0530)]
Merge pull request #45981 from rhcs-dashboard/box-no-more-ultron

cephadm/box: Rootless podman box implementation

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: anthonyeleven <NOT@FOUND>
Reviewed-by: melissa-kun-li <NOT@FOUND>
3 years agoosd/SnapMapper: fix pacific legacy key conversion and introduce test 46908/head
Manuel Lausch [Thu, 30 Jun 2022 12:29:53 +0000 (14:29 +0200)]
osd/SnapMapper: fix pacific legacy key conversion and introduce test

Octopus modified the SnapMapper key format from

  <LEGACY_MAPPING_PREFIX><snapid>_<shardid>_<hobject_t::to_str()>

to

  <MAPPING_PREFIX><pool>_<snapid>_<shardid>_<hobject_t::to_str()>

When this change was introduced, 94ebe0ea also introduced a conversion
with a crucial bug which essentially destroyed legacy keys by mapping them
to

  <MAPPING_PREFIX><poolid>_<snapid>_

without the object-unique suffix.  This commit fixes this conversion going
forward, but a fix for existing clusters still needs to be developed.

Fixes: https://tracker.ceph.com/issues/56147
Signed-off-by: Manuel Lausch <manuel.lausch@1und1.de>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
3 years agoMerge pull request #46650 from faithuniterh/Modifying-boto3-examples
Yuval Lifshitz [Thu, 14 Jul 2022 08:03:20 +0000 (11:03 +0300)]
Merge pull request #46650 from faithuniterh/Modifying-boto3-examples

Modifying boto3 examples

reviewed-by: ylifshit@redhat.com

3 years agoMerge pull request #47039 from rosinL/fix-perf-cirmson-msgr
Yingxin [Thu, 14 Jul 2022 06:25:53 +0000 (14:25 +0800)]
Merge pull request #47039 from rosinL/fix-perf-cirmson-msgr

tools/crimson/perf_crimson_msgr:fix perf_crimson_msgr abort

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #47081 from zhscn/fix-map-exist
Samuel Just [Wed, 13 Jul 2022 20:59:51 +0000 (13:59 -0700)]
Merge pull request #47081 from zhscn/fix-map-exist

crimson/os/seastore: fix bug of Transaction::is_retired

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agodoc/cephadm: add note about OSDs being recreated to OSD removal section 47066/head
Adam King [Tue, 12 Jul 2022 20:54:19 +0000 (16:54 -0400)]
doc/cephadm: add note about OSDs being recreated to OSD removal section

Signed-off-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #46918 from bigjust/wip-jcaratza-mib-rpm
Justin Caratzas [Wed, 13 Jul 2022 16:56:53 +0000 (12:56 -0400)]
Merge pull request #46918 from bigjust/wip-jcaratza-mib-rpm

monitoring:package SNMP MIB file as an rpm

3 years agomgr/dashboard: fix rbdconfiguration init type 46644/head
Pere Diaz Bou [Wed, 6 Jul 2022 15:47:31 +0000 (17:47 +0200)]
mgr/dashboard: fix rbdconfiguration init type

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: prevent alert redirect 47011/head
Tatjana Dehler [Thu, 7 Jul 2022 15:21:14 +0000 (17:21 +0200)]
mgr/dashboard: prevent alert redirect

Prevent Alertmanager alerts from being redirected to the active mgr
dashboard instance. There are two reasons for it:

1. It doesn't bring any additional benefit. The Alertmanager config
   includes all available mgr instances - active and passive ones. In
   case of an alert, it will be sent to all of them. It ensures that
   the active mgr dashboard will receive the alert in any case.
2. The redirect URL includes the mgr IP and NOT the FQDN. This leads
   to issues in environments where an SSL certificate is configured and
   matches the FQDNs, only.

Fixes: https://tracker.ceph.com/issues/56401
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
3 years agocephadm/box: Choose between docker or podman with --engine 45981/head
Pere Diaz Bou [Tue, 12 Jul 2022 10:28:47 +0000 (12:28 +0200)]
cephadm/box: Choose between docker or podman with --engine

With ./box.py --engine docker you can specify you want to use docker
instead of podman. With docker box.py command should be run with sudo.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #46114 from myoungwon/wip-dedup-tool-object-dedup-snapshot
Yuri Weinstein [Wed, 13 Jul 2022 14:32:32 +0000 (07:32 -0700)]
Merge pull request #46114 from myoungwon/wip-dedup-tool-object-dedup-snapshot

tool/ceph-dedup-tool: add performing dedup option on cloned object

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #47021 from aisakaki/wip-cache-opt
Yingxin [Wed, 13 Jul 2022 14:01:13 +0000 (22:01 +0800)]
Merge pull request #47021 from aisakaki/wip-cache-opt

crimson/os/seastore/cache: fine-grained lru cache control with GC

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore: fix bug of Transaction::is_retired 47081/head
Zhang Song [Wed, 13 Jul 2022 12:45:05 +0000 (20:45 +0800)]
crimson/os/seastore: fix bug of Transaction::is_retired

The retired extent may exist as a RetiredExtentPlaceholder, casting
this extent to LogicalCachedExtent will cause undefined behavior.

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
3 years agoMerge pull request #46898 from rhcs-dashboard/cleanup-55720-master
Pere Diaz Bou [Wed, 13 Jul 2022 12:43:06 +0000 (14:43 +0200)]
Merge pull request #46898 from rhcs-dashboard/cleanup-55720-master

mgr/dashboard: don't log tracebacks on 404s

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #46987 from rhcs-dashboard/fix-ingress-backend-service-filter
Pere Diaz Bou [Wed, 13 Jul 2022 12:41:31 +0000 (14:41 +0200)]
Merge pull request #46987 from rhcs-dashboard/fix-ingress-backend-service-filter

mgr/dashboard: ingress backend service should list all supported services

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
3 years agoMerge pull request #47006 from teuchert/fix_56269
Venky Shankar [Wed, 13 Jul 2022 09:29:30 +0000 (14:59 +0530)]
Merge pull request #47006 from teuchert/fix_56269

mgr/snap_schedule: Use rados.Ioctx.remove_object() instead of remove().

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoqa/tasks: add thrash test for persistent write log cache
Yin Congmin [Fri, 7 Jan 2022 07:03:44 +0000 (15:03 +0800)]
qa/tasks: add thrash test for persistent write log cache

add thrash test for persistent write log cache. run rbd bench
on persistent write log cache, thrashes rbd bench, test the
recovery function of persistent write log cache.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
3 years agocrimson/os/seastore/cache: fine-grained lru cache control with GC 47021/head
Xinyu Huang [Tue, 12 Jul 2022 10:19:04 +0000 (10:19 +0000)]
crimson/os/seastore/cache: fine-grained lru cache control with GC

GC transaction is not sourced by user behaviors, so the extent read
operations from GC transaction don’t satisfy the time locality
principle. These extents should not be added to LRU cache.

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
3 years agoMerge pull request #47064 from rzarzynski/wip-crimson-fix-client-request-leak
Samuel Just [Tue, 12 Jul 2022 21:10:52 +0000 (14:10 -0700)]
Merge pull request #47064 from rzarzynski/wip-crimson-fix-client-request-leak

crimson/osd: don't leak ClientRequests on stopping OSD

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoceph-volume: Remove refs to get_block_devs_lsblk 46375/head
Zack Cerza [Tue, 21 Jun 2022 17:37:49 +0000 (11:37 -0600)]
ceph-volume: Remove refs to get_block_devs_lsblk

It's been replaced by get_block_devs_sysfs

Signed-off-by: Zack Cerza <zack@redhat.com>
3 years agoceph-volume: Rename env var; add warning
Zack Cerza [Tue, 21 Jun 2022 17:28:30 +0000 (11:28 -0600)]
ceph-volume: Rename env var; add warning

So that we can have a nice big warning that fires once per invocation, I
think using a callable class with a class attribute seems like a decent
approach. A closure could work too.

Signed-off-by: Zack Cerza <zack@redhat.com>
3 years agoceph-volume: Optionally consume loop devices
Zack Cerza [Tue, 17 May 2022 17:29:02 +0000 (11:29 -0600)]
ceph-volume: Optionally consume loop devices

A similar proposal was rejected in #24765; I understand the logic
behind the rejection, but this will allow us to run Ceph clusters on
machines that lack disk resources for testing purposes. We just need to
make it impossible to accidentally enable, and make it clear it is
unsupported.

Signed-off-by: Zack Cerza <zack@redhat.com>
3 years agomgr/dashboard: fix api tests + lint
Pere Diaz Bou [Tue, 5 Jul 2022 12:57:16 +0000 (14:57 +0200)]
mgr/dashboard: fix api tests + lint

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: cleanups
Pere Diaz Bou [Tue, 5 Jul 2022 11:22:42 +0000 (13:22 +0200)]
mgr/dashboard: cleanups

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: RbdConfiguration.list reuse image context.
Pere Diaz Bou [Tue, 5 Jul 2022 10:39:12 +0000 (12:39 +0200)]
mgr/dashboard: RbdConfiguration.list reuse image context.

Image contexts are reopen even though we pass the context as an
argument. This commit changes that so you can forget about reopening
a rbd image context again.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: change sort char to '+' and page icons
Pere Diaz Bou [Tue, 5 Jul 2022 09:55:51 +0000 (11:55 +0200)]
mgr/dashboard: change sort char to '+' and page icons

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: fix linting errors
Pere Diaz Bou [Tue, 5 Jul 2022 08:10:23 +0000 (10:10 +0200)]
mgr/dashboard: fix linting errors

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: add rbd list sorting support
Pere Diaz Bou [Mon, 4 Jul 2022 14:57:43 +0000 (16:57 +0200)]
mgr/dashboard: add rbd list sorting support

Support sorting with name, pool name and namespace

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: update rbd list api version to 2
Pere Diaz Bou [Mon, 4 Jul 2022 10:56:03 +0000 (12:56 +0200)]
mgr/dashboard: update rbd list api version to 2

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: debounce reload data with serverSide
Pere Diaz Bou [Mon, 4 Jul 2022 10:33:36 +0000 (12:33 +0200)]
mgr/dashboard: debounce reload data with serverSide

Whenever we use serverSide (paginate through backend) we should
debounce reloadData since it might call api calls too much times.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: add rbd list search and disable sorting
Pere Diaz Bou [Fri, 1 Jul 2022 08:16:13 +0000 (10:16 +0200)]
mgr/dashboard: add rbd list search and disable sorting

- Disable sorting in each column because it will not be possible to
sort with this pagination implementation.
- Add search capabilities to the rbd list pagination endpoint.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: fix rbd list tests
Pere Diaz Bou [Wed, 29 Jun 2022 11:37:17 +0000 (13:37 +0200)]
mgr/dashboard: fix rbd list tests

X-Total-Count header was missing from rbdservice.list spy mock

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: fix frontend unit tests
Nizamudeen A [Tue, 28 Jun 2022 17:33:04 +0000 (23:03 +0530)]
mgr/dashboard: fix frontend unit tests

Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agomgr/dashboard: if null tablecontext then create empty one
Pere Diaz Bou [Tue, 28 Jun 2022 13:58:11 +0000 (15:58 +0200)]
mgr/dashboard: if null tablecontext then create empty one

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: more linting
Pere Diaz Bou [Fri, 17 Jun 2022 11:57:53 +0000 (13:57 +0200)]
mgr/dashboard: more linting

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: fix tests and iscsi rbd list
Pere Diaz Bou [Fri, 17 Jun 2022 11:02:28 +0000 (13:02 +0200)]
mgr/dashboard: fix tests and iscsi rbd list

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: bind serverSide to externalPaging
Pere Diaz Bou [Thu, 16 Jun 2022 09:15:35 +0000 (11:15 +0200)]
mgr/dashboard: bind serverSide to externalPaging

Count won't be used unless externalPaging is set to true.
remove paginated import

task list service needed to be updated in order to have extra
params. This commit introduces that param.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: rbd pagination poc
Pere Diaz Bou [Mon, 13 Jun 2022 15:15:45 +0000 (17:15 +0200)]
mgr/dashboard: rbd pagination poc

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agocrimson/osd: improve debugs along stopping OperationRegistry 47064/head
Radoslaw Zarzynski [Tue, 12 Jul 2022 16:55:47 +0000 (16:55 +0000)]
crimson/osd: improve debugs along stopping OperationRegistry

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/osd: stop ShardServices when stopping OSD
Radoslaw Zarzynski [Tue, 12 Jul 2022 16:52:44 +0000 (16:52 +0000)]
crimson/osd: stop ShardServices when stopping OSD

This patch fixes leaks of `ClientReqtest` instances observed
previously on OSD's stop.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #46510 from nSedrickm/accessibility-improvements
Ernesto Puerta [Tue, 12 Jul 2022 15:19:30 +0000 (17:19 +0200)]
Merge pull request #46510 from nSedrickm/accessibility-improvements

mgr/dashboard: improve accessibility of login page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoexamples/boto3: Updating Python examples to work with Python 3 46650/head
dux [Tue, 12 Jul 2022 13:29:37 +0000 (13:29 +0000)]
examples/boto3: Updating Python examples to work with Python 3

Signed-off-by: dux <faithuniterh@tutanota.com>
3 years agocephadm: update LATEST_STABLE_RELEASE 47062/head
Guillaume Abrioux [Tue, 12 Jul 2022 14:37:17 +0000 (16:37 +0200)]
cephadm: update LATEST_STABLE_RELEASE

The latest stable release of Ceph is Quincy.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoMerge pull request #46764 from Matan-B/wip-matanb-osd-cleanup
Yuri Weinstein [Tue, 12 Jul 2022 13:35:13 +0000 (06:35 -0700)]
Merge pull request #46764 from Matan-B/wip-matanb-osd-cleanup

osd: MOSDPGCreate, MOSDScrub cleanup

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
3 years agoMerge pull request #46950 from dparmar18/fix_dirfrags_doc
Venky Shankar [Tue, 12 Jul 2022 06:56:45 +0000 (12:26 +0530)]
Merge pull request #46950 from dparmar18/fix_dirfrags_doc

doc/cephfs/dirfrags: clarify the unit of threshold limits

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
3 years agocmake: add ndctl and daxctl to build pmdk 46260/head
Yin Congmin [Fri, 24 Jun 2022 16:11:18 +0000 (00:11 +0800)]
cmake: add ndctl and daxctl to build pmdk

In order to enable the pmem character device, add ndctl=y to the parameter
of compiling the pmdk library when WITH_BLUESTORE_PMEM is ON. Because
after find_ndctl and find_daxctl are added, the dependency packages
required by WITH_BLUESTORE_PMEM and WITH_RBD_RWL become more different.
So separated these two. libpmem has no version required. libpmemobj
required version >=1.8.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
3 years agocmake: add findndctl and finddaxctl function
Yin Congmin [Fri, 13 May 2022 12:47:07 +0000 (20:47 +0800)]
cmake: add findndctl and finddaxctl function

In order to support the character device of pmem usage in bluestore via
libpmem built by Ceph itself, we need to enable daxctl and ndctl
dependency. add the installation of ndctl and find it. the version of
ndctl and daxctl library requires >63. "apt-get install" meet the version
under ubuntu focal.

the installation of ndctl-devel in ceph.spec.in has not been verified.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
3 years ago.github/labeler.yml: add core label to some tools 47052/head
Neha Ojha [Mon, 11 Jul 2022 19:11:50 +0000 (19:11 +0000)]
.github/labeler.yml: add core label to some tools

Signed-off-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #47045 from zdover23/wip-doc-2022-07-12-dev-guide-dependabot-GitHu...
zdover23 [Mon, 11 Jul 2022 18:24:38 +0000 (04:24 +1000)]
Merge pull request #47045 from zdover23/wip-doc-2022-07-12-dev-guide-dependabot-GitHub-typo

doc/dev: s/github/GitHub/ in essentials.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
3 years agoMerge pull request #47031 from zdover23/wip-doc-2022-07-10-internals-cephfs-delayed...
zdover23 [Mon, 11 Jul 2022 18:17:44 +0000 (04:17 +1000)]
Merge pull request #47031 from zdover23/wip-doc-2022-07-10-internals-cephfs-delayed-deletion-edit

doc/dev: edit delayed-delete.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
3 years agodoc/dev: s/github/GitHub/ in essentials.rst 47045/head
Zac Dover [Mon, 11 Jul 2022 16:58:18 +0000 (02:58 +1000)]
doc/dev: s/github/GitHub/ in essentials.rst

This addresses a typo that Anthony D'Atri pointed out in
PR#47032. It didn't get addressed there, so it's being
addressed here.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agoMerge pull request #47032 from zdover23/wip-doc-2022-07-10-dev-guide-dependabot
zdover23 [Mon, 11 Jul 2022 16:28:28 +0000 (02:28 +1000)]
Merge pull request #47032 from zdover23/wip-doc-2022-07-10-dev-guide-dependabot

doc/dev: add Dependabot section to essentials.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
3 years agodoc/dev: add Dependabot section to essentials.rst 47032/head
Zac Dover [Sat, 9 Jul 2022 18:05:47 +0000 (04:05 +1000)]
doc/dev: add Dependabot section to essentials.rst

This PR adds a section to the Developer Guide chapter
"Essentials" that explains what Dependabot is. This
section is adapted from an email from Ernesto Puerta
to the CLT that was sent on 08 Jul 2022.

Co-authored-by: Ernesto Puerta <epuertat@redhat.com>>
Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agotools/crimson/perf_crimson_msgr: correct default addr to msgr v2 version 47039/head
luo rixin [Mon, 11 Jul 2022 02:31:55 +0000 (10:31 +0800)]
tools/crimson/perf_crimson_msgr: correct default addr to msgr v2 version

As msgr protocol V1 is no longer supported, correct default addr
to msgr v2 version and replace the option 'v1-crc-enabled' to
'crc-enabled'.

Fixes: https://tracker.ceph.com/issues/56520
Signed-off-by: luo rixin <luorixin@huawei.com>
3 years agotools/crimson/perf_crimson_msgr: init ConfigPorxy when pref_crimson_msgr start running
luo rixin [Mon, 11 Jul 2022 02:28:58 +0000 (10:28 +0800)]
tools/crimson/perf_crimson_msgr: init ConfigPorxy when pref_crimson_msgr start running

While constructing socket connections, there are some config items needed(
`ceph::msgr::v2::FrameAssembler` need the value of `common::local_conf()->ms_crc_data`,
Access `common::local_conf()->ms_crc_data` will cause SIGSEGV without the ConfigProxy init).
So adding ConfigPorxy init before pref_crimson_msgr start running.

Fixes: https://tracker.ceph.com/issues/56520
Signed-off-by: luo rixin <luorixin@huawei.com>
3 years agoMerge pull request #45519 from zhscn/wip-split-extent
Samuel Just [Mon, 11 Jul 2022 05:22:21 +0000 (22:22 -0700)]
Merge pull request #45519 from zhscn/wip-split-extent

crimson: split ObjectDataBlock when overwriting

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #47036 from tchaikov/wip-spec-with-crimson
Kefu Chai [Mon, 11 Jul 2022 03:33:53 +0000 (11:33 +0800)]
Merge pull request #47036 from tchaikov/wip-spec-with-crimson

ceph.spec.in: fixes for "with seastar"

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agotest/crimson/seastore: add test case for splitting ObjectDataBlock 45519/head
Zhang Song [Mon, 13 Jun 2022 08:56:28 +0000 (16:56 +0800)]
test/crimson/seastore: add test case for splitting ObjectDataBlock

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
3 years agotest/crimson/seastore: add test case for TransactionManager::map_existing_extent
Zhang Song [Mon, 13 Jun 2022 08:53:20 +0000 (16:53 +0800)]
test/crimson/seastore: add test case for TransactionManager::map_existing_extent

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
3 years agocrimson/os/seastore: update TransactionManager::get_extent_if_live
zhscn [Wed, 29 Jun 2022 12:26:54 +0000 (20:26 +0800)]
crimson/os/seastore: update TransactionManager::get_extent_if_live

TransactionManager::get_extents_if_live should return a list of
extents that are located in range paddr~len. When SegmentCleaner
invokes get_extents_if_live, the target extent may have been split into
multiple pieces by other transaction, so only search the paddr as key
will lose other pieces need to be rewritten.

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
3 years agocrimson/os/seastore: split ObjectDataBlock when overwriting
Zhang Song [Mon, 13 Jun 2022 08:43:01 +0000 (16:43 +0800)]
crimson/os/seastore: split ObjectDataBlock when overwriting

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
fix obj

3 years agocrimson/os/seastore: introduce TransactionManager::map_existing_extent
Zhang Song [Mon, 13 Jun 2022 08:40:54 +0000 (16:40 +0800)]
crimson/os/seastore: introduce TransactionManager::map_existing_extent

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
3 years agocrimson/os/seastore: new extent state: EXIST_CLEAN and EXIST_MUTATION_PENDING
Zhang Song [Mon, 13 Jun 2022 02:37:52 +0000 (10:37 +0800)]
crimson/os/seastore: new extent state: EXIST_CLEAN and EXIST_MUTATION_PENDING

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
3 years agoMerge pull request #47029 from tchaikov/wip-link-against-ceph-common
Kefu Chai [Mon, 11 Jul 2022 00:27:21 +0000 (08:27 +0800)]
Merge pull request #47029 from tchaikov/wip-link-against-ceph-common

cmake: link librados applications against ceph-common

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #47030 from tchaikov/wip-crimson-cleanup
Kefu Chai [Mon, 11 Jul 2022 00:26:30 +0000 (08:26 +0800)]
Merge pull request #47030 from tchaikov/wip-crimson-cleanup

crimson/osd: drop unused .cc file

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #47033 from tchaikov/wip-crimson-modified-by
Kefu Chai [Mon, 11 Jul 2022 00:26:11 +0000 (08:26 +0800)]
Merge pull request #47033 from tchaikov/wip-crimson-modified-by

crimson/osd: use a named type for user_modify

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoceph.spec.in: disable WITH_JAEGER if "with seastar" 47036/head
Kefu Chai [Mon, 11 Jul 2022 00:21:19 +0000 (20:21 -0400)]
ceph.spec.in: disable WITH_JAEGER if "with seastar"

this is an intermediate solution for building crimson rpm packages.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoceph.spec.in: enable WITH_SEASTAR if "with seastar"
Kefu Chai [Mon, 11 Jul 2022 00:19:51 +0000 (20:19 -0400)]
ceph.spec.in: enable WITH_SEASTAR if "with seastar"

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agodoc/dev: edit delayed-delete.rst 47031/head
Zac Dover [Sat, 9 Jul 2022 16:10:31 +0000 (02:10 +1000)]
doc/dev: edit delayed-delete.rst

This PR improves the English in the "CephFS Delayed
Deletion" chapter of the Ceph Internals Guide (which
is stored in the doc/dev/ directory).

There was no pressing need to rewrite this. I just saw
that it could be tightened up, and I had the fifteen
minutes I needed to do it.

Co-author: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>