]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agoMerge pull request #47054 from nSedrickm/grafana-component-accessibility-update
Pere Diaz Bou [Wed, 7 Sep 2022 08:30:00 +0000 (10:30 +0200)]
Merge pull request #47054 from nSedrickm/grafana-component-accessibility-update

mgr/dashboard: Improve level A accessibility for grafana component

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #47987 from idryomov/wip-52810
Ilya Dryomov [Tue, 6 Sep 2022 18:17:15 +0000 (20:17 +0200)]
Merge pull request #47987 from idryomov/wip-52810

librbd: retry ENOENT in V2_REFRESH_PARENT as well

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
2 years agoMerge pull request #47783 from soumyakoduri/wip-skoduri-sync-flow
Casey Bodley [Tue, 6 Sep 2022 16:29:53 +0000 (12:29 -0400)]
Merge pull request #47783 from soumyakoduri/wip-skoduri-sync-flow

radosgw-admin: fix crash with 'sync flow' cmd without 'flow-type'

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2 years agoMerge pull request #45883 from ifed01/wip-ifed-fix-bluefs-truncate
Yuri Weinstein [Tue, 6 Sep 2022 16:12:29 +0000 (09:12 -0700)]
Merge pull request #45883 from ifed01/wip-ifed-fix-bluefs-truncate

os/bluestore: fix no metadata update on truncate+fsync

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2 years agoMerge pull request #47630 from sseshasa/wip-add-mclk-cpu-util-rel-note
Sridhar Seshasayee [Tue, 6 Sep 2022 15:15:02 +0000 (20:45 +0530)]
Merge pull request #47630 from sseshasa/wip-add-mclk-cpu-util-rel-note

PendingReleaseNotes: Note the fix for high CPU utilization during recovery

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #47908 from dang/wip-dang-zipper-cleanup
Daniel Gryniewicz [Tue, 6 Sep 2022 13:48:49 +0000 (09:48 -0400)]
Merge pull request #47908 from dang/wip-dang-zipper-cleanup

RGW - Sal - remove unused default constructors

Reviewed-by: Casey Bodley <cbodley@redat.com>
2 years agoMerge pull request #47907 from cbodley/wip-57195
Casey Bodley [Tue, 6 Sep 2022 11:52:34 +0000 (07:52 -0400)]
Merge pull request #47907 from cbodley/wip-57195

rgw: fix d3n crash in StoreManager::get_config()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2 years agoMerge pull request #46823 from lightmelodies/librados-c-api-stat2
Kefu Chai [Tue, 6 Sep 2022 00:40:46 +0000 (08:40 +0800)]
Merge pull request #46823 from lightmelodies/librados-c-api-stat2

librados: export stat2 in C API

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #47962 from tchaikov/wip-3rd-time-is-a-charm
Kefu Chai [Tue, 6 Sep 2022 00:37:45 +0000 (08:37 +0800)]
Merge pull request #47962 from tchaikov/wip-3rd-time-is-a-charm

test/{librbd, rgw}: retry when bind fail with port 0

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agolibrbd: make RefreshRequest tests compatible with clone v1 47987/head
Ilya Dryomov [Sun, 4 Sep 2022 17:14:04 +0000 (19:14 +0200)]
librbd: make RefreshRequest tests compatible with clone v1

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agolibrbd: retry ENOENT in V2_REFRESH_PARENT as well
Ilya Dryomov [Sun, 4 Sep 2022 15:52:51 +0000 (17:52 +0200)]
librbd: retry ENOENT in V2_REFRESH_PARENT as well

With auto-deletion of trashed snapshots, it is relatively easy to lose
a race to "rbd flatten" as follows:

- when V2_GET_PARENT runs, the image is technically still a clone
- when V2_REFRESH_PARENT runs, the image is fully flattened and the
  snapshot in the parent image is deleted

This results in a spurious ENOENT error, mainly when trying to open the
image (e.g. for "rbd info").  This race condition has always been there
but auto-deletion of trashed snapshots makes it much worse.

Retry ENOENT in V2_REFRESH_PARENT the same way as in V2_GET_SNAPSHOTS.

Fixes: https://tracker.ceph.com/issues/52810
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agolibrbd: limit the number of ENOENT retries in RefreshRequest
Ilya Dryomov [Sun, 4 Sep 2022 10:40:36 +0000 (12:40 +0200)]
librbd: limit the number of ENOENT retries in RefreshRequest

If the image header is corrupt, ENOENT error may be persistent.  Avoid
an infinite loop in that case.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agolibrbd: fix a bunch of issues with restarting RefreshRequest
Ilya Dryomov [Fri, 2 Sep 2022 14:58:36 +0000 (16:58 +0200)]
librbd: fix a bunch of issues with restarting RefreshRequest

Make RefreshRequest properly restartable, at least up until and including
V2_REFRESH_PARENT step:

- clear m_migration_spec when skipping GET_MIGRATION_HEADER
- don't rely on potentially stale m_incomplete_update on retry
- reset m_legacy_parent when retrying more than just V2_GET_PARENT
- don't rely on potentially stale m_parent_md.overlap and
  m_head_parent_overlap on retry
- clear m_metadata before fetching image metadata (but not before
  fetching pool metadata)
- clear m_op_features when skipping V2_GET_OP_FEATURES
- clear m_group_spec on EOPNOTSUPP error in V2_GET_GROUP
- reset m_legacy_snapshot when retrying more than just V2_GET_SNAPSHOTS
- don't rely on potentially stale m_snap_parents on retry

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agolibrbd: check *result consistently in RefreshRequest
Ilya Dryomov [Tue, 30 Aug 2022 19:33:04 +0000 (21:33 +0200)]
librbd: check *result consistently in RefreshRequest

Stick to *result >= 0 checks everywhere and add missing checks for
op_features_get_finish() and image_group_get_finish() errors.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agolibrbd: reflect V2_GET_SNAPSHOTS ENOENT retry in state diagram
Ilya Dryomov [Tue, 30 Aug 2022 18:38:10 +0000 (20:38 +0200)]
librbd: reflect V2_GET_SNAPSHOTS ENOENT retry in state diagram

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #47964 from yuvalif/wip-yuval-fix-57418
Casey Bodley [Mon, 5 Sep 2022 18:17:07 +0000 (14:17 -0400)]
Merge pull request #47964 from yuvalif/wip-yuval-fix-57418

rgw/lua: handle null lua manager in librgw

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #47456 from sseshasa/wip-fix-mon-cmd-nack
Yuri Weinstein [Mon, 5 Sep 2022 18:16:00 +0000 (11:16 -0700)]
Merge pull request #47456 from sseshasa/wip-fix-mon-cmd-nack

osd: Implement Context based completion for mon cmd to set a config option

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #37767 from shun-s/add-mons-ms-tos-support
Yuri Weinstein [Mon, 5 Sep 2022 18:14:33 +0000 (11:14 -0700)]
Merge pull request #37767 from shun-s/add-mons-ms-tos-support

msg: add min delay packets support for mons

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agotest/{librbd, rgw}: retry when bind fail with port 0 47962/head 47963/head
Kefu Chai [Sun, 4 Sep 2022 12:37:32 +0000 (20:37 +0800)]
test/{librbd, rgw}: retry when bind fail with port 0

there is chance that the bind() call may fail if we have another test
happen to pick the free port picked by operating system. in this case,
we just retry up to 42 times.

in theory, this change does not fully address the racing, but it should
help to alleviate this issue.

See-also: https://tracker.ceph.com/issues/57116
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #47250 from nSedrickm/progress-bar-accessibility-update
Pere Diaz Bou [Mon, 5 Sep 2022 11:02:00 +0000 (13:02 +0200)]
Merge pull request #47250 from nSedrickm/progress-bar-accessibility-update

mgr/dashboard: Improve level A accessibility for usagebar

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #47507 from zenomri/wip-omri-fix-tracer-provider
Yuval Lifshitz [Mon, 5 Sep 2022 09:19:56 +0000 (12:19 +0300)]
Merge pull request #47507 from zenomri/wip-omri-fix-tracer-provider

common/tracer: fix for tracer provider destruction

reviewed-by: yuvalif

2 years agoMerge pull request #47905 from cbodley/wip-57326
Yuval Lifshitz [Mon, 5 Sep 2022 07:33:27 +0000 (10:33 +0300)]
Merge pull request #47905 from cbodley/wip-57326

rgw: avoid string_view to temporary in RGWBulkUploadOp

reviewed-by: dang, yuvalif

2 years agoMerge pull request #47518 from aravind-wdc/zns-wip
Yingxin [Mon, 5 Sep 2022 03:34:31 +0000 (11:34 +0800)]
Merge pull request #47518 from aravind-wdc/zns-wip

crimson/os/seastore: improve ZNS support for SeaStore

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agorgw/lua: handle null lua manager in librgw 47964/head
Yuval Lifshitz [Sun, 4 Sep 2022 13:11:36 +0000 (16:11 +0300)]
rgw/lua: handle null lua manager in librgw

Fixes: https://tracker.ceph.com/issues/57418
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2 years agoMerge pull request #47571 from rhcs-dashboard/nfs-form-squash-norequired
Nizamudeen A [Sun, 4 Sep 2022 07:30:17 +0000 (13:00 +0530)]
Merge pull request #47571 from rhcs-dashboard/nfs-form-squash-norequired

mgr/dashboard: fix nfs exports form issues with squash field

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agoMerge pull request #47634 from rhcs-dashboard/kcli-fedora-36
Nizamudeen A [Sun, 4 Sep 2022 04:26:19 +0000 (09:56 +0530)]
Merge pull request #47634 from rhcs-dashboard/kcli-fedora-36

mgr/dashboard: fix "can't read .ssh/known_hosts: No such file or directory

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #47871 from zdover23/wip-doc-2022-08-30-start-documenting-ceph...
zdover23 [Sun, 4 Sep 2022 02:10:37 +0000 (12:10 +1000)]
Merge pull request #47871 from zdover23/wip-doc-2022-08-30-start-documenting-ceph-branch-name-update

doc/start: update documenting-ceph branch names

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #47906 from ivancich/wip-bad-move-s3-getobj
J. Eric Ivancich [Sat, 3 Sep 2022 22:26:42 +0000 (18:26 -0400)]
Merge pull request #47906 from ivancich/wip-bad-move-s3-getobj

rgw: fix bug where variable referenced after data moved out

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2 years agoMerge pull request #47941 from adk3798/tp-no-overwrite-doc2
Adam King [Sat, 3 Sep 2022 19:47:21 +0000 (15:47 -0400)]
Merge pull request #47941 from adk3798/tp-no-overwrite-doc2

doc/cephadm: add info about --no-overwrite to note about tuned-profiles

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2 years agodoc/cephadm: add info about --no-overwrite to note about tuned-profiles 47941/head
Adam King [Sat, 3 Sep 2022 16:19:54 +0000 (12:19 -0400)]
doc/cephadm: add info about --no-overwrite to note about tuned-profiles

Follow up to fix to this flag in
https://github.com/ceph/ceph/commit/028cb031ddb72c1f37048c8568ecdf43f5b77b50

Signed-off-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #47733 from adk3798/prometheus-retention
Adam King [Sat, 3 Sep 2022 15:59:32 +0000 (11:59 -0400)]
Merge pull request #47733 from adk3798/prometheus-retention

mgr/cephadm: allow setting prometheus retention time

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #47815 from rkachach/fix_issue_57032
Adam King [Sat, 3 Sep 2022 15:58:01 +0000 (11:58 -0400)]
Merge pull request #47815 from rkachach/fix_issue_57032

cephadm/mgr: adding logic to handle --no-overwrite for tuned profiles

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2 years agoMerge pull request #47859 from pcuzner/fix-gather-facts-disk-size
Adam King [Sat, 3 Sep 2022 15:55:05 +0000 (11:55 -0400)]
Merge pull request #47859 from pcuzner/fix-gather-facts-disk-size

cephadm: Fix disk size calculation

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #47882 from rkachach/fix_issue_57060
Adam King [Sat, 3 Sep 2022 15:53:23 +0000 (11:53 -0400)]
Merge pull request #47882 from rkachach/fix_issue_57060

mgr/cephadm: Fix how we check if a host belongs to public network

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #47904 from cbodley/wip-57329
J. Eric Ivancich [Sat, 3 Sep 2022 15:21:11 +0000 (11:21 -0400)]
Merge pull request #47904 from cbodley/wip-57329

rgw: avoid use-after-move in RGWDataSyncSingleEntryCR ctor

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2 years agoMerge pull request #47921 from rkachach/fix_issue_56953
Adam King [Fri, 2 Sep 2022 18:36:22 +0000 (14:36 -0400)]
Merge pull request #47921 from rkachach/fix_issue_56953

doc/cephadm/services: fix example for specifying rgw placement

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #47388 from ronen-fr/wip-rf-snapper
Ronen Friedman [Fri, 2 Sep 2022 18:31:59 +0000 (21:31 +0300)]
Merge pull request #47388 from ronen-fr/wip-rf-snapper

osd/scrub: verify SnapMapper consistency

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2 years agoMerge pull request #44589 from xijiacun/lc-transition
J. Eric Ivancich [Fri, 2 Sep 2022 16:38:56 +0000 (12:38 -0400)]
Merge pull request #44589 from xijiacun/lc-transition

rgw: Get canonical storage class when storage class is empty in lifecycle configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #44610 from xijiacun/append-object
J. Eric Ivancich [Fri, 2 Sep 2022 16:37:10 +0000 (12:37 -0400)]
Merge pull request #44610 from xijiacun/append-object

rgw: Use STANDARD storage class in objects appending operation when the

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #46930 from zealot-xzd/wip-rgw-list-optimization2
J. Eric Ivancich [Fri, 2 Sep 2022 16:36:27 +0000 (12:36 -0400)]
Merge pull request #46930 from zealot-xzd/wip-rgw-list-optimization2

rgw: bucket list operation slow down in special scenario

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2 years agoMerge pull request #47831 from a16bitsysop/libaio-musl
Casey Bodley [Fri, 2 Sep 2022 16:29:48 +0000 (12:29 -0400)]
Merge pull request #47831 from a16bitsysop/libaio-musl

rgw: rgw_d3n_datacache.cc add check for musl

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agoMerge pull request #47819 from yehudasa/wip-cloud-sync-fixes
Casey Bodley [Fri, 2 Sep 2022 16:26:47 +0000 (12:26 -0400)]
Merge pull request #47819 from yehudasa/wip-cloud-sync-fixes

rgw: cloud sync fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #47527 from shraddhaghatol/shr/etag_cosmetic_change
Casey Bodley [Fri, 2 Sep 2022 15:20:10 +0000 (11:20 -0400)]
Merge pull request #47527 from shraddhaghatol/shr/etag_cosmetic_change

rgw: Added Cosmetic change in etag of copyObjectResult

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #47667 from AISHU-Technology/fix-rgw-data-corruption
Casey Bodley [Fri, 2 Sep 2022 15:18:37 +0000 (11:18 -0400)]
Merge pull request #47667 from AISHU-Technology/fix-rgw-data-corruption

rgw: fix data corruption due to network jitter

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Or Friedmann <ofriedma@redhat.com>
2 years agoMerge pull request #47818 from petrutlucian94/fix_is_err
David Galloway [Fri, 2 Sep 2022 13:41:40 +0000 (09:41 -0400)]
Merge pull request #47818 from petrutlucian94/fix_is_err

include: fix IS_ERR on Windows

2 years agoMerge pull request #47916 from rhcs-dashboard/applitools-master-main
Ernesto Puerta [Fri, 2 Sep 2022 13:05:14 +0000 (15:05 +0200)]
Merge pull request #47916 from rhcs-dashboard/applitools-master-main

mgr/dashboard: s/master/main in applitools config

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #47796 from adk3798/test_cephadm_fix
Adam King [Fri, 2 Sep 2022 12:57:15 +0000 (08:57 -0400)]
Merge pull request #47796 from adk3798/test_cephadm_fix

qa/cephadm: remove fsid dir before bootstrap in test_cephadm.sh

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #47620 from lxbsz/wip-56249
Rishabh Dave [Fri, 2 Sep 2022 12:24:22 +0000 (17:54 +0530)]
Merge pull request #47620 from lxbsz/wip-56249

client: stop the remount_finisher thread in the Client::unmount()

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #47593 from lxbsz/wip-57126
Rishabh Dave [Fri, 2 Sep 2022 12:23:07 +0000 (17:53 +0530)]
Merge pull request #47593 from lxbsz/wip-57126

client: abort the client if we couldn't invalidate dentry caches

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2 years agoMerge pull request #46988 from rishabh-d-dave/qa-suites-upgraded-client
Rishabh Dave [Fri, 2 Sep 2022 12:21:25 +0000 (17:51 +0530)]
Merge pull request #46988 from rishabh-d-dave/qa-suites-upgraded-client

qa/cephfs: clean up evicted client in 4-compat_client.yaml

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #47919 from SUSE/wip-fix-57390
Tim Serong [Fri, 2 Sep 2022 10:04:31 +0000 (20:04 +1000)]
Merge pull request #47919 from SUSE/wip-fix-57390

ceph-dencoder: Add erasure_code to denc-mod-osd's target_link_libraries

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agodoc/cephadm/services: fix example for specifying rgw placement 47921/head
Redouane Kachach [Fri, 2 Sep 2022 09:57:43 +0000 (11:57 +0200)]
doc/cephadm/services: fix example for specifying rgw placement
fixes: https://tracker.ceph.com/issues/56953

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2 years agoceph-dencoder: Add erasure_code to denc-mod-osd's target_link_libraries 47919/head
Tim Serong [Fri, 2 Sep 2022 08:44:09 +0000 (18:44 +1000)]
ceph-dencoder: Add erasure_code to denc-mod-osd's target_link_libraries

Fixes: https://tracker.ceph.com/issues/57390
Signed-off-by: Tim Serong <tserong@suse.com>
2 years agoMerge pull request #47881 from idryomov/wip-57343
Ilya Dryomov [Fri, 2 Sep 2022 08:00:23 +0000 (10:00 +0200)]
Merge pull request #47881 from idryomov/wip-57343

test/cli-integration/rbd: iSCSI REST API responses aren't pretty-printed anymore

Reviewed-by: Xiubo Li <xiubli@redhat.com>
2 years agoosd/scrub: modify SnapMapper.cc to use ceph::buffer::list 47388/head
Ronen Friedman [Tue, 23 Aug 2022 14:16:09 +0000 (14:16 +0000)]
osd/scrub: modify SnapMapper.cc to use ceph::buffer::list

... systematically, over ceph::bufferlist.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agotests/osd: creating a Teuthology test re missing SnapMapper entries
Ronen Friedman [Tue, 23 Aug 2022 05:12:18 +0000 (05:12 +0000)]
tests/osd: creating a Teuthology test re missing SnapMapper entries

The test (in the standalone/scrub suite) verifies that the scrubber
detects (and issues a cluster-log error) whenever a mapping entry
("SNA_") is missing in the SnapMapper DB.

Specifically, here the entry is corrupted - shortened as per
https://tracker.ceph.com/issues/56147.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoosd/scrub: verify SnapMapper consistency
Ronen Friedman [Mon, 1 Aug 2022 10:14:58 +0000 (10:14 +0000)]
osd/scrub: verify SnapMapper consistency

Whenever the scrubber access the SnapMapper for the snaps of a specific
clone, the mapper will now verify that the snaps have the required
mapping DB entries (the 'SNA_' keys).

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoosd/scrub: move snap_mapper_{fix/op}_t to the Scrub namespace
Ronen Friedman [Wed, 3 Aug 2022 13:04:00 +0000 (13:04 +0000)]
osd/scrub: move snap_mapper_{fix/op}_t to the Scrub namespace

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoosd/scrub: improve SnapMapper's API used by the scrubber
Ronen Friedman [Sun, 24 Jul 2022 13:25:55 +0000 (13:25 +0000)]
osd/scrub: improve SnapMapper's API used by the scrubber

By:
- defining the interface;
- avoiding 'out' parameters where possible
- (forced to) improved const correctness

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoMerge pull request #47847 from xxhdx1985126/wip-simplify-may-interrupt
Yingxin [Fri, 2 Sep 2022 07:03:34 +0000 (15:03 +0800)]
Merge pull request #47847 from xxhdx1985126/wip-simplify-may-interrupt

crimson/common/interrupt_future: simplify may_interrupt

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agomgr/dashboard: s/master/main in applitools config 47916/head
Nizamudeen A [Fri, 2 Sep 2022 05:21:41 +0000 (10:51 +0530)]
mgr/dashboard: s/master/main in applitools config

Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agocrimson/zns: fix compilation warnings for C++20. 47518/head
Aravind [Thu, 1 Sep 2022 07:31:43 +0000 (13:01 +0530)]
crimson/zns: fix compilation warnings for C++20.

Fix the following warning which is manifesting as a result of
the ceph adopting C++20.

warning: implicit capture of ‘this’ via ‘[=]’ is deprecated in C++20 [-Wdeprecated]

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agocrimson/block: Rename Device::get_size() to get_available_size()
Aravind [Thu, 1 Sep 2022 05:35:35 +0000 (11:05 +0530)]
crimson/block: Rename Device::get_size() to get_available_size()

Due to maintaining super block and other tracking information in the
disk, the entire disk size is not available, so rename the function
to represent that it actually returns available size on the device.
get_available_size() represents together free and used space available
on the device.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agocrimson/block: fix the device size calculation.
Aravind Ramesh [Tue, 30 Aug 2022 11:33:27 +0000 (17:03 +0530)]
crimson/block: fix the device size calculation.

In BlockSegmentManager, super block is updated with device size.
But a small amount of device capacity is reserved to store the
super block information and other tracking information.
Number of segments is calculated after discounting super block
size and tracking information size. This creates a mismatch
with the actual available size versus actual number of segments.

Update the available size after considering the reserved device
capacity and the number of segments and segment size.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agocrimson/zns: crimson osd crashes when device size is huge
Aravind Ramesh [Thu, 25 Aug 2022 03:44:52 +0000 (09:14 +0530)]
crimson/zns: crimson osd crashes when device size is huge

In reset_device(), if the total number of 512B sectors on the device
is more than INT_MAX then there was a overflow happening, rendering
the nr_sectors as 0, which was causing the failure of the ioctl and
subsequent crash, fix the overflow.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agocrimson/zns: Add zone-capacity support.
Aravind Ramesh [Thu, 18 Aug 2022 15:42:13 +0000 (21:12 +0530)]
crimson/zns: Add zone-capacity support.

ZNS SSDs have an attribute called zone_capacity which can be less than or
equal to zone_size. zone_capacity represents the actual writable media in
a zone. When zone_capacity is less than zone_size, writing to offsets
beyond zone_capacity will cause write errors.
Set the segment size as equal to zone_capacity, so that segment managers
writes only upto capacity of the zone/segment.
Update device size to actual available bytes so that the gc can kick in
at appropriate thresholds.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agocrimson/zns: fix the order of decoding zns_sm_metadata_t members
Aravind Ramesh [Tue, 2 Aug 2022 06:30:16 +0000 (12:00 +0530)]
crimson/zns: fix the order of decoding zns_sm_metadata_t members

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agocrimson/zns: ZNSSegmentManager::release() should reset the zone.
Aravind Ramesh [Thu, 18 Aug 2022 09:06:48 +0000 (14:36 +0530)]
crimson/zns: ZNSSegmentManager::release() should reset the zone.

For a ZNS device, a open/full zone has to be reset before it can be
reused to write from start. Seastore releases a segment/zone and marks
it empty and expects to be able to write to it from start. So as a part
of release reset the zone, so it moves to empty state on the device.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agocrimson/zns: segment_close() should finish the zone.
Aravind Ramesh [Tue, 26 Jul 2022 13:46:01 +0000 (19:16 +0530)]
crimson/zns: segment_close() should finish the zone.

Zones in IMP-OPEN, EXP-OPEN, CLOSED states in a ZNS device are
counted as active resources. ZNS SSDs can have a limit on the
number of zones that can be active at the same time (max_active_resources).
If CLOSED zones reach max_active_zones supported by the device, then
opening/writing to newer zones will fail.
So a close_segment() from Seastore is essentially a FINISH
operation on a ZNS zone.

Do FINISH operation on a zone instead of CLOSE from segment_close().

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agocrimson/zns: advance write pointer before writing tail-info.
Aravind Ramesh [Tue, 26 Jul 2022 09:52:17 +0000 (15:22 +0530)]
crimson/zns: advance write pointer before writing tail-info.

SegmentAllocator::close_segment() writes tail information to a
segment before closing the segment, and this is written at the
end of segment. However, for ZNS SSDs, the writes have to always happen
at write pointer, so writing tail info at the end of a zone fails if
the WP is not at the offset requested by close_segment().

If the write pointer is not at lba where the tail information is written,
then advance write pointer by writing zeroes to the zone from it's current
write pointer. Then write the tail information at the end of zone.

Added advance_wp() function which advances the write pointer and then write
tail information, in case of ZNS devices but for a regular device it
continues to write at the end of segment.

Do close_segment() call after writing tail information, closing a segment
first and then writing tail information can cause potential race conditions
on a zns backed segment.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
2 years agoMerge pull request #47650 from rzarzynski/wip-bug-57119
Yuri Weinstein [Thu, 1 Sep 2022 23:03:02 +0000 (16:03 -0700)]
Merge pull request #47650 from rzarzynski/wip-bug-57119

osd: fix the "heap" admin cmd printing always to error stream

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Prashant D <pdhange@redhat.com>
2 years agodoc/cephadm: documentation for setting prometheus retention time 47733/head
Adam King [Mon, 22 Aug 2022 17:47:53 +0000 (13:47 -0400)]
doc/cephadm: documentation for setting prometheus retention time

Signed-off-by: Adam King <adking@redhat.com>
2 years agomgr/cephadm: allow setting prometheus retention time
Adam King [Mon, 22 Aug 2022 15:14:12 +0000 (11:14 -0400)]
mgr/cephadm: allow setting prometheus retention time

When we deploy Prometheus server, we don't provide any
ability to define the tsdb retention time - so it defaults to 15d.

This change adds a field that can be passed in a prometheus service
spec that will be passed as an arg to the --storage.tsdb.retention.time
parameter for the prometheus daemon.

Fixes: https://tracker.ceph.com/issues/54308
Signed-off-by: Adam King <adking@redhat.com>
2 years agomgr/dashboard: Improve level A accessibility for progress bars 47250/head
nsedrickm [Sat, 23 Jul 2022 18:38:01 +0000 (19:38 +0100)]
mgr/dashboard: Improve level A accessibility for progress bars

Add titles for usage bars so that they can be picked up by screenreaders

Fixes: https://tracker.ceph.com/issues/56008
Signed-off-by: nsedrickm <nsedrick101@gmail.com>
2 years agomgr/dashboard: enable tslint a11y rules 47054/head
nsedrickm [Fri, 22 Jul 2022 09:05:27 +0000 (10:05 +0100)]
mgr/dashboard: enable tslint a11y rules

enable tslint accessibility rules

Signed-off-by: nsedrickm <nsedrick101@gmail.com>
2 years agomgr/dashboard: Improve level A accessibility for grafana component
nsedrickm [Mon, 11 Jul 2022 20:37:45 +0000 (21:37 +0100)]
mgr/dashboard: Improve level A accessibility for grafana component

*Add descriptive iframe titles for embedded grafana panels

Fixes: https://tracker.ceph.com/issues/55890
Signed-off-by: nsedrickm <nsedrick101@gmail.com>
2 years agoradosgw-admin: fix crash with 'sync flow create/remove' cmd 47783/head
Soumya Koduri [Wed, 24 Aug 2022 05:38:38 +0000 (11:08 +0530)]
radosgw-admin: fix crash with 'sync flow create/remove' cmd

Avoid dereferencing an empty optional "flow-type" (if not specified).

Fixes: https://tracker.ceph.com/issues/57275
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agoMerge pull request #47817 from rkachach/fix_issue_57304
Adam King [Thu, 1 Sep 2022 18:17:31 +0000 (14:17 -0400)]
Merge pull request #47817 from rkachach/fix_issue_57304

mgr/cephadm: allow binding to loopback for rgw daemons

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #47098 from rkachach/fix_issue_56508
Adam King [Thu, 1 Sep 2022 18:15:29 +0000 (14:15 -0400)]
Merge pull request #47098 from rkachach/fix_issue_56508

mgr/cephadm: Adding logic to store grafana cert/key per node

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Francesco Pantano <fpantano@redhat.com>
2 years agoMerge pull request #47884 from dang/wip-dang-zipper-invalidate
Daniel Gryniewicz [Thu, 1 Sep 2022 18:12:53 +0000 (14:12 -0400)]
Merge pull request #47884 from dang/wip-dang-zipper-invalidate

RGW - Zipper - Pass invaliate down from RadosStore

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #47631 from guits/bz2112730
Adam King [Thu, 1 Sep 2022 18:02:43 +0000 (14:02 -0400)]
Merge pull request #47631 from guits/bz2112730

ceph-volume: add a retry in util.disk.remove_partition

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2 years agoMerge pull request #47665 from adk3798/apply-spec-error-code
Adam King [Thu, 1 Sep 2022 18:00:00 +0000 (14:00 -0400)]
Merge pull request #47665 from adk3798/apply-spec-error-code

cephadm: return nonzero exit code when applying spec fails in bootstrap

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #47666 from adk3798/offline-no-check
Adam King [Thu, 1 Sep 2022 17:58:29 +0000 (13:58 -0400)]
Merge pull request #47666 from adk3798/offline-no-check

mgr/cephadm: don't try to write client/os tuning profiles to known offline hosts

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #47763 from phlogistonjohn/jjm-object-format-fixes
Adam King [Thu, 1 Sep 2022 17:54:13 +0000 (13:54 -0400)]
Merge pull request #47763 from phlogistonjohn/jjm-object-format-fixes

pybind/mgr: object_format.py decorator updates & docs

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #47804 from p-se/wip-7294-pse-custom-prometheus-alerts
Adam King [Thu, 1 Sep 2022 17:44:16 +0000 (13:44 -0400)]
Merge pull request #47804 from p-se/wip-7294-pse-custom-prometheus-alerts

mgr/dashboard: enable addition of custom Prometheus alerts

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2 years agoMerge pull request #47788 from dang/wip-dang-zipper-admin
Daniel Gryniewicz [Thu, 1 Sep 2022 16:57:35 +0000 (12:57 -0400)]
Merge pull request #47788 from dang/wip-dang-zipper-admin

RGW - Zipper - Remove a number of casts from rgw_admin

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Soumya Koduri <skoduri@redhat.com>
2 years agoRGW - Sal - remove unused default constructors 47908/head
Daniel Gryniewicz [Thu, 1 Sep 2022 15:37:27 +0000 (11:37 -0400)]
RGW - Sal - remove unused default constructors

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #47719 from yuvalif/wip-yuval-lua-filter2
Casey Bodley [Thu, 1 Sep 2022 15:31:31 +0000 (11:31 -0400)]
Merge pull request #47719 from yuvalif/wip-yuval-lua-filter2

rgw/lua: allow read access to object data

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #47184 from ljflores/wip-telemetry-memory-stats
Yuri Weinstein [Thu, 1 Sep 2022 15:29:25 +0000 (08:29 -0700)]
Merge pull request #47184 from ljflores/wip-telemetry-memory-stats

mgr/telemetry: add `perf_memory_metrics` collection to telemetry

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
2 years agorgw: fix d3n crash in StoreManager::get_config() 47907/head
Casey Bodley [Thu, 1 Sep 2022 15:08:06 +0000 (11:08 -0400)]
rgw: fix d3n crash in StoreManager::get_config()

config options with `type: size` are not actually `size_t` but
`uint64_t`, so accessing them with `get_val<size_t>()` leads to
a `bad_variant_access` exception. use the `Option::size_t` type
instead

Fixes: https://tracker.ceph.com/issues/57195
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agorgw: fix bug where variable referenced after data moved out 47906/head
J. Eric Ivancich [Thu, 1 Sep 2022 15:06:44 +0000 (11:06 -0400)]
rgw: fix bug where variable referenced after data moved out

Addresses RGWGetObj_ObjStore_S3::override_range_hdr using a variable
for logging after the data had been moved out. This was caught by
coverity.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2 years agorgw: avoid string_view to temporary in RGWBulkUploadOp 47905/head
Casey Bodley [Thu, 1 Sep 2022 14:50:46 +0000 (10:50 -0400)]
rgw: avoid string_view to temporary in RGWBulkUploadOp

the `else` block below constructs a temporary std::string that destructs
at the end of the statement, leaving `filename` as a dangling view:
```
filename = file_prefix + std::string(header->get_filename());
```
store a copy of the `std::string` instead

Fixes: https://tracker.ceph.com/issues/57326
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agorgw: avoid use-after-move in RGWDataSyncSingleEntryCR ctor 47904/head
Casey Bodley [Thu, 1 Sep 2022 14:43:39 +0000 (10:43 -0400)]
rgw: avoid use-after-move in RGWDataSyncSingleEntryCR ctor

the constructor body below accesses the constructor's 'obligation'
parameter instead of the member variable it was moved into

Fixes: https://tracker.ceph.com/issues/57329
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #47885 from ceph/boost-focal
David Galloway [Thu, 1 Sep 2022 14:02:07 +0000 (10:02 -0400)]
Merge pull request #47885 from ceph/boost-focal

install-deps: Actually install libboost we built for Jammy

2 years agoMerge pull request #47886 from ceph/chillout
Ernesto Puerta [Thu, 1 Sep 2022 09:18:43 +0000 (11:18 +0200)]
Merge pull request #47886 from ceph/chillout

.github: Give folks 30 seconds to fill out the checklist

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
3 years agomgr/dashboard: fix "can't read .ssh/known_hosts: No such file or directory" 47634/head
Nizamudeen A [Tue, 16 Aug 2022 14:17:13 +0000 (19:47 +0530)]
mgr/dashboard: fix "can't read .ssh/known_hosts: No such file or directory"

Updates the fedora version too

Fixes: https://tracker.ceph.com/issues/57365
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #47816 from cyx1231st/wip-seastore-split-trimmer
Yingxin [Thu, 1 Sep 2022 01:45:51 +0000 (09:45 +0800)]
Merge pull request #47816 from cyx1231st/wip-seastore-split-trimmer

crimson/os/seastore/async_cleaner: factor out journal trimming implementation

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Zhang Song <zhangsong325@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agocrimson/common/interrupt_future: simplify may_interrupt 47847/head
Xuehan Xu [Mon, 29 Aug 2022 08:07:47 +0000 (16:07 +0800)]
crimson/common/interrupt_future: simplify may_interrupt

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agoMerge pull request #47880 from ionutbalutoiu/qa-update-branch-name
David Galloway [Wed, 31 Aug 2022 18:38:34 +0000 (14:38 -0400)]
Merge pull request #47880 from ionutbalutoiu/qa-update-branch-name

qa: update branch name for Windows job

3 years ago.github: Give folks 30 seconds to fill out the checklist 47886/head
David Galloway [Wed, 31 Aug 2022 18:21:16 +0000 (14:21 -0400)]
.github: Give folks 30 seconds to fill out the checklist

Otherwise GitHub sends an annoying e-mail right away when you file a PR that doesn't have the checklist filled out.  It's easier IMO to create the PR, then check the boxes instead of putting Xes in brackets while filling out the PR comment.

Signed-off-by: David Galloway <dgallowa@redhat.com>