]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Xuehan Xu [Thu, 2 Jun 2022 13:33:57 +0000 (21:33 +0800)]
test/crimson/seastore: add test case for parallel extent retrieval
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Xuehan Xu [Wed, 1 Jun 2022 10:44:30 +0000 (18:44 +0800)]
crimson/os/seastore/cache: make access to Transaction::read_set atomic
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Xuehan Xu [Sat, 28 May 2022 08:38:30 +0000 (16:38 +0800)]
crimson/os/seastore/segment_cleaner: retrieve different live extents in parallel
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Yingxin [Fri, 27 May 2022 08:23:22 +0000 (16:23 +0800)]
Merge pull request #46356 from xxhdx1985126/wip-backref-gc-opt
crimson/os/seastore/segment_cleaner: add dedicated backref trimming process
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Venky Shankar [Fri, 27 May 2022 05:43:43 +0000 (11:13 +0530)]
Merge pull request #45544 from wxypro/fix-typos-cephfs
common/options: fix typo
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Xuehan Xu [Fri, 20 May 2022 09:23:03 +0000 (17:23 +0800)]
crimson/os/seastore/segment_cleaner: add dedicated backref trimming process
Space reclamation needs to merge backrefs up to the point where the latest
release of extents within the scope of the reclamation process happened.
When the journal size is large, that merge may generate a transaction
record with size exceeds the max record size threshold. So we need have a
backref trimming process that merge most of the backrefs before the space
reclamation happens.
This commit also fixes issue: https://tracker.ceph.com/issues/55692, by
repeating the inflight backrefs trimming transaction when it's
invalidated by other trans on the ROOT block
Fixes: https://tracker.ceph.com/issues/55692
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Yingxin [Thu, 26 May 2022 02:24:37 +0000 (10:24 +0800)]
Merge pull request #46271 from xxhdx1985126/wip-backref-new_mapping-opt
crimson/os/seastore: backref related clean ups
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Patrick Donnelly [Wed, 25 May 2022 20:01:57 +0000 (16:01 -0400)]
Merge commit '
b76b6ea16899861d6c266797b3c11d9f68f924d7 ' into main
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
David Galloway [Wed, 25 May 2022 19:47:07 +0000 (15:47 -0400)]
Merge pull request #46390 from gabriellasroman/inclusive-lang
teuthology-cronjobs: master->main
Gabriella [Wed, 25 May 2022 12:01:11 +0000 (08:01 -0400)]
teuthology-cronjobs: master->main
Signed-off-by: Gabriella Roman <gsroman@bu.edu>
Venky Shankar [Wed, 25 May 2022 01:24:53 +0000 (06:54 +0530)]
Merge pull request #44486 from batrick/fs-workload-kclient-switches
qa: reduce fs:workload use of random selection
Reviewed-by: Venky Shankar <vshankar@redhat.com>
David Galloway [Tue, 24 May 2022 21:04:50 +0000 (17:04 -0400)]
Merge pull request #46385 from ceph/inclusive-rename-yaml
Rename/re-symlink whitelist_*.yaml
Zack Cerza [Tue, 24 May 2022 19:31:12 +0000 (13:31 -0600)]
Rename/re-symlink whitelist_*.yaml
whitelist_health.yaml -> ignorelist_health.yaml
whitelist_wrongly_marked_down.yaml -> ignore_wrongly_marked_down.yaml
This was mostly addressed in
https://github.com/ceph/ceph/commit/
2ee9365d0bc220523632753f02c61eb5c2f4fbdd ,
but the rename wasn't done there.
Signed-off-by: Zack Cerza <zack@cerza.org>
Zack Cerza [Tue, 24 May 2022 16:23:48 +0000 (10:23 -0600)]
Merge pull request #44967 from batrick/nested-subset-doc
doc/dev: document nested subsets in teuthology
Yuri Weinstein [Tue, 24 May 2022 15:02:26 +0000 (08:02 -0700)]
Merge pull request #46099 from NitzanMordhai/wip-nitzan-rados-test-delete-pools-parallel
systest: fix race with pool delete after socket injection
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Tue, 24 May 2022 15:01:45 +0000 (08:01 -0700)]
Merge pull request #45825 from NitzanMordhai/wip-nitzan-watch-notify-reconnect-107
librados/watch_notify: reconnect after socket injection
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Yuri Weinstein [Tue, 24 May 2022 15:01:19 +0000 (08:01 -0700)]
Merge pull request #45805 from ifed01/wip-ifed-more-stats
os/bluestore: more debug information from BlueStore
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Yuri Weinstein [Tue, 24 May 2022 14:59:36 +0000 (07:59 -0700)]
Merge pull request #45409 from NUABO/tancz
mon: add proxy to cache tier options
Reviewed-by: Neha Ojha <nojha@redhat.com>
Adam King [Tue, 24 May 2022 14:27:03 +0000 (10:27 -0400)]
Merge pull request #46324 from ceph/dgalloway-nodockerio
qa: Pull from quay.io not docker.io
Reviewed-by: Adam King <adking@redhat.com>
kalebskeithley [Tue, 24 May 2022 11:06:50 +0000 (07:06 -0400)]
Merge pull request #46366 from kalebskeithley/rocksdb
rocksdb: build with rocksdb-7.y.z
Rishabh Dave [Tue, 24 May 2022 10:43:48 +0000 (16:13 +0530)]
Merge pull request #46185 from dparmar18/fix_rm_error_output
cephfs-shell: fix rm cmd's error output
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Rishabh Dave [Tue, 24 May 2022 10:40:26 +0000 (16:10 +0530)]
Merge pull request #46160 from dparmar18/fix_tracker_40860
cephfs-shell: fix rmdir cmd
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Ernesto Puerta [Tue, 24 May 2022 10:38:14 +0000 (12:38 +0200)]
Merge pull request #46249 from rhcs-dashboard/rbd-mirroring-commands
mgr/dashboard: enable/disable/promote/demote/resync mirroring images
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Xuehan Xu [Sun, 15 May 2022 23:59:54 +0000 (07:59 +0800)]
crimson/os/seastore/segment_cleaner: downgrade the level of unnecessary logs to DEBUG
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Xuehan Xu [Sun, 15 May 2022 07:24:03 +0000 (15:24 +0800)]
crimson/os/seastore/backref: rename BackrefManager::batch_insert_from_cache to BackrefManager::merge_cached_backrefs
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Xuehan Xu [Sun, 15 May 2022 06:45:51 +0000 (14:45 +0800)]
crimson/os/seastore/segment_cleaner: don't repeat backref mapping retrieval when gc transactions get invalidated
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Xuehan Xu [Fri, 13 May 2022 08:50:10 +0000 (16:50 +0800)]
crimson/os/seastore: mandate all access to backrefs to go through backref manager
this would avoid other components' unnecessary dependency on Cache
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Venky Shankar [Tue, 24 May 2022 02:24:16 +0000 (07:54 +0530)]
Merge pull request #45728 from lxbsz/wip-55144
client: stop retrying the request when exceeding 256 times
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Venky Shankar [Tue, 24 May 2022 01:43:32 +0000 (07:13 +0530)]
Merge pull request #46034 from mchangir/mgr-snap_schedule-remove-subvolume-support
mgr/snap_schedule: remove subvol interface
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yingxin [Tue, 24 May 2022 01:34:48 +0000 (09:34 +0800)]
Merge pull request #46351 from cyx1231st/wip-seastore-cleaner-cleanup
crimson/os/seastore/segment_cleaner: minor cleanups
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Venky Shankar [Tue, 24 May 2022 01:26:13 +0000 (06:56 +0530)]
Merge pull request #46314 from rishabh-d-dave/qa-fs-fsid
qa/vstart_runner: store FSID in LocalContext instances
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Kaleb S. KEITHLEY [Mon, 23 May 2022 11:41:26 +0000 (07:41 -0400)]
rocksdb: build with rocksdb-7.y.z
RocksDB 7, specifically 7.2.2 has landed in Fedora 37/rawhide.
https://tracker.ceph.com/issues/55730
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Neha Ojha [Mon, 23 May 2022 23:26:27 +0000 (16:26 -0700)]
Merge pull request #46346 from ljflores/wip-lflores-testing-recovery
qa/suites/rados/thrash-erasure-code-big/thrashers: add `osd max backfills` setting to mapgap and pggrow
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Vikhyat Umrao [Mon, 23 May 2022 19:49:35 +0000 (12:49 -0700)]
Merge pull request #46124 from SMIL-Infra/reopen_cluster_logs
mon/LogMonitor: reopen log files on SIGHUP
Ernesto Puerta [Mon, 23 May 2022 18:30:08 +0000 (20:30 +0200)]
Merge pull request #46233 from rhcs-dashboard/fix-overlapping-form-icons
mgr/dashboard: form field validation icons overlap with other icons
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Casey Bodley [Mon, 23 May 2022 16:55:41 +0000 (12:55 -0400)]
Merge pull request #45985 from soumyakoduri/wip-skoduri-dbstore-tests
rgw/qa: Run s3tests on dbstore backend in teuthology
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Pere Diaz Bou [Thu, 12 May 2022 18:29:01 +0000 (20:29 +0200)]
mgr/dashboard: expose image mirroring commands as endpoints
Expose:
- enable/disable mirroring in image
- promote/demote (primary and non-primary)
- resync
- snapshot mode:
- mirror image snapshot (manual snapshot)
- schedule
Fixes: https://tracker.ceph.com/issues/55645
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
Xiubo Li [Thu, 31 Mar 2022 07:16:49 +0000 (15:16 +0800)]
client: stop retrying the request when exceeding 256 times
The type of 'retry_attempt' in 'MetaRequest' is 'int', while in
'ceph_mds_request_head' the type of 'num_retry' is '__u8'. So in
case the request retries exceeding 256 times, the MDS will receive
a incorrect retry seq.
In this case it's ususally a bug in MDS and continue retrying the
request makes no sense. For now let's limit it to 256. In future
this could be fixed in ceph code, so avoid using the hardcode here.
Fixes: https://tracker.ceph.com/issues/55144
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Venky Shankar [Mon, 23 May 2022 05:57:33 +0000 (11:27 +0530)]
Merge pull request #45844 from lxbsz/wip-55253
client: switch AT_NO_ATTR_SYNC to AT_STATX_DONT_SYNC
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Venky Shankar [Mon, 23 May 2022 05:35:03 +0000 (11:05 +0530)]
Merge pull request #46337 from rishabh-d-dave/missing-arg-exit-code
cephfs-shell: fixes related to Cmd2ArgparseError
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Soumya Koduri [Thu, 5 May 2022 18:33:56 +0000 (00:03 +0530)]
rgw/qa: Run s3tests on dbstore backend in teuthology
Add a new test-suite to run s3tests on dbstore backend.
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Kefu Chai [Sat, 21 May 2022 12:27:06 +0000 (20:27 +0800)]
Merge pull request #44264 from tchaikov/wip-mipel
cmake: do not use GCC extension when detecting 16-byte atomic op
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Sat, 21 May 2022 12:10:30 +0000 (20:10 +0800)]
Merge pull request #45835 from tchaikov/wip-55189
common/ceph_json: dump bool using f->dump_bool()
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Kefu Chai [Sat, 21 May 2022 09:34:02 +0000 (17:34 +0800)]
Merge pull request #46355 from tchaikov/wip-no-annobin
ceph.spec.in: disable annobin plugin if compile with gcc-toolset
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Kefu Chai [Sun, 10 Apr 2022 01:23:59 +0000 (09:23 +0800)]
common/ceph_json: dump bool using f->dump_bool()
as per https://www.json.org/json-en.html, JSON encodes bool as
"true" or "false", without the quotes. before this change, the quotes
are always added when encoding boolean values.
but this change is not backward compatible.
encode_json()'s bool overload is used by rgw. it uses JSONObj
defined in common/ceph_json.h to decode JSON-encoded structs.
and it does not differentiate bool from str when decoding a boolean
value despite that it could have check the "quoted" member variable
of JSONObj for validating the type of value. so we should be fine.
Fixes: https://tracker.ceph.com/issues/55189
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Ronen Friedman [Sat, 21 May 2022 07:03:17 +0000 (10:03 +0300)]
Merge pull request #46293 from ronen-fr/wip-rf-sched-test
test/osd: unit-tests for the scrubber scheduler
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Adam King [Fri, 20 May 2022 22:36:24 +0000 (18:36 -0400)]
Merge pull request #45467 from phlogistonjohn/jjm-format-exp2
A module and decorator for generically handling format= in python mgr modules
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Adam King [Fri, 20 May 2022 22:33:36 +0000 (18:33 -0400)]
Merge pull request #45786 from adk3798/staggered-upgrade
mgr/cephadm: staggered upgrade
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Kefu Chai [Fri, 20 May 2022 06:25:10 +0000 (14:25 +0800)]
ceph.spec.in: disable annobin plugin if compile with gcc-toolset
we are seeing FTBFS failures when compiling the packages with seastar
enabled, like:
Package gcc-toolset-9-annobin-9.08-4.el8.x86_64 is already installed.
...
Package redhat-rpm-config-130-1.el8.noarch is already installed.
...
CFLAGS=-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
...
-- The CXX compiler identification is GNU 9.2.1
....
Building CXX object CMakeFiles/cmTC_386fb.dir/testCXXCompiler.cxx.o
/opt/rh/gcc-toolset-9/root/usr/bin/c++ -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -o CMakeFiles/cmTC_386fb.dir/testCXXCompiler.cxx.o -c /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/
17.2.0-209-g3e01cd41 /rpm/el8/BUILD/
ceph-17.2.0-209-g3e01cd41 /x86_64-redhat-linux-gnu/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
cc1plus: fatal error: inaccessible plugin file /opt/rh/gcc-toolset-9/root/usr/lib/gcc/x86_64-redhat-linux/9/plugin/gcc-annobin.so expanded from short plugin name gcc-annobin: No such file or directory
the root cause is that redhat-rpm-config expects gcc-annobin:
$ cat /usr/lib/rpm/redhat/redhat-annobin-cc1 *cc1_options: + %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()} -fplugin=gcc-annobin}
while this plugin is not installed by gcc-toolset-9-annobin, which installs annobin.so though:
$ rpm -ql gcc-toolset-9-annobin| grep annobin.so /opt/rh/gcc-toolset-9/root/usr/lib/gcc/x86_64-redhat-linux/9/plugin/annobin.so /opt/rh/gcc-toolset-9/root/usr/lib/gcc/x86_64-redhat-linux/9/plugin/annobin.so.0 /opt/rh/gcc-toolset-9/root/usr/lib/gcc/x86_64-redhat-linux/9/plugin/annobin.so.0.0.0
but gcc-toolset-8-annobin provides this file. upgrading to
gcc-toolset-11 does not help. see https://centos.pkgs.org/8-stream/centos-appstream-x86_64/gcc-toolset-11-annobin-plugin-gcc-10.23-1.el8.x86_64.rpm.html
so, the intermediate solution would be to disable the plugin, if
we want to use gcc-toolset to build rpm packages.
in this change, _annotated_build is undefined to prevent the compiler
from adding extra information to the binary. in general this change
shuold be safe, without these information, it'd be hard to tell if
the binary is hardened or what ABI version it expects. see
also https://fedoraproject.org/wiki/Changes/Annobin
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Rishabh Dave [Thu, 19 May 2022 18:29:25 +0000 (23:59 +0530)]
qa/cephfs: remove temporary files
These temporary files don't matter for test execution with teuthology
but they do matter for execution with vstart_runner.py since the test
fails if these files exist already. And tests are often run repeatedly
with vstart_runner.py, unlike with teuthology.
Fixes: https://tracker.ceph.com/issues/55719
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Yingxin Cheng [Thu, 19 May 2022 07:44:16 +0000 (15:44 +0800)]
crimson/os/seastore/cache: ignore maybe unused variables
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 20 May 2022 03:04:09 +0000 (11:04 +0800)]
crimson/os/seastore/segment_cleaner: cleanup logs
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 20 May 2022 02:30:11 +0000 (10:30 +0800)]
crimson/os/seastore/segment_cleaner: move implementations to cc
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Samuel Just [Fri, 20 May 2022 02:16:29 +0000 (19:16 -0700)]
Merge pull request #46335 from cyx1231st/wip-seastore-onode-fix-unaligned-access
crimson/onode-staged-tree: fix unaligned reference to shard_pool_t::pool
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 20 May 2022 02:14:29 +0000 (19:14 -0700)]
Merge pull request #46333 from athanatos/sjust/wip-55705
crimson: generalize journal_seq_t to tolerate different paddr_t types
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Samuel Just [Fri, 20 May 2022 02:11:44 +0000 (19:11 -0700)]
Merge pull request #46310 from cyx1231st/wip-seastore-tune-cleaner
crimson/os/seastore: increase journal size and decrease rewrite-dirty size
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Kefu Chai [Thu, 19 May 2022 23:55:39 +0000 (07:55 +0800)]
Merge pull request #46122 from tchaikov/wip-pmem
blk/pmem: refactor pmem_check_file_type() using std::filesystem
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Laura Flores [Mon, 16 May 2022 22:59:42 +0000 (17:59 -0500)]
qa/suites/rados/thrash-erasure-code-big/thrashers: add `osd max backfills` setting to mapgap and pggrow
All `rados/thrash-erasure-code-big` tests that die due to the “wait_for_recovery” timeout have one thing in common: They contain either `thrashers/pggrow` or `thrashers/mapgap`.
The difference between pggrow and mapgap vs. all other non-offending thrashers (default, careful, fastread, and morepggrow) is that they lack an override setting for `osd max backfills`. `osd max backfills` is the max number of backfill operations allowed to/from an OSD. The higher the number, the quicker the recovery. By default, this value is 1. On all of the non-offending thrashers (default, careful, fastread, and morepggrow), the default 1 value gets overridden in their .yaml files with a value > 1. This is not the case for pggrow and mapgap, however, as they lack an `osd max backfills` override setting.
The mclock op scheduler is known to override `osd max backfills` with a high value, but all of the thrash-erasure-code-big thrashers have their op queue set to “debug_random”, which chooses randomly between op queues (the debug_random op queue is set to override the default mclock_scheduler in qa/config/rados.yaml). So, coupled with the “debug_random” op queue, the low `osd max backfill` setting is causing some tests to time out in recovery.
WITHOUT `osd max backfills`, as they are now, “mapgap” and “pggrow” tests die due to timed-out recovery about 17/100 times, as seen here with a pggrow test: http://pulpito.front.sepia.ceph.com/lflores-2022-05-18_14:24:29-rados:thrash-erasure-code-big-master-distro-default-smithi/
WITH `osd max backfills` specified, as I have suggested in this PR, 99/100 tests passed, with one test failing for a different reason:
http://pulpito.front.sepia.ceph.com/lflores-2022-05-17_22:40:27-rados:thrash-erasure-code-big-master-distro-default-smithi/
I also scheduled 145 tests WITH `osd max backfills` that are a mix of pggrow and mapgap thrashers. 144/145 tests passed, with one test failing for a different reason. http://pulpito.front.sepia.ceph.com/lflores-2022-05-17_15:27:54-rados:thrash-erasure-code-big-master-distro-default-smithi/
Fixes: https://tracker.ceph.com/issues/51076
Signed-off-by: Laura Flores <lflores@redhat.com>
Radoslaw Zarzynski [Thu, 19 May 2022 22:08:57 +0000 (00:08 +0200)]
Merge pull request #45562 from tchaikov/wip-vstart-with-pmem
vstart.sh: add options to enable PMEM bluestore backend
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Adam King [Tue, 19 Apr 2022 17:20:45 +0000 (13:20 -0400)]
doc/cephadm: staggered upgrade docs
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Wed, 13 Apr 2022 04:36:02 +0000 (00:36 -0400)]
mgr/cephadm: unit test for staggered upgrade param validation
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Tue, 12 Apr 2022 16:39:26 +0000 (12:39 -0400)]
qa/suites/orch/cephadm: staggered upgrade test
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Tue, 5 Apr 2022 21:38:20 +0000 (17:38 -0400)]
mgr/cephadm: autopep8 changes
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Fri, 1 Apr 2022 13:41:01 +0000 (09:41 -0400)]
mgr/cephadm: make use of new upgrade control parameters
Fixes: https://tracker.ceph.com/issues/54135
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Fri, 1 Apr 2022 12:20:28 +0000 (08:20 -0400)]
mgr/cephadm: make UpgradeState from_json a bit safer
This way, for downgrades to whatever versions
this lands in onward, having added new parameters to
UpgradeState shouldn't break anything. Can't do much
about downgrades to older versions from this one
but this should help in the future.
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Wed, 30 Mar 2022 13:49:56 +0000 (09:49 -0400)]
mgr/cephadm: add new args and validation for staggered upgrade
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Mon, 28 Mar 2022 16:10:15 +0000 (12:10 -0400)]
mgr/cephadm: split _do_upgrade into sub functions
This function was around 500 lines and difficult to work
with. Splitting it into sub functions should hopefully make
it a bit easier to understand and make changes to.
Signed-off-by: Adam King <adking@redhat.com>
Adam King [Thu, 19 May 2022 21:03:05 +0000 (17:03 -0400)]
Merge pull request #45425 from rkachach/fix_issue_54309
mgr/cephadm: Adding prometheus service discovery endpoints
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Adam King [Thu, 19 May 2022 20:08:21 +0000 (16:08 -0400)]
Merge pull request #46301 from rkachach/fix_issue_55685
mgr/cephadm: adding support to copy ceph conf to per fsid config location
Reviewed-by: Adam King <adking@redhat.com>
Adam King [Thu, 19 May 2022 19:42:27 +0000 (15:42 -0400)]
Merge pull request #46054 from rkachach/fix_issue_55466
mgr/cephadm: fix unit.stop for ceph daemons
Reviewed-by: Adam King <adking@redhat.com>
Adam King [Thu, 19 May 2022 18:19:17 +0000 (14:19 -0400)]
Merge pull request #46129 from rkachach/fix_issue_ssh
mgr/cephadm: Adding an early ssh connectivity check during bootsrap
Reviewed-by: Adam King <adking@redhat.com>
Ernesto Puerta [Thu, 19 May 2022 18:11:42 +0000 (20:11 +0200)]
Merge pull request #45951 from rhcs-dashboard/feature-set-login-banner
mgr/dashboard: customizable log-in page text/banner
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Rishabh Dave [Thu, 19 May 2022 15:33:54 +0000 (21:03 +0530)]
cephfs-shell: check version before importing Cmd2ArgparseError
Cmd2ArgparseError is available only cmd2 version 1.0.1 onwards. Before
that, SystemExit(2) is raised. This commit creates an empty class
Cmd2ArgparseError for earlier version so that similar error won't creep
up again.
Fixes: https://tracker.ceph.com/issues/55716
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Casey Bodley [Thu, 19 May 2022 14:37:50 +0000 (10:37 -0400)]
Merge pull request #46067 from sumedhak27/fix/user-policy/response-correction
rgw_rest_user_policy: Fix GetUserPolicy & ListUserPolicies responses
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Rishabh Dave [Thu, 19 May 2022 12:37:39 +0000 (18:07 +0530)]
cephfs-shell: set exit code when Cmd2ArgparseError is caught
Not doing so, sets the exit code to zero which is not desired in case of
a command failure.
Fixes: https://tracker.ceph.com/issues/55710
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Soumya Koduri [Thu, 19 May 2022 10:23:54 +0000 (15:53 +0530)]
Merge pull request #46008 from soumyakoduri/wip-skoduri-cloudtier-tests
rgw/qa: enable s3-tests related to cloud-transition feature
Reviewed-by: casey Bodley <cbodley@redhat.com>
Reviewed-by: Maredia, Ali <amaredia@redhat.com>
Yingxin Cheng [Wed, 18 May 2022 02:22:21 +0000 (10:22 +0800)]
crimson/os/seastore: increase journal size and decrease rewrite-dirty size
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin [Thu, 19 May 2022 08:40:15 +0000 (16:40 +0800)]
Merge pull request #46334 from athanatos/sjust/wip-55706
crimson: clarify use of tm_make_config_t
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Soumya Koduri [Fri, 6 May 2022 17:10:12 +0000 (22:40 +0530)]
rgw/qa: Run tests on multiple cloudtier config
Run cloudtier tests with parameter 'retain_head_object'
set to true and false.
However having multiple cloudtier storage classes in the same task
is increasing the transition time and resulting in spurious failures.
Hence until there is a consistent way of running the tests, without
having to depend on lc_debug_interval, disabled one of the config for
now.
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Samuel Just [Thu, 19 May 2022 04:39:02 +0000 (04:39 +0000)]
crimson: generalize journal_seq_t to tolerate different paddr_t types
Introduced:
bdc1c1490ff11269c7808e762893fcbdea3a0503
Fixes: https://tracker.ceph.com/issues/55706
Signed-off-by: Samuel Just <sjust@redhat.com>
Yingxin Cheng [Thu, 19 May 2022 07:15:43 +0000 (15:15 +0800)]
crimson/onode-staged-tree: fix unaligned reference to shard_pool_t::pool
../src/crimson/os/seastore/onode_manager/staged-fltree/stages/key_layout.h:844:44:
runtime error: reference binding to misaligned address 0x6250013ee905
for type 'const crimson::os::seastore::onode::pool_t' (aka 'const
long'), which requires 8 byte alignment
from UndefinedBehaviorSanitizer
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Samuel Just [Thu, 19 May 2022 05:15:51 +0000 (05:15 +0000)]
crimson: clarify use of tm_make_config_t
Fixes: https://tracker.ceph.com/issues/55706
Introduced:
9fe59429fc2ef17e2bec329109713eb63ffc74c9
Signed-off-by: Samuel Just <sjust@redhat.com>
Ronen Friedman [Thu, 12 May 2022 15:46:23 +0000 (15:46 +0000)]
test/osd: unit-tests for the scrubber scheduler
Adding unittest_scrub_sched, testing the way scrubs are scheduled.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
David Galloway [Thu, 19 May 2022 04:17:54 +0000 (00:17 -0400)]
Merge pull request #46329 from ceph/1629
doc: 16.2.9 Release notes
Samuel Just [Thu, 19 May 2022 04:05:02 +0000 (21:05 -0700)]
Merge pull request #44233 from myoungwon/wip-add-rbm-inline-outline-alloc
seastore: add CircularBoundedJournal
Reviewed-by: Samuel Just <sjust@redhat.com>
myoungwon [Mon, 16 May 2022 04:41:00 +0000 (13:41 +0900)]
seastore/cbjournal: add a log if crc is mismatched
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Fri, 13 May 2022 08:28:09 +0000 (17:28 +0900)]
seastore/cbjournal: add assert(initialized), revmoving open_device_read_header()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Fri, 13 May 2022 07:47:41 +0000 (16:47 +0900)]
seastore/cbjournal: rename init to initialized
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Fri, 13 May 2022 07:41:07 +0000 (16:41 +0900)]
seastore/cbjournal: move written_to from the header to CBJournal
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Fri, 13 May 2022 07:30:51 +0000 (16:30 +0900)]
seastore/cbjournal: remove header_checksum field and add calculating header checkksum
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Fri, 13 May 2022 06:38:49 +0000 (15:38 +0900)]
seastore: use config value and avoid passing a hint
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 07:58:40 +0000 (16:58 +0900)]
seastore: rename journal_type to journal_type_t
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 07:45:35 +0000 (16:45 +0900)]
test/cbjournal: do rb_device.reset() unconditionally
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 07:32:34 +0000 (16:32 +0900)]
test/seastore: use tm_make_config_t to get default_placement_hint
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 06:33:50 +0000 (15:33 +0900)]
test/seastore: return journal type with less lines
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 06:15:21 +0000 (15:15 +0900)]
seastore: add a log line after set_disable_trim is called
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 05:59:42 +0000 (14:59 +0900)]
seastore: add the comment to explain disable_trim
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 05:56:38 +0000 (14:56 +0900)]
seastore: add space between ) and {
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 05:32:23 +0000 (14:32 +0900)]
seastore: add a comment (blk_paddr_t is absolute)
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Thu, 12 May 2022 05:24:37 +0000 (14:24 +0900)]
seastore: do assert if paddr type is not SEGMENT in add_relative
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>