]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Kefu Chai [Sun, 29 Aug 2021 23:42:53 +0000 (07:42 +0800)]
Merge pull request #42961 from tchaikov/wip-docker-test
test/docker-test.sh: add support for fedora 34 and debian stretch
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Kefu Chai [Sun, 29 Aug 2021 13:43:23 +0000 (21:43 +0800)]
test/docker-test.sh: add support for fedora 34 and debian strech
and drop fedora-31, fedora32, debian-jessie and ubuntu-16.04 since
they are not maintained anymore. see
- https://fedoraproject.org/wiki/End_of_life
- https://wiki.debian.org/LTS
- https://wiki.ubuntu.com/Releases
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Samuel Just [Sat, 28 Aug 2021 07:14:22 +0000 (00:14 -0700)]
Merge pull request #42957 from athanatos/sjust/wip-lba-rewrite-working
crimson: add object_info, log support to crimson-store-nbd, fix lba btree bugs
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Samuel Just [Fri, 27 Aug 2021 21:19:54 +0000 (21:19 +0000)]
crimson/os/seastore/lba_manager/btree: don't assert deltas are empty
We use the contents to update counters when transactions conflict,
deltas may validly be empty there.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 27 Aug 2021 20:56:42 +0000 (13:56 -0700)]
crimson/os/seastore/cache: retire_extent_addr: add addr asserts
retire_extent_addr can only be called on absolute or record-relative
addrs. Record-relative addrs are only valid on extents allocated as
part of the current transaction.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 27 Aug 2021 05:42:38 +0000 (05:42 +0000)]
crimson/os/seastore/lba_manager/btree: fix get_val() paddr value from iterator
This was causing a stray RETIRED_PLACEHOLDER to be created resulting in
a segment cleaner segfault in release on an invalid segment and a crash
upon adding a duplicate lba btree pin since the returned addr didn't
match the addr of the INITIAL_PENDING extent on the transaction.
Fixes: https://tracker.ceph.com/issues/52434
Fixes: https://tracker.ceph.com/issues/52435
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 27 Aug 2021 21:41:22 +0000 (14:41 -0700)]
crimson/tools/store_nbd: add log emulation to fs_driver
Signed-off-by: Samuel Just <sjust@redhat.com>
Patrick Donnelly [Sat, 28 Aug 2021 01:31:04 +0000 (21:31 -0400)]
Merge PR #42899 into master
* refs/pull/42899/head:
src/client: fix dump mds twice
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Sat, 28 Aug 2021 01:30:22 +0000 (21:30 -0400)]
Merge PR #42751 into master
* refs/pull/42751/head:
cephfs-mirror: shutdown ClusterWatcher on termination
Reviewed-by: Jos Collin <jcollin@redhat.com>
Patrick Donnelly [Sat, 28 Aug 2021 01:29:17 +0000 (21:29 -0400)]
Merge PR #42687 into master
* refs/pull/42687/head:
qa: test the "ms_mode" options in kclient workflows
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Sat, 28 Aug 2021 01:28:02 +0000 (21:28 -0400)]
Merge PR #42620 into master
* refs/pull/42620/head:
mds: switch mds_lock to fair mutex
common/Timer: make SafeTimer a template
common/fair_mutex: add is_locked_by_me support
common: do not compile condition_variable_debug in none debug mode
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Sat, 28 Aug 2021 01:26:41 +0000 (21:26 -0400)]
Merge PR #38481 into master
* refs/pull/38481/head:
qa/vstart_runner: inherit methods instead of duplicating them
qa/ceph_manager: make it possible to reuse few methods
qa/vstart_runner: don't use "shell=False" in run_ceph_w()
qa/ceph_manager: minor refactor
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Kefu Chai [Sat, 28 Aug 2021 00:08:48 +0000 (08:08 +0800)]
Merge pull request #42955 from tchaikov/wip-fio-tag
cmake: use ceph repo with tag
Reviewed-by: Neha Ojha <nojha@redhat.com>
Samuel Just [Thu, 26 Aug 2021 20:48:30 +0000 (20:48 +0000)]
crimson/tools/store_nbd: add object_info emulation to fs_driver
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 27 Aug 2021 05:03:30 +0000 (22:03 -0700)]
crimson/common/interruptible_future: move int future refcounting to trace
We probably want to add a dedicated subsys for this, but for now it's just
too much for debug level.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 27 Aug 2021 02:10:26 +0000 (19:10 -0700)]
crimson/tools/store_nbd/fs_driver: collid should be the msb
This way, objects in a collection will sort together. Probably
a typo.
Signed-off-by: Samuel Just <sjust@redhat.com>
Neha Ojha [Fri, 27 Aug 2021 18:30:30 +0000 (11:30 -0700)]
Merge pull request #39377 from ifed01/wip-ifed-fix-huge-omap-rename
os/bluestore: cap omap naming scheme upgrade transaction
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Neha Ojha [Fri, 27 Aug 2021 18:29:20 +0000 (11:29 -0700)]
Merge pull request #42918 from ifed01/wip-ifed-fix-52398
test/store_test: fix broken test case
Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Ilya Dryomov [Fri, 27 Aug 2021 17:04:05 +0000 (19:04 +0200)]
Merge pull request #42926 from zdover23/wip-doc-2021-08-26-nomad-cleanup-cherry-picker
doc: clean rbd-nomad.rst
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Kefu Chai [Fri, 27 Aug 2021 16:14:55 +0000 (00:14 +0800)]
cmake: use ceph repo with tag
to address following warning:
fatal: reference is not a tree:
7f02f21f53ccd5e2448086f8e9015489693dd2dc
CMake Error at /home/jenkins-build/build/workspace/ceph-pull-requests/build/fio_ext-prefix/tmp/fio_ext-gitclone.cmake:40 (message):
Failed to checkout tag: '
7f02f21f53ccd5e2448086f8e9015489693dd2dc '
it seems that the shallow option does not work with a sha1 tag option,
let's continue using the ceph repo with a tag.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Kefu Chai [Fri, 27 Aug 2021 13:48:18 +0000 (21:48 +0800)]
Merge pull request #42944 from dengchl01/dengchl01
mgr/restful/api/crush.py: delete unused reference
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Daniel Gryniewicz [Fri, 27 Aug 2021 13:47:43 +0000 (09:47 -0400)]
Merge pull request #39933 from linuxbox2/wip-bl-opslog
rgw: permit logging of list-bucket (and any other no-bucket op)
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Kefu Chai [Fri, 27 Aug 2021 13:41:50 +0000 (21:41 +0800)]
Merge pull request #42934 from tchaikov/wip-fio-cxx
cmake: use upstream repo for fio
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Ilya Dryomov [Fri, 27 Aug 2021 12:10:49 +0000 (14:10 +0200)]
doc/rbd/rbd-nomad: fix indentation in config snippets
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Zac Dover [Thu, 26 Aug 2021 04:47:44 +0000 (14:47 +1000)]
doc: clean rbd-nomad.rst
This PR improves the rbd-nomad.rst file.
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Samuel Just [Fri, 27 Aug 2021 05:38:46 +0000 (22:38 -0700)]
Merge pull request #42901 from athanatos/sjust/wip-lba-rewrite
crimson/os/seastore: replace lba btree implementation with simpler, more correct iterator based implementation
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
dengchl01 [Fri, 27 Aug 2021 03:19:27 +0000 (11:19 +0800)]
mgr/restful/api/crush.py: delete unused reference
This commit delete unused reference of the crush page.
Signed-off-by: Charlie Developer <dengchl01@inspur.com>
Patrick Donnelly [Fri, 27 Aug 2021 02:36:28 +0000 (22:36 -0400)]
Merge PR #42530 into master
* refs/pull/42530/head:
qa: ignore expected metadata cluster log error
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Samuel Just [Tue, 13 Jul 2021 02:02:18 +0000 (02:02 +0000)]
crimson/os/seastore/lba_manager: replace btree implementation
Replace previous implementation with one based around an internal
iterator interface. Besides simplifying the implementation and
removing duplicate lookups in the allocation pathway, this implementation
should correct a design problem in the prior implementation wherein
LBALeafNode::find_hole couldn't see the first element of the subsequent
node and therefore assumed that there was one at get_meta().end.
This patch removes the btree logic from lba_btree_node_impl.* leaving
the LBAInternalNode and LBALeafNode layout in lba_btree_node.*.
lba_btree.h/cc now have the main btree update/query logic.
Signed-off-by: Samuel Just <sjust@redhat.com>
Neha Ojha [Thu, 26 Aug 2021 22:39:30 +0000 (15:39 -0700)]
Merge pull request #42941 from kamoltat/ksirivad-add-release-notes-quincy
PendingReleaseNotes: add autoscaler profile
Reviewed-by: Neha Ojha <nojha@redhat.com>
Samuel Just [Thu, 19 Aug 2021 05:13:08 +0000 (22:13 -0700)]
crimson/os/seastore/seastore_types: add lba_root_t::set_depth
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 19 Aug 2021 05:12:38 +0000 (22:12 -0700)]
crimson/os/seastore/lba_manager: fix comment
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 19 Aug 2021 05:11:30 +0000 (22:11 -0700)]
crimson/common/fixed_kv_node_layout: add iterator operator-- and misc
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Mon, 19 Jul 2021 22:01:57 +0000 (15:01 -0700)]
crimson/os/seastore: remove unneeded TransactionManager::find_hole
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 20 Aug 2021 03:41:50 +0000 (20:41 -0700)]
crimson/tools/store_nbd/tm_driver: reset segment_manager in mkfs()
Otherwise, instantiating segment_manager in mount() results in two
instances and a duplicate metric exception.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 20 Aug 2021 03:40:35 +0000 (20:40 -0700)]
crimson/os/seastore/cache: fix formatting in get_extent
In particular, this->invalidate and *ref.get() are superfluous.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 19 Aug 2021 21:42:21 +0000 (21:42 +0000)]
key_value_store/kv_flat_btree_async: add using namespace std
Not worth annotating the specific elements of std used.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 20 Aug 2021 08:03:22 +0000 (01:03 -0700)]
crimson/os/seastore: get_next_dirty_extents: record in transaction read set
Record the extents in the read set after wait_io() as in get_extent. This
should ensure that the interruptible_future machinery will handle the
event that one of them gets invalidated prior to beging rewritten.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Mon, 23 Aug 2021 17:56:22 +0000 (10:56 -0700)]
crimson/os/seastore/segment_manager/block: add segment_size to make_superblock debugging
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Mon, 23 Aug 2021 20:51:50 +0000 (13:51 -0700)]
test/crimson/seastore/test_btree_lba_manager: add better merge test
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Tue, 24 Aug 2021 05:38:36 +0000 (05:38 +0000)]
test/crimson/seastore/test_transaction_manager: update find_hole_assert_trigger
Signed-off-by: Samuel Just <sjust@redhat.com>
chunmei-liu [Wed, 14 Jul 2021 00:42:55 +0000 (17:42 -0700)]
crimson/seastore:: add find_hole/add_pin assert test case
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Samuel Just [Tue, 24 Aug 2021 16:29:22 +0000 (09:29 -0700)]
crimson/os/seastore: fix some clang capture warnings
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 26 Aug 2021 01:53:27 +0000 (18:53 -0700)]
crimson/os/seastore/segment_cleaner: fix update_usage call
Signed-off-by: Samuel Just <sjust@redhat.com>
Kamoltat [Thu, 26 Aug 2021 19:50:14 +0000 (19:50 +0000)]
PendingReleaseNotes: add autoscaler profile
Added autoscaler profile release notes for
>=17.0.0
Signed-off-by: Kamoltat <ksirivad@redhat.com>
Ilya Dryomov [Thu, 26 Aug 2021 19:49:29 +0000 (21:49 +0200)]
Merge pull request #42843 from majianpeng/pwl-ssd-restart-failed
librbd/cache/pwl/ssd: fix first_valid_entry calculation in retire_entries()
Reviewed-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Kamoltat Sirivadhna [Thu, 26 Aug 2021 17:13:02 +0000 (13:13 -0400)]
Merge pull request #42741 from sosodev/master
mgr/pg_autoscaler: add threshold module option
Yuri Weinstein [Thu, 26 Aug 2021 16:41:48 +0000 (09:41 -0700)]
Merge pull request #42569 from ljflores/perf-histograms
mgr, mgr/telemetry: add access to osd commands in mgr and workload metrics to telemetry
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
Kefu Chai [Thu, 26 Aug 2021 15:57:00 +0000 (23:57 +0800)]
cmake: use upstream repo for fio
this change partially reverts
10baab3fc8293b8c30ca90a4acd76f70d011f1b5 ,
but since the fix for C++ build is not included by any tag or branche so
far. let's just use the sha1 for now.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Joseph Sawaya [Thu, 26 Aug 2021 15:12:59 +0000 (11:12 -0400)]
Merge pull request #42858 from josephsawaya/wip-mgr-rook-osd-removal
mgr/rook: implementing OSD removal in the rook orchestrator
Matt Benjamin [Tue, 9 Mar 2021 02:57:57 +0000 (21:57 -0500)]
rgw: permit logging of list-bucket (and any other no-bucket op)
This limitation is from inception (2011), but just recently reported by
a site using ops-log for system audit.
Fixes: https://tracker.ceph.com/issues/49313
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Casey Bodley [Thu, 26 Aug 2021 14:32:23 +0000 (10:32 -0400)]
Merge pull request #42869 from cbodley/wip-49955
qa/rgw: move mixed-version multisite tests out of upgrade suite
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Daniel Gryniewicz [Thu, 26 Aug 2021 13:45:17 +0000 (09:45 -0400)]
Merge pull request #42922 from dang/wip-dang-zipper-cleanup
RGW Zipper Cleanups
Reviewed-by: Soumya Kadouri <skoduri@redhat.com>
Venky Shankar [Thu, 26 Aug 2021 12:32:34 +0000 (18:02 +0530)]
Merge pull request #42893 from jan--f/snap-sched-fix-multi-retention
snap-schedule: count retained snapshots per retention policy
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Igor Fedotov [Thu, 26 Aug 2021 11:08:40 +0000 (14:08 +0300)]
Merge pull request #42895 from ifed01/wip-ifed-improve-alloc-replay
test/allocator_replay_test: implement "try_alloc" command.
This tool has just limited manual application when troubleshooting allocator's issues hence omitting QA run...
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Igor Fedotov [Wed, 25 Aug 2021 12:37:19 +0000 (15:37 +0300)]
test/store_test: fix broken test case
After merging NCB stuff missed shared blob detection doesn't reveal
extent leak any more - which is expected. Hence we need to adjust the
relevant UT.
Fixes:https://tracker.ceph.com/issues/52398
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
Samuel Just [Thu, 26 Aug 2021 01:50:41 +0000 (18:50 -0700)]
Merge pull request #42916 from cyx1231st/wip-seastar-tree-metrics2
crimson/seastore: improve metrics about onode and lba trees
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Wed, 25 Aug 2021 19:23:54 +0000 (12:23 -0700)]
Merge pull request #42917 from rzarzynski/wip-crimson-watch-list
crimson/osd: implement CEPH_OSD_OP_LIST_WATCHERS.
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Daniel Gryniewicz [Wed, 25 Aug 2021 17:02:12 +0000 (13:02 -0400)]
RGW Zipper - chunk/stripe size are RadosStore only
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Yuri Weinstein [Wed, 25 Aug 2021 16:07:29 +0000 (09:07 -0700)]
Merge pull request #42886 from sebastian-philipp/prometheus-url-missing-slash
mgr/{prometheus,restful}: Fix url generation again
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Kefu Chai [Wed, 25 Aug 2021 16:01:39 +0000 (00:01 +0800)]
Merge pull request #42921 from tchaikov/wip-crimson-cleanup
crimson/os: use structured binding in loop
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Wed, 25 Aug 2021 14:25:54 +0000 (22:25 +0800)]
crimson/os: use structured binding in loop
also avoid using `map[key] = val` for setting an item in map, as, if
the key does not exist in map, `map[key]` would have to create a value
using its default ctor, and then call the `operator=(bufferlist&&)` to
set it.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Joseph Sawaya [Wed, 25 Aug 2021 13:54:17 +0000 (09:54 -0400)]
mgr/rook: add better error handling to remove
This commit adds better error handling to the remove method
in the DefaultRemover by hiding stack traces from the user
and displaying meaningful error messages.
Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
Joseph Sawaya [Tue, 24 Aug 2021 16:01:39 +0000 (12:01 -0400)]
mgr/rook: use label selector to fetch pods and jobs in remover
This commit adds a label_selector to the fetching of the osd pods and osd prepare
jobs in the DefaultRemover.
Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
Amnon Hanuhov [Wed, 25 Aug 2021 11:34:18 +0000 (14:34 +0300)]
Merge pull request #42915 from AmnonHanuhov/wip-fix_omap_set_values
crimson/os: Use operator[] in CyanStore when inserting values to omap
Jianpeng Ma [Fri, 20 Aug 2021 06:29:37 +0000 (14:29 +0800)]
librbd/cache/pwl/ssd: fix first_valid_entry calculation in retire_entries()
Consider one control_block which cotain multi encode(WriteLogCacheEntry):
Log1: WriteLogEntry
Log2: WriteLogEntry
Log3: Non-WriteLogEntry
For this case, currently calc method is: control_block_pos + sizeof(control_block).
But in fact, it should: control_block_pos + sizeof(control_block) +
data_length(Log1 + Log2).
Wrong first_valid_entry will persist to superblock and restart to read.
This cause read wrong position and when decode(WriteLogCacheEntry) it
will report bug.
Fixes: https://tracker.ceph.com/issues/52323
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
Radoslaw Zarzynski [Wed, 25 Aug 2021 10:36:00 +0000 (10:36 +0000)]
crimson/osd: implement CEPH_OSD_OP_LIST_WATCHERS.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Ilya Dryomov [Wed, 25 Aug 2021 09:28:46 +0000 (11:28 +0200)]
Merge pull request #42712 from idryomov/wip-malignant-noexcept
common: drop noexcept on thread entry points
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Alfonso Martínez [Wed, 25 Aug 2021 08:52:39 +0000 (10:52 +0200)]
Merge pull request #42090 from cypherean/feedback
mgr/dashboard: report ceph tracker bug/feature through CLI/API
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Juan Miguel Olmo [Wed, 25 Aug 2021 08:14:46 +0000 (10:14 +0200)]
Merge pull request #41918 from SudhanAruna/feature_parse
behave_tests: Implemented basic behave test scenario's for cephadm, ceph shell and OSD commands
Yingxin Cheng [Mon, 23 Aug 2021 07:31:46 +0000 (15:31 +0800)]
crimson/seastore: improve metrics about onode and lba trees
Absolute value:
* tree_depth
By transaction source:
* tree_inserts_committed
* tree_inserts_invalidated
* tree_erases_committed
* tree_erase_invalidated
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Amnon Hanuhov [Wed, 25 Aug 2021 06:22:30 +0000 (09:22 +0300)]
crimson/os: Use operator[] in CyanStore when inserting values to object omap
std::map::insert() doesn't update the value associated with an existing
key, therefore we should use operator[] instead
Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
Samuel Just [Wed, 25 Aug 2021 02:40:29 +0000 (19:40 -0700)]
Merge pull request #42885 from cyx1231st/wip-seastar-tree-metrics
crimson/seastore: add metrics about onode and lba trees
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Wed, 25 Aug 2021 02:38:25 +0000 (19:38 -0700)]
Merge pull request #42907 from rzarzynski/wip-crimson-smphack
crimson/osd: implicitly append '--smp 1' when invoked without it
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Xiubo Li [Mon, 2 Aug 2021 03:53:37 +0000 (11:53 +0800)]
mds: switch mds_lock to fair mutex
The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the lock will be
acquired by threads in the order that they called the lock().
In most case this works well, but in corner case in the Finisher
thread in mds daemon, which may call more than one complete()s
once the mdlog flushing succeeds, after the mdlog flushing is done
it will call the queued complete callbacks and the Finisher thread
could always successfully acquire the mds_lock in successive
complete callbakcs even there may have other threads already being
stuck waiting the mds_lock. This will make the other threads starve
and if they are client's requests, it will cause several or even
tens of seconds long delay for user's operations.
This will switch the mds_lock to fair mutex and it could make sure
that the all the mds_lock waiters are in FIFO order and the Finisher
thread won't hold the mds_lock that long.
At the same time, if the finisher thread has many completes needed
to run the fair mutex could guarantee that the finisher won't be
scheduled out due to fair mutex unlock() if no any other mds_lock
waiter queued.
Fixes: https://tracker.ceph.com/issues/51722
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Mon, 2 Aug 2021 06:20:52 +0000 (14:20 +0800)]
common/Timer: make SafeTimer a template
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Mon, 2 Aug 2021 03:52:57 +0000 (11:52 +0800)]
common/fair_mutex: add is_locked_by_me support
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
Xiubo Li [Thu, 5 Aug 2021 03:55:06 +0000 (11:55 +0800)]
common: do not compile condition_variable_debug in none debug mode
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Radoslaw Zarzynski [Tue, 24 Aug 2021 14:02:36 +0000 (14:02 +0000)]
crimson/osd: implicitly append '--smp 1' when invoked without it
This commit is basically a hack supposed to fulfil the obligation
of crimson being a drop-in replacement for the classical OSD
we already made by packaging it under `/usr/bin/ceph-osd`.
The discussion whether the interface-exactness should be continued
or not is out of scope of the commit; it's supposed just to handle
the issue unveiled by the Rook integration effort: `crimson-osd`
is unable to `--mkfs` because Seastar, if not restricted by passing
`--smp N`, considers all CPU cores available in the system when
allocating resources. This leads to the following error:
```
ERROR 2021-08-24 14:17:32,105 [shard 5] seastar - Could not setup Async I/O: Resource temporarily unavailable. The most common cause is not enough request capacity in /proc/sys/fs/aio-max-nr. Try increasing that number or reducing the amount of logical CPUs available for your application
```
This hack will need to be dropped when integrating multi-reactor
support in crimson.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Neha Ojha [Tue, 24 Aug 2021 23:27:32 +0000 (16:27 -0700)]
Merge pull request #42910 from zdover23/doc-2021-08-25-doc-dev-config-verb-agreement-typo
doc/dev/config: fix verb disagreement
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Tue, 24 Aug 2021 23:26:43 +0000 (16:26 -0700)]
Merge pull request #42713 from ronen-fr/wip-ronenf-explicit-config
common: removing the explicit attribute from a public copy constructor
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Zac Dover [Tue, 24 Aug 2021 19:27:16 +0000 (05:27 +1000)]
doc/dev/config: fix verb disagreement
This PR changes the following sentence:
The other is to register an observer
that will be called every time the
relevant configuration values changes.
This sentence has been rewritten so that the
verb agrees with its subject, like this:
The other is to register an observer
that will be called every time the
relevant configuration values change.
Signed-off-by: Zac Dover <zac.dover@gmail.com>
cypherean [Tue, 24 Aug 2021 09:55:42 +0000 (15:25 +0530)]
mgr/dashboard: report ceph tracker bug/feature through CLI/API
Fixes: https://tracker.ceph.com/issues/44851
Signed-off-by: Shreya Sharma <shreyasharma.ss305@gmail.com>
Ilya Dryomov [Tue, 24 Aug 2021 15:41:57 +0000 (17:41 +0200)]
Merge pull request #42806 from zdover23/wip-doc-2021-08-17-rbd-nomad-3-of-x
doc: rewrite configure ceph-csi to "conf. nomad"
Reviewed-by: Etienne Menguy <etienne.menguy@croit.io>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Kefu Chai [Tue, 24 Aug 2021 15:32:39 +0000 (23:32 +0800)]
Merge pull request #42845 from tchaikov/wip-51483
common/options: validate see-also
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Zac Dover [Tue, 17 Aug 2021 05:07:26 +0000 (15:07 +1000)]
doc: rewrite configure ceph-csi to "conf. nomad"
This PR rewrites the sections
- Configure Ceph-CSI
- Configure Nomad
in the rbd-nomad.rst Chapter of the RBD
Guide.
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Sebastian Wagner [Tue, 24 Aug 2021 14:55:02 +0000 (16:55 +0200)]
Merge pull request #42857 from dsavineau/cephadm_tcmalloc_ceph_daemons_only
cephadm: Set tcmalloc env var for Ceph daemons
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Sebastian Wagner [Tue, 24 Aug 2021 14:54:50 +0000 (16:54 +0200)]
Merge pull request #42833 from oleeander/cephadm_openntpd
cephadm: check for openntpd.service as time sync service
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Sebastian Wagner [Tue, 24 Aug 2021 14:54:34 +0000 (16:54 +0200)]
Merge pull request #42824 from pcuzner/cephadm-add-ports
cephadm:Add listening ports to gather-facts output
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Sebastian Wagner [Tue, 24 Aug 2021 14:54:14 +0000 (16:54 +0200)]
Merge pull request #42699 from sebastian-philipp/cephadm-duplicate-ips
cephadm: list-networks: Avoid duplicated IPs
Reviewed-by: Francesco Pantano <fpantano@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sage@newdream.net>
Sebastian Wagner [Tue, 24 Aug 2021 14:52:28 +0000 (16:52 +0200)]
Merge pull request #42405 from Daniel-Pivonka/cephadm_rm_offline_host
mgr/cephadm: add ability to remove offline host
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Ilya Dryomov [Tue, 24 Aug 2021 14:32:42 +0000 (16:32 +0200)]
Merge pull request #42805 from zdover23/wip-doc-2021-08-17-rbd-nomad-2-of-x
doc: improve "Create a Pool" in rbd-nomad.rst
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Kefu Chai [Thu, 19 Aug 2021 10:01:03 +0000 (18:01 +0800)]
cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/
as FindPython3.cmake only sets Python3_EXECUTABLE for us, we should
stick with Python3_EXECUTABLE instead of Python_EXECUTABLE.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Thu, 19 Aug 2021 09:24:32 +0000 (17:24 +0800)]
common/options: validate see-also
y2c.py is like a compiler which translates .yaml to .cc and .h files,
it does not have access to all .yaml files. to validate the dangling
see-also issue, we need to do this with a "linker".
in this change, validate-options.py is introduced to check if any of
option name included by the see-also property is valid.
Fixes: https://tracker.ceph.com/issues/51483
Signed-off-by: Kefu Chai <kchai@redhat.com>
Zac Dover [Tue, 17 Aug 2021 04:44:52 +0000 (14:44 +1000)]
doc: improve "Create a Pool" in rbd-nomad.rst
This PR improves the English in the "Create
a Pool" section of the "RBD & Nomad Integration"
chapter of the RBD Guide.
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Kefu Chai [Tue, 24 Aug 2021 14:04:28 +0000 (22:04 +0800)]
Merge pull request #42826 from tchaikov/wip-test-fair-mutex
test/common/test_fair_mutex: increase NR_ROUNDS from 256 to 512
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Kefu Chai [Tue, 24 Aug 2021 13:51:34 +0000 (21:51 +0800)]
Merge pull request #42881 from wjwithagen/wjw-fix-vstart-mgr
src/vstart.sh: put modules option for MGR in the [mgr] block.
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Kefu Chai [Tue, 24 Aug 2021 13:49:42 +0000 (21:49 +0800)]
Merge pull request #42882 from cyx1231st/wip-seastar-cleanup
crimson/common: minor cleanups to futures
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Ilya Dryomov [Tue, 24 Aug 2021 11:19:03 +0000 (13:19 +0200)]
Merge pull request #42802 from Rethan/fix-rbd-bench-unit
rbd: avoid overflow of ios and clarify io-size limit for bench
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Jan Fajerski [Mon, 23 Aug 2021 13:37:57 +0000 (15:37 +0200)]
snap-schedule: add test for multiple retention set calculation
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>