test/{librbd, rgw}: retry when bind fail with port 0
there is chance that the bind() call may fail if we have another test
happen to pick the free port picked by operating system. in this case,
we just retry up to 42 times.
in theory, this change does not fully address the racing, but it should
help to alleviate this issue.
Zac Dover [Tue, 30 Aug 2022 11:48:08 +0000 (21:48 +1000)]
doc/start: update documenting-ceph branch names
This PR updates the branch names in the
documenting-ceph.rst file. It gets rid of all references
to the "master" branch, and updates the language to
reflect the state of play in 2022.
inb4: This PR merely removes the most egregious inaccuracies,
the ones that were most readily evident on a cursory perusal.
The full text remains to be carefully read and fitted together
with care.
Ilya Dryomov [Thu, 18 Aug 2022 16:48:39 +0000 (18:48 +0200)]
librbd/cache/pwl: generate image cache state json under m_lock
The previous commit moved the entirety of write_image_cache_state()
from under m_lock. This was a step too far because the generated image
cache state json is no longer guaranteed to be consistent.
Arrange for m_lock to still be held during image cache json generation
but released before owner_lock is grabbed.
librbd/cache/pwl: move write_image_cache_state() out of m_lock
periodic_stats() will get m_lock, then get owner_lock. It is opposite
to the lock getting order of SnapshotCreateRequest::handle_notify_quiesce().
move write_image_cache_state() out of m_lock scope. After calling
update_image_cache_state(), and m_lock auto released, then call
write_image_cache_state() to update state in osds.
Nizamudeen A [Fri, 2 Sep 2022 10:19:31 +0000 (15:49 +0530)]
Merge pull request #47867 from MrFreezeex/quincy-ceph-mixin-backports
quincy: monitoring: ceph mixin backports
Reviewed-by: Aashish Sharma <aasharma@redhat.com> Reviewed-by: Anthony D Atri <anthony.datri@gmail.com> Reviewed-by: Nizamudeen A <nia@redhat.com> Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Ilya Dryomov [Tue, 30 Aug 2022 09:45:44 +0000 (11:45 +0200)]
rbd-mirror: skip setting error code on snapshot replayer shutdown
This is regarding failures in unregister_remote_update_watcher() and
unregister_local_update_watcher(). handle_replay_complete() can't be
called in these cases anymore as it would blindly attempt to unregister
watchers from scratch again. Dropping handle_replay_complete() calls
there means that these failures would only be logged and would not be
surfaced by snapshot replayer. But the only caller ignores them
anyway:
void ImageReplayer<I>::shut_down(int r) {
...
// close the replayer
if (m_replayer != nullptr) {
ctx = new LambdaContext([this, ctx](int r) {
m_replayer->destroy();
m_replayer = nullptr;
ctx->complete(0); <------
});
ctx = new LambdaContext([this, ctx](int r) {
m_replayer->shut_down(ctx);
});
}
Ilya Dryomov [Wed, 24 Aug 2022 10:56:31 +0000 (12:56 +0200)]
rbd-mirror: resume pending shutdown on error in snapshot replayer
If a shutdown is requested, e.g. by update_pool_replayers() because
remote RADOS instance got blocklisted, and Replayer::shut_down() pends
it on completion of current snapshot sync, it gets stuck if replayer
encounters an error in the interim. This is particularly likely in the
blocklist case: a higher layer may detect that client got blocklisted
and request a shutdown first, and then when replayer sees EBLOCKLISTED
in turn, it calls handle_replay_complete() -- which does not resume
a pending shutdown. Because update_pool_replayers() blocks on shutdown
with Mirror::m_lock held, eventually the entire daemon hangs in
perpetuity.
Ilya Dryomov [Sat, 27 Aug 2022 09:09:00 +0000 (11:09 +0200)]
librbd: use actual monitor addresses when creating a peer bootstrap token
Relying on mon_host config option is fragile, as the user may confuse
v1 and v2 addresses, group them incorrectly, etc. Get mon_host value
only as a fallback.
Adam King [Thu, 18 Aug 2022 12:49:57 +0000 (08:49 -0400)]
qa/cephadm: specify using container host distros for workunits
Right now, the OS Type and OS Version for these workunits
tests is left blank on pulpito and they appear to be trying to
run ubuntu jammy currently which is causing failures. We should
specify what distros the tests should run on then very explicitly
tell it to start trying new distros when we can get the tests to
pass.
胡玮文 [Sun, 9 Jan 2022 15:17:38 +0000 (23:17 +0800)]
mon/MDSMonitor: remove redundant state change check
There are two sets of checks to state change in prepare_beacon.
Since the last commit, many of these checks are covered by
`MDSMap::state_transition_valid`. So merging these checks.
This fixes the bug that standby-replay is evicted unexpectedly.
This bug is introduced in 794d13c9ff4 (mon/MDSMonitor: reject illegal want_states from MDS)
but only reveal itself after 20509bb6c82 (MDSMonitor: handle damaged from standby-replay)
The addition of unselectable prompts to these three files
completes the work begun in PR#47810 (d8064b4), which sought
to bring dashboard.rst into line with the unselectable prompt
standard introduced by Kefu Chai in 2020.
In 4a3afcf, the $PATH is set for the test, but we cannot set multiple
properties with a single `set_property()` cmake command. We fix that by
adding the installation path of jsonnet-bundler
(CMAKE_CURRENT_BINARY_DIR) to the $PATH used for every tox test.
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch> Co-Authored-By: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit d46e14c71bffda1381dac7da244ab8347d035769)
Xiubo Li [Wed, 1 Jun 2022 02:32:58 +0000 (10:32 +0800)]
mds: notify the xattr_version to replica MDSes
When one client changes a xattr's value in the auth MDS, when replying
to the client the MDS possibly will drop the increased xattr_version
and new value in the reply message if no 'Xs' caps will be issued to
the client together.
And when the client wants to get this xattr's value, and if it sends
the request to a replicated MDS, since the replicated MDS still has
the old value of the xattr_version, and then the client will drop the
xattr value since xattr_version is not changed.
We need to notify the xattr_version to the replicated MDSes together
with the xattrs when notifying the lock state.
Adam King [Thu, 25 Aug 2022 16:09:49 +0000 (12:09 -0400)]
mgr/orchestrator/tests: don't match exact whitespace in table output
It seems that the exact spacing may differ a bit between
python versions. Currently seeing py3 (which cooresponds to py 3.6
on my system) passing these tests and py37 (which is python 3.7
obviously) failing. I think verifying against the exact whitespace
is unnecessary anyhow. As long as it isn't egregious, we don't
really need to worry about exactly what the spacing is.
Laura Flores [Wed, 24 Aug 2022 22:23:45 +0000 (22:23 +0000)]
src/pybind/mgr/telemetry: parse `outb` instead of `outs`
Following the merge of https://github.com/ceph/ceph/pull/47650, which
fixes the confusion between std out and std err in admin socket
commands, we will need to reference the out stream (outb) instead
of the error stream (outs) when we parse heap stats.
Laura Flores [Tue, 5 Jul 2022 22:06:15 +0000 (22:06 +0000)]
mgr/telemetry: add `perf_memory_metrics` collection to telemetry
This new collection includes heap stats and mempool metrics for
mon and mds daemons.
A `tell_command` function was introduced to the mgr module as a wrapper
around the `send_command` function to make it easier to run "tell"
admin socket commands.
osd, mds: fix the "heap" admin cmd printing always to error stream
Before the patch `ceph::osd_cmds::heap()` was confusing
the concepts of _stderr_ and _stdout_. This was the direct
cause of the differences in output between `ceph tell` and
`ceph daeamon`.
Thanks to Laura Flores who made the extremely useful observation
noted in https://tracker.ceph.com/issues/57119#note-3.
Zac Dover [Thu, 25 Aug 2022 15:56:41 +0000 (01:56 +1000)]
doc/mgr: add prompt directives to dashboard.rst
This commit adds prompt directives (.. prompt:: bash $) to
the commands in dashboard.rst.
There are several ".. include::" directives in the dashboard.rst
file, which means that part of this page is sourced from elsewhere
than the dashboard.rst file. Because I have not yet added prompt
directives to those files, there is an inconsistency in the rendering
of this file. Most of the commands on this page have unselectable
prompts (unselectable prompts are the prompts that don't get added to
the buffer when you copy them to one of the clipboards). But the
commands on this page that come from those ".. include::" directives
do not yet have unselectable prompts.
This file is over 1600 lines long. It was perhaps not optimally wise
of me to have edited all of it in one fell swoop. It took many hours,
and carefully checking it will probably take at least one hour. I
suggest that whoever reviews this should not spend much time on it,
but should instead make a quick pass over the page and make sure that
it looks passable.
The English syntax on this page (and throughout the Dashboard doc-
umentation) will be tightened to remove ambiguity and to improve
readability in the near future, so hold all English-language-related
comments for a future pull request.
Adam King [Wed, 24 Aug 2022 14:36:53 +0000 (10:36 -0400)]
doc/cephadm: fix example for specifying networks for rgw
count_per_host must be used with underscores rather
than dashes to work, you need to pass service_id not
service_name and the option for the port is called
rgw_frontend_port not just "port"
Lucian Petrut [Tue, 16 Nov 2021 13:53:37 +0000 (13:53 +0000)]
msg: Fix Windows IPv6 support
The Windows AF_INET6 definition doesn't match the Linux one, thus
sockaddr structures must be converted when being sent over the wire.
Otherwise, Linux hosts will not be able to recognize the address
family and reject Windows client connections:
handle_client_ident peer is trying to reach
v2:(unrecognized address family 23)/0 which is not us
Note that this isn't the case with AF_INET, which has the same
definition on both platforms.
Prashant D [Fri, 24 Jun 2022 15:10:03 +0000 (11:10 -0400)]
mgr: Remove service_daemon handling from MMgrUpdate
The service_daemon is only required if client services
needs to be registered with mgr. The service_daemon
handling is not required in case of MMgrUpdate as it
handles mon metadata updates only.
Fixes: https://tracker.ceph.com/issues/55322 Signed-off-by: Prashant D <pdhange@redhat.com>
(cherry picked from commit 6540937a6d1865642dd799b4996d85d2ee9a99ad)
Prashant D [Mon, 28 Mar 2022 13:02:08 +0000 (14:02 +0100)]
mgr, mon: Keep upto date metadata with mgr for MONs
The mgr updates mon metadata through handle_mon_map which
gets triggered when MONs were removed/added from/to cluster or
the active mgr is restarted or mgr failsover.
We could have handled metadata update through MMgrOpen or
early MMgrReport messages but these are sent before monitor
electin completes and lead monitor updates pending metadata
in monstore. Instead of relying on fetching mon metadata using
'ceph mon metadata <id>' command, explicitly send metadata
update request with mon metadata to mgr.
Fixes: https://tracker.ceph.com/issues/55088 Signed-off-by: Prashant D <pdhange@redhat.com>
(cherry picked from commit 1a065043b964f8c014ebb5bc890a243c398ff07c)
Zac Dover [Tue, 23 Aug 2022 06:59:04 +0000 (16:59 +1000)]
doc/mgr: edit orchestrator.rst
This PR improves the English language in the "Orchestrator CLI"
section of the MGR documentation. It adds a couple of section
headers in order to signpost the information in the document
a bit more than had already been done, but it makes no major
structural changes to the presentation of the information here.
This PR was motivated by feedback from the 2022 Ceph User Survey
in which one of the respondents wrote "better ceph orch documen-
tation".
The final section on this page, "Current Implementation Status",
must be verified by someone who is familiar with the current state
of "ceph orch" and a date stamp should be applied to the top of
the section so that the word "current" has a meaningful referent.
projects / ceph.git / log