Sage Weil [Sun, 6 Oct 2019 14:08:28 +0000 (09:08 -0500)]
Merge PR #30217 into master
* refs/pull/30217/head:
crimson: common/admin_socket kludge so that it builds
mon/MonClient: fix sending mon command to a specific rank
src/.gitignore: ignore .tox
mon/MonClient: interpret numeric mon target name as rank
mgr,mgr/MgrClient: use fsid to signal mon-mgr vs cli MCommands
qa/workunits/cephtool: fix errpr checks for 'ceph daemon' commands
common/ceph_context: make 'config unset' idempotent
qa/tasks/dump_stuck: mon.a, not mon.0
qa/suites/rados/singleton/all/admin-socket: fix test
common/config: EPERM setting config option after startup
qa/workunits/cephtool/test.sh: fix tell output error check
common/admin_socket: pass Formatter from generic infrastructure
common/admin_socket: pass ostream to call() for error output
os/bluestore: fix asok hook return value
rgw: fix asok return value
common/ceph_context: return error code from asok commands
test/pybind/test_rados: fix accidental mon tell test
mon: print entity_name along with caps to debug log
PendingReleaseNotes: notes about asok changes
mgr/MgrClient: empty target string for 'tell' means active mgr
common/admin_socket: report error code as part of output string
osd: change trigger_[deep_]scrub tommands to a pg tell command
osd: remove old command workqueue, threadpool
osd: drop MMonCommand handling
osdc/Objecter: resend OSD tell commands on EAGAIN
osd: route tell commands to asok; migrate commands
osd: use unique_ptr<Formatter> for asok_command
common/ceph_context: add generic asok 'injectargs'
common/admin_socket: allow dup prefixes
common/admin_socket: refactor with sync and async execute_command variants
common/admin_socket: pass input bufferlist
osd: transition to call_async() for asok
common/admin_socket: support alternative call_async()
mon/MonClient: send tell commands out of band via MCommand
mon: accept tell commands via MCommand and send them to asok handler
common/admin_socket: return int from hook call()
mgr/DaemonServer: route MCommand (for octopus+) to asok commands
do not use 'ceph tell mgr'
pybind/ceph_argparse: disambiguate mgr tell and CLI commands
ceph: make 'ceph tell mgr.*' send to the active mgr
ceph: send 'ceph tell mgr.X' to the right mgr
librados: add rados_mgr_command_target
mgr/MgrClient: add start_command variant that takes a target
common/admin_socket: drop unregister_command(); use per-hook variant
common/admin_socket: drop explicit prefix arg to register_command
common/admin_socket: simplify command routing
common/admin_socket: add ability to process MCommand via asok queue
common/admin_socket: pass cmdvec to execute_command
common/admin_socket: use pipe for general wakeup
include/compat: add flags arg to pipe_cloexec
common/admin_socket: drop unused args
Sage Weil [Sat, 5 Oct 2019 01:42:07 +0000 (20:42 -0500)]
Merge PR #29035 into master
* refs/pull/29035/head:
mgr/pg_autoscaler: changes made reflect jdurgin's request
mgr/pg_autoscaler: current pg_num compared to distance between the last initial_pg_num
mgr/progress & mgr/pg_autoscaler: changes reflect liewegas' comment
mgr/pg_autoscaler: get rid of white space
mgr/progress: change threshold value to origin
mgr/progress: cleaning up for pg_autoscaler
mgr/progress: Added Pg Autoscaler Event
Sage Weil [Fri, 4 Oct 2019 14:49:36 +0000 (09:49 -0500)]
osd/PeeringState: base lease support checks on features, not require_osd_release
The require_osd_release value will change at an arbitrary point in time,
midway through a peering interval. That can leave us in a state where
we are midway through an interval and suddenly enforcing checks on leases
we don't have.
In contrast, the upacting features change at an interval boundary, which
lets us adjust our behavior for the whole interval.
Sage Weil [Mon, 30 Sep 2019 15:58:40 +0000 (10:58 -0500)]
mon/MonClient: interpret numeric mon target name as rank
This allows us to behave when a rank is passed to mon_command(..., target=),
which will call rados_mon_command_target() -> MonClient::start_mon_command
with a (string) target name.
We could make an integer variant of rados_mon_command_target, and do the
int vs string differentiation in python, but this is much easier.
Sage Weil [Fri, 27 Sep 2019 21:01:44 +0000 (16:01 -0500)]
mgr,mgr/MgrClient: use fsid to signal mon-mgr vs cli MCommands
We can't use the feature bit for the MCommand connection to tell whether
it is a tell or CLI command because new clients may have to send CLI
commands via MCommand for old clusters, and they don't always know whether
this mgr is new or old yet.
Prior to octopus, MCommand contained a mon/mgr CLI command, and did not
have the fsid field set. Start populating the fsid field, and use this
to signal whether a client is a new MgrClient that knows MCommand vs
MMgrCommand. If we get an MCommand with the fsid set, that means it is
a tell command; otherwise, it's an old client sending a CLI command.
Sage Weil [Thu, 19 Sep 2019 16:11:55 +0000 (11:11 -0500)]
common/admin_socket: pass Formatter from generic infrastructure
The implementation can choose to either use the provided Formatter, or
put something directly into outbl. The implementation may choose to
flush the formatter to the output buffer|stream, or let the caller do it
for them (usually the latter).
Lots of fiddling/cleanup in the implementations to make this build,
including dropping the (seeminlyg unused?) ostream& output mode for
the librbd asok implementations.
Sage Weil [Tue, 10 Sep 2019 15:21:39 +0000 (10:21 -0500)]
common/admin_socket: report error code as part of output string
This isn't ideal, but it's better than nothing.
Net behavior change here is that calling an non-existant command now
succeeds with an error message instead of exiting with an error code
and no informative output.
Sage Weil [Tue, 10 Sep 2019 21:41:18 +0000 (16:41 -0500)]
osdc/Objecter: resend OSD tell commands on EAGAIN
Request map *and* resend. We don't have map epoch info about when the
reply was sent, and the OSD isn't ordering with respect to epochs anyway.
So, resend now, just in case we already saw a map change, or because we
were suffering from a peering vs command race on the OSD side, and then
also request a new map from the mon, in case we are missing a map update.
Sage Weil [Tue, 10 Sep 2019 03:07:03 +0000 (22:07 -0500)]
osd: route tell commands to asok; migrate commands
- move items from _do_command to asok_command in OSD.cc
- update PG::do_command to take a std::function on_finish
- sprinkle in some osd_lock locking (_do_command implicitly locks osd_lock,
asok_command() does not; most commands don't need it)
Paul Cuzner [Fri, 20 Sep 2019 04:55:06 +0000 (16:55 +1200)]
mgr/k8sevents: Add support for remote kubernetes
The initial implementation sent events to the
kubernetes cluster Ceph is running under managed
by rook-ceph. This patch extends this support to
include sending events to an external kubernetes
cluster, that may just be consuming ceph resources
Additional docs added to help people use the
module, either with or without rook-ceph.
It also address a bug where the module was loaded
outside of a rook-ceph environment.
Fixes: https://tracker.ceph.com/issues/41737 Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
Sage Weil [Thu, 3 Oct 2019 18:00:45 +0000 (13:00 -0500)]
osd/PrimaryLogPG: skip obcs that don't exist during backfill scan_range
We already skip objects we encounter that we do getattr() on and get
ENOENT, but sometimes the object is in our obc cache with exists=false.
Skip those too.
Fixes: https://tracker.ceph.com/issues/42177 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Thu, 3 Oct 2019 15:04:19 +0000 (10:04 -0500)]
osd: set affinity for *all* threads
It turns out schedule_setaffinity(2) only sets the current *thread*'s
affinity, not the entire process. Look in /proc to identify all of our
children and map those, too.
Note that this doesn't seem to affect the numa_preferred_nid value in
/proc/$pid/[task/$tid/]sched... at least not immediately. That value does
seem to change for some threads after some load is applied.
Fixes: https://tracker.ceph.com/issues/42054 Signed-off-by: Sage Weil <sage@redhat.com>
* refs/pull/30251/head:
doc: Update requirements for using CephFS
Reviewed-by: Rishabh Dave <ridave@redhat.com> Reviewed-by: Douglas Fuller <dfuller@redhat.com> Reviewed-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
projects / ceph.git / log