J. Eric Ivancich [Mon, 14 Oct 2019 20:21:35 +0000 (16:21 -0400)]
rgw: reduce per-shard entry count during ordered bucket listing
Currently, if a client requests the 1000 next entries from a bucket,
each bucket index shard will receive a request for the 1000 next
entries. When there are hundreds, thousands, or tens of thousands of
bucket index shards, this results in a huge amplification of the
request, even though only 1000 entries will be returned.
These changes reduce the per-bucket index shard requests. These also
allow re-requests in edge cases where all of one shard's returned
entries are consumed. Finally these changes improve the determination
of whether the resulting list is truncated.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Sage Weil [Fri, 8 Nov 2019 19:28:45 +0000 (13:28 -0600)]
Merge PR #31493 into master
* refs/pull/31493/head:
ceph-daemon: 'profile ...' not 'allow profile ...'
mgr/ssh: 'profile ...' not 'allow profile ...'
mgr/orchestrator_cli: rearrange things a bit
doc/mgr/orchestrator_cli: remove irrelevant line
mgr/ssh: learn to deploy rbd-mirror daemons
mgr/orchestrator: add rbd-mirror commands and hooks
ceph-daemon: learn to deploy rbd-mirror daemon
mgr/ssh: handle lack of node hints more gracefully
mgr/ssh: factor out update_{rgw,mds} into common helper
mgr/ssh: fix update_rgw, update_mgr
* refs/pull/31400/head:
mds: establish session with mgr only after added to FSMap
mds: do not register as a service daemon
mds: do not try to diagnose cause of MDSMap removal
mds: fix handling of initial MDS states
mds: remove unnecessary const qualifier
mds: cleanup type decl and map iteration
mds: define stream operator for mds_info_t
This commit undoes the service daemon registration for the MDS. It doesn't look
absolutely necessary and it causes the MDS to be listed twice in the `ceph
versions` output:
Fixing that requires looking for duplicates or ignoring MDSs in the
service daemons when the mon processes `ceph versions`. I have a feeling
that it wasn't actually designed to be used by the MDS this way however.
Additionally, the reason for "unknown" version is because the metadata
sent to the mgr does not include "ceph_version".
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
- Make explicit the check for getting removed from the MDSMap. This was
only done before by checking if MDS held a rank which does not check the
case where a standby is removed from the FSMap.
- Use mds_info_t::dump to simplify various debug output.
- Add a few sanity asserts for invalid state transitions.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Sage Weil [Fri, 8 Nov 2019 17:49:00 +0000 (11:49 -0600)]
Merge PR #29437 into master
* refs/pull/29437/head:
mgr/diskprediction_local: Reverted dependencies, added HGST models
mgr/diskprediction_local: Updated dependencies in ceph.spec.in, debian/control to match requirements.txt
mgr/diskprediction_local: Updated Red Hat developed prediction model. Updated module options to choose between Red Hat and ProphetStor models.
mgr/diskprediction_local: Updated prediction models to use only supported python packages.
mgr/diskprediction_local: Replaced old models and updated predictor.
Sage Weil [Fri, 8 Nov 2019 16:12:27 +0000 (10:12 -0600)]
mgr/orchestrator: add rbd-mirror commands and hooks
This is somewhat different from the other services in that the name is
basically unused: we have a single pool of rbd-mirror daemons for the
whole cluster.
Thomas Bechtold [Thu, 7 Nov 2019 15:41:23 +0000 (16:41 +0100)]
ceph-daemon: Move ceph-daemon executable to own directory
Moving ceph-daemon into src/ceph-daemon/ makes it simpler to add extra
code (eg. tox.ini, README, unittests, ...) specific to ceph-daemon.
That way related files are in a single directory.
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
Nathan Cutler [Fri, 8 Nov 2019 14:14:10 +0000 (15:14 +0100)]
ceph-backport.sh/interactive_setup: abort if Step 3 fails
Fixes confusing error message reported by a user:
Setup step 3 of 4 - remote repos
---------------------------------------------------------------------
Searching "git remote -v" for remote repos
ceph-backport.sh: ERROR: Cannot auto-determine fork remote
ceph-backport.sh: (Could not find GitHub user callithea's fork of ceph/ceph in
"git remote -v")
ceph-backport.sh: ERROR: (internal error) fork_remote not set, even after
completing Steps 1-3 of interactive setup
ceph-backport.sh: This could be reported as a bug!
Sage Weil [Fri, 8 Nov 2019 13:10:53 +0000 (07:10 -0600)]
ceph-daemon: add --skip-pull
It occurs to me there might be cases where the user *doesn't* want to pull
the latest image (e.g., because it is a partially disconnected enviroment,
and they know the image is already in the local registry).
Sage Weil [Fri, 8 Nov 2019 13:08:46 +0000 (07:08 -0600)]
Merge PR #31464 into master
* refs/pull/31464/head:
ceph-daemon: help users find the shell/CLI too
ceph-daemon: enable the dashboard during bootstrap
ceph-daemon: add CLI helper to bootstrap
Reviewed-by: Paul Cuzner <pcuzner@redhat.com> Reviewed-by: Kai Wagner <kwagner@suse.com> Reviewed-by: Sebastian Wagner <swagner@suse.com>
A user did "ceph-backport.sh --setup" and entered the interactive setup routine.
After entering a valid GitHub personal access token, she got the following
message:
ceph-backport.sh: my GitHub username is callithea
ceph-backport.sh: You must provide a valid GitHub personal access token
ceph-backport.sh: ERROR: problem detected in your setup
ceph-backport.sh: Run "ceph-backport.sh --setup" to fix
and the script aborted. This commit fixes the issue.
Sage Weil [Thu, 7 Nov 2019 23:14:52 +0000 (17:14 -0600)]
ceph-daemon: make mon container privileged
libudev needs to be privileged in order to query the underlying hardware
devices, as reported by the 'ceph device ...' command set, and to scrape
smart metrics, etc.
Sage Weil [Thu, 7 Nov 2019 22:35:19 +0000 (16:35 -0600)]
mgr/devicehealth: ensure we don't store empty objects
I'm not sure why devid would end up empty, but it is possible if we get
wonky data, and users seem to have seen empty objects in their pools,
so add a check!
Maybe-fixes: https://tracker.ceph.com/issues/41383 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Thu, 7 Nov 2019 18:54:00 +0000 (12:54 -0600)]
mon/MonMap: encode (more) valid compat monmap when we have v2-only addrs
If we have 1 or more mons with v2-only addrs, pre-nautilus clients can't
talk to them. If there are more than 1 such mons in the map, they also
fail when loading the map because they expect the addrs to be unique. In
such situations, lie by giving them v1 addrs that are actually v2 ip:port
(so not actually valid). Hopefully there are enough other mons that do
have v1 addrs that the clients can still connect.
Fixes: https://tracker.ceph.com/issues/42600 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Thu, 7 Nov 2019 18:51:04 +0000 (12:51 -0600)]
mgr/DaemonServer: warn when we reject reports
If this is triggered it can be disruptive, since we're marking the
connection down. It points to a real bug, so better visibility is good
(e.g., failing the teuthology jobs).
Sage Weil [Thu, 7 Nov 2019 16:57:56 +0000 (10:57 -0600)]
Merge PR #31064 into master
* refs/pull/31064/head:
test: Test balancer module commands
mgr: Improve balancer module status
mgr: Release GIL before calling OSDMap::calc_pg_upmaps()
projects / ceph.git / log