xie xingguo [Fri, 4 Jan 2019 00:39:01 +0000 (08:39 +0800)]
mon/OSDMonitor: do not populate void pg_temp into nextmap
Due to commit ea723fb, pg_temp with clean acting set are added to inc map.
The original intent was to clear out pg_temps during priming, but as
written it would set a new_pg_temp item clearing the pg_temp even if one
didn't already exist. Adding the up != acting condition in there makes us
only take that path if there is an existing pg_temp entry to remove.
Fixes: https://tracker.ceph.com/issues/37784 Signed-off-by: Aleksei Zakharov <zakharov.a.g@yandex.ru>
(cherry picked from commit b1d3ca5e78eaee509c923f06e9024c23cc6ce31a)
Alfredo Deza [Thu, 13 Dec 2018 14:49:41 +0000 (09:49 -0500)]
ceph-volume tests.functional.lvm stop all osds before activating again to test permissions
Otherwise, ``ceph-volume lvm activate --all`` will skip OSDs that are
already running. So the test that verifies that permissions are
corrected will fail because ``activate`` will simply skip them
Andrew Schoen [Wed, 12 Dec 2018 20:10:59 +0000 (14:10 -0600)]
ceph-volume: include the ceph-facts role when needed during testing
The ceph-ansible stable-3.2 branch does not have the ceph-facts role
but master does. We need to make sure to import this role only when
it's supported.
The patch to enforce bounds on max-keys/max-uploads/max-parts had a few
issues that would prevent us from compiling it. Instead of changing the
code provided by the submitter, we're addressing them in a separate
commit to maintain the DCO.
Signed-off-by: Joao Eduardo Luis <joao@suse.de> Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 29bc434a6a81a2e5c5b8cfc4c8d5c82ca5bf538a)
mimic specific fixes:
As the largeish change from master g_conf() isn't in mimic yet, use the g_conf
global structure, also make rgw_op use the value from req_info ceph context as
we do for all the requests
Robin H. Johnson [Fri, 21 Sep 2018 21:49:34 +0000 (14:49 -0700)]
rgw: enforce bounds on max-keys/max-uploads/max-parts
RGW S3 listing operations provided a way for authenticated users to
cause a denial of service against OMAPs holding bucket indices.
Bound the min & max values that a user could pass into the max-X
parameters, to keep the system safe. The default of 1000 is chosen to
match AWS S3 behavior.
Affected operations:
- ListBucket, via max-keys
- ListBucketVersions, via max-keys
- ListBucketMultiPartUploads, via max-uploads
- ListMultipartUploadParts, via max-parts
The Swift bucket listing codepath already enforced a limit, so is
unaffected by this issue.
Prior to this commit, the effective limit is the lower of
osd_max_omap_entries_per_request or osd_max_omap_bytes_per_request.
Backport: luminous, mimic Fixes: http://tracker.ceph.com/issues/35994 Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
(cherry picked from commit d79f68a1e31f4bc917eec1b6bbc8e8446377dc6b)
Conflicts:
src/common/options.cc:
Conflicts due to options from master
mon/config-key: limit caps allowed to access the store
Henceforth, we'll require explicit `allow` caps for commands, or for the
config-key service. Blanket caps are no longer allowed for the
config-key service, except for 'allow *'.
(for luminous and mimic, we're also ensuring MonCap's parser is able to
understand forward slashes '/' when parsing prefixes)
xie xingguo [Wed, 21 Nov 2018 01:36:21 +0000 (09:36 +0800)]
osd/OSDMap: add pg-existence sanity check
The reason why __get_pg_pool_size(pg)__ or __get_pg_pool_crush_rule(pg)__ fails is
that the pg does not exist anymore. So it generally makes sense to check __pg_exists(pg)__
before moving further.
xie xingguo [Wed, 20 Jun 2018 01:04:19 +0000 (09:04 +0800)]
osd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone
If an osd is gone or moved out from the specific crush rule,
we should cancel any pg_upmap/pg_upmap_items still bound to
that osd too.
The original code does not work for the above case because
get_parent_of_type() will fail if that osd does not belong
to the crush_rule passed in and hence hits the assert below:
Noah Watkins [Mon, 1 Oct 2018 23:54:19 +0000 (16:54 -0700)]
luminous: doc: show edit on github links and version warnings
backport of #24452 that adds edit on
github links to documentation and notification banners that display
warnings when old documentation is being viewed.
this is not a cherry-pick: it removes from the original patch the
dynamic generation of the releases schedule from a yaml database file.
backporting this portion requires modifying the patch to deal with a
different file / directory structure [in luminous] with no real added value.
Alfredo Deza [Tue, 27 Nov 2018 14:26:33 +0000 (09:26 -0500)]
ceph-volume util.disk fix issue when capturing partition information
Before, the `if` condition meant that it would only work when the output
was '1', which is incorrect as that would only happen if a partition was
the first one, ignoring any other partition. The contents of that file
is the partition number, not a boolean to tell if it is a partition or
not.
It now includes the `holders` file contents which is needed for
dm-mapper work
Venky Shankar [Wed, 31 Oct 2018 06:47:24 +0000 (02:47 -0400)]
client: remove redundant abort logic during dentry invalidation test
commit d1471f070c added retry logic when remounting is
used to invalidate kernel dcache. ceph-fuse performs a
dcache invalidation test during startup, thereby logging
(and continuing) or aborting as per certain configuration
settings. the abort logic is pretty much redundant and
can be cleaned up preserving the existing behavior.
Also, assert is none of dcache invalidation mecahnisms
(`dentry_invalidate_cb` or `remount_cb`) are available.
Venky Shankar [Tue, 28 Aug 2018 10:35:18 +0000 (06:35 -0400)]
client: retry remount on dcache invalidation failure
For some (unknown) reason, there have been reports of
ceph-fuse crash due to failure is remounting at the
time of invalidating kernel dentry cache. This issue
is not also reproducible yet.
Therefore, as suggested by Patrick and Zheng, for a
temporary workaround, client would ignore the failure
as the invalidation would be retried again. There is
a max cap on the number of consecutive remount failures
after which client would abort.
Fedora 29 still ships a Python 2 binary, but some of Ceph's build
dependencies are only available in py3 versions there. In other
words, from F29 on, it is no longer possible to do a py2 Ceph build
on Fedora, even if a python2 binary exists on the system.
If that were not enough, the Python 2 that ships with Fedora 29 is
linked against a non-compatible version of OpenSSL.
Before this commit, install-deps.sh was overriding the spec file's
Python build setting based on the presence or absence of a python2
binary. As the bug cited below indicates, this was not a good idea.
It's better for the spec file to be explicit about which OS versions
are py2 and which are py3, and just stick to that.
Nathan Cutler [Wed, 1 Aug 2018 10:52:45 +0000 (12:52 +0200)]
build/ops: refrain from installing/using lsb_release in install-deps.sh
Unfortunately the mapping between release number and codename (which is only
relevant for Debian and Ubuntu btw) is not available from /etc/os-release.
In that one respect, lsb_release was "better".
However, when I weigh the advantages of obtaining that mapping from an external
tool, with the (substantial) risk that the external dependency might cause
trouble on one or more supported distros (to say nothing of the non- or
semi-/pseudo-supported ones), against the work involved in maintaining a
hard-coded mapping (negligible), the needle on my scale immediately swings
toward eliminating the dependency.
Also, I see this commit as part of the longer-term effort to completely expunge
lsb_release from our codebase. See git log --grep lsb_release.
For another example of an external distro-detection tool (albeit one that was
included in Python 2) gone awry, see http://tracker.ceph.com/issues/18163.
Dan van der Ster [Sat, 15 Sep 2018 12:46:00 +0000 (14:46 +0200)]
mgr/balancer: add crush_compat_metrics param
Add a crush_compat_metrics options to allow choosing between pgs,
objects, and bytes when rebalancing with crush-compat mode.
Multi-values are supported, so that the placement scoring can be
a function of several metrics. By default we score based on the
mean of pgs, objects, and bytes scores for all roots.
projects / ceph.git / log