so we check SNAPPY_VERSION to tell if we should use `uint32_t` or
`uint32`.
in this change, snappy version used to build win32 client is bumped
to the latest stable version, v1.1.9, to include the fix of
SNAPPY_VERSION. this paves the road to fix of https://tracker.ceph.com/issues/50934
Patrick Donnelly [Wed, 28 Jul 2021 17:45:08 +0000 (10:45 -0700)]
Merge PR #42349 into master
* refs/pull/42349/head:
mon/MDSMonitor: propose if FSMap struct_v is too old
mon/MDSMonitor: give a proper error message if FSMap struct_v is too old
mds/FSMap: use DECODE_OLDEST to gate FSMap version
qa: add tests for fs dump of epoch and trimming
qa: add file system support for dumping epoch
mon/MDSMonitor: return mon_mds_force_trim_to even if equal to current epoch
mon: add debugging for trimming methods
mon: fix debug spacing
qa: add nofs upgrade suite
Patrick Donnelly [Wed, 28 Jul 2021 17:34:12 +0000 (10:34 -0700)]
Merge PR #41025 into master
* refs/pull/41025/head:
qa: wait pgs to be clean before using the pools
qa: ignore PG_RECOVERY_FULL and PG_DEGRADED for mds-full
qa: wait more time since there have many more pgs than before
qa: do not multiple the full ratio twice
qa: do not raise for kclient for _fsync test
qa: use the pg autoscale mode to calcuate the pg_num
qa: set the object_size to 1M
qa: move the is_full() to parent class
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 15 Jul 2021 01:02:20 +0000 (18:02 -0700)]
mon/MDSMonitor: propose if FSMap struct_v is too old
To flush older versions which may still be an empty MDSMap (for clusters
that have never used CephFS), we need to force a proposal so older
versions of the struct are trimmed.
This is the main fix of this branch. We removed code which processed old
encodings of the MDSMap in the mon store via 60bc524. That broke old
ceph clusters which never used CephFS (see cited ticket below). This is
because the initial epoch is an empty MDSMap (back in Infernalis/Hammer)
that is never updated. So, the fix here is to just do proposals
periodically until all of the old structs are automatically trimmed by
the mons.
Fixes: 60bc524827bac072658203e56b1fa3dede9641c5 Fixes: https://tracker.ceph.com/issues/51673 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Wed, 14 Jul 2021 20:31:21 +0000 (13:31 -0700)]
mon/MDSMonitor: return mon_mds_force_trim_to even if equal to current epoch
The PaxosService code already excludes the value returned by
PaxosService::get_trim_to as the upper bound of the range of epochs to
trim. Without this fix, you need to set mon_mds_force_trim_to to one
greater than the epoch you want to trim _and_ force the current epoch to
be one greater than that; the net result being that you can only force
trimming up to 2 epochs behind the current epoch.
This change is helpful for resolving issue 51673, but not strictly
necessary.
Related-to: https://tracker.ceph.com/issues/51673 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Wed, 14 Jul 2021 16:16:26 +0000 (09:16 -0700)]
qa: add nofs upgrade suite
This adds an upgrade suite to ensure that a Ceph cluster without a
CephFS file system does not blow up on upgrade (in particular, that the
MDSMonitor does not trip). This was developed to potentially reproduce
tracker 51673 but the actual cause for that issue was an old encoding
for the MDSMap which was obsoleted in Pacific. You must create a cluster
older than the FSMap (~Hammer or Infernalis) to reproduce. In any case,
this upgrade suite may be useful in the future so let's keep it!
Related-to: https://tracker.ceph.com/issues/51673 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Alfonso Martínez [Wed, 28 Jul 2021 07:51:52 +0000 (09:51 +0200)]
mgr/dashboard: backend unit tests: decouple from build dir
After https://github.com/ceph/ceph/pull/42255 some unit tests got coupled
to the frontend folder of the build dir.
Backend unit tests do not need a build folder: they only test our business logic.
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Sage Weil [Tue, 27 Jul 2021 18:11:56 +0000 (14:11 -0400)]
Merge PR #42406 into master
* refs/pull/42406/head:
mgr/nfs: remove unused 'realm' arg for 'nfs export create rgw'
doc/mgr/rook: update title
doc/mgr/nfs: reference customizing ingress
doc/mgr/nfs: add section for manual ganesha config; reframe
doc/mgr/nfs: document ingress in more detail
doc/mgr/nfs: typo
doc/mgr/nfs: add note about incomplete ingress
qa/suites/orch/cephadm: add rgw nfs export test
mgr/cephadm: ingress: tolerate no daemons
mgr/nfs: add --squash option to 'nfs export create rgw ...'
mgr/nfs: use bucket owner creds for rgw bucket export
mgr/cephadm: use new CEPH_IMAGE_TYPES for all daemons using ceph container image
qa/tasks/python: simple task to run python code
doc/mgr/nfs: revisions
mgr/nfs/export: nicer exceptions on cap update
test/test_intarith: adapt to the change in intarith.h
back in 2623fec1cdd5fd9b15ee67c4b115385c67129ef4, the vaiants of, for
instance, ctz() are consolidated to a single template. so the
ctz<>() dispatches by the size of argument after that change.
but the tests were not updated accordingly.
in this change:
* the tests are updated to use the template.
* instead of using integer literal postfix, use the macros like
UINT64_C to define integer constants for better portability on
different architectures where the lengths of integer *might* be
different from amd64. also, it's more readable than postfixes
like ULL in this context, as we really care about the exact
length of an integer in this context when counting the leading
zeros.
common/bloom_filter: return by return values not by input params
it'd be easier for the static analyzer (like GCC), to reason about if
a variable is initialized before being used.
this change also helps to improve the readability, and to silence the
false alarm like:
In file included from ../src/os/bluestore/BlueStore.h:42,
from ../src/os/bluestore/BlueStore.cc:26:
../src/common/bloom_filter.hpp: In member function 'void std::vector<_Tp, _Alloc>::_M_fill_insert(std::vector<_Tp, _Alloc>::iterator, std::vector<_Tp, _Alloc>::size_type, const value_type&) [with _Tp = bloom_filter; _Alloc = mempool::pool_allocator<mempool::mempool_bluestore_fsck, bloom_filter>]':
../src/common/bloom_filter.hpp:118:46: warning: '*((void*)(& __tmp)+8).bloom_filter::table_size_' may be used uninitialized in this function [-Wmaybe-uninitialized]
118 | mempool::bloom_filter::alloc_byte.deallocate(bit_table_, table_size_);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
this change silences following false positive warning:
In file included from ../src/include/encoding.h:41,
from ../src/kv/KeyValueDB.h:12,
from ../src/os/bluestore/bluestore_common.h:20,
from ../src/os/bluestore/BlueFS.cc:5:
../src/include/denc.h: In function ‘std::enable_if_t<(is_same_v<T, bluefs_extent_t> || is_same_v<T, const bluefs_extent_t>)> _denc_friend(T&, P&) [with T = bluefs_extent_t; P = ceph::buffer::v15_2_0::p$
../src/include/denc.h:639:11: warning: ‘shift’ may be used uninitialized in this function [-Wmaybe-uninitialized]
639 | shift += 7;
| ~~~~~~^~~~
../src/include/denc.h:613:7: note: ‘shift’ was declared here
613 | int shift;
| ^~~~~
Xiubo Li [Mon, 26 Apr 2021 08:24:31 +0000 (16:24 +0800)]
qa: use the pg autoscale mode to calcuate the pg_num
Setting the pg_num to 8 is too small that some osds maybe not covered by the
pools, some osds maybe overloaded. Remove the hardcodeing pg_num here and let
the pg autoscale mode to calculate it as needed, and at the same time set the
pg_num_min to 64 to avoid the pg_num to small.
If ec pool is used, for the test cases most datas will go to the ec pool and
the primary replicated pool will store a small amount of metadata for all the
files only, so set the target size ratio to 0.05 should be enough.
Fixes: https://tracker.ceph.com/issues/45434 Signed-off-by: Xiubo Li <xiubli@redhat.com>
Deepika Upadhyay [Wed, 23 Jun 2021 05:12:38 +0000 (10:42 +0530)]
mon/PGMap: DIRTY field as N/A in `df detail` when cache tier not in use
'ceph df detail' reports a column for DIRTY objects under POOLS even
though cache tiers not being used. In replicated or EC pool all objects
in the pool are reported as logically DIRTY as they have never been
flushed .
we display N/A for DIRTY objects if the pool is not a cache tier.
Sage Weil [Mon, 19 Jul 2021 17:46:38 +0000 (13:46 -0400)]
mgr/nfs: use bucket owner creds for rgw bucket export
The bucket owner can always read/write to the bucket, so use those creds
for the export. This is less complicated than setting up a dedicated
user anyway.
replace strict_iecstrtoll(const char *str,..) with
strict_iecstrtoll(std::string_view, ..). which is more convenient.
and both of them share the same implementation:
strict_sistrtoll() is but an alias of strict_si_cast<uint64_t>(..),
let's just drop the former. there are way too many thin wrappers in
strtol.{h,cc}. they don't offer lots of benefit to us.
projects / ceph.git / log