Ramana Raja [Fri, 13 Mar 2020 14:00:05 +0000 (19:30 +0530)]
mgr/volumes: type convert uid and gid to int
This fix is only needed in nautilus. The issue
was observed during upstream teuthology testing in CentOS 7.5
machines.
File "/usr/share/ceph/mgr/volumes/fs/async_cloner.py", line 153, in do_clone
bulk_copy(fs_handle, src_path, dst_path, should_cancel)
File "/usr/share/ceph/mgr/volumes/fs/async_cloner.py", line 144, in bulk_copy
cptree(source_path, dst_path)
File "/usr/share/ceph/mgr/volumes/fs/async_cloner.py", line 134, in cptree
sync_attrs(fs_handle, d_full_dst, stx)
File "/usr/share/ceph/mgr/volumes/fs/async_cloner.py", line 77, in sync_attrs
fs_handle.lchown(target_path, source_statx["uid"], source_statx["gid"])
The issue wasn't observed in master/octopus teuthology
testing.
Rishabh Dave [Wed, 20 Feb 2019 16:36:41 +0000 (22:06 +0530)]
qa: allow keeping sudo in command arguments
sudo cannot be omitted from the given command's arguments, when running
passwd, chown and, specially, when sudo is used for running the given
command as different user.
Venky Shankar [Fri, 7 Feb 2020 05:26:03 +0000 (00:26 -0500)]
mgr/volumes: synchronize inode attributes for cloned subvolumes
Synchronize ownership, permission and inode timestamp (access and
modification times) for all supported inode types. Note that inode
timestamps are synchronized upto seconds granularity.
Nizamudeen [Tue, 10 Mar 2020 16:32:41 +0000 (22:02 +0530)]
mgr/dashboard: NoRebalance flag is added to the Dashboard
This commit will add a norebalance flag into the Cluster-wide Flags in the OSDs which can be set/unset. Fixes: https://tracker.ceph.com/issues/44543 Signed-off-by: Nizamudeen <nia@redhat.com>
(cherry picked from commit 7e30c261c0c57ccb26097e56ee442f3d6bf08e4a)
Marcus Watts [Wed, 30 May 2018 20:37:31 +0000 (16:37 -0400)]
rgw: making implicit_tenants backwards compatible.
In jewel, "rgw keystone implicit tenants" only applied to swift. As of
luminous), this option applies to s3 also.
Sites that used this feature with jewel now have outstanding data that
depends on the old behavior.
The fix here is to expand "rgw keystone implicit tenants" so that it
can be set to any of "none", "all", "s3" or "swift" (also 0=false=none,
1=true=all). When set to "s3" or "swift", the actual id lookup
is also partitioned.
Formerly "rgw keystone implicit tenants" was a legacy opt.
This change converts it to the new style of option,
including support for dynamically changing it.
Fixes: http://tracker.ceph.com/issues/24348 Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit 3ba7be8d1ac7ee43e69eebb58263cd080cca1d38)
Kefu Chai [Thu, 5 Mar 2020 15:42:13 +0000 (23:42 +0800)]
mgr: update metadata if an osd just joins
instead of using "front_address" for checking if we have a new OSD
reusing existing a known identify shows up in the osdmap, it'd be
simpler to compare the up_from epoch with the osdmap's epoch. as
objecter will subscribe **every** osdmap after mgr boots. so mgr
should be able to see the osdmap when the osd joins the cluster
where the up_from epoch is identical to osdmap's epoch.
this way is simpler than existing approach. but it will involve
more overhead if osd reboots frequently without changing their
metadata.
before this change, the metadata is requested/updated only if
the public (front) address is changed.
after this change, the metadata is requested/updated whenever
an osd reboots.
Kefu Chai [Thu, 5 Mar 2020 15:35:46 +0000 (23:35 +0800)]
mgr: update "hostname" when we already have the daemon state from the same entity
there is chance that we reuse the identity of a daemon and deploy the
daemon on a different host. in that case, the existing daemon state
should be updated with the new hostname.
Sage Weil [Mon, 2 Dec 2019 13:43:54 +0000 (07:43 -0600)]
mon: cap keys in mon_sync messages
The previous cap was set at 1 MB. However, a user was experiencing mon
timeouts while syncing the purged_snap_epoch * keys, which are ~20 bytes
each. Reducing the max payload to 64K resolved the problem, which maps
to (very!) roughly 1500 keys per message. Set our limit a bit higher than
that since we just made this quite a bit more efficient. Most of the time
the keys are larger than 20 bytes and we wouldn't hit the key limit, but
having one ensures that we won't burn too much CPU in one go when we do
have lots of these little keys.
Sage Weil [Tue, 12 Nov 2019 20:51:41 +0000 (14:51 -0600)]
mon/MonitorDBStore: improve get_chunk_tx limits
The old version was horribly inefficient in that it would reencode the
transaction on every iteration.
Instead, estimate the size if we add an item and stop it if looks like it
will go over. This isn't super precise, but it's close enough, since the
limits are approximate.
Drop the single-use helper since it only makes the code harder to
follow.
Brad Hubbard [Tue, 3 Mar 2020 05:58:35 +0000 (15:58 +1000)]
mgr/run-tox-tests: Fix issue with PYTHONPATH
Something changed recently on Bionic which caused tox to fail when
PYTHONPATH is a relative path. For some reason the path is mangled by
the time it gets to pytest so we need to ensure we are using an absolute
path. This seems to be nautilus specific, at least ATM.
Ulrich Weigand [Thu, 23 Jan 2020 12:07:08 +0000 (13:07 +0100)]
cmake: Improve test for 16-byte atomic support on IBM Z
Commit d1b9d14324586d02dc6c8dd0a9bdf4b98ae16416 added a check for
16-byte atomics without library support on IBM Z. Unfortunately
it turns out this test only works correctly when the test case
is built without optimization. (This is normally true, but it
may not be the case when passing explicit CXXFLAGS to cmake.)
The underlying reason is that GCC may choose to use either an
inline implementation of the 16-byte atomics or library calls,
depending on whether or not it is able to prove the atomic
variable is properly aligned. At -O0 it is never able to prove
that, but at higher optimization levels it depends on the complexity
of the expression (in particular, whether GCC can track down the
definition of the underlying object).
As the test case uses a very simple expression, it may happen that
this test can be built without requiring library support, but some
of the "real" uses of atomics in Ceph code cannot. This defeats
the whole purpose of the test at configure time.
Fixed by making the access pattern in the test more complex, so
that the test fails even at high optimization levels.
Alex Zhang [Sun, 29 Sep 2019 09:33:58 +0000 (02:33 -0700)]
common: Fix multiple logical errors in get_device_id.
0. If blkdev.serial exists, the serial should be used. The original impl seems wrong (if serial does not exist, then use the value from the uninitialized buffer, or even worse, use the value from the last call (model))
1. When using fallback methods, device id should only be returned when both model and serial are present. The original impl looks like a logical error.
Kefu Chai [Wed, 29 May 2019 09:45:35 +0000 (17:45 +0800)]
common/blkdev.c: check retval of snprintf()
as snprintf()'ed string could be truncated, to properly use this
function, we need to check its return value.
to silence warning like
../src/common/blkdev.cc: In member function ‘int64_t
BlkDev::get_string_property(blkdev_prop_t, char*, size_t) const’:
../src/common/blkdev.cc:165:15: warning: ‘%s’ directive output may be
truncated writing up to 4095 bytes into a region of size between 4085
and 4089 [-Wformat-truncation=]
165 | "%s/block/%s/%s", sysfsdir(), dev, propstr);
| ^~
In file included from /usr/include/stdio.h:873,
from /usr/include/c++/9/cstdio:42,
from /usr/include/c++/9/ext/string_conversions.h:43,
from /usr/include/c++/9/bits/basic_string.h:6493,
from /usr/include/c++/9/string:55,
from /usr/include/c++/9/bits/locale_classes.h:40,
from /usr/include/c++/9/bits/ios_base.h:41,
from /usr/include/c++/9/ios:42,
from /usr/include/c++/9/ostream:38,
from /usr/include/c++/9/iterator:64,
from
/opt/ceph/include/boost/iterator/iterator_traits.hpp:10,
from
/opt/ceph/include/boost/range/iterator_range_core.hpp:26,
from
/opt/ceph/include/boost/algorithm/string/replace.hpp:16,
from ../src/common/blkdev.cc:31:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:35: note:
‘__builtin___snprintf_chk’ output 9 or more bytes (assuming 4108) into a
destination of size 4096
67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL
- 1,
|
~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
68 | __bos (__s), __fmt, __va_arg_pack ());
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There is a problem with mimic releases where pg_creates can stall for a
long time doing the build_pg_history while holding osd_lock. That lock
is also used by the tell command processing queue, which means that
commands like 'flush_pg_stats' can block for long periods...and time out.
This is currently happening with mimic->nautilus upgrades. Note that
the problem is mostly fixed in nautilus and totally fixed in octopus, so
this is just a matter of tolerating slow behavior in old releases for the
purposes of the upgrade tests.
Work around this by increasing the timeout from 120s -> 900s.
Fixes: https://tracker.ceph.com/issues/43914 Signed-off-by: Sage Weil <sage@redhat.com>
Matt Benjamin [Fri, 19 Jul 2019 20:32:20 +0000 (16:32 -0400)]
RGWLC: fix expiration header tag match
Need to match key->value
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit af327f21aa377a7abd0290814bfa7333db5443c3) Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
Matt Benjamin [Fri, 3 May 2019 17:48:31 +0000 (13:48 -0400)]
rgw: fix header timestamp
The AWS example of this header intends to be RFC822-compliant.
Found by Tyler Brekke <tbrekke@redhat.com>.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 6da5be5aba0820dc91aa44d4b63cd490b39371db) Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
Matt Benjamin [Tue, 19 Feb 2019 16:17:45 +0000 (11:17 -0500)]
rgw: complete expiration header (object tags)
The expiration header tag processing is complete, but the
passed RGWObjTags argument was never initialized. Now it is
initialized in RGWGetObj and RGWPutObj, as required.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 8981c5e9f688b13a00345e069c2ce1e62fb0a588) Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
projects / ceph.git / log