Sage Weil [Wed, 21 Oct 2015 20:45:11 +0000 (16:45 -0400)]
os/newstore: make key names more efficient
- pack u32 and u64 in binary (instead of in hex)
- avoid duplicating the object name while making things still
sort by (key,name). Use < when key < name, = when key == name,
> when key > name) as a prefix. And in the = case (which is
basically always) include the name just once.
Sage Weil [Mon, 21 Sep 2015 01:56:50 +0000 (21:56 -0400)]
os/newstore: fixed fragment size
Instead of a single, variable-length fragment for each object,
set a fixed size (newstore_min_frag_size = 1 MB) and stripe the
object over these. The last fragment will be smaller
than 1 MB if the object is not a multiple of 1 MB.
On write, this is basically free: we can just as cheaply write
4 inodes created together and fsync them than we can one. On
overwrite, it allows us to replace individual fragments and avoid
write-ahead many cases.
On read it is a bit slower because of inode lookups and disk
seeks. In the common case (big object written sequentially) we
hope that fs prefetching will hide most of it (e.g., all inodes
will be loaded together in the same metadata btree node, and the
files' data is written sequentially on disk).
Allowing for a singe large fragment in the case of a sequentially
written large object may save us something, but it complicates
the code significantly.
Kefu Chai [Wed, 23 Dec 2015 06:36:39 +0000 (14:36 +0800)]
ceph-detect-init: fix py3 test
* in python3, None can not be compared with a str. and
we should always set codename with a non empty str, so update
TestCephDetectInit.test_debian so it always set a code name.
and assert(codename and distroname) in choose_init().
Yehua [Wed, 23 Dec 2015 05:08:44 +0000 (13:08 +0800)]
doc:adding "--allow-shrink" in decreasing the size of the rbd block to distinguish from the increasing option
In the original file, the increasing and decreaing of the size of the rbd block shares the same option:
"rbd resize --size 2048 foo".
However, it is not proper, as the "--allow-shrink" needs to be added while decreasing the size of the rbd block.
As a result, it is necessary to make a distinguish between these two options as follows:
"rbd resize --size 2048 foo (to increase)"
"rbd resize --size 2048 foo --allow-shrink (to decrease)"
Ilya Shipitsin [Tue, 22 Dec 2015 09:02:36 +0000 (14:02 +0500)]
ceph-fuse: fix double free of args
Checking src/ceph_fuse.cc...
[src/ceph_fuse.cc:55]: (error) Memory pointed to by 'argv' is freed twice.
[src/ceph_fuse.cc:55]: (error) Deallocating a deallocated pointer: argv
Sage Weil [Mon, 21 Dec 2015 17:58:32 +0000 (12:58 -0500)]
Merge pull request #6926 from dachary/wip-14080-ceph-disk-udevadm
ceph-disk: fix failures when preparing disks with udev > 214
On CentOS 7.1 and other operating systems with a version of udev greater or equal to 214,
running ceph-disk prepare triggered unexpected removal and addition of partitions on
the disk being prepared. That created problems ranging from the OSD not being activated
to failures because /dev/sdb1 does not exist although it should.
Loic Dachary [Fri, 18 Dec 2015 23:53:03 +0000 (00:53 +0100)]
ceph-disk: protect deactivate with activate lock
When ceph-disk prepares the disk, it triggers udev events and each of
them ceph-disk activate. If systemctl stop ceph-osd@2 happens while
there still are ceph-disk activate in flight, the systemctl stop may be
cancelled by the systemctl enable issued by one of the pending ceph-disk
activate.
This only matters in a test environment where disks are destroyed
shortly after they are activated.
Loic Dachary [Fri, 18 Dec 2015 16:03:21 +0000 (17:03 +0100)]
ceph-disk: use blkid instead of sgdisk -i
sgdisk -i 1 /dev/vdb opens /dev/vdb in write mode which indirectly
triggers a BLKRRPART ioctl from udev (starting version 214 and up) when
the device is closed (see below for the udev release note). The
implementation of this ioctl by the kernel (even old kernels) removes
all partitions and adds them again (similar to what partprobe does
explicitly).
The side effects of partitions disappearing while ceph-disk is running
are devastating.
sgdisk is replaced by blkid which only opens the device in read mode and
will not trigger this unexpected behavior.
The problem does not show on Ubuntu 14.04 because it is running udev <
214 but shows on CentOS 7 which is running udev > 214.
git clone git://anonscm.debian.org/pkg-systemd/systemd.git
systemd/NEWS:
CHANGES WITH 214:
* As an experimental feature, udev now tries to lock the
disk device node (flock(LOCK_SH|LOCK_NB)) while it
executes events for the disk or any of its partitions.
Applications like partitioning programs can lock the
disk device node (flock(LOCK_EX)) and claim temporary
device ownership that way; udev will entirely skip all event
handling for this disk and its partitions. If the disk
was opened for writing, the close will trigger a partition
table rescan in udev's "watch" facility, and if needed
synthesize "change" events for the disk and all its partitions.
This is now unconditionally enabled, and if it turns out to
cause major problems, we might turn it on only for specific
devices, or might need to disable it entirely. Device Mapper
devices are excluded from this logic.
Loic Dachary [Wed, 16 Dec 2015 14:57:03 +0000 (15:57 +0100)]
ceph-disk: dereference symlinks in destroy and zap
The behavior of partprobe or sgdisk may be subtly different if given a
symbolic link to a device instead of an actual device. The debug output
is also more confusing when the symlink shows instead of the device it
points to.
Always dereference the symlink before running destroy and zap.
The default of 120 seconds may be exceeded when the disk is very slow
which can happen in cloud environments. Increase it to 600 seconds
instead.
The partprobe command may fail for the same reason but it does not have
a timeout parameter. Instead, try a few times before failing.
The udevadm settle guarding partprobe are not necessary because
partprobe already does the same. However, partprobe does not provide a
way to control the timeout. Having a udevadm settle after another is
going to be a noop most of the time and not add any delay. It matters
when the udevadm settle run by partprobe fails with a timeout because
partprobe will silentely ignores the failure.
Yuan Zhou [Mon, 21 Dec 2015 07:30:44 +0000 (15:30 +0800)]
cache-tier: Allow to config eviction check max size
This patch adds the option for eviction check size in cache-tier.
On a busy setup, it's better to check bigger number of objects so
the eviction is faster.
Matt Benjamin [Sun, 20 Dec 2015 18:32:13 +0000 (13:32 -0500)]
Dispatcher.h: include assert.h
The inline ms_fast_dispatch implementation calls assert, which
puts asset.h in the interface.
The fact that many Dispatcher descendants don't implement
ms_fast_dispatch prevents making the method pure virtual--which
suggests that maybe there is a need for a FastDispatcher
interface that inherits Dispatcher and introduces ms_fast_dispatch
and related?
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
xiexingguo [Wed, 18 Nov 2015 09:57:17 +0000 (17:57 +0800)]
Objecter: potential null pointer access in list_(n)objects.
In list_objects and list_nobjects, we are possibly access a null returned pointer from the osdmap->get_pg_pool() call. Fixes: #13822 Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
projects / ceph.git / log