Kadu Ribeiro [Tue, 25 Aug 2015 01:25:57 +0000 (22:25 -0300)]
doc: update ruby doc with the aws-sdk gem usage
Since I'm using ceph with the `aws-sdk` gem (https://github.com/aws/aws-sdk-ruby) instead `aws-s3` (https://github.com/marcel/aws-s3) because the aws-s3 have a trouble with the new active support (https://github.com/marcel/aws-s3/issues/98) (and the downgrade active-support wasn't a option), I proposed change the doc to receive the usage instructions with the aws-sdk gem.
I used ceph with aws-sdk gem with this commands.
Thanks so much
Signed-off-by: Carlos E Ribeiro <mail@carlosribeiro.me>
buffer: make buffer::exception classes undefined in dynamic objects
On OSX, if the an exception class is declared and defined in header file,
but it ends up being compiled as private symbols in different binaries.
The exception handling code will take the two compiled exception classes
as different types! In our case, the one in libcls_xxx.so and the one is
ceph-osd are considered as different classes, thus the try-catch statement
fails to work.
The fix is force buffer::exception classes undefined in libcls_xxx.so. The
ibcls_xxx.so are compiled with '-undefined dynamic_lookup' option. when
it is loaded into ceph-osd, buffer::exception classes in ceph-osd will be
used.
On Darwin, getxattr/removexattr return ENOATTR when extended
attribute does not exist. But lots of ceph codes expect ENODATA
is returned in that case. On Darwin, ENOATTR and ENODATA have
different values, so we convert ENOATTR to ENODATA.
there is no sem_timedwait on OSX. For inter-threads communication,
we can replace sem_t with pthread_cond_t. But for inter-processes
communication, it's extremely hard to figure out how to do a timed
wait. So disable the test case on OSX
msg: fix encoding/decoding sockaddr_storage on DARWIN/FreeBSD
sockaddr_storage on DARWIN/FreeBSD is different from sockaddr_storage
on Linux. sockaddr_storage on DARWIN/FreeBSD includes a ss_len field.
Besides, Its ss_len and ss_family fields are 'unsigned char'.
Loic Dachary [Thu, 27 Aug 2015 20:22:43 +0000 (22:22 +0200)]
ceph-disk: integration tests for multipath
Add integration tests for multipath to the ceph-disk workunit, with the
following caveats:
A workaround is added (explicit call to ceph-disk activate) until the
CentOS activation bug http://tracker.ceph.com/issues/12786 is fixed.
The tests do not run on Ubuntu because of the multipath / device mapper
bug
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1488688
and it has not been tested on Debian.
Loic Dachary [Thu, 27 Aug 2015 20:17:21 +0000 (22:17 +0200)]
ceph-disk: implement workunit
This new ceph-disk workunit re-implements the tests that previously were
in the src/test/ceph-disk.sh src/test/ceph-disk-root.sh scripts and is
meant to run in a virtual machine instead of docker.
Loic Dachary [Thu, 27 Aug 2015 11:14:13 +0000 (13:14 +0200)]
ceph-disk: implement list --format json
The ceph-disk list command is reworked in two parts:
1) the list_devices function that build an internal structure with all
the information regarding disks and partitions.
2) a function to display the internal structure in plain text or json
The ceph-disk list show the plain text version and is backward
compatible.
The ceph-disk list --format json output has more information about each
device than the plain text version and is intended for scripts.
The unit tests cover all modified lines (2610 to 2849).
Loic Dachary [Thu, 27 Aug 2015 10:21:48 +0000 (12:21 +0200)]
tests: obsolete ceph-disk root tests
They were designed to run in a docker container using loop devices
instead of disks. Although this was fit for ceph-disk activate tests for
regular and dmcrypt devices, a docker instance does not have its own
udev instance it is not possible to run tests involving udev events
without interfering with the host.
Loic Dachary [Sun, 16 Aug 2015 00:37:01 +0000 (02:37 +0200)]
ceph-disk: add multipath support
A multipath device is detected because there is a
/sys/dev/block/M:m/dm/uuid file with the mpath- prefix (or part\w+-mpath
prefix).
When ceph-disk prepares data or journal devices on a multipath device,
it sets the partition typecode to MPATH_JOURNAL_UUID, MPATH_OSD_UUID and
MPATH_TOBE_UUID to
a) help the udev rules distinguish them from other devices in
devicemapper
b) allow ceph-disk to fail if an attempt is made to activate a device
with this type without accessing it via a multipath device
The 95-ceph-osd.rules call ceph-disk activate on partitions of type
MPATH_JOURNAL_UUID, MPATH_OSD_UUID. It relies on ceph-disk to do nothing
if the device is not accessed via multipath.
Loic Dachary [Mon, 17 Aug 2015 22:18:51 +0000 (00:18 +0200)]
ceph-disk: is_held must ignore multipath devices
Always return an empty list when is_held is called on a multipath
device.
The dmcrypt logic base decisions depending on the holders/slaves
relationship. Such relationships can also exists for multipath devices
but do not have the same semantic.
Loic Dachary [Mon, 17 Aug 2015 22:04:32 +0000 (00:04 +0200)]
ceph-disk: rework get_partition_{type,uuid}
Mimic the get_partition_type implementation after get_partition_uuid
and factorize them to reduce the code footprint.
The get_partition_type implementation is based on blkid: it is complex
and fragile. Since sgdisk is consistently used to create partitions, use
it instead. It is already used for get_partition_uuid and there does not
seem to be any reason for concern.
Loic Dachary [Mon, 17 Aug 2015 21:51:24 +0000 (23:51 +0200)]
ceph-disk: multipath support for split_dev_base_partnum
split_dev_base_partnum returns the path of the whole disk in
/dev/mapper. The base variable name to designate the device for the
whole disk is a misnomer since it cannot be used as a basename to
rebuild the parition device name in the case of multipath.
The logic of split_dev_base_partnum for devices is reworked to use
/sys/dev/block/M:m/partition instead of device name parsing.
Loic Dachary [Mon, 17 Aug 2015 21:25:45 +0000 (23:25 +0200)]
ceph-disk: multipath support for is_partition and list_partitions
The is_partition predicate and the list_partitions function support
devices managed by multipath.
A set of helpers dedicated to multipath devices is implemented because
the content of the corresponding /sys directory does not use the same
conventions as regular devices regarding partitions.
Instead of relying on subdirectories such as /sys/block/name/name1, the
devicemapper uuid file is used and expected to start with part\d+. The
holders/slaves directories provide pointers between the whole device and
the partition devices.
Although these structural differences reduce the opportunity for
code factorization, it is easier for backward compatibility since the
multipath specific logic is limited to if is_mpath(dev) branches.
Loic Dachary [Mon, 17 Aug 2015 21:02:40 +0000 (23:02 +0200)]
ceph-disk: --verbose shows a stack trace on error
When running with --verbose, do not hide the stack trace from the user
when an exception is raised. It is most helpful to figure out when the
exception actually happened.
Loic Dachary [Mon, 17 Aug 2015 20:52:25 +0000 (22:52 +0200)]
ceph-disk: is_mpath predicate for multipath devices
The is_mpath predicate returns True if a device is managed by
multipath. It is based on the devicemapper uuid content which is
expected to always contain the mpath- string to identify the multipath
subsystem.
The block_path helper is added to convert the path to a device to the
/sys directory that describes it. It uses the major and minor number
instead of the device name because it is more reliable. The rationale
including an actual example is added as a comment for future
maintainers.
Loic Dachary [Mon, 17 Aug 2015 20:22:12 +0000 (22:22 +0200)]
tests: ceph-disk tests may use system ceph-{mon,osd}
Allow ceph-disk.sh to run to test ceph as installed from packages.
When run from sources, ceph-disk.sh is expected to use the binaries from
the source tree. It is enough to prepend . to the PATH. There is no need
to prefix each binary with ./
The virtualenv is also only necessary when running from sources and
setting it up for ceph-detect-init is only done if in the source tree.
projects / ceph.git / log