Loic Dachary [Tue, 19 Jan 2016 09:49:40 +0000 (16:49 +0700)]
ceph-disk: refactor prepare
The logic / code path is only modified to the extent necessary for the
refactor.
The Prepare class roughly replaces the prepare_main function but also
handles the prepare subcommand argument parsing. It creates the data and
journal objects and delegate the actual work to them via the prepare()
method.
The Prepare class assumes that preparing an OSD consists on the
following phases:
* optionally prepare auxiliary devices, such as the journal
* prepare a data directory or device
* populate the data directory with fsid etc. and optionally
symbolic links to the auxiliary devices
The PrepareDefault class is derived from Prepare and implements the
current model where there only is one auxiliary device, the journal.
The PrepareJournal class implements the *journal* functions
and is based on a generic class, PrepareSpace which handles the
allocation of an auxiliary device. The only journal specific feature is
left to the PrepareJournal class: querying the OSD to figure out if
a journal is wanted or not.
The OSD data directory is prepared via the PrepareData class. It creates
a file system if necessary (i.e. if a device) and populate the data
directory. Further preparation is then delegated to the auxiliary
devices (i.e. adding a symlink to the device for a journal).
There was some code paths related dmcrypt / multipath devices in
the prepare functions, although it is orthogonal. A class tree for
Devices was created to isolate that.
Although that was the primary reason for adding a new class tree, two
other aspects have also been moved there: ptypes and partition creation.
The ptypes are organized into a data structure with a few helpers in
the hope it will be easier to maintain. All references to the *_UUID
variables have been updated.
The creation of a partition is delegated to sgdisk and a wrapper helps
reduce the code redundancy.
The ptype of a given partition depends on the type of the device (is it
dmcrypt'ed or a multipath device ?). It is best implemented by
derivation so the prepare function does not need to be concerned about
how the ptype of a partition is determined.
Many functions could be refactored into a Device class and its
derivatives, but that was not done to minimize the size of the refactor.
Device knows how to create a partition and figure out the ptype tobe
DevicePartition a regular device partition
DevicePartitionMultipath a partition of a multipath device
DevicePartitionCrypt base class for luks/plain dmcrypt, can map/unmap
DevicePartitionCryptPlain knows how to setup dmcrypt plain
DevicePartitionCryptLuks knows how to setup dmcrypt plain
The CryptHelpers class is introduced to factorize the code snippets that
were duplicated in various places but that do not really belong
because they are convenience wrappers to figure out:
* if dmrypt should be used
* the keysize
* the dmcrypt type (plain or luks)
Loic Dachary [Tue, 19 Jan 2016 11:33:05 +0000 (18:33 +0700)]
tests: workaround ceph-disk global side effects
Because some variables are global in ceph-disk, tests that modify them
interact with each other in non-predictable ways. This will go away
eventually but requires a significant refactor. Workaround by running
one py.test per test file.
Loic Dachary [Tue, 19 Jan 2016 09:19:03 +0000 (16:19 +0700)]
ceph-disk: make all must setup.py install
Refactor the test / virtualenv setup in the same way it was done for
ceph-detect-init.
All shell tests use ceph-helpers.sh which is modified to add ceph-disk /
ceph-detect-init virtualenv/bin to the PATH to ensure the source version
is used even if ceph is installed.
See "ceph-detect-init: make all must setup.py install"
Loic Dachary [Tue, 19 Jan 2016 09:15:43 +0000 (16:15 +0700)]
tests: fix ceph-disk unit tests
Because ceph-disk unit tests were not run as part of make check, part of
the most recent changes broke them. This is a batch fix to sanitize the
situation. Since it is now run with make check, that won't happen again.
Loic Dachary [Tue, 19 Jan 2016 08:55:52 +0000 (15:55 +0700)]
ceph-detect-init: make all must setup.py install
When make all runs in the ceph-detect-init module, it does a "setup.py
build" which is not used. Replace it with a python setup.py install in a
virtualenv so that tests can add the virtualenv/bin to their PATH and
call ceph-detect-init from sources as they would if it was installed.
Part of run-tox.sh is moved to tools/setup-virtualenv.sh so that it can
be re-used by ceph-disk and other python modules.
Kefu Chai [Tue, 2 Feb 2016 06:24:46 +0000 (14:24 +0800)]
rgw: do not include "expat.h" in heade file
Building Ceph without the Expat development library currently results in
the following build error:
~/ceph> ./configure --without-fuse --without-tcmalloc --without-radosgw
...
~/ceph> make
...
CXX ceph_dencoder-rgw_dencoder.o
In file included from rgw/rgw_dencoder.cc:6:0:
rgw/rgw_acl_s3.h:9:19: fatal error: expat.h: No such file or directory
The ceph-dencoder binary is built with rgw_dencoder.cc, which includes
rgw_acl_s3.h, which in turn includes expat.h. However, configure.ac
currently only checks for Expat as part of radosgw.
but we don't need expat.h for testing the encoding of rgw structs, so
move "#include <expat.h>" into the source file to avoid this dependency.
and as a side effect, this also speed up the compilation.
Sage Weil [Mon, 1 Feb 2016 18:32:30 +0000 (13:32 -0500)]
osd/PG: fix scrub start object
This was changed in 05d79faa512210b0f0a91640d18db33b887a6e73 but it needs
to be more precise to avoid breaking BlueStore's assertions (that the
start position is within the collection range).
Igor Fedotov [Fri, 29 Jan 2016 17:19:24 +0000 (20:19 +0300)]
osd/PGBackend: PGBackend interface cleanup - make some interface methods pure virtual instead of assertion usage. Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>