rgw: fix get_resource_mgr() to correctly identify resource

Fixes: #5262
The original test was not comparing the correct string, ended up
with the effect of just checking the substring of the uri to match
the resource.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit 8d55b87f95d59dbfcfd0799c4601ca37ebb025f5)

PG: don't write out pg map epoch every handle_activate_map

We don't actually need to write out the pg map epoch on every
activate_map as long as:
a) the osd does not trim past the oldest pg map persisted
b) the pg does update the persisted map epoch from time
to time.

To that end, we now keep a reference to the last map persisted.
The OSD already does not trim past the oldest live OSDMapRef.
Second, handle_activate_map will trim if the difference between
the current map and the last_persisted_map is large enough.

Fixes: #4731
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

rgw: only append prefetched data if reading from head

Fixes: #5209
Backport: bobtail, cuttlefish
If the head object wrongfully contains data, but according to the
manifest we don't read from the head, we shouldn't copy the prefetched
data. Also fix the length calculation for that data.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit c5fc52ae0fc851444226abd54a202af227d7cf17)

rgw: don't copy object idtag when copying object

Fixes: #5204
When copying object we ended up also copying the original
object idtag which overrode the newly generated one. When
refcount put is called with the wrong idtag the count
does't go down.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit b1312f94edc016e604f1d05ccfe2c788677f51d1)

osd: initialize new_state field when we use it

If we use operator[] on a new int field its value is undefined; avoid
reading it or using |= et al until we initialize it.

Fixes: #4967
Backport: cuttlefish, bobtail
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>
(cherry picked from commit 50ac8917f175d1b107c18ecb025af1a7b103d634)

HashIndex: sync top directory during start_split,merge,col_split

Otherwise, the links might be ordered after the in progress
operation tag write.  We need the in progress operation tag to
correctly recover from an interrupted merge, split, or col_split.

Fixes: #5180
Backport: cuttlefish, bobtail
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 5bca9c38ef5187c7a97916970a7fa73b342755ac)

Merge remote-tracking branch 'upstream/wip_scrub_tphandle_bobtail' into bobtail

Fixes: #5159
Reviewed-by: Sage Weil <sage@inktank.com>

PG: ping tphandle during omap loop as well

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: reset timeout in _scan_list for each object, read chunk

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD,PG: pass tphandle down to _scan_list

Signed-off-by: Samuel Just <sam.just@inktank.com>

rgw: iterate usage entries from correct entry

Fixes: #5152
When iterating through usage entries, and when user id was
provided, we started at the user's first entry and not from
the entry indexed by the request start time.
This commit fixes the issue.

Backport: bobtail

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit 8b3a04dec8be13559716667d4b16cde9e9543feb)

mon: be a bit more verbose about osd mark down events

Put these in the cluster log; they are interesting.

This is a backport of 87767fb1fb9a52d11b11f0b641cebbd9998f089e.

Signed-off-by: Sage Weil <sage@inktank.com>

rgw: protect ops log socket formatter

Fixes: #4905
Ops log (through the unix domain socket) uses a formatter, which wasn't
protected.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit d48f1edb07a4d8727ac956f70e663c1b4e33e1dd)

librbd: make image creation defaults configurable

Programs using older versions of the image creation functions can't
set newer parameters like image format and fancier striping.

Setting these options lets them use all the new functionality without
being patched and recompiled to use e.g. rbd_create3().
This is particularly useful for things like qemu-img, which does not
know how to create format 2 images yet.

Refs: #5067
backport: cuttlefish, bobtail
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit aacc9adc4e9ca90bbe73ac153cc754a3a5b2c0a1)

rbd.py: fix stripe_unit() and stripe_count()

These matched older versions of the functions, but would segfault
using the current versions.

backport: cuttlefish, bobtail
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 53ee6f965e8f06c7256848210ad3c4f89d0cb5a0)

cls_rbd: make sure stripe_unit is not larger than object size

Test a few other cases too.

backport: cuttlefish, bobtail
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 810306a2a76eec1c232fd28ec9c351e827fa3031)

mon: fix validatation of mds ids in mon commands

Fixes: #4996
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 5c305d63043762027323052b4bb3ae3063665c6f)

Conflicts:

src/mon/MDSMonitor.cc

v0.56.6

ceph.spec.in:  Fix platform dependecies

Picked up an incorrect dependency merging the rbd udev rules update.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

rgw: don't send tail to gc if copying object to itself

Fixes: #4776
Backport: bobtail
Need to make sure that when copying an object into itself we don't
send the tail to the garbage collection.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit de5d1da810732ee48f41e8be18257053d862301b)

ceph_common.sh: re-sync get_name_list with master

We backported various items but didn't catch all the changes!  :(

Signed-off-by: Sage Weil <sage@inktank.com>

v0.56.5

ceph.spec.in: fix udev rules.d files handling

Move 50-rbd.rules into the ceph base package since the related
ceph-rbdnamer binary is part of this package. Use correct install
pattern.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

debian: only start/stop upstart jobs if upstart is present

This avoids errors on non-upstart distros (like wheezy).

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 030bf8aaa15837f898e453161eeaf1d52fc5779d)

Conflicts:
debian/ceph-mds.postinst

debian: stop ceph-mds before uninstalling ceph-mds

Fixes: #4384
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 9eb0d91b867ab980135d7c6ff6347d69d0a8a794)

Makefile.am: Add -lpthread to fix build on newer ld in Raring Ringtail

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 98f532e8000af281fa03b24da9ad2fda81755270)

Conflicts:
src/Makefile.am

Merge remote-tracking branch 'gh/bobtail-deploy' into bobtail-next

init-ceph: use remote config when starting daemons on remote nodes (-a)

If you use -a to start a remote daemon, assume the remote config is present
instead of pushing the local config.  This makes more sense and simplifies
things.

Note that this means that -a in concert with -c foo means that foo must
also be present on the remote node in the same path.  That, however, is a
use case that I don't particularly care about right now.  :)

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit cd7e52cc76878eed0f084f7b9a6cf7c792b716c6)

PG: call check_recovery_sources in remove_down_peer_info

If we transition out of peering due to affected
prior set, we won't trigger start_peering_interval
and check_recovery_sources won't get called.  This
will leave an entry in missing_loc_sources without
a matching missing set.  We always want to
check_recovery_sources with remove_down_peer_info.

Fixes: 4805
Backport: bobtail
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 81a6165c13c533e9c1c6684ab7beac09d52ca6b5)

PG: clear want_acting when we leave Primary

This is somewhat annoying actually.  Intuitively we want to
clear_primary_state when we leave primary, but when we restart
peering due to a change in prior set status, we can't afford
to forget most of our peering state.  want_acting, on the
other hand, should never persist across peering attempts.
In fact, in the future, want_acting should be pulled into
the Primary state structure.

Fixes: #3904
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>
(cherry picked from commit a5cade1fe7338602fb2bbfa867433d825f337c87)

mon: communicate the quorum_features properly when declaring victory.

Fixes #4747.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit fe68afe9d10bc5d49a05a8bafa644d57783447cf)

elector: trigger a mon reset whenever we bump the epoch

We need to call reset during every election cycle; luckily we
can call it more than once. bump_epoch is (by definition!) only called
once per cycle, and it's called at the beginning, so we put it there.

Fixes #4858.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 6ae9bbb5d03cb5695a4ebb7a3c20f729de1bd67a)

Conflicts:

src/mon/Elector.cc

Fix journal partition creation

With OSD sharing data and journal, the previous code created the
journal partiton from the end of the device. A uint32_t is
used in sgdisk to get the last sector, with large HD, uint32_t
is too small.
The journal partition will be created backwards from the
a sector in the midlle of the disk leaving space before
and after it. The data partition will use whichever of
these spaces is greater. The remaining will not be used.

This patch creates the journal partition from the start as a workaround.

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
(cherry picked from commit 56619ab91732bd2eacbef388311954f4e0230a30)

ceph.spec.in: put ceph-disk-* et al in correct sbindir

Signed-off-by: Sage Weil <sage@inktank.com>

debian: fix ceph.install

This got out of sync somewhere in cherry-picking all of these patches.

Signed-off-by: Sage Weil <sage@inktank.com>

Makefile.am: install ceph-* python scripts to /usr/bin directly

Install ceph-* scripts directly to $(prefix)$(sbindir) (which
normaly would be /usr/sbin) instead of moving it around after
installation in SPEC file or debian files.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 4d16f38f48e276497190c8bc03abc55c40e18eed)

ceph-disk:  CalledProcessError has no output keyword on 2.6

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
(cherry picked from commit a793853850ee135de14b9237f7023cadcdb8575c)

ceph-disk: fix some (local) variable names

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit c4eb7e6ddd593cd45ab8343da01355be7382723e)

ceph-disk: fix naming of local variable in is_mounted()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 3dd8b461219e64bb0f7a210dba5a9ab7c644f014)

ceph-disk: merge twice defined function is_mounted(dev)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit eaf31bf9f90ba9709a57a6870dbafa21142dae2c)

ceph-disk: define exception type

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 4c6d6442a89adc5b56e99cb4d2ed572f2ad192c9)

ceph-disk: fix Redefining name 'uuid' from outer scope

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 058eb923c5b7dab611901fdd1724ce2a7c180827)

ceph-disk: add missing space after comma

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 0080d1df7c7950e051840a543fc4bdabe6cc53e5)

ceph-disk list: say 'unknown cluster $UUID' when cluster is unknown

This makes it clearer that an old osd is in fact old.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e9b3f2e6e93426d631d4f8101fb431a523b0d88a)

ceph-disk: handle missing journal_uuid field gracefully

Only lower if we know it's not None.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 562e1716bd4b5372716e502a5e4dd872d381bfb7)

ceph-disk: implement 'list'

This is based on Sandon's initial patch, but much-modified.

Mounts ceph data volumes temporarily to see what is inside.  Attempts to
associated journals with osds.

Resolves: #3120
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 6a65b9131c444041d16b880c6a7f332776063a78)

ceph-disk: reimplement list_all_partitions

Use /dev/disk/by-id to list disks and their partitions.  This is more
accurate and correct than the previous (as-yet unused) implementation.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d3e49047ff405573aa41f45864cf315be23f5c50)

ceph-disk: reimplement is_partition

Previously we were assuming any device that ended in a digit was a
partition, but this is not at all correct (e.g., /dev/sr0, /dev/rbd1).
Instead, look in /dev/disk/by-id and see if there is a symlink that ends in
-partNN that links to our device.

There is probably still a better way...

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 20d594a889d62110ad03b761d8703f79f8eea6ad)

ceph-disk: conditionally remove mount path

umount removes it on success; only remove it here if it is still there.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 4362934a94c5a6f7521f06aa1255e111094e1d88)

ceph-disk: ignore udevadm settle return code

If we time out, just continue and let the next step fail.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 2ae297ccb812c00204ec0c4dc614821c506062b9)

ceph-disk: rename some local variabels in list_*partitions

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit b7d7e6894c550a7afa8dfb5bfa2bc54b5d331178)

ceph-disk: fix naming of a local variable in find_cluster_by_uuid

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 0b5fcfffe6d2f69bd4318cc93ef73195d948b9c5)

ceph-disk: rename some constants to upper case variable names

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 8a999ded088e688fd3f4a7c27127b7c06f0b2f66)

ceph-disk: add some more docstrings

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 86e55f5448c4b5b46b74d2d89b01d1e64b1ea826)

ceph-disk: print subprocess.CalledProcessError on error

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 543327b1f2a9efe8083bb196433c4bcf83883269)

ceph-disk: fix indention

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 2d26bcc07162a5176cdbc1748b829e3f39653697)

ceph-disk:  udevadm settle before partprobe

After changing the partition table, allow the udev event to be
processed before calling partprobe.  This helps prevent partprobe
from getting a resource busy error on some platforms.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
(cherry picked from commit 9eda8e5d5abf0743a2ad484806cfb2018243515f)

ceph-disk: re-add python 2.7 dependency comment

FIXME!

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 6d63752c8fde91cdab306d1ca689690b269fe977)

ceph-disk: cast output of subprocess.Popen() to str()

Cast output of subprocess.Popen() to str() to be able to use
str.split() and str.splitlines() without warnings from pylint.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 87691dc46edfba11c370592dbb533772190be4b2)

ceph-disk: fix adjust_symlink() replace 'journal' with 'target'

Replace 'journal' variable with 'target' since journal doesn't
exist in this function.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 3575feb7d2e5f4e35c5df193a1f8c9f08f88fcf4)

ceph-disk: fix adjust_symlink() replace 'canonical' with 'path'

Replace 'canonical' variable with 'path' since canonical doesn't
exist in this function.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit a27cb85b21610b4a9f8eb24f8f6cd04902349728)

ceph-disk: there is no os.path.lstat use os.lstat

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 6d3247b5c02c39a66666a5833106dbc23044e436)

ceph-disk: remove unused variable key from prepare_journal_dev()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 97b4f8d66bef2328fa53f9e508eb38f8b8d49632)

ceph-disk: fix except to catch OSError

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 9daf6cfce2d57509d896eae28bb97146a682a306)

ceph-disk: add missing space after >> operator

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 0ada43f79d2b1f9f84367e558c6d1a3e90e5c9b2)

fix: Redefining name 'uuid' from outer scope (line 14)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit d3c60dc8cad1db1d5df1c740bc805aaf9ba606ba)

ceph-disk: remove unused variables from list_partitions()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 6a8120d4b0c4cfa851d473532eb2366534f8653d)

ceph-disk: fix /dev/dm-[0-9] handling list_all_partitions()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 3af7a1ac5bf24bf832d7180002281d6b5853e85a)

ceph-disk: rename local variable shadowing builtin

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 9bcf5b64f45ab6c4bdedf820ed111319b2dbd778)

ceph-disk: remove twice defined identical function unmount

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 4adf088107586be7b356d1e963570cdab232809e)

ceph-disk: remove twice defined function mount

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit ea26ea0d81a23aa76076ad5441c3b1aadfba3b44)

ceph-disk: remove double defined function get_conf

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit c57daa3c6e03e8974e133d3a2d9bc3d6f06f4faf)

ceph-disk: rename local variable shadowing builtin

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 57dde5c8b18ff4ccd53a30bb94119c0ffceb52b7)

ceph-disk: install and package

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit c2602d749023b24ac22d8cfce6e04889078f14d8)

Conflicts:

debian/ceph.install

ceph-disk: simplify command dispatch

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit f287c6f90af0dfdd41358846b069aa3c54b600b3)

ceph-disk: consolidate exceptions

Use a single exception type, and catch it at the top level.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit a019753bd3897ee0f5d9c3563c653c5457d5e67d)

ceph-disk: consolidate ceph-disk-* into a single binary

 ceph-disk prepare ...
 ceph-disk activate ...
 ceph-disk ...

This let's us share code (we were already duplicating a bunch of stuff!)
and to add new commands easily.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 20e4ba5cebb7b4989d1745932f7a412c28504ee1)

ceph-disk-prepare: 'mkfs -t' instead of 'mkfs --type='

Older mkfs (el6) doesn't like --type=.

Fixes: #4495
Reported-by: Alexandre Maragone <alexandre.maragone@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d47759429a6e2fdd392265e90b67046302c97644)

ceph-disk-activate: identify cluster .conf by fsid

Determine what cluster the disk belongs to by checking the fsid defined
in /etc/ceph/*.conf.  Previously we hard-coded 'ceph'.

Note that this has the nice side-effect that if we have a disk with a
bad/different fsid, we now fail to activate it.  Previously, we would
mount and start ceph-osd, but the daemon would fail to authenticate
because it was part of the wrong cluster.

Fixes: #3253
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 80af5fb887f30792c342ac16da9ed95d7e89e491)

ceph-disk-activate: abort if target position is already mounted

If the target position is already a mount point, fail to move our mount
over to it.  This usually indicates that a different osd.N from a
different cluster instances is in that position.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 7370b5564606474f11b9ac5afb7cc60e0ac36ed1)

Revert "ceph-disk-activate: rely on default/configured keyring path"

This reverts commit 936b8f20af1d390976097c427b6e92da4b39b218.

This is necessary because we mount the osd in a temporary location.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit b6102c0945a0d68b7c5ff7f2254a1a994788ca0f)

Revert "ceph-disk-activate: don't override default or configured osd journal path"

This reverts commit 813e9fe2b4291a1c1922ef78f031daa9b78fe53b.

We run --mkfs with the osd disk mounted in a temporary location, so it is
necessary to explicitly pass in these paths.

If we want to support journals in a different location, we need to make
ceph-disk-prepare update the journal symlink accordingly.. not control it via
the config option.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 3e628eee770508e750f64ea50179bbce52e7b8e0)

ceph-disk-activate: rely on default/configured keyring path

No reason to override the default or configured value here.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 936b8f20af1d390976097c427b6e92da4b39b218)

ceph-disk-activate: don't override default or configured osd journal path

There is no reason not to rely on the default or obey any configured
value here.

Fixes: #4031
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 813e9fe2b4291a1c1922ef78f031daa9b78fe53b)

ceph-disk-prepare: move in-use checks to the top, before zap

Move the in-use checks to the very top, before we (say) zap!

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 32407c994f309cd788bf13fe9af27e17a422309a)

ceph-disk-prepare: verify device is not in use by device-mapper

Be nice and tell the user which devices/mappings are consuming the device,
too.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit a6196de9e2f3ca9d67691f79d44e9a9f669443e9)

ceph-disk-prepare: verify device is not mounted before using

Make sure the data and/or journal device(s) are not in use (mounted)
before using them.  Make room for additional "in-use" checks in the future.

Closes: #3256
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 3bd0ac0ab011c4cdf0121f0d9732938d085fb8bf)

ceph-disk-prepare: clean up stupid check for a digit

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit f03f62697f170d42b4b62c53d2860ff2f24a2d73)

ceph-disk-prepare: use os.path.realpath()

My janky symlink resolution is broken in various ways.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 59505546e52a175435881b431bd349d532ae627e)

ceph.spec.in: add new Requires from ceph-disk-prepare

Added new Requires from ceph-disk-prepare: cryptsetup, gptfdisk,
parted and util-linux.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 5c3f5c5b69a9edc99138d4f1ddb016689303dc28)

Conflicts:

ceph.spec.in

debian: require cryptsetup-bin

This is needed for ceph-disk-prepare's dmcrypt support.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit cfcaceac44d6b7b7c55e81d0bfb05f4893f3b1d0)

Conflicts:

debian/control

Fix: use absolute path with udev

Avoids the following: udevd[61613]: failed to execute '/lib/udev/bash'
'bash -c 'while [ ! -e /dev/mapper/....

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
(cherry picked from commit 785b25f53dc7f8035eeba2aae8a196e3b102d930)

ceph-disk-prepare: -f for mkfs.xfs only

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit fecc3c3abf1176f4c7938e161559ea2db59f1cff)

debian: fix start of ceph-all

Tolerate failure, and do ceph-all, not ceph-osd-all.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit aff0bb6fdc8ca358f7ac1e941bb9cfecbefb4bb6)

ceph-disk-prepare: always force mkfs.xfs

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
(cherry picked from commit d950d83250db3a179c4b629fd32cd7bc8149997e)

udev: trigger on dmcrypted osd partitions

Automatically map encrypted journal partitions.

For encrypted OSD partitions, map them, wait for the mapped device to
appear, and then ceph-disk-activate.

This is much simpler than doing the work in ceph-disk-activate.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e090a92a20f4161f473d16bc966f7d6aacac75ee)

ceph-disk-prepare: add initial support for dm-crypt

Keep keys in /etc/ceph/dmcrypt-keys.

Identify partition instances by the partition UUID.  Identify encrypted
partitions by a parallel set of type UUIDs.

Signed-off-by: Alexandre Marangone <alexandre.maragone@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit c6ac0ddf91915ba2aeae46d21367f017e18e82cd)

ceph-disk-activate: pull mount options from ceph.conf

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
(cherry picked from commit e7040f55f01db3de7d5cebfc79de50c8b6ad5d45)

ceph-disk-activate: use full paths for everything

We are run from udev, which doesn't get a decent PATH.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit b1c0fccba42dd184a2891ee873c0d6d8f8c79d14)

ceph-disk-prepare: do partprobe after setting final partition type

This is necessary to kick udev into processing the updated partition and
running its rules.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 95835de9f80712eb26652ac6b66ba9c5eeb093d6)