Merge remote-tracking branch 'gh/bobtail-deploy' into bobtail-next

init-ceph: use remote config when starting daemons on remote nodes (-a)

If you use -a to start a remote daemon, assume the remote config is present
instead of pushing the local config.  This makes more sense and simplifies
things.

Note that this means that -a in concert with -c foo means that foo must
also be present on the remote node in the same path.  That, however, is a
use case that I don't particularly care about right now.  :)

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit cd7e52cc76878eed0f084f7b9a6cf7c792b716c6)

PG: call check_recovery_sources in remove_down_peer_info

If we transition out of peering due to affected
prior set, we won't trigger start_peering_interval
and check_recovery_sources won't get called.  This
will leave an entry in missing_loc_sources without
a matching missing set.  We always want to
check_recovery_sources with remove_down_peer_info.

Fixes: 4805
Backport: bobtail
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 81a6165c13c533e9c1c6684ab7beac09d52ca6b5)

PG: clear want_acting when we leave Primary

This is somewhat annoying actually.  Intuitively we want to
clear_primary_state when we leave primary, but when we restart
peering due to a change in prior set status, we can't afford
to forget most of our peering state.  want_acting, on the
other hand, should never persist across peering attempts.
In fact, in the future, want_acting should be pulled into
the Primary state structure.

Fixes: #3904
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>
(cherry picked from commit a5cade1fe7338602fb2bbfa867433d825f337c87)

mon: communicate the quorum_features properly when declaring victory.

Fixes #4747.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit fe68afe9d10bc5d49a05a8bafa644d57783447cf)

elector: trigger a mon reset whenever we bump the epoch

We need to call reset during every election cycle; luckily we
can call it more than once. bump_epoch is (by definition!) only called
once per cycle, and it's called at the beginning, so we put it there.

Fixes #4858.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 6ae9bbb5d03cb5695a4ebb7a3c20f729de1bd67a)

Conflicts:

src/mon/Elector.cc

Fix journal partition creation

With OSD sharing data and journal, the previous code created the
journal partiton from the end of the device. A uint32_t is
used in sgdisk to get the last sector, with large HD, uint32_t
is too small.
The journal partition will be created backwards from the
a sector in the midlle of the disk leaving space before
and after it. The data partition will use whichever of
these spaces is greater. The remaining will not be used.

This patch creates the journal partition from the start as a workaround.

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
(cherry picked from commit 56619ab91732bd2eacbef388311954f4e0230a30)

ceph.spec.in: put ceph-disk-* et al in correct sbindir

Signed-off-by: Sage Weil <sage@inktank.com>

debian: fix ceph.install

This got out of sync somewhere in cherry-picking all of these patches.

Signed-off-by: Sage Weil <sage@inktank.com>

Makefile.am: install ceph-* python scripts to /usr/bin directly

Install ceph-* scripts directly to $(prefix)$(sbindir) (which
normaly would be /usr/sbin) instead of moving it around after
installation in SPEC file or debian files.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 4d16f38f48e276497190c8bc03abc55c40e18eed)

ceph-disk:  CalledProcessError has no output keyword on 2.6

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
(cherry picked from commit a793853850ee135de14b9237f7023cadcdb8575c)

ceph-disk: fix some (local) variable names

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit c4eb7e6ddd593cd45ab8343da01355be7382723e)

ceph-disk: fix naming of local variable in is_mounted()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 3dd8b461219e64bb0f7a210dba5a9ab7c644f014)

ceph-disk: merge twice defined function is_mounted(dev)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit eaf31bf9f90ba9709a57a6870dbafa21142dae2c)

ceph-disk: define exception type

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 4c6d6442a89adc5b56e99cb4d2ed572f2ad192c9)

ceph-disk: fix Redefining name 'uuid' from outer scope

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 058eb923c5b7dab611901fdd1724ce2a7c180827)

ceph-disk: add missing space after comma

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 0080d1df7c7950e051840a543fc4bdabe6cc53e5)

ceph-disk list: say 'unknown cluster $UUID' when cluster is unknown

This makes it clearer that an old osd is in fact old.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e9b3f2e6e93426d631d4f8101fb431a523b0d88a)

ceph-disk: handle missing journal_uuid field gracefully

Only lower if we know it's not None.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 562e1716bd4b5372716e502a5e4dd872d381bfb7)

ceph-disk: implement 'list'

This is based on Sandon's initial patch, but much-modified.

Mounts ceph data volumes temporarily to see what is inside.  Attempts to
associated journals with osds.

Resolves: #3120
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 6a65b9131c444041d16b880c6a7f332776063a78)

ceph-disk: reimplement list_all_partitions

Use /dev/disk/by-id to list disks and their partitions.  This is more
accurate and correct than the previous (as-yet unused) implementation.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d3e49047ff405573aa41f45864cf315be23f5c50)

ceph-disk: reimplement is_partition

Previously we were assuming any device that ended in a digit was a
partition, but this is not at all correct (e.g., /dev/sr0, /dev/rbd1).
Instead, look in /dev/disk/by-id and see if there is a symlink that ends in
-partNN that links to our device.

There is probably still a better way...

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 20d594a889d62110ad03b761d8703f79f8eea6ad)

ceph-disk: conditionally remove mount path

umount removes it on success; only remove it here if it is still there.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 4362934a94c5a6f7521f06aa1255e111094e1d88)

ceph-disk: ignore udevadm settle return code

If we time out, just continue and let the next step fail.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 2ae297ccb812c00204ec0c4dc614821c506062b9)

ceph-disk: rename some local variabels in list_*partitions

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit b7d7e6894c550a7afa8dfb5bfa2bc54b5d331178)

ceph-disk: fix naming of a local variable in find_cluster_by_uuid

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 0b5fcfffe6d2f69bd4318cc93ef73195d948b9c5)

ceph-disk: rename some constants to upper case variable names

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 8a999ded088e688fd3f4a7c27127b7c06f0b2f66)

ceph-disk: add some more docstrings

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 86e55f5448c4b5b46b74d2d89b01d1e64b1ea826)

ceph-disk: print subprocess.CalledProcessError on error

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 543327b1f2a9efe8083bb196433c4bcf83883269)

ceph-disk: fix indention

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 2d26bcc07162a5176cdbc1748b829e3f39653697)

ceph-disk:  udevadm settle before partprobe

After changing the partition table, allow the udev event to be
processed before calling partprobe.  This helps prevent partprobe
from getting a resource busy error on some platforms.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
(cherry picked from commit 9eda8e5d5abf0743a2ad484806cfb2018243515f)

ceph-disk: re-add python 2.7 dependency comment

FIXME!

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 6d63752c8fde91cdab306d1ca689690b269fe977)

ceph-disk: cast output of subprocess.Popen() to str()

Cast output of subprocess.Popen() to str() to be able to use
str.split() and str.splitlines() without warnings from pylint.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 87691dc46edfba11c370592dbb533772190be4b2)

ceph-disk: fix adjust_symlink() replace 'journal' with 'target'

Replace 'journal' variable with 'target' since journal doesn't
exist in this function.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 3575feb7d2e5f4e35c5df193a1f8c9f08f88fcf4)

ceph-disk: fix adjust_symlink() replace 'canonical' with 'path'

Replace 'canonical' variable with 'path' since canonical doesn't
exist in this function.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit a27cb85b21610b4a9f8eb24f8f6cd04902349728)

ceph-disk: there is no os.path.lstat use os.lstat

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 6d3247b5c02c39a66666a5833106dbc23044e436)

ceph-disk: remove unused variable key from prepare_journal_dev()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 97b4f8d66bef2328fa53f9e508eb38f8b8d49632)

ceph-disk: fix except to catch OSError

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 9daf6cfce2d57509d896eae28bb97146a682a306)

ceph-disk: add missing space after >> operator

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 0ada43f79d2b1f9f84367e558c6d1a3e90e5c9b2)

fix: Redefining name 'uuid' from outer scope (line 14)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit d3c60dc8cad1db1d5df1c740bc805aaf9ba606ba)

ceph-disk: remove unused variables from list_partitions()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 6a8120d4b0c4cfa851d473532eb2366534f8653d)

ceph-disk: fix /dev/dm-[0-9] handling list_all_partitions()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 3af7a1ac5bf24bf832d7180002281d6b5853e85a)

ceph-disk: rename local variable shadowing builtin

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 9bcf5b64f45ab6c4bdedf820ed111319b2dbd778)

ceph-disk: remove twice defined identical function unmount

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 4adf088107586be7b356d1e963570cdab232809e)

ceph-disk: remove twice defined function mount

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit ea26ea0d81a23aa76076ad5441c3b1aadfba3b44)

ceph-disk: remove double defined function get_conf

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit c57daa3c6e03e8974e133d3a2d9bc3d6f06f4faf)

ceph-disk: rename local variable shadowing builtin

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 57dde5c8b18ff4ccd53a30bb94119c0ffceb52b7)

ceph-disk: install and package

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit c2602d749023b24ac22d8cfce6e04889078f14d8)

Conflicts:

debian/ceph.install

ceph-disk: simplify command dispatch

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit f287c6f90af0dfdd41358846b069aa3c54b600b3)

ceph-disk: consolidate exceptions

Use a single exception type, and catch it at the top level.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit a019753bd3897ee0f5d9c3563c653c5457d5e67d)

ceph-disk: consolidate ceph-disk-* into a single binary

 ceph-disk prepare ...
 ceph-disk activate ...
 ceph-disk ...

This let's us share code (we were already duplicating a bunch of stuff!)
and to add new commands easily.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 20e4ba5cebb7b4989d1745932f7a412c28504ee1)

ceph-disk-prepare: 'mkfs -t' instead of 'mkfs --type='

Older mkfs (el6) doesn't like --type=.

Fixes: #4495
Reported-by: Alexandre Maragone <alexandre.maragone@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d47759429a6e2fdd392265e90b67046302c97644)

ceph-disk-activate: identify cluster .conf by fsid

Determine what cluster the disk belongs to by checking the fsid defined
in /etc/ceph/*.conf.  Previously we hard-coded 'ceph'.

Note that this has the nice side-effect that if we have a disk with a
bad/different fsid, we now fail to activate it.  Previously, we would
mount and start ceph-osd, but the daemon would fail to authenticate
because it was part of the wrong cluster.

Fixes: #3253
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 80af5fb887f30792c342ac16da9ed95d7e89e491)

ceph-disk-activate: abort if target position is already mounted

If the target position is already a mount point, fail to move our mount
over to it.  This usually indicates that a different osd.N from a
different cluster instances is in that position.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 7370b5564606474f11b9ac5afb7cc60e0ac36ed1)

Revert "ceph-disk-activate: rely on default/configured keyring path"

This reverts commit 936b8f20af1d390976097c427b6e92da4b39b218.

This is necessary because we mount the osd in a temporary location.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit b6102c0945a0d68b7c5ff7f2254a1a994788ca0f)

Revert "ceph-disk-activate: don't override default or configured osd journal path"

This reverts commit 813e9fe2b4291a1c1922ef78f031daa9b78fe53b.

We run --mkfs with the osd disk mounted in a temporary location, so it is
necessary to explicitly pass in these paths.

If we want to support journals in a different location, we need to make
ceph-disk-prepare update the journal symlink accordingly.. not control it via
the config option.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 3e628eee770508e750f64ea50179bbce52e7b8e0)

ceph-disk-activate: rely on default/configured keyring path

No reason to override the default or configured value here.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 936b8f20af1d390976097c427b6e92da4b39b218)

ceph-disk-activate: don't override default or configured osd journal path

There is no reason not to rely on the default or obey any configured
value here.

Fixes: #4031
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 813e9fe2b4291a1c1922ef78f031daa9b78fe53b)

ceph-disk-prepare: move in-use checks to the top, before zap

Move the in-use checks to the very top, before we (say) zap!

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 32407c994f309cd788bf13fe9af27e17a422309a)

ceph-disk-prepare: verify device is not in use by device-mapper

Be nice and tell the user which devices/mappings are consuming the device,
too.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit a6196de9e2f3ca9d67691f79d44e9a9f669443e9)

ceph-disk-prepare: verify device is not mounted before using

Make sure the data and/or journal device(s) are not in use (mounted)
before using them.  Make room for additional "in-use" checks in the future.

Closes: #3256
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 3bd0ac0ab011c4cdf0121f0d9732938d085fb8bf)

ceph-disk-prepare: clean up stupid check for a digit

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit f03f62697f170d42b4b62c53d2860ff2f24a2d73)

ceph-disk-prepare: use os.path.realpath()

My janky symlink resolution is broken in various ways.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 59505546e52a175435881b431bd349d532ae627e)

ceph.spec.in: add new Requires from ceph-disk-prepare

Added new Requires from ceph-disk-prepare: cryptsetup, gptfdisk,
parted and util-linux.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 5c3f5c5b69a9edc99138d4f1ddb016689303dc28)

Conflicts:

ceph.spec.in

debian: require cryptsetup-bin

This is needed for ceph-disk-prepare's dmcrypt support.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit cfcaceac44d6b7b7c55e81d0bfb05f4893f3b1d0)

Conflicts:

debian/control

Fix: use absolute path with udev

Avoids the following: udevd[61613]: failed to execute '/lib/udev/bash'
'bash -c 'while [ ! -e /dev/mapper/....

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
(cherry picked from commit 785b25f53dc7f8035eeba2aae8a196e3b102d930)

ceph-disk-prepare: -f for mkfs.xfs only

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit fecc3c3abf1176f4c7938e161559ea2db59f1cff)

debian: fix start of ceph-all

Tolerate failure, and do ceph-all, not ceph-osd-all.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit aff0bb6fdc8ca358f7ac1e941bb9cfecbefb4bb6)

ceph-disk-prepare: always force mkfs.xfs

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
(cherry picked from commit d950d83250db3a179c4b629fd32cd7bc8149997e)

udev: trigger on dmcrypted osd partitions

Automatically map encrypted journal partitions.

For encrypted OSD partitions, map them, wait for the mapped device to
appear, and then ceph-disk-activate.

This is much simpler than doing the work in ceph-disk-activate.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e090a92a20f4161f473d16bc966f7d6aacac75ee)

ceph-disk-prepare: add initial support for dm-crypt

Keep keys in /etc/ceph/dmcrypt-keys.

Identify partition instances by the partition UUID.  Identify encrypted
partitions by a parallel set of type UUIDs.

Signed-off-by: Alexandre Marangone <alexandre.maragone@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit c6ac0ddf91915ba2aeae46d21367f017e18e82cd)

ceph-disk-activate: pull mount options from ceph.conf

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
(cherry picked from commit e7040f55f01db3de7d5cebfc79de50c8b6ad5d45)

ceph-disk-activate: use full paths for everything

We are run from udev, which doesn't get a decent PATH.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit b1c0fccba42dd184a2891ee873c0d6d8f8c79d14)

ceph-disk-prepare: do partprobe after setting final partition type

This is necessary to kick udev into processing the updated partition and
running its rules.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 95835de9f80712eb26652ac6b66ba9c5eeb093d6)

debian: start/stop ceph-all event on install/uninstall

This helps us avoid the confusing situation with upstart where an individual
daemon job is running (like ceph-osd id=2) but the container jobs ceph-osd-all
and ceph-all are not.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit b7b9af5c0d531dcee7ce9b10043a29b0a1b31f47)

ceph-disk-activate: catch daemon start errors

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 690ae05309db118fb3fe390a48df33355fd068a0)

udev: trigger ceph-disk-activate directly from udev

There is no need to depend on upstart for this.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 5bd85ee5aa31bfd1f4f0e434f08c2a19414358ef)

Conflicts:

ceph.spec.in

ceph-disk-activate: auto detect init system

Look for an option 'init' in ceph.conf.  Otherwise, check if we're ubuntu.
If so, use upstart.  Otherwise, use sysvinit.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d1904b2a848af3c02d2065ac2a42abe0e2699d0f)

ceph-disk-activate: specify full path for blkid, initctl, service

/sbin apparently isn't in the path when udev runs us.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit f06b45e66315310abb0720e021da377186455048)

upstart: ceph-hotplug -> ceph-osd-activate

This is a more meaningful name.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e011ad128e7f302cb6955d9a7171ac0ec8890ddf)

upstart/ceph-hotplug: tell activate to start via upstart

This will mark the OSD data dir as upstart-managed.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 792e45c63dd7a9622fddd6e15ee4c075f995ea56)

ceph-disk-prepare: refactor to support DIR, DISK, or PARTITION for data or journal

Lots of code reorganization collapsed into a single commit here.

- detect whether the user gave us a directory, disk, or partition, and Do The
Right Thing
- allow them to force that the input was of type X, for the careful/paranoid.
- make --zap-disk an option -- no longer the default

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit b2ff6e8c9d96dee2c063b126de7030a5c2ae0d02)

ceph-disk-activate: detect whether PATH is mount or dir

remove in-the-way symlinks in /var/lib/ceph/osd

This is simpler.  Just detect what the path is and Do The Right Thing.

Closes #3341 (which wanted to make --mount the default)

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 191d5f7535f8d96d493e1b35b43a421c67c168ea)

ceph-disk-activate: add --mark-init INITSYSTEM option

Do not assume we will manage via upstart; let that be passed down via the
command line.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit fd4a921085a861e4aa428376219bb39055731f2b)

ceph-disk-activate: factor mounting out of activate

The activate stuff is generic for any OSD, regardless of whether we want
to mount it or not.  Pull that part out.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 07655288281c9c6f691f87352dc26b7c11ae07e8)

debian: put ceph-mds upstart conf in ceph-mds package

Fixes: #3157
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 23ad3a46a0099e263f43e0f0c1df1d21cfe58b3f)

debian: include /var/lib/ceph/bootstrap-mds in package

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e80675a0f333c04452d4822fd0eb3c6e92eda3df)

ceph-create-keys: create mds bootstrap key

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 809143f16c70483ba5bb429dea812d31b67f2b49)

upstart/ceph-hotplug: drop -- in ceph-disk-activate args

We would like to transition to

 ceph-disk-activate --mount DEV

and away from a generic multi-definition PATH argument.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 4698b6a1035dee8509ce2d4dab7b34a16b78f7cd)

init-ceph: iterate/locate local sysvinit-tagged directories

Search /var/lib/ceph/$type/ceph-$id and start/stop those daemons if
present and tagged with the sysvinit file.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit c8f528a4070dd3aa0b25c435c6234032aee39b21)

init-ceph: consider sysvinit-tagged dirs as local

If there is a 'sysvinit' file in the daemon directory in the default
location (/var/lib/ceph/$type/ceph-$id), consider it sysvinit-managed.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit b8aa4769a62e0d88174678cbefd89d9ee2baceea)

ceph-disk-prepare: align mkfs, mount config options with mkcephfs

'osd mkfs ...', not 'osd fs mkfs ...'.  Sigh.  Support both.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit af2372ca4a702da70275edd1b1357fcff51e6ae2)

init-ceph: fix (and simplify) pushing ceph.conf to remote unique name

The old code would only do the push once per remote node (due to the
list in $pushed_to) but would reset $unique on each attempt.  This would
break if a remote host was processed twice.

Fix by just skipping the $pushed_to optimization entirely.

Fixes: #4794
Reported-by: Andreas Friedrich <andreas.friedrich@ts.fujitsu.com>
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit ccbc4dbc6edf09626459ca52a53a72682f541e86)

Merge pull request #238 from ceph/wip-bobtail-rbd-backports-req-order

Reviewed-by: Sage Weil <sage.weil@inktank.com>

rbd: only set STRIPINGV2 feature when needed

Only set the STRIPINGV2 feature if the striping parameters are non-default.
Specifically, fix the case where the passed-in size and count are == 0.

Fixes: #4710
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 5926ffa576e9477324ca00eaec731a224195e7db)

Conflicts:

src/rbd.cc

LibrbdWriteback:  complete writes strictly in order

RADOS returns writes to the same object in the same order. The
ObjectCacher relies on this assumption to make sure previous writes
are complete and maintain consistency. Reads, however, may be
reordered with respect to each other. When writing to an rbd clone,
reads to the parent must be performed when the object does not exist
in the child yet. These reads may be reordered, resulting in the
original writes being reordered. This breaks the assmuptions of the
ObjectCacher, causing an assert to fail.

To fix this, keep a per-object queue of outstanding writes to an
object in the LibrbdWriteback handler, and finish them in the order in
which they were sent.

Fixes: #4531
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 06d05e5ed7e09fa873cc05021d16f21317a1f8ef)

LibrbdWriteback: removed unused and undefined method

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 909dfb7d183f54f7583a70c05550bec07856d4e4)

LibrbdWriteback: use a tid_t for tids

An int could be much smaller, leading to overflow and bad behavior.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 9d19961539b2d50d0c9edee1e3d5ac6912a37f24)

WritebackHandler: make read return nothing

The tid returned by reads is ignored, and would make tracking writes
internally more difficult by using the same id-space as them. Make read
void and update all implementations.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 870f9cd421ca7b0094f9f89e13b1898a8302c494)

ObjectCacher: deduplicate final part of flush_set()

Both versions of flush_set() did the same thing. Move it into a
helper called from both.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit f5b81d8d167d1aa7f82a5776bbb1f319063ab809)