osd_types.h: pass eversion_t by reference to operator<<

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SimpleLock.h: remove unused private function clear_more()

Remove unused private function clear_more(), it's replaced by
try_clear_more().

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SimpleLock.h: remove twice included osd_types.h

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

linux_fiemap.h: remove twice included int_types.h

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Dumper::dump_entries(): remove not needed variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_bucket.cc: return error if update_containers_stats() fails

In case need_stats is set on rgw_read_user_buckets() and the
update_containers_stats() call fails with !-ENOENT, not only
print out a message but also return the error to the function
caller.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_common.cc: reduce scope of 'fpos' variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_admin.cc: remove unused string variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

PGBackend.cc: remove unused to_remove variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

KeyValueStore.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_op.cc: remove unused map variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_main.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_main.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ObjectStore.h: pass const string parameter by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

CDentry.cc: fix bool comparison using relational operator

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

buffer.cc: catch exception by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SimpleLock.h: fix bool comparison using relational operator

[src/mds/SimpleLock.h:287]: (warning) Comparison of a boolean value
 using relational operator (<, >, <= or >=).

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

hitset.cc: fix format string to unsigned int

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

OSDMap.cc: prefer prefix ++operator for non-trivial iterator

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

OSDMonitor.cc: prefer prefix ++operator for non-trivial iterator

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

KeyValueStore: rename s/logger/perf_logger/

[src/os/KeyValueStore.h:368] -> [src/os/ObjectStore.h:100]: (warning) The class
 'KeyValueStore' defines member variable with name 'logger' also defined in its
 parent class 'ObjectStore'.

KeyValueStore logger is of type PerfCounters while ObjectStore logger is Logger.
Change the name in KeyValueStore to more obvious name perf_logger.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

crush/mapper.c: fix printf format for unsigned variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #1629 from ceph/wip-die-mkcephfs

remove mkcephfs (merge post-firefly?)

Reviewed-by: Alfredo Deza <alfredo.deza@inktank.com>

Merge pull request #1741 from ceph/wip-early-reply

Wip early reply

mds: avoid adding replicas of target dentry to rename witnesses

When the rename target dentry is NULL, we can use MDentryLink messages
instead of slave requests to update its replicas.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

mds: allow early reply when request's witness list is empty

Early reply should be Ok when there were slave requests, but all
of them were for acquiring locks.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

Merge pull request #1121 from ceph/wip-no-anchor

mds: remove anchor table (merge post-firefly only)

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1670 from yuyuyu101/wip-test-clone

Add clone test on store_test

Reviewed-by: Sage Weil <sage@inktank.com>

mds: include authority of the source inode in rename witnesses

rename updates source inode's ctime

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

mds: remove discover ino

Anchor table was the main user of MDCache::discover_ino(), it has
been removed. MDCache::discover_path() can replace discover_ino()
in remaining places. This patch removes discover ino related code.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

mds: remove anchor table

use backtrace instead of anchors to find/open remote inodes

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

Copy range using fiemap not entire length

Under rbd usage, if a volume has tens of thousands of objects and each 4M
object only has several KB(run fio on this volume or other cases), this volume
will be very low performance during a long time after create snapshot on
this volume. The OSD will be busy with large bandwidth read/write although
the object actually has few bytes needed to be copied.

This commit try to use fiemap if backend fs support, it can skip unnecessary
range to write. It also can be beneficial to space effective, because the copied
object will be regard as snapshot object which is access infrequently.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

rbd-fuse: fix unlink

The path contains a leading / that needs to be ignored.

Fixes: #8197
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #1701 from ceph/wip-libkrbd

libkrbd convenience library

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #1648 from ceph/wip-client-sleep

Wip client sleep

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1709 from dachary/wip-brag

brag : useability changes

Reviewed-by: Babu Shanmugam <anbu@enovance.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge branch 'wip-rbd-invalidate'

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1737 from steveftaylor/add_rbd_fuse_image_restriction

Added a new command line parameter (-i or --image=) that allows rbd-fuse...

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1699 from chrisglass/python-api-cleanup

Simple mechanical cleanups

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

librbd: add an interface to invalidate cached data

This is useful for qemu to guarantee live migration with caching is
safe, by invalidating the cache on the destination before starting it.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

librbd: check return code and error out if invalidate_cache fails

This will only happen when shrinking or rolling back an image is done
while other I/O is in flight to the same ImageCtx. This is unsafe, so
return an error before performing the resize or rollback.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Changed the -i parameter to -r in order to avoid a conflict with a generic flag interpreted by the common code.

Merge pull request #1652 from ceph/wip-5170-firefly

Wip 5170 firefly

Merge pull request #1734 from ceph/wip-8202

rgw: fix url escaping

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1736 from ceph/wip-7500-wusui

Fix s3 tests in the rgw workunit.

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

Fix s3 tests in the rgw workunit.

Make it possible to set RGW_PORT with ENV variable.

Fixes: 7500
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Added a new command line parameter (-i or --image=) that allows rbd-fuse to specify a single image to be made available within
the mount directory. The purpose of this is to allow a single RBD to be "mounted" in userspace without opening (and locking)
the other RBDs in the pool.

This is accomplished by performing a case-sensitive string compare in enumerate_images() when an image name is
specified on the command line. If no image name is specified, all images appear in the mount directory. If a non-existent
image name is specified, the mount directory is empty.

Signed-off-by: Stephen F Taylor <steveftaylor@gmail.com>

doc/release-notes: v0.67.8 notes

Signed-off-by: Sage Weil <sage@inktank.com>

rgw: fix url escaping

Fixes: #8202
This fixes the radosgw side of issue #8202. Needed to cast value
to unsigned char, otherwise it'd get padded.

Backport: dumpling

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

client: cleanup unsafe requests if MDS session is reset

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

client: wake up cap waiters if MDS session is reset

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

client: drop dirty/flushing caps if auth MDS' session is reset

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

client: add asok command to kick sessions that were remote reset

Fixes: #8021
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

Merge pull request #1729 from ceph/wip-7966

readlink result in resapwn

Reviewed-by: Yan, Zheng <zheng.z.yan@intel.com>

mds: terminate readlink result in resapwn

readlink(2) does not null terminate the buffer; we need to do that.

Fixes: #7966
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1727 from ceph/wip-8193

ceph_test_rados_api_tier: increase HitSetTrim timeouts

Merge pull request #1725 from FlorentCoppint/master

Skipping '_netdev' Debian fstab option

Reviewed-by: Sage Weil <sage@inktank.com>

ceph_test_rados_api_tier: increase HitSetTrim timeouts

...so that they pass when they get unlucky with thrashing.

This will vastly decrease the probability of failure, but failure will
always be possible when a timeout is in place.

Fixes: #8193
Signed-off-by: Sage Weil <sage@inktank.com>

Skipping '_netdev' Debian fstab option

Signed-off-by: Florent Bautista <florent@coppint.com>

Merge pull request #1717 from dachary/wip-auid

mon: add ceph osd pool set <pool> auid

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #1724 from ceph/wip-uselocalgithubforqemu-wusui

Use new git mirror for qemu-iotests

Use new git mirror for qemu-iotests

Fixes: 8191
Signed-off-by: Warren Usui <warren.usui@inktank.com>

brag : implement --verbose on client

Signed-off-by: Loic Dachary <loic@dachary.org>

brag : document the zero argument behavior

Signed-off-by: Loic Dachary <loic@dachary.org>

brag : meaningfull error messages

To help figure out problems, include the error message in the output
when a submission fails.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge remote-tracking branch 'gh/firefly'

Merge pull request #1720 from jdurgin/wip-list-children-test

test_rbd.py: ignore children in cache pools

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1719 from ceph/wip-8168

Wip 8168

Reviewed-by: Sage Weil <sage@inktank.com>

ReplicatedPG::do_osd_ops: consider head whiteout in list-snaps

Signed-off-by: Samuel Just <sam.just@inktank.com>

ReplicatedPG::do_op: don't return ENOENT for whiteout on snapdir read

Signed-off-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1718 from ceph/wip-7882-wusui

Support latest qemu iotest code

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

test_rbd.py: ignore children in cache pools

This is necessary until http://tracker.ceph.com/issues/8187 is fixed.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

mon: add ceph osd pool set <pool> auid

When a pool is created with ceph osd pool create, the auid is not
inferred from the session auid and is set to zero. Add the

  ceph osd pool set <pool> auid <int>

command to set it after it is created, and the matching get:

  ceph osd pool get <pool> auid

Signed-off-by: Loic Dachary <loic@dachary.org>

Support latest qemu iotest code

Modified qemu-iotests workunit script to check for versions
that use the latest qemu (currently only Trusty).  Limit the
tests to those that are applicable to rbd.

Fixes: 7882
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #1714 from ceph/wip-fs-client

two small fixes for client

Reviewed-by: Sage Weil <sage@inktank.com>

rbd: deprecate --no-settle option

Waiting for udev has been the default for a while now, and, after
switching to libkrbd, is no longer an option.  (libkrbd waits for
specific devices to show up, the old behaviour was to wait for
everything udev.)

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

doc: 'rbd showmapped' doesn't need privileges

No need to run 'rbd showmapped' with sudo.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

doc: do not mention modprobe in rbd docs

rbd binary will load rbd.ko itself, with appropriate options.  Loading
it by hand with default options is undesirable.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

rbd: switch to libkrbd for 'rbd {map,showmapped,unmap}' operations

Thanks to libkrbd, 'rbd map' now outputs the device node it mapped to
to stdout:

    $ sudo rbd map foo
    /dev/rbd0

This will allow us to get rid of a lot of ad-hoc poll/sleep code in our
qa scripts.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

mount.ceph: switch to module_load()

Implement modprobe() in terms of module_load() from common/module.h

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

rbd: add libkrbd convenience library

Add libkrbd libtool convenience library to provide an interface for
mapping and unmapping rbd images programmatically.  This will be used
by the rbd binary itself and the librbd_fsx testing tool.

libkrbd takes care of the kernel module stuff (common/module.h) and
makes use of libudev to be able to properly wait for block device
creation and deletion and tell which block device got assigned by the
kernel to the newly created mapping.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

client: check cap ID when handling cap export message

handle following sequence of events:
- mds0 exports an inode to mds1. client receives the cap import
  message from mds1. caps from mds0 are removed while handling
  the cap import message.
- mds1 exports an inode to mds0. client receives the cap export
  message from mds1. handle_cap_export() adds placeholder caps
  for mds0
- client receives the first cap export message (for exporting
  inode from mds0 to mds1)

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

client: avoid releasing caps that are being used

To avoid releasing caps that are being used, encode_inode_release()
should send implemented caps to MDS.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

Merge pull request #1713 from ceph/wip-7439

Wip 7439

Reviewed-by: Sage Weil <sage@inktank.com>

ReplicatedPG: handle ec pools in mark_all_unfound_lost

Signed-off-by: Samuel Just <sam.just@inktank.com>

0.80-rc1

ReplicatedPG: enable mark_unfound_lost delete for ec pools

revert is tricky to implement at this time for ec pools, so
we'll instead just implement delete for ec pools.

Fixes: #7439
Signed-off-by: Samuel Just <sam.just@inktank.com>

qa/workunits/rbd/copy.sh: skip some tests when tiering is enabled

The rados ls bit doesn't work.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

qa/workunits/rbd/copy.sh: fix test

I broke this in commit 9d64ac66082bd108ec3c2a74e2e77475b5564eae.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #1691 from ceph/wip-8139

osd_types: pg_t: allow is_split to handle checks for splits prior to the most recent

Reviewed-by: Samuel Just <sam.just@inktank.com>

ECBackend: use std::swap for boost::optional

Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Samuel Just <sam.just@inktank.com>

common: add module_{load,has_parameter}()

Add two kernel module helpers: module_{module,has_parameter}().  They
are going to live in common/module.[ch].

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

Makefile: build common/secret.c with libtool

Turn common/secret.c into a libtool convenience library, libsecret.la.
Currently it is build directly, twice: for mount.ceph and rbd binaries.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

configure: check for blkid/blkid.h header

The check for the presence of blkid/blkid.h was missing.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

rbd: use stringify() in options parsing routines

Use stringify() in map_option_{uuid,ip,int}_cb() instead of essentially
open-coding it.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

stringify: use ostringstream instead of stringstream

Use ostringstream, as we don't need both input and output of the
stringstream in stringify().

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

Merge pull request #1710 from ceph/wip-coverity

a couple coverity fixes

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

Merge pull request #1711 from ceph/wip-coverity-respawn

mds: make strncpy in ::respawn safer

Reviewed-by: Sage Weil <sage@inktank.com>

mds: make strncpy in ::respawn safer

Previous code assumed null terminated argv[0]
was not longer than PATH_MAX and the resulting
strncpy was not strictly safe.

Modify the bounds to ensure that copy will not
result in an unterminated string if argv[0]
is oversized.

Signed-off-by: John Spray <john.spray@inktank.com>