StoreTest: Clean up warning

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

MemStore: Return -ENODATA when collection_getattr hit nonexist attr

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

KeyValueStore: Fix collection_getattrs problem when zero-length attr

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

KeyValueStore: Return -ENODATA if collection_getattr is nonexist

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

StoreTest: Add collection_getattr(s) tests

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

KeyValueStore: Remove assert for collection_getattr method

Merge 22b6c1a5317504e87042ffeff2b2d0d0b71cac5e introduce
5dc990c1ca821620993417ef4acc51a2b0a34606 commit which will lookup "remove"
attr. It shouldn't exist for most of cases but KeyValueStore will crash
if attr isn't exist. It seemed that this assert isn't correct.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

vstart.sh: complain less about fullish mon disks

Lots of people develop on file systems  > 70% full.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/next'

mds: don't overwrite reply's snapbl

set_trace_dist() updates reply's snapbl, don't overwrite it. For MKSNAP
request, just need to set mdr->tracei, set_trace_dist() will set reply's
snapbl.

Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #2772 from ceph/wip-init-radosgw

init-radosgw.sysv: set ulimit -n before starting daemon

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #2920 from ceph/wip-10095

crush: fix detach_bucket

Reviewed-by: Samuel Just <sjust@redhat.com>

crush/CrushWrapper: fix detach_bucket

In commit 9850227d2f0ca2f692a154de2c14a0a08e751f08 we changed the call that
changed the weight of all instances of item to one that explicitly
changes it in the parent bucket, but parent_id may not be valid at the
call site.  Move this into the conditional block to fix.

Fixes: #10095
Signed-off-by: Sage Weil <sage@redhat.com>

Makefile: include 'ceph' in base target

This makes 'make base' enough to vstart.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2906 from ceph/wip-9835

osd: fix misdirected ec ops stuck in optracker (9835)

Reviewed-by: Samuel Just <sam.just@inktank.com>

msg/Makefile: add new header to tarball

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2909 from yuyuyu101/wip-10079

AsyncMessenger: Add select support and tests for async drivers

Merge pull request #2912 from dachary/wip-warning

tests: fix compilation warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #2911 from keichwa/wip-doc-rados

Wip doc rados

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: fix compilation warning

Signed-off-by: Loic Dachary <ldachary@redhat.com>

doc: remove superfluous white spaces (for testing)

Signed-off-by: Karl Eichwalder <ke@suse.de>

doc: Fix 2 syntax errors.

Signed-off-by: Karl Eichwalder <ke@suse.de>

AsyncMessenger: Fix select driver for add_event

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Add tests for async drivers

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Support select for other OS such as Windows

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

osd/OSD: use OSDMap helper to determine if we are correct op target

Use the new helper.  This fixes our behavior for EC pools where targetting
a different shard is not correct, while for replicated pools it may be. In
the EC case, it leaves the op hanging indefinitely in the OpTracker because
the pgid exists but as a different shard.

Fixes: #9835
Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: add osd_is_valid_op_target()

Helper to check whether an osd is a given op target for a pg.  This
assumes that for EC we always send ops to the primary, while for
replicated we may target any replica.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_objectstore_tool: Fixes to make import work again

The is_pg() call is now true even for pgs pending removal, fix broken
    finish_remove_pgs() by removing is_pg() check.
Need to add create_collection() to the initial transaction on import

Fixes: #10090
Signed-off-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2903 from ceph/wip-crush-tree

crush: fix tree bucket functions

Reviewed-by: Sage Weil <sage@redhat.com>

vstart.sh: warn less

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2892 from leseb/doc-juno-client-socket

doc: enable RBD cache and socket on OpenStack deployments

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #2902 from dachary/wip-10083-noisy-osd

qa: handle CEPH_CLI_TEST_DUP_COMMAND on ceph osd create

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'origin/next'

qa: handle CEPH_CLI_TEST_DUP_COMMAND on ceph osd create

If CEPH_CLI_TEST_DUP_COMMAND is set when ceph osd create is called, it
will create two osd. They must be cleaned up afterwards instead of
assuming only one is going to be created.

http://tracker.ceph.com/issues/10083 Fixes: #10083

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #2124 from ceph/wip-objectstore

simplify ObjectStore interface

sage-2014-11-11_08:26:01-rados-wip-sage-testing-distro-basic-multi

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #2722 from ceph/wip-9598

msgr: make mark_down not block on fast dispatch

sage-2014-11-11_08:26:01-rados-wip-sage-testing-distro-basic-multi

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #2901 from ktdreyer/doc-spelling-grammar

doc: spelling & grammar fixes

Reviewed-by: John Wilkins <jowilkin@redhat.com>

doc: correct grammar in RGW SSL cert installation

The SSL certificate generation instruction was missing a preposition.
Add it.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc: correct spelling of "fqdn"

The "fqdn" (fully-qualified domain name) was misspelled in a couple
places as fgdn". Use the correct spelling.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #2899 from jdurgin/wip-import-export-next

qa: allow small allocation diffs for exported rbds

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: enable RBD cache and socket on OpenStack deployments

Enabling the RBD cache improves sequential IOs and the socket helps a
lot while troubleshooting. These 2 items are considered as best
practice for OpenStack deployments with Ceph.

Signed-off-by: Sébastien Han <sebastien.han@enovance.com>

qa: allow small allocation diffs for exported rbds

The local filesytem may behave slightly differently. This isn't
foolproof, but seems to be reliable enough on rhel7 rootfs, where
exact comparison was failing.

Fixes: #10002
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #2898 from ceph/wip-rbd-python-tests

librbd: Python unit tests now use unique pools and images

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #2820 from ceph/wip-9854

osdc: Constrain max number of in-flight read requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #2890 from zhouyuan/fix_mailmap

Fix Yuan's mailmap

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/next'

Fix Yuan's mailmap

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

doc/release-notes: v0.88

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2897 from ceph/wip-da-revert-10b68b

Revert "osd: detect (some) misordered ondisk tmaps"

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

crush: fix tree bucket functions

There are incorrect nodes' weight in tree bucket when construct tree
bucket. The tree bucket don't store item id in items array, so the tree
bucket will not work correctly. The patch fix above bugs and add a
simple test for tree bucket.

Signed-off-by: Rongze Zhu <zrzhit@gmail.com>

crush/builder: replace printf with an empty dprintk macro

This mirrors mapper.c.

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "osd: detect (some) misordered ondisk tmaps"

This reverts commit 10b68b2e1a1479c478843fab8232537675053c9b.

The check for 'nextkey < last_disk_key' makes not much sense since
last_disk_key is an empty string and not set before. Comparing a
decoded string to be less than an empty string will be never true.

Since this if() isn't part of a loop last_disk_key is only set
once and there is no other consumer: revert this dead code.

Conflicts:
src/osd/ReplicatedPG.cc

0.88

Merge pull request #2562 from ceph/wip-5595

Wip 5595

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/rados/configuration fix 'ods mkfs options' default

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2466 from nereocystis/osd-filesystem-options

doc: Add file system osd config settings

Merge pull request #2883 from ceph/wip-da-SCA-20141028

Fixes for coverity and SCA issues

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2889 from xuechendi/move_buffers_to_BufferTransaction

KeyValueStore: move buffers from strip_header to BufferTransaction

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #2739 from majianpeng/fix2

buffer: Judge bufferlist whether need  rebuild

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2894 from ceph/wip-scrub-coverity

Wip scrub coverity

AuthSessionHandler.h: init protocol in constructor

Init protocol in AuthSessionHandler constructor with
with CEPH_AUTH_UNKNOWN.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/librbd/librbd.cc: fix potential null pointer deref

Fix potential deref after null check. Move check for max_snaps in
rbd_snap_list up to fail early before call any functions.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: remove redundant check

Fix for:

[src/osd/ReplicatedPG.cc:2171]: (style) Redundant checking of STL
 container element existence before removing it.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/librados/c_read_operations.cc: fix invalid checks

Wrong variable for return of rados_getxattrs_next() and
rados_omap_get_next() used (len vs. val_len).

Fix for:

CID 1219464 (#1 of 1): Logically dead code (DEADCODE)
 dead_error_line: Execution cannot reach this expression
 key == NULL inside statement if (len == 0UL && key == NU...

CID 1219465 (#1 of 1): Logically dead code (DEADCODE)
 dead_error_line: Execution cannot reach this expression
 key == NULL inside statement if (len == 0UL && key == NU...

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: remove dead code

Fix for:

CID 1128391 (#1 of 1): Logically dead code (DEADCODE)
 dead_error_line: Execution cannot reach this statement goto done_err;

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados_sync.cc: fix xattr_diff() for the only_in_b checks

In the checks to build only_in_b up the wrong const_iterator x is
build up. it should compare rhs->xattrs with xattrs entries and
not twice rhs->xattrs.

Fix for:

CID 716957 (#1 of 1): Invalid iterator comparison (MISMATCHED_ITERATOR)
 mismatched_comparison: Comparing x from rhs->xattrs to this->xattrs.end()
 from this->xattrs.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

bench_log.cc: catch ceph::FailedAssertion exception

Fix for:

CID 717177 (#2-1 of 3): Uncaught exception (UNCAUGHT_EXCEPT)
 root_function: In function main(int, char const **) an exception of
 type ceph::FailedAssertion is thrown and never caught.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PGLog.cc: prefer empty() over size() for emptiness check

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd: Python unit tests now use unique pools and images

RBD python unit tests no longer utilize the 'rbd' pool for
test cases.  Instead, a new temporary pool is created and
deleted. Additionally, each unit test now uses a unique
image name to reduce the possibility of test case failures
affecting subsequent tests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #2873 from ceph/wip-rbd-test-fixtures

librbd: Refactor librbd unit tests to use test fixtures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

bufferlist: Don't call rebuild(), c_strt() call it if bufferlist not contiguous.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: Constrain max number of in-flight read requests

Constrain the number of in-flight RADOS read requests to the
cache size.  This reduces the chance of the cache memory
ballooning during certain scenarios like copy-up which can
invoke many concurrent read requests.

Fixes: #9854
Backport: giant, firefly, dumpling
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #2893 from ceph/wip-10045

common/Readahead: use correct lock when waiting on the pending ops

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2878 from ceph/wip-10025

#10025 Fix 'journal import' (aka undump)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

common/Readahead: use correct lock when waiting on the pending ops

Readahead was using the incorrect lock with the pending condition,
resulting in a failed assertion.  It now uses the lock associated
with pending ops.

Fixes: #10045
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mds: remove dead code from inode_t::compare()

CID 1251354

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

mds: CInode: catch exception by reference, not value

CID 1251353

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #2380 from osynge/wip-suse-systemd-integration

systemd support in ceph-disk activate

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2446 from xylv/wip-9359

rgw: Export user stats in get-user-info Adminops API

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #2725 from CzBiX/rgw-modified-since

rgw: correct "If-Modified-Since" handle.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #2782 from boydc2014/master

rgw: fix can not disable max_size quota

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #2624 from ceph/wip-8911

rgw: swift GET / HEAD object returns X-Timestamp field

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2851 from ceph/wip-9986

objecter: fix map skipping

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2667 from yuandong1222/avoid-unused-pg_to_acting_osds-call

Avoid unused pg_to_acting_osds call in OSDMap::get_primary_shard

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #2474 from dachary/wip-9420-erasure-code-non-regression

erasure-code: store and compare encoded contents

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #2456 from ceph/wip-9031-9262

Wip 9031 9262

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2553 from ceph/wip-aio-journal

osd: clean up journal aio+dio header issue

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #2745 from ceph/wip-6756

JounralingObjectStore: journal->committed_thru after replay

Merge pull request #2773 from ceph/wip-9852

mon: MDSMonitor: proper error output if pool DNE on 'add_data_pool'

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #2822 from ceph/wip-9887

Wip 9887

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #2850 from ceph/wip-9987

mon: fix min_last_epoch_started tracking

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #2862 from ceph/wip-log-client

osd, mds: fix cluster log

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

qa/workunits/cephtool/test.sh: ceph, not ./ceph

Fixes: #10053
Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/fs/misc: combine sudo and echo effectively

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit abc995b6b4e0117b74c301abcb4faf4ea2ebef25)

Merge pull request #2689 from zhurongze/fix-crush

crush: fix incorrect use of adjust_item_weight method

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2881 from ceph/wip-10030

librbd: don't close an already closed parent image upon failure

Reviewed-by: Sage Weil <sage@redhat.com>

crush: fix incorrect use of adjust_item_weight method

adjust_item_weight method will adjust all buckets which the item
inside. If the osd.0 in host=fake01 and host=fake02, we execute
"ceph osd crush osd.0 10 host=fake01", it not only will adjust fake01's
weight, but also will adjust fake02's weight.

the patch add adjust_item_weightf_in_loc method and fix remove_item,
_remove_item_under, update_item, insert_item, detach_bucket methods.

Signed-off-by: Rongze Zhu <zrzhit@gmail.com>

erasure-code: erasure_code_benchmark exhaustive erasure exploration

Add the --erasure-generation exhaustive flag to try all combinations of
erasures, not just one at random.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

erasure-code: add erasure_code_benchmark --verbose

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

erasure_code: implement ceph_erasure_code to assert the existence of a plugin

This is handy when scripting in the context of teuthology and only
conditionally run tests for the isa plugin, for instance.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

erasure-code: ceph_erasure_code does not need to avoid dlclose

The only reason for not dlclosing plugins at exit is for callgrind but
ceph_erasure_code has no workload that would require callgrind.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>