ceph-objectstore-tool: mount an ObjectStore via fuse

Currently only read-only.

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: put the OSD journal in the normal location

Also, drop the ability to run without a journal.

Signed-off-by: Sage Weil <sage@redhat.com>

[g]hobject_t: normalize operator<< result, and implement parse

- print hash in *bitwise reverse* order
- include all fields, every time
- use : delimitor for hobject_t
- use @ delimitor for ghobject_t
- escape unprintable characters <32, >=127, :, %.
  - also /, for fuse layer's benefit

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7195 from liewegas/wip-rados-bench

rados: rados bench improvements

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7269 from liewegas/wip-bluestore-btree

bluestore: use btree_map for allocator

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #6 from ifed01/wip-bluestore-btree-test

UT for interval_set implementations

Merge pull request #6869 from chuanhong-wang/chuanhong-wang-wip-14026

osd: fix OSD utilization is abnormal after data disk lost

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7201 from ErwanAliasr1/evelu-cleanup-deadstuff

cleanup: remove misc dead code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7217 from vumrao/wip-14356-vumrao

osd: FileStore: cleanup: remove obsolete option "filestore_xattr_use_omap"

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7245 from Sandy4999/wip-mon-election

mon: add an independent option for max election time

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #7253 from wido/mon-err-stuck-pg

mon: go into ERR state if multiple PGs are stuck inactive

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7281 from xuechendi/omap_sync_assert

osd: FileStore: add error check for object_map->sync()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7309 from asheplyakov/bug-14428

osd: PG::activate(): handle unexpected cached_removed_snaps more gracefully

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7319 from chenji-kael/wip-cls-clear-pool

test/cls: clean useless pools

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7320 from yuyuyu101/kill-keyvaluestore

os/keyvaluestore: kill KeyValueStore

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7375 from rzarzynski/wip-14526

rgw: fix dangeours usage of strtok in RGWAccessControlPolicy_SWIFT

Add UT for interval_set implementations

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #7379 from branch-predictor/bp-async-fixes-0127

msg/async: bunch of fixes

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async: bunch of fixes

1. Use MSG_MORE in sendmsg to reduce number of actual sends and pass hint
   to do_sendmsg in callers.
2. Move restore_sigpipe() outside of loop in do_sendmsg().
3. restore_sigpipe when hit the error condition in do_sendmsg().
3. When sent_bytes >= outcoming_bl.length, don't swap outgoing_bl with
   bl and wait for bl to get out of scope, just clear out outgoing_bl.
4. Don't swap empty outcoming_bl with bl, always claim_append(bl).

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #7329 from fullerdj/wip-14476

librbd: remove canceled tasks from timer thread

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: fix dangeours usage of strtok in RGWAccessControlPolicy_SWIFT.

Fixes: #14526
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #7120 from zaitcev/wip-5073-update1

rgw: fix up the use of tenant before it's available

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #7366 from tchaikov/wip-cmake-snappy

cmake: check for libsnappy in default path also

cmake: check for libsnappy in default path also

* some distros install libsnappy into /usr/lib/x86_64-linux-gnu
  which is completely resonable. but this directory is not listed
  in the Findsnappy.cmake.
* also add SNAPPY_ROOT_DIR, in case user install snappy in some
  other directory.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7304 from liewegas/wip-tcmalloc

osd: pass tcmalloc env through to daemon

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #7358 from chenyehua11692/master

mds:judgment added to avoid the risk of visiting the NULL pointer

Reviewed-by: Yan, Zheng <zyan@redhat.com>

upstart/ceph-osd.conf: pass TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES through

Signed-off-by: Sage Weil <sage@redhat.com>

mds:judgment added to avoid the risk of visting the NULL pointer

Signed-off-by: chenyehua11692 chen.yehua@h3c.com

Merge pull request #7251 from tchaikov/wip-rados-manpage

doc: amend the rados.8

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7353 from renhwsky/renhw-wip-14473-2

mon: fix locking in preinit error paths

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7321 from renhwsky/renhw-wip-14473

mon: drop useless rank init assignment

Reviewed-by: Sage Weil <sage@redhat.com>

mon: drop useless rank init assignment

drop useless rank init assignment

Fixes:14508
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #7294 from branch-predictor/bp-buffer-spinlock

common/buffer: replace RWLock with spinlocks

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: song baisen <song.baisen@zte.com.cn>

Merge pull request #7303 from cbodley/wip-tx-move

os/ObjectStore: add custom move operations for ObjectStore::Transaction

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7283 from xiexingguo/xxg-wip-14407

os/bluestore/kstore: fix nid overwritten logic

http://pulpito.ceph.com/sage-2016-01-25_07:39:08-rados-wip-sage-testing2---basic-smithi/

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7267 from liewegas/wip-fiemap

osd: drop fiemap len=0 magic

http://pulpito.ceph.com/sage-2016-01-25_07:39:08-rados-wip-sage-testing2---basic-smithi/

Reviewed-by: Sage Weil <sage@redhat.com>

mon: fix locking in preinit error paths

fix locking in preinit error paths

Fixes: #14473
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

rados: bench: add --max-objects

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7314 from tchaikov/wip-fix-cmake

cmake: let ceph-client-debug link with tcmalloc

Merge pull request #7331 from XinzeChi/wip-RepModify-bytes-written

osd: delete useless bytes_written defined in RepModify

Reviewed-by: Kefu Chai <kchai@redhat.com>

PG::activate(): handle unexpected cached_removed_snaps more gracefully

PGPool::update(): ditto

Fixes: #14428
Backport: infernalis, hammer, firefly

Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>

Merge pull request #7342 from tchaikov/wip-fix-ceph-dencoder

os/bluestore: fix bluestore_wal_transaction_t encoding test

Reviewed-by: Loic Dachary <ldachary@redhat.com>

os/bluestore: fix bluestore_wal_transaction_t encoding test

bluestore_wal_op_t::op is encoded/dumped but it is not
initialized in its ctor. so set it to 0 in its ctor. 0 is not an
valid op though, but it helps with debugging and testing.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7315 from tchaikov/wip-kill-warnings

Wip kill warnings

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #7284 from ceph/wip-make-remove-libedit

makefile: remove libedit from libclient.la

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

osd: delete useless bytes_written defined in RepModify

this would avoid calling encode function.

Signed-off-by: Xinze Chi <xinze@xsky.com>

Merge pull request #7301 from oritwas/wip-14375

rgw: fix subuser rm command failure

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rbd: remove canceled tasks from timer thread

When canceling scheduled tasks using the timer thread, TaskFinisher::cancel
does not call SafeTimer::cancel_event, so events fire anyway. Add this call.

Fixes: #14476
Signed-off-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #7327 from igrcic/master

doc: Fixes a CRUSH map step take argument

Reviewed-by: John Wilkins <jowilkin@redhat.com>

doc: Fixes a CRUSH map step take argument

Signed-off-by: Ivan Grcic <igrcic@gmail.com>

os: add custom move ctor/assignment for Transaction

Signed-off-by: Casey Bodley <cbodley@redhat.com>

os: add custom move ctor/assignment for TransactionData

Signed-off-by: Casey Bodley <cbodley@redhat.com>

os: Transaction sets default values to avoid duplication in ctors

Signed-off-by: Casey Bodley <cbodley@redhat.com>

os: move Transaction ctors closer to member variables

this makes it easier to update constructors when adding/removing fields

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test: add unit test for Transaction move/copy

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cleanup: remove obsolete option "filestore_xattr_use_omap"

This patch does cleanup for option "filestore_xattr_use_omap",
as this option was removed in #7408.

Fixes: #14356
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

KeyValueStore: Kill this

Signed-off-by: Haomai Wang <haomai@xsky.com>

test/cls ：clean useless pools

After do ./ceph_test_cls_log ,pools remains
need be destroyed.

Signed-off-by: Ji Chen <insomnia@139.com>

rgw: fix up the use of tenant before it's available

A big problem that cropped up with the multi-tenancy as it
went in was that it used tenant to initialize bucket names
before tenant was actually available by parsing a token.
Thus, although multi-tenancy worked, accessing anything across
tenants was actually impossible. This patch fixes that oversight.
Unfortunately, a new method postauth_init() was needed for it.

We also fix small bugs:

 - validate_tenant_name() used a proper-looking C++ but was
   incorrect. We simply use C for now, which may be not as
   pretty but is correct.

 - We now make a distinction between empty tenant syntax that
   means "the logged-in tenant" ("bucket") and the syntax
   that means "the legacy compatibility tenant with no name
   (":bucket").

 - We make rgw_make_bucket_entry_name() more robust against
   empty inputs. It's not used for work, but produces odd-looking
   debugging logs.

Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>

Merge pull request #7317 from dmick/master

configure.ac: make "--with-librocksdb-static" default to 'check'

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

configure.ac: make "--with-librocksdb-static" default to 'check'

Fixes: #14463
Signed-off-by: Dan Mick <dan.mick@redhat.com>

librbd: fixed "hidden" virtual methods

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: let ceph-client-debug link with tcmalloc

to fix the link problem of
libcephfs.so.1.0.0: undefined reference to `IsHeapProfilerRunning'
when tcmalloc is used and profiler header is detected.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/objectstore: fix unused variable warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/[btree_]interval_set: drop useless assert

This mirrors the condition above.

Signed-off-by: Sage Weil <sage@redhat.com>

include/[btree_]interval_set: use std::swap

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/FreelistManager: use btree_map::erase return value

The btree_map erase() returns an iterator for the next element.
Perfect!

Reported-by: Igor Fedotov <ifedotov@mirantis.com>
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7310 from trociny/fix-rbd-replay-eof

rbd-replay: handle EOF gracefully

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay: handle EOF gracefully

Fixes: #14452
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #7292 from tchaikov/wip-submittingpatches

doc: revise SubmittingPatches

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #7263 from theanalyst/vstart/update-opts

vstart: update help with a few more commandline opts

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: vstart update a few commandline switches

Add a few more available commandline switches to vstart quickstart
documentation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7240 from cxwshawn/org-fix

organizationmap: modify org mail info.

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

SubmittingPatches: rebase to resolve conflicts

pull request is not supposed to contain merge commits

Signed-off-by: Kefu Chai <kchai@redhat.com>

SubmittingPatches: sending kernel patches to mailing list

Signed-off-by: Kefu Chai <kchai@redhat.com>

SubmittingPatches: add doc for commit title

explain that we need a prefix in the commit title

Signed-off-by: Kefu Chai <kchai@redhat.com>

SubmittingPatches: add doc for "Fixes:" tag

Signed-off-by: Kefu Chai <kchai@redhat.com>

SubmittingPatches: update with reST syntax

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7293 from xiexingguo/xxg-wip-14436

os/bluestore: fix assert

Reviewed-by: Sage Weil <sage@redhat.com>

init-ceph: pass TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES through

..when set in the sysconfig/default file.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7197 from ceph/wip-kill-warnings

build: fix warnings

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7232 from lzhng2000/master

cls/cls_rbd: pass string by reference

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7242 from xiexingguo/xxg-wip-14370

os/memstore: drain finisher first before we really begin a umount-process
os/bluestore: drain wal_wq on replay error

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7247 from ifed01/wip-14383

os/bluestore: use intrusive_ptr for Dir

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7254 from guangyy/wip-counter

common: change the type of counter total/unhealthy_workers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7255 from yuyuyu101/wip-14386

msg/async: fix send closed local_connection message problem

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/jewel'

Merge pull request #7300 from ceph/wip-cmake

cmake: test_build_libcephfs needs ${ALLOC_LIBS}

rgw: fix subuser rm command failure

Fixes: 14375
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake: test_build_libcephfs needs ${ALLOC_LIBS}

Signed-off-by: Ali Maredia <amaredia@redhat.com>

include/interval_set: clean up std usage

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/StupidAllocator: use btree_map<> instead of map<>

btree_map uses about half the memory.  According to the cpp-btree docs they
are also faster than STL maps... especially when they get big.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/FreelistManager: use btree_map instead of map

btree_map is about twice as memory efficient.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_rados: clean up std usage

Signed-off-by: Sage Weil <sage@redhat.com>

include: add btree_interval_map

A few small changes to interval_map to deal with the fact that any btree
change invalidates all iterators.

Signed-off-by: Sage Weil <sage@redhat.com>

include/cpp-btree: add google's cpp-btree library

From https://github.com/algorithm-ninja/cpp-btree.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7231 from renhwsky/renhw-wip-14369

osd: remove unused OSDMap::set_weightf()

Reviewed-by: Kefu Chai <kchai@redhat.com>

BlueStore/StupidAllocator: fix assert

Fixes: #14436
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7164 from flaf/master

doc: fix "mon osd down out subtree limit" option name

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix "mon osd down out subtree limit" option name

The name of the parameter `mon osd down out subtree limit`
was incorrect (with `downout` instead of `down out`).
Furthermore, the description seemed to me a little bit
ambiguous.

Signed-off-by: François Lafont <francois.lafont@ac-versailles.fr>