mon: PG Monitor should report backfill_wait

Instead of backfill as it is actually waiting for backfill instead of
actually backfilling

Fixes: #12744
Reported-by: Dan Mick <dmick@redhat.com>
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7075 from shun-s/shun-fix

global: do not start two daemons with a single pid-file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7111 from chenji-kael/master

osd: FileStore: print file name before osd assert if read file failed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7268 from ifed01/wip-14400

common: snappy decompressor may assert when handling segmented input bufferlist

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7276 from caibo2014/fix-messager-null

mon/MonClient: avoid null pointer error when configured incorrectly

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7330 from ifed01/wip-pool-stat-cleanup

librados: remove duplicate definitions for rados pool_stat_t and cluster_stat_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7340 from ceph/wip-portable-time

common: ake ceph_time clocks work under BSD

Reviewed-by: Erwan Velu <erwan@redhat.com>

Merge pull request #7341 from songbaisen/song7

mon: consider the pool size when setting pool crush rule

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7355 from yuanjingkai/master

os/fs: fix io_getevents argument

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7348 from xiexingguo/xxg-wip-14484

osd: BlueStore: fix space leak

http://pulpito.ceph.com/sage-2016-01-27_18:39:48-rados-wip-sage-testing---basic-smithi/

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7368 from somnathr/wip-jemalloc-build

build: allow jemalloc enabled build with rocksdb-static

http://pulpito.ceph.com/sage-2016-01-27_18:39:48-rados-wip-sage-testing---basic-smithi/

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'origin/greg-fs-testing-126'

Merge remote-tracking branch 'origin/jewel'

Merge pull request #7312 from jcsp/wip-14365

mds, client: fix locking around handle_conf_change

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #7297 from ukernel/jewel-13903

client: properly trim unlinked inode

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7262 from ukernel/jewel-14380

Be more careful about the "mds setmap" command to prevent breakage.

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #7369 from mauri/fix-message-typo

rbd: fix error message typo

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7387 from theanalyst/fix/gitignore

remove unused src/messages/#MOSDOp.h# file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7345 from x11562/rbd_snap_rename

rbd: add "--dest-snap" optional for snap rename

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7393 from tchaikov/wip-cmake

fix the autotools and cmake build (the new fusestore needs libfuse)

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7403 from rzarzynski/wip-rgw-dumpconlen-cosmetics

rgw: cosmetic improvements in dump_content_length()

rgw: cosmetic improvements in dump_content_length().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

ceph-objectstore-tool: disable fuse support if !HAVE_LIBFUSE

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: disable FuseStore if !HAVE_LIBFUSE

Signed-off-by: Kefu Chai <kchai@redhat.com>

make: disable FuseStore if !WITH_FUSE

* configure.ac: "#define HAVE_LIBFUSE" if libfuse is found
* os/Makefile.am: do not compile FuseStore.cc if libfuse is not found

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: link against libfuse if HAVE_LIBFUSE

* use ${FUSE_LIBRARIES} instead of "fuse" when referencing libfuse.
* do not compile FuseStore.cc if !HAVE_LIBFUSE.
* CMakeLists.txt: s/HAVE_FUSE/HAVE_LIBFUSE/, to match with the one
  defined for config.h.in.cmake, otherwise we need two vars:
  HAVE_LIBFUSE, and HAVE_FUSE

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: remove the copyright of FindSnappy.cmake

as we've basically removed all the HyperTable code.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6875 from dx9/wip-cmake-ccache

cmake: support ccache via a WITH_CCACHE build option

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd: simplify snap rename arguments

Signed-off-by: Na Xie <xie.na@h3c.com>

Merge pull request #7337 from clever215/rbd_nbd

rbd-nbd: Add an argument validation for nbds_max

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7385 from trociny/fix-journal-flush-commit-position

journal: flush commit position on metadata shutdown

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7382 from dillaman/wip-14510

librbd: block maintenance ops until after journal is ready

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7259 from trociny/wip-14264

librbd: API: async open and close

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7302 from dillaman/wip-14434

librbd: journal shut down flush race condition

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7299 from dillaman/wip-librbd-update-features

librbd: fix internal handling of dynamic feature updates

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

gitignore: ignore backup files from editors etc.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

remove unused source file

This file was possibly accidentally committed in the tree, thanks
@BranchPredictor for bringing this up in irc

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7381 from ceph/wip-rgw-static-website-yehuda

RGW static website feature
 - currently disabled by default
 - partial error page handling
 - a few known failures

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Robin H. Johnson <robin.johnson@dreamhost.com>

journal: flush commit position on metadata shutdown

A crash was observed on JournalTrimmer destroy, which was called
by the Journaler after metadata shutdown. JournalTrimmer destructor
also calls flush_commit_position, but at that time metadata m_timer
is already null and it crushed there.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #7223 from liewegas/wip-collection-handles

osd: collection handles

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7134 from liewegas/wip-fusestore

osd: implement fuse interface for ObjectStore

Reviewed-by: Samuel Just <sjust@redhat.com>

osd: use collection handle where possible

Signed-off-by: Sage Weil <sage@redhat.com>

osd: associate a CollectionHandle& with PGBackend

Make this a ref to the PG's handle because the backend is
constructed before the handle is opened.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: associate a CollectionHandle with each PG

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: implement handle-based read methods

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: implement open_collection

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: implement CollectionHandle based read methods

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: implement open_collection

Fix up create and remove so that they poison a Collection that
gets removed.

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add CollectionHandle wrappers for all read ops

The backend can optionally implement the handle-based variant. If
it does not, the wrappers will call the legacy coll_t version.

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add CollectionHandle

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: block maintenance until after journal is ready

Ops were already blocked when the requests were received via
watch/notify.  Requests are now blocked for local requests as
well.

Fixes: #14510
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge branch 'wip-rgw-static-website-yehuda'

Conflicts:
src/rgw/rgw_common.h
src/rgw/rgw_op.h
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_rest_s3.h

os/FuseStore: improve readdir for omap and attr

Use new-style fuse readdir interface for omap and attr.  We aren't
particularly efficient, but we won't completely break if there are a
lot of attrs or omap entries.

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: wait for commit and flush in all updates

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: use offset for readdir

This allows us to list large directories.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/objectstore/test_fuse.sh: simple workunit test

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: add 'type' file in root dir

This also acts as a sentinel to indicate whether we have mounted.

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add get_type()

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: drop filestore-specific sanity check

Just let mount() fail

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: add --op mkfs

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: simplify error message

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: 'whoami' is not required for an ObjectStore

OSD creates it, but it won't be there if we did --op mkfs

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: do not call exists() on an out-of-bounds oid

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: no-op chmod

This makes mkdir(1) happy.

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: allow object and collection creates

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: more ENOENT on getattr for non-existent items

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: hide bitwise_hash_end if bits unknown; add bitwise_hash_bits

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: present bitwise_hash_end

Note this is the *last* hash value, inclusive.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: implement collection_bits

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add collection_bits

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: respect fuse_debug

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: no need to check for fsck here

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: use myexit(1) for fsck

Signed-off-by: Sage Weil <sage@redhat.com>

osd: osd_objectstore_fuse

Expose underlying ObjectStore via fuse at $osd_data/fuse/.  Can be enabled/
disabled at runtime.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: mount an ObjectStore via fuse

Currently only read-only.

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: put the OSD journal in the normal location

Also, drop the ability to run without a journal.

Signed-off-by: Sage Weil <sage@redhat.com>

[g]hobject_t: normalize operator<< result, and implement parse

- print hash in *bitwise reverse* order
- include all fields, every time
- use : delimitor for hobject_t
- use @ delimitor for ghobject_t
- escape unprintable characters <32, >=127, :, %.
  - also /, for fuse layer's benefit

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7195 from liewegas/wip-rados-bench

rados: rados bench improvements

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7269 from liewegas/wip-bluestore-btree

bluestore: use btree_map for allocator

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #6 from ifed01/wip-bluestore-btree-test

UT for interval_set implementations

Merge pull request #6869 from chuanhong-wang/chuanhong-wang-wip-14026

osd: fix OSD utilization is abnormal after data disk lost

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7201 from ErwanAliasr1/evelu-cleanup-deadstuff

cleanup: remove misc dead code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7217 from vumrao/wip-14356-vumrao

osd: FileStore: cleanup: remove obsolete option "filestore_xattr_use_omap"

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7245 from Sandy4999/wip-mon-election

mon: add an independent option for max election time

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #7253 from wido/mon-err-stuck-pg

mon: go into ERR state if multiple PGs are stuck inactive

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7281 from xuechendi/omap_sync_assert

osd: FileStore: add error check for object_map->sync()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7309 from asheplyakov/bug-14428

osd: PG::activate(): handle unexpected cached_removed_snaps more gracefully

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7319 from chenji-kael/wip-cls-clear-pool

test/cls: clean useless pools

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7320 from yuyuyu101/kill-keyvaluestore

os/keyvaluestore: kill KeyValueStore

Reviewed-by: Sage Weil <sage@redhat.com>

tools/rbd: fix error message typo

Signed-off-by: Mauricio Garavaglia <mauriciogaravaglia@gmail.com>

Merge pull request #7375 from rzarzynski/wip-14526

rgw: fix dangeours usage of strtok in RGWAccessControlPolicy_SWIFT

Add UT for interval_set implementations

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #7379 from branch-predictor/bp-async-fixes-0127

msg/async: bunch of fixes

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async: bunch of fixes

1. Use MSG_MORE in sendmsg to reduce number of actual sends and pass hint
   to do_sendmsg in callers.
2. Move restore_sigpipe() outside of loop in do_sendmsg().
3. restore_sigpipe when hit the error condition in do_sendmsg().
3. When sent_bytes >= outcoming_bl.length, don't swap outgoing_bl with
   bl and wait for bl to get out of scope, just clear out outgoing_bl.
4. Don't swap empty outcoming_bl with bl, always claim_append(bl).

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #7329 from fullerdj/wip-14476

librbd: remove canceled tasks from timer thread

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: fix dangeours usage of strtok in RGWAccessControlPolicy_SWIFT.

Fixes: #14526
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

test: add unitest test_pidfile.sh

Fixes: #13422
Signed-off-by: shun song <song.shun3@zte.com.cn>

Mon: When set pool crush rule should consider the pool size.

Fixes: #14495
Signed-off-by: song baisen <song.baisen@zte.com.cn>

rbd: Add an argument-check for nbds_max in rbd-nbd

Fixes: #14482
This modification adds several checks to the argument of nbds_max in the module of rbd-nbd. The value of nbds_max is strictly limited to non-negative integer now.

Signed-off-by: Kongming Wu <wu.kongming@h3c.com>

Build: Allow jemalloc enabled build with rocksdb-static

Presently, we can't do jemalloc enabled OSD build along with
rocksdb-static.Rocksdb is now mandatory with Bluestore and that's why
jemalloc builds are failing. Sometimes back the following commit
fd7a35731854528dde10deb3444a82ae4c1a9206 is introduced to build rocksdb
with jemalloc as well, so, we can allow jemalloc build along with
rocksdb-static now. Removing the redundant check from configure.ac.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>