qa/workunits/objectstore/test_fuse.sh: no bashism

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'origin/greg-fs-testing-126'

Merge remote-tracking branch 'origin/jewel'

Merge pull request #7312 from jcsp/wip-14365

mds, client: fix locking around handle_conf_change

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #7297 from ukernel/jewel-13903

client: properly trim unlinked inode

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7262 from ukernel/jewel-14380

Be more careful about the "mds setmap" command to prevent breakage.

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #7369 from mauri/fix-message-typo

rbd: fix error message typo

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7387 from theanalyst/fix/gitignore

remove unused src/messages/#MOSDOp.h# file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7345 from x11562/rbd_snap_rename

rbd: add "--dest-snap" optional for snap rename

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7393 from tchaikov/wip-cmake

fix the autotools and cmake build (the new fusestore needs libfuse)

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7403 from rzarzynski/wip-rgw-dumpconlen-cosmetics

rgw: cosmetic improvements in dump_content_length()

rgw: cosmetic improvements in dump_content_length().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

ceph-objectstore-tool: disable fuse support if !HAVE_LIBFUSE

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: disable FuseStore if !HAVE_LIBFUSE

Signed-off-by: Kefu Chai <kchai@redhat.com>

make: disable FuseStore if !WITH_FUSE

* configure.ac: "#define HAVE_LIBFUSE" if libfuse is found
* os/Makefile.am: do not compile FuseStore.cc if libfuse is not found

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: link against libfuse if HAVE_LIBFUSE

* use ${FUSE_LIBRARIES} instead of "fuse" when referencing libfuse.
* do not compile FuseStore.cc if !HAVE_LIBFUSE.
* CMakeLists.txt: s/HAVE_FUSE/HAVE_LIBFUSE/, to match with the one
  defined for config.h.in.cmake, otherwise we need two vars:
  HAVE_LIBFUSE, and HAVE_FUSE

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: remove the copyright of FindSnappy.cmake

as we've basically removed all the HyperTable code.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6875 from dx9/wip-cmake-ccache

cmake: support ccache via a WITH_CCACHE build option

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd: simplify snap rename arguments

Signed-off-by: Na Xie <xie.na@h3c.com>

Merge pull request #7337 from clever215/rbd_nbd

rbd-nbd: Add an argument validation for nbds_max

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7385 from trociny/fix-journal-flush-commit-position

journal: flush commit position on metadata shutdown

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7382 from dillaman/wip-14510

librbd: block maintenance ops until after journal is ready

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7259 from trociny/wip-14264

librbd: API: async open and close

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7302 from dillaman/wip-14434

librbd: journal shut down flush race condition

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7299 from dillaman/wip-librbd-update-features

librbd: fix internal handling of dynamic feature updates

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

gitignore: ignore backup files from editors etc.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

remove unused source file

This file was possibly accidentally committed in the tree, thanks
@BranchPredictor for bringing this up in irc

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7381 from ceph/wip-rgw-static-website-yehuda

RGW static website feature
 - currently disabled by default
 - partial error page handling
 - a few known failures

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Robin H. Johnson <robin.johnson@dreamhost.com>

journal: flush commit position on metadata shutdown

A crash was observed on JournalTrimmer destroy, which was called
by the Journaler after metadata shutdown. JournalTrimmer destructor
also calls flush_commit_position, but at that time metadata m_timer
is already null and it crushed there.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #7223 from liewegas/wip-collection-handles

osd: collection handles

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7134 from liewegas/wip-fusestore

osd: implement fuse interface for ObjectStore

Reviewed-by: Samuel Just <sjust@redhat.com>

osd: use collection handle where possible

Signed-off-by: Sage Weil <sage@redhat.com>

osd: associate a CollectionHandle& with PGBackend

Make this a ref to the PG's handle because the backend is
constructed before the handle is opened.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: associate a CollectionHandle with each PG

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: implement handle-based read methods

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: implement open_collection

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: implement CollectionHandle based read methods

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: implement open_collection

Fix up create and remove so that they poison a Collection that
gets removed.

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add CollectionHandle wrappers for all read ops

The backend can optionally implement the handle-based variant. If
it does not, the wrappers will call the legacy coll_t version.

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add CollectionHandle

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: block maintenance until after journal is ready

Ops were already blocked when the requests were received via
watch/notify.  Requests are now blocked for local requests as
well.

Fixes: #14510
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge branch 'wip-rgw-static-website-yehuda'

Conflicts:
src/rgw/rgw_common.h
src/rgw/rgw_op.h
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_rest_s3.h

os/FuseStore: improve readdir for omap and attr

Use new-style fuse readdir interface for omap and attr.  We aren't
particularly efficient, but we won't completely break if there are a
lot of attrs or omap entries.

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: wait for commit and flush in all updates

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: use offset for readdir

This allows us to list large directories.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/objectstore/test_fuse.sh: simple workunit test

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: add 'type' file in root dir

This also acts as a sentinel to indicate whether we have mounted.

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add get_type()

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: drop filestore-specific sanity check

Just let mount() fail

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: add --op mkfs

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: simplify error message

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: 'whoami' is not required for an ObjectStore

OSD creates it, but it won't be there if we did --op mkfs

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: do not call exists() on an out-of-bounds oid

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: no-op chmod

This makes mkdir(1) happy.

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: allow object and collection creates

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: more ENOENT on getattr for non-existent items

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: hide bitwise_hash_end if bits unknown; add bitwise_hash_bits

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: present bitwise_hash_end

Note this is the *last* hash value, inclusive.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: implement collection_bits

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add collection_bits

Signed-off-by: Sage Weil <sage@redhat.com>

os/FuseStore: respect fuse_debug

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: no need to check for fsck here

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: use myexit(1) for fsck

Signed-off-by: Sage Weil <sage@redhat.com>

osd: osd_objectstore_fuse

Expose underlying ObjectStore via fuse at $osd_data/fuse/.  Can be enabled/
disabled at runtime.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: mount an ObjectStore via fuse

Currently only read-only.

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: put the OSD journal in the normal location

Also, drop the ability to run without a journal.

Signed-off-by: Sage Weil <sage@redhat.com>

[g]hobject_t: normalize operator<< result, and implement parse

- print hash in *bitwise reverse* order
- include all fields, every time
- use : delimitor for hobject_t
- use @ delimitor for ghobject_t
- escape unprintable characters <32, >=127, :, %.
  - also /, for fuse layer's benefit

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7195 from liewegas/wip-rados-bench

rados: rados bench improvements

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7269 from liewegas/wip-bluestore-btree

bluestore: use btree_map for allocator

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #6 from ifed01/wip-bluestore-btree-test

UT for interval_set implementations

Merge pull request #6869 from chuanhong-wang/chuanhong-wang-wip-14026

osd: fix OSD utilization is abnormal after data disk lost

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7201 from ErwanAliasr1/evelu-cleanup-deadstuff

cleanup: remove misc dead code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7217 from vumrao/wip-14356-vumrao

osd: FileStore: cleanup: remove obsolete option "filestore_xattr_use_omap"

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7245 from Sandy4999/wip-mon-election

mon: add an independent option for max election time

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #7253 from wido/mon-err-stuck-pg

mon: go into ERR state if multiple PGs are stuck inactive

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7281 from xuechendi/omap_sync_assert

osd: FileStore: add error check for object_map->sync()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7309 from asheplyakov/bug-14428

osd: PG::activate(): handle unexpected cached_removed_snaps more gracefully

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7319 from chenji-kael/wip-cls-clear-pool

test/cls: clean useless pools

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7320 from yuyuyu101/kill-keyvaluestore

os/keyvaluestore: kill KeyValueStore

Reviewed-by: Sage Weil <sage@redhat.com>

tools/rbd: fix error message typo

Signed-off-by: Mauricio Garavaglia <mauriciogaravaglia@gmail.com>

Merge pull request #7375 from rzarzynski/wip-14526

rgw: fix dangeours usage of strtok in RGWAccessControlPolicy_SWIFT

Add UT for interval_set implementations

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #7379 from branch-predictor/bp-async-fixes-0127

msg/async: bunch of fixes

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async: bunch of fixes

1. Use MSG_MORE in sendmsg to reduce number of actual sends and pass hint
   to do_sendmsg in callers.
2. Move restore_sigpipe() outside of loop in do_sendmsg().
3. restore_sigpipe when hit the error condition in do_sendmsg().
3. When sent_bytes >= outcoming_bl.length, don't swap outgoing_bl with
   bl and wait for bl to get out of scope, just clear out outgoing_bl.
4. Don't swap empty outcoming_bl with bl, always claim_append(bl).

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #7329 from fullerdj/wip-14476

librbd: remove canceled tasks from timer thread

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: fix dangeours usage of strtok in RGWAccessControlPolicy_SWIFT.

Fixes: #14526
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rbd: Add an argument-check for nbds_max in rbd-nbd

Fixes: #14482
This modification adds several checks to the argument of nbds_max in the module of rbd-nbd. The value of nbds_max is strictly limited to non-negative integer now.

Signed-off-by: Kongming Wu <wu.kongming@h3c.com>

Merge pull request #7120 from zaitcev/wip-5073-update1

rgw: fix up the use of tenant before it's available

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge branch 'wip-14374' of git://github.com/jcsp/ceph into greg-fs-testing

https://github.com/ceph/ceph/pull/7295

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Conflicts:
src/mds/MDSDaemon.cc

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-14365' of git://github.com/jcsp/ceph into greg-fs-testing

https://github.com/ceph/ceph/pull/7312

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge branch 'jewel-13903' of git://github.com/ukernel/ceph into greg-fs-testing

https://github.com/ceph/ceph/pull/7297

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7343 from ukernel/jewel-dump-requests-lockdep

objecter: avoid recursive lock of Objecter::rwlock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7366 from tchaikov/wip-cmake-snappy

cmake: check for libsnappy in default path also

cmake: check for libsnappy in default path also

* some distros install libsnappy into /usr/lib/x86_64-linux-gnu
  which is completely resonable. but this directory is not listed
  in the Findsnappy.cmake.
* also add SNAPPY_ROOT_DIR, in case user install snappy in some
  other directory.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7304 from liewegas/wip-tcmalloc

osd: pass tcmalloc env through to daemon

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #7358 from chenyehua11692/master

mds:judgment added to avoid the risk of visiting the NULL pointer

Reviewed-by: Yan, Zheng <zyan@redhat.com>

upstart/ceph-osd.conf: pass TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES through

Signed-off-by: Sage Weil <sage@redhat.com>

mds:judgment added to avoid the risk of visting the NULL pointer

Signed-off-by: chenyehua11692 chen.yehua@h3c.com

Merge pull request #7251 from tchaikov/wip-rados-manpage

doc: amend the rados.8

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7353 from renhwsky/renhw-wip-14473-2

mon: fix locking in preinit error paths

Reviewed-by: Sage Weil <sage@redhat.com>