pkg: add ceph_volume_client to python-cephfs

Signed-off-by: John Spray <john.spray@redhat.com>

mds: fix `tell session evict`

Don't try waiting on empty context gatherbuilder.

Signed-off-by: John Spray <john.spray@redhat.com>

pybind: minor fixes to volumeclient

Signed-off-by: John Spray <john.spray@redhat.com>

pybind: stray comment in volumeclient

Signed-off-by: John Spray <john.spray@redhat.com>

pybind/cephfs: exception on extra init args

Signed-off-by: John Spray <john.spray@redhat.com>

pybind: volumeclient fixes

Generalize authorization so that an ID can be authorized
for multiple shares.

Remove commented blocks.

Separate purge from delete.  Up to the caller to background
the purge part.

Signed-off-by: John Spray <john.spray@redhat.com>

pybind: fix mkdir_p in ceph_volume_client

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: add libcephfs to cephfs_testing target

Tests for VolumeClient will use this.

Signed-off-by: John Spray <john.spray@redhat.com>

pybind: add ceph_volume_client

This is for FSaaS interfaces like Manila.

Signed-off-by: John Spray <john.spray@redhat.com>

libcephfs: add ceph_create_from_rados

This is for applications that want to use
librados and libcephfs in the same process,
without trying to have two CephContexts

Signed-off-by: John Spray <john.spray@redhat.com>

pybind: add LibCephFS.create_with_rados

Enable setting up cephfs while borrowing
the CephContext from an already-set-up
rados python client.

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: link libcephfs the same as librados

Things got weird (tcmalloc exceptions) if one used
both libraries from the same process, presumably
because of some of these dependencies.

Signed-off-by: John Spray <john.spray@redhat.com>

pybind: allow CephFS constructor to take ID

For those folks who aren't running as 'admin'.  We
already allow this at the C layer, it was just left
out of the python binding.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #7100 from dachary/wip-14220-make-check

tests: configure with rocksdb by default

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7063 from mathslinux/rgw-dont-abort-for-bad-argumentment

rgw: do not abort radowgw server when using admin op API with bad parameters

Merge pull request #7099 from jcsp/wip-cmake-bluestore

Fix cmake build with bluestore

tests: configure with rocksdb by default

Otherwise bluestore won't compile and run-make-check.sh will fail.

http://tracker.ceph.com/issues/14220 Fixes: #14220

Signed-off-by: Loic Dachary <loic@dachary.org>

cmake: build and link rocksdb etc for bluestore

Signed-off-by: John Spray <john.spray@redhat.com>

cmake: add rocksdb include path for bluestore

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6854 from jecluis/wip-monmapmon

mon: MonmapMonitor: don't expose uncommitted state to client

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mon: MonmapMonitor: don't expose uncommitted state to client

During prepare_command(), we were returning to the user based on
pending_map's state. Even though this weren't causing any issues we are
aware of, we really shouldn't do that.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #6760 from ddiss/bug13943_ceph_disk_validate_part_uuid

ceph-disk: warn for prepare partitions with bad GUIDs

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6992 from dachary/wip-mailmap

mailmap updates

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Chengyuan Li affiliation

Reviewed-by: Chengyuan Li <chengyli@ebay.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Hervé Rousseau name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Chris Holcombe name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Jianhui Yuan affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Joe Julian affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: David Coles affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Vasu Kulkarni affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Tobias Suckow affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Rahul Aggarwal affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Brian Felton affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #7030 from ceph/wip-detect-bz2-lz4

configure: detect bz2 and lz4

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #7016 from xiexingguo/xxg-wip-14151

osd: fix sparse-read result code checking logic

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7031 from wonzhq/log-monitor-2

mon: fix the can't change subscribe level bug in monitoring log

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7009 from majianpeng/osd-scrub-sleep-cause-thread-suicide

osd: avoid osd_op_thread suicide because osd_scrub_sleep

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7007 from tchaikov/wip-13990

osd: do not keep ref of old osdmap in pg

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6977 from stiopaa1/msg_override

msg: add override to virutal methods

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #6916 from xiexingguo/xxg-wip-14072

osd: fix null pointer access and race condition

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7028 from yuyuyu101/wip-async-emplace

msg/async: don't use shared_ptr to manage EventCallback

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6896 from liewegas/wip-bluestore

bluestore: latest and greatest

qa/workunits/rados/test_rados_tool.sh: fix path

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/rados/test_rados_tool: adapt to new buffer hexdump

Signed-off-by: Sage Weil <sage@redhat.com>

uniform the type format

Signed-off-by: Ning Yao <zay11022@gmail.com>

os/bluestore/BlueStore: fix overlay keys

Signed-off-by: YiQiang Chen <cyqsign@163.com>

fs: fix unclosed formatter session

Fixes: #14210
Signed-off-by: xie.xingguo <xie.xingguo@zte.com.cn>

qa/workunits/cephtool/test.sh: leave sortbitwise set

New backends don't work if it's off.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: remove deleted file from dirty list

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: Don't forget close fd in _setup_block_symlink_or_file.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

os/bluestore: write 'type' file

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: don't require journal path

Assume journal symlink is present.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: get store type from 'type' file, if present

Just like ceph-osd.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: wait for flush aios to complete before returning

This makes all the IOs effectively synchronous (while holding the lock),
which isn't great, but at least it's correct.  We can contemplate async
later..

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlockDevice: clean up aio completion a bit

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: delay IOContext dtor until after io completes

It's possible for the IO to be in flight when the caller closes the
writer handle (although dangerous of them).  Queue the IOContext for
async cleanup when we sync everything to disk.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: fix a few leaks

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: close cct on shutdown

This lets us do leak checking.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueRocksEnv: fix several leaks

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: fix leak of iocs in writer

Signed-off-by: Sage Weil <sage@redhat.com>

kv/RocksDBStore: fix leak of custom Env

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlockDevice: fix FS leak

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: no need for 'quit' to ceph cli

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: k=2 for ec

It didn't like k=1 with the default profile.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: be tolerant of collection_list bounds

In particular, we may get a shard specified along with hobject_t min or
max (from PGBackend::objects_list_range()).

Signed-off-by: Sage Weil <sage@redhat.com>

CMakeLists: update for os/ reorg

Signed-off-by: Sage Weil <sage@redhat.com>

os/Makefile: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

os/kstore: cache in-flight stripe updates in memory

The write process may do a read/modify/write on a stripe.  In order to
allow multiple writes to coexist within the same transaction, we need to
be able to "see" our writes.

Clear the "cached" stripe values when the last TransContext touching an
onode is finished.  In theory we could pin memory with a constant stream
of updates to an object; we may need to address that later.

Signed-off-by: Sage Weil <sage@redhat.com>

common/buffer: make hexdump look like 'hexdump -C ...'

...the better to diff them with.

Signed-off-by: Sage Weil <sage@redhat.com>

os/kstore: debug nid

Signed-off-by: Sage Weil <sage@redhat.com>

os/kstore: fix bugs in read stripe logic

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: clean out test dir for each test

Signed-off-by: Sage Weil <sage@redhat.com>

os/kstore: clear coll_map on umount

Signed-off-by: Sage Weil <sage@redhat.com>

osd: do not catch mkfs exceptions

We either want to take a clean error path, or dump core.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/StupidAllocator: be less stupid about alloc_unit

Make sure the extent we return is aligned to alloc_unit.  If the first
extent isn't suitable, try the next one.

Keep bin 0 for things that are smaller than min_alloc_size.

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bluestore_types: fix warnings

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/bluestore_types: fix bluestore_extent_ref_map_t test inst

Signed-off-by: Sage Weil <sage@redhat.com>

os/Makefile: cosmetic

Signed-off-by: Sage Weil <sage@redhat.com>

os/fs: move btrfs_ioctl.h to os/fs

Signed-off-by: Sage Weil <sage@redhat.com>

os/ZFS: move to os/fs

Signed-off-by: Sage Weil <sage@redhat.com>

os/keyvaluestore: move KeyValueStore into os/keyvaluestore/*

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: move MemStore into os/memstore/*

Signed-off-by: Sage Weil <sage@redhat.com>

os/filestore: move FileStore to os/filestore/*

Signed-off-by: Sage Weil <sage@redhat.com>

ceph:small_io_bench_dumb: drop unused include

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: drop unused FileSTore include

Signed-off-by: Sage Weil <sage@redhat.com>

remove ceph_streamtest

This is an ancient obsolete test that is never used.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: allocate shouldn't fail if reserve succeeded

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/Allocator: add unreserve()

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/StupidAllocator: fix reservation release on allocate

Use up the amount we allocated, not the amount we asked for.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/StupidAllocator: add reservation assertions

Signed-off-by: Sage Weil <sage@redhat.com>

os/kstore: do not include header in omap iterator result

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: do not include header in iterator result

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: print omap_digest in debug log

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: add omap iterator test

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: add fsck command

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: add fsck to interface

Only bluestore and kstore implement this currently.

Signed-off-by: Sage Weil <sage@redhat.com>

os/kstore: clear onode on _do_remove

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: clear onode in _do_remove

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PGBackend: fix omap digest error message

Print the *omap* digest, not *data* digest.

Signed-off-by: Sage Weil <sage@redhat.com>

os: remove {get,set}_allow_sharded_objects from interface

We've already forced everyone to upgrade through hammer, so everyone
supports this.  Just unconditionally set the feature if it is not set
(for consistency's sake).

Signed-off-by: Sage Weil <sage@redhat.com>