cmake: add common/fs_types.cc to libcommon

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #7098 from liewegas/wip-file-layout

mds, client: add namespace to file_layout_t (previously ceph_file_layout)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7849 from ceph/wip-cmake

Wip cmake

cmake: simplified heap_profiler_objs dependencies

The heap_profiler object library only needs to be
added to mon,mds, and osd instead of the numerous
targets it is part of

Signed-off-by: Ali Maredia <amaredia@redhat.com>

client/Inode: fix has_dir_layout

Compare to default file_layout_t() -- not zeroed memory.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7869 from rzarzynski/wip-rgw-aws4-puthash

rgw: calculate payload hash in RGWPutObj_ObjStore only when necessary.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge remote-tracking branch 'gh/jewel'

Merge pull request #7612 from liewegas/wip-buffer-combined

buffer: raw_combined allocations buffer and ref count together

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7690 from xiexingguo/xxg-wip-14804

common/TrackedOp: fix inaccurate counting for slow requests

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7704 from xiexingguo/xxg-wip-14814

osd: filestore: fix wrong scope of result code for error cases during mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7812 from xiexingguo/xxg-wip-14872

osd: fix wrong return type of find_osd_on_ip()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7815 from branch-predictor/bp-conditional-crc

msg/async: don't calculate msg header crc when not needed

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7711 from xiexingguo/xxg-wip-14817

osd: filestore: fix result code overwritten for clone

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7790 from tchaikov/wip-log-segv

log: segv in a portable way

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #7842 from xiexingguo/xxg-wip-14921

osd: fix overload of '==' operator for pg_stat_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7390 from badone/wip-13829

common: Allow OPT_INT settings with negative values

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #7854 from yehudasa/wip-authors-update

AUTHORS: update email

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7846 from dillaman/wip-14551

rbd: update default image features

Conflicts:
PendingReleaseNotes (trivial)

Merge pull request #7841 from dillaman/wip-14550

rbd: deprecate image format 1

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7839 from dillaman/wip-14809

librbd: remove last synchronous librados calls from open/close state machine

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7785 from dillaman/wip-14822

librbd: journal replay needs to support re-executing maintenance ops

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7776 from dillaman/wip-librbd-journal-tweaks

journal: improve commit position tracking

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cmake: Turned LTTng OFF

This is temporary, until a way to stop repetitive
unecessary generation of headerfiles is figured out.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

AUTHORS: update email

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: calculate payload hash in RGWPutObj_ObjStore only when necessary.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #7767 from athanatos/wip-sam-journal-throttle-4

osd: filestore: restructure journal and op queue throttling

Reviewed-by: Samuel Just <sjust@redhat.com>

mds: deny access to file with pool_ns layout if feature is missing

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: fix inodestat size calculation

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: properly purge backtrace object

backtrace objects are always in default namespace

Signed-off-by: Yan, Zheng <zyan@redhat.com>

messages/MClientCaps: fix decoding

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds/Server: fix whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

mds/Locker: fix Session leak on access denial

Signed-off-by: Sage Weil <sage@redhat.com>

fix init of new fields in MCLientCaps

mds/MDSMap: add file layout v2 incompat feature

The existing MDSMonitor machinations already add this to the MDSMap to
boot out old MDS's.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: deny access to directories with pool_ns layouts if feature is missing

If the client feature is missing, given them EIO.  Change check_access to
return an error code to distinguish this from EACCES.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MDSMonitor: prevent pool 0 from being used as a data pool

Pool 0 means no change or default in the legacy ceph_file_layout in the
layout ioctl and file create arguments. Prevent it from being used to avoid
putting users in an awkward situation later.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: don't break compatibility of MClientCaps/MClientReply

Signed-off-by: Yan, Zheng <zyan@redhat.com>

fs_types: file_layout_t: convert pool -1 (undefined) to 0 in legacy encoding

Old code assumes that fl_pg_pool == 0 means the pool is not defined, while
file_layout_t uses -1.  Translate between the two.

Note that this means a valid file_layout_t with pool_id == 0 cannot be
accurately translated to a legacy file_layout_t.  That is somewhat
unavoidable, and should not be a problem since real clusters create 'rbd'
as pool 0 and it does not use any file layouts.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/fs/misc/layout_vxattrs: test layout.pool_namespace

Signed-off-by: Sage Weil <sage@redhat.com>

client: expose layout.pool_namespace vxattr

Signed-off-by: Sage Weil <sage@redhat.com>

mds: allow adjustment of layout.pool_namespace vxattr

Signed-off-by: Sage Weil <sage@redhat.com>

struct ceph_file_layout -> file_layout_t

- drop the global
- do not memset!
- encode with features
- field names are different
- use get_period() method where appropriate
- fix is layout empty checks

Signed-off-by: Sage Weil <sage@redhat.com>

mds: make inode_t, old_inode_t featureful

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MClientCaps: avoid fixed struct for encoding

At least for new clients.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: encode inode reply information directly

Drop the ceph_mds_reply_inode struct and encode directly.  Note
that we already decode directly to InodeStat in MClientReply.h.

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MClientReply: stop using ceph_mds_reply_inode struct to unpack

Decode directly.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: pass features to various CInode encoders and helpers

We'll need this shortly.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSMap: cache up_features; fix logic

& these, not | them.

Signed-off-by: Sage Weil <sage@redhat.com>

libradosstriper: move default stripe parameters inline

The extern is going away.

Signed-off-by: Sage Weil <sage@redhat.com>

include/encoding: featureful encoder for list of shared_ptrs

Signed-off-by: Sage Weil <sage@redhat.com>

include/compact_map: featureful encoders, too

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-dencoder: TYPE_FEATUREFUL_NOCOPY

Signed-off-by: Sage Weil <sage@redhat.com>

mds: pass features to LogEvents, EMetaBlob

Signed-off-by: Sage Weil <sage@redhat.com>

mds,mon: include features in beacon messages, MDSMap

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSMap: add features to MDSMap's mds_info_t

Signed-off-by: Sage Weil <sage@redhat.com>

add fs_types.h, with file_layout_t

Signed-off-by: Sage Weil <sage@redhat.com>

declare Formatter for everyone

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/fs/misc/layout_vxattr.sh: detect data pool name

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7823 from SUSE/wip-14906

packaging: add lsb-release dependency for SUSE and Debian

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #7837 from rldleblanc/wpq-documentation

doc: osd-config Add Configuration Options for op queue.

Merge pull request #7851 from jupiturliu/obj-bencher-bug

rados: fix bug for write bench

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7729 from yuyuyu101/wip-spdk2

osd: bluestore: Revert NVMEDevice task cstor and refresh interface changes

packaging: lsb_release build and runtime dependency

The lsb_release executable is being run in multiple places, not least in
src/common/util.cc, which calls it via shell in the collect_sys_info() code
path.

This patch addresses this issue on SUSE- and Debian-derivatives, as well
as reinstating the dependency for RHEL/Fedora after it was dropped in
15600572265bed397fbd80bdd2b7d83a0e9bd918.

http://tracker.ceph.com/issues/14906 Fixes: #14906

Signed-off-by: Nathan Cutler <ncutler@suse.com>

buffer: use alignof for raw_combined allocation arithmetic

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: clean up raw_combined construction

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: size append_buffer so that it fits into page-multiple allocations

We drop some unittest assertions about alloc buffer size.  Sorry!

Signed-off-by: Sage Weil <sage@redhat.com>

rbd-replay: s/CEPH_BUFFER_APPEND_SIZE/CEPH_PAGE_SIZE/

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: alloc right-sized buffer from read_fd

This may as well fit the input; this doesn't relate to the
append buffer.

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: use raw_combined for certain allocations

If the alignment is on a page boundary, or the allocation is big,
a separate buffer::raw goes faster.  The rest of the time,
a raw_combined does.

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: align unspecified allocations to a word

This will let us put policy create_aligned.

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: combine data and buffer::raw into single allocation

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: add front(), back(), get_num_buffers() methods

These eliminate most callers of buffers(), which exposes the
internal list<ptr>.

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bufferlist: benchmark some allocations

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bufferlist: fix append_bench

- fix source
- include larger sizes

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bufferlist: fix ptr move test

Do not assume there is a trailing null the terminate the string.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7739 from xiexingguo/xxg-wip-14387

osd: fix race condition for heartbeat_need_update

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7703 from xiexingguo/xxg-wip-statfs

osd: filestore: fast abort if statfs encounters ENOENT

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7766 from adamemerson/wip-portable-mutex

common: Do not use non-portable constants in mutex_debug

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #7799 from xiexingguo/xxg-wip-fixnvme

osd: bluestore: NVMEDevice: fix error handling

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7804 from jjhuo/fixes

osd: kstore: fix a race condition in _txc_finish()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7816 from marcan/build-fixes

configure.ac: boost_iostreams is required, not optional

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: use aio watch/unwatch during open/close state machines

Removed the place holders in the open/close async state machines with
the newly added aio watch/unwatch methods.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados_test_stub: added aio_watch/aio_unwatch/aio_watch_flush

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: adjust rbd test case guards to handle new defaults

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: use default RBD features in test cases

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: permit deep-flatten to be dynamically disabled

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: update default image features

Exclusive lock, object map, fast-diff, and deep-flatten have been
enabled by default for all new images.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: apply orphaned maintenance ops after journal replay

If a client recorded a maintenance op to the journal but crashed
before writing the op finish event, the image will be in an
inconsistent state.  Therefore, once the end of the journal is
reached, attempt to apply all queued ops.

Fixes: #14822
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: only cancel replay of journal op events upon error

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: delay commit of op start event

If the start event is flagged as committed before the op is
actually executed, librbd won't be able to replay the event
should a crash occur.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: permit fire-and-forget aio commit position flush

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: support replay of maintenance ops

Ignore errors that are to be expected when replaying
a maintenance op two or more times.

Fixes: #14822
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7858 from jdurgin/wip-rbd-formatted-output-progress

test/cli-integration/rbd: disable progress output

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #7745 from sileht/sileht/rados-cython

pybind: move cephfs to Cython

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #7831 from yuyuyu101/wip-14912

AsyncMessenger: fix several bugs

Merge pull request #7852 from xiexingguo/xxg-wip-14928

msg: async: start over after failing to bind a port in specified range

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kefu@redhat.com>

msg: async: start over after failing to bind a port in specified range

The original report associated with this fix can be found at
http://tracker.ceph.com/issues/13002. I reopen one in case it
needs to be backported or something alike.

Fixes: #14928
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test/cli-integration/rbd: disable progress output

This test is intended to check machine-readable output and usage, not
human-readable output.

Fixes: #14931
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7778 from marcan/pybind-fixes

pybind/rados: fix object lifetime issues and other bugs in aio

Reviewed-by: Mehdi Abaakouk <sileht@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7855 from tchaikov/wip-fix-ftbfs

test/TestPGLog: fix the FTBFS

Reviewed-by: Loic Dachary <ldachary@redhat.com>