Modifications to rbd-openstack.rst

Original configuration is not working

'enabled_backends' should be changed accordingly

Signed-off-by: RustShen <rustinpeace@163.com>
Update rbd-openstack.rst

rgw: indexless buckets

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: configurable index type

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #7764 from rzarzynski/wip-rgw-etag-dlo

rgw: multiple improvements regarding etag calculation for SLO/DLO of Swift API.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge remote-tracking branch 'gh/jewel'

v10.0.4

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7897 from branch-predictor/bp-bench-errors-fatal

common/obj_bencher.cc: make verify error fatal

Reviewed-by: Loic Dachary <ldachary@redhat.com>

common/obj_bencher.cc: make verify error fatal

When run without "--no-verify", all verification errors are noted,
but they are not forwarded/reported anywhere else but to cerr, which
will cause automated testing to ignore them. Make seq_read_bench and
rand_read_bench return -EIO on any verification error which will,
in turn, return it back to caller.

Fixes: #14971
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #7757 from ukernel/jewel-14684

mon/MDSMonitor.cc: properly note beacon when health metrics changes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #7882 from branch-predictor/bp-unconfuse-health-1

mon: unconfuse object count skew message

Reviewed-by: Sage Weil <sage@redhat.com>

PGMonitor: unconfuse object count skew message

"Pool <pool> has too few pgs" is okay assuming it does not take other
pools into account. And since it does, it is confusing in the following
scenario:

1. Create two pools, one with small pg count and one with large
   pg count
2. Put a whole lot of objects in smaller pool, resulting in "too few
   pgs" warning on that pool, which is expected behavior.
3. Put a whole lot of objects in larger pool, warning goes away.
   Suddenly smaller pool has plenty of PGs?

Current message suggests adding more nodes (or PGs) to pool, when
actually it's warning about significantly more objects in that
particular pool than in the other pools.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #7898 from oritwas/wip-cmake-add-fs_types

cmake: add common/fs_types.cc to libcommon

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #7881 from aclamk/ceph-df-used-fix

mon: fix calculation of %USED

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7892 from majianpeng/bluestore-leak-unwritten-extent

osd: bluestore: fix check for write falling within the same extent

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7886 from stiopaa1/blueStoreOverrides

osd: bluestore: add 'override' to virtual functions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7895 from majianpeng/bluestore-trim-overlay

os/bluestore/BlueStore: Don't leak trim overlay data before write.

os/bluestore/BlueStore: Don't leak trim overlay data before write.

Suppose: bluestore_overlay_max_length=bluestore_min_alloc_size;
bluestore_overlay_max = 2;
For the following ops:
write(off=0, len=4096) --->write into overlay
write(off=4096, len=4096)-->write into overlay
write(off=0, len=bluestore_min_alloc_size)-->because overlay_map.size()
>=2, it allocate a extent.

It should trim overlay data(0,4096) &(4096, 4096),and then write(0,
bluestore_min_alloc_size).
But the original code don't trim overlay data.
This make the later read data is orignal data rather that new data.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

cmake: add common/fs_types.cc to libcommon

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

os/bluestore/BlueStore: Fix bug when calc offset & end whether locate in the a extent.

Suppose: bluestore_overlay_max_length == bluestore_min_alloc_size
The orignal code which calc content of written whether locate in a
extent:
(offset / min_alloc_size) == (offset + length) /min_alloc_size
This will make the case which offset=0 & length =min_alloc_size locate
in the different extent.
In fact, this content is in the same extent.

Change end = offset + length - 1 make work.

Fixes: #14954
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/obj_bencher.cc: use more readable constant instead of magic number

When clean_up_slow() fails, it returns "-5" which is equal to -EIO.
Change it in source, so it's not confusing for someone who does not
remember all error codes (functionality remains the same).

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #7098 from liewegas/wip-file-layout

mds, client: add namespace to file_layout_t (previously ceph_file_layout)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Bluestore.h:Added 'override' to virtual functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #7849 from ceph/wip-cmake

Wip cmake

cmake: simplified heap_profiler_objs dependencies

The heap_profiler object library only needs to be
added to mon,mds, and osd instead of the numerous
targets it is part of

Signed-off-by: Ali Maredia <amaredia@redhat.com>

client/Inode: fix has_dir_layout

Compare to default file_layout_t() -- not zeroed memory.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7867 from SUSE/wip-14940

RPM: move %post(un) ldconfig calls to ceph-base

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #7869 from rzarzynski/wip-rgw-aws4-puthash

rgw: calculate payload hash in RGWPutObj_ObjStore only when necessary.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge remote-tracking branch 'gh/jewel'

Merge pull request #7612 from liewegas/wip-buffer-combined

buffer: raw_combined allocations buffer and ref count together

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7690 from xiexingguo/xxg-wip-14804

common/TrackedOp: fix inaccurate counting for slow requests

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7704 from xiexingguo/xxg-wip-14814

osd: filestore: fix wrong scope of result code for error cases during mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7812 from xiexingguo/xxg-wip-14872

osd: fix wrong return type of find_osd_on_ip()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7815 from branch-predictor/bp-conditional-crc

msg/async: don't calculate msg header crc when not needed

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7711 from xiexingguo/xxg-wip-14817

osd: filestore: fix result code overwritten for clone

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7790 from tchaikov/wip-log-segv

log: segv in a portable way

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #7842 from xiexingguo/xxg-wip-14921

osd: fix overload of '==' operator for pg_stat_t

Reviewed-by: Sage Weil <sage@redhat.com>

[MON] Fixed calculation of %USED. Now it is shows (space used by all replicas)/(raw space available on OSDs). Before it was (size of pool)/(raw space available on OSDs).

Signed-off-by: Adam Kupczyk akupczyk@mirantis.com

Merge pull request #7390 from badone/wip-13829

common: Allow OPT_INT settings with negative values

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #7854 from yehudasa/wip-authors-update

AUTHORS: update email

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7846 from dillaman/wip-14551

rbd: update default image features

Conflicts:
PendingReleaseNotes (trivial)

Merge pull request #7841 from dillaman/wip-14550

rbd: deprecate image format 1

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7839 from dillaman/wip-14809

librbd: remove last synchronous librados calls from open/close state machine

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7785 from dillaman/wip-14822

librbd: journal replay needs to support re-executing maintenance ops

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7776 from dillaman/wip-librbd-journal-tweaks

journal: improve commit position tracking

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cmake: Turned LTTng OFF

This is temporary, until a way to stop repetitive
unecessary generation of headerfiles is figured out.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

RPM: move scriptlets from ceph to ceph-base

This addresses the following RPMLINT error:

ceph-base.x86_64: E: library-without-ldconfig-postun (Badness:
300) /usr/lib64/libosd_tp.so.1.0.0
ceph-base.x86_64: E: library-without-ldconfig-postun (Badness:
300) /usr/lib64/libos_tp.so.1.0.0
This package contains a library and provides no %postun scriptlet
containing a call to ldconfig.

ceph-base.x86_64: E: library-without-ldconfig-postin (Badness:
300) /usr/lib64/libosd_tp.so.1.0.0
ceph-base.x86_64: E: library-without-ldconfig-postin (Badness:
300) /usr/lib64/libos_tp.so.1.0.0
This package contains a library and provides no %post scriptlet
containing a call to ldconfig.

http://tracker.ceph.com/issues/14940 Fixes: #14940

Signed-off-by: Nathan Cutler <ncutler@suse.com>

AUTHORS: update email

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: calculate payload hash in RGWPutObj_ObjStore only when necessary.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #7767 from athanatos/wip-sam-journal-throttle-4

osd: filestore: restructure journal and op queue throttling

Reviewed-by: Samuel Just <sjust@redhat.com>

mds: deny access to file with pool_ns layout if feature is missing

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: fix inodestat size calculation

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: properly purge backtrace object

backtrace objects are always in default namespace

Signed-off-by: Yan, Zheng <zyan@redhat.com>

messages/MClientCaps: fix decoding

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds/Server: fix whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

mds/Locker: fix Session leak on access denial

Signed-off-by: Sage Weil <sage@redhat.com>

fix init of new fields in MCLientCaps

mds/MDSMap: add file layout v2 incompat feature

The existing MDSMonitor machinations already add this to the MDSMap to
boot out old MDS's.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: deny access to directories with pool_ns layouts if feature is missing

If the client feature is missing, given them EIO.  Change check_access to
return an error code to distinguish this from EACCES.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MDSMonitor: prevent pool 0 from being used as a data pool

Pool 0 means no change or default in the legacy ceph_file_layout in the
layout ioctl and file create arguments. Prevent it from being used to avoid
putting users in an awkward situation later.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: don't break compatibility of MClientCaps/MClientReply

Signed-off-by: Yan, Zheng <zyan@redhat.com>

fs_types: file_layout_t: convert pool -1 (undefined) to 0 in legacy encoding

Old code assumes that fl_pg_pool == 0 means the pool is not defined, while
file_layout_t uses -1.  Translate between the two.

Note that this means a valid file_layout_t with pool_id == 0 cannot be
accurately translated to a legacy file_layout_t.  That is somewhat
unavoidable, and should not be a problem since real clusters create 'rbd'
as pool 0 and it does not use any file layouts.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/fs/misc/layout_vxattrs: test layout.pool_namespace

Signed-off-by: Sage Weil <sage@redhat.com>

client: expose layout.pool_namespace vxattr

Signed-off-by: Sage Weil <sage@redhat.com>

mds: allow adjustment of layout.pool_namespace vxattr

Signed-off-by: Sage Weil <sage@redhat.com>

struct ceph_file_layout -> file_layout_t

- drop the global
- do not memset!
- encode with features
- field names are different
- use get_period() method where appropriate
- fix is layout empty checks

Signed-off-by: Sage Weil <sage@redhat.com>

mds: make inode_t, old_inode_t featureful

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MClientCaps: avoid fixed struct for encoding

At least for new clients.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: encode inode reply information directly

Drop the ceph_mds_reply_inode struct and encode directly.  Note
that we already decode directly to InodeStat in MClientReply.h.

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MClientReply: stop using ceph_mds_reply_inode struct to unpack

Decode directly.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: pass features to various CInode encoders and helpers

We'll need this shortly.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSMap: cache up_features; fix logic

& these, not | them.

Signed-off-by: Sage Weil <sage@redhat.com>

libradosstriper: move default stripe parameters inline

The extern is going away.

Signed-off-by: Sage Weil <sage@redhat.com>

include/encoding: featureful encoder for list of shared_ptrs

Signed-off-by: Sage Weil <sage@redhat.com>

include/compact_map: featureful encoders, too

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-dencoder: TYPE_FEATUREFUL_NOCOPY

Signed-off-by: Sage Weil <sage@redhat.com>

mds: pass features to LogEvents, EMetaBlob

Signed-off-by: Sage Weil <sage@redhat.com>

mds,mon: include features in beacon messages, MDSMap

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSMap: add features to MDSMap's mds_info_t

Signed-off-by: Sage Weil <sage@redhat.com>

add fs_types.h, with file_layout_t

Signed-off-by: Sage Weil <sage@redhat.com>

declare Formatter for everyone

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/fs/misc/layout_vxattr.sh: detect data pool name

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7823 from SUSE/wip-14906

packaging: add lsb-release dependency for SUSE and Debian

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

rgw: send proper ETag value during GET on DLO/SLO through S3 API.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #7837 from rldleblanc/wpq-documentation

doc: osd-config Add Configuration Options for op queue.

Merge pull request #7851 from jupiturliu/obj-bencher-bug

rados: fix bug for write bench

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7729 from yuyuyu101/wip-spdk2

osd: bluestore: Revert NVMEDevice task cstor and refresh interface changes

packaging: lsb_release build and runtime dependency

The lsb_release executable is being run in multiple places, not least in
src/common/util.cc, which calls it via shell in the collect_sys_info() code
path.

This patch addresses this issue on SUSE- and Debian-derivatives, as well
as reinstating the dependency for RHEL/Fedora after it was dropped in
15600572265bed397fbd80bdd2b7d83a0e9bd918.

http://tracker.ceph.com/issues/14906 Fixes: #14906

Signed-off-by: Nathan Cutler <ncutler@suse.com>

buffer: use alignof for raw_combined allocation arithmetic

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: clean up raw_combined construction

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: size append_buffer so that it fits into page-multiple allocations

We drop some unittest assertions about alloc buffer size.  Sorry!

Signed-off-by: Sage Weil <sage@redhat.com>

rbd-replay: s/CEPH_BUFFER_APPEND_SIZE/CEPH_PAGE_SIZE/

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: alloc right-sized buffer from read_fd

This may as well fit the input; this doesn't relate to the
append buffer.

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: use raw_combined for certain allocations

If the alignment is on a page boundary, or the allocation is big,
a separate buffer::raw goes faster.  The rest of the time,
a raw_combined does.

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: align unspecified allocations to a word

This will let us put policy create_aligned.

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: combine data and buffer::raw into single allocation

Signed-off-by: Sage Weil <sage@redhat.com>

buffer: add front(), back(), get_num_buffers() methods

These eliminate most callers of buffers(), which exposes the
internal list<ptr>.

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bufferlist: benchmark some allocations

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bufferlist: fix append_bench

- fix source
- include larger sizes

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bufferlist: fix ptr move test

Do not assume there is a trailing null the terminate the string.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7739 from xiexingguo/xxg-wip-14387

osd: fix race condition for heartbeat_need_update

Reviewed-by: Sage Weil <sage@redhat.com>