doc/cephfs: rename early-adopters to best-practices

...and give it a link in the TOC (in addition
to where it is referenced from the top of index.rst)

Signed-off-by: John Spray <john.spray@redhat.com>

doc/cephfs: remove some scary warnings

...and restructure the "early adopters" page
into a "best practices" guide.  Early adopters
are now just adopters :-)

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #9891 from diluga/rgw_socket_backlog_with_sign

RGW : setting socket backlog for via ceph.conf

Merge pull request #9852 from dx9/wip-rgw-ldap-link

rgw-ldap: add ldap lib to rgw lib deps based on build config

Merge pull request #9931 from Yan-waller/yj-wip-rgwmainccfix

rgw: remove uneccessary temporary variable and fix some indents

cosmetic changes only--build verified, f23

Merge pull request #9979 from Yan-waller/yj-wip-rgwrests3cc

rgw: fix potential memory leaks and remove unnecessary assignment

lgtm--should backport

Merge pull request #10031 from yehudasa/wip-16530

rgw: finish error_repo cr in stop_spawned_services()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9985 from Yan-waller/yj-wiph-rgwopccfixstring

rgw: use string::npos while 'string::find()' find nothing

lgtm--build verified, f23

Merge pull request #9898 from liewegas/wip-bluefs-async-flush

os/bluestore/BlueFS: do not hold internal lock while waiting for IO

Merge pull request #10002 from Yan-waller/yj-wip-rgwradosccremoveunused

rgw: remove uneccessary temporary variable 'r'

lgtm--build verified, f23

Merge pull request #10067 from liewegas/wip-do-cmake

do_cmake.sh: set up initial plugin dir

os/bluestore/BlueFS: debug statements to check log_fnode size in super block

Signed-off-by: Varada Kari <varada.kari@sandisk.com>

os/bluestore/Bluefs: Unit test cases for bluefs async flush

Signed-off-by: Varada Kari <varada.kari@sandisk.com>

os/bluestore/BlueFS: Make fysnc return the status

Signed-off-by: Varada Kari <varada.kari@sandisk.com>

os/bluestore/BlueFS: make _sync_and_flush_log smarter

If we know what event we need to wait for, only wait long enough for it
to flush.  This helps the situation where another thread flushed what we
needed, and more dirty stuff was added to log_t, but we don't need to
wait for that too for our caller to be happy.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: drop lock while we flush the log

Handle cases where we have multiple racing threads trying to flush the
log by only allowing one concurrent log flush to be in progress at a time,
and behave if, after flushing, there are no more dirty records to flush.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: use existing IOContext for _write_super

We need to be careful with IOContext destruction racing with the bdev
io completion thread.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: pass unique_lock down through _fsync and log flush

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: carry lock for mkfs

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/KernelDevice: rebuild buffer if we exceed IOV_MAX

Signed-off-by: Sage Weil <sage@redhat.com>

common/buffer: assert that we do not exceed IOV_MAX in prepare_iov

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bluefs: don't silence log

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: _flush_bdev -> flush_bdev

It is safe to call without a lock.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: drop lock while waiting for user io to complete

_flush_wait is safe to call without a lock, as long as our reference is
stable.  Rename it wait_for_aio() to be more clear about what it does and
the fact that it doesn't require a lock.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: track dirty by log_seq, log_seq_stable

Note when we dirty a file, and clean it only if that seq has been
committed.  Currently this is always the case because we don't drop the
lock, but that will change shortly.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: do not dirty log file when size changes

We only need to dirty the log file if the allocate changes.  Replay is
smart enough to learn the file size as it goes.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: track log_seq_stable

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: _flush_log -> _flush_and_sync_log

Be accurate.

No functional change.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10018 from ifed01/wip-bluestore-squeeze-lextents

os/bluestore: apply "small encoding" for onode_t::extents map

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10059 from tchaikov/wip-cmake

cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #10035 from Yan-waller/yj-wip-rgwradoscc-0630

rgw: fix the missing return value and remove temporary variables

lgtm, build verified f23

cmake: install rbd-replay-many and its manpage

rpm packaging needs it explicitly.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10029 from wjwithagen/patch-1

msg/async/net_handler.cc: make it more compatible with BSDs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10030 from stiopaa1/msg_asyncmess_removeCodeDup

msg/AsyncMessenger.cc: remove code duplication

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10066 from Ved-vampir/isal_submodule

isa-l: add isa-l library as a submodule

Merge pull request #9999 from xiexingguo/xxg-wip-bluestore-2016-06-29

os/bluestore: misc fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9946 from xiexingguo/xxg-wip-bluestore-cache

os/bluestore: make trim() of 2Q cache more fine-grained

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9968 from ceph/wip-no-start-update

rpm: Do not start targets on update

Reviewed-by: Nathan Cutler <ncutler@suse.com>

cmake: add compressor_plugins as dep for libos

Signed-off-by: Sage Weil <sage@redhat.com>

common/PluginRegistry: try $plugin_dir/$foo if $plugin_dir/$type/$foo fails

This is helpful for running out of a build dir, where e.g. cmake puts
stuff in lib/* and type separated into subdirs by type.

Signed-off-by: Sage Weil <sage@redhat.com>

do_cmake.sh: create initial ceph.conf setting dirs

This is enough for unit tests to run.

Signed-off-by: Sage Weil <sage@redhat.com>

isa-l: add isa-l library as a submodule
Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>

Merge pull request #9935 from xiexingguo/xxg-wip-make-zone-configurable

os/bluestore: fix memory leak during bit_alloc testing

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #9789 from batrick/i16164

mds: add maximum fragment size constraint

Reviewed-by: John Spray <john.spray@redhat.com>

os/bluestore: check against if tail_pad is already 0

So we may skip the potential has_any_lextents() check.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

cmake: use selinux/devel/Makefile for building selinux policy

was using selinux/devel/include/Makefile, which was wrong.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: install libs into ${CMAKE_INSTALL_LIBDIR}

this address rpm build failures on amd64 platforms:

Directory not found:
/srv/autobuild-ceph/gitbuilder.git/build/rpmbuild/BUILDROOT/ceph-11.0.0-119.gcd3ee3c.x86_64/usr/lib64/rados-classes
File not found by glob:
/srv/autobuild-ceph/gitbuilder.git/build/rpmbuild/BUILDROOT/ceph-11.0.0-119.gcd3ee3c.x86_64/usr/lib64/rados-classes/*
File not found by glob:
/srv/autobuild-ceph/gitbuilder.git/build/rpmbuild/BUILDROOT/ceph-11.0.0-119.gcd3ee3c.x86_64/usr/lib64/libos_tp.so*
File not found by glob:
/srv/autobuild-ceph/gitbuilder.git/build/rpmbuild/BUILDROOT/ceph-11.0.0-119.gcd3ee3c.x86_64/usr/lib64/libosd_tp.so*

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: install systemd files

add an option "WITH_SYSTEMD", off by default

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: install ceph_test_keyvaluedb

qa task rados/objectstore/keyvaluedb.yaml uses it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9351 from Yan-waller/yj-hwip-rgwfixlisterr

rgw/rados: remove confused error printout
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

os/bluestore: remove duplicated initialization of max_alloc_size

_set_alloc_sizes() already did this.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: merge _rm_buffer() to reduce code redundancy

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: kill unused buffer state "read"

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix potential access violation

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix incorrect skipping of partial hole reading

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9700 from JiYou/fix-monitor-crush

mon: Monitor: validate prefix on handle_command()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

rgw: fix the missing return value

If `get_bucket_index_object` returns an error code (-ENOTSUP)
with ivalid `hash_type`, we should return this negative error code
that resulting in `BucketShard::init` failed as expected,
oherwise it will go wrong.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

rgw: remove uneccessary temporary variable

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

os/bluestore: make trim() of 2Q cache more fine-grained

1. count total bytes as per buffer types respectively,
   so we know the exact size of each kind of buffer.

2. evict by list actual size against expected size,
   which is more accurate.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rgw: finish error_repo cr in stop_spawned_services()

Fixes: http://tracker.ceph.com/issues/16530
Need to call finish, otherwise drain_all() wouldn't be able to
drain it.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #9993 from weiqiaomiao/wqm-wip-clean-bufferlist

rgw: remove unused bufferlist

(build verified, provably unused/not aliased)

Merge pull request #10025 from tchaikov/wip-cmake

cmake: install ceph_test_cls_rgw

Reviewed-by: Ali Maredia <amaredia@redhat.com>

rgw-ldap: add ldap lib to rgw lib deps based on build config

Fixes autotools and Cmake linking on Alpine.

Signed-off-by: John Coyle <dx9err@gmail.com>

net_handler.cc: make it more compatible with BSDs

According the Linux manual page:
POSIX.1-2001 does not require the inclusion of <sys/types.h>, and this header file is not required on Linux.  However, some historical (BSD) implementations required this header file, and portable applications are probably wise to include it.

Submitted-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #9745 from weiqiaomiao/wqm-wip-rgw-usage

rgw:add missing "--staging" to radosgw-admin usage

lgtm (build verified f23)

msg/AsyncMessenger.cc: remove code duplication

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

debian: package etc/init.d/ceph in ceph-base

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9606 from gregsfortytwo/wip-16002

ObjectCacher: fix bh_read_finish offset logic

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cmake: install ceph_test_cls_rgw

it is used by cls/test_cls_rgw.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9754 from dx9/wip-fix-rgw-include

rgw/rgw_http_client: add compat.h include for TEMP_FAILURE_RETRY

lgtm (build verified, f23)

Merge pull request #9994 from xiexingguo/xxg-wip-kill-client-warn

client: kill compiling warning

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9831 from linuxbox2/wip-ssl3

cmake: rework NSS and SSL

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9807 from Yan-waller/yj-wip-rgwtoolsadderrnostr

rgw: ext_mime_map_init add string describing for error number

lgtm  (build verified)

automake, ssl:  real openssl detection, fix ssl linkage w/NSS

This change mainly intends to prevent linking with libssl when the
crypto provider is NSS, which provides an SSL implementation (the
implementation we must use and prefer when NSS is selected).

Secondarily, actually detect openssl when it is selected, which
happens by default and co-selects with cryptopp.  To do this, we
import the ax_check_openssl.m4 from the automake archive.

Since upstream builds now prefer NSS crypto, make this the default.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

cmake: rework NSS and SSL

The cmake build should select NSS or CryptoPP consistently with
current autoconf logic, as used in our RHEL builds (-DWITH_NSS),
and the SSL linkage must match (for NSS, -lssl3).

With this change, there is no direct linkage with libssl when NSS
is selected (I still see indirect linkage via libcurl, in turn
via libssh2, looking into that).

With guidance from upstream and downstream, NSS builds are now
selected by default.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #9851 from cbodley/wip-rgw-tombstone-pgver

rgw: add pg_ver to tombstone_cache

Merge pull request #9995 from tchaikov/wip-cmake

cmake changes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cmake: build install libcls_numops

test_cls_numops uses it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: install ceph_test_filestore_idempotent_sequence

run_seed_to.sh in task "filestore_idempotent" uses it

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: use full path for substitution and definitions

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: use CMAKE_POSITION_INDEPENDENT_CODE property instead of -fPIC

use the global setting of CMAKE_POSITION_INDEPENDENT_CODE instead of
hardwiring the CMAKE_C_FLAGS with -fPIC. this helps to reduce the
overhead of symbol resolution when user is using static libraries.
see
https://cmake.org/cmake/help/v2.8.11/cmake.html#variable:CMAKE_POSITION_INDEPENDENT_CODE

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: move librados.cc out of libradosstriper

NObjectIterator::__EndObjectIterator is a static member variable
whose destructor deletes a pointer. librados.cc is the source file that
defines __EndObjectIterator. before this change rados cli is linked
against libradosstriper and librados. both of them include librados.cc.
that's why the dtor is called twice, hence double free when the dso
is closed and the static variables are destructed. in this change,
the librados.cc is moved out of libradosstriper, RadosClient.cc,
RadosXattrIter.cc and IoCtxImpl.cc kept as the OBJECT library, as
libradostriper is accessing the non-public symbols defined by them.

Fixes: http://tracker.ceph.com/issues/16504
Signed-off-by: Kefu Chai <kchai@redhat.com>

run-make-check.sh: modprobe won't work on containers; not needed

Signed-off-by: Dan Mick <dan.mick@redhat.com>

debian/rules: specify sysconf dir explicitly

Signed-off-by: Sage Weil <sage@redhat.com>

CMakeLists: nss only, ignore cryptopp

Signed-off-by: Sage Weil <sage@redhat.com>

debian/rules: build deb with cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge branch 'kraken'

os/bluestore: no need to use SIGNED varint encoding for bluestore_extent_ref_map_t offsets

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: apply small encoding for bluestore_onode_t:extents

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

debian/rules: package in a more debhelper way

this helps to keep the build rule simpler, and easier to customize

* -j$(NUMJOBS) is taken care of by 'dh --parallel'
* use 'autoreconf' dh add-on to autoconf cleanup
* add dh-autoreconf to Build-Depends
* bump debhelper compatibility level to 8, as jewel's supported
  debian based distro is jessie (with debhelper 9.20150101),
  ubuntu trusty (with debhelper 9.20131227ubuntu1)

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: remove uneccessary temporary variable 'r'

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

client: kill compiling warning

The 'done' lable is not appliable any more.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rgw: remove unused bufferlist

Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>

Merge pull request #9992 from yuyuyu101/wip-fix-rebase

msg/async: add "queue_reset" argument

this fixes the broken build

Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async: add "queue_reset" argument

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #9991 from stiopaa1/msg_asyncmess_moveCProcessorAccept_class

msg/AsyncMessenger: move C_processor_accept class

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #9783 from yuyuyu101/wip-16366

msg/async: Support close idle connection feature

os/bluestore: promote output to derr if out of memory

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: make block array of test_bmap_alloc self-adapting

So we are be able change total_blocks or zone_size flexibly and
thus append more test cases.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix memory leak during bit alloc ut

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>