librados: add timeout parameter to rados_watch

Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>

osd: add comfigurable timeout for OSD_OP_WATCH

Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>

osd: add check of ceph_osd_op for compatibility

Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>

Merge pull request #11173 from tchaikov/wip-17306

test/cli/crushtool: fix the test of compile-decompile-recompile.t

Reviewed-by: Samuel Just <sjust@redhat.com>

test/cli/crushtool: fix the test of compile-decompile-recompile.t

should read the map from $TESTDIR,
it's a regression introduced by b2c0a07

Fixes: http://tracker.ceph.com/issues/17306
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11166 from xiexingguo/xxg-wip-bluestore-0921

os/bluestore: simplify calculation of collection key range

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11131 from ceph/rgw-lifecycle-testing

Rgw lifecycle testing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Lifecycle testing

Add new config option - rgw_lc_debug_interval - that modifies the
lifecycle algorithm.  It treats days as being the configured number of
seconds.  This allows lifecycle to be tested in minutes, rather than
days.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Lifecycle - Fix config locking

The lifecycle config lock was not setting a cookie.  This means that all
instances of that lock conflict, causing errors to be returned to the
client.

Set a unique cookie on each lock instance.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #11031 from yangdongsheng/du

rbd: return error if we specified a wrong image name for rbd du

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11163 from jcsp/wip-cmake-vstart

doc: Update developer docs for cmake paths

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

doc: update vstart guide for cmake-ish paths

Signed-off-by: John Spray <john.spray@redhat.com>

doc: update build instructions for cmake

Signed-off-by: John Spray <john.spray@redhat.com>

os/bluestore: simplify calculation of collection key range

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11083 from ifed01/wip-load-plugins-fix

common/compressor: add libcommon as a dependency for zlib and snappy p…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10975 from stiopaa1/osd_replicatedbackend_moveSomeClassesToCC

osd/PGBackend.h: move structs to .cc file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11073 from badone/wip-remove-check_disk_write_cache

FileJournal: Remove obsolete _check_disk_write_cache function

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #11060 from stiopaa1/mon_osdmonitor_removeAssign

mon/OSDMonitor.cc: remove use of boost assign

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11039 from stiopaa1/osd_inprogress_removingCount

osd/OSD.cc: remove unneeded searching of maps

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge branch 'wip-rocksdb-librados-test'

qa: add test of envlibrados for rocksdb

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

add qa test of envlibrados for rocksdb

Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>

Merge pull request #11146 from tchaikov/wip-17307

os/filestore/FileJournal: error out if FileJournal is not a file

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #11103 from liewegas/wip-clone-range

ceph_test_objectstore: test clone_range and fix a few bugs

ceph_test_objectstore: disable filestore_fiemap

This very reliably triggers a test failure for
ObjectStore/StoreTest.Synthetic/1.

FIEMAP is bad!  Do not use it!

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: exercise clone_range from synthetic

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: fix bug in BufferlistObject::write

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: clarify clone_range semantics

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11157 from cbodley/wip-memstore-clone

memstore: clone zero-fills holes from source range

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10991 from weiqiaomiao/wqm-wip-sync-error

rgw: fix failed to create bucket if a non-master zonegroup has a single zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10942 from JellevdK/master

systemd: add install section to rbdmap.service file

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: add tests for memstore clone_range

Signed-off-by: Casey Bodley <cbodley@redhat.com>

memstore: clone zero-fills holes from source range

Signed-off-by: Casey Bodley <cbodley@redhat.com>

memstore: use const_iterator instead of const_cast

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11070 from VictorDenisov/consistency_groups_extract_module

librbd: extract group module from librbd/internal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9844 from rzarzynski/wip-rgw-swift-website

rgw: add support for Static Website of Swift API

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11149 from xiexingguo/xxg-wip-bluestore-0920

os/bluestore: narrow condition of sanity check when get_object_key()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11121 from stiopaa1/osd_osd_moveMembersToPrivate

osd/OSD.h: move some members under private

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10563 from stiopaa1/mon_healthmonitor_removeUnneededInclude

mon/HealthMonitor: remove unneeded include

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10515 from stiopaa1/mon_monitor_addOverride

mon/Monitor.h: add override to virtual functions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10333 from stiopaa1/msg_EventSelect_RemoveIncFromHeader

msg/EventSelect: move includes to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10269 from stiopaa1/osd_watch_removeUnneededForwardDecl

osd/Watch.h: remove unneeded forward declaration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9644 from stiopaa1/osdc_ObjectCacher_const

osdc/ObjectCacher: change iterator to const_iterator and add const to member functions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11093 from ceph/wip-jlayton-statx

Small interface cleanups for struct ceph_statx

client: always provide the file type in stx_mode

The proposed statx interface does this, and this would allow ganesha
to get away without requesting CEPH_STATX_MODE when all it wants is
enough to instantiate a filehandle.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: remove unimplemented fields from ceph_statx

We don't provide stx_version and stx_information is currently unused.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Revert "client: move the device bitshift handling macros to Client.h"

This reverts commit 2115de04a417e6df4272fc836829b70bd6a2b97e.

This is unnecessary now that we're using dev_t's in ceph_statx.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: convert ceph_statx to use dev_t for devices

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: switch ceph_statx time fields to struct timespec

This is just much more convenient for applications that want to use
the API. We're not talking to the kernel here, so there's no real
reason we need to use fixed length fields.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: remove unneeded extra padding from ceph_statx

Holdover from the proposed kernel API that doesn't really help anything
here. If we ever want to extend this struct, we'll just append to the
end and add new want mask bits.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

os/filestore/FileJournal: fail out if FileJournal is not block device or regular file

otherwise JournalingFileStore will assert when deleting FileJournal
which still has the non block/regular file opened.

Fixes: http://tracker.ceph.com/issues/17307
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11144 from tchaikov/wip-17306

crush/CrushCompiler: error out as long as parse fails

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

crush/CrushCompiler: error out as long as parse fails

do not output compiled crush map if anything goes wrong when parsing
crush map.

Fixes: http://tracker.ceph.com/issues/17306
Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: narrow condition of sanity check when get_object_key()

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11129 from xiexingguo/xxg-wip-bluestore-0919

os/bluestore: misc. fixes

Reviewed-by: Sage Weil <sage@redhat.com.

Merge pull request #11116 from badone/wip-rm-modprobe-run-cmake-check

run-cmake-check.sh: Remove redundant calls

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #10943 from jdurgin/wip-tox-coverage

tox.ini: remove extraneous coverage --omit option

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #11006 from iain-buclaw-sociomantic/pyaiostat

python-rados: implement new aio_stat.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

fix test_rgw_ldap.cc for search filter

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge branch 'ldap-auth-custom-searchfilter' of https://github.com/haklein/ceph into haklein-ldap-auth-custom-searchfilter

mon/Monitor.h: add override to virtual functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

mon/HealthMonitor: remove unneeded include

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #11134 from tserlin/wip-tserlin-fix-radosgw-admin-manpage

Add two options to radosgw-admin.rst manpage

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Add two options to radosgw-admin.rst manpage

Add '--job-id' and '--max-concurrent-ios' to Orphan Search Options

Fixes: http://tracker.ceph.com/issues/17281
Signed-off-by: Thomas Serlin <tserlin@redhat.com>

Merge pull request #11051 from prallabh/wip-17256

rgw: Do not archive metadata by default

Reviewed-by: Casey Bodley <cbodley@redhat.com>

client: reinstate clearing of *fhp in ll_create and don't allow fhp == NULL

The logic in ll_create relies on *fhp being zeroed out on entry into the
function, but that's no longer being done since commit e08210dda. Fix
this by reinstating that initialization.

Luckily, it looks like the two main callers (FUSE and nfs-ganesha)
already handle this safely, but the SyntheticClient does not. Move the
zeroing of the pointer into ll_create, and allow callers to pass in a
pointer to an uninitialized value. With this, we can remove the
initialization in the fuse client code as well.

Also, we can get rid of some unneeded NULL pointer checks if we don't
allow callers to pass in a NULL fhp pointer. As best I can tell, no one
does that currently and no comments over ceph_ll_create seem to claim
that this should be allowed.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #11132 from ceph/wip-17167

rgw: fix decoding of creation_time and last_update.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: fix decoding of creation_time and last_update.

Fixes: http://tracker.ceph.com/issues/17167
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11115 from petermaloney/peter-mds-docs3

doc: manual instructions to set up mds daemon

Reviewed-by: John Wilkins<jowilkin@redhat.com>

Merge pull request #11063 from ceph/wip-doc-BZ#1368528

Removed reference about pool ownership based on BZ#1368528

Reviewed-by: Samuel Just <sjust@redhat.com>

os/bluestore: simplify marking unused logic of blob tail

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11104 from dillaman/wip-17283

rbd-mirror: concurrent access of event might result in heap corruption

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11123 from batrick/ceph-create-keys-bugfix

ceph-create-keys: add missing argument comma

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10815 from ceph/wip-qa-workunits-py3

Python 3 compatibility for workunits

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: exercise handle when trying to get through throttlers

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: confirm buffer is empty before moving it into AOUT queue

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: avoid reset cache_private if we haven't to

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11100 from bassamtabbara/rocksdb-build-improvements

cmake: enable ccache for rocksdb too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11105 from badone/wip-dnf-enable-build-scripts

build scripts: Enable dnf for Fedora >= 22

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11111 from bassamtabbara/wip-tcmalloc-minimal

cmake: Support tcmalloc_minimal allocator

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix sanity check of object key length

We now encode shard-id into exactly one byte.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11109 from dmick/master

unittest_chain_xattr: account for existing xattrs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9569 from stiopaa1/osdc_objectCacher_const

osdc/ObjectCacher.h: add const to member functions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9701 from stiopaa1/osd_removeUnneededReturn

OSD.cc: remove unneeded return

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9932 from stiopaa1/test_simple_dispatcher_removeUnusedVariable

test/simple_dispatcher.cc: remove unused variable

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10071 from stiopaa1/compatset_removeUnneededInline_mc

CompatSet.h: remove unneeded inline

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10503 from stiopaa1/msg_queuestrategy_addOverrideNew

msg/QueueStrategy: add override to virtual functs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10575 from stiopaa1/mon_osdmonitor_removeUnneededCasts

mon/OSDMonitor.cc: remove unneeded casts

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10599 from stiopaa1/mon_monclient_deleteCopyConstrAndAssing

mon/MonClient.h: delete copy constr and assing op

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10618 from stiopaa1/mon_configkeyservice_passStringByConstRef

mon/ConfigKeyService: pass strings by const ref

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10639 from stiopaa1/common_trackedop_moveShardedTrackingData

common/TrackedOp: move ShardedTrackingData to .cc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10721 from stiopaa1/mon_logmonitor_moveCLogStruct

mon/LogMonitor: move C_Log struct to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10549 from stiopaa1/mon_healthmonitor_addOverride

mon/HealthMonitor: add override to virtual functs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10529 from stiopaa1/mon_paxosservice_moveClassesTocc

mon/PaxosService: move classes to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10422 from stiopaa1/msg_accepter_addOverride

msg/Accepter.h: add override to virtual function

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10416 from stiopaa1/mon_elector_moveCElectionExpire

mon/Elector:move C_ElectionExpire class to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10383 from stiopaa1/mon_monclient_moveCTickClass

mon/MonClient: move C_Tick struct to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10441 from stiopaa1/msg_accepter_moveIncludeToCC

msg/Accepter: move include to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10513 from stiopaa1/mon_monitor_moveCScrubCScrubTimeout

mon/Monitor: move C_Scrub, C_ScrubTimeout to .cc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10482 from stiopaa1/msg_faststrategy_addOverride

msg/FastStrategy.h: add override to virtual funct

Reviewed-by: Sage Weil <sage@redhat.com>

osd/OSD.h: move some members under private

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>