server: avoid duplicated call to utime_t

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mds/SessionMap: avoid potential memory leak

This is safer and clean.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mds/SessionMap: explicitly require importing_count to be postive

The importing_count is of type int, thus it can go underflow and
assert(importing_count) can still return false positive while it
should not supposed to do so.

Also this can make the assert itself more sensitive, e.g., can
catch underflow errors.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: remove redundant semicolon

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: set terminating_sessions flag correctly

So caller won't do this multiple times.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9484 from jmunhoz/aws4-streaming

rgw: aws4: add STREAMING-AWS4-HMAC-SHA256-PAYLOAD support

verified (by hand)

Merge pull request #9664 from dillaman/wip-16253

rbd-mirror: do not re-use image id from mirror directory if creating image

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9588 from renhwztetecs/renhw-wip-mdsboot

mds: Ignore create_root_inode's access value

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9502 from tchaikov/wip-buffer-noexcept

common/buffer: add noexcept to ensure move ctor is used

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9658 from runsisi/ss-wip-fix-rbd-mirror-doc

doc: fix typo for rbd-mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9657 from dillaman/wip-16235

librbd: avoid holding owner_lock waiting for exclusive lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9661 from dillaman/wip-16248

qa/workunits: ensure replay has started before checking position

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: reset local image id when creating new image

Fixes: http://tracker.ceph.com/issues/16253
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: handle an attempt to delete a non-mirrored image

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: disallow setting an empty mirror image entry to disabling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9642 from yuyuyu101/wip-16237

msg/async/AsyncConnection: make verify auth callback without connection lock

Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #9213 from stiopaa1/replicatedpg_movetoforward

osd/ReplicatedPG.h: remove incorrect use of move

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6967 from majianpeng/osd-full-skip-readop

osd: bypass readonly ops when osd full.

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8913 from vumrao/wip-vumrao-15704

rados: Add cleanup message with time to rados bench output

Reviewed-by: David Zafman <dzafman@redhat.com>

qa/workunits: ensure replay has started before checking position

Fixes: http://tracker.ceph.com/issues/16248
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: fix typo for rbd-mirroring

Signed-off-by: runsisi <runsisi@zte.com.cn>

librbd: avoid holding owner_lock waiting for exclusive lock

Fixes: http://tracker.ceph.com/issues/16235
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9374 from batrick/i16024

client: use fh min readahead to enable readahead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8739 from ukernel/wip-14271

mds,client: sort dentries in hash order

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9640 from dillaman/wip-16235

librbd: recursive lock possible when disabling journaling

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9645 from trociny/rbd-mirror-teuthology-fix

qa/workunits/rbd: specify source path for rbd-mirror helper functions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mds/MDSRank: Ignore create_root_inode's access value

add (void), Ignore create_root_inode's access value

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

qa/workunits/rbd: specify source path

otherwise it looks in $PATH for the script

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8436 from dreamhost/wip-python-prefix

python: Pass prefix/sbindir from autoconf to distutils.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9364 from H3C/H3C

osd: sparse_read offset may not be zero for ecpool

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9387 from stiopaa1/osd_osd_constiter

osd/OSD.h: change iterator to const_iterator

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9508 from x11562/auth_rm

mon: fix typo of 'ceph auth rm'

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9395 from tchaikov/wip-16010

buffer: handle integer underflow in iterator::copy(large_int, dest)

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async/AsyncConnection: make verify auth callback without connection lock

Fixes: http://tracker.ceph.com/issues/16237
Signed-off-by: Haomai Wang <haomai@xsky.com>

client: move readahead ref get/put to obj constr

The readahead context was incrementing the file handle reference count but only
decreasing in finish which is not called if readahead is a no-op.

This fixes a bug caught in testing where readahead was causing inode to become
disconnected:

    2016-06-10 19:46:48.953018 7f2a4351be80  1 client.4125 dump_inode: DISCONNECTED inode 10000000502 #10000000502 ref 110000000502.head(faked_ino=2307 ref=1 ll_ref=0 cap_refs={1024=0,2048=0,4096=0,8192=0} open={1=0,2=0} mode=100666 size=4194304/0 mtime=2016-06-10 19:29:45.107417 caps=-(0=pAsLsXsFscr) objectset[10000000502 ts 2/4012653 objects 0 dirty_or_tx 0] 0x7f2a24300d00)
    2016-06-10 19:46:48.953032 7f2a4351be80  2 client.4125 cache still has 0+155 items, waiting (for caps to release?)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #9637 from stiopaa1/osdc_ObjectCacher_deleteCopyAssign

osdc/ObjectCacher.h: delete copy constr and assign

Reviewed-by: Sage Weil <sage@redhat.com>

client: use fh min readahead to enable readahead

The client was using the configured maximum readahead of 0 which was changed in
95ee69989129750fddce6a3b5644238c4b88ed74. This prevented readahead from ever
running (without setting a different default from 0).

Fixes: http://tracker.ceph.com/issues/16024
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

client: update comment

Default was changed to unlimited in 11e4680e7394e6e02612441c964e05207b97737b.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

client: use layout.get_period() helper

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #9593 from Yan-waller/yj-hwip-rgwbasictype

rgw/basic_type: use the standard usage of string::npos

verified

osdc/ObjectCacher.h: delete copy constr and assign

Delete copy constructor and assignment operator, so the checking can
be moved from linking to compile time - as it was when the above
where only declared, but not defined.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9636 from linuxbox2/xio-entity

msg/xio: fix entity_addr_t encoding

Merge pull request #9625 from ceph/ldap-build

Allow building RGW with LDAP disabled

validated

librbd: recursive lock possible when disabling journaling

If pool-level mirroring is enabled, deleting the journal
could cause a deadlock attempting to delete remote peer
sync-point snapshots.

Fixes: http://tracker.ceph.com/issues/16235
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

xio: fix build (entity_addr_t encoding)

Currently XioMessenger encodes an entity_addr_t in its wire header.
The new 'features' argument to entity_addr_t::encode should have
the actual XioMessenger::local_features value when actually encoding
(and some value otherwise).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

cmake:  fix mis-matched arguments to if warning

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #9534 from shehbazj/master

os/bluestore: benchmark Host Aware SMR Drives for linear writes

Merge pull request #9615 from stiopaa1/osdc_objectCacher_removeUnneededPublic

osdc/ObjectCacher.h: remove public access specifier

Reviewed-by: Sage Weil <sage@redhat.com>

Allow building RGW with LDAP disabled

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #8718 from stiopaa1/avoid_copying_for_loop

rgw/rgw_op.cc: avoid copying in for loops

I'm going with, if it compiles, it must be correct (because const).

Merge pull request #9622 from ceph/wip-ceph-disk-test

cmake, test: enable ceph-disk.sh to run with cmake

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #9515 from liewegas/wip-kraken-defaults

default to async messenger, wpq

Reviewed-by: Samuel Just <sjust@redhat.com>

test: ceph-disk.sh: use ceph-helper.sh helpers

so we can use the utlitiy functions offered by ceph-helper.sh, and enable
cmake to run this test.

* ceph-disk/tests/ceph-disk.sh: it checks ../ for the executables for which
  fixtures are set up. but this assumption does not hold when we are using
  cmake for out-of-source build. so we need to pass the build directory to
  ceph-disk.sh. ${CEPH_BIN} is used for this purpose in this change, it is
  ".." by default, so this change is backward compatible. and one can run
  tox without setting up the ${CEPH_BIN} env variable.
* ceph-disk/test-requirements.txt: bump up tox to 2.0+
  in the case of "make check", both cmake and automake pass ${CEPH_BIN} env
  variables to the "make check" tests, so we can simpy pass it down via tox.ini's
  "passenv" setting. per
  https://testrun.org/tox/latest/example/basic.html#passing-down-environment-variables,
  this feature was introduce in tox 2.0. so bump up the minimum required
  tox version to 2.0 in ceph-disk/test-requirements.txt.
* ceph-disk/tox.ini: pass CEPH_BIN, CEPH_BIN, and CEPH_ROOT down to ceph-disk.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: respect CEPH_*_VIRTUALENV when running run-tox-ceph-*

so ceph-{detect-init,disk}/run-tox.sh is able to pick it up

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9621 from dillaman/wip-16229

rbd-mirror: do not propagate deletions when pool unavailable

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9619 from dillaman/wip-16197

test: improve rbd-mirror test case coverage

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9616 from liewegas/wip-osd-markdown-test

test/osd/osd-markdown: make test more reliable

Reviewed-by: Loic Dachary <ldachary@redhat.com>

qa/workunits/rbd: additional rbd-mirror stress tests

Fixes: http://tracker.ceph.com/issues/16197
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

vstart: add --nolockdep option

rbd-mirror stress test cases need a way to disable lockdep when
initiated in a vstart environment.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9618 from dillaman/wip-16226

rbd-mirror: fix deletion propagation edge cases

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9620 from trociny/rbd-mirror-error-formatting

rbd-mirror: fix error message formatting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9464 from vumrao/wip-vumrao-15807

rbd: add error message "snapshot is already protected"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: do not propagate deletions when pool unavailable

Fixes: http://tracker.ceph.com/issues/16229
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9420 from weiqiaomiao/wqm-wip-rgw-fixmemleak

rgw:fix data sync memory leaks

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd-mirror: fix error messages formatting

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8386 from ukernel/wip-monntbyfsname

client: mount non-default filesystem by name

Reviewed-by: John Spray <john.spray@redhat.com>

rbd-mirror: image deleter properly handles multiple snapshots

Fixes: http://tracker.ceph.com/issues/16226
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: propagate deletions even if image replayer is stopped

If an image deletion which causes the replayer to stop (due to an error)
before the deletion is detected, the deletion should still occur.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: improve deletion propagation test case

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

messages: fix MFSMap::get_type_name()

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: mount non-default filesystem by name

To mount non-default filesytem, user needs to provide mds namespace ID.
This is inconvenience.

This patch makes user be able to mount filesystem by name. To do this,
client first subscribes to FSMapUser. Subscribe to mdsmap.<ID> after
knowning ID of the filesystem.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

MDSMonitor: handle fsmap.user subscription

return FSMap data that are visiable to normal users.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

messages: add MFSMapUser

For mon clients that subscribe to 'fsmap.user'

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: add FSMapUser

FSMapUser is compact version of FSMap, it includes FSMap data that are
visiable to normal users.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9617 from liewegas/wip-fix-ec-min-size-test

test/mon/misc: fix pool set test given new default ec k

Reviewed-by: Loic Dachary <ldachary@redhat.com>

test/mon/misc: fix pool set test given new default ec k

k is no longer min_size by default; adjust test accordingly.

Signed-off-by: Sage Weil <sage@redhat.com>

test/osd/osd-markdown: make test more reliable

This can fail if the osd hasn't gotten the map for the
last time it was marked down and come back up.  This
should make this test much more reliable (fewer false
positives).

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/ObjectCacher.h: remove public access specifier

This is duplicated 3 times, so can be removed.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9594 from dillaman/wip-16165

rbd-mirror: fix journal shut down ordering

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9598 from dillaman/wip-16196

rbd-mirror: refresh image after creating sync point

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9433 from jbernard/master

librbd: refresh image if needed in mirror functions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9538 from chhabaramesh/master

os/bluestore: Unaligned device length and Correct alloc size fix for BitMap allocator

Merge pull request #9600 from stiopaa1/osdc_objectcacher_moveCwritecommit

osdc/ObjectCacher.h: move classes to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9596 from liewegas/wip-statfs

os: fix statfs for filestore and fusestore

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9597 from dillaman/wip-cls_rbd-encoding

cls_rbd: encode entity_addr_t with features

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8779 from ukernel/wip-cephfs-rstat

mds: fix rstat bugs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8780 from ukernel/wip-renew-caps

client: renew caps for read/write if mds session got killed.

Reviewed-by: John Spray <john.spray@redhat.com>

correct allocation block size for bluestore and bluefs

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

remove truncation of unaligned blocks at init

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #9443 from vishalkanaujia/master

mds: Removed tmap fetch code from Cdir

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9536 from xiexingguo/xxg-wip-fix-prdwrv

client: fix sanity check of preadv/pwritev

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9523 from liewegas/wip-pg-to-osds

osd/OSDMap: pg_to_osds -> pg_to_raw_osds, fix libcephfs caller to get acting osds

Reviewed-by: Song Baisen <song.baisen@zte.com.cn>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9507 from xiexingguo/xxg-wip-fix-mdsutility

tools/cephfs: fix message leak; add flush for dump cmd

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8890 from ukernel/wip-15634

client: enable fuse_use_invalidate_cb by default

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9509 from ukernel/wip-16137

client: skip executing async invalidates while umounting

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9541 from renhwztetecs/renhw-wip-rbd

rbd: cleanup unnecessary switch breaks

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osdc/ObjectCacher.h: move classes to cc file

Move C_WriteCommit and C_WaitForWrite to cc file.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9595 from tchaikov/wip-librados-do-not-return-local-ref

librados: get_pool_name() should not return ref of a local variable

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

rbd-mirror: refresh image after creating sync point snapshot

Fixes: http://tracker.ceph.com/issues/16196
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: consistently use dout/derr for debug logging

Signed-off-by: Jason Dillaman <dillaman@redhat.com>