ceph-volume: retry when 'vagrant up' fails

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: run tests without waiting on ceph repos

This provides a new playbook that installs ceph on all nodes and
generates the ceph config. Then it will rsync ceph-volume from the
control node, or jenkins slave, to all testing vms before completing
the deployment. This means we can run tests on PRs without waiting
for repos to be built for the branch in the PR.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23719 from bstillwell/config-doc-fixes

doc: Fixed spelling errors in configuration section

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23724 from cyx1231st/test-config

test: fix compile error in test/crimson/test_config.cc

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: fix compile error in test/crimson/test_config.cc

Change to use ceph::md_config_obs_impl.

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

Merge pull request #23690 from dongbula/add-comment-in-reg-next-scrub

osd: add log when pg reg next scrub

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23692 from tchaikov/wip-silence-gcc-warnings

test,common: silence GCC warnings

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

osd: add comment when pg reg next scrub

Signed-off-by: lvshuhua <lvshuhua@cmss.chinamobile.com>

Merge pull request #23715 from bstillwell/bluestore-doc-fixes

doc: Spelling fixes in BlueStore config reference

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23711 from tspmelo/wip-26999

mgr/dashboard: Fix growing table in firefox

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #23698 from badone/wip-mark-jewel-EOL

doc/releases: Mark Jewel EOL

Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc: Fixed spelling errors in configuration section

Correct a number of spelling mistakes and word omissions in the cluster
configuration section of the docs.

Signed-off-by: Bryan Stillwell <bstillwell@godaddy.com>

doc: Spelling fixes in BlueStore config reference

Signed-off-by: Bryan Stillwell <bstillwell@godaddy.com>

Merge pull request #23376 from dzafman/wip-25108

object errors found in be_select_auth_object() aren't logged the same

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23518 from dzafman/wip-25084

osd: When possible check CRC in build_push_op() so repair can eventually stop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: Update tests for error message changes

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Use colon to separate error message from other info

Signed-off-by: David Zafman <dzafman@redhat.com>

qa: Add new message to whitelist for scrub/repair tests

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Verify cluster logging of scrub error messages

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Log shard errors in be_select_auth_object() as other errors

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Move object error check out of be_select_auth_object()

Fixes: http://tracker.ceph.com/issues/25108
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #23708 from tchaikov/wip-sudo-sysctl

run-make-check: should use sudo for running sysctl

Merge pull request #23695 from dzafman/wip-27056

test: Use pids instead of jobspecs which were wrong

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: silence -Wsign-compare warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

test,common: silence -Wunused-variable warnings

if compiled with -NDEBUG, assert() is optimised out.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix growing table in firefox

It seems Firefox 52 has some kind of problem when
rendering our table inside a fieldset.

Since we don't really require the fieldset in the monitor page,
I have removed it to "fix" this problem.

Fixes: http://tracker.ceph.com/issues/26999
Signed-off-by: Tiago Melo <tmelo@suse.com>

run-make-check: should use sudo for running sysctl

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23667 from thinkercui/bugfix

tools/rados: correct the read offset of bench

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23664 from zy751713126/delete_redundant_code

pybind/rbd: make the code more concise

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23648 from sakhinov/sakhinov-fix-bcrypt

change ceph-mgr package depency from py-bcrypt to python2-bcrypt

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23647 from wjwithagen/wjw-fix-pidfile-err

common: be more informative if set PID-file fails

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23643 from tchaikov/wip-cmake-dmclock

dmclock, cmake: sync up with ceph/dmclock, dmclock related cleanups

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #23624 from david-z/wip-fix-mgr-fs-status

mgr/status: Add standby-replay MDS ceph version

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23576 from ifed01/wip-ifed-fix-perf-counters

os/bluestore: bluestore_buffer_hit_bytes perf counter doesn't reset

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21983 from chardan/jfw-wip-libradosstriper_ultimate-final-battle

libradosstriper: conditional compile

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23689 from tchaikov/wip-run-make-check-more-aio

run-make-check: increase fs.aio-max-nr to 1048576

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>

doc/releases: Mark Jewel EOL

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

test: Use pids instead of jobspecs which were wrong

Fixes: http://tracker.ceph.com/issues/27056
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #23675 from tserlin/tserlin-require-fuse

debian/control: require fuse for ceph-fuse

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23655 from tchaikov/wip-silence-Wsign-compare-warnings

test: silence -Wsign-compare warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

run-make-check: increase fs.aio-max-nr to 1048576

the solution was suggested by Yingxin Cheng. otherwise we could have
EAGAIN returned by io_setup(2).

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23630 from wjwithagen/wjw-fix-rbd-ggate-kldload

test/rbd: rbd_ggate test improvements

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23684 from tchaikov/wip-doc-balancer

doc/rados/operations: add balancer.rst to TOC

Reviewed-by: John Spray <john.spray@redhat.com>

doc/rados/operations: add balancer.rst to TOC

this helps user to find the balancer feature, and
also addresses the FTBFS of doc like

Warning, treated as error:
/home/jenkins-build/build/workspace/ceph-pr-docs/doc/rados/operations/balancer.rst:document
isn't included in any toctree

this doc was moved from doc/mgr/balancer.rst to
doc/rados/operations/balancer.rst in
a78d600e6b368255096d6f0bbee8e53553307118

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23642 from tchaikov/wip-crimson-msgr

cmake,crimson/net: add keepalive support, and enable unittest_seastar_messenger in "make check"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

test/mon: no need to include OSD.h

what it needs is the definition of `struct statfs`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common,test: update mClockPriorityQueue to accomodate the change in dmc

* cost is required to be a positive number
* PullPriorityQueue's construct uses an enum instead of a bool for
  specifying the behavior when request breaches the limit.

Signed-off-by: Kefu Chai <kchai@redhat.com>

dmclock: pick up tag.reservation calc fix and cmake changes

see
- ceph/dmclock#58
- ceph/dmclock#60
- ceph/dmclock#61

Fixes: http://tracker.ceph.com/issues/26998
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: dmclock related cleanups

* do not include_directories() globally. use dmclock::dmclock's
  interface properties instead
* add_subdirectory(dmclock) instead of add_subdirectory(dmclock/src)
  for including its
* set dmclock_TEST instead of including the subdirectories under
  dmclock/
* objclass/class_api.cc includes PrimaryLogPG.h, which in turn includes
  indirect_intrusive_heap.h, so it'd better to compile class_api.cc as
  a part of osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #23635 into master

* refs/pull/23635/head:
mds: use monotonic clock in beacon
mds: simplify beacon init

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #21114 from colletj/colletj_rbd_image_timestamps

librbd: add rbd image access/modified timestamps

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: add image access/last modified timestamps

Add access and modify timestamps and associated tests
to RBD images.

Access (resp. modify) timestamps are updated on
read (resp. write) operations. A configurable throttling
mechanism is implemented (default to 60s).

Signed-off-by: Julien Collet <julien.collet@cern.ch>

debian/control: require fuse for ceph-fuse

This is the Ubuntu equivalent for:

https://github.com/ceph/ceph/pull/17120
http://tracker.ceph.com/issues/21057

Installing "ceph-fuse" should pull in the "fuse" package automatically.

Signed-off-by: Thomas Serlin <tserlin@redhat.com>

Merge pull request #23649 from trociny/wip-26939

librbd: journaling unable request can not be sent to remote lock owner

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23311 from Songweibin/wip-fix-du-display

tools/rbd: minor fixes for rbd du display

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23638 from Songweibin/wip-filter-user-snapshots

rbd: filter out group/trash snapshots from snap_list

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23615 from dmick/wip-ceph-mgr-deb

debian/rules: fix ceph-mgr .pyc files left behind

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23606 from ifed01/wip-ifed-stupid-get-fragmentation-fix

os/bluestore: fix assertion in StupidAllocator::get_fragmentation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23587 from xiexingguo/wip-recover-order

osd/PrimaryLogPG: optimize recover order

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23558 from liewegas/wip-mgr-always-on

mgr: keep status, balancer always on

Merge pull request #23573 from rzarzynski/wip-msg-26937

msg: entity_addr_t::parse doesn't do memset(this, 0, ...) for clean-up

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23641 from wjwithagen/wjw-fix-pthread_get_name_np

include/compat.h: make pthread_get_name_np work when available

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #23658 into master

* refs/pull/23658/head:
qa: use cephfs standard configs for kclient

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test: silence -Wsign-compare warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23657 from trociny/wip-wsign-compare

test/librbd: fix compiler -Wsign-compare warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23591 from tchaikov/wip-lockless-auth

auth,common: add lockless auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23567 from tchaikov/wip-cmake-cleanup

cmake: fix Debug build `WITH_SEASTAR=ON`

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

common: be more informative if set PID-file fails

The errors returned by fcntl(...F_SETLK...) can be:
 -  -1 for any odd error
 -  EAGAIN if locking does not work because the file is already locked.

Differentiate in these to be more informative on what is going on.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #23631 from cyx1231st/test-seastar-config

crimson/common: apply config changes also on shard.0

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/test: improve test_config with observers

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

Merge pull request #23582 from badone/wip-remove-upstart

Remove upstart files and references

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/rbd: Improve/update rbd-ggate

* check if geom_gate can be loaded before doing the actual tests

Otherwise continuing does not make sense.
Major reason for this problem is due to mismatch between
kernel and module versions.

* After FreeBSD kernevel 1200078 ggate resizing is possible

So set the flag that resizing can be tested

* Only sudo commands that really need sudo

rbd-ggate list is available in regular user mode

* be a bit more verbose during testing and list the test purpose

* list-mapped is an option in rbd-nbd, not (yet) in rbd-ggate

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

pybind/rbd: make the code more concise

Signed-off-by: Zheng Yin <zhengyin@cmss.chinamobile.com>

cmake: should use lowercase for component name in FindSanitizers.cmake

and s/undefined-behavior/undefined_behavior/ to be compatible with
seastar

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/crimson: exercise keepalive also

allow one to pass options to unittest_seastar_messenger for specifying
- verbose
- keepalive-ratio
- rounds

so we can have better coverage of crimson/messenger. and make it less
chatty unless it's asked to do so.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: add keepalive support to Connection

* do not mix buffered writes and zero-copy writes.

/root/ceph/src/seastar/include/seastar/core/iostream-impl.hh:114:
seastar::future<> seastar::output_stream::write(seastar::net::packet)
[with CharType = char]: Assertion `!_end && "Mixing buffered writes and
zero-copy writes not supported yet"' failed.
Aborting on shard 0.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add unittest_seastar_messenger to "make check"

as it's relatively stable at the moment. and as suggested by @cyx1231st
.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: use HAVE_FEATURE instead

so we can check the marker the bit's incarnation also.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rados: correct the read offset of bench

We need read with the specified offset but not 0.

Signed-off-by: Xiaofei Cui <cuixiaofei@sangfor.com.cn>

Merge pull request #23654 from cbodley/wip-crypto-assert-always

crypto: use ceph_assert_always for assertions

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22668 into master

* refs/pull/22668/head:
mds: reorganize and curate perf log counters
mds: show session load average in dump session
mds: allow request load average decay rate configurable via admin socket
mds: decay counter for tracking request load average
mds: add perf counters to track current open/stale sessions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23438 into master

* refs/pull/23438/head:
client: fix choose_target_mds for requests that do name lookup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23467 into master

* refs/pull/23467/head:
cephfs-journal-tool: enable purge_queue journal's event commands

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #23529 into master

* refs/pull/23529/head:
mon: fix idempotent FS creation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge PR #23543 into master

* refs/pull/23543/head:
tools/cephfs: make cephfs-data-scan prints the max used ino

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23584 into master

* refs/pull/23584/head:
mds: don't modify filepath when printing
mds: hold slave request refernce when dumping MDRequestImpl

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23589 into master

* refs/pull/23589/head:
mds: more description for failed authpin
mds: cleanup CDir freezing/frozen tree check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use monotonic clock in beacon

Also update other parts of MDS which interact.

Fixes: http://tracker.ceph.com/issues/26959
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use cephfs standard configs for kclient

Specifically, qa/cephfs/begin.yaml.

Fixes: http://tracker.ceph.com/issues/26967
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

test/librbd: fix compiler -Wsign-compare warnings

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23315 from Devp00l/wip-clean-notification-messages

mgr/dashboard: Cleaner notifications

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #23637 from noahdesu/qa-mgr-mod-path

qa: fix manager module paths

crypto: use ceph_assert_always for assertions

use ceph_assert_always() to enforce this comment in ceph_crypto.h:

This assert MUST NOT be compiled out, even on non-debug builds.

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23639 from trociny/wip-24910

librbd: potential race on image create request complete

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rpm: change ceph-mgr package depency from py-bcrypt to python2-bcrypt

Signed-off-by: Konstantin Sakhinov <sakhinov@gmail.com>

Merge pull request #23110 from Songweibin/wip-rbd-snap-rollback

librbd: use object map when doing snap rollback

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: journaling unable request can not be sent to remote lock owner

Fixes: http://tracker.ceph.com/issues/26939
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23609 from ErwanAliasr1/evelu-ccache

run-make-check: Showing configuration before the build

crimson/common: apply config changes also on shard.0

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

Remove upstart files and references

Nautilus is not supported on any distros still running upstart so these
are redundant.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>