common: fix the macros for malformed_input::what()

the thrown exception of malformed_input should carry the function name in
which it was thrown. but what we have now is something like:
"buffer::malformed_input: __PRETTY_FUNCTION__ unknown encoding version >
100"
because __PRETTY_FUNCTION__ is not a macro any more. see
- https://gcc.gnu.org/onlinedocs/gcc-3.1/gcc/Function-Names.html
- https://gcc.gnu.org/onlinedocs/gcc/Function-Names.html
and it is not a string literal, so neither can we can concat it with
the literal err message.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4656 from ceph/wip-rbd-xfstests-201505

RBD: expunge test broken by xfs commit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4655 from ceph/wip-add-ceph-create-keys.rst

doc: add ceph-create-keys.rst to dist tarball

Reviewed-by: Loic Dachary <ldachary@redhat.com>

RBD: expunge test broken by xfs commit

expunge generic/231, broken for disk and rbd by kernel commit 04162bb

Signed-off-by: Douglas Fuller <dfuller@redhat.com>

doc: add ceph-create-keys.rst to dist tarball

Fixes: #10725
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4412 from tchaikov/wip-add-man-page

doc: add man page for ceph-create-keys

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4522 from ceph/wip-cephfs-reset-fixes

Fixes to resets of tables and journal with damaged metadata

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tools: fix tabletool reset of nonexistent sessionmap

If the object didn't exist, the omap clear was failing
and preventing the subsueent omap set header from
executing.  Set the FAILOK flag on the omap clear
sub-operation.

Signed-off-by: John Spray <john.spray@redhat.com>

tools: improve journaltool reset

Handle the case where no journal objects exist at all,
rather than just resetting a journal that already exists.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: expose static default file layout methods

So that tools can invoke these to get the defaults
without a fully baked MDCache.

Signed-off-by: John Spray <john.spray@redhat.com>

tools: fix tabletool reset snap

SnapServer has an encode method defined that
is different to encode_state, whereas in InoTable
the two were synonmous.  This code was working
previously for inotable but not for snapserver.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #4652 from ceph/wip-11541

mds: clear CDir::STATE_REJOINUNDEF after fetching dirfrag

Reviewed-by: John Spray <john.spray@redhat.com>

mds: clear CDir::STATE_REJOINUNDEF after fetching dirfrag

Fixes: #11541
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #4545 from dachary/wip-mailmap

mailmap updates

mailmap: Jean-Charles Lopez affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Hervé Rousseau affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Rajesh Nambiar affiliation

Reviewed-by: Rajesh Nambiar <rajesh.n@msystechnologies.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Douglas Fuller affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Orit Wasserman affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Nathan Cutler affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Tim Serong affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Fabio Alessandro Locati affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Henry Chang affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Gerhard Muntingh affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Alistair Israel affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Sergey Arkhipov affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Neha Ummareddy affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Xingyi Wu affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Ning Yao affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Joao Eduardo Luis affiliation

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4525 from athanatos/wip-11358

Wip 11358

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4648 from athanatos/wip-11439

Add separate suicide timeout configs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4365 from ceph/wip-digest-repair

osd: fix repair when recorded digest is wrong

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #4252 from wonzhq/make_writeable

osd: fixes for make_writeable

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4450 from big-henry/wip-journal

FileJournal: fix check_for_full

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4293 from ajs1984/master

librados: Add config observer for Objecter instance.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4352 from majianpeng/misc-fix

Misc fix

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #4452 from tchaikov/wip-interactive-ceph-cli

Wip interactive ceph cli

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4494 from ceph/wip-int-option-overflow

Wip int option overflow

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4645 from ceph/wip-fix-BTBFS-message-monmetadata

mon: fix the FTBFS

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

mon: fix the FTBFS

put the messages/MMonMetadata.h into the dist tarball.

Fixes: #10904
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4626 from trhoden/bootstrap-rgw

debian: add bootstrap-rgw dir to ceph package

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #4609 from trhoden/ceph-spec-rgw

Add rgw placeholder directories to ceph-radosgw package

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4227 from tchaikov/node-ls

mon,osd,mds: add "{mon,mds} metadata" and "node ls" commands

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #4551 from theanalyst/doc/release/v0.94.2

doc: release notes for hammer v0.94.2 (draft)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: release notes for hammer v0.94.2

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
Conflicts:
doc/release-notes.rst

Merge pull request #4624 from rajukv/wip-xio-rajukv

xio: Update XIO client connection IP and nonce

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4620 from yuyuyu101/wip-async-fix-10

AsyncConnection: Avoid assert since replacing may inject when unlock

Update XIO client connection IP and nonce

Obtain the local IP of the client and save the nonce provided when the messenger was created. This is required for RBD lock/unlock
Fix script error in RBD concurrent test
Reset did_bind during messenger shutdown

Signed-off-by: Raju Kurunkad <raju.kurunkad@sandisk.com>

Merge pull request #4640 from SUSE/wip-rbd-recover-tool-typo-fix

doc: rbd-recover-tool: fix typos in comments

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4637 from SUSE/wip-rados-gw-comment-fix

doc: rgw: fix typo in comments

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: rbd-recover-tool: fix typos in comments

Signed-off-by: Nathan Cutler <ncutler@suse.cz>

doc: rgw: fix typo in comments

Signed-off-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #4628 from dachary/wip-releases

doc: add release numbering explanation

doc: add release numbering explanation

Mostly copy pasted from Sage's email "The first infernalis dev release
will be v9.0.0".

Signed-off-by: Loic Dachary <ldachary@redhat.com>

doc: move the release cycle after the timeline

The timeline is a useful reference and the reader should not have to
scroll down to see it. Move the release cycle after the timeline.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4541 from jbernard/master

common/admin_socket: close socket descriptor in destructor

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4623 from ceph/wip-11577

librbd: possible deadlock during copyup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

debian: add bootstrap-rgw dir to ceph package

Signed-off-by: Travis Rhoden <trhoden@redhat.com>

packaging: Add rgw placeholder dirs in proper packages

Automatically create /var/lib/ceph/radosgw with the
ceph-radosgw package, and /var/lib/ceph/bootstrap-rgw
via the ceph package.

Signed-off-by: Travis Rhoden <trhoden@redhat.com>

librbd: object map clean state valid only for fast-diff

The deep-flatten feature bit is incorrectly used to determine
whether to set the object map to clean.  It should use the
fast-diff feature bit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/admin_socket: close socket descriptor in destructor

Long-running processes that do not reuse a single client connection will
see accumulating file descriptors as a result of not closing the
listening socket.  In this case, eventually the system will reach
file-max and subsequent connections will fail.

Fixes: #11535
Signed-off-by: Jon Bernard <jbernard@tuxion.com>

AsyncConnection: Avoid assert since replacing may inject when unlock

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4621 from dachary/wip-pip-timeout

install-deps.sh: increase pip install timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4601 from oritwas/nss-cmake

cmake fixes

install-deps.sh: increase pip install timeout

It is not uncommon for pip install to timeout with:

   Cannot fetch index base URL https://pypi.python.org/simple

because the default timeout is 15 seconds. When running from a CI, it
translates into bursts of false negatives when the network is saturated.

Increase the timeout to 10 minutes which is hopefully large enough to
only happen when there is a serious network problem.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4588 from dachary/wip-11398-kill-daemons

tests: ceph-helpers kill_daemons fails when kill fails

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4616 from ceph/wip-fix-FTBFS-boost-1.58

mon: remove unused variable

Reviewed-by: Loic Dachary <ldachary@redhat.com>

cmake: add common_utf8 lib to link unittest_str_map

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake: Add missing pthread lib

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake add mds and librbd missing files

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake: add crc assembley files

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #4615 from dachary/wip-11575-pip-log

tests: pip must not log in $HOME/.pip

Reviewed-by: Kefu Chai <kchai@redhat.com>

tests: ceph-helpers kill_daemons fails when kill fails

Instead of silently leaving the daemons running, it returns failure so
the caller can decide what to do with this situation. The timeout is
also extended to minutes instead of seconds to gracefully handle the
rare situations when a machine is extra slow for some reason.

http://tracker.ceph.com/issues/11398 Fixes: #11398

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mon: remove unused variable

* as a side effect, this change silences
  http://tracker.ceph.com/issues/11576

Fixes: #11576
Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: exit on error if dependencies cannot be installed

Now that pre-installing pip dependencies is done at the end of the
script, the last command to run is no longer the installation
command. Therefore the status of the script is no longer the status of
the install command and no longer reflect success or failure to install
the dependencies. Add explicit || exit 1 to commands that are to be
treated as fatal errors.

Also set -e so that another error has a better chance to be caught.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: pip must not log in $HOME/.pip

Because it may not have permission to when running in a container and
scripts run from source are not expected to modify anything outside of
the source tree anyway.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #4614 from ceph/wip-11574-fix-FTBFS

json_sprit: fix the FTBFS on old gcc

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

json_sprit: fix the FTBFS on old gcc

Fixes: #11574
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4611 from yuyuyu101/wip-async-fix-9

AsyncConnection: verify connection's state is expected

AsyncConnection: verify connection's state is expected

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4604 from ceph/wip-11568

#11568: fix error handling in check_pool_perm

Merge pull request #4608 from ceph/wip_11547

Added a "ceph hello world" for a simple check for ceph-deploy qa suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Added a "ceph hello world" for a simple check for ceph-deploy qa suite

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #4520 from ceph/wip-mds-damage-handling

MDS: improve handling of damaged metadata

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4607 from dachary/wip-11532-limits

tests: need 1024 files per process not 100024

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tests: need 1024 files per process not 100024

Signed-off-by: Loic Dachary <ldachary@redhat.com>

librbd: ObjectMap::aio_update can acquire snap_lock out-of-order

Detected during an fsx run where a refresh and CoR were occurring
concurrently.  The refresh held the snap_lock and was waiting on
the object_map_lock, while the CoR held object_map_lock and was
waiting for snap_lock.

Fixes: #11577
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #4605 from dachary/wip-11532-limits

tests: fail make check if nproc is too low

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tests: fail make check if nproc is too low

When running tests in parallel with make -jX, the ulimit -u (number of
processor / thread per user) needs to be at least X * 1024. If not it
will fail in mysterious ways. Since there is no convenient way to figure
out the value of X ( see
http://blog.jgc.org/2015/03/gnu-make-insanity-finding-value-of-j.html
for a non trivial an entertaining solution) add a very conservative
check that assumes the user will run make -jX where X is nproc / 2.

It will be annoying for users who want to run make check, not use -j,
and have a low ulimit -u. But the error suggest a way to override this
with

   make CHECK_ULIMIT=false check

This is a minor irritation compared to the puzzling behavior of make
check when ulimit is exceeded.

http://tracker.ceph.com/issues/11532 Fixes: #11532

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4599 from dachary/wip-releases

release cycle / timeline updates

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4532 from dachary/wip-init-system-detection

ceph-detect-init helper and associated tests

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

client: fix error handling in check_pool_perm

Previously, on an error such as a pool not existing,
the caller doing the check would error out, but
anyone waiting on waiting_for_pool_perm would
block indefinitely (symptom was that reads on a
file with a bogus layout would block forever).

Fix by triggering the wait list on errors and
clear the CHECKING state so that the other callers
also perform the check and find the error.

Additionally, don't return the RADOS error code
up to filesystem users, because it can be
misleading.  For example, nonexistent pool is
ENOENT, but we shouldn't give ENOENT on IO
to a file which does exist, we should give EIO.

Signed-off-by: John Spray <john.spray@redhat.com>

client: use SaferCond in check_pool_perm

Just because it's easier to read.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #4543 from yuyuyu101/wip-async-fix-8

Wip async fix 8

Merge pull request #4003 from ceph/wip-fix-cmake-nss

Fix nss for cmake

Merge pull request #3743 from trociny/wip-osd_create

mon: osd create: add optional 'id' parameter

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #4422 from ceph/wip-10154

librbd: deep-flatten of cloned images

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4603 from ceph/wip-11557

librbd: fast diff should treat all _EXISTS objects as dirty

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mon: add 'node ls {all,mds,mon,osd}' asok

Fixes: #10904
Signed-off-by: Kefu Chai <kchai@redhat.com>