client: add failure injection for not advancing oldest_tid

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: include size of completed_requests in session dump

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: warn when clients are not advancing their oldest_client_tid

Fixes: #10657
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #4343 from ceph/wip-11356

client: fix uninline data funtion

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4223 from ceph/wip-11254

mds: make sure lock state not stay in XLOCK/XLOCKDONE indefinitely

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4218 from ceph/wip-11048

mds: persist completed_requests reliably

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4204 from ceph/wip-10368-mk3

mds: separate MDLog::safe_pos from journaler

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4380 from ceph/wip-nodebug-build

tests: separate check_PROGRAMS from TESTS

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: separate check_PROGRAMS from TESTS

Not all programs that should be built during 'make check' should
be executed as tests.  A new variable, check_TESTPROGRAMS, was
introduced to specify which programs should be executed as a test
during a 'make check'.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #4311 from yuyuyu101/wip-async-fix-4

Wip async fix 4

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4358 from gerhardqux/gerhardqux-nokey-exit-1

ceph-authtool: exit(1) when printing a non-existing key

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4012 from majianpeng/cache-tier-fadvise

Cache tier fadvise

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3606 from wonzhq/hitset-insert

osd: avoid inserting an op into hit set multiple times

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4378 from dachary/wip-11359-tier-tests-disable

tests: comment out unstable tiering tests

tests: comment out unstable tiering tests

So that make check does not randomly fails.

http://tracker.ceph.com/issues/11359 Refs: #11359

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4377 from liewegas/wip-git.ceph.com

use git.ceph.com

Reviewed-by: Alfredo Deza <adeza@redhat.com>

use git://git.ceph.com

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-authtool: exit(1) when printing a non-existing key

When printing a non-existing key, ceph-authtool exits with a success
value:

[root@ip-172-31-3-178 ~]# ceph-authtool /etc/ceph/ceph.mon.keyring -p -n
client.doesntexist ; echo $?
entity client.doesntexist not found
0

Expected result: 1

Signed-off-by: Gerhard Muntingh <gerhard@qux.nl>

Merge pull request #4373 from dachary/wip-releases

releases: table of estimated end of life

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

releases: table of estimated end of life

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4339 from ceph/wip-da-SCA-20150317

Various SCA and Coverity fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4367 from ceph/wip-ceph-disk-default-cluster

ceph-disk: set '--cluster=ceph' in 'prepare' subcommand

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4366 from BCLibCoop/bclibcoop/rgw-hostnames

rgw: Document & improve region hostnames

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

TestPGLog.cc: fix -Wsign-compare

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec.cc: fix -Wsign-compare

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_async_driver.cc: fix UNINIT

Fix for:

CID 1254380 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use_in_call: Using uninitialized value sa. Field
 sa.sin_zero is uninitialized when calling connect

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

psim.cc: change initialzation of VLAs

Use memset to fill integer VLAs.

Fix for:

CID 1219649 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use: Using uninitialized value primary_count[primary]

CID 1219648 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use: Using uninitialized value first_count[osds[0UL]]

CID 1219647 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use: Using uninitialized value count[osds[i]]

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

systest_runnable.h: mark copy ctor as explicit

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/Thread.h: mark copy/move ctor as explicit

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

cls_rbd.cc: reduce scope of local variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ReplicatedBackend.cc: init 'Message *commit' to NULL

Init variable with NULL to make sure it crashes in error
case (would be a bug) if NDEBUG disable the assert() call.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PG.h: mark copy ctor explicit

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

msg/Dispatcher.h: make copy/move ctor explicit

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_shared_cache.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rest_user.cc: UserQuotas init vars in ctor init list

Fix for:

[src/rgw/rgw_rest_user.cc:631]: (performance) Variable 'bucket_quota' is
 assigned in constructor body. Consider performing initialization in
 initialization list.
[src/rgw/rgw_rest_user.cc:632]: (performance) Variable 'user_quota' is
 assigned in constructor body. Consider performing initialization in
 initialization list.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

client/Client.cc: fix uninit variables

This is caused by assert(). If assert is disabled we run into use
of uninitialized variables. Return -EINVAL in case assert() is
disabled.

Fix for:

[src/client/Client.cc:7791]: (error) Uninitialized variable: lock_state
[src/client/Client.cc:7829]: (error) Uninitialized variable: lock_type

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

libcephfs.h: fix unnecessary forward declarations

This is an issue in case non-cpp code. Move the declaration of
inodeno_t and vinodeno_t structs into __cplusplus block where
they are needed.

Fix for cppcheck issue:

[src/include/cephfs/libcephfs.h:89] -> [src/include/cephfs/libcephfs.h:74]:
 The struct 'inodeno_t' forward declaration is unnecessary. Type struct is
 already declared earlier.
[src/include/cephfs/libcephfs.h:93] -> [src/include/cephfs/libcephfs.h:83]:
 The struct 'vinodeno_t' forward declaration is unnecessary. Type struct is
 already declared earlier.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: fix format qualifier in COMMIT_SNAP_ITEM

All places COMMIT_SNAP_ITEM is used the variables are
'unsigned long long'. Change the format qualifier to reflect
this and to fix:

[src/os/FileStore.cc:1341]: (warning) %lld in format string (no. 1)
 requires 'long long *' but the argument type is 'unsigned long long *'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDCache.cc: fix potential null pointer dereference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/../chain_xattr.cc: fix suspicious usage of 'sizeof'

Use sizeof(char) instead of sizeof('\0').

Fix for:

[src/test/objectstore/chain_xattr.cc:164]: (warning) Suspicious
 usage of 'sizeof' with a numeric constant as parameter.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec: fix some -Wsign-compare

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rest_log.h: remove http_ret from some RGWOp_*_list classes

Remove http_ret from child class since there is already an integer
with the same name in the parent/base class.

Fix for:

[src/rgw/rgw_rest_log.h:79] -> [src/rgw/rgw_rest.h:275]: (warning) The
 class 'RGWOp_MDLog_List' defines member variable with name 'http_ret'
 also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:99] -> [src/rgw/rgw_rest.h:275]: (warning) The
 class 'RGWOp_MDLog_Info' defines member variable with name 'http_ret'
 also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:182] -> [src/rgw/rgw_rest.h:275]: (warning) The
 class 'RGWOp_DATALog_List' defines member variable with name 'http_ret'
 also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:202] -> [src/rgw/rgw_rest.h:275]: (warning) The
 class 'RGWOp_DATALog_Info' defines member variable with name 'http_ret'
 also defined in its parent class 'RGWRESTOp'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: set '--cluster=ceph' in 'prepare' subcommand

otherwise the 'ceph' default value won't be catched by "main_prepare()",
this breaks TEST_corrupt_and_repair_replicated. seems the behaviour
was changed in python 2.7.9, so we move the default value to where
the '--cluster' argument is added instead of setting it in the
main argument parser.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: Changes in region hostname do not seem to take effect until radosgw is restarted.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>

Merge pull request #4335 from liewegas/wip-osd-df

mon: fix min/max variance for 'cpeh osd df' when osds are marked out

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: Document a lurking problem I see with subdomain/domain splitting

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>

rgw: Speed up the building of valid de-duplicated hostnames for regions.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>

doc: Document region hostnames settings.

The hostnames key in region settings was undocumented. Also includes how
the setting interacts with the ``rgw dns name`` configuration setting.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>

Merge pull request #3481 from ktdreyer/wip-rpm-rm-old-fedoras

ceph.spec.in: remove reference to EOL Fedoras

Merge pull request #4316 from Ved-vampir/librdb

librdb: add perf counters descriptions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4361 from oritwas/wip-11091

rgw: shouldn't return content-type: application/xml if content length is 0

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4351 from ceph/wip-11256

Wip 11256

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4363 from nilamdyuti/wip-doc-ceph-object-gateway

doc: Corrected the steps of Apache configuration for Debian-based
distros in object gateway installation guide.

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: Corrected the steps of Apache configuration for Debian-based distros
in object gateway installation guide.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

mon: make note about osd df calc bug

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix min variance calc in 'osd df'

If the reweight is 0, ignore the osd for the purposes of calculating the
min.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4354 from zhouyuan/fix_ct_doc

doc: Fix misleading overlay settting in Cache Tier

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4359 from ceph/wip-fix-ceph-dencoder-build-master

Move ceph-dencoder build to client
Rework mds/Makefile.am to support a dencoder client build
rgw/Makefile.am: Populate DENCODER_SOURCES properly
Dencoder should never be built with tcmalloc

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rgw: shouldn't return content-type: application/xml if content length is 0

Fixed: #11091

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Move ceph-dencoder build to client

The patch simply moves the ceph-dencoder build from server part of the
Makefiles to client part of the Makefiles.

Refs: #10691

Signed-off-by: Boris Ranto <branto@redhat.com>

Rework mds/Makefile.am to support a dencoder client build

The patch adds all the mds sources to DENCODER_SOURCES to allow a
dencoder client build. The patch also splits the Makefile.am file to
better accomodate the change.

Refs: #10691

Signed-off-by: Boris Ranto <branto@redhat.com>

rgw/Makefile.am: Populate DENCODER_SOURCES properly

Dencoder is built if ENABLE_CLIENT is set. However, the rgw/Makefile.am
populated DENCODER_SOURCES only if WITH_RADOSGW was set. The patch fixes
this and populates DENCODER_SOURES if ENABLE_CLIENT is set.

Signed-off-by: Boris Ranto <branto@redhat.com>

Dencoder should never be built with tcmalloc

The patch adds disabled perfglue stubs to DENCODER sources in order to
avoid tcmalloc-enabled ceph-dencoder builds.

Refs: #10691

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #4348 from ceph/wip-mon-clean

mon: remove dead MonitorStore code

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

librdb: add perf counters descriptions

Sign-off-by: Alyona Kiseleva <akiselyova@mirantis.com>

doc: Fix misleading overlay settting in Cache Tier

set-overlay is needed in either writeback or readonly mode of Cache
Tier setup.

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

client: fix uninline data funtion

For CEPH_OSD_CMPXATTR_MODE_U64, OSD expects the u64 to be encoded
as string in object's xattr

Fixes: #11356
Signed-off-by: Yan, Zheng <zyan@redhat.com>

ceph_json: add decode / encoder for multimap

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: use multimap to keep pending operations in bucket index

Fixes: #11256
Multiple concurrent requests might be sent using the same tag, need the
entry map to be able to hold multiple entries.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: generate new tag for object when setting object attrs

Fixes: #11256
Backport: firefly, hammer

Beforehand we were reusing the object's tag, which is problematic as
this tag is used for bucket index updates, and we might be clobbering a
racing update (like object removal).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4350 from almightybeeij/master

Check that delta_sum.stats.sum.num_object_copies and delta_sum.stats.sum...

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4349 from ceph/wip-dist-systemd-tmpfiles-d

#11383: packaging: ship systemd/ceph.tmpfiles.d in tarballs

packaging: ship systemd/ceph.tmpfiles.d in tarballs

Prior to this commit, the tarballs did not contain
any files under the top-level "systemd" directory. This caused problems
with RPM builds on Fedora and RHEL 7, because as of commit
aa88364f30e2d2f254ade185a83ba263b48e2a73, those RPMs depend on the
systemd/ceph.tmpfiles.d file.

(Longer-term we might want to improve the tarball generation code to be
less complex/manual.)

http://tracker.ceph.com/issues/11383 Refe: #11383

Reported-by: Greg Farnum <gfarnum@redhat.com>
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Check that delta_sum.stats.sum.num_object_copies and delta_sum.stats.sum.num_object are greater than zero

This fixes division by zero.

Signed-off-by BJ Lougee <almightybeeij@gmail.com>

mon: remove dead MonitorStore code

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: v0.94.1

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4317 from ceph/wip-rpm-with-systemd-comparison

ceph.spec.in: set _with_systemd on RHEL 7 and Fedora

Merge pull request #4327 from dachary/wip-wheezy

install-deps.sh: Debian GNU/Linux wheezy needs backports

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4341 from dachary/wip-test-timeout

tests: display ceph report when stuck

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #4334 from smithfarm/master

do_autogen: add missing '--without-lttng' option to usage()

Reviewed-by: Loic Dachary <ldachary@redhat.com>

AsyncConnection: Discard all prefetch buffer when replacing

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

tests: display ceph report when stuck

When the cluster is stuck (wait_for_clean times out), display ceph
report to stderr for debugging purposes.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4239 from yuyuyu101/wip-librbd-conf

Wip librbd conf

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4323 from ceph/wip-11363

librbd: ImageWatcher should cancel in-flight ops on watch error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4140 from ceph/wip-rbd-rebuild-object-map

rbd: add ability to rebuild corrupt/missing object maps

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4325 from wonzhq/async-resize-2

librbd/AsyncResize: avoid dup incrementing refresh seq

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4336 from ceph/wip-11369

librbd: failure to update the object map should always return success

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

install-deps.sh: Debian GNU/Linux wheezy needs backports

It is not enough for the backports to be available, they also need to be
explicitly allowed to take precedence whenever a package is installed
indirectly. This is causing problems with libp11-kit-dev pulled by
libcurl4-gnutls-dev.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4319 from ceph/wip-11348

librbd: update default image format to version 2

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

do_autogen.sh: add missing '--without-lttng' option to usage

Signed-off-by: Nathan Cutler <ncutler@suse.cz>

librbd: notify of header update after rebuilding object map

This will allow clients to detect that the object map is no longer
invalid and to reload the object map from disk.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: rebuilding object map shouldn't update piecemeal

The object map is now rebuilt in-memory and written back to the
OSDs as a single operation.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: add test case for cls_rbd object_map_save

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd_client: add object_map_save helper method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: add object_map_save method

Allow the object map rebuild process to build the full object map
in memory and save it to disk with one operation.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: add ImageWatcher test for new rebuild object map request

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: added rebuild_object_map test to test_librbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: librados_test_stub reads should deep-copy

If a client of librados_test_stub modified a bufferlist
retrieved via a read call, the client will actually be
changing the contents of the file.  Therefore, read calls
should deep-copy the contents of the buffer::ptrs.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: allow snapshot object maps to be updated

Rebuild will need to be able to update/resize the object maps
for image snapshots.  This was previously not permitted.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: update in-memory object map before on-disk update completes

When multiple IO updates are occurring within the same object, this
will allow only a single object map update request to be sent to the
OSD.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>