PG: For needs_recovery, asap return if need recover.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

PG: remove the duplicated code.

assert(!actingbackfill.empty()) is equal to
assert(actingbackfill.end() != actingbackfill.begin()).

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

PG: fix logic error: don't set need_recovery if peer_missing don't have acting osd.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #4399 from trociny/fix-osd_df_avgutil

mon: osd df: fix average_util calculation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4386 from ceph/wip-10657

mds: warn when clients are not advancing their oldest_client_tid

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4331 from ceph/wip-mds-misc

Wip mds misc

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4424 from ceph/wip-doc-nuke-setmap

doc: don't mention ceph osd setmap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4420 from tchaikov/wip-man-fix-desc-in-name

man: fix the description in NAME section

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4409 from ceph/wip-11437

rgw: don't use end_marker for namespaced object listing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4408 from ceph/wip-11436

rgw: adjust return code if can't find upload

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4407 from ceph/wip-11435

rgw: fail if parts not specified on complete-multipart-upload

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: don't mention ceph osd setmap

ceph osd setmap command was disabled and subsequently removed 5 years
ago.  See 9aadd41b2095 ("mon: disable 'osd setmap'") and 6d292397aa26
("mon: remove dead 'setmap' code").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #4423 from ceph/wip-11370

librbd: better handling for duplicate flatten requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: better handling for duplicate flatten requests

A proxied flatten request could be replayed, resulting in a
-EINVAL error code being generated on the second attempt. Filter
out that error if it is known the parent did exist before the
op started.

Fixes: #11370
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #4372 from xiaoxichen/fix_kv_compact_on_mount

Fix kv compact on mount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4418 from oritwas/wip-11438

rgw: force content-type header for swift account responses without body

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

client: add failure injection for not advancing oldest_tid

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: include size of completed_requests in session dump

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: warn when clients are not advancing their oldest_client_tid

Fixes: #10657
Signed-off-by: Yan, Zheng <zyan@redhat.com>

rgw: force content-type header for swift account responses without body

This regression was introduced in commit 4cfeca03ade21861ab70ca759f94eb244c16cb39

Fixes: #11438
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

man: fix the description in NAME section

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4370 from guangyy/wip-ec-test-fix

qa : misc fixes to stabilize test-erasure-code suite

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4411 from hjwsm1989/wip-monstore-pool

mon: ceph-monstore-tool must do out_store.close() in "store-copy" command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4417 from osynge/wip-LimitNOFILE

Increase max files open limit for OSD daemon.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tools: ceph-monstore-tool must do out_store.close()

this change fixes the "store-copy" command.
Like the bug reported in http://tracker.ceph.com/issues/10093.

Signed-off-by: huangjun <hjwsm1989@gmail.com>

Increase max files open limit for OSD daemon.

Under heavy load the number of file descriptors opened
by the OSD can go beyond the 64K file limit. This patch
increases the default to 128K.

Signed-off-by: Owen Synge <osynge@suse.com>

client: inlcude inline data in FLUSHSNAP cap message

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: flush dirty snap data before allowing new writes

this guarantees that new writes do not overwrite existing dirty
snap data.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: properly update capability's client_follow

If snap inode exists, capability's client_follow has already been
updated by MDCache::cow_inode(). So we only update client_follow
when the inode was not COWed.

Besides, we need to check return of SnapRealm::get_snap_following().
It can return CEPH_NOSNAP, which is clearly invalid for client_follows.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: fix Locker::_do_null_snapflush

the snapid check in Locker::_do_null_snapflush() prevents us from
flushing old snap inodes. The check does not make sense.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: do null snap flush when processing embeded cap releases

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: make sure snap inode's last match existing snapshots

MDCache::pick_inode_snap() and snap caps flushing code require
this.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: remove stray dentry from delayed list when it's queued for purging

this avoids assert(!dn->state_test(CDentry::STATE_PURGING)) in
eval_stray() being triggered when processing the delayed list.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: move snap inode to new dirfrag's dirty_rstat_inodes

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: keep auth pin on CDir when marking it dirty

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #4257 from rzarzynski/wip-10683

rgw: rectify support for GET on Swift's account with "limit" parameter == 0

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't use end_marker for namespaced object listing

Fixes: #11437
Don't deal with end_marker if there's a namespace set for the object
listing. The cur_end_marker.empty() check is incorrect if no end_marker
is provided, but a namespace is set. Just avoid this issue entirely, as
it's not needed.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: adjust return code if can't find upload

Fixes: #11436
Return NoSuchUpload (instead of NoSuchKey).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fail if parts not specified on complete-multipart-upload

Fixes: #11435
A complete multipart upload should not succeed if parts were not
specified. Also, adjust return codes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4406 from ceph/wip-11433

librbd: always initialize perf counters

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: always initialize perf counters

If ImageCtx::init() fails, perf counters are still used when closing
the image. To prevent uninitialized use, it's easiest to always
initialize them before anything that can fail in ImageCtx::init().

This was moved in 6055df398557e4dbcabc66f463a0a126f9b5dc74, but it's
not necessary to delay perf counter initialization for any extra
configuration.

Fixes: #11433
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4302 from ceph/wip-11339

client: drop inode when rmdir request finishes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4282 from ceph/wip-3645

mds: rename snapshot support

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com

Merge pull request #4405 from ceph/wip-11380

cls_rbd: get_features needs to support legacy negative tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4402 from ceph/wip-rbd-unmap-by-spec

rbd unmap <image-name>

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls_rbd: get_features needs to support legacy negative tests

During Ceph upgrade testing, older Ceph test suites assume that
get_features will return -ENOENT if provided a missing snapshot.
Support these negative tests until the older releases are no
longer supported.

Fixes: #11380
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #3991 from rzarzynski/wip-11087

rgw: multiple fixes for copying Swift objects

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3863 from rzarzynski/wip-10971

rgw: multiple fixes for Content-Length header generation in Swift API

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: make rgw_read_user_buckets() backward compatible.

Fixes: #10683
Backport: hammer
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: rectify support for GET on Swift's account with limit == 0.

Fixes: #10683
Backport: hammer
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #4400 from ceph/update-organization-1

Update Haomai's organization

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3847 from trociny/wip-test_with_crushtool.master

mon: ceph osd setcrushmap: test_with_crushtool improvements

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4343 from ceph/wip-11356

client: fix uninline data funtion

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4223 from ceph/wip-11254

mds: make sure lock state not stay in XLOCK/XLOCKDONE indefinitely

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4218 from ceph/wip-11048

mds: persist completed_requests reliably

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4204 from ceph/wip-10368-mk3

mds: separate MDLog::safe_pos from journaler

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mon: osd df: fix average_util calculation

average_util should be calculated from the same set of nodes that
dump_item considers when calculating the stddev.

While here, remove unnecessary check for kb > 0:
get_bucket_utilization returns false if kb <= 0.

Also, update get_bucket_utilization accordingly to the preferred
convention to use pointers and not references for output arguments.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Update Haomai's organization

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test/cli-integration/rbd: add unmap test

Add a test to exercise rbd unmap.  This was inspired by and is mostly
for rbd unmap <image-name> (aka spec), but rbd unmap <device> is tested
as well.  Any significant change to krbd.cc or corresponding rbd cli
should be tested in the following three configurations at least:

- kernel < 3.14
- kernel >= 3.14, rbd.single_major=N
- kernel >= 3.14, rbd.single_major=Y

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: allow unmapping by spec

Make use of krbd_unmap_by_spec() and allow unmapping by <image-name>
(i.e. pool/image@snap spec) with a bunch of careful adjustments to the
fragile command args/options parsing process to make --pool, --image
and --snap options work as expected for rbd unmap <image-name> case.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

krbd: add krbd_unmap_by_spec()

Make unmapping by (pool, image, snap) spec possible.  For specs mapped
multiple times unmap one device per krbd_unmap_by_spec() call.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

krbd: rename should_match_minor() to have_minor_attr()

Rename and clarify why simply checking kernel version doesn't work.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

client: drop inode when rmdir request finishes

Current client code relies on cap message to trim unlinked inode
from cache. This method is too sensitive to message ordering. If
client receives the cap message while directory inode still contains
dentries that are referenced by unsafe requests, the inode can't
get trimmed.

The fix is, when rmdir/rename request finishes, try trimming the
unlinked inode again.

Fixes: #11339
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #4380 from ceph/wip-nodebug-build

tests: separate check_PROGRAMS from TESTS

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: separate check_PROGRAMS from TESTS

Not all programs that should be built during 'make check' should
be executed as tests.  A new variable, check_TESTPROGRAMS, was
introduced to specify which programs should be executed as a test
during a 'make check'.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #4311 from yuyuyu101/wip-async-fix-4

Wip async fix 4

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4358 from gerhardqux/gerhardqux-nokey-exit-1

ceph-authtool: exit(1) when printing a non-existing key

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4012 from majianpeng/cache-tier-fadvise

Cache tier fadvise

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3606 from wonzhq/hitset-insert

osd: avoid inserting an op into hit set multiple times

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4378 from dachary/wip-11359-tier-tests-disable

tests: comment out unstable tiering tests

tests: comment out unstable tiering tests

So that make check does not randomly fails.

http://tracker.ceph.com/issues/11359 Refs: #11359

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4377 from liewegas/wip-git.ceph.com

use git.ceph.com

Reviewed-by: Alfredo Deza <adeza@redhat.com>

use git://git.ceph.com

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-authtool: exit(1) when printing a non-existing key

When printing a non-existing key, ceph-authtool exits with a success
value:

[root@ip-172-31-3-178 ~]# ceph-authtool /etc/ceph/ceph.mon.keyring -p -n
client.doesntexist ; echo $?
entity client.doesntexist not found
0

Expected result: 1

Signed-off-by: Gerhard Muntingh <gerhard@qux.nl>

Merge pull request #4373 from dachary/wip-releases

releases: table of estimated end of life

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

releases: table of estimated end of life

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4339 from ceph/wip-da-SCA-20150317

Various SCA and Coverity fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4367 from ceph/wip-ceph-disk-default-cluster

ceph-disk: set '--cluster=ceph' in 'prepare' subcommand

Reviewed-by: Loic Dachary <ldachary@redhat.com>

os/LevelDBStore:fix bug when compact_on_mount

Initialize the performance counter(l_rocksdb_compact)
before use it in compact().

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

os/RocksDBStore: fix bug when compact_on_mount

Initialize the performance counter(l_rocksdb_compact)
before use it in compact().

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #4366 from BCLibCoop/bclibcoop/rgw-hostnames

rgw: Document & improve region hostnames

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

TestPGLog.cc: fix -Wsign-compare

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec.cc: fix -Wsign-compare

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_async_driver.cc: fix UNINIT

Fix for:

CID 1254380 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use_in_call: Using uninitialized value sa. Field
 sa.sin_zero is uninitialized when calling connect

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

psim.cc: change initialzation of VLAs

Use memset to fill integer VLAs.

Fix for:

CID 1219649 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use: Using uninitialized value primary_count[primary]

CID 1219648 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use: Using uninitialized value first_count[osds[0UL]]

CID 1219647 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use: Using uninitialized value count[osds[i]]

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

systest_runnable.h: mark copy ctor as explicit

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/Thread.h: mark copy/move ctor as explicit

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

cls_rbd.cc: reduce scope of local variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ReplicatedBackend.cc: init 'Message *commit' to NULL

Init variable with NULL to make sure it crashes in error
case (would be a bug) if NDEBUG disable the assert() call.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PG.h: mark copy ctor explicit

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

msg/Dispatcher.h: make copy/move ctor explicit

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_shared_cache.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rest_user.cc: UserQuotas init vars in ctor init list

Fix for:

[src/rgw/rgw_rest_user.cc:631]: (performance) Variable 'bucket_quota' is
 assigned in constructor body. Consider performing initialization in
 initialization list.
[src/rgw/rgw_rest_user.cc:632]: (performance) Variable 'user_quota' is
 assigned in constructor body. Consider performing initialization in
 initialization list.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

client/Client.cc: fix uninit variables

This is caused by assert(). If assert is disabled we run into use
of uninitialized variables. Return -EINVAL in case assert() is
disabled.

Fix for:

[src/client/Client.cc:7791]: (error) Uninitialized variable: lock_state
[src/client/Client.cc:7829]: (error) Uninitialized variable: lock_type

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

libcephfs.h: fix unnecessary forward declarations

This is an issue in case non-cpp code. Move the declaration of
inodeno_t and vinodeno_t structs into __cplusplus block where
they are needed.

Fix for cppcheck issue:

[src/include/cephfs/libcephfs.h:89] -> [src/include/cephfs/libcephfs.h:74]:
 The struct 'inodeno_t' forward declaration is unnecessary. Type struct is
 already declared earlier.
[src/include/cephfs/libcephfs.h:93] -> [src/include/cephfs/libcephfs.h:83]:
 The struct 'vinodeno_t' forward declaration is unnecessary. Type struct is
 already declared earlier.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: fix format qualifier in COMMIT_SNAP_ITEM

All places COMMIT_SNAP_ITEM is used the variables are
'unsigned long long'. Change the format qualifier to reflect
this and to fix:

[src/os/FileStore.cc:1341]: (warning) %lld in format string (no. 1)
 requires 'long long *' but the argument type is 'unsigned long long *'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDCache.cc: fix potential null pointer dereference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/../chain_xattr.cc: fix suspicious usage of 'sizeof'

Use sizeof(char) instead of sizeof('\0').

Fix for:

[src/test/objectstore/chain_xattr.cc:164]: (warning) Suspicious
 usage of 'sizeof' with a numeric constant as parameter.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec: fix some -Wsign-compare

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rest_log.h: remove http_ret from some RGWOp_*_list classes

Remove http_ret from child class since there is already an integer
with the same name in the parent/base class.

Fix for:

[src/rgw/rgw_rest_log.h:79] -> [src/rgw/rgw_rest.h:275]: (warning) The
 class 'RGWOp_MDLog_List' defines member variable with name 'http_ret'
 also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:99] -> [src/rgw/rgw_rest.h:275]: (warning) The
 class 'RGWOp_MDLog_Info' defines member variable with name 'http_ret'
 also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:182] -> [src/rgw/rgw_rest.h:275]: (warning) The
 class 'RGWOp_DATALog_List' defines member variable with name 'http_ret'
 also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:202] -> [src/rgw/rgw_rest.h:275]: (warning) The
 class 'RGWOp_DATALog_Info' defines member variable with name 'http_ret'
 also defined in its parent class 'RGWRESTOp'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>