rgw: fix ListParts response

The response XML element name should be 'ListPartsResult'.

Fixes: #11494
Signed-off-by: Henry Chang <henry@bigtera.com>

Merge pull request #4426 from liewegas/wip-11330

logrotate: prefer service over invoke-rc.d

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4425 from ceph/wip-10243

civetweb: update max num of threads

Merge pull request #4421 from dachary/wip-11441-osd-bench

tests: osd-bench.sh must be verbose on failure

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4430 from zhouyuan/fix_make_debs

misc: fix make debs script

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4427 from big-henry/wip-mds-fix-integer-truncation

mds: fix integer truncation

misc: fix make debs script

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Merge pull request #4428 from wkennington/master.pgrefdebugging

configure.ac: fix the disablement of pgrefdebugging

Reviewed-by: Kefu Chai <kchai@redhat.com>

Fix the disablement of pgrefdebugging

Currently, if you add the flag --disable-pgrefdebugging during configure
it enables pgrefdebugging. The only way for it to actually be disabled
is to remove the flag entirely. This patch makes the flag respect the
--enable and --disable options correctly. Therefore,
--enable-pgrefdebugging is the only way to enable this now and
--enable-pgrefdebugging=no and --disable-pgrefdebugging work as
expected.

Signed-off-by: William A. Kennington III <william@wkennington.com>

mds: fix integer truncation

Client ID is a 64-bit integer. Unlike Ceph daemons, it may go beyond
2147483647 and cause problems.

Signed-off-by: Henry Chang <henry@bigtera.com>

Merge pull request #4376 from majianpeng/needs-recovery

osd: fix bugs for PG::needs_recovery()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4269 from majianpeng/add-recoverable

osd: For ec pool recovery, only for can recoverable object.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4222 from majianpeng/scrub-fix1

osd/PG: check scrub state when handle CEPH_OSD_OP_SCRUB_MAP.

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: add test-case for repair unrecovery-ec pg.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd: Remove the duplicated func MissingLoc::get_all_missing.

It is the same as MissingLoc::get_needs_recovery.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd: For object op, first check object whether unfound.

If object unfound, asap return -EIO.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd: Fix ec pg repair endless when met unrecover object.

In repair_object, if bad_peer is replica, it don't add soid in
MissingLoc for ec pool. If there are more bad replica for ec pool
which cause object can't recover, the later recoverying will endless.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

PG: For needs_recovery, asap return if need recover.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

PG: remove the duplicated code.

assert(!actingbackfill.empty()) is equal to
assert(actingbackfill.end() != actingbackfill.begin()).

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

PG: fix logic error: don't set need_recovery if peer_missing don't have acting osd.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

logrotate.conf: prefer service over invoke-rc.d

On trusty 14.04, service works but invoke-rc.d does not (but is present).

Fixes: #11330
Reported-by: Wim Heirman <wim@heirman.net>
Signed-off-by: Sage Weil <sage@redhat.com>

tests: osd-bench.sh must be verbose on failure

When a command fail as it should but it does not contain the expected
output, display the entire output to confirm it's a false negative.

http://tracker.ceph.com/issues/11441 Fixes: #11441

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: convert osd-bench.sh to ceph-helpers.sh

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph-helpers: implement test_expect_failure

To display the output in case the command did not fail with the expected
output.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4399 from trociny/fix-osd_df_avgutil

mon: osd df: fix average_util calculation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4386 from ceph/wip-10657

mds: warn when clients are not advancing their oldest_client_tid

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4331 from ceph/wip-mds-misc

Wip mds misc

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4424 from ceph/wip-doc-nuke-setmap

doc: don't mention ceph osd setmap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

civetweb: update max num of threads

Fixes: #10243
cherry-picked upstream fixed into submodule

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4420 from tchaikov/wip-man-fix-desc-in-name

man: fix the description in NAME section

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4409 from ceph/wip-11437

rgw: don't use end_marker for namespaced object listing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4408 from ceph/wip-11436

rgw: adjust return code if can't find upload

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4407 from ceph/wip-11435

rgw: fail if parts not specified on complete-multipart-upload

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: don't mention ceph osd setmap

ceph osd setmap command was disabled and subsequently removed 5 years
ago.  See 9aadd41b2095 ("mon: disable 'osd setmap'") and 6d292397aa26
("mon: remove dead 'setmap' code").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #4423 from ceph/wip-11370

librbd: better handling for duplicate flatten requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: better handling for duplicate flatten requests

A proxied flatten request could be replayed, resulting in a
-EINVAL error code being generated on the second attempt. Filter
out that error if it is known the parent did exist before the
op started.

Fixes: #11370
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #4372 from xiaoxichen/fix_kv_compact_on_mount

Fix kv compact on mount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4418 from oritwas/wip-11438

rgw: force content-type header for swift account responses without body

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

client: add failure injection for not advancing oldest_tid

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: include size of completed_requests in session dump

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: warn when clients are not advancing their oldest_client_tid

Fixes: #10657
Signed-off-by: Yan, Zheng <zyan@redhat.com>

rgw: force content-type header for swift account responses without body

This regression was introduced in commit 4cfeca03ade21861ab70ca759f94eb244c16cb39

Fixes: #11438
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

man: fix the description in NAME section

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4370 from guangyy/wip-ec-test-fix

qa : misc fixes to stabilize test-erasure-code suite

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4411 from hjwsm1989/wip-monstore-pool

mon: ceph-monstore-tool must do out_store.close() in "store-copy" command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4417 from osynge/wip-LimitNOFILE

Increase max files open limit for OSD daemon.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tools: ceph-monstore-tool must do out_store.close()

this change fixes the "store-copy" command.
Like the bug reported in http://tracker.ceph.com/issues/10093.

Signed-off-by: huangjun <hjwsm1989@gmail.com>

Increase max files open limit for OSD daemon.

Under heavy load the number of file descriptors opened
by the OSD can go beyond the 64K file limit. This patch
increases the default to 128K.

Signed-off-by: Owen Synge <osynge@suse.com>

client: inlcude inline data in FLUSHSNAP cap message

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: flush dirty snap data before allowing new writes

this guarantees that new writes do not overwrite existing dirty
snap data.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: properly update capability's client_follow

If snap inode exists, capability's client_follow has already been
updated by MDCache::cow_inode(). So we only update client_follow
when the inode was not COWed.

Besides, we need to check return of SnapRealm::get_snap_following().
It can return CEPH_NOSNAP, which is clearly invalid for client_follows.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: fix Locker::_do_null_snapflush

the snapid check in Locker::_do_null_snapflush() prevents us from
flushing old snap inodes. The check does not make sense.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: do null snap flush when processing embeded cap releases

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: make sure snap inode's last match existing snapshots

MDCache::pick_inode_snap() and snap caps flushing code require
this.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: remove stray dentry from delayed list when it's queued for purging

this avoids assert(!dn->state_test(CDentry::STATE_PURGING)) in
eval_stray() being triggered when processing the delayed list.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: move snap inode to new dirfrag's dirty_rstat_inodes

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: keep auth pin on CDir when marking it dirty

Signed-off-by: Yan, Zheng <zyan@redhat.com>

osd/PG: check scrub state when handle CEPH_OSD_OP_SCRUB_MAP.

There is a chance which cause scrub stop before replica-node send
scrub-map.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #4257 from rzarzynski/wip-10683

rgw: rectify support for GET on Swift's account with "limit" parameter == 0

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't use end_marker for namespaced object listing

Fixes: #11437
Don't deal with end_marker if there's a namespace set for the object
listing. The cur_end_marker.empty() check is incorrect if no end_marker
is provided, but a namespace is set. Just avoid this issue entirely, as
it's not needed.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: adjust return code if can't find upload

Fixes: #11436
Return NoSuchUpload (instead of NoSuchKey).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fail if parts not specified on complete-multipart-upload

Fixes: #11435
A complete multipart upload should not succeed if parts were not
specified. Also, adjust return codes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4406 from ceph/wip-11433

librbd: always initialize perf counters

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: always initialize perf counters

If ImageCtx::init() fails, perf counters are still used when closing
the image. To prevent uninitialized use, it's easiest to always
initialize them before anything that can fail in ImageCtx::init().

This was moved in 6055df398557e4dbcabc66f463a0a126f9b5dc74, but it's
not necessary to delay perf counter initialization for any extra
configuration.

Fixes: #11433
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4302 from ceph/wip-11339

client: drop inode when rmdir request finishes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4282 from ceph/wip-3645

mds: rename snapshot support

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com

Merge pull request #4405 from ceph/wip-11380

cls_rbd: get_features needs to support legacy negative tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4402 from ceph/wip-rbd-unmap-by-spec

rbd unmap <image-name>

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls_rbd: get_features needs to support legacy negative tests

During Ceph upgrade testing, older Ceph test suites assume that
get_features will return -ENOENT if provided a missing snapshot.
Support these negative tests until the older releases are no
longer supported.

Fixes: #11380
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #3991 from rzarzynski/wip-11087

rgw: multiple fixes for copying Swift objects

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3863 from rzarzynski/wip-10971

rgw: multiple fixes for Content-Length header generation in Swift API

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: make rgw_read_user_buckets() backward compatible.

Fixes: #10683
Backport: hammer
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: rectify support for GET on Swift's account with limit == 0.

Fixes: #10683
Backport: hammer
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #4400 from ceph/update-organization-1

Update Haomai's organization

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3847 from trociny/wip-test_with_crushtool.master

mon: ceph osd setcrushmap: test_with_crushtool improvements

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4343 from ceph/wip-11356

client: fix uninline data funtion

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4223 from ceph/wip-11254

mds: make sure lock state not stay in XLOCK/XLOCKDONE indefinitely

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4218 from ceph/wip-11048

mds: persist completed_requests reliably

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4204 from ceph/wip-10368-mk3

mds: separate MDLog::safe_pos from journaler

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mon: osd df: fix average_util calculation

average_util should be calculated from the same set of nodes that
dump_item considers when calculating the stddev.

While here, remove unnecessary check for kb > 0:
get_bucket_utilization returns false if kb <= 0.

Also, update get_bucket_utilization accordingly to the preferred
convention to use pointers and not references for output arguments.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Update Haomai's organization

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test/cli-integration/rbd: add unmap test

Add a test to exercise rbd unmap.  This was inspired by and is mostly
for rbd unmap <image-name> (aka spec), but rbd unmap <device> is tested
as well.  Any significant change to krbd.cc or corresponding rbd cli
should be tested in the following three configurations at least:

- kernel < 3.14
- kernel >= 3.14, rbd.single_major=N
- kernel >= 3.14, rbd.single_major=Y

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: allow unmapping by spec

Make use of krbd_unmap_by_spec() and allow unmapping by <image-name>
(i.e. pool/image@snap spec) with a bunch of careful adjustments to the
fragile command args/options parsing process to make --pool, --image
and --snap options work as expected for rbd unmap <image-name> case.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

krbd: add krbd_unmap_by_spec()

Make unmapping by (pool, image, snap) spec possible.  For specs mapped
multiple times unmap one device per krbd_unmap_by_spec() call.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

krbd: rename should_match_minor() to have_minor_attr()

Rename and clarify why simply checking kernel version doesn't work.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

client: drop inode when rmdir request finishes

Current client code relies on cap message to trim unlinked inode
from cache. This method is too sensitive to message ordering. If
client receives the cap message while directory inode still contains
dentries that are referenced by unsafe requests, the inode can't
get trimmed.

The fix is, when rmdir/rename request finishes, try trimming the
unlinked inode again.

Fixes: #11339
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #4380 from ceph/wip-nodebug-build

tests: separate check_PROGRAMS from TESTS

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: separate check_PROGRAMS from TESTS

Not all programs that should be built during 'make check' should
be executed as tests.  A new variable, check_TESTPROGRAMS, was
introduced to specify which programs should be executed as a test
during a 'make check'.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #4311 from yuyuyu101/wip-async-fix-4

Wip async fix 4

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4358 from gerhardqux/gerhardqux-nokey-exit-1

ceph-authtool: exit(1) when printing a non-existing key

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4012 from majianpeng/cache-tier-fadvise

Cache tier fadvise

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3606 from wonzhq/hitset-insert

osd: avoid inserting an op into hit set multiple times

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4378 from dachary/wip-11359-tier-tests-disable

tests: comment out unstable tiering tests

tests: comment out unstable tiering tests

So that make check does not randomly fails.

http://tracker.ceph.com/issues/11359 Refs: #11359

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4377 from liewegas/wip-git.ceph.com

use git.ceph.com

Reviewed-by: Alfredo Deza <adeza@redhat.com>

use git://git.ceph.com

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-authtool: exit(1) when printing a non-existing key

When printing a non-existing key, ceph-authtool exits with a success
value:

[root@ip-172-31-3-178 ~]# ceph-authtool /etc/ceph/ceph.mon.keyring -p -n
client.doesntexist ; echo $?
entity client.doesntexist not found
0

Expected result: 1

Signed-off-by: Gerhard Muntingh <gerhard@qux.nl>

Merge pull request #4373 from dachary/wip-releases

releases: table of estimated end of life

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

releases: table of estimated end of life

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4339 from ceph/wip-da-SCA-20150317

Various SCA and Coverity fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>