Merge pull request #6445 from ceph/wip-init-rbdmap

Init rbdmap patchset.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6468 from cxwshawn/wip-mail-fix

modify member info in mailmap

Reviewed-by: Loic Dachary <ldachary@redhat.com>

[mailmap] add member info.
Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

Merge tag 'v9.2.0'

v9.2.0

doc/release-notes: edits from Nathan Cutler

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: final infernalis notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6334 from xinxinsh/wip-13533

rbd : fix clone isssue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6465 from oritwas/wip-13596

radosgw-admin: metadata list user should return an empty list when user pool is empty

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

radosgw-admin: metadata list user should return an empty list when pool is empty

Fixes: #13596
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #6340 from joaquimrocha/rados_write_benchmark_new_options

Rados write benchmark new options

Reviewed-by: Loic Dachary <ldachary@redhat.com>

test/test_rados_tool.sh: Add tests for the new bench's write options

Those are the combinations of --write-omap, --write-xattr and --write-object.

Signed-off-by: Joaquim Rocha <joaquim.rocha@cern.ch>

tools/rados/rados.cc: Add options to choose the benchmark's write destination

Signed-off-by: Joaquim Rocha <joaquim.rocha@cern.ch>

tools/rados/rados.cc: Write to different destinations

The write benchmark only writes the objects' contents, not the omap
nor the extended attributes, which is something also interesting to
measure depending on the cluster's configuration.

These changes add a way to specify where the contents should be
written, allowing to combine any of the destinations mentioned above.

Signed-off-by: Joaquim Rocha <joaquim.rocha@cern.ch>

Merge pull request #6400 from ceph/wip-12997

rgw: Add default quota config

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5738 from dwj192/dwj192_bkt_index

rgw: fix objects can not be displayed which object name does not cont…

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

9.2.0

Merge pull request #6452 from dachary/wip-13664-test-ec

tests: test/librados/test.cc must create profile

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6410 from Vicente-Cheng/ceph-disk-list-plain-single-failure

ceph-disk: get Nonetype when ceph-disk list with --format plain on single device.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rbd: fix clone issue when we specify image feature

Fixes: #13553
Signed-off-by: xinxin shu <xinxin.shu@intel.com>

tests: test/librados/test.cc must create profile

Now that the create_one_ec_pool function removes the testprofile each
time it is called, it must create the testprofile erasure code profile
again for the test to use.

http://tracker.ceph.com/issues/13664 Refs: #13664

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6446 from dachary/wip-13664-test-ec

tests: destroy testprofile before creating one

Reviewed-by: Samuel Just <sjust@redhat.com>

tests: destroy testprofile before creating one

The testprofile erasure code profile is destroyed before creating a new
one so that it does not fail when another testprofile erasure code
profile already exists with different parameters.

This must be done when creating erasure coded pools with the C++
interface, in the same way it's done with the C interface.

http://tracker.ceph.com/issues/13664 Fixes: #13664

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: add destroy_ec_profile{,_pp} helpers

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6444 from liewegas/wip-pg-key

osd/PG: tolerate missing epoch key

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6406 from xiexingguo/xxg-wip-13626

FileJournal:_fdump wrongly returns if journal is currently unreadable.

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

ceph.spec.in: We no longer need redhat-lsb-core

Drop the redhat-lsb-core dependency as it is no longer necessary on
fedora/rhel.

The other two init scripts do not use redhat-lsb-core either. The
init-ceph.in conditionally requires /lib/lsb/init-functions and does not
use any of the functions defined in that file (at least not directly).
The init-radosgw file includes /etc/rc.d/init.d/functions on non-debian
platforms instead of /lib/lsb/init-functions file so it does not require
redhat-lsb-core either.

Signed-off-by: Boris Ranto <branto@redhat.com>

init-rbdmap: Rewrite to use logger + clean-up

This patch rewrites the init-rbdmap init script so that it uses logger
instead of the log_* functions. The patch also fixes various smaller
bugs like:
* MAP_RV was undefined if mapping already existed
* UMNT_RV and UMAP_RV were almost always empty (if they succeeded) ->
  removed them
* use of continue instead RET_OP in various places (RET_OP was not being
  checked after the switch to logger messages)
* removed use of DESC (used only twice and only one occurrence actually
  made sense)

Signed-off-by: Boris Ranto <branto@redhat.com>

osd/PG: tolerate missing epoch key

An orphan PG may have an info attr but no epoch key.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5133 from stiopaa1/ThreadMoveCopyConstr

Thread.h: disable copy constructor and assignment op

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6344 from dillaman/wip-13559-infernalis

librbd: potential assertion failure during cache read

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6329 from dzafman/wip-zafman-ceph

Wip zafman ceph

Reviewed-by: Dan Mick <dmick@redhat.com>

ceph.in: Notify user that 'tell' can't be used in interactive mode

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #6435 from ceph/docs-new-url

updates all http://ceph.com/ to download.ceph.com

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: update rpm links to download.ceph.com

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: update debian links to download.ceph.com

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #5783 from dzafman/wip-12738

Fix scrub error handling and log output
Add set-size and remove-clone-metadata to ceph-objectstore-tool
Fix trim_object() to not crash on corrupt snapset
Add scrub clones test and add clones to ceph-objectstore-tool test

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

test: osd-scrub-snaps.sh uses ceph-helpers.sh and added to make check

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Use boost::optional instead of snap 0 for "all_clones"

Signed-off-by: David Zafman <dzafman@redhat.com>

osd, test: When head missing a snapset, clones not an error

Fix log message for this case and make into "info"

Signed-off-by: David Zafman <dzafman@redhat.com>

osd, test: Keep missing count and log number of missing clones

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Eliminate check for bogus "obj13/head snaps empty" error

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Add new remove-clone-metadata object op

Fixes: #12740
Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix trim_object() to not crash on corrupt snapset

Add test case to check for trim_object() osd crash

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Improve object spec error handling

Add test cases

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Add undocumented clear-snapset command for testing

Add test cases using new feature to corrupt SnapSet

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Add set-size command for objects

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Enhanced dump command replaces dump-info

Show object stat information from objectstore
Show SnapSet if present
Add --head for search by object name

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Add some clones to ceph-objectstore-tool test

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: For corrupt objectstores, don't abort listing on errors

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Improve some error messages

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: White space fixes

Signed-off-by: David Zafman <dzafman@redhat.com>

tools/rados: Improve xattr import handling so future internal xattrs ignored

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Test scrubbing of snapshot problems

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Don't crash if OI_ATTR attribute is missing or corrupt

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Additional _scrub() check for snapset inconsistency

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Better SnapSet scrub checking (find issues instead of asserting)

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Make the _scrub routine produce good output and detect errors properly

Catch decode errors so osd doesn't crash on corrupt OI_ATTR or SS_ATTR
Use boost::optional<> to make current state clearer
Create next_clone as needed using head/curclone
Add equivalent logic after getting to end of scrubmap.objects

Fixes: #12738
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #5865 from dzafman/wip-13016

osd: Add config option osd_read_ec_check_for_errors for testing

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6045 from Sandy4999/wip-13133

rgw: refuse to calculate digest when the s3 secret key is empty

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6424 from badone/curl_easy_perform_fix

rgw: Fix typo in RGWHTTPClient::process error message

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6363 from ceph/wip-init-rbdmap

Drop redhat-lsb-core dependency

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: Fix typo in RGWHTTPClient::process error message

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #6358 from dx9/wip-12406-cmake-arch-detection

cmake: Use uname instead of arch.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6310 from dx9/wip-12406-test-libcephfs-flock

test/libcephfs/flock: add sys/file.h include for flock operations

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6377 from xiexingguo/wip-xxg-13597

FileStore: potential memory leak if _fgetattrs fails

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6414 from dzafman/wip-13625

messages/MOSDOp: Cast in assert to eliminate warnings

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6418 from tchaikov/wip-fix-ceph-monstore-update-crush

tools: ceph-monstore-update-crush: add "--test" when testing crushmap

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tools: ceph-monstore-update-crush: add "--test" to crushtool

in addition to testing the references to unknown name/type, we should
perform necessary test the crushmap when looking for a good crushmap

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: ceph-disk: coverage list_format_dev_plain() new behavior.

    modify the unittest to coverage the list_format_dev_plain()
    new behavior.

Signed-off-by: Vicente Cheng <freeze.bilsted@gmail.com>

FileJournal:_fdump wrongly returns if journal is currently unreadable.

_fdump wrongly returns if journal is currently unreadable, and fix some other errors also.
Fixes: #13626
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #5875 from Aran85/parsemetadatakey

rgw:modify the conditional statement in parse_metadata_key method.

Merge pull request #6409 from dachary/wip-mailmap

mailmap: Xie Xingguo affiliation

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5879 from H3C/wip-rgw-bugfix1

rgw/rgw_admin: Checking the legality of the parameters

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

messages/MOSDOp: Cast in assert to eliminate warnings

Fixes: #13625
Caused by 0bf2a79e

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #5262 from jmunhoz/wip-12352

rgw: remove extra check in RGWGetObj::execute

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: Add default quota config

Add a default quota config that is applied when no specific quota is
enabled.  There are both per-bucket and per-user defaults.  Defaults are
enabled if either max count or max size is enabled.

This fixes issues 12912 and 12997.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

ceph-disk: get Nonetype when ceph-disk list with --format plain on single device.

    with dmcrypt device, ceph-disk list will get Nonetype with the real device
    i.e. /dev/sdb1 for /dev/dm-1, `ceph-disk --list /dev/sdb1` will get Nonetype

    that because the holder we query is `dm-x`, the devices includ the path
    is `sdX`, it cannot match.

    if we list format plain and meet dmcrypt device, query again with the
    holder (dm-x).

    Also change the list_devices() parameter to let other function to use it
    easily.

Signed-off-by: Vicente Cheng <freeze.bilsted@gmail.com>

mailmap: Xie Xingguo affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6320 from JiYou/mything

Fix examples/librados/Makefile error.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6381 from xiexingguo/xxg-wip-fs-cleanup

FileStore: remove unused local variable 'handle'

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6384 from ktdreyer/wip-doc-download-gpg-key

doc: download GPG key from download.ceph.com

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6246 from liewegas/wip-13477

crush: validate bucket id before indexing buckets array

Reviewed-by: Loic Dachary <ldachary@redhat.com>

crush/mapper: ensure take bucket value is valid

Ensure that the take argument is a valid bucket ID before indexing the
buckets array.

Signed-off-by: Sage Weil <sage@redhat.com>

crush/mapper: ensure bucket id is valid before indexing buckets array

We were indexing the buckets array without verifying the index was within
the [0,max_buckets) range.  This could happen because a multistep rule
does not have enough buckets and has CRUSH_ITEM_NONE
for an intermediate result, which would feed in CRUSH_ITEM_NONE and
make us crash.

Fixes: #13477
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6239 from zhouyuan/swift_testing_access

vstart.sh: grant full access to Swift testing account

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6303 from dx9/wip-12406-use-freopen

test_rgw_admin:  Use freopen for output redirection.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6387 from dachary/wip-test-ports

tests: port uniqueness reminder

Reviewed-by: Vicente Cheng <freeze.bilsted@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

tests: port uniqueness reminder

When copy/pasting a tests, it is easy to forget (or not know) that the
port used must be unique to allow for multiple tests to run in
parallel (make -j8). Add a reminder next to each port.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6379 from tchaikov/wip-13554

rocksdb: remove rdb source files from dist tarball

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6258 from swamireddy/wip-v910-mailmap

mailmap updates

Reviewed-by: Loic Dachary <ldachary@redhat.com>

FileStore: remove unused local variable 'handle'

Remove unused local variable 'handle' in FileStore::collection_empty
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

FileStore: potential memory leak if _fgetattrs fails

Memory leak happens if _fgetattrs encounters some error and simply returns.
Fixes: #13597
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #6366 from liewegas/wip-osd-fixboot-infernalis

osd: fix OSDService vs Objecter init order

Reviewed-by: Samuel Just <sjust@redhat.com>

doc: download GPG key from download.ceph.com

Fixes: #13603
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc/release-notes: v0.94.5

Signed-off-by: Sage Weil <sage@redhat.com>

release-notes: draft v0.94.5 release notes

Signed-off-by: Loic Dachary <ldachary@redhat.com>

rocksdb: remove rdb source files from dist tarball

rdb is based on NodeJS, and we don't build rdb to use rocksdb as
a keyvaluestore backend.

Fixes: #13554
Signed-off-by: Kefu Chai <kchai@redhat.com>

vstart.sh: grant full access to Swift testing account

This makes Swift testing more convinent.

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Merge pull request #6375 from david-z/wip-fix-kv-stripe-size-conf

KeyValueStore: fix the name's typo of keyvaluestore_default_strip_size

Reviewed-by: Kefu Chai <kchai@redhat.com>

KeyValueStore: fix the name's typo of keyvaluestore_default_strip_size

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>