cmake: do not link ec_jerasure_generic against crush

crush is not used in erasure plugins at this moment.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: exclude private symbols in librbd

missed it in 0edfad0.
* librbd: pass '--exclude-libs,ALL' to linker
* rbd: link against used libraries explicitly, rbd is accessing some
  internal symbols not exposed by librbd.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10379 from xiexingguo/xxg-wip-bluestore-2016-07-21

os/bluestore: fix bitmap allocating failure if max_alloc_size is 0

Mark's Comments:

This passed Jenkins checks.
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #10140 from ifed01/wip-bluestore-fsck-update

os/bluestore: replace interval_set with a bitset for used_block conta…

Mark's Comments:

This passed Jenkins checks.
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests, however no fsck specific performance or memory tests were performed.

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #10060 from trociny/wip-16386

librbd: discard hangs when 'rbd_skip_partial_discard' is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10472 from tchaikov/wip-cmake

cmake: exclude non-public symbols in shared libraries

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #10273 from yonghengdexin735/wip-zzz-pip-delete-no_casting_required

msg/simple/Pipe: eliminating casts for the comparing of len and recv_max_prefetch

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10477 from cbodley/wip-16839

radosgw-admin: zone[group] modify can change realm id
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9988 from ifed01/wip-bluestore-reduce-blob4

os/bluestore: get rid off blob's ref_map for non-shared objects

Mark's Comments:

This passed Jenkins checks.
This PR did not appear to have a significant impact on performance tests.
This initially failed "ceph_test_objectstore --gtest_filter=*/2" segfaulting in one of the tests.  See the attached log in the comments for the PR for details.  Subsequent reruns passed, and a run through valgrind's memcheck also passed.  Igor indicated that he has seen this particular segfault previously and believes it is related to running out of memory.  Given that the PR has passed the object store tests multiple times and the segfault has been observed in the past, I believe this PR is unrelated.

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #10439 from weiqiaomiao/wqm-wip-rgw-zonedump

rgw: add lc_pool when decode or encode struct RGWZoneParams

Reviewed-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: zone[group] modify can change realm id

allows the default zone and zonegroup (created with empty realm_id) to
be later added to a realm. the 'modify' command now accepts either
--realm_id=id or --rgw-realm=name

Fixes: http://tracker.ceph.com/issues/16839
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10476 from yonghengdexin735/wip-zzz-CDir

mds/CDir: remove the part of judgment for _next_dentry_on_set

Reviewed-by: John Spray <john.spray@redhat.com>

mds/CDir: remove the part of judgment for _next_dentry_on_set

Signed-off-by: zhang.zezhu <zhang.zezhu@zte.com.cn>

Merge pull request #10428 from jcsp/wip-16592

mon: validate states transmitted in beacons

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mon: validate states transmitted in beacons

Since FSMap was added, the state of a daemon can lead
to an entirely invalid map, but we were letting daemons
send any state they wanted.

Especially, we must not allow standby daemons to set
any state other than STANDBY.

Fixes: http://tracker.ceph.com/issues/16592
Signed-off-by: John Spray <john.spray@redhat.com>

cmake: link rgw_a against cls_lock_client

rgw_lc.cc is using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10429 from jcsp/wip-16807-assertion

mds: add assertion in handle_slave_rename_prep

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

cmake: exclude private symbols in public libs

we should avoid exposing non-public symbols from user facing dynamic
libraries. so pass '--exclude-libs' with appropriate argument to linker.
as libcephfs does not add the ((visibility ("default"))) specifier to
the exported symbols, we can not set the default visiblity to hidden for
libcephfs, and "-export-symbols-regex" is a libtool option, we need to
develop a way for cmake to fix the visibility of libcephfs. it's still a
TODO.

* librados
  - pass '--exclude-libs=ALL' to linker
  - add buffer.cc to librados, so we can use '--exclude-libs=ALL'.
* libcephfs: pass '--exclude-libs=libcommon.a,libclient.a,libosdc.a' to
  linker
* libcommon
  - extract common_buffer_obj from libcommon, to avoid compilation
    this source file repeatly.
* tests:
  - link against common_internal_objs explicitly if the test in
    question is using the internal symbols.
* ceph-client-debug:
  - link against client explicitly, and do not link against librados
    anymore, as it is not used in this tool.

Fixes: http://tracker.ceph.com/issues/16556
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: move common_utf8 closer to where it is used

for example, json_spirit uses common_utf8, so link json_spirit against common_utf8.
because libcommon includes json_spirit, this change does not impact the
consumers of libcommon.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9925 from fullerdj/wip-djf-11171

mds: Add path filtering for dump cache

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10281 from vishalkanaujia/wip_inotable_repair

cephfs: Inotable repair during forward scrub

Reviewed-by: John Spray <john.spray@redhat.com>

test: fix librbd tests for rbd_skip_partial_discard

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: discard hangs when 'rbd_skip_partial_discard' is enabled

Fixes: http://tracker.ceph.com/issues/16386
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

msg/simple/Pipe: eliminating casts for the comparing of len and recv_max_prefetch

Signed-off-by: zhang.zezhu <zhang.zezhu@zte.com.cn>

Merge pull request #10457 from SUSE/wip-15497

doc: add Backporting section to Essentials chapter

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: add Backporting section to Essentials chapter

... and fix some broken links.

Fixes: http://tracker.ceph.com/issues/15497
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10414 from SUSE/wip-doc-dev

doc: add Upload logs to archive server section...

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: add Upload logs to archive server section...

...to Testing in the cloud testing chapter

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10456 from cbodley/wip-16834

rgw: use endpoints from master zone instead of zonegroup
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: use endpoints from master zone instead of zonegroup

Fixes: http://tracker.ceph.com/issues/16834
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10396 from zhouruisong/master

rgw:add a s3 API of make torrent for a object

Merge pull request #10438 from ceph/wip-arm64

ARM64 fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Tested-by: tone zhang <tone.zhang@linaro.org>

Merge pull request #10418 from wjwithagen/wip-wjw-freebsd-rocksdb

cmake: pass cmake's compiler and flags to compile RocksDB into build

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9334 from yangdongsheng/rbd_clone_hiberit

rbd: inherit the parent image features when cloning an image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10087 from yangdongsheng/rbd_snap_remove_force

rbd: Allow user to remove snapshot with --force to auto flatten children

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9371 from ukernel/wip-osx-fuse

build ceph-fuse on OSX

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10436 from gregsfortytwo/wip-snapshot-doc

Add documentation about snapshots

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10399 from renhwztetecs/renhw-wip-mds-cleanup2

mds: unuse Class and cleanup

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10444 from SUSE/wip-drop-cephfs-mopup

man/Makefile-client.am: drop legacy cephfs tool

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

mds: unuse Class and cleanup

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

src/CMakeLists.txt: pass compiler and flags used to compile into RocksDB build

 - make sure the right compiler is used in CXX
 - NDEBUG if the current code is not being build with debugging

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #10433 from SUSE/wip-16808

deb: merge ceph-fs-common into ceph-common

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10342 from songbaisen/b6

logmon: check is_leader() before doing any work on get_trim_to()

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #10343 from xiexingguo/xxg-wip-pipe-2016-07-19

msg/simple: fix wrong condition checking of writing TAG_CLOSE on closing

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #10345 from tchaikov/wip-dump-cmdline

global/signal_handler: dump cmdline instead of arg[0]

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #10356 from stiopaa1/msg_message_changeConstructorsRemoveCodeDuplication

msg/Message.h: remove code duplication

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9290 from xiexingguo/xxg-wip-replicatedpg-cleaner

tools: do not closed stdout ; fix overload of "<" operator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9362 from xiexingguo/xxg-wip-texttable-df

tools: use TextTable for "rados df" plain output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9894 from gaowanlong/osd_recovery_max_omap_entries_per_chunk

osd: limit omap data in push op

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #10369 from ceph/wip-16715

msg/async/Event: ensure not refer to member variable which may destroyed

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #10358 from xiexingguo/xxg-wip-bluestore-2016-07-20

kv/MemDB: fix wrong output target and add sanity checks

Mark's Comments:

This passed Jenkins checks.
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #10349 from xiexingguo/xxg-wip-bluestore-2016-07-19

os/bluestore: add a boundary check of cache read

Mark's Comments:

This passed Jenkins checks.
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #10442 from weiqiaomiao/wqm-wip-rgw-getlc

rgw: return "NoSuchLifecycleConfiguration" if lifecycle config does not exist

Merge pull request #10446 from cbodley/wip-16817

rgw: clear realm watch on failed watch_restart
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10391 from ceph/wip-cmake

cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>

qa/qa_scripts/cephscrub.sh: do not try to remove ceph-fs-common package

The ceph-fs-common package is being dropped.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rgw: clear realm watch on failed watch_restart

if the realm is deleted while the gateway has a watch, it's disconnected
and watch_restart() will fail. this results in a watch handle of 0,
which leads to a segfault on ~RGWRealmWatcher when we pass it to
unwatch()

this commit cleans up the watch when watch_restart() fails, so we don't
try to unwatch() on destruction

Fixes: http://tracker.ceph.com/issues/16817
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10443 from wjwithagen/patch-13

src/CMakeLists.txt: remove double flag -Wno-invalid-offsetof

Reviewed-by: Kefu Chai <kchai@redhat.com>

man/Makefile-client.am: drop legacy cephfs tool

Somehow c76c31d312ce7a623acd99dbdedfe471c0cd445f missed this bit.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10401 from theanalyst/rgw/test-multi-nose-attr

rgw: test/multi.py add a destructive attr to tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: describe snapshots for developers, and warn about multi-fs

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

src/CMakeLists.txt: remove double flag -Wno-invalid-offsetof

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #10375 from xiexingguo/xxg-wip-fix-bmap-ut-failure

os/bluestore: use small encoding for bluefs extent and fnode

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: use small encoding for bluefs extent and fnode

Also change bdev of extent from uint16_t to uint8_t, which matches
the fnode prefer_bdev field better, and is good for efficiency.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test_librbd: add a test for newly introduced api of snap_remove2

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: introduce no-progress for rbd snap remove.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge pull request #10440 from tchaikov/wip-16686

msg/async: remove static member variable

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

deb: merge ceph-fs-common into ceph-common

Fixes: http://tracker.ceph.com/issues/16808
Signed-off-by: Nathan Cutler <ncutler@suse.com>

rgw: return "NoSuchLifecycleConfiguration" if lc config does not exist

return error code "NoSuchLifecycleConfiguration" if lifecycle configuration does not exist
as AmazonS3 latest API does.

Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>

rgw:add a s3 API of make torrent for a object

When you execute the command gettorrent of a object, a torrent file will be produced and returned.
The torrent also will be save into a pool named default.rgw.torrent.
If the torrent of a object exists in default.rgw.torrent, it will be returned.

Signed-off-by: zhouruisong <236131368@qq.com>

Merge pull request #10430 from tchaikov/wip-cephtool-test.sh

qa/workunits/cephtool/test.sh: use absolute path for TEMP_DIR

Reviewed-by: Loic Dachary <ldachary@redhat.com>

pybind: add remove_snap2 api in python binding

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

tools/rbd: introduce an option of --force for snap remove

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

librbd: fix a incorrect input for cpp_strerror()

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

librbd: coding style: fix the indent problem

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

librbd: coding style: add space after comma

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

librbd: introduce a new flag of RBD_SNAP_REMOVE_FLATTEN

introduce RBD_SNAP_REMOVE_FLATTEN for flags of Image::snap_remove2()
to auto flatten the children of snapshot which we want to remove.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

librbd: introduce a new api of snap_remove2

currently, we only have one api for snap_remove, but if we want to
pass more options about snapshot removal, that's impossible.

This patch introduce a new api of snap_remove2 here to solve
this problem.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge pull request #10423 from ceph/wip-16801

osd: fix the mem leak of RepGather

Reviewed-by: Samuel Just <sjust@redhat.com>

cmake: move flock.cc from libmds into libcommon

as we don't need to compile the whole mds just for building the
libcephfs

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async: remove static member variable

msgr is part of libcommon. and the later is linked into both libcephfs
and librados. and hence into cephfs.so and rados.so. that's why we have
double free.

Fixes: http://tracker.ceph.com/issues/16686
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10304 from ukernel/wip-16610

ObjectCacher: fix last_write check in bh_write_adjacencies()
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

rgw: add lc_pool when decode or encode struct RGWZoneParams

Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>

Update to upstream rocksdb

Signed-off-by: Dan Mick <dan.mick@redhat.com>

src/test/CMakeLists.txt: don't assume Intel (sse/sse2)

Fixes ARM64 build of ceph_perf_local test

Signed-off-by: Dan Mick <dan.mick@redhat.com>

erasure-code/jerasure/CMakeLists.txt: set NEON flags for neon objs

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #10243 from jcsp/wip-cephfs-cleanup

CephFS: misc. cleanups and remove legacy cephfs tool

Reviewed-by: Nathan Cutler <ncutler@suse.com>

mds: add assertion in handle_slave_rename_prep

This function assumes that the source path
does not point to a null dentry, and that
path_traverse is therefore setting the
passed inode.

Candidate code path for crash
in http://tracker.ceph.com/issues/16807

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #9716 from xiexingguo/xxg-wip-server_droppinlock2

Server: drop locks and auth pins if wait for pending truncate

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

os/bluestore: fix unittest_bluestore_types to properly handle different ref_map locations

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

qa/workunits/cephtool/test.sh: use absolute path for TEMP_DIR

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10261 from songbaisen/b1

mon: remove the redundant cancel_probe_timeout function

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: fix the mem leak of RepGather

ReplicatedPG::new_repop() returns a pointer to RepGather with two refcounts,
one is held by ReplicatedPG::repop_queue, the other is supposed to be
held by the caller of this function. but it's caller
ReplicatedPG::submit_log_entries() assigns it to a
boost::intrusive_ptr<RepGather>() directly, why by default add_ref() in
its constructor. this makes the refcount 3. that's why we have a leak of
RepGather in `ReplicatedPG::new_repop(ObcLockManager&&,
boost::optional<std::function<void ()>>&&)`.

Fixes: http://tracker.ceph.com/issues/16801
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: do not link against profiler if gperftools is found

this behaviour matches the autotools. and we should not enable profiling
unless asked to do so.

Fixes: http://tracker.ceph.com/issues/16804
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: cleanup Findgperftools.cmake

remove unused bits

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: cleanup Findudev.cmake

remove the crufts for handling the REQUIRED argument.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: cleanup Findblkid.cmake

* remove unused bits
* fix the wrong mark_as_advanced() call.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: fix libaio detection

* we used to check the header file of "aio.h" which is part of glibc,
  the glibc async i/o, and the corresponding library is librt. but what
  we are using is libaio.
* reference libaio using AIO_LIBRARIES for consistency.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: do not include undefind paths

for example
```
include_directories(${LEVELDB_PREFIX}/include)
```
does not make sense if `LEVELDB_PREFIX` is not defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: cleanup Findbabeltrace.cmake

* we don't need the BABELTRACE_PATH_HINT setting, as packaging script
  requires the the babeltrace pacakges explicitly.

Signed-off-by: Kefu Chai <kchai@redhat.com>