client/Makefile: ship InodeRef

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4753 from ceph/wip-client-inode-refs

client: hold reference for returned inode

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5466 from ceph/wip-12578

mds: open base inode's snaprealm after decoding snapblob

Reviewed-by: John Spray <john.spray@redhat.com>

mds: open base inode's snaprealm after decoding snapblob

base inode has no parent, so can we mark its snaprealm open
immediately. This change makes sure replica mdsdir's snaplream
is opened.

Fixes: #12578
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #5140 from tchaikov/wip-pg-ls-update-pending-rel-notes

doc: add changes to "pg ls*" commands to pending release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: add changes to "pg ls*" commands to pending release notes

"pg ls*" commands now accept recovering instead of recovery as
one of its states.

Fixes: #11569
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5337 from branch-predictor/bp-reduce-mallocs

Thread.cc: remove malloc/free pair

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5000 from ceph/wip-da-SCA-20150601

Various SCA and Coverity fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5312 from xinxinsh/wip-sparse-copy

use SEEK_HOLE/SEEK_DATA for sparse copy

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5420 from ceph/wip-12429

osd/OSDMap: handle incrementals that modify+del pool

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5431 from kylinstorage/test_cache_pool_script_fix

fix: qa/workunits/rados/test_cache_pool.sh

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5430 from ceph/wip-12537

rgw: url encode exposed bucket

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5400 from tchaikov/wip-12451

tools/rados: update the help message

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5440 from vumrao/wip-vumrao-12458-doc

doc: change --keyfile description in man page of rbd help

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: change "--keyfile" description in man page of rbd help

"--keyring" has higher priority than "--keyfile".
"--keyfile" option is overridden by "--keyring"
if latter is also specified.

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

Merge pull request #5455 from ceph/wip-cephfs-cmake

CMake fixes for CephFS

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5456 from kylinstorage/wip-do_autogen.sh

Allow do_autogen.sh pass configure parameters

Reviewed-by: Sage Weil <sage@redhat.com>

CMake: add crushtool

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: fix librados build

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: fix rbd build

Don't link with 'global' from rbd (to match
autotools update fa78739).  Fixes "g_ceph_context
defined twice" error.

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: libblkid not only needed with rbd

It's linked in libcommon.

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: smalliobenchrbd should dep on rbd

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: set WITH_CEPHFS by default

Option was referenced in CMakeLists but
not actually set.

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: fix cephfs cls build

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: fix cephfs test targets

These had bad lib names.

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: add cephfs tools

cephfs-table-tool, cephfs-data-scan, cephfs-journal-tool

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5422 from ceph/wip-fix-filestore-getattr

tests: fix the ObjectStore/StoreTest.SimpleCloneTest in "ceph_test_objectstore"

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5375 from ceph/wip-config-root-ino

Make UID of new root inos configurable

add help to do_autogen.sh

Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Reviewed-by: Li Wang <liwang@ubuntukylin.com>

Allow do_autogen.sh to pass configure parameters

Currently do_autogen.sh does not allow to pass parameters
of configure to disable/enable components during building, which is not
flexible, this patch add the ability to it.

Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Reviewed-by: Li Wang <liwang@ubuntukylin.com>

client: use smart pointer to track temporary inode reference

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: use smart pointer to track 'cwd' and 'root_parents'

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: convert Inode::snapdir_parent to smart pointer

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: convert CapSnap::in to smart pointer

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: convert Fh::inode to smart pointer

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: use smart pointers in MetaRequest

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: convert Dentry::inode to smart pointer

Signed-off-by: Yan, Zheng <zyan@redhat.com>

client: hold reference for returned inode

CLient::make_request() returns pointer of target inode, but it does not
increase reference count of the returned inode. The inode may get freed
when CLient::make_request() releases MetaRequest.

he fix is hold a reference for the returned inode. Since there are many
of places use CLient::make_request() directly or indirectly, it's easy
to leak reference. This patch uses intrusive_ptr to track the reference.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #5445 from ceph/wip-12432

Wip 12432

rgw: set http status in civetweb

Need to set the http status in civetweb so that we report it correctly.
Fixes: #12432
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

civetweb: update submodule to support setting of http status

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5258 from zhouyuan/rgw_skip_first_chunk

rgw: skip prefetch first chunk if range get falls to shadow objects

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc: krbd supports clones since 3.10

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #5438 from ceph/fix-doc-build

doc: fix the build of placement-groups.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/FileStore: getattr() should return 0 if success

* fix the test of ObjectStore/StoreTest.SimpleCloneTest, the expected
  return value of getattr() should be 0. otherwise it otherwise it fails
  on objectstore backends of "memstore" and "keyvaluestore".
* change FileStore::getattr() to return 0 on success
* tighten related tests in ceph_test_objectstore, to test the return value
  of getattr() using ASSERT_EQ(r, 0) instead of ASSERT_GE(r, 0)

Signed-off-by: Kefu Chai <kchai@redhat.com>

KeyValueStore: Fix incorrect rm_keys behaviors

GenericObjectMap will copyup parents keys when rm keys, but if
caller(KeyValueStore) already buffer "set keys", it shouldn't
copyup reset keys.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

doc: fix the build of placement-groups.rst

add a reference target of "mapping pgs to osds"

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5432 from ceph/wip-fix-rbd-build

make: fix the build of "rbd"

Reviewed-by: Loic Dachary <ldachary@redhat.com>

fix: qa/workunits/rados/test_cache_pool.sh

Signed-off-by: Mingxin Liu <mingxinliu@ubuntukylin.com>

Merge pull request #5434 from tobegit3hub/remove_execute_mode

Remove execute mode of source files

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Remove execute mode of source files

Merge pull request #5398 from mingyuez/master

rbd:If the image has any protected snaps,'rbd snap purge image' will return failure without removing any snaps.

Reviewed-by: Josh During <jdurgin@redhat.com>
Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

make: fix the build of "rbd"

`CEPH_GLOBAL` was removed in fa78739, but the linked libraries does not
offer all needed symbols for "rbd". so add them back.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd:'rbd purge image' will return failure without removing any snaps if the image has a protected snap

Signed-off-by: mingyuez <zhao.mingyue@h3c.com>

Merge pull request #5395 from tchaikov/wip-12512

common/syncfs: fall back to sync(2) if syncfs(2) not available

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5402 from ceph/wip-claim-sparse-read-buffer

osd/ReplicatedPG: claim the read buffer instead of copying it

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5428 from ceph/wip-kill-rest-bench

kill rest-bench

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rgw: skip prefetch first chunk if range get falls to shadow objects

Currently the head object will be prefetched in each GET:
 a) This is unnecessary if the Range GET falls to shadow objects.
 b) The GET request would be quite slow if we have a big head object
This patch adds some check on the Range. If it's not in the head
object(>=rgw_max_chunk_size) then skip the prefetch.

Fixes: #12539
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Merge remote-tracking branch 'origin/next'

Merge pull request #5425 from ceph/wip-12479-next

librbd: test crash during client upgrade suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: url encode exposed bucket

Fixes: #12537
Don't send the bucket name back without url encoding it.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5423 from oddomatik/patch-1

doc: No CRUSH involvement in deciding PG target

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5426 from ceph/wip-12383-next

rbd: remove dependency on non-ABI controlled CephContext

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5399 from tchaikov/wip-12458

rbd: add "--keyring" option to help message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5414 from cxwshawn/master

doc: add bucket object version description.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

remove libs3 submodule

Signed-off-by: Sage Weil <sage@redhat.com>

remove rest-bench

This is a weak tool; users should look to cosbench or others instead.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5419 from ceph/wip-kfsx-invalidate

test_librbd_fsx: invalidate before discard in krbd mode

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

No CRUSH involvement in deciding PG target

Quoting from elsewhere "[we calculate] a placement group using the object name, a hash code, the number of PGs in the pool and the pool name"

osd/OSDMap: handle incrementals that modify+del pool

Because new_pools was processed after old_pools, if something
was modified and then deleted in the same incremental
map, then the resulting state would be a pool in the map
that had no entry in pool_name.

Fixes: #12429
Signed-off-by: John Spray <john.spray@redhat.com>

lockdep: allow lockdep to be dynamically enabled/disabled

librbd test cases attempt to enable lockdep coverage via the librados
API.  Use a configuration observer to register/unregister lockdep
support.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: librbd API test cannot use private md_config_t struct

Remove all depencencies on md_config_t and instead use librados API
methods to get/set configuration values.

Fixes: #12479
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados_test_stub: implement conf get/set API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: add "--keyring" option to help message

Refs: #12458
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5401 from ceph/wip-12450

tools/rados: change the default max_ops to 16

Reviewed-by: David Zafman <dzafman@redhat.com>

doc: add bucket object version description.
bucket object version has been supported, but do not have description in the docs,
so add this part.

Signed-off-by: shawn chen <cxwshawn@gmail.com>

test_librbd_fsx: invalidate before discard in krbd mode

Commit bd050240ceef ("test_librbd_fsx: flush before discard in krbd
mode") added an fsync() before BLKDISCARD.  Don't know what I was
thinking at the time, but I missed the invalidate part, for which we
need to use the BLKFLSBUF ioctl.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Log::reopen_log_file: take m_flush_mutex

Otherwise, _flush() might continue to write to m_fd after it's closed.
This might cause log data to go to a data object if the filestore then
reuses the fd during that time.

Fixes: #12465
Backport: firefly, hammer
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 8778ab3a1ced7fab07662248af0c773df759653d)

Merge pull request #5348 from athanatos/wip-12465

log: take mutex when reopening log fd

Reviewed-by: Sage Weil <sage@redhat.com>

tools/rados: change the first op id to 0

the first op id was 16 by default, which is okay, but a non-zero
magic number could lead to questions. max_op was mixed up with
max_ops, and changed to 16 in 51e402e3 by mistake.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rados: change the default max_ops to 16

otherwise the load-gen will not emit any ops by default.

Fixes: #12450
Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rados: update the help message

Fixes: #12451
Signed-off-by: Kefu Chai <kchai@redhat.com>

common/syncfs: fall back to sync(2) if syncfs(2) not available

Fixes: #12512
Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd: remove dependency on non-ABI controlled CephContext

The rbd CLI tool no longer attempts to initialize a CephContext
and pass said context to librados since it's possible that the
structure will not be ABI compatible between rbd and librados.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5139 from jmunhoz/wip-7817-ms

rgw: add minimum support for copy multipart part

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5313 from guce/rgw_access_invalid_argument

rgw: check subuser illegal access parameter.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

crypto: use NSS_InitContext/NSS_ShutdownContex to avoid memory leak

Switched to context-aware NSS init/shutdown functions to avoid conflicts
with parent application.  Use a reference counter to properly shutdown the
NSS crypto library when the last CephContext is destroyed.  This avoids
memory leaks with the NSS library from users of librados.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5364 from xdonghai/master

doc:modified librados-intro.rst an error of document representation

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5191 from theanalyst/v9.0.2/mailmap

doc: mailmap for v9.0.2 release

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5130 from tchaikov/wip-12167

ceph-detect-init: do not require "argparse" on py2.6

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5338 from hjwsm1989/wip-12459

common/WorkQueue: fix the FTBFS with GCC 4.4.7

* add nested-name-specifier ThreadPool before WorkQueueVal

Reviewed-by: Kefu Chai <kchai@redhat.com>

tools/cephfs: respect mds_root_ino_[gid|uid]

...while injecting inodes.  So that if you're a cephfs
admin but not uid=0 on the client, you can get things
injected as your choice of uid/gid.

Definitely useful for dev/test, potentially useful
elsewhere too.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: configurable uid/gid on new root inos

This is to deal with the slightly perverse situation
where a user might have all the permissions they need
to create a filesystem and mount it with fuse, but then
be unable to do anything inside the mount because
we default to creating a root-owned "/" inode.

This is immediatley useful for dev/test environments without
root, but also potentially in production environments
where the filesystem configuration/mounting is done by
someone less privileged than root.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5350 from ceph/wip-pgls-filter-enc

osdc: fix pgls_filter op encoding

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5355 from ceph/wip-anotate-s-files

Anotate all the .s files

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5116 from yuyuyu101/wip-compressor

Wip compressor

Reviewed-by: Sage Weil <sage@redhat.com>

modified librados-intro.rst

Signed-off-by: x11507 <xu.donghai@h3c.com>

Merge pull request #5254 from vuhuong/wip-xio

@vuhuong sorry for the latency, could you please rebase & repush , to poke the built bot again?

once the build completes, will get it merged.

thanks,

osd/ReplicatedPG: claim the read buffer instead of copying it

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5349 from yuyuyu101/wip-async-fix-21

AsyncConnection: Exit process loop if entering fault