rgw: do not include "expat.h" in heade file

Building Ceph without the Expat development library currently results in
the following build error:

~/ceph> ./configure --without-fuse --without-tcmalloc --without-radosgw
...
~/ceph> make
...
CXX ceph_dencoder-rgw_dencoder.o
In file included from rgw/rgw_dencoder.cc:6:0:
rgw/rgw_acl_s3.h:9:19: fatal error: expat.h: No such file or directory

The ceph-dencoder binary is built with rgw_dencoder.cc, which includes
rgw_acl_s3.h, which in turn includes expat.h. However, configure.ac
currently only checks for Expat as part of radosgw.

but we don't need expat.h for testing the encoding of rgw structs, so
move "#include <expat.h>" into the source file to avoid this dependency.
and as a side effect, this also speed up the compilation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6813 from manens/pybind_rados_snapshot_read

pybind: Implementation of Ioctx.set_read to allow read from snapshots

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7445 from xiexingguo/xxg-wip-mem

osd: memstore: fix blocksize for statfs

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: fix typo (again)

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7400 from yuyuyu101/wip-13938

librbd: Avoid create two threads per image

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7409 from dillaman/wip-librbd-watch-notify

tests: improve simulation of watch/notify handling

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6562 from Aran85/authedpermcheck

rgw: fix the condition of  authenticated user's permit check .

Merge pull request #7291 from majianpeng/vstart.sh

vstart.sh: If use '-n' delete related source rather than consider '-k'.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7397 from tchaikov/wip-kill-warnings

build: kill warnings

Reviewed-by: Sage Weil <sage@redhat.com>

civetweb: update submodule to silence a compile warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

common,osd: remove _process(T *t) to silence warnings

* and mark the WorkQueue::_process(T *, TPHandle&) a pure virtual
  method. so we must override it
* and mark all override methods with the "override" specifier.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: fix the condition of authenticated user's permit check

before fix:
put a bucket under one user with authenticated-read permit
> ./s3curl.pl http://myobs.com/mybucket --put --H "x-amz-acl: authenticated-read"
GET the bucket with anonymous user
> curl http://myobs.com/mybucket -X GET

> successed!

after fix:
anonymous cant GET the authenticated-read bucket
other users who authenticated could GET it

Signed-off-by: Zengran Zhang <zhangzengran@h3c.com>

Merge pull request #7266 from sahithi-rv/master

common/str_map: cleanup: replaced get_str_map() function overloading by using default parameters for delimiters

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7344 from songbaisen/song8

crush: add safety assert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7363 from cxwshawn/mon-sync-fix

mon: add mon_config_key prefix when sync full

Fixes: #14577
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7383 from liewegas/wip-bluestore

osd: bluestore: some new c++11-isms, a couple reverts

cmake: fix "jobserver unavailable" warning

use ${MAKE} so the make command can be properly propagated to their
children

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: disable unused-variable warning for rocksdb

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: fix the signed/unsigned comparison warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

rados: bench: fix the signed/unsigned comparison warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: remove duplicated HAVE_EXECINFO_H macro

fixes the `"HAVE_EXECINFO_H" redefined` warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/*stores: fix hidden warnings

the CollectionHandle variant methods are hidden by the subclasses
who do not override it. so the compiler complains. by adding
"using" statement we expose them again.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix unsigned/signed comparision warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7014 from efirs/ef_replace_coll_t_by_const_coll_t_ref

osd: reduce CPU usage by replacing coll_t by const coll_t&

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7333 from xdonghai/xdh1

osd: FileStore: modify the format of colon

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7392 from tianshan/wip-14536

rados: fix race condition in load-gen complete callback

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #7395 from xiexingguo/xxg-wip-fuse

os/fusestore: add error handling

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7396 from tchaikov/wip-14537

mon: compact full epochs also

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7402 from liewegas/wip-rados-bench

rados: make rados bench metadata backward compatible

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7418 from renhwsky/renhw-wip-monmapmonitor-cleanup

mon: some cleanup in MonmapMonitor.cc

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #7420 from mikulely/fix-osd-mon-exit-issue

osd, mon: fix exit issue

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7430 from xiexingguo/xxg-wip-kstore

osd: kstore: fix decode logic and add flush

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7432 from branch-predictor/bp-fix-read-bench

rados: fix wrong op/object sizes in rand/seq bench

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7436 from liewegas/wip-bash

tests: qa/workunits/objectstore/test_fuse.sh: no bashism

os/memstore: return empty list if offset overflows

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/memstore: fix wrong blocksize for statfs

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7404 from sahithi-rv/Mutex

rbd_fuse: replaced pthread_mutex*() functions with ceph Mutex wrapper

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tools/rados/rados.cc: fix race condition in load-gen complete callback

Fixes: #14536
Signed-off-by: Tianshan Qu <tianshan@xsky.com>

Merge pull request #6131 from rzarzynski/wip-admincreds-in-s3-keystone

rgw: support admin credentials in S3-related Keystone authentication.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: support admin credentials in S3-related Keystone authentication.

Fixes: #13302
Backport: infernalis, hammer
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #7298 from racpatel/wip-doc-rac2

doc :- fixing image in section ERASURE CODING

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph_osd.cc: fix unreachable flush call

The calling chain for generic_server_usage():

  generic_server_usage()
    ->generic_usage(true)
    ->exit(1)
  cout.flush()

any statements after generic_server_usage() would not be reached,so we
need to flush cout in generic_usage().

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

cmake: musl libc does not implement backtrace. Added feature check and fallback to libexecinfo.

Signed-off-by: John Coyle <dx9err@gmail.com>

Merge pull request #7264 from ceph/wip-doc-RGW

doc: rgw: port changes from downstream to upstream

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7244 from mikulely/fix-authtool-doc

doc: remove redundant space in ceph-authtool/monmaptool doc

doc: Updated the RPM section for RHEL and CentOS

Fixes: #10245
Signed-off-by: Aron Gunn <ritz_303@yahoo.com>

Merge pull request #7027 from mathslinux/fix-s3-csharp-doc

doc: fix S3 C# example

Merge pull request #7000 from zhouyuan/fix_cache_tier_doc

doc: fix misleading configuration guide on cache tiering

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6930 from wido/rgw-key-docs

doc: use 'ceph auth get-or-create' for creating RGW keyring

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7384 from wido/mirroring

doc: script and guidelines for mirroring Ceph

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7435 from xiexingguo/xxg-wip-14561

osd: BlueStore: fix null pointer access

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7271 from somnathr/wip-transaction-delete

osd: avoid heap allocations for ObjectStore::Transaction

http://pulpito.ceph.com/sage-2016-01-28_12:50:51-rados-wip-sage-testing3---basic-mira/

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6440 from stiopaa1/scrubbingInfo

mon: warn if pg(s) not scrubbed

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #7075 from shun-s/shun-fix

global: do not start two daemons with a single pid-file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7111 from chenji-kael/master

osd: FileStore: print file name before osd assert if read file failed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7268 from ifed01/wip-14400

common: snappy decompressor may assert when handling segmented input bufferlist

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7276 from caibo2014/fix-messager-null

mon/MonClient: avoid null pointer error when configured incorrectly

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7330 from ifed01/wip-pool-stat-cleanup

librados: remove duplicate definitions for rados pool_stat_t and cluster_stat_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7340 from ceph/wip-portable-time

common: ake ceph_time clocks work under BSD

Reviewed-by: Erwan Velu <erwan@redhat.com>

Merge pull request #7341 from songbaisen/song7

mon: consider the pool size when setting pool crush rule

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7355 from yuanjingkai/master

os/fs: fix io_getevents argument

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7348 from xiexingguo/xxg-wip-14484

osd: BlueStore: fix space leak

http://pulpito.ceph.com/sage-2016-01-27_18:39:48-rados-wip-sage-testing---basic-smithi/

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7368 from somnathr/wip-jemalloc-build

build: allow jemalloc enabled build with rocksdb-static

http://pulpito.ceph.com/sage-2016-01-27_18:39:48-rados-wip-sage-testing---basic-smithi/

Reviewed-by: Sage Weil <sage@redhat.com>

BlueStore: fix null pointer access

Fixes: #14561
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/workunits/objectstore/test_fuse.sh: no bashism

Signed-off-by: Sage Weil <sage@redhat.com>

include/str_map.h removed unnecessary function overloading of
get_str_map()

Signed-off-by: Sahithi R V <sahithi.rv1@gmail.com>
tests successful

Signed-off-by: Sahithi R V <sahithi.rv1@gmail.com>

BlueStore: fix typo

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rbd_fuse:replaced pthread_mutex*() functions with ceph Mutex wrapper

Signed-off-by: Sahithi R V <tansy.rv@gmail.com>

kstore: flush before we really start a clone

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

kstore: flush before we really start a truncate

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: add fast check against empty list

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

kstore: add fast check against empty list

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

tools/rados: fix wrong op/object sizes in rand/seq bench

Metadata contain correct information about last bench write run, yet
these are ignored in subsequent bench rand/seq runs, resulting in
wrong perf numbers and failures when not using --no-verify.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

kstore: simplify open_collection logic a bit

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

kstore: fix unmatched type of decode

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph_osd.cc/ceph_mon.cc: cleanup unreachable exit call

The calling chain of usage() is:

  usage()
    ->generic_server_usage()
      ->exit(1)
  exit(0)

so the exit(0) after usage() would not be reached.

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

mon:some cleanup in MonmapMonitor.h

delete unused "get_monmap(MonMap &m)"

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mon:some cleanup in MonmapMonitor.cc

remove the processing of  ”mon add“ && “mon remove” in the preprocess_command;
delete unused "get_monmap(MonMap &m)"

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #7312 from jcsp/wip-14365

mds, client: fix locking around handle_conf_change

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge remote-tracking branch 'origin/greg-fs-testing-126'

Merge remote-tracking branch 'origin/jewel'

Merge pull request #7297 from ukernel/jewel-13903

client: properly trim unlinked inode

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7262 from ukernel/jewel-14380

Be more careful about the "mds setmap" command to prevent breakage.

Reviewed-by: John Spray <john.spray@redhat.com>

librbd: use task finisher per CephContext

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7369 from mauri/fix-message-typo

rbd: fix error message typo

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librados_test_stub: watch/notify now behaves similar to librados

Notifications are executed via the same librados AIO callback
thread, so it's now possible to catch deadlock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

FuseStore: fix mountpoint leak

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

FuseStore: return error if encounter collection_list error

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

FuseStore: fix unhandled error cases for open

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

FuseStore: fix memory leak

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7387 from theanalyst/fix/gitignore

remove unused src/messages/#MOSDOp.h# file

Reviewed-by: Kefu Chai <kchai@redhat.com>

tests: simulate writeback flush during snap create

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7345 from x11562/rbd_snap_rename

rbd: add "--dest-snap" optional for snap rename

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7393 from tchaikov/wip-cmake

fix the autotools and cmake build (the new fusestore needs libfuse)

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7403 from rzarzynski/wip-rgw-dumpconlen-cosmetics

rgw: cosmetic improvements in dump_content_length()

rgw: cosmetic improvements in dump_content_length().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rados: make rados bench metadata backward compatible

We recently made a distinction between op size and object size, and broke
our ability to read metadata from old runs.  Change our format to be
backward compatible.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: disable fuse support if !HAVE_LIBFUSE

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: disable FuseStore if !HAVE_LIBFUSE

Signed-off-by: Kefu Chai <kchai@redhat.com>