rgw: swift: The http referer should be parsed to compare in swift API
The http referer should be parsed to compare with the url set on the container read acl. If we set .r:www.example.com on container read acl, we should parse the hostname 'www.example.com' of the http referer like 'http://www.example.com' from the http request.

Fixes: http://tracker.ceph.com/issues/18685
Signed-off-by: Jing Wenjun <jingwenjun@cmss.chinamobile.com>

Merge pull request #12778 from badone/wip-check_new_interval-logging

osd/osd_type: Fix logging output

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12671 from badone/wip-ceph_test_rados_api_cmd-valgrind-fixes

test/librados/cmd.cc: Fix trivial memory leaks

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/osd_type: Fix logging output

Logging should reflect the actual function name.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12656 from badone/wip-c_read_operations-leak

test/librados/c_read_operations.cc: Fix trivial memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12442 from badone/wip-coverity-librados-TestCase

test/librados: Silence Coverity memory leak warnings

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12690 from badone/wip-ceph_test_rados_api_snapshots-valgrind-fixes

test/librados/snapshots.cc: Fix memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12770 from yuriw/wip-fix-distro-master

qa: fixed distros links

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: fixed distros links

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #11787 from SUSE/wip-17802

doc: clarify "ceph quorum" syntax

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12385 from dlan17/master

cmake: explictly call find_package(PythonInterp) first to fix build err

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #12658 from oritwas/wip-rgw-17239

rgw: clear master_zonegroup when reseting RGWPeriodMap

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12723 from gaurav36/rbd-remove-unused-condition

rbd: remove unused condition within group action handler

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12509 from SUSE/wip-18262

doc: add verbiage to rbdmap manpage

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12667 from trociny/wip-librados_test_stub-tmap_udate

test: librados_test_stub: tmap_update: return -ENOENT when removing nonexisent key

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12712 from liewegas/wip-18366

mon/OSDMonitor: set last_force_op_resend on overlay pool too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12717 from hrchu/patch-1

doc: Update keystone.rst

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Update keystone.rst

Make it clear that project/domain options accept name instead of id.

Signed-off-by: Chu, Hua-Rong <hrchu@cht.com.tw>

Merge pull request #12734 from liupan1111/wip-fix-nvme-driver-compile-issue

spdk: upgrade spdk to v16.12

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12751 from tchaikov/wip-18395

ceph_test_rados_api_watch_notify: move global variables into test class

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12730 from Liuchang0812/fix-dev-doc-pglog

doc/dev: update log_based_pg.rst, fix some display problem

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph_test_rados_api_watch_notify: move global variables into test class

so the class instances using mempool are destructed before the mempools
are destroyed. this avoids the "invalid read" issue reported by
valgrind.

Fixes: http://tracker.ceph.com/issues/18395
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12713 from badone/wip-ceph_test_rados_api_watch_notify-valgrind-fixes

test_rados_watch_notify: Fix trivial memory leaks

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12680 from badone/wip-ceph_test_rados_api_misc-valgrind-fixes

ceph_test_rados_api_misc: Fix trivial memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/dev: update log_based_pg.rst, fix some display problem

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

os/bluestore: add more useful log information in NVMEDevice.cc

Signed-off-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12747 from SUSE/wip-18388

tests: rbd/test_lock_fence.sh: fix rbdrw.py relative path

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests: rbd/test_lock_fence.sh: fix rbdrw.py relative path

This commit fixes a regression introduced by
cf294777ea92f0911813a7132068584d4f73a65a

Fixes: http://tracker.ceph.com/issues/18388
Signed-off-by: Nathan Cutler <ncutler@suse.com>

os/bluestore: fixed compilation issue when use spdk v16.12

Signed-off-by: Pan Liu <pan.liu@istuary.com>

spdk: fixed the compilation error for testcases when use spdk v16.12

Signed-off-by: Pan Liu <pan.liu@istuary.com>

dpdk: make compilation successful when use dpdk v16.11

Signed-off-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12706 from Liuchang0812/wip-qa-rados-1228

test: update test_rados_tool.sh, use POOL and OBJ var

Reviewed-by: Kefu Chai <kchai@redhat.com>

spdk: upgrade spdk version to 16.12

Signed-off-by: Pan Liu <pan.liu@istuary.com>

dpdk: upgrade dpdk version to 16.11

Signed-off-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12716 from kylinstorage/wip-ctypos_h

misc: fix code typos in header files

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12721 from chendave/wip-authx-options

auth: Let's not use the deprecated cephx option

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12676 from chendave/master

common/ceph_context: Show clear message if all features are enabled

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12633 from liewegas/wip-15784

msg/simple: call clear_pipe in wait() shutdown path

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12692 from XinzeChi/wip-bluestore-statfs

os/bluestore: _txc_release_alloc when do wal cleaning

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12691 from runsisi/wip-dump-osdop

osd/OpRequest: dump both name and addr for the client op

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12363 from wjwithagen/wip-wjw-freebsd-strerror

common/module.c: do not use strerror_r the GNU way.

Reviewed-by: Sage Weil <sage@redhat.com>

rbd: remove unused condition

Signed-off-by: Gaurav Kumar Garg <garg.gaurav52@gmail.com>

cephx: Let's not use the deprecated option

Enable cephx by setting the option of "auth supported" has been
deprecated for a long time, it's better to use the recommended
options instead.

Signed-off-by: Dave Chen <wei.d.chen@intel.com>

Merge pull request #12718 from kylinstorage/wip-orig

organizationmap: add Xianxia Xiao to Kylin Cloud team

organizationmap: add Xianxia Xiao to Kylin Cloud team

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

src/tools/rbd_mirror/Replayer.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/messages/MMDSSlaveRequest.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/tools/rbd_recover_tool/epoch_h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/librados/IoCtxImpl.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/journal/JournalTrimmer.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/common/hobject.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

Merge pull request #12652 from badone/wip-coverity-TracepointProvider

tracing: Fix handle leak in TracepointProvider

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tracing: Fix handle leak in TracepointProvider

Silences Coverity 1397733

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

test_rados_watch_notify: Fix trivial memory leaks

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12711 from yuyuyu101/wip-fix-rdma

msg/async/rdma: fix ceph_clock_now calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12451 from liewegas/wip-osd-on-applied

osd: never send rados ack (only commit)

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12480 from dx9/wip-boost-python

cmake: fix hard coded boost python lib

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12406 from songbaisen/s5

tool: add some ceph relate processes to ps-ceph.pl

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12621 from fiskn/master

doc: Update disk thread section to reflect that scrubbing is no longe…

mon/OSDMonitor: set last_force_op_resend on overlay pool too

We currently set the last_force_op_resend field on the
base pool when we set or clear the overlay.  Set it on
the cache/overlay pool too.  The Objecter should resend
even with a change only to the base pool, but the OSD
needs to see the change on the overlay pool to correctly
discard the op.

Fixes: http://tracker.ceph.com/issues/18366
Signed-off-by: Sage Weil <sage@redhat.com>

test: update test_rados_tool.sh, use POOL and OBJ var

* replace hard-code pool name with $POOL
* replace hard-code object name with $OBJ
* introduce a new variable called $POOL_EC
* clean up pool
* simplify test case

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

msg/async/rdma: fix ceph_clock_now calls

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12403 from liewegas/wip-mon-mgr-cleanups

mon: clean up some osdmon/pgmon interactions

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits: remove crash_replay_interval tests

Signed-off-by: Sage Weil <sage@redhat.com>

doc: remove crash_replay_interval mentions

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MDSMonitor: no need to set crash_replay_interval on data pools

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: remove waiting_for_ack

No longer used.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: simplify dup/resent logic

No need to special ack path.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: remove replay_queue

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: do not go into replay state

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: do not queue ops for replay

We no longer send ACKs, so this can't happen anyway.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: we can discard "replay" ops

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: do not send ACK separate from COMMIT

This never comes before COMMIT, making it useless to clients.  And it's
just unnecessary overhead.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: remove 'rlat' read latency metrics

These are measuring how long of a delay we see before ops are readable.
They are calculated in the on_applied path, which we'd like to remove,
and weren't reliably calculated anyway.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: kill hit_set_flushing

This was added before we started taking the ondisk lock in order to allow
reading a just-written hitset.  Since this is a rare path anyway, just
take the ondisk lock.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12522 from hjwsm1989/ecwrite

osd/ECBackend: Send write message to peers first, then do local write

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12524 from yuyuyu101/wip-bluestore-flush

os/bluestore: prevent lock for almost "flush" calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12528 from kylinstorage/wip-JournalingObjectStore-cleanup

os/filestore/JournalingObjectStore cleanup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12670 from liewegas/wip-nits

mon: fix a few nits

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12684 from xiexingguo/xxg-wip-fix-onodesizetracking

os/bluestore: fix OnodeSizeTracking testing

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12707 from tchaikov/wip-dpdk-v16.07

builds with dpdk v16.07

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async: bump dpdk to v16.07

as some distros include it already. so sync it up to avoid building
failures on them with dpdk-dev package(s) shipped by distro.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async: be compatible with dpdk v16.07

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12698 from yuyuyu101/wip-dpdk-fix

msg/async/dpdk: fix compile errors

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12699 from wjwithagen/patch-15

qa/test_rados_tool.sh: POSIX dd only accepts 'k' as multiplier

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/librados/snapshots.cc: Fix memory leak

Fix for valgrind reported leak of librados::ObjectWriteOperation heap
object.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

osd/OpRequest: dump both name and addr for the client op

Signed-off-by: runsisi <runsisi@zte.com.cn>

mon/OSDMonitor: remove workaround for dumpling-era bug

We will definitely have published (and stored) a new osdmap since then.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12703 from kylinstorage/wip-remove-unused-variable

os/bluestore/BlueStore.cc: remove unused variable

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12594 from wjwithagen/wip-wjw-freebsd-rebind-timing

common/config_opts.h: FreeBSD timing changed due to no SO_REUSEADDR

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueStore.cc: remove unused variable

Signed-off-by: Li Wang <li.wang@kylin-cloud.com>

qa/test_rados_tool.sh: POSIX dd only accepts 'k' as multiplier

FreeBSD does not want KB as 1k multiplier.
see also: http://www.unix.com/man-page/posix/1posix/dd/

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

test_userspace_event: fix missing std

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/dpdk/TCP: remove unneeded CephContext argument

Signed-off-by: Haomai Wang <haomai@xsky.com>

os/bluestore: _txc_release_alloc when do wal cleaning

Signed-off-by: Xinze Chi <xinze@xsky.com>

Merge pull request #12688 from Adirl/clean_line_end

msg/async/rdma: clean line endings

Reviewed-by: Haomai Wang <haomai@xsky.com>

os/bluestore: fix OnodeSizeTracking testing

[  FAILED  ] 1 test, listed below:
[  FAILED  ] ObjectStore/StoreTest.OnodeSizeTracking/2, where GetParam() = "bluestore"

 1 FAILED TEST

The above test failure happens as the bluestore mount() process
will try to load all collections and put them into the coll_map,
which will be also tracked as mempool::bluestore_meta_other.
So total_bytes from mempool won't be equal to zero.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

common/ceph_context: Show clear message if all features are enabled

It's saying the following dangerous and experimental featuresare enabled
while the value is "*", it's not quite clear what's "*" stands for.

The patch updates the message so that it's more clear when "*" is used
in place.

Signed-off-by: Dave Chen <wei.d.chen@intel.com>

Merge pull request #12336 from kylinstorage/wip-cleanup-use-string-ref

osd: cleanup: use string & to avoid unnecessary copy

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12569 from liewegas/wip-bluestore-raw-unused

os/bluestore/bluestore_types: drop std::bitset for blob unused

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>