Merge pull request #8074 from tchaikov/wip-ceph-detect-init-test

ceph-detect-init: add debian/jessie test

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph-detect-init: return None at seeing unknown debian distro

* return None at seeing unknown debian distros
* add test for this accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8046 from xiexingguo/xxg-wip-fixpc

msg: async: fix perf counter description and simplify _send_keepalive_or_ack

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7949 from wjin/fix

Event: fix clock skew problem

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8076 from ceph/wip-rgw-ldap

Wip rgw ldap

This follows a revert of the previous PR. This branch has some build related fixes.

Merge pull request #8075 from ceph/revert-7985-rgw-token-7

Revert "rgw ldap"

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Revert "rgw ldap"

Merge pull request #8066 from yehudasa/wip-rgw-swift-versioning-disabled

Wip rgw swift versioning disabled

Reviewed-by: Josh Durgin <joshd@redhat.com>

Merge pull request #7985 from linuxbox2/rgw-token-7

rgw ldap (Matt Benjamin)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge remote-tracking branch 'origin/master' into wip-rgw-ldap

test: disable test_rgw_ldap

for now

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw/Makefile.am: add a missing .h file

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_ldap:  merge cleanups

* match LDAPHelper op return codes to RGW (-POSIX)
* make temporary LDAP handle local (MT-safe)
* add required --encode and --ttype to usage
* use versioned encoding macros

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: require openldap library client headers

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: free components on shutdown

Also, use ldout in rgw_file.h in 3 places.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: LDAP pass-through authentication

Implement a new external authenticator based on LDAP and the
new external token format.

External LDAP auth now works, at least with openldap/X.500 style
naming and ldaps:// (SSL).

The latter is AD-friendly, but since AD uses dnattr=cn (IIRC)
everywhere, AD will need testing.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge remote-tracking branch 'gh/jewel'

Merge branch 'wip-15086' of git://github.com/SUSE/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8068 from SUSE/wip-15104

qa/workunits/rbd: use POSIX function definition

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7663 from javacruft/wip-pybind-flags

pybind: Ensure correct python flags are passed

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7856 from jdurgin/wip-cython-cleanup

configure.ac: update help strings for cython

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge remote-tracking branch 'origin/wip-josh-testing'

Merge pull request #8067 from athanatos/wip-fix-filestore-warning

FileStore: fix initialization order for m_disable_wbthrottle

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa/workunits/rbd: use POSIX function definition

In scripts with /bin/sh shebang, we must use POSIX function definitions to
avoid syntax errors on platforms that link /bin/sh to something other than
/bin/bash.

See https://wiki.ubuntu.com/DashAsBinSh#function for more information.

http://tracker.ceph.com/issues/15104 Fixes: #15104

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rgw: disable swift versioning by default

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: improve naming of Swift versioned object's copies.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: dump X-Versions-Location HTTP header of Swift API.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: copy swift versioned object to destination if overwritten

Only allow if target bucket owner is the source bucket owner.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: swift object versioning data structure changes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8034 from SUSE/wip-15047

packaging: make infernalis -> jewel upgrade work

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

debian: make infernalis -> jewel upgrade work

Attempts to install jewel ceph-common, ceph-mon, ceph-osd, and ceph-base
package over infernalis ceph package fail due to files existing in both.

See comment #4 in the tracker issue for a deeper analysis.

http://tracker.ceph.com/issues/15047 Fixes: #15047

Signed-off-by: Nathan Cutler <ncutler@suse.com>

FileStore: fix initialization order for m_disable_wbthrottle

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph-detect-init: return systemd on Debian Jessie

http://tracker.ceph.com/issues/15086 Fixes: #15086

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Revert "test/time: no need to abs(uint64_t) for comparing"

This reverts commit da0f6608dd6cc3356e727e739575f496c2b6a590.

unittest_time fails:

[==========] Running 4 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 2 tests from RealClock
[ RUN      ] RealClock.Sanity
[       OK ] RealClock.Sanity (0 ms)
[ RUN      ] RealClock.Conversions
test/common/test_time.cc:125: Failure
Expected: ((Clock::from_double(bd) - brt).count()) < (30U), actual: 18446744073709551595 vs 30
[  FAILED  ] RealClock.Conversions (0 ms)
[----------] 2 tests from RealClock (0 ms total)

[----------] 2 tests from CoarseRealClock
[ RUN      ] CoarseRealClock.Sanity
[       OK ] CoarseRealClock.Sanity (0 ms)
[ RUN      ] CoarseRealClock.Conversions
test/common/test_time.cc:125: Failure
Expected: ((Clock::from_double(bd) - brt).count()) < (30U), actual: 18446744073709551595 vs 30
[  FAILED  ] CoarseRealClock.Conversions (0 ms)
[----------] 2 tests from CoarseRealClock (0 ms total)

[----------] Global test environment tear-down
[==========] 4 tests from 2 test cases ran. (0 ms total)
[  PASSED  ] 2 tests.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] RealClock.Conversions
[  FAILED  ] CoarseRealClock.Conversions

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8063 from gregsfortytwo/wip-fix-fsmap-decode

Wip fix fsmap decode

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7752 from somnathr/wip-odsync-write

FileStore: Added O_DSYNC write scheme

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

FSMap: actually include the legacy MDSMap in Filesystems when upgrading

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

FSMap: decode old MDSMaps properly

We accidentally left a few members out that the MDSMap had!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8058 from yehudasa/wip-rgw-clean-log

cleanup

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

rgw_ldap:  merge cleanups

* match LDAPHelper op return codes to RGW (-POSIX)
* make temporary LDAP handle local (MT-safe)
* add required --encode and --ttype to usage
* use versioned encoding macros

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: silence compilation warnings

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cleanup of debug messages

use ldout instead of derr and lderr and increase logging level

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #7944 from trociny/wip-rbd-mirror-image-replayer-async

rbd-mirror: ImageReplayer async start/stop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8028 from dillaman/wip-librbd-journal-replay

librbd: replaying a journal op post-refresh requires locking

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7483 from liewegas/wip-osd-boot

osd: resolve boot vs NOUP set + clear race

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8030 from yehudasa/wip-rgw-sync-status

adds the radosgw-admin sync status command that gives a human readable status of the sync process at a specific zone

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #7615 from ceph/wip-14678

rgw: store system object meta in cache when creating it

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #8037 from yehudasa/wip-bucket-link

rgw:bucket link now set the bucket.instance acl (bug fix)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8013 from yehudasa/wip-rgw-multisite-fixes

rgw multisite fixes

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #6953 from jcsp/wip-fsmap

Multi-filesystem support

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7465 from liewegas/wip-promote-throttle

osd: probabilistic cache tier promotion throttling

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge tag 'v10.0.5'

v10.0.5

Conflicts:
debian/changelog

v10.0.5

Signed-off-by: Sage Weil <sage@redhat.com>

debian/changelog: Remove stray 'v' in version

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 0f7730c8f444657d4aaffa28c12bbcd602a2e3b8)

msg: async: improve _send_keepalive_or_ack() a little

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

msg: async: fix connection perf counter description

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge branch 'master' of github.com:ceph/ceph

Merge branch 'debian-stretch-dep-fix' of git://github.com/rldleblanc/ceph

Reviewed-by: Sage Weil <sage@redhat.com>
# Conflicts:
# debian/control

Merge pull request #8045 from tchaikov/wip-scrubstore-cleanup

osd/ScrubStore: remove unused function

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7707 from xiexingguo/xxg-wip-14815

osd: filestore: fix wrong scope of result code for error cases during mount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7726 from tchaikov/wip-fix-ftbfs-gcc6

test/time: no need to abs(uint64_t) for comparing

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #7955 from ErwanAliasr1/evelu-check-bufferlist

test/bufferlist: Avoid false-positive tests

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7980 from liewegas/wip-14952

mon: initialize last_* timestamps on new pgs to creation time

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #7982 from liewegas/wip-14606

qa/workunits/rest/test.py: add confirmation to 'mds setmap'

Merge pull request #7983 from liewegas/wip-14616

log: do not repeat errors to stderr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8004 from liewegas/wip-rados-test-stderr

qa/workunits/rados/test.sh: capture stderr too

Merge pull request #8006 from liewegas/wip-big-pages

os/bluestore/KernelDevice: force block size

Merge pull request #8019 from kmroz/wip-upstream-civetweb-low-ports-6

rgw: drop permissions of rgw/civetweb after startup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8007 from wido/se-mirror-mail

mirrors: Change contact e-mail address for se.ceph.com

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7991 from xiexingguo/xxg-wip-remove-upthrupending

osd: remove up_thru_pending field, which is never used

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8023 from roidayan/xio-remove_unused_variable

xio: remove unused variable

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

test: add rbd-mirror test script

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

test: rbd_mirror_image_replay improve commit position parsing

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

test: ImageReplayer on_stop/on_start error tests

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: ImageReplayer async start/stop

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: protect against duplicate journal events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: protect against journal replay shutdown race condition

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: replaying a journal op post-refresh requires locking

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: periodic failure in TestJournalReplay.SnapRename

Force the refresh of the image to pick up the new snapshot
name before testing that it successfully was renamed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7923 from RustShen/master

doc: add cinder backend section to rbd-openstack.rst

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ScrubStore: remove unused function

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7999 from liewegas/wip-armhf

a few armhf (32-bit build) fixes

Reviewed-by: Dan Mick <dmick@redhat.com>

rgw: store system object meta in cache when creating it

Fixes: #14678
When creating the system object we can easily also get the object's meta, so
keep it in the cache.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw:bucket link now set the bucket.instance acl

Fixes: #11076
Signed-off-by: Zengran Zhang <zhangzengran@h3c.com>

Merge pull request #7891 from rzarzynski/wip-rgw-content-disp

rgw: multiple Swift API compliance improvements for TempURL (Radoslaw Zarzynsk)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #7990 from x11562/rgw_unused_vector

rgw: remove unused vector

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8031 from ceph/wip-rgw-usage

rgw: admin api for retrieving usage info (Ji Chen)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #8036 from yehudasa/wip-rgw-support-json-format-v2

rgw: support json format for admin policy API (Dunrong Huang)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #7874 from badone/wip-coverity-rgw_lib-uninit

[rgw] RGWLib::env is not used so remove it

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: policy dump --xml backward compatibility

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

osd: make promote_probability_millis atomic_t

This is almost always read, so it should be cheap.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: remove unused header file

Signed-off-by: Dunrong Huang <riegamaths@gmail.com>

rgw: support json format output for rados-admin policy command

when use rados-admin to get policy with parameter --format=json:
$ radosgw-admin policy --bucket=m8x --format=json

policy information with json format should be output.

Signed-off-by: Dunrong Huang <riegamaths@gmail.com>

rgw: support json format for admin policy API

Fixes: #14090, #14091
- before this patch:
$ curl -s -X GET http://radosrgw:9090/admin/bucket?format=json&policy&bucket=m8x | xmllint --format -

got XML response with error format

- after this patch:
got correct response with json format

Signed-off-by: Dunrong Huang <riegamaths@gmail.com>

Merge pull request #8027 from liewegas/wip-reweight-fix

test/pybind/test_ceph_argparse: fix reweight-by-utilization tests

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6656 from jmunhoz/wip-bucket-request-payment-usage-log

rgw: add bucket request payment feature usage statistics integration

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: also dump realm in radosgw-admin sync status command

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8021 from branch-predictor/bp-bench-full-bw

common/obj_bencher.cc: bump the precision of bandwidth field

Reviewed-by: Sage Weil <sage@redhat.com>

FileStore: Added O_DSYNC write scheme

lfn_open can now open file handle in O_DSYNC mode based on the
filestore_odsync_write config option. This should help in getting
more stable and higher performance out in case of SSDs configured as
OSD data store.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>

Merge pull request #7675 from vmcloud/wip-12917

rgw: adjust the request_uri to support absoluteURI of http request