os/bluestore/BlueStore.cc: when do omapclear, it should clear omap_head.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/ReplicatedPG: when do omapclear, it clear FLAG_OMAP  at the same time.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/ReplicatedPG: When do omapclear, it should check object whether is omap.

Though is_omap don't make sure object has omap header or k/v now, if
object isn't has FLAG_OMAP, it must don't have omap header or k/v.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/ReplicatedPG: For omap read ops, it should check object wether has omap

For set omap heaader or k/v, it set FLAG_OMAP.  When remove header
or k/v, it don't remove FLAG_OMAP  because it don't  know object whether
have omap header or other k/vs.
Though object has FLAG_OMAP dont mean now it has omap header or k/v, it
better check FLAG_OMAP than pool whether support omap operations for omap reads.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #8235 from trociny/wip-rbd-mirror-asok

rbd-mirror: asok commands to get status and flush on Mirror and Replayer level

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8280 from rjfd/wip-15238

rbd-mirror: switch fsid over to mirror uuid

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7902 from jcsp/wip-pycephfs-flock

pybind: add flock to libcephfs python bindings

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #7901 from jcsp/wip-pycephfs-fixes

libcephfs: fix python tests and fix getcwd on missing dir

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #8271 from vumrao/wip-vumrao-15244

doc: rgw_region_root_pool option should be in [global]

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8184 from theanalyst/fix/rgw-admin-zone-msg

rgw: add zone delete to rgw-admin help

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8264 from cbodley/wip-rgw-admin-errors

radosgw-admin: 'period commit' supplies user-readable error messages

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8190 from yehudasa/wip-rgw-sync-fixes-4

Wip rgw sync fixes 4

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd: rbd-mirroring: Replayer registers in remote journal with mirror_uuid

Signed-off-by: Ricardo Dias <rdias@suse.com>
Fixes: #15238

Merge pull request #8282 from dillaman/wip-15250

rbd: allow librados to prune the command-line for config overrides

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7907 from dachary/wip-subman-facts

script: subscription-manager support

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #8022 from roidayan/fix_compile_against_latest_accelio

xio: fix compilation against latest accelio

rbd: allow librados to prune the command-line for config overrides

Fixes: #15250
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/rgw: use bucket for data checkpoint instead of data

data checkpoint is not accurate, as data log might not show repeating changes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: data sync, update and flush high marker

We need to update the high marker even if we skip entries, and eventually
flush it. This is needed so that our position in the bucket index log that
we follow is reflected correctly.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: update data log even if canceling index operation

Need to update the data log anyway, so that it reflects that there were
changes in the bucket index log.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: break out of run_sync() if going down

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix bucket_instance_meta_info mtime json decoding

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8277 from SUSE/wip-15246

rpm: prefer UID/GID 167 when creating ceph user/group

Reviewed-by: David Disseldorp ddiss@suse.de
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #8222 from SUSE/wip-14984

systemd: set up environment in rbdmap unit file

Reviewed-by: Boris Ranto <branto@redhat.com>

xio: fix compilation against latest accelio

Do not use C struct initilization style to avoid eclipse errors.
We also don't use c++ struct initalize as members are not specified
and it just happens that in latest accelio version one of the fields
has changed in the middle of the struct.

Signed-off-by: Roi Dayan <roid@mellanox.com>

Merge pull request #8268 from yehudasa/wip-15243

rgw: convert plain object to versioned (with null version) when removing

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8148 from roidayan/xio-add_prefix_to_xio_logs

xio: add prefix to xio msgr logs

Merge pull request #8227 from roidayan/xio-call_xio_init_first

xio: xio_init needs to be called before any other xio function

Merge pull request #8207 from theanalyst/doc/mstart

doc: very basic doc on mstart

Merge pull request #8237 from oritwas/wip-rgw-fix-mrun

cmake: fix mrun to handle cmake build structure

rgw: convert plain object to versioned (with null version) when removing

Fixes #15243

When removing a plain null versioned object (was created prior to bucket versioning
enabled), need to convert the bucket index representation to a versioned one. This
is needed so that all the versioning mechanics play together.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

radosgw-admin: add explanation for EACCESS on 'realm pull'

References #15200

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: 'period commit' supplies user-readable error messages

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: parse error messages from failed requests

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8273 from wjin/fix

osd: fix log info

Reviewed-by: Kefu Chai <kchai@redhat.com>

RPM: prefer UID/GID 167 when creating ceph user/group

This change affects openSUSE/SLE only.

http://tracker.ceph.com/issues/15246 Fixes: #15246

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8260 from liewegas/wip-ec-test

test/test-erasure-code.sh: disable pg_temp priming

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: rgw_region_root_pool option should be in [global]
     add new option rgw_zonegroup_root_pool from Jewel.

Fixes: #15244
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

osd: fix log info

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

Merge pull request #8267 from athanatos/wip-14309

osd: bail out of _committed_osd_maps if we are shutting down

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8266 from athanatos/wip-test-exit-value

Test exit values on test.sh, fix tier.cc

Reviewed-by: Sage Weil <sage@redhat.com>

test/librados/tier.cc: Fix Whiteout tests to force promotion

Otherwise, there may not actually be whiteouts created.

Fixes: 15177
Signed-off-by: Samuel Just <sjust@redhat.com>

qa/workunits/rados/test.sh: check subprocess return values

Fixes: #15165
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #8263 from tchaikov/wip-15031

msg/async: avoid log spam on throttle

Reviewed-by: Sage Weil <sage@redhat.com>

test/test-erasure-code.sh: disable pg temp priming

If we prime pg_temp, there is a race where this test expects an out OSD
to not be in the acting set but it will be there initially.

Fixes: #15211
Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: avoid log spam on throttle

increase the log level from 1 to 10, the log is printed when message
throttler kicks in.

Fixes: #15031
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8252 from xiexingguo/xxg-wip-15164

doc: fix wrong type of hyphen

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8239 from jcsp/wip-15223

mon: fix mixed-version MDSMonitor

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: fix mrun to handle cmake build structure

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

doc: fix wrong type of hyphen

Which makes picture of http://docs.ceph.com/docs/master/start/quick-start-preflight/
look sick.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7934 from ceph/wip-default-tcmalloc-cache-size

etc/sysconfig/ceph: set 128MB tcmalloc cache size

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #8244 from linuxbox2/jewel-ldap-fixes

rgw_ldap fixes

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8225 from SUSE/wip-radosgw-description

packaging: align radosgw package description

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8217 from ceph/wip-rbd-mirror-init

rbd-mirror: systemd and upstart init scripts

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8245 from theanalyst/fix/delete-zone-msg

rgw: fix error message for zone delete

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8243 from theanalyst/fix/zone-rm-msg

rgw: typo fix for zone delete message

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix error message for zone delete

Currently if a zone is not a part of a zonegroup, an error message
is printed that zone doesn't exist, and the zone gets deleted anyway.
Since this is exhibited only if the zone isn't a part of a zonegroup
and we allow creation of such zones, clarify that the zone wasn't a
part of zonegroup instead

Fixes: #14951
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rbd-mirror: asok commands to get status and flush on Mirror and Replayer level

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: async flush for ImageReplayer

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rgw_ldap:  move update of s->perm_mask

This should happen on successful auth, not only when a new user
record is stored.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_ldap:  two bug fixes

1. only attempt auth on tokens which pass valid() check
2. ignore false positive search result (and fix result in that case)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #8240 from dillaman/wip-15224

cls_rbd: throw error if set_features mask included non-mutable feature

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: fix zone delete message

Though this error message is hard to reach, it is still wrongly printed
as failed to create instead of failed to delete.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #8130 from tchaikov/wip-15113

mon: do not panic at seeing requests without session

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

cls_rbd: throw error if set_features mask included non-mutable feature

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: fix mixed-version MDSMonitor

Encode old-style until mon quorum features indicate
jewel, forbid creating multiple filesystems, and
when creating a single filesystem use the legacy fscid.

Fixes: #15223
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #8016 from ErwanAliasr1/evelu-15042

rpm,deb: remove conditional BuildRequires for btrfs-progs

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8214 from athanatos/wip-feature-bits

Deprecate or free up a bunch of feature bits

Reviewed-by: Sage Weil <sage@redhat.com>

qa: test_rbdmap_RBDMAPFILE.sh workunit

Before a7a3658 the rbdmap script was logging bogus messages and not working
on systemd platforms because the unit file was not defining the RBDMAPFILE
environment variable.

This workunit asserts that the bug has been fixed.

http://tracker.ceph.com/issues/14984 References: #14984

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rgw: add zone delete to rgw-admin help

This operation is already implemented (OPT_ZONE_DELETE), however not
mentioned in the help, just adding to the help options

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

xio: xio_init needs to be called before any other xio function

Latest accelio code removes the attribute to call xio_init on load
and XioMessenger will first try to initialize portals and create xio context.
This commit will fix this behavior and call xio_init first.

Signed-off-by: Roi Dayan <roid@mellanox.com>

Merge pull request #8162 from rjfd/wip-15142

rbd-mirror: use the mirroring directory to detect candidate images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8204 from rjfd/wip-15143

librbd: enable/disable image mirroring automatically for pool mode

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

packaging: align radosgw package description

First, make the Debian package description mention that RGW aims to
implement the Swift API.

Second, replace the RPM package description with the Debian one, both for
consistency and because the Debian one is better.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8172 from trociny/fix-test-rbd-mirror-position

test: rbd-mirror: compare positions using all fields

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8185 from trociny/wip-rbd-mirror-faster-exit

rbd-mirror: fix long termination due to 30sec wait in main loop

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8221 from jdurgin/wip-test-rbd-mirror

qa: add workunit to run ceph_test_rbd_mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: fix long termination due to 30sec wait in Mirror::run loop

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8191 from liewegas/wip-15180

qa/workunits/rest/test.py: don't use newfs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8101 from jjhuo/bluestore

os/bluestore: small fixes in bluestore StupidAllocator

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8178 from majianpeng/bluefs-closewriter

os/bluestore/BlueFS: Before reap ioct, it should wait io complete

Reviewed-by: Sage Weil <sage@redhat.com>

rbdmap: drop init- prefix from logger tag

Since init-rbdmap is sysvinit-specific, tagging log lines "init-rbdmap" is
potentially misleading.

In systemd- and upstart-based systems, /usr/bin/rbdmap is run directly.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rbdmap: default RBDMAPFILE to reasonable value

Signed-off-by: Nathan Cutler <ncutler@suse.com>

systemd: set up environment in rbdmap unit file

http://tracker.ceph.com/issues/14984 Fixes: #14984

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8220 from dillaman/wip-15202

journal: prevent race injecting new records into overflowed object

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: add workunit to run ceph_test_rbd_mirror

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8219 from jdurgin/wip-test-pool-create-cleanup

test_pool_create.sh: put test files in the test dir so they are cleaned up

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

journal: prevent race injecting new records into overflowed object

The recorded added after the overflow will not have been detached
from the object recorder.

Fixes: #15202
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8149 from trociny/test-rbd-mirror

test: rbd-mirror: add "switch to the next tag" test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw_admin: new command to get bilog status

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8182 from trociny/wip-rbd-mirror-args

rbd-mirror: make remote context respect env and argv config params

Conflicts:
qa/workunits/rbd/rbd_mirror.sh (asok command format changed)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8159 from trociny/wip-rbd-mirror-asok

rbd-mirror: use pool/image names in asok commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

packaging: added rbd-mirror startup scripts

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

upstart: new rbd-mirror daemon scripts

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

systemd: new ceph-rbd-mirror scripts

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

OSD: bail out of _committed_osd_maps if we are shutting down

Introduced in b839a06c1aba24ecf0780387a605192ff929f158
Fixes: 14309
Signed-off-by: Samuel Just <sjust@redhat.com>

test_pool_create.sh: put test files in the test dir so they are cleaned up

If they aren't cleaned up, tarball gitbuilders are unhappy.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8216 from ceph/wip-rbd-maint-ops

qa/workunits/rbd: new online maintenance op tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: don't record fetch remote obj in new multisite

No need to record the ops status in new multisite. It was used in v1
to record fetching remote objs during the operation, since these could
time out (were triggered by RESTful copy object that could take too long).
With v2 it can't time out, so no need to make it stateful.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8183 from ceph/wip-rgw-fix-mdlog

Wip rgw fix mdlog

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>