script: subscription-manager support (part 3)

Renaming subman.py in subman is cute but does not prevent the generation
of the py[co].

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #8308 from wido/cloudstack-docs

doc: Updated CloudStack RBD documentation

Merge pull request #8327 from vumrao/wip-vumrao-15288

doc: Remove Ceph Monitors do lots of fsync()

Merge pull request #8328 from scienceluo/master

doc: Fixes headline different font size and type

Merge pull request #8293 from dachary/wip-subman-facts

script: subscription-manager support (part 2)

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: Remove Ceph Monitors do lots of fsync()
     and change the ligature of "fl" to "f" and "l"

Fixes: #15288
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

Fixes headline different font size and type

Merge pull request #8299 from liewegas/wip-15230

messages/MOSDOp: clear reqid inc for v6 encoding

Merge pull request #8318 from tchaikov/wip-15240

test/system/*: use dynamically generated pool name

Reviewed-by: Sage Weil <sage@redhat.com>

test/system/*: use dynamically generated pool name

was using "foo" for the pool name. this works if we are performing
tests in serial. but if we do it in parallel, the tests interfere
with each other.

Fixes: #15240
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7094 from songbaisen/wipdf

mon: show the pool quota info on ceph df detail command

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #8275 from xiexingguo/xxg-wip-fixdirtyinfo

osd: fix dirtying info without correctly setting drity_info field

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8230 from majianpeng/clear-omap

osd: fix bugs for omap ops

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8254 from xiexingguo/xxg-wip-osd

osd: fix reference count, rare race condition etc.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8291 from xiexingguo/xxg-wip-fixhistorymerge

osd: more fixes for incorrectly dirtying info; resend reply for duplicated scrub-reserve req

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8247 from liewegas/wip-split-creates

mon: do not send useless pg_create messages for split pgs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8208 from liewegas/wip-15027

mon: mark_down_pgs in lockstep with pg_map's osdmap epoch

Reviewed-by: Kefu Chai <kchai@redhat.com>

script: subscription-manager support (part 2)

Strip the .py suffix so that no pyc / pyo files are generated.
Do not install on CentOS.

http://tracker.ceph.com/issues/14972 Fixes: #14972

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph_test_rados_api_misc: debug LibRadosMiscConnectFailure.ConnectFailure

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MOSDOp: clear reqid inc for v6 encoding

For the new v7 encoding, we put the client incarnation in the
reqid so that we can have it at the beginning of the encoding.
However, we do *not* want to do this for the v6 encoding, as that
version of get_reqid() will return reqid if it is not ==
osd_reqid_t().

Fixes: #15230
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8265 from athanatos/wip-journal-default

config_opts: disable filestore throttle soft backoff by default

Merge pull request #8306 from tchaikov/wip-doc-formatting

doc/rados/operations/crush: fix the formatting

Merge pull request #8202 from XinzeChi/wip-objecter-tid

osdc/Objecter: dout log after assign tid

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/jewel'

Merge pull request #8310 from tchaikov/wip-fix-rbdmap-manpage

makefile: fix rbdmap manpage

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: conditionally install man pages

this matches its counterpart in automake
* install fuse related man pages only if HAVE_LIBFUSE
* install rbd related man pages only if WITH_RBD

Signed-off-by: Kefu Chai <kchai@redhat.com>

autotools: add rbdmap to dist_man_MANS

this fixes the rpm/deb build failure
was introduced by 20c8650

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8038 from badone/wip-crushtool-crash

crushtool: Don't crash when called on a file that isn't a crushmap

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8309 from tchaikov/wip-15240

test/system/rados_list_parallel: print oid if rados_write fails

Reviewed-by: Sage Weil <sage@redhat.com>

xxHash: fix submodule commit

This was inadvertantly rewinded in b9a14312edd6a4edb0f4783ab08ad97e0e9ba03f

Signed-off-by: Sage Weil <sage@redhat.com>

osd/pg: publish stats to osd if pg is currently marked as down

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/pg: publish stats to osd if we recovered from degraded

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/pg: avoid publishing pg_stat twice in a short interval

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/pg: fold sanity check into one single "if" qualification

Just as "boost::statechart::result
PG::RecoveryState::WaitRemoteBackfillReserved::react(const RemoteBackfillReserved &evt)"
does.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/pg: drop unnecessary update_heartbeat_peers()

The proc_replica_info() method will do the same job in a
smarter way, so the succeeding call to update_heartbeat_peers()
is unnecessary and cost expensive.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8211 from liewegas/wip-jewel

doc: draft notes for jewel

Merge pull request #8044 from dzafman/wip-8885

osd: fix dump_ops_in_flight races

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8156 from liewegas/wip-pg-query

osd/PG: indicate in pg query output whether ignore_history_les would help

Revewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8177 from majianpeng/signal-handle

global/signal_handler: print thread name in signal handler

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/system/rados_list_parallel: print oid if rados_write fails

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/release-notes: rgw typo

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8294 from theanalyst/rgw/bucket-quota-check

rgw:Use count fn in RGWUserBuckets for quota check

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

doc: Updated CloudStack RBD documentation

Signed-off-by: Wido den Hollander <wido@42on.com>

doc/rados/operations/crush: fix the formatting

unlike in LaTeX, we should use ``foo`` instead of ``bar'' in the
sphinx markdown for the double quote.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'origin/wip-15065'

Support new standby_for_fscid option, letting us assign daemons to FS clusters.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-mfsmap-features' of git://github.com/jcsp/ceph

Clean up some code in the new multi-fs handling. Make its output
more precise and add some debug/repair config options.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Conflicts:
src/mds/FSMap.cc

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7952 from ukernel/wip-14360

mds: allow client to request caps when opening file

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8180 from ukernel/jewel-15167

Be more careful about directory fragmentation and scrubbing

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8115 from ukernel/jewel-cephfs-data-scan-misc

Minor fixes around data scan in some scenarios

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8297 from dillaman/wip-cls-rbd-mirror-image-list

cls_rbd: mirror_image_list should return global image id

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8253 from wido/issue-15119

rgw: Do not send a Content-Type on a '304 Not Modified' response

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8224 from SUSE/wip-15212

rbdmap: add manpage

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls_rbd: mirror_image_list should return global image id

The global image id is needed to crosslink images when an image is
replicated between multiple clusters.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw:Use count fn in RGWUserBuckets for quota check

We already have a count function from RGWUserBuckets class which gives
the total size of the map, use that instead of getting the map and doing
a count.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

10.1.0

Merge pull request #8285 from dmick/jewel

debian/control: try installing virtualenv first, if it exists

Reviewed-by: Alfredo Deza <adeza@redhat.com>

os/bluestore/BlueStore.cc: when do omapclear, it should clear omap_head.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/ReplicatedPG: when do omapclear, it clear FLAG_OMAP  at the same time.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/ReplicatedPG: When do omapclear, it should check object whether is omap.

Though is_omap don't make sure object has omap header or k/v now, if
object isn't has FLAG_OMAP, it must don't have omap header or k/v.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/ReplicatedPG: For omap read ops, it should check object wether has omap

For set omap heaader or k/v, it set FLAG_OMAP.  When remove header
or k/v, it don't remove FLAG_OMAP  because it don't  know object whether
have omap header or other k/vs.
Though object has FLAG_OMAP dont mean now it has omap header or k/v, it
better check FLAG_OMAP than pool whether support omap operations for omap reads.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/pg: set dirty_info if we succeeding in updating log from master

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/pg: set dirty_info if proc_master_log() dirties info

There are newly added branches for proc_master_log() method will
potentially dirty info but without setting dirty_info flag.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8251 from majianpeng/vstart

vstart: make -k with optional mon_num.

Reviewed-by: Kefu Chai <kchai@redhat.com>

vstart: make -k with optional mon_num.

Default mon_num=3, if we change this and using -k to create a new
cluster. It don't work. So we must record the old mon_num.
The same for osd_num/mds_num/rgw_num.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #8288 from dmick/wip-default-virtualenv-path

Makefile-env.am: set a default for CEPH_BUILD_VIRTUALENV

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/pg: set dirty_info after we dirty history

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Makefile-env.am: set a default for CEPH_BUILD_VIRTUALENV

We need to shorten the virtualenv path for non-Jenkins builds
too; gitbuilders were also overflowing the interpreter path limit

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #8235 from trociny/wip-rbd-mirror-asok

rbd-mirror: asok commands to get status and flush on Mirror and Replayer level

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

debian/control: try installing virtualenv first, if it exists

Debian Jessie (at least) now has the virtualenv command in
package 'virtualenv', which depends on python3-virtualenv, and
that's how you get the virtualenv command there.  Earlier
Debian-based releases still use python-virtualenv.  Jessie's
virtualenv command still creates a python2 venv by default.

This will require a followup fix for install-deps.sh
to handle the | correctly, because currently it sends
the string to apt-get, which treats it as a regular expression
alternation of the two package names, so will try to install both.

The problem occurs when packages are installed without Recommended
packages (because jessie's virtualenv package currently Recommends
python-virtualenv); this is the case under pbuilder, and also using
'mk-build-deps --install', and I suspect under other automated
package-building tools.

Note that Build-Requires processing is not specified to
perform the searches in left-to-right order, so even this is
a tenuous workaround.  We probably need distro-specific
debian/ directories.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #8280 from rjfd/wip-15238

rbd-mirror: switch fsid over to mirror uuid

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7976 from ukernel/jewel-15008

Jewel 15008

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7902 from jcsp/wip-pycephfs-flock

pybind: add flock to libcephfs python bindings

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #7901 from jcsp/wip-pycephfs-fixes

libcephfs: fix python tests and fix getcwd on missing dir

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #8271 from vumrao/wip-vumrao-15244

doc: rgw_region_root_pool option should be in [global]

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8184 from theanalyst/fix/rgw-admin-zone-msg

rgw: add zone delete to rgw-admin help

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8264 from cbodley/wip-rgw-admin-errors

radosgw-admin: 'period commit' supplies user-readable error messages

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8190 from yehudasa/wip-rgw-sync-fixes-4

Wip rgw sync fixes 4

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd: rbd-mirroring: Replayer registers in remote journal with mirror_uuid

Signed-off-by: Ricardo Dias <rdias@suse.com>
Fixes: #15238

Merge pull request #8282 from dillaman/wip-15250

rbd: allow librados to prune the command-line for config overrides

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7907 from dachary/wip-subman-facts

script: subscription-manager support

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #8022 from roidayan/fix_compile_against_latest_accelio

xio: fix compilation against latest accelio

rbd: allow librados to prune the command-line for config overrides

Fixes: #15250
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8279 from dachary/wip-virtualenv-jessie

build/ops: in jessie virtualenv is in package virtualenv

Reviewed-by: Alfredo Deza <adeza@redhat.com>

build/ops: in jessie virtualenv is in package virtualenv

The python scripts are not yet compatible with python3, yet it is the
default on jessie. Force the creation of the virtualenv to use python2.7
instead. The wheelhouse is already explicitly populated for both python3
and python2.7 by install-deps.sh, regardless of the default interpreter.

Signed-off-by: Loic Dachary <loic@dachary.org>

test/rgw: use bucket for data checkpoint instead of data

data checkpoint is not accurate, as data log might not show repeating changes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: data sync, update and flush high marker

We need to update the high marker even if we skip entries, and eventually
flush it. This is needed so that our position in the bucket index log that
we follow is reflected correctly.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: update data log even if canceling index operation

Need to update the data log anyway, so that it reflects that there were
changes in the bucket index log.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: break out of run_sync() if going down

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix bucket_instance_meta_info mtime json decoding

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8277 from SUSE/wip-15246

rpm: prefer UID/GID 167 when creating ceph user/group

Reviewed-by: David Disseldorp ddiss@suse.de
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #8222 from SUSE/wip-14984

systemd: set up environment in rbdmap unit file

Reviewed-by: Boris Ranto <branto@redhat.com>

xio: fix compilation against latest accelio

Do not use C struct initilization style to avoid eclipse errors.
We also don't use c++ struct initalize as members are not specified
and it just happens that in latest accelio version one of the fields
has changed in the middle of the struct.

Signed-off-by: Roi Dayan <roid@mellanox.com>

Merge pull request #8268 from yehudasa/wip-15243

rgw: convert plain object to versioned (with null version) when removing

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8148 from roidayan/xio-add_prefix_to_xio_logs

xio: add prefix to xio msgr logs

Merge pull request #8227 from roidayan/xio-call_xio_init_first

xio: xio_init needs to be called before any other xio function

Merge pull request #8207 from theanalyst/doc/mstart

doc: very basic doc on mstart

mon/PGMonitor: do not send pg_creates for split PGs

They will split from the parent on their own, and the OSD drops these
PGs on the floor anyway in OSD::handle_pg_create().

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMonitor: minor optimization in send_pg_creates

Avoid repeated hash lookup.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8237 from oritwas/wip-rgw-fix-mrun

cmake: fix mrun to handle cmake build structure

rgw: convert plain object to versioned (with null version) when removing

Fixes #15243

When removing a plain null versioned object (was created prior to bucket versioning
enabled), need to convert the bucket index representation to a versioned one. This
is needed so that all the versioning mechanics play together.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>