mailmap: Pan Liu affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Ilya Shipitsin affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Fang Yuxiang affiliation

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13297 from dachary/wip-buildpackages

buildpackages: remove because it does not belong

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #13162 from liewegas/wip-bluestore-reshard

os/bluestore: partial reshard support

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

buildpackages: remove because it does not belong

It should live in teuthology, not in Ceph. And it is currently broken:
there is no need to keep it around.

Fixes: http://tracker.ceph.com/issues/18846
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #13035 from yehudasa/wip-18621

rgw: change default chunk size to 4MB

Merge pull request #13259 from jcsp/wip-jcsp-relnotes

PendingReleaseNotes: note the fuse fstab format change

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13279 from smithfarm/wip-fix-regression

tests: fix regression in qa/tasks/ceph_master.py

rgw: don't update bucket index multiple times in overwrite

Instead of this for overwrites:
 prepare (index),
   write (head) [-EEXIST]
 cancel (index)
 read (head)
 prepare (index)
   write (head)
 complete (index)

We now do:
 prepare (index),
   write (head) [-EEXIST]
   read (head)
   write (head)
 complete (index)

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: assume obj write is a first write

if fails and object already exists then retry. This improves first obj
write performance on the expense of overwrites.

Fixes: http://tracker.ceph.com/issues/18622
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: configurable write obj window size

Fixes: http://tracker.ceph.com/issues/18623
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

tests: fix regression in qa/tasks/ceph_master.py

https://github.com/ceph/ceph/pull/13194 introduced a regression:

2017-02-06T16:14:23.162 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
  File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 722, in wrapper
    return func(self)
  File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 839, in do_thrash
    self.choose_action()()
  File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 305, in kill_osd
    output = proc.stderr.getvalue()
AttributeError: 'NoneType' object has no attribute 'getvalue'

This is because the original patch failed to pass "stderr=StringIO()" to run().

Fixes: http://tracker.ceph.com/issues/16263
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12979 from theanalyst/scripts/release-notes-tracker

script: ceph-release-notes check orig. issue only for backports

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13277 from jan--f/janf-teuth-cloud-doc

doc: minor change to a cloud testing paragraph

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13275 from idryomov/wip-require-osd-features-fix

osd/OSDMap: require OSD features only of OSDs

Reviewed-by: Sage Weil <sage@redhat.com>

doc: minor change to a cloud testing paragraph

Use terminate for the instances to match the terminology used by the
horizon interface.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #13256 from dongbula/correct-cached_crc-comment

osd: correct comment of perfcounter cached_crc in code

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13220 from rzarzynski/wip-rgw-18364-2

rgw: be aware abount tenants on cls_user_bucket -> rgw_bucket conversion

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12993 from jcsp/wip-cephfs-doc-authcaps

doc: clarify the path restriction mds cap example

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #13242 from Jing-Scott/dev/fix-swift-anon-getobj-error-code

rgw: swift: fix anonymous user's error code of getting object

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

osd/OSDMap: require OSD features only of OSDs

Fix up commit 1a5cc32f0a3b ("osd/OSDMap: reflect REQUIRE_*_OSDS flag in
required features") -- require_*_osds shouldn't affect older clients.

Fixes: http://tracker.ceph.com/issues/18831
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #13271 from Liuchang0812/fix-file-description-leak

msg/async: fix file description leak in NetHandler

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async: fix file description leak in NetHandler

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

rgw: swift: fix anonymous user's error code of getting object

The openstack swift will return 401 rather than 403 when
the anon user has no permission to get objects.

Fixes: http://tracker.ceph.com/issues/18806
Signed-off-by: Jing Wenjun <jingwenjun@cmss.chinamobile.com>

Merge pull request #13237 from smithfarm/wip-18799

tests: Thrasher: eliminate a race between kill_osd and __init__

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13263 from Adirl/fix_log_spaces

msg/async/rdma: fix log line spacing

Reviewed-by: Haomai Wang <haomai@xsky.com>

PendingReleaseNotes: note the fuse fstab format change

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #13257 from tchaikov/wip-less-verbose-ceph-helper

tests: ceph-helpers.sh reduce get_timeout_delays() verbosity

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: ceph-helpers.sh reduce get_timeout_delays() verbosity

`set +o` prints out the full command line which is echoed if "xtrace" is
enabled. this increases the verbosity of get_timeout_delays().
in this change, we follow the way of kill_daemons() to kill the extra
output. see aefcf6d.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: correct comment of perfcounter cached_crc in code

Signed-off-by: lvshuhua <lvshuhua@cmss.chinamobile.com>

Merge pull request #13080 from yuyuyu101/wip-async-fd

msg/async/AsyncConnection: socket's fd can be zero, avoid false assert

Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #13251 from scuttlemonkey/master

script: adding contributor credits script

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9396 from songbaisen/doc2

doc: fix the librados c api can not compile problem

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13252 from linuxbox2/wip-rgw-setattr

rgw_file:  fix RGWLibFS::setattr for directory objects

rgw_file:  fix RGWLibFS::setattr for directory objects

Fixes: http://tracker.ceph.com/issues/18808
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13147 from yehudasa/wip-18331

rgw: fix use of marker in List::list_objects()

Merge pull request #12023 from yuyuyu101/wip-msgr-type

msg: allow different ms type for cluster network and public network

Reviewed-by: Adir Lev <adirl@mellanox.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13114 from liewegas/wip-osdmap-required-features

osd: condition object_info_t encoding on required (not up) features

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13206 from liewegas/wip-osd-enxio

osd: do not send ENXIO on misdirected op by default

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13213 from ovh/bp-osd-updateable-throttles

osd: allow client throttler to be adjusted on-fly, without restart

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13063 from ovh/wip-mailmap-ovh-contributors

mailmap: Update OVH contributors

Reviewed-by: Loic Dachary <ldachary@redhat.com>

adding contributor credits script

Signed-off-by: Patrick McGarry <pmcgarry@redhat.com>

Merge pull request #13223 from wjwithagen/wip-wjw-freebsd-sphynx

build/install-deps.sh: Add sphinx package for building docs on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11448 from ledgr/change/mount_fuse_ceph_options_in_fs_mntops_and_python

Rewrite mount.fuse.ceph (to python) and move ceph-fuse options to fs_mntops

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13194 from smithfarm/wip-16263

tests: ignore bogus ceph-objectstore-tool error in ceph_manager

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

os/bluestore: drop ExtentMap::Shard offset field

This is mirrored in shard_info, which we have a pointer to.

Note that shard_info is also a bit redundant as we can also look it up in
the onode vector in the same position, but it makes it more awkward to use
iterators.  Something to consider later to save memory.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: simplify dirty shard handling in ExtentMap::update

We can do this with a single allocation with a simple vector<>.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ExtentMap: reshard only requested range

Only reshard extents in the requested range.

Note that the strategy for unspanning blobs changed; we now span or
unspan specific blobs at the end based on our reshard result instead
of unspanning all at the start.  This keeps the spanning id stable,
which is important because it may be referenced from another that
we aren't even looking at.

Also note that this requires a bit of restructuring: an encode_some may
hit a spanning shard, *requiring* us to reshard, which means we should
always conduct the initial pass through update to discover other reshard
requirements, even if we already know some resharding will be needed.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: dump num spanning blobs from dump_onode

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: trivial cleanups in csum error code

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fsck: dump_onode after shards are loaded

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: error out on fsck failure

It's much more helpful this way.

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: return 0 from fsck

Signed-off-by: Sage Weil <sage@redhat.com>

tests: Thrasher: eliminate a race between kill_osd and __init__

If Thrasher.__init__() spawns the do_thrash thread before initializing the
ceph_objectstore_tool property, do_thrash races with the rest
of Thrasher.__init__() and in some cases do_thrash can call kill_osd() before
Trasher.__init__() progresses much further. This can lead to an exception
("AttributeError: Thrasher instance has no attribute 'ceph_objectstore_tool'")
being thrown in kill_osd().

This commit eliminates the race by making sure the ceph_objectstore_tool
attribute is initialized before the do_thrash thread is spawned.

Fixes: http://tracker.ceph.com/issues/18799
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13234 from ifed01/wip-bluestore-performance-sandbox2017

os/bluestore: fix potential assert in cache _trim method.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12965 from linuxbox2/wip-librgw-expand-argv

rgw_file:  split last argv on ws, if provided

rgw_file:  split last argv on ws, if provided

This is intended to allow an "extra" unparsed argument string
containing various cmdline options to be passed as the last argument
in the argv array of librgw_create(), which nfs-ganesha is
expecting to happen.

While at it, hook env_args() too.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: be aware abount tenants on cls_user_bucket -> rgw_bucket conversion.

Fixes: http://tracker.ceph.com/issues/18364
Fixes: http://tracker.ceph.com/issues/16355
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

mailmap: Update OVH contributors

Signed-off-by: Bartłomiej Święcki <bartlomiej.swiecki@corp.ovh.com>

Merge pull request #12335 from theanalyst/mailmap/v11.1.0

mailmap updates for v11.1.0

Reviewed-by: Loic Dachary <ldachary@redhat.com>

os/bluestore: fsck: check for shards past EOF

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: reshard on truncate if shards past EOF

This ensure we mop up shards past EOF instead of encoding them
empty and confusing future code (that, say, assumes no shards past
EOF).

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/bluestore_types: beautify use_tracker_t operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ExtentMap: request reshard range from update() intelligently

Split ourselves, or merge with our immediate predecessor or
successor.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ExtentMap: use normal iterator for reshard loop

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ExtentMap: put un-spanning in normal loop

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ExtentMap: remove reshard 0 extent special case

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ExtentMap: factor old shard key removal into reshard()

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: cosmetic cleanup in ExtentMap::update

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ExtentMap: count extents per Shard

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ExtentMap: use reshard ranges from update()

This gives us a single point of control for reshard, and tells us
which range we care about at the same time.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: replace needs_reshard flag with a range

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: match clone_range src and dst offset

This is what the OSD does, and BlueStore doesn't necessarily handle
all combinations of different offsets.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13211 from liewegas/wip-clone-offsets

ceph_test_objectstore: match clone_range src and dst offset

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12904 from ifed01/wip-bluestore-onode-refmap3

os/bluestore: replace Blob ref_map with reference counting

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13192 from liewegas/wip-bluestore-min-alloc-size

os/bluestore: fix min_alloc_size at mkfs time

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: fix potential assert in cache _trim method.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

msg/async/rdma: fix log line spacing

issue: none

Change-Id: Ieef3284ee673f012331d21fffcb499e22473f62a
Signed-off-by: Adir Lev <adirl@mellanox.com>

OSD: allow client throttler to be adjusted on-fly, without restart

This patch allows the osd_client_message_cap and
osd_client_message_size_cap to be adjusted on-fly, using admin socket
functionality.

Fixes: http://tracker.ceph.com/issues/18791
Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>

ceph_mds: adopt ms public type

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13158 from dillaman/wip-18594

qa: integrate OpenStack 'gate-tempest-dsvm-full-devstack-plugin-ceph'

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13225 from tmuthamizhan/wip-ansible-m

qa: do not mention ceph branch explicitly

do not mention ceph branch explicitly

no need to mention ceph_dev_branch explicitly. it will be taken from the
ceph branch value mentioned in the teuthology-suite command

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

build/install-deps.sh: Add sphinx package for DOCS

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13009 from liewegas/wip-mempool-bloom-filter

mempool: put bloom_filter in mempool

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13136 from liewegas/wip-scrub-waitlist

osd: use separate waitlist for scrub

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12104 from pritha-srivastava/wip_sts_role_rest

rgw: Added code for REST APIs for AWS Roles.

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13189 from gaurav36/bug-18326-image-rename-non-rbd-pool

rbd: destination pool should be source pool if it is not specified

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12948 from trociny/wip-17020

rbd-mirror: add support for active/passive daemon instances

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13013 from wangzhengyong/master

os/bluestore: remove no use parameter in bluestore_blob_t::map_bl

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12782 from atheism/rgw-orphan-fix-error-messages

rgw: orphan: fix error messages

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13199 from chipitsine/master

mds,mon: Clean issues detected by cppcheck

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13203 from ifed01/wip-bluestore-more-dump

os: extend ObjectStore interface to dump store's performance counters

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12974 from ukernel/wip-multimds-misc

mds: miscellaneous fixes

Reviewed-by: John Spray <john.spray@redhat.com>

qa/suites/rbd: add rbd-mirror-ha test

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

qa/workunits/rbd: tweaks to use RBD_MIRROR_USE_EXISTING_CLUSTER locally

This var is mostly used when running rbd_mirror test scripts on
teuthology.  It can be used locally though to speedup re-running the
tests:

Set a test temp directory:

  export RBD_MIRROR_TEMDIR=/tmp/tmp.rbd_mirror

Run the tests the first time with NOCLEANUP flag (the cluster and
daemons are not stopped on finish):

  RBD_MIRROR_NOCLEANUP=1 ../qa/workunits/rbd/rbd_mirror.sh

Now, to re-run the test without restarting the cluster, run cleanup
with USE_EXISTING_CLUSTER flag:

  RBD_MIRROR_USE_EXISTING_CLUSTER=1 \
    ../qa/workunits/rbd/rbd_mirror_ha.sh cleanup

and then run the tests:

  RBD_MIRROR_USE_EXISTING_CLUSTER=1
    ../qa/workunits/rbd/rbd_mirror_ha.sh

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

qa/workunits/rbd: tests for rbd mirror HA

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

qa/workunits/rbd: allow to start several daemons for the same cluster

by optionally specifyning daemon instance after cluster name and
colon, like:

  start_mirror ${cluster}:${instance}

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror HA: pool replayer should be started/stopped when lock acquired/released

Fixes: http://tracker.ceph.com/issues/17020
Signed-off-by: Mykola Golub <mgolub@mirantis.com>