erasure-code: set max_size to chunk_count() instead of 20 for shec

See 8b64fe93b088a3a33d357869c47e6bf928c3f0e4 for a detailed explanation.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5249 from tchaikov/wip-11101

ceph.in: improve the error message

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5110 from ceph/wip-11864-rgw-mime-types-pkg

11864: packaging: RGW depends on /etc/mime.types

Reviewed-by: Boris Ranto <branto@redhat.com>

packaging: RGW depends on /etc/mime.types

If the mimecap RPM or mime-support DEB is not installed, then the
/etc/mime.types file is not present on the system. RGW attempts to read
this file during startup, and if the file is not present, RGW logs an
error:

  ext_mime_map_init(): failed to open file=/etc/mime.types ret=-2

Make the radosgw package depend on the mailcap/mime-support packages so
that /etc/mime.types is always available on RGW systems.

http://tracker.ceph.com/issues/11864 Fixes: #11864

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #4699 from joehandzik/master

FileStore: Collect device partition information

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4876 from ceph/wip-11855

crush: fix 'ceph osd crush reweight-subtree'

Reviewed-by: Li Wang <liwang@ubuntukylin.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph.in: print more detailed warning for 'ceph <type> tell'

put the full signature of "tell <target> <command> [options...]"
instead of "tell {0}.<id>", which could be misleading somehow.

Fixes: 11101
Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.in: print more detailed error message for 'tell' command

* we do not allow user specify a certain daemon when starting an
  interactive session. the existing error message could lead to
  some confusion. so put more details in it.

Fixes: #11101
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5245 from yuyuyu101/wip-async-fix-18

AsyncConnection: Fix incorrect sign_message usages

Merge pull request #5244 from zhouyuan/rgw_obj_stripe_size

rgw: doc: Fix radosgw stripe size config ref

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5240 from dachary/wip-ceph-release-notes

tools: src/script/ceph-release-notes normalization

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5165 from zhouyuan/librados_default_crush_ruleset

Make librados pool_create respect default_crush_ruleset

Reviewed-by: Kefu Chai <kchai@redhat.com>

librados: Make librados pool_create respect default_crush_ruleset

Make pool_create in librados use the osd_pool_default_crush_replicated_ruleset

Fixes: #11640
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

AsyncConnection: Make sign_message ahead of construct message bufferlist

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Fix non-fastdispatch message doesn't prepare case

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

rgw: doc: Fix radosgw stripe size config ref

s/rgw object stripe size/rgw obj stripe size
https://github.com/ceph/ceph/blob/master/src/common/config_opts.h#L1024

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

tools: src/script/ceph-release-notes normalization

Issue an error if for the title does not start with a known prefix.
Issue an error if no issue is associated to a pull request.

Group the pull requests fixing the same issue together.

Add link to the issue and the pull request.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #5230 from majianpeng/check_ops_in_flight_cleanup

common: clean up code for OpTracker::check_ops_in_flight.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

common: clean up code for OpTracker::check_ops_in_flight.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #5220 from vumrao/wip-vumrao-12318

doc: change tcp rcvbuf and tcp nodelay to ms tcp rcvbuf and ms tcp nodelay

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: change tcp rcvbuf and tcp nodelay to ms tcp rcvbuf
     and ms tcp nodelay

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

Merge pull request #4939 from ceph/wip-offline-backward

cephfs-data-scan (offline backward file recovery)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5190 from guce/master

radosgw-admin: after subuser modify print only once user info

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5183 from zhouyuan/rgw_max_put_size_configurable

rgw: Make RGW_MAX_PUT_SIZE configurable

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5213 from wido/wip-rgw-keepalive

rgw: If the client sends a Connection: close header respond accordingly.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5210 from tweenk/docs-improvements

Document librbd::parent_spec and librbd::parent_info.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5215 from SUSE/wip-fdupes

ceph.spec.in: do not run fdupes, even on SLE/openSUSE

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

ceph.spec.in: do not run fdupes, even on SLE/openSUSE

In openSUSE there is a policy to use %fdupes in the spec file if RPMLINT
complains about duplicate files wasting space in the filesystem.

However, RPMLINT is not so complaining, so drop fdupes.

http://tracker.ceph.com/issues/12301 Fixes: #12301

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5102 from ceph/wip-mds-refactor-small

Minor MDS:: refactor

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5209 from zhouyuan/rgw_swift_user

rgw: Make vstart.sh print out swift user info

Reviewed-by: Sage Weil <sage@redhat.com>

packaging: add cephfs-data-scan

Signed-off-by: John Spray <john.spray@redhat.com>

tools/cephfs: add cephfs-data-scan

For scraping cephfs data pools for metadata in
backtraces, and injecting that metadata into
the cephfs metadata pool (initially offline).

Signed-off-by: John Spray <john.spray@redhat.com>

doc: add some docs about cephfs-data-scan

These are deliberately fairly sparse, because:
 * These tools are for experts
 * These tools may well be wrapped in a higher
   level recovery tool that orchestrates parallel
   workers at some stage.

Signed-off-by: John Spray <john.spray@redhat.com>

include/ceph_fs.h: define magic LOST+FOUND ino

So that recovery tools can create this dir without
having to consult/modify InoTable.

Signed-off-by: John Spray <john.spray@redhat.com>

cls: add CephFS object class

For use in cephfs-data-scan for file size and mtime
recovery, by doing a set-if-greater for size/mtime
xattrs on the 0th object of a file.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: include damage_flags in inter-mds encoding

...and correspondingly bump the cluster protocol
version as these structures are not versioned
themselves.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: bump required version on fnode_t encoding

So that older MDS instances can't miss/ignore
the damage flags.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add damage_flags_t to inode+frag

Used to mark inodes+fnodes with flags indicating
that particular types of damage have either
been detected (scrub/fetch) or caused (cephfs-data-scan
inject).

This should serve as a cue to the MDS to do online
repair of stats for the nodes that have been injected.

Signed-off-by: John Spray <john.spray@redhat.com>

mds/CInode: move hash_dentry_name up into InodeStore

Because it only depends on the data in the store, not
anything from a live MDS, and it's useful when
we're dealing with fragmented directories from
an offline tool.

Signed-off-by: John Spray <john.spray@redhat.com>

tools: fix journal reset error handling

When we failed and prompted for --force, it still
looked like a success.  ::reset never had any
proper error handling since it was lifted out
of the MDS, so add it now.

Signed-off-by: John Spray <john.spray@redhat.com>

client: fix typo

s/bye/byte/

Signed-off-by: John Spray <john.spray@redhat.com>

mds: correct typo in log message

s/issueing/issuing/

Silly, but it annoyed me every time I saw it...

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5164 from ceph/wip-12088

client: reference counting 'struct Fh'

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: If the client sends a Connection: close header respond accordingly.

HTTP/1.1 assumes Keep-Alive by default, but if a Connection: close header is send
the server should respond with it as well.

This makes the client close the connection after the request.

Fixes: #12298

Merge pull request #5201 from ceph/wip-12272

tools: Fix rados export to use io_ctx in the right way

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: Make vstart.sh print out swift user info

Currently vstart.sh only print out S3 user info. This patch updates it
to print swift user info too. Also changes the swift account/user/password
to test/tester/testing which align with openstack-swift better:
https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L270

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Fixes: #12286 radosgw-admin: after subuser modify print only once user info.
remove rgw_admin.cc OPT_SUBUSER_MODIFY, show_user_info code block.

switch (opt_cmd) {
...
case OPT_SUBUSER_MODIFY:
show_user_info(info, formatter);       //show first time (remove this)
break;
...
}

// output the result of a user operation
if (output_user_info) {
...
show_user_info(info, formatter);       //show second time
}

test fix:
before: after subuser modify print twice user info.
after changes, do the same procedure, print only once user info.

Signed-off-by: guce guce@h3c.com

rgw: Make RGW_MAX_PUT_SIZE configurable

The 5GB limit of a single operation uploading was part of S3 spec.
However some private setups may have some special requirements
on this limit. It's more convinent to have a configurable value.

Fixes: #6999
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

tools: Fix rados export to use io_ctx in the right way

Cause by: 868f6b7626db4a802b38d1d8ce5135108b322f74

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #5197 from ceph/wip-cot-meta

Wip cot meta

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5198 from tchaikov/wip-doc-erasure-fix

doc/erasure-code: fix couple typos

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/erasure-code: fix couple typos

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools, test: Some ceph-objectstore-tool error handling fixes

Improve various error messages generated with invalid syntax
Add test cases for most of these error messages

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Add debug argument to the ceph-objectstore-tool test

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Check for valid --op earlier so we can get a better error message

Signed-off-by: David Zafman <dzafman@redhat.com>

tools, test: Add ceph-objectstore-tool to operate on the meta collection

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Fix newlines in output of --op list

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Fix dump-super which doesn't require pgid

A valid pgid had to be specified in order for superblock to be displayed,
but it was otherwise ignored.
Fix message that didn't include dump-super

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Check and specify commands that require the pgid specification

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #4404 from rzarzynski/wip-11431

rgw: fix lack of account name attribute in XML-formatted bucket listing of Swift account

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5078 from rzarzynski/wip-12158

rgw: send Content-Length in response for GET on Swift account.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge branch 'wip-temp'

Conflicts:
src/osd/ReplicatedPG.cc
src/test/osd/RadosModel.h

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

osd, tools: Always filter temp objects since not being exported

Can't use object_locator_to_pg() directly with a temp object
because the pool is negative.

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Don't export temporary objects until we have persistent-temp objects

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: fix temp clearing in OSD

Signed-off-by: David Zafman <dzafman@redhat.com>

Document librbd::parent_spec and librbd::parent_info.

Add an explanatory comment to parent_types.h that explains what
the "overlap" member in parent_info means. This was not exactly easy
to figure out.

Signed-off-by: Krzysztof Kosiński <krzysztof.kosinski@intel.com>

Merge pull request #5175 from david-z/wip-mds-dump-crash

Fix mds dump_ops_in_flight crashing ocassionally

Fix mds dump_ops_in_flight crashing ocassionally

Merge pull request #4977 from theanalyst/rados/wip-9737

rados/tool: handle --snapid correctly

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5185 from yuyuyu101/wip-store-test-zero

StoreTest: Add zero test for SyntheticTest

Merge pull request #3273 from majianpeng/fix4

cleanup code for RecoveyMessages

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #4143 from majianpeng/ec-notsupp-omap

fix bugs about cache tier.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4329 from majianpeng/filestore-copy

Filestore copy:using splice instead read/write for clone

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #4458 from wonzhq/dup-promote

osd/ReplicatedPG: need to queue op when avoiding dup promotion in maybe_handle_cache

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #5075 from SUSE/wip-logrotate-requires

packaging: add find and which dependencies

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

packaging: add find and which dependencies

The postrotate script in src/logrotate.conf uses the which and find utilities
to do its work. Although logrotate itself is only a Recommends, I think which
and find are so ubiquitous that it makes sense to have them as hard
dependencies.

Also, I checked and the package names which and findutils are the same on all
the RPM distros we are currently targeting in the spec file.

In Debian, find is also in a package called findutils while the package
containing 'which' is called debianutils.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5187 from SUSE/wip-12261

ceph.spec.in: install 95-ceph-osd.rules, mount.ceph, and mount.fuse.c…

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5181 from SUSE/wip-snappy-devel

ceph.spec.in: snappy-devel for all supported distros

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5180 from SUSE/wip-add-missing-specfile-bit

ceph.spec.in: add missing -%{release}

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5076 from SUSE/wip-logrotate-hard-dependency

packaging: make logrotate a hard dependency across all distros

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
 Conflicts:
debian/control

ceph.spec.in: install 95-ceph-osd.rules, mount.ceph, and mount.fuse.ceph properly on SUSE

http://tracker.ceph.com/issues/12261 Fixes: #12261

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5079 from SUSE/wip-12166

ceph.spec.in: use %_udevrulesdir to eliminate conditionals

Merge pull request #5126 from SUSE/wip-12201

ceph.spec.in: /var/run/ceph fixes

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

ceph.spec.in: use _udevrulesdir to eliminate conditionals

The conditionals governing where 50-rbd.rules is installed were not doing the
right thing on SUSE distros.

Start using the %_udevrulesdir RPM macro, while taking care that it is defined
and set to the right value. Use it to eliminate some conditionals around other
udev rules files as well.

http://tracker.ceph.com/issues/12166 Fixes: #12166

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5172 from ceph/wip-rm-git-build-dep

Remove git build-time dependency

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5167 from zaitcev/wip-rgw-doc-layout

rgw: Document the layout of pools and objects

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5184 from majianpeng/compiler-fix

Compiler fix warning message

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

rgw: fix signed/unsigned compare warning.

rgw/rgw_op.cc: In member function 'virtual void
RGWCompleteMultipart::execute()':
rgw/rgw_op.cc:3133:27: warning: comparison between signed and unsigned
integer expressions [-Wsign-compare]
   if (parts->parts.size() > s->cct->_conf->rgw_multipart_part_upload_limit) {

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

test: fix signed/unsigned compare warning.

 In file included from test/objectstore/chain_xattr.cc:29:0:
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of
‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*,
const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 =
int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from
‘static testing::AssertionResult
testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*,
const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 =
int; bool lhs_is_null_literal = false]’
test/objectstore/chain_xattr.cc:205:314:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison
between signed and unsigned integer expressions [-Wsign-compare]
   if (expected == actual) {

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #5176 from zhouyuan/rest_bench_https

tools: Fix rest_bench to support https

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5174 from yuyuyu101/fix-bufferlist-zero

Buffer: Fix bufferlist::zero bug

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

ceph.spec.in: snappy-devel for all supported distros

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: make /var/run/ceph conditional

/var/run/ceph should only be installed on Fedora, RH/CentOS, and
all supported SUSE distros that do not have systemd (e.g. SLE11-SP3).

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: add missing -%{release}

We have it everywhere else and I can't think of any reason why
it should be omitted here.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5179 from yghannam/yghannam-workunit-fixes-pjd

Workunits : suites/pjd.sh : Do make clean so make can build on curren…

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5178 from yghannam/yghannam-workunit-fixes-chmod

Workunits : fs/misc/chmod.sh : Include ACL characters in permission c…

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Workunits : fs/misc/chmod.sh : Include ACL characters in permission check.

Signed-off-by: Yazen Ghannam <yazen.ghannam@linaro.org>

Workunits : suites/pjd.sh : Do make clean so make can build on current arch.

Signed-off-by: Yazen Ghannam <yazen.ghannam@linaro.org>

Fix rest_bench to support https

Fixes: #3968
There's typo in the code which has compared 'http' twice

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

buffer: Fix bufferlist::zero bug with special case

Fixes: #12252
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>