mon: Monitor: use 'ceph mon compact' instead of 'ceph compact'

Makes it easier to identify the command as being related with the
monitor instead of cluster-wide.

This entails adding an exception to module 'mon' in order to have this
command handled by the Monitor class instead of MonmapMonitor (which is
the one traditionally handling 'mon' module commands).

Fixes: #11545
Signed-off-by: Joao Eduardo Luis <joao@suse.de>

qa/workunits: cephtool: test deprecated commands

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

PendingReleaseNotes: note deprecation of commands

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: MonCommands.h: DEPRECATE 'ceph sync force'

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: MonCommands.h: DEPRECATE 'ceph compact'

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: MonCommands.h: DEPRECATE 'ceph scrub'

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: Monitor: allow deprecating commands and debugging as obsolete

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: Monitor: add is_noforward() helper to MonCommand

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: Monitor: add support to have OBSOLETE commands

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: MonCommand: don't match help string in is_compat()

Otherwise it's virtually impossible to change a command's help string
without triggering a mismatch with the leader's command set.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: MonCommands: add NOFORWARD to 'ceph sync force'

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: MonCommand: add FLAG_NONE

Instead of passing '0' for commands without flags, pass FLAG_NONE
instead.  It's prettier and more obvious -- and it only costs 64 bits.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: MonCommand: have flags as uint32_t instead of enum

Makes much more sense if we're OR'ing flags.  Or not as weird.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: MonCommands: accept FLAG(f) instead of 'f' in command sig

This allows us to do nifty stuff like 'FLAG(foo) | FLAG(bar)' and expand
it to (MonCommand::FLAG_foo | MonCommand::FLAG_bar), instead of being
bound by a single flag on macro expansion.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5143 from trociny/fix-mds_metadata

mon: fix mds metadata

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4917 from hjwsm1989/wip-rest-bench

 tools/rest_bench.cc: bucket is not mandatory bc we have a default one

Reviewed-by: Kefu Chai <kchai@redhat.com>

  rest_bench: bucketname is not mandatory as we have a default name

Signed-off-by: huangjun <hjwsm1989@gmail.com>

  rest_bench: drain the work queue to fix a crash
Fixes: #3896
Signed-off-by: huangjun <hjwsm1989@gmail.com>

Merge pull request #5249 from tchaikov/wip-11101

ceph.in: improve the error message

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5110 from ceph/wip-11864-rgw-mime-types-pkg

11864: packaging: RGW depends on /etc/mime.types

Reviewed-by: Boris Ranto <branto@redhat.com>

packaging: RGW depends on /etc/mime.types

If the mimecap RPM or mime-support DEB is not installed, then the
/etc/mime.types file is not present on the system. RGW attempts to read
this file during startup, and if the file is not present, RGW logs an
error:

  ext_mime_map_init(): failed to open file=/etc/mime.types ret=-2

Make the radosgw package depend on the mailcap/mime-support packages so
that /etc/mime.types is always available on RGW systems.

http://tracker.ceph.com/issues/11864 Fixes: #11864

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #4699 from joehandzik/master

FileStore: Collect device partition information

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4876 from ceph/wip-11855

crush: fix 'ceph osd crush reweight-subtree'

Reviewed-by: Li Wang <liwang@ubuntukylin.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph.in: print more detailed warning for 'ceph <type> tell'

put the full signature of "tell <target> <command> [options...]"
instead of "tell {0}.<id>", which could be misleading somehow.

Fixes: 11101
Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.in: print more detailed error message for 'tell' command

* we do not allow user specify a certain daemon when starting an
  interactive session. the existing error message could lead to
  some confusion. so put more details in it.

Fixes: #11101
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5245 from yuyuyu101/wip-async-fix-18

AsyncConnection: Fix incorrect sign_message usages

Merge pull request #5244 from zhouyuan/rgw_obj_stripe_size

rgw: doc: Fix radosgw stripe size config ref

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5240 from dachary/wip-ceph-release-notes

tools: src/script/ceph-release-notes normalization

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5165 from zhouyuan/librados_default_crush_ruleset

Make librados pool_create respect default_crush_ruleset

Reviewed-by: Kefu Chai <kchai@redhat.com>

librados: Make librados pool_create respect default_crush_ruleset

Make pool_create in librados use the osd_pool_default_crush_replicated_ruleset

Fixes: #11640
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

AsyncConnection: Make sign_message ahead of construct message bufferlist

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Fix non-fastdispatch message doesn't prepare case

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

rgw: doc: Fix radosgw stripe size config ref

s/rgw object stripe size/rgw obj stripe size
https://github.com/ceph/ceph/blob/master/src/common/config_opts.h#L1024

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

tools: src/script/ceph-release-notes normalization

Issue an error if for the title does not start with a known prefix.
Issue an error if no issue is associated to a pull request.

Group the pull requests fixing the same issue together.

Add link to the issue and the pull request.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #5230 from majianpeng/check_ops_in_flight_cleanup

common: clean up code for OpTracker::check_ops_in_flight.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

common: clean up code for OpTracker::check_ops_in_flight.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #5220 from vumrao/wip-vumrao-12318

doc: change tcp rcvbuf and tcp nodelay to ms tcp rcvbuf and ms tcp nodelay

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: change tcp rcvbuf and tcp nodelay to ms tcp rcvbuf
     and ms tcp nodelay

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

Merge pull request #4939 from ceph/wip-offline-backward

cephfs-data-scan (offline backward file recovery)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5190 from guce/master

radosgw-admin: after subuser modify print only once user info

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5183 from zhouyuan/rgw_max_put_size_configurable

rgw: Make RGW_MAX_PUT_SIZE configurable

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5213 from wido/wip-rgw-keepalive

rgw: If the client sends a Connection: close header respond accordingly.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5210 from tweenk/docs-improvements

Document librbd::parent_spec and librbd::parent_info.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5215 from SUSE/wip-fdupes

ceph.spec.in: do not run fdupes, even on SLE/openSUSE

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

ceph.spec.in: do not run fdupes, even on SLE/openSUSE

In openSUSE there is a policy to use %fdupes in the spec file if RPMLINT
complains about duplicate files wasting space in the filesystem.

However, RPMLINT is not so complaining, so drop fdupes.

http://tracker.ceph.com/issues/12301 Fixes: #12301

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5102 from ceph/wip-mds-refactor-small

Minor MDS:: refactor

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5209 from zhouyuan/rgw_swift_user

rgw: Make vstart.sh print out swift user info

Reviewed-by: Sage Weil <sage@redhat.com>

packaging: add cephfs-data-scan

Signed-off-by: John Spray <john.spray@redhat.com>

tools/cephfs: add cephfs-data-scan

For scraping cephfs data pools for metadata in
backtraces, and injecting that metadata into
the cephfs metadata pool (initially offline).

Signed-off-by: John Spray <john.spray@redhat.com>

doc: add some docs about cephfs-data-scan

These are deliberately fairly sparse, because:
 * These tools are for experts
 * These tools may well be wrapped in a higher
   level recovery tool that orchestrates parallel
   workers at some stage.

Signed-off-by: John Spray <john.spray@redhat.com>

include/ceph_fs.h: define magic LOST+FOUND ino

So that recovery tools can create this dir without
having to consult/modify InoTable.

Signed-off-by: John Spray <john.spray@redhat.com>

cls: add CephFS object class

For use in cephfs-data-scan for file size and mtime
recovery, by doing a set-if-greater for size/mtime
xattrs on the 0th object of a file.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: include damage_flags in inter-mds encoding

...and correspondingly bump the cluster protocol
version as these structures are not versioned
themselves.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: bump required version on fnode_t encoding

So that older MDS instances can't miss/ignore
the damage flags.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add damage_flags_t to inode+frag

Used to mark inodes+fnodes with flags indicating
that particular types of damage have either
been detected (scrub/fetch) or caused (cephfs-data-scan
inject).

This should serve as a cue to the MDS to do online
repair of stats for the nodes that have been injected.

Signed-off-by: John Spray <john.spray@redhat.com>

mds/CInode: move hash_dentry_name up into InodeStore

Because it only depends on the data in the store, not
anything from a live MDS, and it's useful when
we're dealing with fragmented directories from
an offline tool.

Signed-off-by: John Spray <john.spray@redhat.com>

tools: fix journal reset error handling

When we failed and prompted for --force, it still
looked like a success.  ::reset never had any
proper error handling since it was lifted out
of the MDS, so add it now.

Signed-off-by: John Spray <john.spray@redhat.com>

client: fix typo

s/bye/byte/

Signed-off-by: John Spray <john.spray@redhat.com>

mds: correct typo in log message

s/issueing/issuing/

Silly, but it annoyed me every time I saw it...

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5164 from ceph/wip-12088

client: reference counting 'struct Fh'

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: If the client sends a Connection: close header respond accordingly.

HTTP/1.1 assumes Keep-Alive by default, but if a Connection: close header is send
the server should respond with it as well.

This makes the client close the connection after the request.

Fixes: #12298

Merge pull request #5201 from ceph/wip-12272

tools: Fix rados export to use io_ctx in the right way

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: Make vstart.sh print out swift user info

Currently vstart.sh only print out S3 user info. This patch updates it
to print swift user info too. Also changes the swift account/user/password
to test/tester/testing which align with openstack-swift better:
https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L270

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Fixes: #12286 radosgw-admin: after subuser modify print only once user info.
remove rgw_admin.cc OPT_SUBUSER_MODIFY, show_user_info code block.

switch (opt_cmd) {
...
case OPT_SUBUSER_MODIFY:
show_user_info(info, formatter);       //show first time (remove this)
break;
...
}

// output the result of a user operation
if (output_user_info) {
...
show_user_info(info, formatter);       //show second time
}

test fix:
before: after subuser modify print twice user info.
after changes, do the same procedure, print only once user info.

Signed-off-by: guce guce@h3c.com

rgw: Make RGW_MAX_PUT_SIZE configurable

The 5GB limit of a single operation uploading was part of S3 spec.
However some private setups may have some special requirements
on this limit. It's more convinent to have a configurable value.

Fixes: #6999
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

tools: Fix rados export to use io_ctx in the right way

Cause by: 868f6b7626db4a802b38d1d8ce5135108b322f74

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #5197 from ceph/wip-cot-meta

Wip cot meta

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5198 from tchaikov/wip-doc-erasure-fix

doc/erasure-code: fix couple typos

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/erasure-code: fix couple typos

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools, test: Some ceph-objectstore-tool error handling fixes

Improve various error messages generated with invalid syntax
Add test cases for most of these error messages

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Add debug argument to the ceph-objectstore-tool test

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Check for valid --op earlier so we can get a better error message

Signed-off-by: David Zafman <dzafman@redhat.com>

tools, test: Add ceph-objectstore-tool to operate on the meta collection

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Fix newlines in output of --op list

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Fix dump-super which doesn't require pgid

A valid pgid had to be specified in order for superblock to be displayed,
but it was otherwise ignored.
Fix message that didn't include dump-super

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Check and specify commands that require the pgid specification

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #4404 from rzarzynski/wip-11431

rgw: fix lack of account name attribute in XML-formatted bucket listing of Swift account

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5078 from rzarzynski/wip-12158

rgw: send Content-Length in response for GET on Swift account.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge branch 'wip-temp'

Conflicts:
src/osd/ReplicatedPG.cc
src/test/osd/RadosModel.h

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

osd, tools: Always filter temp objects since not being exported

Can't use object_locator_to_pg() directly with a temp object
because the pool is negative.

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Don't export temporary objects until we have persistent-temp objects

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: fix temp clearing in OSD

Signed-off-by: David Zafman <dzafman@redhat.com>

Document librbd::parent_spec and librbd::parent_info.

Add an explanatory comment to parent_types.h that explains what
the "overlap" member in parent_info means. This was not exactly easy
to figure out.

Signed-off-by: Krzysztof Kosiński <krzysztof.kosinski@intel.com>

Merge pull request #5175 from david-z/wip-mds-dump-crash

Fix mds dump_ops_in_flight crashing ocassionally

Fix mds dump_ops_in_flight crashing ocassionally

Merge pull request #4977 from theanalyst/rados/wip-9737

rados/tool: handle --snapid correctly

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5185 from yuyuyu101/wip-store-test-zero

StoreTest: Add zero test for SyntheticTest

Merge pull request #3273 from majianpeng/fix4

cleanup code for RecoveyMessages

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #4143 from majianpeng/ec-notsupp-omap

fix bugs about cache tier.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4329 from majianpeng/filestore-copy

Filestore copy:using splice instead read/write for clone

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #4458 from wonzhq/dup-promote

osd/ReplicatedPG: need to queue op when avoiding dup promotion in maybe_handle_cache

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #5075 from SUSE/wip-logrotate-requires

packaging: add find and which dependencies

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

packaging: add find and which dependencies

The postrotate script in src/logrotate.conf uses the which and find utilities
to do its work. Although logrotate itself is only a Recommends, I think which
and find are so ubiquitous that it makes sense to have them as hard
dependencies.

Also, I checked and the package names which and findutils are the same on all
the RPM distros we are currently targeting in the spec file.

In Debian, find is also in a package called findutils while the package
containing 'which' is called debianutils.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5187 from SUSE/wip-12261

ceph.spec.in: install 95-ceph-osd.rules, mount.ceph, and mount.fuse.c…

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5181 from SUSE/wip-snappy-devel

ceph.spec.in: snappy-devel for all supported distros

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5180 from SUSE/wip-add-missing-specfile-bit

ceph.spec.in: add missing -%{release}

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5076 from SUSE/wip-logrotate-hard-dependency

packaging: make logrotate a hard dependency across all distros

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
 Conflicts:
debian/control

ceph.spec.in: install 95-ceph-osd.rules, mount.ceph, and mount.fuse.ceph properly on SUSE

http://tracker.ceph.com/issues/12261 Fixes: #12261

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5079 from SUSE/wip-12166

ceph.spec.in: use %_udevrulesdir to eliminate conditionals

Merge pull request #5126 from SUSE/wip-12201

ceph.spec.in: /var/run/ceph fixes

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>