doc: Fixes spelling and grammatical errors

Signed-off-by: Vartika Rai <vartikarai17@gmail.com>

Merge pull request #4086 from tchaikov/fix-doxygen-warnings

doc: fix doxygen warnings

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4011 from ceph/wip-mds-damaged

mds: Add 'damaged' state and handle errors during rank startup

Reviewed-by: Greg Farnum <gfarnum@rehdat.com>

mds: handle read/replay errors in MDLog with damaged()

Signed-off-by: John Spray <john.spray@redhat.com>

mds: call damaged() on what were assertions in replay

i.e. categorize an unreplayable journal as a damaged rank.

NB there will be other cases where replay can assert out
while calling into other classes, but that's for another day.

Signed-off-by: John Spray <john.spray@redhat.com>

osdc/Journaler: improved error handling

Exceptions from ::decode are caught and turned
into return codes that we can handle via
the contexts used for completions.

The assertion that entry_size != 0 is removed in
JournalStream::read so that if it is zero that
just becomes a normal deserialization error
later when someone tries to decode the
zerolength journal event.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: catch exceptions in ::decode

...and translate them into more readily handlable -EINVALs

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #4102 from larkly/patch-docerror-usermgmt

Minor fix: bootstrap-mds in user-management.rst

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mds: handle encoding errors on MDSTable load

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #3893 from ceph/wip-10390

#10390: Refactor stray handling code and add throttling

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #3371 from nilamdyuti/wip-doc-ceph-object-gateway

Updates radosgw man page with configuration for mod_proxy_fcgi

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #4137 from majianpeng/assert-version

osd/ReplicatedPG: For CEPH_OSD_OP_ASSERT_VER, it should use

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4141 from tchaikov/sync-rst-with-manpages

doc: update .rst file with their .8 counterparts

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4117 from ceph/wip-libcephfs-flock

Fixes for libcephfs flock tests

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc: Adds updated radosgw-admin man page under man/

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Updates radosgw-admin man page.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Adds updated radosgw man page under man/

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Updates radosgw man page.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Adds updated radosgw man page under man/

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Updates radosgw man page with configuration for mod_proxy_fcgi .

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

mds: call damaged() on errors loading SessionMap

Signed-off-by: John Spray <john.spray@redhat.com>

mon/MDSMonitor: add "mds repaired"

For clearing an MDS rank out of `damaged`

Signed-off-by: John Spray <john.spray@redhat.com>

mds: emit DAMAGED on MDSTable load error

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add damaged() method

A hook to fire off one last DAMAGED beacon
to mon before respawning.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: implement Beacon::send_and_wait

For callers that want to send a beacon, and then
wait up to some timeout for a response
from the mon.  Specifically, shutdown handlers.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: report damaged ranks in health

Signed-off-by: John Spray <john.spray@redhat.com>

mon/MDSMonitor: handle DAMAGED in beacon

Signed-off-by: John Spray <john.spray@redhat.com>

mds: report damaged in MDSMap::print_summary

Signed-off-by: John Spray <john.spray@redhat.com>

mds/MDSMap: update print() for DAMAGED

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add `damaged` set to MDSMap

To track which ranks have been reported in state
DAMAGED by an MDS daemon.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: introduce DAMAGED state

This state is for MDS daemons to report to
the monitor that they encountered an unfixable
error during startup/replay.  The implication
is that this rank should not be assigned to
another MDS daemon until offline repair is
performed, thus avoiding it flapping around
between failing daemons.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: remove unused Beacon::last_send

This was just a typo in the header, although someone
dutifully also added initialization of it to the constructor
at some stage too!

Signed-off-by: John Spray <john.spray@redhat.com>

mds: remove a spurious #if 1

This crept in in f93bf385.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: be fairer in enqueuing purges

Don't let purges that require fewer ops skip ahead of those
that require more ops.

Signed-off-by: John Spray <john.spray@redhat.com>

doc: update .rst file with their .8 counterparts

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/ReplicatedPG: For CEPH_OSD_OP_ASSERT_VER, it should use
op.assert_ver rather than op.watch.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #4135 from ceph/wip-11198

Fix ceph_test_async_driver failed

Fix ceph_test_async_driver failed

This test will create 10000 sockets which will failed because of limited system fd. Actually we only need to create several hundreds sockets and it's enough to get the test's goal.

Fix bug #11198

Merge pull request #3873 from tchaikov/cli-human-readable-errno

rados: translate errno to str in CLI

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3841 from ceph/wip-10217

qa/workunits/fs/misc: fix filelock_interrupt.py

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #3829 from liewegas/wip-recovery-sleep

osd: inject simple sleep in recovery

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3685 from XinzeChi/wip-mark-started

osd: clear extra mark_started

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3282 from tchaikov/pq-test

common/PrioritizedQueue: add basic tests for it

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3177 from davidzhang84/wip-inject-eio-ec

Add EIO injection on EC pool for mdata and data error only for now

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3889 from XinzeChi/wip-tier-stat

osd: add l_osd_op_cache_hit perf counter for cache pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4010 from majianpeng/osdmap-nspace

mon:make 'ceph osd map' accept namespace.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4130 from dachary/wip-backports

doc: what does it mean for a release to be supported

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

doc: what does it mean for a release to be supported

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4122 from trhoden/rgw-quickstart

doc: Add RGW quick start info

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4098 from ceph/wip-11128

auth: reinitialize NSS modules after fork()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

auth: use crypto_init_mutex to protect NSS_Shutdown()

Signed-off-by: Yan, Zheng <zyan@redhat.com>

doc: Add RGW quick start info

Add details of how to use ceph-deploy to deploy the CivetWeb-based
RGW present in Hammer.

Signed-off-by: Travis Rhoden <trhoden@redhat.com>

Merge pull request #4120 from trhoden/quickstart-mon-create

doc: remove reference to old ceph-deploy steps

Reviewed-by: Sage Weil <sage@redhat.com>

doc: remove reference to old ceph-deploy steps

ceph-deploy 1.1.3 is ancient, and isn't easily found or installed.

No need to reference it and call out the steps necessary to use if
using it.

Signed-off-by: Travis Rhoden <trhoden@redhat.com>

Merge pull request #4103 from ceph/wip-error-str

mds: fix error_str formatting in scrub output

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4116 from ceph/wip-10368

mds: call wait_for_safe twice in command_flush_journal

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mds: call wait_for_safe twice in command_flush_journal

...so that we wait until everyone who was waiting for the first
wait_for_safe has run their completion before we proceed, thus
avoiding the possibility that those completions will execute
during our trimming and cause problems.

Fixes: #10368
Signed-off-by: John Spray <john.spray@redhat.com>

auth: reinitialize NSS modules after fork()

Fixes: #11128
Signed-off-by: Yan, Zheng <zyan@redhat.com>

auth: properly return error of cephx_calc_client_server_challenge

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Minor fix: Documentation referred to bootstrap-osd instead of bootstrap-mds

Signed-off-by: Bosse Klykken <larkly@gmail.com>

mds: improved doxygen comments in StrayManager

Signed-off-by: John Spray <john.spray@redhat.com>

mds: update StrayManager op limit on mds_max_purge_ops_per_pg

Previously was only updating on mds_max_purge_ops

Signed-off-by: John Spray <john.spray@redhat.com>

mds: fix parent check in MDCache::eval_remote

This code path wasn't touched until the change
in "mds/Server: fix stray reintegration".  Now we
see that it will crash in fsstress in some circumstances
when the client is doing a link operation, where
we are checking the projected linkage on the DN
before going into this function, but not the projected
parenthood of the inode.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: make purge op limit dynamic based on PGs/MDSs

Signed-off-by: John Spray <john.spray@redhat.com>

mds: throttle purge stray operations

Pull out the stray-handling code into a separate StrayManager
class.

There is an additional improvement to stray reintegration here,
by passing the DN that triggered the reintegration through,
we should make a better choice about *which* hardlink to
reintegrate the inode into, rather than just picking the first.

Fixes: #10390
Signed-off-by: John Spray <john.spray@redhat.com>

common/config_opts: add purge throttle options

These are to be respected by the MDS in the number
of concurrent RADOS delete operations and
number of files to be purged at once.

Signed-off-by: John Spray <john.spray@redhat.com>

osdc: make op count in Filer::purge configurable

Previously this was simply hardcoded to do up to
10 RADOS delete ops at a time.  Make this a config
option so that Filer consumers (like the MDS) can
know how many concurrent operations will be done
on their behalf.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: eval stray on remove dentry replica

This happens because when handling an MCacheExpire,
we expire inodes first and then dentries, so when
the inodes are put() their dentries still are replicated
so eval_stray fails.  Do an explicit call to maybe_eval_stray
when handling the expire for the dentry in order to catch
this.

One hits this path when migrating a stray away from
a stopping MDS rank.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: expire other ranks' mydirs during 'stopping'

Avoids the surviving MDSs thinking their base dir is still
replicated after a rank has been stopped.

Signed-off-by: John Spray <john.spray@redhat.com>

mds/Server: fix stray reintegration

This was checking tracei's primary parent DN for remoteness
as a condition to calling eval_remote.  However, the *primary*
parent DN in this case is the stray, which is not remote.

The correct check is tracedn (the remote dentry that got us here)
rather than tracei->get_parent_dn() (the primary dentry for the
inode that we have looked up).

This leads to eval_remote being invoked, and reintegration
of stray hard-linked inodes working.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #4114 from apeters1971/wip-isa-lrucache

erasure-code: make ErasureCodeIsaTableCache drop entries according to LRU

Reviewed-by: Loic Dachary <ldachary@redhat.com>

erasure-code: make ErasureCodeIsaTableCache drop entries according to LRU

There was a bug in the arguments used by the splice function to move elements from the LRU list to the head of the list.
To simplify the implementation the direction of the LRU list has been changed e.g. newest entries are pushed in front.

Signed-off-by: Andreas-Joachim Peters <andreas.joachim.peters@cern.ch>

Merge pull request #4084 from ceph/wip-switch-to-breathe

Wip switch to breathe

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: s/Asphyxiate/Breathe/ in documenting.rst

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: enable doxygen for enum

* since we switched to breathe, which supports rendering enum
  types

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: Switch doxygen integration back to breathe

* asphyxiate needs more toolings, see
  https://github.com/ceph/asphyxiate/issues/1
* this commit basically reverts c96064
* use `autodoxygen` directive to doxygen referenced source files
* do not call `doxygen` explicitly in `build-doc`, `autodoxygen`
  will take care of it.

Fixes: #6115
Fixes: #6115
Signed-off-by: Kefu Chai <kchai@redhat.com>

test/libcephfs/flock.cc: don't release lock before EWOULDBLOCK check

make sure the main thread/process not release the file lock before
workers execute the operation that supposes to return EWOULDBLOCK.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

test/libcephfs/flock.cc: fix synchronization points

In the threesome test cases, the main thread/process call PING_WORKER()
twice at each synchronization point. It's possible that one worker gets
both notifications and takes two steps forward, another worker gets no
notification.

The fix is use seperate semaphores for odd/even number synchronization
points.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #3983 from yuyuyu101/wip-async-fix-3

Wip async fix 3

Reviewed-by: Sage Weil <sage@redhat.com>

Thread.cc: Make set_affinity private and correct behavior

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

doc/release-notes: more hammer release notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4085 from ceph/wip-11160

rgw: don't use rgw_socket_path if frontend is configured

Reviewed-by: Sage Weil <sage@redhat.com>

doc/release-notes: make hammer note about filestore format upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3885 from Ved-vampir/master

common: add perf counters description

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #3840 from ceph/wip-libcephfs-fsetxattr

libcephfs: add ceph_f{get,set,list,remove)_xattr

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #3750 from ceph/wip-dump-cache

mds: Add 'dump cache' asok

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mds: implement 'dump cache' asok command

Retains old text format output when user specifies
a filename, else returns JSON.  JSON output will
cope less well with large caches, but will be more
useful for automated tests.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: use MDSCacheObject::dump in CDir::dump

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add CDentry::dump

Signed-off-by: John Spray <john.spray@redhat.com>

mds: extended dump() for CInode

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add dump() to MDSCacheObject

Subclasses extend this with their own attributes.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add SimpleLock::dump

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add a dump() method to MutationImpl

This is just an externally callable wrapper
to _dump that doesn't require caller
to pass in a time.

Signed-off-by: John Spray <john.spray@redhat.com>

include/frag: add a dump() method to fragtree_t

Signed-off-by: John Spray <john.spray@redhat.com>

mds: move auth_pins attrs into MDSCacheObject

These were common between CInode, CDir, CDentry.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge branch 'master' of github.com:ceph/ceph

Merge remote-tracking branch 'gh/hammer'

Conflicts:
PendingReleaseNotes
src/Makefile.am
src/gmock
src/test/Makefile.am

Merge pull request #4035 from ceph/wip-11113

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3988 from ceph/wip-refine-build-configuration-hammer

Refine build configuration for hammer

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4104 from rzarzynski/wip-11148

rgw: rectify 202 Accepted in response for PUT on already existing bucket

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>