doc: s/Asphyxiate/Breathe/ in documenting.rst

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: enable doxygen for enum

* since we switched to breathe, which supports rendering enum
  types

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: Switch doxygen integration back to breathe

* asphyxiate needs more toolings, see
  https://github.com/ceph/asphyxiate/issues/1
* this commit basically reverts c96064
* use `autodoxygen` directive to doxygen referenced source files
* do not call `doxygen` explicitly in `build-doc`, `autodoxygen`
  will take care of it.

Fixes: #6115
Fixes: #6115
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3983 from yuyuyu101/wip-async-fix-3

Wip async fix 3

Reviewed-by: Sage Weil <sage@redhat.com>

Thread.cc: Make set_affinity private and correct behavior

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

doc/release-notes: more hammer release notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4085 from ceph/wip-11160

rgw: don't use rgw_socket_path if frontend is configured

Reviewed-by: Sage Weil <sage@redhat.com>

doc/release-notes: make hammer note about filestore format upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3885 from Ved-vampir/master

common: add perf counters description

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #3840 from ceph/wip-libcephfs-fsetxattr

libcephfs: add ceph_f{get,set,list,remove)_xattr

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #3750 from ceph/wip-dump-cache

mds: Add 'dump cache' asok

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mds: implement 'dump cache' asok command

Retains old text format output when user specifies
a filename, else returns JSON.  JSON output will
cope less well with large caches, but will be more
useful for automated tests.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: use MDSCacheObject::dump in CDir::dump

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add CDentry::dump

Signed-off-by: John Spray <john.spray@redhat.com>

mds: extended dump() for CInode

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add dump() to MDSCacheObject

Subclasses extend this with their own attributes.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add SimpleLock::dump

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add a dump() method to MutationImpl

This is just an externally callable wrapper
to _dump that doesn't require caller
to pass in a time.

Signed-off-by: John Spray <john.spray@redhat.com>

include/frag: add a dump() method to fragtree_t

Signed-off-by: John Spray <john.spray@redhat.com>

mds: move auth_pins attrs into MDSCacheObject

These were common between CInode, CDir, CDentry.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge branch 'master' of github.com:ceph/ceph

Merge remote-tracking branch 'gh/hammer'

Conflicts:
PendingReleaseNotes
src/Makefile.am
src/gmock
src/test/Makefile.am

Merge pull request #4035 from ceph/wip-11113

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3988 from ceph/wip-refine-build-configuration-hammer

Refine build configuration for hammer

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4104 from rzarzynski/wip-11148

rgw: rectify 202 Accepted in response for PUT on already existing bucket

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4105 from athanatos/wip-11110

Wip 11110

Reviewed-by: Sage Weil <sage@redhat.com>

init-radosgw*: don't require rgw_socket_path to be defined

Fixes: #11159
Backport: hammer, firefly

Scripts required rgw_socket_path to exist in order to start radosgw.
This is not needed.

Reported-by: Dan Mick <dmick@redhat.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 6823bcdcd0ce72cd223e809291f46d82da76115c)

Merge pull request #4087 from ceph/wip-11159

init-radosgw*: don't require rgw_socket_path to be defined

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4064 from dachary/wip-10488-jerasure-idempotent

mon: informative message when erasure-code-profile set fails

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4066 from dachary/wip-11144-erasure-code-profile-rm

osd: erasure-code-profile incremental rm before set

Reviewed-by: Sage Weil <sage@redhat.com>

PG: ensure that info.last_epoch_started only increases

See doc/dev/osd_internals/last_epoch_started.rst

Fixes: #11110
Signed-off-by: Samuel Just <sjust@redhat.com>

doc: add last_epoch_started.rst

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #4054 from diurchenko/wip-10662

rgw: Swift API. Allows setting attributes with COPY object operation.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rectify 202 Accepted in response for PUT on existing bucket.

Fixes: #11148
Backport: hammer
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #4025 from ceph/wip-fix-configure-noversion

configure.ac: add --disable-gitversion

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

configure.ac: add --disable-gitversion

During development each git commit cause currently a complete
rebuild of the source tree. By passing --disable-gitversion to
configure we set a generic version string for ceph to prevent
getting after each commit a new version set.

remove src/check_version and replace with a extended version
of src/make_version which allows detecting changes in .git_version
and ceph_ver.h to avoid not needed updates.

This change also forces to call make_version each time to make
sure that changes from configure or the NO_VERSION env variable
are processed.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

init-radosgw*: don't require rgw_socket_path to be defined

Fixes: #11159
Backport: hammer, firefly

Scripts required rgw_socket_path to exist in order to start radosgw.
This is not needed.

Reported-by: Dan Mick <dmick@redhat.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't use rgw_socket_path if frontend is configured

Fixes: #11160
Backport: hammer, firefly

Previously if we wanted to use the tcp fcgi socket, we needed to clear
rgw_socket_path.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3951 from ceph/wip-cephfs-snaps

Wip cephfs snaps

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #4072 from javierguerragiraldez/master

rados.py: fix Object.write() method

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rados.py: fix Object.write() method

the file position should advance by the number of bytes written,
but the ioctx.write() function returns 0 on success

Signed-off-by: Javier Guerra <javier@guerrag.com>

Merge pull request #3898 from jdurgin/wip-rbd-merge-diff

qa: make rbd-fuse exit cleanly with lttng

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4071 from athanatos/wip-10718

Wip 10718

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4016 from ceph/wip-11123

osd: fix whiteout handling for delete+create compound ops

Reviewed-by: Samuel Just <sjust@redhat.com>

PG: make sure to update history.last_epoch_started with first write

Otherwise, we might remember the write, but not that the activation
interval was history.last_epoch_started.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #4067 from ceph/wip-rbd-warning

tests: remove unused variable

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

tests: remove unused variable

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

osd: erasure-code-profile incremental rm before set

It is possible for an incremental change to have both a rm and a set for
a given erasure code profile. It only happens when a rm is followed by a
set. When a set is followed by a rm, the rm will remove the pending set
in the incremental change.

The logic is the same for pool create and pool delete.

We must apply the incremental erasure-code-profile removal before the
creation otherwise rm and set in the same proposal will ignore the set.

This fix is minimal. A better change would be that erasure-code-profile
set checks if there is a pending removal and wait_for_finished_proposal
before creating.

http://tracker.ceph.com/issues/11144 Fixes: #11144

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mon: informative message when erasure-code-profile set fails

When erasure-code-profile set refuses to override an existing profile,
it may be non trivial to figure out why. For instance:

   ceph osd set default ruleset-failure-domain=host

fails with:

   Error EPERM: will not override erasure code profile default

although ruleset-failure-domain=host is documented to be the
default. The error message now includes the two profiles that have been
compared to not be equal so that the user can verify the difference.

   Error EPERM: will not override erasure code profile default
   because the existing profile
   {directory=.libs,k=2,m=1,plugin=jerasure,technique=reed_sol_van}
   is different from the proposed profile
   {directory=.libs,k=2,m=1,plugin=jerasure,ruleset-failure-domain=host,technique=reed_sol_van}

http://tracker.ceph.com/issues/10488 Fixes: #10488

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mds: skip inode that is being puring when opening snap past parent

we can not increase reference count of purging inodes. purging
inodes have no snapshots, so it's OK to skip them.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: don't crash MDS when snapshot data not found

When remote dentry and inode are in different snapshot realms.
The inode may not contain snapshot data for snapshot of the remote
dentry.

This issue are difficult to fix. This patch MDS print an error
instead of triggering assersion.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: pick head inode when caps follows is zero

"caps follows is zero" means client does not flush anything, so
we shoud pick the head object

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: fix purging snapshotted stray inode

If stray inode has no snapshot data, it's OK to purge it even if
it's still referenced by some snapsthots. This is because seperate
snap inodes were created for snapshots.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: check snapset in journal_cow_dentry()

Removing snapshot also increases snaprealm's sequence number.
So only using snaprealm's sequence number to decide if we should
COW dentry is too loose. This patch makes journal_cow_dentry()
also check snaprealm's snapset.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: fix fnode.snap_purged_thru check

We do not alway succeed in purging all stale snap dentries
(stale snap dentris can be in-use or have been trimmed from
cache). Updating fnode.snap_purged_thru prematurely will make
us not purge the remainning stale snap dentries. If CDir
has no cached snap dentry when fetching dirfrag, we can
guaranee that all stale snap dentries will be purged.
So we only update fnode.snap_purged_thru in this case.

Now the code that purges stale snap dentries is executed more
frequently. So this patch also make readdir not purge stale
snap dentries

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: purge stale snap data in CInode with snaprealm

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: fix purging stale snapshot dentries

When fetching dirfrag, record stale snap dentries in CDir. So that
CDir::_omap_commit can remove these stale dentires from OMAP.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #3904 from yuyuyu101/wip-librbd-metadata

Wip librbd metadata

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4013 from ceph/wip-5488

librbd: fix deadlock on image refresh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Librbd: Add "start" and "max" arguments to metadata_list

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3953 from wuxingyi/patch-2

rgw/logrotate.conf: Rename service name

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rgw/logrotate.conf: Rename service name

The service name for ceph rados gateway was changed to "ceph-radosgw",
the previous version of service name "radosgw" would cause a failed reload,
and finally make it impossible to write any log data to the log file.

Signed-off-by: wuxingyi <wuxingyi2015@outlook.com>

Merge pull request #4057 from ceph/wip-rgw-bootstrap

mon: add bootstrap-rgw profile

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4055 from vuhuong/wip-xio

XIO fixes

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-create-keys: create bootstrap-rgw key

Signed-off-by: Sage Weil <sage@redhat.com>

mon: add 'bootstrap-rgw' profile

This profile is allowed to create client.rgw.* librados users.  They have
access to any pool, which is pretty coarse.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: v0.67.11 draft notes

Signed-off-by: Sage Weil <sage@redhat.com>

Compile test_build_librgw only if WITH_BUILD_TESTS

Merge pull request #4045 from NTmatter/docs-raid4-removal

doc-rados-operations-crush-map: Swap Raid4 for Erasure

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/hammer'

Conflicts:
src/gmock
src/msg/xio/XioPortal.h

Merge pull request #3910 from athanatos/wip-11027

Wip 11027

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3918 from tchaikov/hammer-fix-leaks-in-C_TwoContexts

osdc: fix a memory leak in C_TwoContexts

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3972 from tchaikov/hammer-fix-msg-leak

osd: fix a msg leak

Reviewed-by: Sage Weil <sage@redhat.com>

osd: use (robust) helper for setting exists or clearing whiteout

The current blanket check in prepare_transaction() will trigger only when
there is a net obs.exists change from the commited obs to new_obs.
However, this misses the case where the first osd_op is a delete and then a
subsequent osd_op recreates the object.  Changing the whiteout check to
look only at new_obs does not work because it fails to understand when
_delete_oid sets the whiteout and will simply clear it again.

In order to support sequences of delete + create in general, we need to do
the whiteout flag clearing when the actual create happens (to match the
fact that we set it when we process the delete osd_op).  Use a helper to
do this and consolidate most other obs.exists = true code to use it.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3994 from athanatos/wip-11069

PGLog: split divergent priors as well

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

PGLog::merge_log: in tail extend case, log.log might be empty

Even if the tail != last_update, the log might be empty due to split
moving all entries into other logs.

Fixes: 10718
Signed-off-by: Samuel Just <sjust@redhat.com>

doc-rados-operations-crush-map: Swap Raid4 for Erasure

Signed-off-by: Thomas Johnson <NTmatter@gmail.com>

Merge pull request #4001 from rzarzynski/wip-10650

rgw: send Last-Modified header in response for PUT on Swift's object.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3925 from rzarzynski/wip-11088

rgw: fix handling empty metadata items on Swift container.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

xio: Add xio_transport_type

Add xio_transport_type to control and run xio messenger over
rdma or tcp

Signed-off-by: Vu Pham <vu@mellanox.com>

xio: Update README.xio with dependencies

Signed-off-by: Vu Pham <vu@mellanox.com>

Merge pull request #4024 from ceph/wip-unused-var

mds: remove double-define on iterator

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mds: remove double-define on iterator

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #3976 from wonzhq/hit-set-check

osd/ReplicatePG: correctly checking if an object is in hit set

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4017 from ceph/wip-11125

rgw: update keystone cache with token info

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4019 from NTmatter/doc-typo-monap

doc/rados/operations/add-or-rm-mons: correcting minor typo

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/rados/operations/add-or-rm-mons: correcting minor typo

Signed-off-by: Thomas Johnson <NTmatter@gmail.com>

librbd: snap_remove should ignore -ENOENT errors

If the attempt to deregister the snapshot from the parent
image fails with -ENOENT, ignore the error as it is safe
to assume that the child is not associated with the parent.

Fixes: #11113
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: get_parent_info should protect against invalid parent

get_parent_info should return -ENOENT if the image does not
have an associated parent image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

PGLog: split divergent priors as well

Signed-off-by: Samuel Just <sjust@redhat.com>

rgw: update keystone cache with token info

Fixes: #11125
Backport: hammer, firefly

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

PendingReleaseNotes: warn about lttng LD_PRELOAD for daemons

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3997 from ceph/wip-debian-mv-var-lib-ceph-mds

debian: move /var/lib/ceph/mds to ceph-mds package

Reviewed-by: James Page <james.page@ubuntu.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3945 from tchaikov/hammer-fix-xio-requeue

XIO: Handle requeue case of XIO messages

Merge pull request #3977 from tchaikov/fix-xio-loop-conn-cleanup

Fix XioLoopbackConnection Lifecycle.

Reviewed-by: Sage Weil <sage@redhat.com>

ceph_test_rados_tier: add test case for delete+create compound ops

This triggers #11123, originally observed when rgw does a delete + create
sequence in a single op and the whiteout gets left behind.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3872 from dachary/wip-7334-partx-test

ceph-disk: test activate on the second journal

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #4014 from ceph/wip-mon-doc

doc/rados/operations/add-or-rm-mons: revise doc a bit to be less confusing

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3922 from ceph/wip-11053

mds: fix assertion caused by system clock backwards

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #3752 from ceph/wip-10744

mds: give up replicas of a stopping mds's stuff

Reviewed-by: Yan, Zheng <zyan@redhat.com>