rgw: remove meta file after deleting bucket
The meta file is deleted only if the bucket meta data is not synced

Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Fixes: #11149
Backport: hammer, firefly

Merge pull request #3867 from wonzhq/fiemap

FileStore: fiemap implementation using SEEK_HOLE/SEEK_DATA

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3170 from tchaikov/python-rados-refactor

python-rados: extract type-checking into a decorator

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3560 from XinzeChi/wip-pmon-all

ceph.in: add ceph ping mon.* for ping all monitor

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3879 from liewegas/wip-cache-mode-none

mon: warn if cache_mode is none and pool is a read or writ eoverlay tier

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3980 from ceph/wip-rbd-update-features

librbd: add support for updating image features

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge remote-tracking branch 'gh/hammer'

Conflicts:
src/gmock

Merge pull request #4133 from rzarzynski/wip-10645

rgw: improve metadata handling on object copy operation of Swift API.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4280 from jdurgin/wip-lttng-build

debian: remove lttng checking from rules

Reviewed-by: Sage Weil <sage@redhat.com>

debian: remove lttng checking from rules

This can be done better in a separate script, which puts these in
CEPH_EXTRA_CONFIGURE_ARGS. In particular, this lets us enable
lttng for gitbuilder builds, but not release builds.

Fixes: #11333
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit a294daff0bb69a7e0951f98c9497ccb22e91e705)

debian: remove lttng checking from rules

This can be done better in a separate script, which puts these in
CEPH_EXTRA_CONFIGURE_ARGS. In particular, this lets us enable
lttng for gitbuilder builds, but not release builds.

Fixes: #11333
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Update PendingReleaseNotes for firefly change, rgw socket path related

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

librbd: remove object maps when disabling

If the object map feature is disabled, remove any object maps
prior to disabling the feature.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: flag the snapshot object map as invalid when error occurs

Instead of flagging the HEAD image object map as invalid when an
error occurs with a snapshot object map, properly flag the snapshot
as invalid.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: add feature enable/disable support

Mutable image features can now be enabled/disabled via
the rbd CLI.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: add update_features test to test_librbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: add update_features test for pybind

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind: add update_features to rbd.py

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add update_features to librbd API

Allow mutable features to be enabled / disabled.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: add test case for cls_rbd set_features

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd_client: add set_features helper method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: add set_features class method

Allow certain mutable feature flags to be updated within
an image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: remove usage of snapshot features

The features for the image HEAD now apply to all
snapshots within the image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: update cls_rbd_client tests for get_features changes

Images no longer track per-snapshot features. snapshot_list
no longer needs to retrieve per-snapshot features.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd_client: snapshot_list no longer retrieves features

Features should now be consistent across snapshots, so
no reason to retrieve them.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: get_features on snapshots returns HEAD image features

In preparation for dynamic feature bits, it probably doesn't
make sense to have snapshots have different features enabled.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #4246 from ceph/wip-11113-hammer

librbd: snap_remove should ignore -ENOENT errors

doc/release-notes: 0.67.12 are draft notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4250 from athanatos/wip-11265

ECTransaction: write out the hinfo key on touch as well

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4251 from athanatos/wip-11284

ReplicatedPG::promote_object: do not create obc if not promoting

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4278 from nilamdyuti/wip-doc-ceph-object-gateway

doc: Corrects some content for Debian-based distros in simple gateway
configuration file.

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: Corrects some content for Debian-based distros in simple gateway
configuration file.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge pull request #4238 from ceph/wip-9580

ceph-disk: make journal setup conditional on osd backend

Reviewed-by: Travis Rhoden <trhoden@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4244 from dachary/wip-9580

tests for ceph-disk: make journal setup conditional on osd backend

tests: add ceph-disk.sh activate for memstore

Add a test for the activation of the memstore objectstore and verify
that it works without specifying a journal.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: move rados put/get tests in a function

Add the test_pool_read_write function to share the rados put / get test
that demonstrate the osd that has been created can actually be used. Use
it from the both the regular device and dmcrypt tests.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: ceph-disk.sh dmcrypt tests use test_setup_dev_and_run

Instead of duplicating the device construction / destruction logic for
dmcrypt tests, use test_setup_dev_and_run to do it. It is now able to
recover from devmapper leftover which may occur when a cryptsetup test
fails.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: merge ceph-disk.sh activate dmcrypt functions

The activate_dmcrypt_plain_dev_body and activate_dmcrypt_dev_body
functions are almost identical, merge them and differentiate with an
argument.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: add test_setup_dev_and_run to ceph-disk.sh

Move test_activate_dev to test_setup_dev_and_run and make it
run the function given in argument. test_activate_dev calls
test_setup_dev_and_run and no longer needs to implement device
allocation or destruction.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: improve ceph-disk.sh setup/teardown

Address all possible failure cases, when ceph-disk.sh completes or when
it starts with leftover from a previous interrupted run. It is assumed
that ceph-disk.sh will crash at any point.

* umount all mount points that belong to ceph-disk.sh (check the
  absolute path of the directory)
* dmsetup remove all device mapper nodes found to hold a loop device
  that ceph-disks.sh no longer uses
* losetup --detach all loop devices that ceph-disks.sh no longer uses

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: do not use -e in ceph-disk.sh

The tests explicitly return on error when relevant. Add two error cases:

* detect when the allocation of a loop device fails.
* in the outer loop, return immediately if one of the test fails

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: reduce ceph-disk.sh verbosity

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4256 from dachary/wip-dumpling-release-notes

doc/release-notes: final dumpling v0.67.12 release notes

Merge pull request #4263 from dachary/wip-man-install

rhel 6 needs python-sphinx10

Reviewed-by: Kefu Chai <kchai@redhat.com>

man: using sphinx-1.0-build if no sphinx-build

On RHEL 6 and CentOS 6 sphinx 1.0+ which is required to build man pages
is packaged as sphinx10 instead of sphinx and the binary is
sphinx-1.0-build instead of sphinx-build.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

spec.in: sphinx -b man needs sphinx > 1.0

It is used to build the man pages.

Signed-off-by: Loic Dachary <ldachary@rehdat.com>

Merge pull request #4115 from dzafman/wip-dzmanfix

doc, mon: Add ceph osd pool get <poolname> all to man page

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4262 from dachary/wip-man-install

man: add conf.py to the list of distributed files

man: add conf.py to the list of distributed files

Otherwise packages won't build as it's not included in the tarbal.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4260 from nilamdyuti/wip-doc-ceph-object-gateway

Replaces reference to mod_fastcgi with mod_proxy_fcgi in gateway docs

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #4149 from objoo/mailmap-for-hammer-2

mailmap for hammer / master 2nd batch

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mailmap: Ian Kelling affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Vartika Rai affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Alexandre Marangone affiliation

(probably a previous typo)

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Steve Capper affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Simon Guinot affiliation

(somewhat reverse 2afe1e3c4419fb0638d6ed21a83d8204675cd29e )

Signed-off-by: Yann Dupont <yann@objoo.org>
Reviewed-by: Simon Guinot <simon.guinot@sequanux.org>

mailmap: Yazen Ghannam affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Lee Revell affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

mailmap: Zhi (David) Zhang affiliation
And name normalization

Signed-off-by: Yann Dupont <yann@objoo.org>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

mailmap: Bosse Klykken affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

mailmap: Dmitry Yatsushkevich affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

mailmap: Greg Farnum affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

mailmap: Boris Ranto affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: Fix .gitignore for man directory

Caused by: 301fe6cbf2c8b04d0650fe60de65c3cdc7a8ac95

Signed-off-by: David Zafman <dzafman@redhat.com>

doc: Add ceph osd pool get <poolname> all to man page

Caused by: ea6192116cb1af4b8f4964cf7d5c69f164680205

Signed-off-by: David Zafman <dzafman@redhat.com>

mon: Add MIN_READ_RECENCY_FOR_PROMOTE to ONLY_TIER_CHOICES

Signed-off-by: David Zafman <dzafman@redhat.com>

doc: Updates the radosgw man page with some changes.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Updates simple gateway configuration file with some changes.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Updates gateway installation doc with some changes.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge branch 'wip-doc-ceph-object-gateway' of https://github.com/nilamdyuti/ceph into wip-doc-ceph-object-gateway

doc: Updates the configuration of mod_proxy_fcgi in terms of localhost
tcp and unix domain socket.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Corrects syntax highlighting in the simple gateway configuration doc.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Corrects syntax highlighting in gateway installation doc.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Updates the simple gateway configuration doc with configuration for
mod_proxy_fcgi instead of mod_fastcgi.

Fixes: #3299
Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Replaces reference to mod_fastcgi with mod_proxy_fcgi in gateway
installation doc.

Fixes: #3299
Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge pull request #4142 from ceph/wip-no-more-dot-8

doc: generate man pages in man/Makefile

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: Updates the configuration of mod_proxy_fcgi in terms of localhost
tcp and unix domain socket.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge pull request #4220 from jdurgin/wip-librados-global-flags

Revert "librados: remove the unused flags."

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: Corrects syntax highlighting in the simple gateway configuration doc.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Corrects syntax highlighting in gateway installation doc.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Updates the simple gateway configuration doc with configuration for
mod_proxy_fcgi instead of mod_fastcgi.

Fixes: #3299
Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Replaces reference to mod_fastcgi with mod_proxy_fcgi in gateway
installation doc.

Fixes: #3299
Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge pull request #4201 from tchaikov/xlist-iterator

xlist: add compare operator for iterator

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/release-notes: final dumpling v0.67.12 release notes

http://tracker.ceph.com/issues/10560 Refs: #10560

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4068 from dachary/wip-opensuse

tests: add OpenSUSE 13.2 dockerfile

Reviewed-by: Owen Synge <osynge@suse.com>

Merge pull request #4167 from dzafman/wip-11184

Fix 11184 by filtering the log

Reviewed-by: Samuel Just <sjust@redhat.com>

librbd: snap_remove should ignore -ENOENT errors

If the attempt to deregister the snapshot from the parent
image fails with -ENOENT, ignore the error as it is safe
to assume that the child is not associated with the parent.

Fixes: #11113
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cf8094942ccdba831e03e5a79451cfa5d78a135f)

librbd: get_parent_info should protect against invalid parent

get_parent_info should return -ENOENT if the image does not
have an associated parent image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 21afd0ef8e1fb81c5ace5fd403513c542e2413e3)

Merge pull request #4221 from xiaoxichen/rbd_discard_skip

Add rbd_discard_skip_zeroing flag

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4213 from xiaoxichen/wip-new-rocksdb

RocksDB configuration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4208 from dmitryya/ceph_argparse_improvement

ceph_argparse: Remove code duplication and generalize ceph_argparse_with* routines

Reviewed-by: Kefu Chai <kchai@redhat.com>

Add rbd_skip_partial_discard flag

If we want to discard a range of an object, we will zero(use fallocate
to punch a hole) the range now. In general this introduce some overhead(extra writes).

If the filesystem ontop of RBD holding lots of small files, this
behavior will bring big performance penalty.

Adding a flag that allow user to control if they want to zero the
range.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

ceph-disk: follow ceph-osd hints when setting up journal

- If the journal is required, require it.
- If the journal is not allowed, do not allow one to be specified
- If the journal is not wanted, to not set one up by default when none is
provided.

See #9580

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-osd: add --check-wants-journal, --check-allows-journal

Completes OSD side of #9580

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-osd: fix usage

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4235 from ceph/wip-rbd-unit-tests

tests: exercise all RBD features during unit testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4232 from ceph/fix-make-conf

Fix make conf

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4236 from ceph/wip-verify-mds-journal-makefile

mds: update Makefile for removed script

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mds: update Makefile for removed script

Oversight in "mds: remove verify-mds-journal.sh script"

Signed-off-by: John Spray <john.spray@redhat.com>

ceph_argparse: generalize ceph_argparse_with* routines

Remove code duplication by generalizing ceph_argparse_with{int,float,longlong}
routines - make one template function for those cases.

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>