Variable length array of std::strings (not legal in C++) changed to std::vector<std::string>

Signed-off-by: Daniel J. Hofmann <daniel@trvx.org>

osd: check blacklisted clients in ReplicatedPG::do_op()

OSD checks if client is blacklisted only when receiving OSD request.
It's possible that OSD request's sender get blacklisted while OSD
request in in some waiting list.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
(cherry picked from commit f92677c5b2cac03f42f15c826bfb809fec27830d)

Merge pull request #1740 from ceph/wip-8155

mon: OSDMonitor: disallow nonsensical cache-mode transitions

Reviewed-by: Sage Weil <sage@inktank.com>

ceph-object-corpus: rebase onto firefly corpus

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1762 from yuyuyu101/wip-8282

Fix clone problem

Backport: firefly
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1752 from ceph/wip-da-SCA-fixes-20140501

Various fixes from SCA

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1755 from eile/master

Fix out of source builds

Reviewed-by: Sage Weil <sage@inktank.com>

Fix out of source builds

Signed-off-by: Stefan Eilemann <Stefan.Eilemann@epfl.ch>

Fix clone problem

When clone happened, the origin header also will be updated in GenericObjectMap,
so the new header wraper(StripObjectHeader) should be updated too.

Fix #8282
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

mon: OSDMonitor: disallow nonsensical cache-mode transitions

Fixes: 8155
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #1735 from ceph/wip-8113

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1698 from ceph/wip-snapmapper-debug

osd/SnapMapper: debug

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1694 from ceph/wip-throttle-snap-master

osd: throttle snap trimmming with simple delay

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1759 from dachary/wip-mailmap

DNM: mailmap updates

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Revert "ReplicatedPG: block scrub on blocked object contexts"

This reverts commit e66f2e36c06ca00c1147f922d3513f56b122a5c0.
Reviewed-by: Sage Weil <sage@inktank.com>

osd: Prevent divide by zero in agent_choose_mode()

Fixes: #8175
Backport: firefly

Signed-off-by: David Zafman <david.zafman@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>

osd, common: If agent_work() finds no objs to work on delay 5 (default) secs

Add config osd_agent_delay_time of 5 seconds
Honor delay by ignoring agent_choose_mode() calls
Add tier_delay to logger
Treat restart after delay like we were previously idle

Fixes: #8113
Backport: firefly

Signed-off-by: David Zafman <david.zafman@inktank.com>

rados.cc: fix typo in help output

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Dumper::dump_entries(): reduce scope of 'got_data'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

PG::read_info(): pass 'const coll_t coll' by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

PGBackend::be_compare_scrubmaps(): pass pgid by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd_types.h: pass eversion_t by reference to operator<<

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SimpleLock.h: remove unused private function clear_more()

Remove unused private function clear_more(), it's replaced by
try_clear_more().

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SimpleLock.h: remove twice included osd_types.h

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

linux_fiemap.h: remove twice included int_types.h

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Dumper::dump_entries(): remove not needed variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_bucket.cc: return error if update_containers_stats() fails

In case need_stats is set on rgw_read_user_buckets() and the
update_containers_stats() call fails with !-ENOENT, not only
print out a message but also return the error to the function
caller.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/PGMonitor: set tid on no-op PGStatsAck

The OSD needs to know the tid.  Both generally, and specifically because
the flush_pg_stats may be blocking on it.

Fixes: #8280
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon/OSDMonitor: share latest map with osd on dup boot message

If we get a dup boot message, share the newer maps with the osd so that
they know they are living in the past.

Fixes: #8279
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon/MonClient: remove stray _finish_hunting() calls

Callig _finish_hunting() clears out the bool hunting flag, which means we
don't retry by connection to another mon periodically.  Instead, we send
keepalives every 10s.  But, since we aren't yet in state HAVE_SESSION, we
don't check that the keepalives are getting responses.  This means that an
ill-timed connection reset (say, after we get a MonMap, but before we
finish authenticating) can drop the monc into a black hole that does not
retry.

Instead, we should *only* call _finish_hunting() when we complete the
authentication handshake.

Fixes: #8278
Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

mailmap: Florent Bautista affiliation

and name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Warren Usui name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Guang Yang name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1754 from nereocystis/hardware-to-glossary

doc: Include links from hardware-recommendations to glossary

osd/ReplicatedPG: agent_work() fix next if finished early due to start_max

Backport: firefly

Signed-off-by: David Zafman <david.zafman@inktank.com>

osd/SnapMapper: pass snaps set by const ref

Signed-off-by: Sage Weil <sage@inktank.com>

osd/SnapMapper: debug

Signed-off-by: Sage Weil <sage@inktank.com>

doc: Include links from hardware-recommendations to glossary
Included :term: in parts of hardware-recommendations so that glossary
links appear.
Signed-off-by: Kevin Dalley <kevin@kelphead.org>

Merge pull request #1749 from hufman/fix-typo-releasenotes-pyramind

Fixes a very minor typo in the release notes

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds: note MDiscoverReply encoding change in corpus

Signed-off-by: Sage Weil <sage@inktank.com>

mds: bump protocol

In commit f689e5f049736bb0a0fa437e05936f6c1b9c1bb6 we change the encoding
and semantics for MDiscoverReply.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1738 from ceph/wip-8147

osd: automatically scrub PGs with invalid stats

Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

doc/release-notes: changelog link

Signed-off-by: Sage Weil <sage@inktank.com>

doc/release-notes: final v0.67.8 notes and changelog

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1750 from nereocystis/doc-link-to-involved

doc: documenting links to get-involved

Fixes a very minor typo in the release notes

pyramind -> pyramid

doc: documenting links to get-involved
Create a link from documenting-ceph so that it is easy to find the
github repository used for ceph.

Signed-off-by: Kevin Dalley <kevin@kelphead.org>

ReplicatedPG: block scrub on blocked object contexts

Fixes: #8011
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1745 from ceph/wip-7941

rados.h,ReplicatedPG: add CEPH_OSD_FLAG_ENFORCE_SNAPC and use on flush

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1744 from ceph/wip-8161

ECBackend::continue_recovery_op: handle a source shard going down

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1746 from ceph/wip-8086

Wip 8086

Reviewed-by: Sage Weil <sage@inktank.com>

rgw_common.cc: reduce scope of 'fpos' variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_admin.cc: remove unused string variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

PGBackend.cc: remove unused to_remove variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

KeyValueStore.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_op.cc: remove unused map variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_main.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_main.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ObjectStore.h: pass const string parameter by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

CDentry.cc: fix bool comparison using relational operator

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

buffer.cc: catch exception by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SimpleLock.h: fix bool comparison using relational operator

[src/mds/SimpleLock.h:287]: (warning) Comparison of a boolean value
 using relational operator (<, >, <= or >=).

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

hitset.cc: fix format string to unsigned int

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

OSDMap.cc: prefer prefix ++operator for non-trivial iterator

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

OSDMonitor.cc: prefer prefix ++operator for non-trivial iterator

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

KeyValueStore: rename s/logger/perf_logger/

[src/os/KeyValueStore.h:368] -> [src/os/ObjectStore.h:100]: (warning) The class
 'KeyValueStore' defines member variable with name 'logger' also defined in its
 parent class 'ObjectStore'.

KeyValueStore logger is of type PerfCounters while ObjectStore logger is Logger.
Change the name in KeyValueStore to more obvious name perf_logger.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

crush/mapper.c: fix printf format for unsigned variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #1629 from ceph/wip-die-mkcephfs

remove mkcephfs (merge post-firefly?)

Reviewed-by: Alfredo Deza <alfredo.deza@inktank.com>

Merge pull request #1741 from ceph/wip-early-reply

Wip early reply

mds: avoid adding replicas of target dentry to rename witnesses

When the rename target dentry is NULL, we can use MDentryLink messages
instead of slave requests to update its replicas.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

mds: allow early reply when request's witness list is empty

Early reply should be Ok when there were slave requests, but all
of them were for acquiring locks.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

Merge pull request #1121 from ceph/wip-no-anchor

mds: remove anchor table (merge post-firefly only)

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1670 from yuyuyu101/wip-test-clone

Add clone test on store_test

Reviewed-by: Sage Weil <sage@inktank.com>

mds: include authority of the source inode in rename witnesses

rename updates source inode's ctime

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

osd: automatically scrub PGs with invalid stats

If a PG has recnetly split and has invalid stats, scrub it now, even if
it has scrubbed recently.  This helps the stats become valid again soon.

Fixes: #8147
Signed-off-by: Sage Weil <sage@inktank.com>

mon: OSDMonitor: return immediately if 'osd tier cache-mode' is a no-op

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mds: remove discover ino

Anchor table was the main user of MDCache::discover_ino(), it has
been removed. MDCache::discover_path() can replace discover_ino()
in remaining places. This patch removes discover ino related code.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

mds: remove anchor table

use backtrace instead of anchors to find/open remote inodes

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

Copy range using fiemap not entire length

Under rbd usage, if a volume has tens of thousands of objects and each 4M
object only has several KB(run fio on this volume or other cases), this volume
will be very low performance during a long time after create snapshot on
this volume. The OSD will be busy with large bandwidth read/write although
the object actually has few bytes needed to be copied.

This commit try to use fiemap if backend fs support, it can skip unnecessary
range to write. It also can be beneficial to space effective, because the copied
object will be regard as snapshot object which is access infrequently.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

rbd-fuse: fix unlink

The path contains a leading / that needs to be ignored.

Fixes: #8197
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #1701 from ceph/wip-libkrbd

libkrbd convenience library

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #1648 from ceph/wip-client-sleep

Wip client sleep

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1709 from dachary/wip-brag

brag : useability changes

Reviewed-by: Babu Shanmugam <anbu@enovance.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge branch 'wip-rbd-invalidate'

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1737 from steveftaylor/add_rbd_fuse_image_restriction

Added a new command line parameter (-i or --image=) that allows rbd-fuse...

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1699 from chrisglass/python-api-cleanup

Simple mechanical cleanups

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

librbd: add an interface to invalidate cached data

This is useful for qemu to guarantee live migration with caching is
safe, by invalidating the cache on the destination before starting it.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

librbd: check return code and error out if invalidate_cache fails

This will only happen when shrinking or rolling back an image is done
while other I/O is in flight to the same ImageCtx. This is unsafe, so
return an error before performing the resize or rollback.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Changed the -i parameter to -r in order to avoid a conflict with a generic flag interpreted by the common code.

Merge pull request #1652 from ceph/wip-5170-firefly

Wip 5170 firefly

Merge pull request #1734 from ceph/wip-8202

rgw: fix url escaping

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1736 from ceph/wip-7500-wusui

Fix s3 tests in the rgw workunit.

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

Fix s3 tests in the rgw workunit.

Make it possible to set RGW_PORT with ENV variable.

Fixes: 7500
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Added a new command line parameter (-i or --image=) that allows rbd-fuse to specify a single image to be made available within
the mount directory. The purpose of this is to allow a single RBD to be "mounted" in userspace without opening (and locking)
the other RBDs in the pool.

This is accomplished by performing a case-sensitive string compare in enumerate_images() when an image name is
specified on the command line. If no image name is specified, all images appear in the mount directory. If a non-existent
image name is specified, the mount directory is empty.

Signed-off-by: Stephen F Taylor <steveftaylor@gmail.com>

ReplicatedPG: we can get EAGAIN on missing clone flush

Signed-off-by: Samuel Just <sam.just@inktank.com>

ReplicatedPG: do not preserve op context during flush

Any information stashed in the OpContext may be obsolete by the time we
actually mark the object clean.  Instead, let the start_flush caller
clean up its OpContext and in try_flush_mark_clean we'll create a new
one.  The primary reason to keep the OpContext would have been locking,
but we can set the obc as blocking without holding an OpContext, and
that would allow trimming to happen in the mean time (which is good
since trim_object does not respect rw locks since it doesn't change user
visible state).  In try_flush_mark_clean, we requeue the fop->op along
with (but ahead of) the fop->dup_ops.

Fixes: #8068
Signed-off-by: Samuel Just <sam.just@inktank.com>

doc/release-notes: v0.67.8 notes

Signed-off-by: Sage Weil <sage@inktank.com>

rgw: fix url escaping

Fixes: #8202
This fixes the radosgw side of issue #8202. Needed to cast value
to unsigned char, otherwise it'd get padded.

Backport: dumpling

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

client: cleanup unsafe requests if MDS session is reset

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>