mailmap: Daniel J. Hofmann affiliation

Reviewed-by: Daniel J. Hofmann <daniel@trvx.org>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Kevin Dalley affiliation

Reviewed-by: Kevin Dalley <kevin@kelphead.org>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Florent Flament affiliation

and name normalization

Reviewed-by: Florent Flament <florent.flament@cloudwatt.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1856 from dachary/wip-pg-documentation

documentation: adapt PG formula for erasure coded pools

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>

Merge pull request #1867 from dachary/wip-8324

documentation: update pid file description

Reviewed-by: Florent Flament <florent.flament@cloudwatt.com>

Merge pull request #1813 from ceph/wip-old-out

mon: restore osd weight after auto-mark out and then osd restart

Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #1811 from ceph/wip-perf-atomic

perf_counter: use atomics instead of mutex

Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1873 from dachary/wip-8440

mon: fix set cache_target_full_ratio

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1876 from atwardowski/patch-4

Fix Documentation Typo

Reviewed-by: Loic Dachary <loic@dachary.org>

Fix Documentation Typo

Signed-off-by: Adam Twardowski <adam.twardowski@gmail.com>

doc: Added primary affinity. Some minor rewrap edits.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #1739 from ceph/wip-journal-tool

cephfs-journal-tool

Merge pull request #1824 from dachary/wip-documentation

erasure-code: fix URL in developer notes

Reviewed-by: koleosfuscus <koleosfuscus@yahoo.com>

mon: fix set cache_target_full_ratio

It was a noop because it was incorrectly using the variable n. Add a
test to protect against regression.

http://tracker.ceph.com/issues/8440 Fixes: #8440

Reported-by: Geoffrey Hartz <hartz.geoffrey@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

documentation: update pid file description

The pid file behavior changed over time and the description is misleading.

http://tracker.ceph.com/issues/8324 Fixes: #8324

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1861 from atwardowski/patch-3

Fix Doc Typo

Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #1846 from ceph/wip-8342

init-ceph: keep trying other OSDs if we fail to set CRUSH position on one OSD

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Dmitry Smirnov <onlyjob@debian.org>

Merge pull request #1858 from ceph/wip-8428

rgw: check appropriate entity permission on put_metadata

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Fix Doc Typo

Signed-off-by: Adam Twardowski <adam.twardowski@gmail.com>

rgw: check appropriate entity permission on put_metadata

Fixes: #8428
Backport: firefly

Cannot use verify_object_permission() to test acls, as the operation
here might either be on object or on bucket.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

documentation: adapt PG formula for erasure coded pools

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1854 from ceph/wip-java-build

cephfs-java: build against older jni headers

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1851 from FlorentFlament/master

documentation: update osd pool default size from 2 to 3.

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>

documentation: update osd pool default size from 2 to 3.

Fixes: #7725
Signed-off-by: Florent Flament <florent.flament@cloudwatt.com>

Merge pull request #1815 from ceph/osd-metadata

osd: include osd_objectstore in metadata reported to mon

Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #1793 from ceph/wip-multimds

Speed up several cross-MDS operations by reducing the number of two-phase commit disk accesses we have to go through.

Reviewed-by: Greg Farnum <greg@inktank.com>

cephfs-java: build against older jni headers

Older versions of the JNI interface expected non-const parameters
to their memory move functions. It's unpleasant, but won't actually
change the memory in question, to do a cast_const in order to satisfy
those older headers. (And even if it *did* modify the memory, that
would be okay given our single user.)

Signed-off-by: Greg Farnum <greg@inktank.com>

Merge pull request #1838 from ceph/wip-client-time

fs: include client timestamp in all mds requests

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #1835 from ceph/wip-8332

HashIndex: in cleanup, interpret missing dir as completed merge

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1834 from ceph/wip-8334

Wip 8334

Reviewed-by: Sage Weil <sage@inktank.com>

doc/release-notes: v0.67.9

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1848 from yuyuyu101/librbd-flush

Avoid extra check for clean object

Backport: firefly, dumpling
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1849 from ceph/wip-mon-get-version

mon: set MMonGetVersionReply tid

Reviewed-by: Sage Weil <sage@inktank.com>

Avoid extra check for clean object

We needn't to check clean object via buffer state, skip the clean object.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

mon: set MMonGetVersionReply tid

Currently we don't set MMonGetVersionReply tid even if the original
MMonGetVersion message had a non-zero tid.  This is bad for the kernel
client, which has the infrastructure in place that relies on tids to
lookup message buffers and contexts.  To kick off transitioning away
from the workaround, set MMonGetVersionReply tid to the tid of the
original MMonGetVersion message.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

Merge pull request #1723 from ctheune/patch-1

Fix formatting of header

Merge pull request #1845 from wdneto/master

doc: Update manual-deployment.rst

Merge pull request #1828 from nereocystis/faq-fix

doc: fix link to FAQ

Merge pull request #1829 from nereocystis/preflight-cleanup

doc: Clean up pre-flight documentation

Merge pull request #1833 from nereocystis/admin-node-doc

doc: mention admin-node in common documentation

Merge pull request #1837 from nereocystis/quick-ceph-deploy-cleanup

doc: quick-ceph-deploy cleanup

Merge pull request #1841 from ceph/wip-8373

Fixes for crush ruleset selection during pool creation (Wip 8373)

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1836 from ceph/wip-doc-openstack

doc: Clarifications to Ceph Block Device and OpenStack documentation.

init-ceph: if we fail to set an OSD's crush position, continue

If we fail to set the CRUSH position for one OSD, continue on to try
starting others, just as we do when we fail to start the daemon.

Fixes: #8342
Signed-off-by: Sage Weil <sage@inktank.com>

init-ceph: continue loop immediately on daemon start failure

If we fail to start the daemon, skip the post_start and other steps.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1842 from ceph/wip-buildroot

ceph.spec.in: remove BuildRoot

Merge pull request #1844 from onlyjob/hppa

Define AO_REQUIRE_CAS (fixes FTBFS on 'hppa')

Backport: firefly
Reviewed-by: Sage Weil <sage@inktank.com>

Update manual-deployment.rst

- When creating the OSD data, specify osd-uuid so that it matches when the osd is first created.
- Modify caps when adding osd auth to match what ceph-deploy does.

Define AO_REQUIRE_CAS (fixes FTBFS on 'hppa')

 to fix FTBFS due to undeclared atomic functions.

As reported

    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=748571

by John David Anglin <dave.anglin@bell.net>

~~~~
./include/atomic.h: In member function 'size_t ceph::atomic_t::inc()':
./include/atomic.h:42:36: error: 'AO_fetch_and_add1' was not declared in this scope
       return AO_fetch_and_add1(&val) + 1;
                                    ^
./include/atomic.h: In member function 'size_t ceph::atomic_t::dec()':
./include/atomic.h:45:42: error: 'AO_fetch_and_sub1_write' was not declared in this scope
       return AO_fetch_and_sub1_write(&val) - 1;
                                          ^
./include/atomic.h: In member function 'void ceph::atomic_t::add(size_t)':
./include/atomic.h:48:36: error: 'AO_fetch_and_add' was not declared in this scope
       AO_fetch_and_add(&val, add_me);
                                    ^
./include/atomic.h: In member function 'void ceph::atomic_t::sub(int)':
./include/atomic.h:52:48: error: 'AO_fetch_and_add_write' was not declared in this scope
       AO_fetch_and_add_write(&val, (AO_t)negsub);
                                                ^
./include/atomic.h: In member function 'size_t ceph::atomic_t::dec()':
./include/atomic.h:46:5: warning: control reaches end of non-void function [-Wreturn-type]
     }
     ^
make[5]: *** [cls/user/cls_user_client.o] Error 1
~~~~

Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>

mon: Fix check of ruleset id on pool update

This code was using CrushWrapper::rule_exists, which
checks for a *rule* existing, whereas the value being
set is a *ruleset*.

Signed-off-by: John Spray <john.spray@inktank.com>

mon: Fix default replicated pool ruleset choice

Specifically, in the case where the configured
default ruleset is CEPH_DEFAULT_CRUSH_REPLICATED_RULESET,
instead of assuming ruleset 0 exists, choose the lowest
numbered ruleset.

In the case where an explicit ruleset is passed to
OSDMonitor::prepare_pool_crush_ruleset, verify
that it really exists.

The idea is to eliminate cases where a pool could
exist with its crush ruleset set to something
other than a value ruleset ID.

Fixes: #8373
Signed-off-by: John Spray <john.spray@inktank.com>

mds: remove unused Mutation::now

Signed-off-by: Sage Weil <sage@inktank.com>

mds: use mds_stamp for mksnap

Use the server timestamp for the snapshot timestamp.  This could arguably
be the client timestamp, but I think snapshot creation times are a bit
more important to have accurate timestamps on, and this should not be
something that existing client apps will strongly depend on.

Signed-off-by: Sage Weil <sage@inktank.com>

mds: reset mds_stamp for readdir, rename, link

These ops to complicated work prior to starting the real operation, like
fetching missing directories, or opening remote dirfrags, creating
snaprealms.  Reset the mds timestamp after this slow work has completed.

Signed-off-by: Sage Weil <sage@inktank.com>

mds: use real timestamp for readdir leases

Signed-off-by: Sage Weil <sage@inktank.com>

mds: use client-provided time stamp for user-visible file metadata

Use the op_stamp from the MDRequest, populated by the MClientRequest when
possible, for setting timestamps on user-visible metadata (like ctime,
mtime).

Signed-off-by: Sage Weil <sage@inktank.com>

mds: do the balancer and propagation timers based on mds_stmap

Use the (more) stable time stamp for doing the balancer's temperature
tracking and associated decay.

Signed-off-by: Sage Weil <sage@inktank.com>

mds: do rstat timestamps (rctime, fragstat mtime) in terms of op stamp

Use the op (client) timestamp for the recursive stats, for santity's sake.
Note that since this is monotonically increasing, the danger here is
that we lose track of nested changes due to skewed client clocks.

Signed-off-by: Sage Weil <sage@inktank.com>

mds: make sure mds_stamp is set when we journal

This is a catch-all that we are carrying over from before.  It may not
be strictly necessary, but I'm not inclined to check the code for
Mutation users who didn't call acquire_locks().

Signed-off-by: Sage Weil <sage@inktank.com>

mds: set mds_stamp on lock acquisition

Normally acquire_locks() will set this for us.

Signed-off-by: Sage Weil <sage@inktank.com>

mds: add {mds,op}_stamp to Mutation

With getters, setters.

Signed-off-by: Sage Weil <sage@inktank.com>

ceph.spec.in: remove BuildRoot

Deprecated

Fixes: #8143
Signed-off-by: Sage Weil <sage@inktank.com>

tools/cephfs: error handling in EventOutput

Print error instead of hitting assertion if
we e.g. can't create directory.

Signed-off-by: John Spray <john.spray@inktank.com>

osdc/Journaler: fix obj count in Journaler:erase

This was off by one (too few) in the case of a
trimmedpos->write_pos range that had length
layout_period+2 and starting position one byte
before a period boundary.

Signed-off-by: John Spray <john.spray@inktank.com>

tools/cephfs-journal-tool: handle --help cleanly

Handle --help before main initialization so that
it works with or without a running mon cluster.

Signed-off-by: John Spray <john.spray@inktank.com>

tools/MDSUtility: fix crash on bad config

Was calling messenger destructor before
calling shutdown in case where config
was bad, e.g. in vstart without "-c" flag.

Signed-off-by: John Spray <john.spray@inktank.com>

tools/JournalTool: update usage()

Signed-off-by: John Spray <john.spray@inktank.com>

qa: Add smoke test for cephfs-journal-tool

Signed-off-by: John Spray <john.spray@inktank.com>

tools/JournalTool: Fix 'header set' corner case

If the write_pos was behind the other pointers, we
would flag the header as invalid.  However, that
would prevent doing a "header set" to correct it.

Signed-off-by: John Spray <john.spray@inktank.com>

mds: Create JournalPointer in MDLog::create

Otherwise it is created at next startup when
we see that it is missing: this is cleaner.

Signed-off-by: John Spray <john.spray@inktank.com>

mds: Make EMetaBlob::dirlump::_decode_bits const

This requires making the late-decoded attributes mutable.
The motivation is to allow get_paths and all the code downstream
of that to be const too.

Signed-off-by: John Spray <john.spray@inktank.com>

osdc: new style encoding for Journal::Header

Switch to the ENCODE_START, DECODE_START macros
for Journal::Header.  As well as being nice generally,
this has the important side effect of making
journal headers written since JOURNAL_FORMAT_RESILIENT
unreadable to older MDSs, as they will fail their check
on Header.magic.

Signed-off-by: John Spray <john.spray@inktank.com>

osdc: Simplify JournalStream::read

It was doing an unnecessary series of splices() after
reading header, replace with a single pass through
the data and one splice at the end.

Signed-off-by: John Spray <john.spray@inktank.com>

osdc: make JournalStream::readable const

Signed-off-by: John Spray <john.spray@inktank.com>

osdc/JournalStream: DRY envelope size calc

Replace repetitive sizeof() calculation for the
envelope overhead per log event with #defines.

Signed-off-by: John Spray <john.spray@inktank.com>

doc: Update ceph-mds manpage

Add missing --hot-standby and --journal-check
arguments.

Fixes: #8258

osdc/Journaler: use pointers for out params

Fixes a style violation in the new JournalStream class.

Signed-off-by: John Spray <john.spray@inktank.com>

tools/cephfs: Update for JournalPointer

Signed-off-by: John Spray <john.spray@inktank.com>

Fix JournalStream::read length assertion

This was checking for new-format length even
when reading from an old-format journal.

Signed-off-by: John Spray <john.spray@inktank.com>

osdc/Journaler: include stream_format in dump()

Signed-off-by: John Spray <john.spray@inktank.com>

mds: Introduce mds_journal_format config setting

This is useful for testing, so that we can create an
old-style journal and then test the version migration
by changing the config setting.

Signed-off-by: John Spray <john.spray@inktank.com>

tools: update Dumper to use JournalPointer

Signed-off-by: John Spray <john.spray@inktank.com>

mds: Generalize JournalPointer functionality

...so that we can use it places like Resetter too.  And use it
there.

Signed-off-by: John Spray <john.spray@inktank.com>

dencoder: add JournalPointer to types.h

Signed-off-by: John Spray <john.spray@inktank.com>

mds: add atomic log rewrite on format change

Two main pieces to this:
 * A new JournalPointer object that stores two journal
   inodes so that we can do a double-buffered update,
   followed by an atomic swap.
 * An extended recovery process in MDLog that dereferences
   the JournalPointer and conditionally rewrites the
   journal to accomodate format updates.

The JournalPointer indirection should also be useful for
making cephfs-journal-tool do updates more safely.

Signed-off-by: John Spray <john.spray@inktank.com>

osdc: Add Journaler.erase

This is used in subsequent commits to delete journals
that are no longer needed, such as after rewriting
a journal in a different format.

Signed-off-by: John Spray <john.spray@inktank.com>

osdc/Journaler: privatize members

Many members were unnecessarily public.

Signed-off-by: John Spray <john.spray@inktank.com>

osdc: fix redundant branch

Signed-off-by: John Spray <john.spray@inktank.com>

osdc: Clean up journalstream readable check

Fix redundant (and subtly incorrect) calculation of
the number of bytes needed.  It worked because waiting
for a few more bytes before reading the entry size
of an old-format entry was harmless.

Signed-off-by: John Spray <john.spray@inktank.com>

mds/CDir: remove redundant inode enc/dec

Use the new CInode::encode_bare/decode_bare fns
in CDir, so that we only have one implementation
of the inode encode/decode code.

Signed-off-by: John Spray <john.spray@inktank.com>

docs: Add cephfs-journal-tool

Signed-off-by: John Spray <john.spray@inktank.com>

gitignore: Add cephfs-journal-tool

Signed-off-by: John Spray <john.spray@inktank.com>

debian: add cephfs-journal-tool to ceph-mds

Signed-off-by: John Spray <john.spray@inktank.com>

rpm: add cephfs-journal-tool to RPM build

Signed-off-by: John Spray <john.spray@inktank.com>

tools: Create cephfs-journal-tool

This is for debugging/repairing CephFS journals.

Signed-off-by: John Spray <john.spray@inktank.com>

Fix formatting of header

The header was displaying the newline as a simple "n" due to quoting in the doc.

Signed-off-by: Christian Theune <ct@gocept.com>

Merge pull request #1839 from ceph/wip-8338

OSD: verify that client ops are targeted correctly in the current epoch

Reviewed-by: Samuel Just <sam.just@inktank.com>

client: set timestamp in MDS requests

Signed-off-by: Sage Weil <sage@inktank.com>

mds: include timestamp in MClientRequest

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1830 from salehe/docfix-1

rados.py: clarify documentation of recent change on write return value

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>