mds: ESessions more modernization for dencoder

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: ESessions now uses modern encoding

To facilitate this (since it had no versioning previously), it
gets a new encoding number and LogEvent::decode() sets a switch
if it's the old encoding.

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: ESession more modernization for dencoder

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: ESession event now uses modern encoding

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EResetJournal modernization for dencoder

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EResetJournal event now uses modern encoding

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EOpen event now uses modern encoding

Signed-off-by: Greg Farnum <greg@inktank.com>

MDS: EMetaBlob more modernization for encoder

Signed-off-by: Greg Farnum <greg@inktank.com>

MDS: EMetaBlob::dirlump more modernization for encoder

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EMetaBlob::nullbit modernization for dencoder

And set defaults in the default constructor.

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EMetaBlob::remotebit more modernization for dencoder

And set defaults for the default constructor.

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EMetaBlob::full_bit more modernization for dencoder

While we're doing so, make the frag stream operator const!

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EMetaBlob and its sub-parts use modern encoding now

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EImportStart event uses modern encoding

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EImportFinish event uses modern encoding

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EFragment event uses modern encoding

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: remove unused EString event

While we're at it, #include LogEvent.h from each of
the log events, some of which didn't include it previously!

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: EExport event uses modern encoding

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: ECommitted now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

SnapServer: use modern encoding for server_state

Signed-off-by: Greg Farnum <greg@inktank.com>

InoTable: use modern encoding for encode_state and decode_state

Signed-off-by: Greg Farnum <greg@inktank.com>

CInode: use modern encoding for encode_export/decode_import

Signed-off-by: Greg Farnum <greg@inktank.com>

CInode: use modern encoding for encode_store

Signed-off-by: Greg Farnum <greg@inktank.com>

AnchorServer: use modern encoding for server state

Signed-off-by: Greg Farnum <greg@inktank.com>

DecayCounter: use modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: old_rstat_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: Capability (and sub-structs) now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: uninline Capability encoders

Signed-off-by: Sage Weil <sage@inktank.com>

mds: build dencoder with more stuff

Add libosdc and perfglue/disabled_heap_profiler to the
dencoder, because those are required for the MDS stuff
we're about to start putting in. *sigh*

Signed-off-by: Greg Farnum <greg@inktank.com>

MDSMap: mds_info_t now uses modern encoding

We have to update the older-style MDSMap encodings to generate
the previous versions for clients as well.

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: MDSMap now uses modern encoding

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: move conditional MDSMap encoding into single encode method

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mdsmap: uninline encode/decode

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: modernize SimpleLock on-wire encoding

This is a wire protocol change.

Signed-off-by: Sage Weil <sage@inktank.com>

mds: mds_table_pending_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: rename MDSTableServer::_pending to mds_table_pending_t

And move it from MDSTableServer into mdstypes.cc, so we can use it
in ceph-dencoder more gracefully (coming up next!).

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: MDSCacheObjectInfo now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>

mds: string_snap_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: session_info_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: move durable Session bits into session_info_t

This keeps the on-disk structure explicitly separate from the in-memory
functional stuff.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: fnode_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: old_inode_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

remove common/types.cc

It no longer has a purpose; the functions it used to host are now
implemented in mds/mdstypes.cc and more properly belong there
anyway.

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: inode_t now uses modern encoding

And we move implementations and the dumper into mdstypes.cc (from
mdstypes.h and common/types.cc).

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: fold byte_range_t into client_writeable_range_t

As part of this, fold byte_range_t into it as a sub-struct
and eliminate its free-standing functions (it's too small
to live alone).

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: client_writeable_range_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: nest_info_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: rename struct default_file_layout to file_layout_policy_t

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: frag_info_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>

mds: default_file_layout now uses modern encoding

And move implementations into mdstypes.cc from CInode and common/types.
mdstypes.cc sadly lives in libcommon; as several of the types we're going
to put there are needed for Messages and similar things.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: inode_back{trace,pointer}_t now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: move SnapRealm into its own h/cc files

Signed-off-by: Sage Weil <sage@inktank.com>

client: rename client/SnapRealm files to avoid automake build conflict

We are about to move the MDS' SnapRealm into its own files, which conflicts.
The MDS is more important, so it wins the "SnapRealm" filename and we
move the client stuff to ClientSnapRealm.[h|cc].

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: SnapInfo, snaplink_t, sr_t now use modern encoding

This commit doesn't enable the dencoder integration due
to some build and compile issues, but we'll turn it
on later.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

 mds: Anchor now uses modern encoding

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

mds: add CEPH_FEATURE_MDSENC feature bit

This will cover the MDS cluster on upgrade, and determine which encoding
of the MDSMap they use for clients.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

osd: remove DecayCounter header

Neither the OSD nor the PG makes any use of this.

Signed-off-by: Greg Farnum <greg@inktank.com>

Merge pull request #39 from dachary/master

Relax Throttle::_reset_max conditions and associated unit tests

Relax Throttle::_reset_max conditions and associated unit tests

Removes a condition in Throttle::_reset_max by which the waiting queue is only
Signal()ed if the new maximum is lower than the current maximum.
There is no evidence of a use case where such a restriction would be
useful. In addition waking up a thread when the maximum increases
gives it a chance to immediately continue the suspended process
instead of waiting for the next put().

Create a new test file covering 100% of src/Throttle.{cc,h} lines of code.
The following methods are tested:

* Throttle::Throttle with and without a maximum
* Throttle::~Throttle when each pending Cond is deleted
* Throttle::take
* Throttle::get when updating the maximum ( lower or higher ),
  when going to sleep waiting for the count to lower under
  the maximum, when going to sleep because another thread is
  already asleep waiting
* Throttle::get_or_fail when there is no maximum,
  when requesting a count that is larger than the maximum, either
  when the current value is under the maximum or above the maximum.
* Throttle::wait when used to reset the maximum and wake up
  another thread asleep waiting

All asserts checking the arguments sanity are exercised ( negative argument
for Throttle::take etc. ).
Adds the LGPLv2+ licensing terms to COPYING along with the others.
Adds a Contributors section to the AUTHORS file.

Notes:
Testing asserts outputs verbose error messages that should be silenced
but it does not seem possible.

Signed-off-by: Loic Dachary <loic@dachary.org>

os: default to 'journal aio = true'

Hooray, testing indicates this is a win!

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #36 from cmello/master

libexpat dependency

Merge pull request #38 from alram/master

Fixes in ./docs/radosgw/config.rst

Edit endpoint-create in ./doc/radosgw/config.rst

internalurl and adminurl are mandatory. Typo in publicurl.

Edit rgw keystone url in ./doc/radosgw/config.rst

Won't work with the public port, it needs to be the admin port.

Note on host in ./doc/radosgw/config.rst

Some people have configured host with a FQDN or an IP
which prevents /etc/init.d/radosgw start to launch the daemon.

doc: Updated to note bobtail supports RGW + Keystone.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

cli test: add pg deep-scrub option to test

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

Add "pg deep-scrub..." missing from ceph usage output

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

rgw: fix setting of NULL to string

Fixes: #3777
s->env->get() returns char * and not string and can return NULL.
Also, remove some old unused code.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

xattr_bench.cc: remove twice included <time.h>

Cleanup includes, remove twice included <time.h>.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-filestore-dump.cc: remove twice included <iostream>

Cleanup includes, remove twice included <iostream>.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

testmsgr.cc: remove twice included <sys/stat.h>

Cleanup includes, remove twice included <sys/stat.h>.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

perf_counters.cc: remove twice included header files

Cleanup includes, remove twice included "global/global_init.h" and
"common/ceph_context.h".

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

testxattr.cc: remove twice included <iostream>

Cleanup includes, remove twice included <iostream>.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

workload_generator.cc: remove twice included "common/debug.h"

Cleanup includes, remove twice included "common/debug.h"

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_idempotent.cc: remove twice included "os/FileStore.h"

Cleanup includes, remove twice included "os/FileStore.h".

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

tp_bench.cc: remove twice included <iostream>

Cleanup includes, remove twice included <iostream>.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

small_io_bench*.cc: remove twice included <iostream>

Cleanup includes, remove twice included <iostream>.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

MDS.cc: remove twice included common/errno.h

Cleanup includes, remove twice included common/errno.h.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon: enforce reweight be between 0..1

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Luis <joao.luis@inktank.com>

qa: smalliobenchrbd workunit

Run a bunch of parallel smalliobenchrbd processes.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote-tracking branch 'gh/wip-rbd-bench'

Conflicts:
ceph.spec.in
debian/ceph-test.install
src/.gitignore

Merge branch 'wip-rpm-update3'

Patches to ceph.spec.in and addition of rbd-fuse package.

Merge branch 'master' of https://github.com/ceph/ceph

doc: Minor edits.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

rgw: key indexes are only link to user info

Instead of keeping multiple copies of the user info,
we just treat the key index as a pointer to the actual
user info (indexed by uid). This helps with two issues:
first, it scales better as we don't need to update the
entire set of keys whenever we make any change. Second,
it helps with the uid index atomicity.
One point to keep in mind is that both the links and the
info can be cached, so effect on performance is minimal.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: caleb miles <caleb.miles@inktank.com>

Build:  Add -n to files and description for rbd-fuse in ceph.sepc.in

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

Makefile:  Install new rdb-fuse.8 man page

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

build:  Add new rbd-fuse package

rdb-fuse is a new facility to map ceph rdb images to files.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

Revert "Don't install rbd-fuse binary"

This reverts commit 35e5d74e5c5786bc91df5dc10b5c08c77305df4e.

-> fix build instead

rbd-fuse: quick and dirty manpage

Signed-off-by: Dan Mick <dan.mick@inktank.com>

rbd-fuse: quick and dirty manpage

Signed-off-by: Dan Mick <dan.mick@inktank.com>

ceph-filestore-dump.cc: don't use po::value<string>()->required()

Don't use po::value<string>()->required() since this breaks build on
RHEL/CentOs6. Check if the options are set as in the code of other
ceph parts.

Move some checks up in the code to validate options as soon
as possible. Remove printing 'help' twice, and check it first.

Fix type description.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: David Zafman <david.zafman@inktank.com>

doc: Added more detail to SSD section. Links to performance blogs.

fixes: #3960

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #37 from alram/master

Add important note in doc/radosgw/config.rst

Add important note in doc/radosgw/config.rst

For CentOS and similar, FastCgiWrapper is turned on by default.
This causes Apache to spawn radosgw processes.

ceph-filestore-dump.cc: don't use po::value<string>()->required()

Don't use po::value<string>()->required() since this breaks build on
RHEL/CentOs6. Check if the options are set as in the code of other
ceph parts.

Move some checks up in the code to validate options as soon
as possible. Remove printing 'help' twice, and check it first.

Fix type description.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: David Zafman <david.zafman@inktank.com>

ceph.spec.in: fix file section for ceph-resource-agents

Create needed dirs (/usr/lib/ocf/resource.d/ceph) for the ceph-resource-agents
subpackage.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph.spec.in: extend fix for libedit-devel on special SUSE versions

Extend fix for libedit-devel on special SUSE versions, use ncurses
also on src/ocf/Makefile and src/java/Makefile

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph.spec.in: don't move libcephfs_jni files around

Don't move libcephfs_jni files around from %{_libdir} to /usr/lib/jni/
in the buildroot. They should be placed in %{_libdir} as all libs.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>