ReplicatedPG: split get_obs_to_trim from snap_trimmer

Factor code for getting next collection/objects to trim into
get_obs_to_trim.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

ReplicatedPG,PG: update snap_collections on replica

Previously, snap_collections did not get updated on the replica.  As a
result, snap collections would not necessarily get trimmed when the
replica recieved and updated purged_snaps via a pg info from the
primary.  Now, the log entries in sub_op_modified are scanned to check
for any new snap collections.  The first and last snaps on a clone entry
are possibly new snap collections.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

ReplicatedPG: update pull_info and data_subset in sub_op_push

data_subset and data_needed in pull_info need to be updated with the
results of running calc_clone_subsets in sub_op_push

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

FileStore: die if we get an ENOENT on a clone

During recovery, ignoring this would allow a bug in the osd to
erroneously mark the pg clean with incorrectly recovered objects.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

librbd: fix read_iterate for the case where block ofs is not 0

Conflicts:

src/librbd.cc

Makefile.am: use libtool convenience libraries

Use libtool convenience libraries rather than explicitly forcing .a
files (static code archives) to be generated or including library .c
files directly into applications.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'next'

Merge remote branch 'origin/stable' into next

Conflicts:
src/librbd.cc

rbd: --help/-h

osd: fix merge

Signed-off-by: Sage Weil <sage@newdream.net>

librbd: sparse read cleanup, fixes

Conflicts:

src/librbd.cc

librbd: fix and cleanup a bit read_iterate

librbd: sparse read cleanup, fixes

librbd: fix and cleanup a bit read_iterate

rgw: some cleanups

qa: simple rbd import/export test

Fixes: #1211
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Objecter.cc: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

initialize g_ceph_context in common_preinit

Initialize g_ceph_context in common_preinit rather than in a global
constructor.

Add comments to all of the major initialization functions.

We still set globals in common_preinit, for the time being.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

CephContext: not copyable

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

make misc items as not copyable

These structures are all not copyable.  Make sure we get a link error if
we try.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

[dex]list: not copyable

The list items are embedded in other structures; copying the list will lead
to duplicated references and other badness.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: avoid copying Pipe* xlist

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'stable' into next

Conflicts:
src/osd/OSD.cc

mds: make avoid copying sessionmap by_state xlist

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: do misdirected request checks before pg active test

We can process+discard a misdirected or invalid request without the pg
being active.  Reorder a few other checks too.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: avoid copying snap client_caps xlist

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mon: do not copy subscription xlist

xlist is not copyable.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: clean up after snaptest-multiple-capsnaps.sh

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Dispatcher: prevent copying

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

msgr: only SO_REUSEADDR when specific port is specified

In general SO_REUSEADDR is slightly dangerous, but avoids waiting for the
timeout when restarting servers.  This is important when binding to a
specific port.

When binding to a random port, it doesn't matter.  Also, it appears that
two processes can bind() to the same port with that flag set, and then one
will fail with EADDRINUSE on listen().  That's racy when starting up
daemons that should be binding to unique/random ports.

Signed-off-by: Sage Weil <sage@newdream.net>

mon: fix log state trimming

We need to stash last_consumed in order for trimming to work.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge branch 'wip-globalfoo'

mon: add 'log ....' command

Just sticks something in the central log.

Signed-off-by: Sage Weil <sage@newdream.net>

Convert g_ceph_context to a pointer.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

gitignore: add test_libcommon_build

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

inode_load_vec_t: de-globalize

Don't call ceph_clock_now from the constructor of inode_load_vec_t.
Update the classes that contain inode_load_vec_t to reflect this.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Create test_libcommon_build

Create test_libcommon_build to test for references to g_ceph_context and
other libglobal symbols from within libcommon.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

DecayCounter: take a utime_t in ::decode

We need to initialize the last_decay time of the DecayCounter when
decoding it. This is not found in the encoded information, but instead
is set to the current time. We need to pass this in explicitly now
because of deglobalization.

This also reduces the number of calls to gettimeofday, which is good in
general.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Thread.cc: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common_init: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

filestore: fix fiemap

rbd: fix read_iterate for sparse read

rgw: use adapting window for put obj

AuthNone: encode entity name in authorizer

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: warn instead of error if cluster addr specified but not public addr

Signed-off-by: Sage Weil <sage@newdream.net>

include/atomic.h:fix build for non-atomic-ops case

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw: put data using a window

MDSMap: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

assert: de-globalize

Implement a simple registration system so that assert output can still
show up non-interleaved in daemons and utilities. Library code will just
use dout_emergency, which is simple and deadlock-free, but might be
interleaved with ongoing messages sometimes.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Messenger: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

DoutStreambuf: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

OSDMap: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: more deglobalization

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

hex.cc: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

MonClient: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ProfLogger: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

tcp.cc: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

LogClient: remove accidental g_conf references

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

msg: remove globals

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

KeyRing: derr -> lderr

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: AuthSupported: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Create global_context.cc to house g_ceph_context

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Move signal, pidfile to global

Move the global parts of signal and pidfile handling into global/

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

move common_init to global_init

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Makefile: put LIBGLOBAL LDFLAGS into LIBGLOBAL_LDA

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libclient: remove client/SyntheticClient.cc

client/SyntheticClient.cc is only of interest to csyn. The other users
of libclient don't need it.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon/MonClient: use ldout instead of dout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon/MonMap: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

SimpleMessenger: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/DecayCounter: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ThreadPool: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: de-globalize LogClient, ProfLogger, Timer

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common_init: misc globals cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: remove remaining globals

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: CephxKeyServer: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: AuthAuthorizeHandler: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: KeyRing, RotatingKeyRing: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: CryptoKey, CryptoAES: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Create ProfLoggerCollection to hold ProfLoggers

Rather than having global variables, create ProfLoggerCollection to
store the current set of ProfLoggers. This collection will be stored
inside of CephContext.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

g_clock.now -> ceph_clock_now

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/Clock: add deglobalized clock stuff

Add clock functions which don't use globals. Remove Clock::recent_now.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: de-globlize CephXAuthorizer

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: de-globalize AuthServiceHandler and pals

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: de-globalize AuthClientHandler and friends

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth: de-globalize TicketManager, TicketHandler

de-globalize CephXTicketManager and CephXTicketHandler. Also
de-globalize two static functions.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

run_cmd: ret empty string on success;err otherwise

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/atomic cleanup

* Don't allow copying of class atomic_t.
* Remove common/Spinlock.h because it's unecessary
* SimpleMessenger: use atomic var for qlen

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/atomic.h: whitespace cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

bufferlist: get rid of derr usage

bufferlist doesn't need to use dout/derr. Only one function used it, and
that was only to return an error string.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/Completion: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ObjectStore: clean up includes a little bit

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/Context.h: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/MemoryModel: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/Finisher: use ldout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

lockdep: code cleanup and de-globalization

common/Mutex.cc: this is a duplicate of the code in common/lockdep.cc.
Delete the duplicate code.

common/lockdep.cc: require the programmer to register a CephContext to
use with lockdep. If we don't have one, we don't print anything out.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Thread.cc: clean up

Don't implement everything in the header file. It slows down compilation
and usually runtime.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Create de-globalized versions of the dout macro

The de-globalized versions are available in dout.h
Versions relying on globals continue to be available in debug.h

Split md_config_obs_t off into common/config_obs.h. This will make it
possible for DoutStreambuf.h to include just that structure, without
pulling in all of config.h.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Remove cdout

cdout just duplicates the functionality of dout.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

lockdep: add lockdep_register_ceph_context,cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

signal: const cleanup, don't use derr

In signal.cc: const cleanup, don't use derr, add a comment.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>