librbd: add C version of get_rados_pools

librbd: add rbd prefix to C version of set_snap

librbd: add set_snap and I/O to the C API

librbd: use RBDClient directly in the C interface

librados, librbd, radosacl: use new include paths

rados, rbd: move include files to their own directories

In order to match the path being used with the packages

librbd: Install the headers in the development package

Signed-off-by: Wido den Hollander <wido@widodh.nl>

librbd: aio write

librbd: convert to dout

librbd: use a pointer instead of a globally constructed RBD instance in the C API

librbd: initialize order in C API test

rbd: dynamically link with librbd

librbd: re-read image header in get_snapc, and call it after snap create and remove

This fixes a bug with image context in which the snap info
in the image header showed fewer snapshots than it should have.

librbd: remove unnecessary argument to trim_image

librbd: store snapshot names in image context and use them when listing snaps

librbd: add packages for librbd

librbd: update C interface and tests to use image contexts

rbd: fix misc problems

librbd: synchronous read and write functions

also update the rbd tool

librbd: use helper to set ImageCtx snapid

rbd: use updated interface with image_t

librbd: use snapshot metadata from ImageCtx insead of refetching it on each call

librbd: remove pool_t parameter from some api calls

librbd: remove set_assert_version on resize

librbd: make ImageCtx stuff compile and remove it from remove/copy/rename

librbd: add ImageCtx and open/close image

RBDClient functions are partially converted to use it.

librbd: implement some more functionality

librbd: update C interface and tests

librbd: return 0 on success in open_pools

librbd: remove do_ prefix from RBDClient methods

librbd: pre-declare RBDClient to avoid a bunch of casts

rbd: get it compiled with updated librbd

librbd: make it compile

librbd: modify interface a bit

rbd: update the command line rbd tool to use librbd

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librbd: add tests for the C api

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librbd: initial implementation created from rbd.cc

Includes image and snapshot create/list/delete functions.
I/O will be added later.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

dout: force reopen after conf read, fg logging set

After the configuration is read, or a program sets foreground logging,
force those changes to be applied to the dout() configuration.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mkcephfs: fix bug in multi-node setups

mkcephfs needs to create $TDIR on the other nodes in a multi-node setup
before trying to scp files into that directory.

Also, do mkdir -p on the directory where the configuration is going to
go.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

safe_io: move to C

Fixes unistd vs XOPEN_SOURCE define weirdness.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

ceph: use new safe_read wrapper

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

add safe_{read/write} wrappers

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

build_debian_image: include bind9-host

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

build_debian_image: fixes

- /dev/pts
- copy console helper scripts
- run from any directory

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

filejournal: fix return code check

pwrite() returns number of bytes, or -1.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: test for proper i_complete handling

This reproduces some buggery with rename vs I_COMPLETE on the kernel
client.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

build_debian_image: include libcrypto++ (need for runtime)

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

kernel console debug output setup scripts

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

script: add script to build uml images

Set up a tap interface in your bridge, and then

./linux ubda=image mem=512M eth0=tuntap,tap0 ip=<ip> netmask=<netmask> gw=<gw>

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

remove old script

remove ancient web content

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: make remove_pid_file fail silently if nothing to remove

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

remove ancient mds_local_osd option

This used to add a few osds to the end of the osdmap, not included in the
crush map, with the assumption that they would be colocated with the MDSs
and used as the primary MDS journal copy.  Not a useful hack anymore; you
can do this all explicitly by setting up your own crush map.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

fakemessenger: remove fakemessenger and related dead code

FakeMessenger has hopelessly drifted from the current abstract interface;
all of this code is years old.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

clock: remove tare function

This was used way back when to set a clock's relative "zero" time to
simplify stat collection when running performance benchmarks.  It has no
use in a real system.  If we need relative time for something, we can
subtract off the zero time in the output.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

config: remove (most) unused options

Left in a few that may be used later:

client_unmount_timeout
osd_shed_reads*
osd_min_raid_width
osd_recovery_forget_lost_objects

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

client: set lru midpoint

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

logger: don't calculate variance by default

Less memory, less CPU.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: check return values

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: check return values for write_pid_file

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: SimpleMessenger::start: fix unchecked return

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

msgr: remove deadcode, warnings

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

msgr: remove_pid_file: fix unchecked returns

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: FileJournal::do_write: fix error handling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'wannoying2'

Conflicts:
src/os/FileJournal.cc

filestore: handle read_op_seq errors

Fixes: #775
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: Return error code when a write is too large.

This should prevent us from crashing the OSD by trying
to pass in a write that is larger than its journal can handle.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

config: add new osd_max_write_size option.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

assert.h: Add ceph_abort

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: _check_disk_write_cache: fix error handling

Fix error handling; use popen instead of a temporary file.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cmds: check return code of chdir

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

unittest_encoding: build without -fstrict-aliasing

unittest_encoding gets "this breaks strict-aliasing" warnings when
compiled under Ubuntu. So don't crank up optimization on the unit test.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

FileStore::mount: fix error handling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: fix zeroing first chunk of FileJournal

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: FileStore::mkjournal: fix error handling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: FileJournal::create: fix error handling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cmon: check return code of chdir

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

tools/common: EINTR/unchecked return fix

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rados: fix pedantic gcc warning on writing stdout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

filejournal: fix unchecked returns, modularize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cosd: check chdir return code

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: Ager: remove Ebofs leftovers

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add do_autogen.sh

It's handy when making builds with various different warning levels.
Also saves you from having to remember --prefix, etc. all the time.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mkcephfs: clean up temp files, honor $TEMPDIR

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

filer: fix probe

ENOENT is allowed and expected at the end of a journal.  It is treated the
same as a zero size object.

Fixes 800507e891170b99cae10a9bf7c5166fd3512927.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

client/fuse_ll.c: const cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Revert "client/fuse_ll: fully initialize callback struct"

Older build environments may not have all the fuse function ptrs that
are present in newer ones. It's best to just let them all silently be
set to 0.

This reverts commit 516ac99bb8c16614757f3b61f4bf266e4531861e.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Revert "client/fuse.cc: explicity zero fuse function ptrs"

Older build environments may not have all the fuse function ptrs that
are present in newer ones. It's best to just let them all silently be
set to 0.

This reverts commit 38b50013e27e79dd43c5f514c8c24bd8b1e69e5f.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'master' of ssh://ceph.newdream.net/git/ceph

filestore: fix wipe_subvol the rest of the way

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

filestore: fix (part of) wipe_subvol

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: add nonce parsing to entity_addr_t::parse()

Reported-by: John Leach <john@brightbox.co.uk>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

os: FileStore::wipe_subvol: fix removing old dir

Older versions of btrfs which don't have BTRFS_IOC_SNAP_DESTROY need to
manually remove snapshots.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cosd.cc: better error message if osd->init fails

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth/ExportControl: fix allow_def leak

allow_def would be leaked if the first branch is not taken.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

Merge branch 'stable'

Conflicts:
src/common/signal.cc
src/os/FileStore.cc

signal: fix redefine warnings

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

ReplicatedPG: snap_trimmer fix leaked lock

Previous patch 7a02070b741d3482ff6b28827c1eb274a2134486 leaks the pg
lock.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

mds: make SETDIRLAYOUT start with ancestor policy

When we set a new policy on a directory, make the starting point the
current policy, as specified by an ancestor.  Remove any reference to
g_default_file_layout.

This aligns the code with the inode creation paths (the other time we
look at the layout policy).

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

ReplicatedPG:snap_trimmer should return if !clean or !active or !primary

The PG may become !clean or !active while in the osd snap_trim_wq.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>