sctp_crc32c: fix crc calculation in the NULL buffer case

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

crc32c: add a special case crc32c NULL buffer handling

Adding a special case crc32c NULL buffer handling for the intel assembly
code.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

common/buffer: pass NULL buffer instead of a zero-filled one

This is about 2x faster on my box using the sctp code.

Signed-off-by: Sage Weil <sage@inktank.com>

common/crc32c: document interface

Signed-off-by: Sage Weil <sage@inktank.com>

common/crc32c_intel_baseline: accept null data buffer

Signed-off-by: Sage Weil <sage@inktank.com>

common/sctp_crc32: accept NULL buffer pointer

If we get NULL we behave as if it were a zero-filled buffer.

Signed-off-by: Sage Weil <sage@inktank.com>

common/buffer: explicitly init zbuf to zeros

This might not be necessary?

Signed-off-by: Sage Weil <sage@inktank.com>

common/buffer: add crc caching performance test

On my old box:
 - matching cached values is a big win (free), obviously
 - the adjustment is the same speed as redoing the calculation.  this
   is probably because the data is already in L1/L2 cache; we still
   save memory bandwidth.

Signed-off-by: Sage Weil <sage@inktank.com>

common/buffer: instrument utilization of cached crcs

This is similar to the alloc tracking, but I've added a method to let
you enable it easily (in this case, from the unit test).

Signed-off-by: Sage Weil <sage@inktank.com>

common/buffer: cache crcs in buffer::raw

Signed-off-by: Samuel Just <sam.just@inktank.com>

include: add Spinlock

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #728 from ceph/wip-6059

Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

test_ceph_argparse: fix typo

Signed-off-by: Sage Weil <sage@inktank.com>

test_ceph_argparse: fix mds {add,remove}_data_pool tests

Signed-off-by: Sage Weil <sage@inktank.com>

common: fix non-daemon init

In commit 4f403c26dc0048ad63e20d20369fa86bfb31c50e we broke the general
non-daemon case.

Also make a note in the release notes.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

test/librados/cmd: fix compile error

From commit f0805cb61fccf9ffaa905b8477a131b4d46379fa.

Signed-off-by: Sage Weil <sage@inktank.com>

ReplicatedPG: remove the other backfill related flushes

Signed-off-by: Samuel Just <sam.just@inktank.com>

RadosModel: send racing read on write

Signed-off-by: Samuel Just <sam.just@inktank.com>

ReplicatedPG: block reads on an object until the write is committed

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD: ping tphandle during pg removal

Fixes: #6528
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #687 from ceph/wip-pool

mon: make 'mon {add,remove}_data_pool ...' take pool name or id

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #723 from ceph/wip-6334

common: don't do special things for unprivileged daemons

Reviewed-by: Greg Farnum <greg@inktank.com>

common: don't do special things for unprivileged daemons

We generally do want an admin socket and log, and people usually run these
as root.  This is more confusing than useful.

Fixes: #6334
Signed-off-by: Sage Weil <sage@inktank.com>

mon, osd: send leveldb log to /dev/null by default

Fixes: #5175
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #542 from ceph/wip-6122

test/librados/cmd.cc: tolerate thrashing on pg_command tests

doc: Update from user feedback. Needed to enable S3/Keystone.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

os/FileStore: fix fiemap double-free(s)

If GenericFileStoreBackend::do_fiemap returns anything other than 0, then
fiemap will not be allocated. However,
GenericFileStoreBackend::detect_features will free fiemap regardless,
triggering an assertion in tcmalloc. The attached patch will fix the bug
by only freeing fiemap when necessary.

In my case, do_fiemap returns -ENOSYS because fiemap is not implemented
on FreeBSD. However, this bug could also happen on Linux when do_fiemap
returns -ENOMEM.

Fixes: #6504
Signed-off-by: Alan Somers <asomers@gmail.com>
Reviewed-by: Sage Weil <sage@inktank.com>

ceph-dencoder: select_generated() should properly validate its input

If m_list.size() == 0, then calling select_generated(0) will result in
uninitialized data being assigned to m_object, which will cause a segfault
down the road. This patch fixes that.

To Reproduce:
$ ceph-dencoder type MWatchNotify select_test 0 encode decode
Segmentation fault (core dumped)

After the patch:
$ ./ceph-dencoder type MWatchNotify select_test 0 encode decode
error: invalid id for generated object
$ echo $?
1

Fixes: #6510
Signed-off-by: Alan Somers <asomers@gmail.com>

vstart.sh: create dev/ automatically

Otherwise you have to explicitly create it on a new checkout.

Also mkdir -p for consistency, tho it's not needed if dev/ is already
there.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #706 from ceph/wip-6059

Wip 6059

Partly-reviewed-by: Greg Farnum <greg@inktank.com>
Partly-reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #717 from dachary/master

mon: osd pool set syntax relaxed, modify unit tests

Reviewed-by: Sage Weil <sage@inktank.com>

mon: osd pool set syntax relaxed, modify unit tests

The argument is no longer restricted to a numerical value. Another
variable was added ( hashpspool ) check that it is a valid variable.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge remote-tracking branch 'gh/next'

ceph.spec.in: do not list ceph-rest-api twice; add missing cls_hello files

Reported-by: hjwsm1989@gmail.com
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #712 from ceph/wip-mon-pool-set

rework ceph osd pool set <pool> <field> <val>

Reviewed-by: Sage Weil <sage@inktank.com>

osd/osd_types: generalize pg_pool_t::get_flags_string()

This way the method doesn't have to be modified when new flags are added.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #694 from grosskur/fix-rpm-junit4

ceph.spec.in: Always depend on junit4 (fixes bug #6216)

Reviewed-by: Loic Dachary <loic@dachary.org>

mon: make 'mon {add,remove}_data_pool ...' take pool name or id

Take either a pool name or an id.  Fix up the tests accordingly.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'wip-rgw-quota'

Conflicts:
src/test/cli/radosgw-admin/help.t

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: swift update obj metadata also add generic attrs

Fixes: #6462
We were missing the generic attributes when we updated the object
metadata (operation that only exists in the swift api).

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge pull request #711 from ceph/wip-objecter-errors

These two patches fix several completion Contexts which are inappropriately setting the Op rval to zero.

Reviewed-by: Greg Farnum <greg@inktank.com>

SignalHandler: fix infinite loop on BSD systems

SignalHandler::entry shouldn't poll for POLLOUT, because it never actually
writes to the pipes in question. Polling for POLLOUT causes poll(2) to
immediately return, so the function spins the CPU and never blocks. Remove
the POLLOUT flag, unnecessarily introduced in commit
8e4a78f169eda716c7d6811cb6db5c757dc67207 when switching from select() to poll().

This fixes the problem on FreeBSD and doesn't break anything (AFAICT) on
Linux. Tested on FreeBSD 9.1 amd64 and Ubuntu Server 13.04 amd64.

Fixes: #6492
Signed-off-by: Alan Somers <asomers@gmail.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

doc: Removed underscore for consistency.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #713 from dachary/master

doc: disable cephx requires auth_supported = none

radosgw-admin: add --quota-scope param to usage

and also look at it when setting quota on a bucket.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge pull request #708 from ceph/wip-truncate2

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #715 from ceph/wip-5025

Reviewed-by: Greg Farnum <greg@inktank.com>

mds: flock: fix F_GETLK

fixes #5025
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

ReplicatedPG.h: while there cannot be a read in progress, there may be a read blocked

Signed-off-by: Samuel Just <sam.just@inktank.com>

doc: disable cephx requires auth_supported = none

    auth cluster required = none
    auth service required = none
    auth client required = none

is not enough to disable cephx if there is a

    auth_supported = cephx

line in the [global] section. It will produce the following error

    # ceph --id myself auth list
    2013-10-13 11:05:00.368638 7f812aea3700  0 librados: client.myself authentication error (1) Operation not permitted
    Error connecting to cluster: PermissionError

Changing the line to

    auth_supported = none

fixes the problem.

Signed-off-by: Loic Dachary <loic@dachary.org>

rgw: switch out param to a pointer instead of reference

following code review

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

test: update cli test for radosgw-admin

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

radosgw-admin: clarify --max-size param

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

formatter: dump_bool dumps unquoted strings

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw_quoa.{h,cc}: add copyright notice

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'wip-6444' into next

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

doc: Created new index to incorporate side-by-side deb/rpm installs.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Created installation doc for Yum priorities. Ceph-specific packages need it.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Created new QEMU install for RPMs with provision for Ceph-specific packages.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Created new QEMU install for debian.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added new libvirt install for RPM.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Created new libvirt install for debian/ubuntu.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Removed Gatway from Ceph Object Storage. Added Yum repo information.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Moved installation portion to the installation section.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Moved installation portion to the installation section.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

librados: add some clarifying comments

about async pool operation handle

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

librados: drop reference to completion in container destructor

Move the PoolAsyncCompletionImpl reference drop from
C_PoolAsync_Safe::finish() to ~C_PoolAsyncSafe(), as finish() is only
called when the async request is actually sent.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

doc: Fixed hyperlinks. Cleanup of old references to Chef.

fixes: #6493

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

osd: osd_types: Output pool's flag names during dump

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

osdc/Objecter: clean up completion handlers that set *prval=0

If the OSD operations returns 0, these are unnecessary.  If it does not,
then these are misleading.  In both cases here, the OSD sets the result to
zero, so this has no client-visible change.r

Signed-off-by: Sage Weil <sage@inktank.com>

osdc/Objecter: only make handlers set *prval if EIO

The normal Objecter handlers set *prval for each operation; we only need to
(re)set it if we have a special reason (like a decoding error).

Signed-off-by: Sage Weil <sage@inktank.com>

mon: OSDMonitor: allow (un)setting 'hashpspool' flag via 'osd pool set'

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: OSDMonitor: split 'osd pool set' out of 'prepare_command'

We should start doing this across the whole 'prepare_command' function.
Makes it prettier to the reader, and easier to add new code.

Change the command to send a string instead of an int to allow us to have
non-integer pool paramters that can be modified.  Support input json with
both int and string values so that we work with all flavors of client.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

test/filestore/run_seed_to_range.sh: fix -d syntax

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #702 from ceph/wip-4405

mds: fix infinite loop of MDCache::populate_mydir().

Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

mds: avoid leaking objects when purging file.

The filer implementation does not delete stripe objects that are truncated
to zero. When purging a deleted file, we need to purge stripe objects up to
the max size the file has ever been.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

radosgw-admin: limit user bucket-level quota

We only allow it if --quota-scope=bucket is specified. This is done in
order to avoid confusion with the future user level quota command.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

ReplicatedPG: remove unused RWTracker::ObjState::clear

Signed-off-by: Samuel Just <sam.just@inktank.com>

radosgw-admin: can set user's bucket quota

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

doc: Minor correction.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

rgw: bucket quota threshold

Add bucket quota threshold so that when we're passed that value we
reread the bucket stats before every write and not rely on cached value.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: configurable bucket quota size

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: protect against concurrent async quota updates

Leverage the cache lru_map locking for making sure that we don't end
up with more than a single concurrent async update on the same bucket
within the same update window.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: async quota update

Asynchronously update bucket stats when a period passed, but bucket
stats are within the ttl window.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

doc: Merge cleanup.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: minor clean up.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Update from wip-doc-quickstart branch.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

mds: fix infinite loop of MDCache::populate_mydir().

make MDCache::populate_mydir() only fetch bare-bone stray dirs.
After all stray dirs are populated, call MDCache::scan_stray_dir(),
it fetches incomplete stray dirs.

Fixes: #4405
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

ReplicatedPG: remove the other backfill related flushes

Signed-off-by: Samuel Just <sam.just@inktank.com>

RadosModel: send racing read on write

Signed-off-by: Samuel Just <sam.just@inktank.com>

ReplicatedPG: block reads on an object until the write is committed

Signed-off-by: Samuel Just <sam.just@inktank.com>

rgw: rearrange includes

Need to have ceph_json.h first due to some boost weirdness.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Go back to $PWD in fsstress.sh if compiling from source.

Although fsstress was being called with a static path the directory
it was writing to was in the current directory so doing a cd to the
source directory that is made in /tmp and then removing it later
caused it to be unable to write the files in a non-existent dir.

This change gets the current path first and cd's back into it after
it is done compiling fsstress.

Issue #6479.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Alfredo Deza <alfredo.deza@inktank.com>

Go back to $PWD in fsstress.sh if compiling from source.

Although fsstress was being called with a static path the directory
it was writing to was in the current directory so doing a cd to the
source directory that is made in /tmp and then removing it later
caused it to be unable to write the files in a non-existent dir.

This change gets the current path first and cd's back into it after
it is done compiling fsstress.

Issue #6479.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Alfredo Deza <alfredo.deza@inktank.com>

Merge pull request #700 from dachary/master

doc: fix openstack rbd installation command

doc: fix openstack rbd installation command

This cannot work because client.volumes.key is not a command

   ssh {your-compute-host} client.volumes.key

replace with a tee to allow for copy/paste as well as using the

   cat client.volumes.key

in the following lines.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #699 from dachary/wip-ghobject-assert

os: stronger assert on FileStore::lfn_open

Reviewed-by: David Zafman <david.zafman@inktank.com>

os: stronger assert on FileStore::lfn_open

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge branch 'next'

Merge pull request #697 from dachary/master

This lets "make check" pass, so I like it.
Reviewed-by: Greg Farnum <greg@inktank.com>