documentation: add osd erasure-code-profile {set,rm,get,ls}

And a link from rados/operations.

http://tracker.ceph.com/issues/8514 Fixes: #8514

Reported-by: Kenneth Waegeman <Kenneth.Waegeman@UGent.be>
Signed-off-by: Loic Dachary <loic@dachary.org>

documentation: update osd pool create erasure

The properties are replaced with erasure code profiles. Remove the
reference to properties and the documentation of each erasure-code
related property.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge remote-tracking branch 'gh/next'

doc/release-notes: v0.81

Signed-off-by: Sage Weil <sage@inktank.com>

RadosClient: Avoid the DispatchQueue for OSD responses.

Prior to this change, reader thread puts the response on the DispatchQueue
and Dispatcher thread would call librados callbacks (C_aio_Ack, C_Safe_Cond).
The single DispatchQueue and Dispatcher thread (per client) would become
a bottleneck with multiple OSDs in the cluster.
Similar to fast-dispatch changes on the OSD, DispatchQueue/thread are removed
and librados internal callbacks will be called from reader thread context.

Signed-off-by: Sushma Gurram <sushma.gurram@sandisk.com>
Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

doc: Updated monitor output and added usage calc explanations.

Fixes: #4948
Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added usage for pool quotas.

Fixes: #5653
Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added more discussion of new CRUSH types.

Fixes: #7727
Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added a section for ceph df.

Fixes: #8281
Signed-off-by: John Wilkins <john.wilkins@inktank.com>

0.81

doc: Updated packages to reference firefly.

Fixes: #8466
Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #1887 from rossturk/wip-rturkfonts

doc: remove Apex Sans from generated doc HTML

Merge pull request #1885 from nereocystis/rados-bench-man

doc: Improve man page for bench

Merge pull request #1857 from ceph/wip-8265

#8265: config: make int parser accept K, M, G, T, P suffix and scale value accordingly

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1902 from dachary/wip-copyright-notices

erasure-code: Ceph distributed storage system

Reviewed-by: Sage Weil <sage@inktank.com>

erasure-code: Ceph distributed storage system

instead of Ceph - scalable distributed file system in the copyright notices

Signed-off-by: Loic Dachary <loic@dachary.org>

scripts/run-coverity: don't explicitly specify tool version

Rely on a symlink instead.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1901 from dachary/wip-erasure-code-concat

erasure-code: make decode_concat virtual

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>

Merge pull request #1900 from dachary/wip-ostream

common: s/stringstream/ostream/ in str_map

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>

erasure-code: make decode_concat virtual

So that a plugin can provide a more efficient implementation.

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1898 from dhxgit/master

Update INSTALL to mention the submodules/recursive

Reviewed-by: Loic Dachary <loic@dachary.org>

common: s/stringstream/ostream/ in str_map

There is no need to specialize more than ostream : it only makes it
impossible to use cerr or cout as a parameter to str_map.

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

Update INSTALL to mention the submodules/recursive

Signed-off-by: Dominik Hannen <cantares1+github@gmail.com>

Merge pull request #1895 from ceph/wip-sock-cmp

msg: remove comparison operators for sockaddr_storage

Reviewed-by: Loic Dachary <loic@dachary.org>

msg: remove comparison operators for sockaddr_storage

We don't need these at all, it turns out.

Signed-off-by: Sage Weil <sage@inktank.com>

qa/workunits/cephtool: test setting options using SI units

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

common/config.cc: allow integer values to be parsed as SI units

We are allowing this for all and any integer values; that is, OPT_INT,
OPT_LONGLONG, OPT_U32 and OPT_U64.

It's on the user to use appropriate units.  For instance, the user should
not use 'E(xabyte)' when setting a signed int, and use his best judgment
when setting options that, for instance, ought to receive seconds.

Fixes: 8265
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

test/strtol.cc: Test 'strict_strtosi()'

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

common/strtol.cc: strict_strtosi() converts str with SI units to uint64_t

Accepts values with a suffix (B, K, M, G, T, P, E) and returns the
appropriate byte value.

E.g., 10B = 10, while 10K = 10240.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge remote-tracking branch 'gh/next'

rgw: calc md5 and compare if user provided appropriate header

Fixes: #8436
Backport: firefly

This was broken in ddc2e1a8e39a5c6b9b224c3eebd1c0e762ca5782. The fix
resurrects and old check that was dropped.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1888 from ceph/wip-8472

msg_types.h: Don't use strncmp for comparing sockaddr_in.

Reviewed-by: Loic Dachary <loic@dachary.org>

msg_types.h: Don't use strncmp for comparing sockaddr_in.

Fixes: #8472
Signed-off-by: Dan Mick <dan.mick@inktank.com>

doc: Improve man page for rados bench

Add information about seq and rand for bench. Also mention
--no-cleanup option.

Signed-off-by: Kevin Dalley <kevin@kelphead.org>

Merge pull request #1886 from nereocystis/pools-doc

doc: update pools documentation

Reviewed-by: Loic Dachary <loic@dachary.org>

doc: update pools documentation

set=>get in "Get Pool Values"
grep for 'replicated size', not 'rep size'

Signed-off-by: Kevin Dalley <kevin@kelphead.org>

doc: Improve man page for bench

Add information about seq and rand for bench. Also mention
--no-cleanup option.

Signed-off-by: Kevin Dalley <kevin@kelphead.org>

Merge pull request #1884 from nereocystis/glossary-rados

doc: Define RADOS in glossary

doc: Define RADOS in glossary

Signed-off-by: Kevin Dalley <kevin@kelphead.org>

doc: Added osd pool default size setting example.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Moved redundant text out of quick-common.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #1882 from ceph/wip-8465

osd: fix bad is_active() assert in share_map()

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1840 from yuyuyu101/wip-8381

Fix set_alloc_hint op cause KeyValueStore crash problem

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1874 from ceph/wip-8292

better error reporting on incompatible device requirements

Reviewed-by: Dan Mick <dan.mick@inktank.com>

osd: fix bad is_active() assert in share_map()

We may be is_stopping() here if we are racing with shutdown().

Fixes: #8465
Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1880 from ailzhang/patch-1

rest-api: key missing for per "rx" and "rwx"

Backport: firefly
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #1847 from ceph/wip-8259

README, Makefile: make install-deps easier

Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #1860 from ceph/wip-filestore

osd: filestore backend improvements

Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #1864 from dachary/wip-filestore

unit tests for #1860

Merge pull request #1879 from ceph/wip-8321

fix incompatible `Counter` usage on Python 2.6

Reviewed-by: Loic Dachary <loic@dachary.org>

doc: Made additional changes s/CRUSH/Ceph for pull request #1855

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #1855 from kazhang/patch-1

doc: Update architecture.rst:Calculating PG IDs

Merge pull request #1866 from cmattson/radosgwdoc

doc: Improvements to radosgw docs

Merge pull request #1881 from hsdhillon/master

doc: Fixes broken hyperlinks

doc: alter doc CSS for the removal of Apex Sans

Signed-off-by: Ross Turk <rturk@redhat.com>

doc: removed Apex Sans font, replace with font from GFE

Signed-off-by: Ross Turk <rturk@redhat.com>

doc: Fixes broken hyperlinks

Makefile: make install-deps rule

This only works if autoconf is happy, but it's better than nothing.

Fixes: #8259
Signed-off-by: Sage Weil <sage@inktank.com>

debian: improve package dep list

This is based on the Build-Depends in debian/control.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1812 from ceph/wip-smart-df

mon: show writeable space in 'ceph df'

Reviewed-by: Samuel Just <sam.just@inktank.com>

mon: set min_size to data chunk count for erasure pools

Make the min_size value meaningful for erasure pools.

Signed-off-by: Sage Weil <sage@inktank.com>

mon: include 'max avail' in df output

Include an estimate of the maximum writeable space for each pool.  Note
that this value is a conservative estimate for that pool based on the
most-full OSD.  It is also potentially misleading as it is the available
space if *all* new data were written to this pool; one cannot (generally)
add up the available space for all pools.

Signed-off-by: Sage Weil <sage@inktank.com>

mon: right justify df values

Signed-off-by: Sage Weil <sage@inktank.com>

vstart.sh: -e to create an ec pool called 'ec'

Signed-off-by: Sage Weil <sage@inktank.com>

crush: add get_rule_weight_map

Calculate a weight map of OSDs for a given rule.

Signed-off-by: Sage Weil <sage@inktank.com>

rest-api: key missing for per "rx" and "rwx"

commit 85a1cf31e6 and db266a3fb2 introduce new per "rx" and "rwx", but key missing for per "rx" and "rwx" in permmap

Signed-off-by: Ailing Zhang <ailzhang@cisco.com>

Merge pull request #1878 from ceph/wip-map-advance

osd: fix map advance limit to handle map gaps

Reviewed-by: Samuel Just <sam.just@inktank.com>

remove unused variables, gets all tox envs passing

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

add backport of collections.Counter for python2.6

Using Raymond Hettinger's MIT backport

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

intial take on ceph-brag-client and tox. Python 2.7 passes

Signed-off-by: Alfredo Deza <alfredo@deza.pe>

Merge pull request #1853 from dachary/wip-mailmap

mailmap updates

each commit has been submitted to the author for review

mailmap: Aristoteles Neto affiliation

and name normalization

Reviewed-by: Aristoteles Neto <aristoteles.neto@webdrive.co.nz>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Chris Glass affiliation

Reviewed-by: Chris Glass <tribaal@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Christian Theune affiliation

Reviewed-by: Christian Theune <ct@gocept.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Shawn Edwards affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Ilya Dryomov name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Stuart Longland affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Walter Huf affiliation

Reviewed-by: Walter J. Huf <hufman@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Michael Riederer affiliation

and name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Stefan Eilemann affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Stephen F Taylor affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Daniel J. Hofmann affiliation

Reviewed-by: Daniel J. Hofmann <daniel@trvx.org>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Kevin Dalley affiliation

Reviewed-by: Kevin Dalley <kevin@kelphead.org>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Florent Flament affiliation

and name normalization

Reviewed-by: Florent Flament <florent.flament@cloudwatt.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1877 from ceph/wip-tier-doc

doc: Fix cache tier docs

Reviewed-by: Loic Dachary <loic@dachary.org>

doc: Fix cache tier docs

The instructions for 'osd tier add' were plain
wrong, claiming that the cache mode could
be passed as a third argument.

Signed-off-by: John Spray <john.spray@inktank.com>

Merge pull request #1856 from dachary/wip-pg-documentation

documentation: adapt PG formula for erasure coded pools

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>

Merge pull request #1867 from dachary/wip-8324

documentation: update pid file description

Reviewed-by: Florent Flament <florent.flament@cloudwatt.com>

os: FileStore::create_backend unit tests

Assert that the fs_type is mapped to a backend that claims to have the
matching name.

Signed-off-by: Loic Dachary <loic@dachary.org>

os: ObjectStore::collect_metadata unit tests

And an emacs-executable reminder about how to run a specific test.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1813 from ceph/wip-old-out

mon: restore osd weight after auto-mark out and then osd restart

Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #1811 from ceph/wip-perf-atomic

perf_counter: use atomics instead of mutex

Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1873 from dachary/wip-8440

mon: fix set cache_target_full_ratio

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1876 from atwardowski/patch-4

Fix Documentation Typo

Reviewed-by: Loic Dachary <loic@dachary.org>

Fix Documentation Typo

Signed-off-by: Adam Twardowski <adam.twardowski@gmail.com>

doc: Added primary affinity. Some minor rewrap edits.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

better error reporting on incompatible device requirements

Signed-off-by: Alfredo Deza <alfredo@deza.pe>

Merge pull request #1739 from ceph/wip-journal-tool

cephfs-journal-tool

Merge pull request #1824 from dachary/wip-documentation

erasure-code: fix URL in developer notes

Reviewed-by: koleosfuscus <koleosfuscus@yahoo.com>

mon: fix set cache_target_full_ratio

It was a noop because it was incorrectly using the variable n. Add a
test to protect against regression.

http://tracker.ceph.com/issues/8440 Fixes: #8440

Reported-by: Geoffrey Hartz <hartz.geoffrey@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>