librbd/test_librbd.cc: fix format qualifiers

Fix for:

[src/test/librbd/test_librbd.cc:2610]: (warning) %lld in format string
 (no. 1) requires 'long long' but the argument type is 'unsigned long long'.
[src/test/librbd/test_librbd.cc:2610]: (warning) %lld in format string
 (no. 2) requires 'long long' but the argument type is 'unsigned long long'.
[src/test/librbd/test_librbd.cc:2610]: (warning) %lld in format string
 (no. 3) requires 'long long' but the argument type is 'unsigned long long'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/test_internal.cc: fix format qualifiers to %lu

Fix for:

[src/test/librbd/test_internal.cc:1054]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1054]: (warning) %ld in format string
 (no. 2) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1063]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1073]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1114]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1121]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1129]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1196]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1196]: (warning) %ld in format string
 (no. 2) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1205]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1215]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: use compare() instead of find()

Fix for:

[src/rgw/rgw_rados.cc:3752]: (performance) Inefficient usage of
 string::find() in condition; string::compare() would be faster.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: fix format qualifiers

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/osd_types.cc: fix sscanf format qualifiers to %u

Fix for:

[src/osd/osd_types.cc:439]: (warning) %d in format string (no. 1) requires
 'int *' but the argument type is 'unsigned int *'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSDMap.cc: fix potential null pointer deref

Use the same logic as in OSDMap::pg_to_raw_up().

Fix for:

[src/osd/OSDMap.cc:2071] -> [src/osd/OSDMap.cc:2077]: (warning) Either the
 condition 'if(primary)' is redundant or there is possible null pointer
 dereference: primary.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/Watcher.cc: remove redundant if()

Fix for:

[src/librbd/Watcher.cc:131]: (style) Condition 'r>=0' is always true

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

CompatSet.h: fix format qualifier

Fix for:

[src/include/CompatSet.h:112]: (warning) %lld in format string (no. 1)
 requires 'long long' but the argument type is 'unsigned long long'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Client.cc: fix format qualifiers

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_admin.cc: remove redundant check

Fix for:

[src/rgw/rgw_admin.cc:4228]: (style) Redundant condition:
 If 'ret < 0', the comparison 'ret' is always true.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_admin.cc: do not compare int with boolean

Fix for:

[src/rgw/rgw_admin.cc:5862]: (warning) Comparison of a boolean
 expression with an integer.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

dpdk/PacketUtil.h: remove consecutive break after abort()

Fix for:

[src/msg/async/dpdk/PacketUtil.h:162]: (style) Consecutive return, break,
 continue, goto or throw statements are unnecessary.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

async/Stack.cc: fix conversion specifier

Fix for:

[src/msg/async/Stack.cc:42]: (warning) %d in format string (no. 1)
 requires 'int' but the argument type is 'unsigned int'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

AsyncConnection.cc: init some members in ctor

Fix for:

[src/msg/async/AsyncConnection.cc:122]: (warning) Member variable
 'AsyncConnection::msg_left' is not initialized in the constructor.
[src/msg/async/AsyncConnection.cc:122]: (warning) Member variable
 'AsyncConnection::cur_msg_size' is not initialized in the constructor.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

MDSMonitor.cc: use string::compare()

Fix for:

[src/mon/MDSMonitor.cc:1717]: (performance) Inefficient usage of
 string::find() in condition; string::compare() would be faster.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

kv/MemDB.h: init some members in constructor

Fix for:
[src/kv/MemDB.h:59]: (warning) Member variable 'MemDB::m_total_bytes'
 is not initialized in the constructor.
[src/kv/MemDB.h:59]: (warning) Member variable 'MemDB::m_allocated_bytes'
 is not initialized in the constructor.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd.cc: fix usage of unsigned var

Fix for:

[src/librbd/librbd.cc:3578]: (style) Checking if unsigned variable
 'data_len' is less than zero.
[src/librbd/librbd.cc:3799]: (style) Checking if unsigned variable
 'data_len' is less than zero.
[src/librbd/librbd.cc:3576]: (style) Checking if unsigned variable
 'data_len' is less than zero.
[src/librbd/librbd.cc:3796]: (style) Checking if unsigned variable
 'data_len' is less than zero.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

LZ4Compressor.h: fix usage of unsigned var

Fix for:

[src/compressor/lz4/LZ4Compressor.h:49]: (style) Checking if
 unsigned variable 'compressed_len' is less than zero.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_op.cc: fix wrong function order

Fix for:

[src/rgw/rgw_op.cc:3829]: (error) Uninitialized variable: calc_md5

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge PR #18062 into master

* refs/pull/18062/head:
client: put MetaSession map container
client: simplify release context ref mgmt

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #18381 from cbodley/wip-21832

Revert "radosgw: fix awsv4 header line sort order."

Merge pull request #18369 from baixueyu/refcount_bug

 rgw: source data in 'default.rgw.buckets.data' may not be deleted after inter-bucket copy

Merge pull request #16605 from theanalyst/tools/release-notes-pyfixes

script: ceph-release-notes: minor fixes for split_component

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18482 from liewegas/wip-luminous-note-fix

doc/release-notes: add jewel->kraken notes

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #18481 from xiexingguo/wip-choose-acting

osd/PG: miscellaneous choose acting changes and cleanups

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18473 from yuyuyu101/wip-objecter-mono

osdc/Objecter: using coarse_mono instead

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #18471 from yuyuyu101/wip-omap-stat

osd/ReplicatedPG: add omap write bytes to delta_stats

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16332 from badone/wip-warn-about-objects-with-too-many-omap-entries

osd: Warn about objects with too many omap entries

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18506 from majianpeng/osd-scrub

osd/PrimaryLogPG: add condition "is_chunky_scrub_active" to check object in chunky_scrub

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

script: ceph-release-notes: minor fixes for split_component

make labels a set as intersection would fail otherwise, also sort the
results of prefixes before creating the title

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #18517 from dillaman/wip-iscsi-docs

doc/rbd: specify additional ESX prerequisites

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18466 from zealoussnow/wip-wrong-parameter

OSD: drop unsed parameter passed to check_osdmap_features

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>

Merge pull request #18425 from ZVampirEM77/wip-options-cleanup

common/options.cc: cleanup readable literals for default sizes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17894 from chenliuzhong/change-aio-max-nr

ceph.spec.in,debian/rules: change aio-max-nr to 1048576

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18505 from badone/wip-unittest-crypto-getentropy-limit

unittest_crypto: Don't exceed limit for getentropy

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18181 from tchaikov/wip-vstart-quoted-print

vstart.sh: quote command only when necessary

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #17266 into master

* refs/pull/17266/head:
qa: update test_ceph_argparse to test fs cmds
qa: use fs rm_data_pool
qa: fix mdsmap lookup
qa: remove usage of mds dump
PendingReleaseNotes: add obsoleted mds commands
qa: remove use of obsolete mds commands
ceph_volume_client: remove use of obsolete mds cmd
doc: update on obsolete mds commands
cephfs: obsolete deprecated mds commands

Reviewed-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #17909 from tchaikov/wip-coverity

test/objectstore/test_bluefs: cleanups

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: update test_ceph_argparse to test fs cmds

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18475 from yuyuyu101/wip-fix-alignment

os/ObjectStore: fix get_data_alignment return -1 causing problem in msg header

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #18297 from dzafman/wip-21803

osd/PG: fix objects degraded higher than 100%

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #18479 from wzliao/wip-correct-scrub-cost

osd/OSD.cc: use option 'osd_scrub_cost' instead

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/rbd: specify additional ESX prerequisites

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #18109 into master

* refs/pull/18109/head:
mds: clean up non-existent data pools in MDSMap
mds: reduce variable scope

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #18366 into master

* refs/pull/18366/head:
mon: s/mds_blacklist_interval/mon_mds_blacklist_interval/
mon: tidy up mds_blacklist_interval

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18384 into master

* refs/pull/18384/head:
mds: disable early reply for dir layout and quota related requests
mds: make mksnap/setlayout wait for unsafe requests on other mds
mds: don't rdlock locks in replica object while auth mds is recovering
mds: move ScatterLock::state_flags into SimpleLock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18393 into master

* refs/pull/18393/head:
ceph_context: re-expand admin_socket metavariables in child process

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18490 from dillaman/wip-21894

rbd-mirror: strip environment/CLI overrides for remote cluster

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

qa: use fs rm_data_pool

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: fix mdsmap lookup

We were returning the file system structure and not the mdsmap.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove usage of mds dump

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18514 from ceph/wip-fix-cv-create-path

ceph-volume lvm.prepare update to use create_osd_path

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume lvm.prepare update to use create_osd_path

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #18500 from scienceluo/wip-luo-fix-doc-log-level-branch

doc: complete and update the subsystem logging level info table

Reviewed-By: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: add condition "is_chunky_scrub_active" to check object in chunky_scrub.

Avoid every time call scrubber.write_block_by_scrub. Most time scrubber
is inactive. And compare to write_block_by_scrub, is_chunky_scrub_active
is light.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

doc/release-notes: fix backticks

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18467 from yuyuyu101/wip-21883

msg/async/AsyncConnection: state will be NONE if replacing by another one

Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #18469 from yuyuyu101/wip-remove-legacy

msg/async/AsyncConnection: remove legacy feature case handle

Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #18415 from majianpeng/msg-cleanup

msg/async: batch handle msg_iovlen

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #18397 from tchaikov/wip-mon-opts

common/options: set max_background_jobs instead of max_background_compactions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18382 from ceph/wip-vstart-mgr-dashboard

cmake: add cython_rbd as a dependency to vstart target

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18480 from tchaikov/wip-doc-cleanup

doc: various cleanups

Reviewed-by: Alfredo Deza <adeza@redhat.com>

OSD: drop unsed parameter passed to check_osdmap_features

Signed-off-by: Leo Zhang <nguzcf@gmail.com>

Merge pull request #18405 from dillaman/wip-21849

librbd: default to sparse-reads for any IO operation over 64K

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

rpm,debian: use download.ceph.com/tarballs instead of ceph.com/download

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix warning reported by "build-doc linkcheck"

all the HTTP 301 (moved permanently) should be killed.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: Add a flag to ScrubMap to signal check needed

If we know none of the scrub maps have errors early in
be_large_omap_check() we can return without doing a lot of unnecessary
work.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

osd: Warn about objects with too many omap entries

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

osd: Move creation of 'master_set' to scrub_compare_maps

Moving it to scrub_compare_maps allows us to pass it to multiple
subsequently called functions.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

mon/OSDMonitor: more options in new-style

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: set max_background_jobs instead of max_background_compactions

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18409 from liu-chunmei/fix_dpdk_assert

Fix two dpdk assert happened in dpdk library.

Reviewed-by: Haomai Wang <haomai@xsky.com>

doc: complete and update the subsystem logging level info table

Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>

ceph.spec.in,debian/rules: change aio-max-nr to 1048576

when osd is more than 14 in one host,it report error that aio is not enough.
As the default aio-max-nr is 65536, one OSD needs 4096 aios and other programs may use aios.
This patch change aio-max-nr to 1048576 when install ceph-osd rpm package and debian package

Signed-off-by: chenliuzhong <liuzhong.chen@easystack.cn>

unittest_crypto: Don't exceed limit for getentropy

CryptoRandom::get_bytes calls getentropy directly if available and it
enforces a limit of 256 bytes.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #18460 from liewegas/wip-straw2-command

mon/OSDMonitor: add 'osd crush set-all-straw-buckets-to-straw2'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18428 from liewegas/wip-21878

os/bluestore/BlueFS: fix race with log flush during async log compaction

Reviewed-by: Varada Kari <varada.kari@gmail.com>

Merge pull request #18470 from yuyuyu101/wip-tracker-lock

osd/OpRequest: reduce overhead when disabling tracking

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18472 from yuyuyu101/wip-kill-asynccompressor

compressor: kill AsyncCompressor which is broken

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: Handle objects on non-actingbackfill nodes during recovery

With this fix we can see more misplaced and no degraded when
doing recovery to a new up/acting set with node(s) containing
objects that are not longer part of up/acting.

Signed-off-by: David Zafman <dzafman@redhat.com>

osd/osd_type: kill log.approx_size()

We used approx_size() because std::list::size() used to be O(n) instead of O(1),
but it is constant now in C++11.

Also counting the precise number of current log entries here is much more
meaningful.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/osd_type.h: kill pg_missing_set.have_old

which has no consumers, and removing the unused code should
always be the preferred option.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/PG: choose replicated acting by last_update

Candidate which has a newer last_update should take precedence,
since it can be technically brought back to normal sooner than others.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: Adjust misplaced calculation

When there are more backfill ndoes then misplaced nodes the
object calculation would cause misplace to drop too quickly.
Assuming that all backfill nodes are at approximate the same
place, we compute backfill per node and only adjust by
the number of misplaced nodes.

Fixes: http://tracker.ceph.com/issues/21898
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge PR #18255 into master

* refs/pull/18255/head:
os/bluestore: handle compressed extents in blob unsharing checks

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17722 from amitkumar50/cov-rgw-6

rgw: Initialization of epoch,len

Merge pull request #17305 from theanalyst/rgw-lc-tagging

RGW: support for tagging in lifecycle policies

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #18291 from ZVampirEM77/wip-copy-errormsg

rgw: copy object add response error messages

Reviewed-by: Amit Kumar amitkuma@redhat.com
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18194 from ivancich/wip-mclock-repops-bitmap

osd: replace mclock subop opclass w/ rep_op opclass; combine duplicated code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18487 from liewegas/wip-20557

/etc/sysconfig/ceph: remove jemalloc option

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rbd-mirror: strip environment/CLI overrides for remote cluster

Fixes: http://tracker.ceph.com/issues/21894
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18448 from ceph/wip-bz1499840

ceph-volume lvm bluestore support

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc/release-notes: add jewel->kraken notes

Forgot to include this!

Also add a bit more warning around the 'crush location' option change.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18403 from dillaman/wip-librbd-flush

 librbd: sync flush should re-use existing async flush logic

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

rbd: bench should flush prior to starting a test

For the new read-based bench tests, flushing prior to the start of the test
will result in the exclusive lock being acquired and the object map being
utilized.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

/etc/sysconfig/ceph: remove jemalloc option

This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: _update_calc_stats: backfill targets do not always affect degraded

Our progress on a backfill target should only count toward degraded if
the target is needed in order for us to reach the target pool size.  If
we have remapped other complete copies, we can backfill and not be degraded
at all.  And we may be some combination of the two if there are multiple
backfill targets; use the target(s) with the most objects.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18486 from liewegas/wip-20557

etc/default/ceph: remove jemalloc option

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

etc/default/ceph: remove jemalloc option

This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18408 from liewegas/wip-buffer-mempool-abi

buffer: avoid changing bufferlist ABI by removing new _mempool field

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>