osd/osd_types: fix ideal lower bound object-id of pg

CEPH_NOSNAP is translated into a large unsigned 64-bit integer
which is bigger than any valid snap-id. So the ideal smallest
object of a specific PG would be a clone of snap-id 0, not a head object.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #21147 from rjfd/wip-dashboard-fix-notification-queue-test

mgr/dashboard: notification queue: fix priority tests

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: notification queue: fix priority tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20979 from ceph/wip-cd-smoke-fix

qa/tests: use ceph-deploy stable branch for single node tests

qa/tests: use stable branch for single node tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #18317 from hitoshikamei/master-rbd

rbd: eager-thick provisioning support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20868 from shun-s/fix-same-cookie

librbd: fix refuse to release lock when cookie is the same at rewatch

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20935 from trociny/wip-get-name

librbd: add API function to get image name

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21117 from smithfarm/wip-23504

tests: remove TestPGLog ASSERT_DEATH test

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #20927 into master

* refs/pull/20927/head:
doc: use actual entity address for clarity
doc: make minor grammatical rectifications

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21131 from trociny/wip-23502

qa/suites/rbd: set qemu task time_wait param

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #20983 into master

* refs/pull/20983/head:
ceph_authtool: add mode option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph_authtool: add mode option

We now have the ability to set the keyring file mode with the help of
'--mode MODE'. The mode needs to be specified in octal using the format:
0600.

Closes: http://tracker.ceph.com/issues/23513
Signed-off-by: Sébastien Han <seb@redhat.com>

qa/suites/rbd: set qemu task time_wait param

so workloads qemu_dynamic_features.sh and qemu_rebuild_object_map.sh,
which check if qemu is finished with periodicity 60 sec, have enough
time to detect this before the rbd image is removed.

Fixes: https://tracker.ceph.com/issues/23502
Signed-off-by: Mykola Golub <mgolub@suse.com>

qa/tasks/qemu: add a parameter to wait for workloads detect qemu finished

In the case when a workload needs to detect qemu finished by running a
check with a periodicity of N sec it needs to set time_wait to 2 * N
in order to avoid races on finish.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21121 from tchaikov/wip-no-relicensing

msg/async/dpdk: remove xsky copyright and LGPL copying

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #21105 from ashitakasam/wip-bug

rgw: comparison between signed and unsigned integer expressions

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21074 from trociny/wip-librados_test_stub-append

test/librados_test_stub: always create copy of buffers passed to operation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21116 from trociny/wip-23499

qa/workunits/rbd: remove sanity check in test_admin_socket.sh

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

msg/async/dpdk: remove xsky copyright and LGPL copying

it's not legitimate to relicense source code without original author's
permission. some of the source files are copied from seastar project
with minor changes. so xsky cannot claim the copyright of them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21107 from tchaikov/wip-pg-stat-cap-trimq

osd: cap snaptrimq_len at 2^32

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #21106 from tchaikov/wip-compressor-zstd

compressor: add zstd back

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21097 from ceph/wip-ifed-fix-reclaim-cap-bug

os/bluestore: fix broken cap in _balance_bluefs_freespace()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20556 from yanghonggang/master

lttng: Trace rgw data transfer, bi entry and object header update processes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

tests: remove TestPGLog ASSERT_DEATH test

Fixes: http://tracker.ceph.com/issues/23504
Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/workunits/rbd: remove sanity check in test_admin_socket.sh

Fixes: https://tracker.ceph.com/issues/23499
Signed-off-by: Mykola Golub <mgolub@suse.com>

osd: cap snaptrimq_len at 2^32

(uint64_t)(1 << 31) evaluates to 0xffffffff80000000 instead of 0x80000000,
because the converted signed number is sign-extended.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21104 from neha-ojha/wip-minimal-perf-suite

qa/suites: add minimal performance suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Varada Kari <varada.kari@gmail.com>

Merge pull request #21092 from runsisi/wip-osdmon-log

mon/OSDMonitor: logging non-active osd id when handling osd beacon

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #21091 from runsisi/wip-elector-cleanup

mon/Elector: remove unused member fields start_stamp and ack_stamp

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21085 from xiexingguo/wip-kill-p2-macros

intarith: get rid of  P2* and ROUND_UP* macros

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21082 from zhsj/fix-java9

cmake: bump target jdk to 1.7

Reviewed-by: Wido den Hollander <wido@42on.com>

Merge pull request #21062 from xiexingguo/wip-hobject-cmp

common/hobject: compare two objects' key directly

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20862 from wido/ipv6-local-address

common/ipaddr: Do not select link-local IPv6 addresses

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21081 from majianpeng/rbd-remove-unused-func

osdc: remove unused function.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

compressor: add zstd back

it's a regression introduced by 16c3552a

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: comparison between signed and unsigned integer expressions

Fixes: Change the data type of 'c' from int to size_t in
src/rgw/rgw_civetweb.cc RGWCivetWeb::read_data

Signed-off-by: ashitakasam <694240887@qq.com>

qa/suites: add minimal performance suite

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #20040 from liewegas/wip-snapmapper

osd: make snapmapper warn+clean up instead of assert

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix broken cap in _balance_bluefs_freespace()

1 << 31 is cast to signed int that results in huge value when casting to
uint64_t making min() call ineffective.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #21071 from mkoderer/vstart-fix

vstart.sh: Fix help text in vstart.sh

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20961 from smithfarm/wip-dashboard-rst

doc: mgr/dashboard.rst: mention ceph.conf and ceph mgr services

Reviewed-by: Ricardo Dias <rdias@suse.com>

osdc: remove unused function.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

vstart.sh: Fix help text

Signed-off-by: Marc Koderer <marc.koderer@sap.com>

Merge pull request #21077 from Ashid6/master

doc: typo fix

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20660 from Leeshine/wip-mon-op-tracker

mon,mgr: make  osd_metric more popular and report slow ops to mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21057 from tchaikov/wip-22114

mon: mark mgr reports as no_reply

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

doc: typo fix

Signed-off-by: Ashita Dashottar <AshitaDashottar6@gmail.com>

cmake: bump target jdk to 1.7

jdk 1.7 was released on 2011. For CentOS 6/7, both have openjdk
1.8. For Debian 8(jessie), Ubuntu 14.04(trusty) have openjdk 1.7.
These are the oldest distributions officially built for during
luminous release.

Since jdk 1.9/9, the minimal supported version is 1.6/6.
jdk 1.0/10(recently released) only supports 1.7/7.
http://openjdk.java.net/jeps/182

Fixes: http://tracker.ceph.com/issues/23458
Signed-off-by: Shengjing Zhu <zhsj@umcloud.com>

doc: mgr/dashboard.rst: adjust TCP port wording

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mon/OSDMonitor: logging non-active osd id when handling osd beacon

Signed-off-by: runsisi <runsisi@zte.com.cn>

Merge pull request #20976 from sebastian-philipp/dashboard-restcontroller-default-get

mgr/dashboard: Fix objects named `default` are inaccessible

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #20870 from rjfd/wip-dashboard-tasks

mgr/dashboard: asynchronous task support

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20884 from zouaiguo/wip_for_mgr_rbd_info

mgr/dashboard: add image id to mgr rbd info instead of block_name_prefix

Reviewed-by: Ricardo Dias <rdias@suse.com>

intarith: get rid of ROUND_UP* macros

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

intarith: kill P2* macros

These macros are originally introduced in https://github.com/ceph/ceph/pull/10128
and get replaced by https://github.com/ceph/ceph/pull/19913.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/Elector: remove unused member fields start_stamp and ack_stamp

Signed-off-by: runsisi <runsisi@zte.com.cn>

mgr: summarize mon metrics in MMgrReport and sent it

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

mon: send health_metrics to mgr

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

mgr,osd: make osd_metric more popular

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

mon: add admin command to dump historic/slow ops

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

[mgr/dashboard] add image id to rbd info instead of block_name_prefix

when create images use data_pool parameter, such as:
rbd create -p pool1 -s 1G --data-pool pool2  image1

then get image info from mgr use request http://192.7.7.36:7000/rbd_pool_data/pool1
we got the response:
[{"parent_name": "", "parent_pool": 18446744073709551615, "name": "image1", "num_objs": 256, "block_name_prefix": "rbd_data.14.f561643c986", "obj_size": 4194304, "features": 189, "features_name": "fast-diff, layering, exclusive-lock, deep-flatten, object-map, data-pool", "order": 22, "size": 1073741824}]

notice:
since block_name_prefix is deprecated,
from block_name_prefix we can not get image id(the real id is ''f561643c9869')
we need add image id to rbd info instead block_name_prefix

Python 2.7.5 (default, Jun 17 2014, 18:11:42)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rados
>>> import rbd
>>> RADOS_NAME = 'client.admin'
>>> cluster_name="ceph"
>>> cluster_handle = rados.Rados(name=RADOS_NAME, clustername=cluster_name, conffile='')
>>> cluster_handle.connect()
>>> ioctx = cluster_handle.open_ioctx("pool1")
>>> image_inst = rbd.Image(ioctx, "image1")
>>> image_inst.stat()
{'parent_name': '', 'parent_pool': 18446744073709551615L, 'num_objs': 256L, 'block_name_prefix': u'rbd_data.14.f561643c986', 'obj_size': 4194304L, 'order': 22, 'size': 1073741824L}
>>> image_inst.id()
u'f561643c9869'

after fix:

[root@tfg36 site-packages]# curl http://192.7.7.36:7000/rbd_pool_data/pool1
[{"parent_name": "", "parent_pool": 18446744073709551615, "name": "image1", "num_objs": 256, "block_name_prefix": "rbd_data.14.f561643c986", "obj_size": 4194304, "features": 189, "id": "f561643c9869", "features_name": "fast-diff, layering, exclusive-lock, deep-flatten, object-map, data-pool", "order": 22, "size": 1073741824}]

Signed-off-by: zouaiguo <zou.aiguo@zte.com.cn>

Merge pull request #21064 from Songweibin/wip-spell-error

osd: correct several spell errors in comments

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21051 from neha-ojha/wip-doc-async-recovery

doc: dev description of async recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21023 from Liuchang0812/delete-asok-directories

test: delete asok directories correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21052 from zhsj/fix-distro-detect-test

test: remove distro_version assert in distro detect test

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: dev description of async recovery

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #20798 into master

* refs/pull/20798/head:
ceph-fuse: trim ceph-fuse -V output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20828 into master

* refs/pull/20828/head:
client: cleanup ::_read return

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #20854 into master

* refs/pull/20854/head:
client: add ctime/btime to Inode dump

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge PR #20908 into master

* refs/pull/20908/head:
ceph-fuse: Fix Segmentation fault --localize-reads

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20913 into master

* refs/pull/20913/head:
client: add ceph_ll_sync_inode

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #20997 into master

* refs/pull/20997/head:
client: fix Client::_read return

Merge PR #21012 into master

* refs/pull/21012/head:
Document the new '$' suite file

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: mgr/dashboard.rst: mgr initial modules can take a list

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: mgr/dashboard.rst: mention ceph.conf and ceph mgr services

Also break down the "Configuration" section into three separate sub-sections
for clarity.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

test/librados_test_stub: always create copy of buffers passed to operation

Otherwise the buffers may be shared between object files if the same
bufferlist was passed to several operations (append and write_full were
affected).

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21070 from ceph/wip-cv-vagrant-reload

ceph-volume improve robustness when reloading vms in tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20494 from wido/influx-time-fixes

mgr/influx: Various time fixes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #21063 from guzhongyan/bs-cleanup

os/bluestore: remove unused variable.

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #21043 from wido/zabbix-osd-numpgs

mgr/zabbix: Send max, min and avg PGs of OSDs to Zabbix

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #21037 from guzhongyan/bf-format

os/bluestore: fix some code formatting

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20951 from tchaikov/wip-FTBFS-bluestore-armhf

bluestore: fix build on armhf

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21034 from rzarzynski/wip-bs-reorder-bluefs_extent_t

os/bluestore: reorder members of bluefs_extent_t for space efficiency

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20900 from ifed01/wip-ifed-fix-interals-in-repair

os/bluestore: bluestore repair should use interval_set::union_insert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20774 from rishabh-d-dave/fix-tilde-expansion

config: expand tilde for ~/.ceph/$cluster.conf

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20558 from chardan/jfw-wip-show_compression_methods-22420

osd,compressor: Expose compression algorithms via MOSDBoot.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20314 from ovh/bp-librados-nntk

librados: extend C API for so it accepts keys with NUL chars

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-volume tests create a retry script for vagrant reload

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests use a reload script in simple functional tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests use a reload script in lvm functional tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard: Fix objects named `default` are inaccessible

These two calls to `default()` are identical: `vpath` and
params` are both empty:

$ curl 'http://localhost/api/cp_path/'

and

$ curl 'http://localhost/api/cp_path/default'

But we need to dinstinguish them. To fix this, we need
to add the missing `default`

We now require that all exposed controllers have a @ApiController
decorator.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Use `@ApiController` for PerfCounters

The next commit will require that all controllers
have a `@ApiController` decorator.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: task manager unit tests implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: add task manager usage instructions to HACKING.rst

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: added tasks info to summary controller

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: async tasks controller

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Asynchronous tasks implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mon: mark mgr reports and osd_failure as no_reply

see also: #20517

Fixes: http://tracker.ceph.com/issues/22114
Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: implemented NotificationQueue listener removal

Signed-off-by: Ricardo Dias <rdias@suse.com>