Merge pull request #21913 from tspmelo/wip-cephfs-table

mgr/dashboard: Add Filesystems list component

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22196 from thinkercui/bugfix

osd: read object attrs failed at EC recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22336 from majianpeng/bluestore-handle-kvdb-error

os/bluestore: firstly delete db then delete bluefs if open db met error.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #22306 from liewegas/wip-msgr-addrvec

msg: addr -> addrvec (part 1)

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #22342 from dillaman/wip-23516

librbd: commit IO as safe when complete if writeback cache is disabled

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22348 from dillaman/wip-24169

rbd-mirror: ensure daemon can cleanly exit if pool is deleted

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21812 from thmour/thmour_ceph-volume

ceph-volume: sort and align `lvm list` output

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22367 from tchaikov/wip-cmake-rgw

cmake: rgw_common should depend on tracing headers

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #22364 from tchaikov/wip-doc

 doc: fix the links in releases/schedule.rst

Reviewed-by: Sage Weil <sage@redhat.com>

osd/OSDMap: switch to addrvec for public_addr

Signed-off-by: Sage Weil <sage@redhat.com>

mon/Session: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: remove get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: avoid get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t contains() and is_same_host()

Signed-off-by: Sage Weil <sage@redhat.com>

osd: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

client: use get_addrs()

These are legacy interfaces that should return an IP only.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t::front()

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t::probably_equals()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addrvec_t::get_ports()

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: change 'osd find' output to list all addrs

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: remove get_inst()

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: use get_addrs

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MRoute: remove unused entity_inst_t dest

This is unused in luminous and mimic, too, although the code was
never removed.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: remove dead try_send_message()

No more users!

Signed-off-by: Sage Weil <sage@redhat.com>

mon: remove dead send_command()

No users.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: remove dead 'osd scrub' implementation from mon

This is now handled by the mgr (as of luminous).

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MOSDMarkMeDown: use addrvec

Signed-off-by: Sage Weil <sage@redhat.com>

cmake: rgw_common should depend on tracing headers

see also: 4c0d3531

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22324 from cbodley/wip-rgw-cache-time-added

rgw: update ObjectCacheInfo::time_added on overwrite

Merge pull request #22327 from dillaman/wip-23512

librbd: utilize the journal disabled policy when removing images

Reviewed-by: Mykola Golub <mgolub@suse.com>

osd: retry to read object attrs at EC recovery

In EC recovery read, if the object's attrs read failed or with errors, we erase the attrs we have read and
try to read it again from left shards. This will make the primary osd get the object's attrs correct and
avoid assert.

Signed-off-by: xiaofei cui <cuixiaofei@sangfor.com>

os/bluestore: firstly delete db then delete bluefs if open db met error.

When create Rocksdb but failed, met the followong error messages:

2018-05-31 22:30:15.017 7facd3f4f240 -1 rocksdb: do_open Failed to create rocksdb column family: L
2018-05-31 22:30:15.017 7facd3f4f240 -1 bluestore(/home/jianpeng/ceph/build/dev/osd0) _open_db erroring opening db:
*** Caught signal (Segmentation fault) **
 in thread 7facd3f4f240 thread_name:ceph-osd
 ceph version 14.0.0-109-g9d317e5d23 (9d317e5d23eb78deec1a048aff5c2a31997c8345) nautilus (dev)
 1: (()+0x910ee0) [0x555727d3aee0]
 2: (()+0x12890) [0x7facc9a8d890]
 3: (pthread_mutex_lock()+0) [0x7facc9a84fa0]
 4: (BlockDevice::queue_reap_ioc(IOContext*)+0x28) [0x555727ce4fa8]
 5: (BlueFS::_close_writer(BlueFS::FileWriter*)+0x62) [0x555727ce9d52]
 6: (BlueRocksWritableFile::~BlueRocksWritableFile()+0x49) [0x555727d0e659]
 7: (rocksdb::WritableFileWriter::Close()+0x1e7) [0x555727ef0c77]
 8: (rocksdb::log::Writer::~Writer()+0x4d) [0x555727dfbb1d]
 9: (rocksdb::DBImpl::CloseHelper()+0x342) [0x555727d6d632]
 10: (rocksdb::DBImpl::~DBImpl()+0x4dd) [0x555727d7cb6d]
 11: (rocksdb::DBImpl::~DBImpl()+0x11) [0x555727d7cc11]
 12: (RocksDBStore::~RocksDBStore()+0xc1) [0x555727c8a6f1]
 13: (RocksDBStore::~RocksDBStore()+0x9) [0x555727c8aa29]
 14: (BlueStore::_open_db(bool, bool)+0x2416) [0x555727c204b6]
 15: (BlueStore::mkfs()+0x805) [0x555727c50105]
 16: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x1b0) [0x5557277ffa70]
 17: (main()+0x4222) [0x5557276ebb92]
 18: (__libc_start_main()+0xe7) [0x7facc8948b97]
 19: (_start()+0x2a) [0x5557277b0a9a]
2018-05-31 22:30:15.789 7facd3f4f240 -1 *** Caught signal (Segmentation fault) **
 in thread 7facd3f4f240 thread_name:ceph-osd

This because the error handle for this case firstly delete bluefs then
db. Like close path, it should firstly delete db than delete bluefs.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

doc: fix some redirected links reported by "build-doc linkcheck"

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix the links in releases/schedule.rst

they were broken in e37e49b2

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21864 from LenzGr/documentation-fixes

doc: fixed restful mgr module SSL configuration commands

Reveiwed-by: John Spray <john.spray@redhat.com>

Merge pull request #22295 from jan--f/cmake-fix-ceph-test-build

cmake: fix cython target in test/CMakeFile.txt

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22317 from MahatiC/wip-doc-zipkin

doc: update blkin changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22332 from TommyLike/bug/fix_typo

test/encoding: Fix typo in encoding/types.h file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22337 from tchaikov/wip-24361

osd,mon: enable level_compaction_dynamic_level_bytes for rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22285 from ricardoasmarques/wip-error-handling

mgr/dashboard: Log frontend errors + @UiController

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22335 from majianpeng/bluestore-cleanup

os/bluestore: remove useless condtion.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #21689 from ovh/bp-reduce-bs-bl-rebuilds

os/bluestore/BlueFS: reduce bufferlist rebuilds during WAL writes

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #21920 from cbodley/wip-async-librados

librados: use ceph::async::Completion for asio bindings

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22056 from jiangxiyxy/Branch_master

mon: change monitor compact command to run asynchronously

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd-mirror: attempt to re-acquire leader lock if watcher recovered

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ensure managed lock can shut down if stuck waiting for register

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: dump long image directory upon rbd-mirror test failure

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22319 from theanalyst/releases/mimic/13.2

doc: release/mimic: correct the changelog to the latest version

mgr/dashboard: Remove unused data from summary controller

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Rename clients component to 'cephfs-clients'

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Rename cephfs component to 'cephfs-detail'

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add Filesystems list component

Also adapted the existing cephfs components for them
to be used inside the new cephfs-list componment.

Fixes: https://tracker.ceph.com/issues/23825
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Refactor CephFS controller

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22330 from neha-ojha/wip-async-up

osd/PG: do not choose stray osds as async_recovery_targets

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22121 from mogeb/bluestore-mono-clock

os/bluestore: use the monotonic clock for perf counters latencies

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #22318 from rjfd/wip-dashboard-rest-methods

mgr/dashboard: rest: add support for query params

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #18241 from hjwsm1989/wip-21756

osd: don't check overwrite flag when handling copy-get

Reviewed-by: Sage Weil <sage@redhat.com>

mon: enable level_compaction_dynamic_level_bytes for rocksdb

Fixes: http://tracker.ceph.com/issues/24361
Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks: rbd_mirror_thrash should wait for daemon exit before restarting

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22174 from zhongyimao/cleanup

doc: fix typos

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: commit IO as safe when complete if writeback cache is disabled

We do not need to flush IO to ensure its safe if the writeback cache is
disabled when performing a journal replay. Instead, immediately mark the
IO as safe and let the journal's periodic commit throttle handle updating
the position.

Fixes: http://tracker.ceph.com/issues/23516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon/OSDMonitor: avoid many/most OSDMap::get_inst() calls

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MOSDFailure: use addrvec instead of addr

Signed-off-by: Sage Weil <sage@redhat.com>

osd: kill get_{cluster,hb_front,hb_back}_inst users

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: kill get_most_recent_inst usage

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: add addrvec accessors

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSMap: remove inst and addr accessors

Signed-off-by: Sage Weil <sage@redhat.com>

client: switch to addrvec for MetaSession

Signed-off-by: Sage Weil <sage@redhat.com>

mds: use addrvec interface

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSMap: add addrvec accessors

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: use entity_addrvec_t for MgrMap

This does several entity_addr_t -> entity_addrvec_t conversions across the
mgr code.

(Note that entity_addr_t and entity_addrvec_t already encode in an
interoperable way.)

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: include type in entity_addr_t::dump()

Signed-off-by: Sage Weil <sage@redhat.com>

msg: add Message addrvec accessors

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Messenger: *myinst -> *myname and *myaddr

Initial step moving Messenger interface away from entity_inst_t.  This
is mostly s/my_inst.addr/my_addr/ and s/my_inst.name/my_name/.

Also add Connection::get_peer_addrs() and Messenger::get_myaddrs().

Signed-off-by: Sage Weil <sage@redhat.com>

SubmittingPatches: fix the spelling error

Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>

librados: fix the spelling error

Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>

doc: update the discarded option '--delay' in command 'rbd trash mv'

I got the following error message when I used the command '#rbd trash
mv foo --delay' as a document:

rbd: unrecognised option '--delay'

so replace the discarded option '--delay' with '--expires-at' avoid
others getting into the same doubt. In addition, some spelling mistakes
were fixed.

Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>

msg/Messenger: add per-type connect_to_* and send_to_*

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Messenger: add send_to() and connect_to() addrvec-base variants

For now, just wrap the existing methods and assume all addrs are legacy
addrs.  (That will change soon.)

Signed-off-by: Sage Weil <sage@redhat.com>

test/direct_messenger: disable for now

This is failing after the addrvec changes (coming next) and it has no
users.  We can fix and resurrect it later when we need it.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: remove useless condtion.

onode must not be null in this func.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

doc: update blkin changes

Update blkin and zipkin changes to reflect current state

Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>

Merge pull request #20822 from Devp00l/table-user-config

mgr/dashboard: Store user table configurations

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

ceph-volume: sort and align output

This commit targets the `ceph-volume lvm list` command.
The output is sorted by the osd id and each device's
attributed are sorted, so the ceph operators can find
relevant information easier. The devices (block,db,..etc)
are now properly aligned.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>

test/encoding: Fix typo in encoding/types.h file

Fix typo in encoding/types.h fie

Signed-off-by: TommyLike <tommylikehu@gmail.com>

Merge pull request #22292 from tchaikov/wip-20924

osd,mon,msg: use intrusive_ptr for holding Connection::priv

Reviewed-by: Sage Weil <sage@redhat.com>

PG: do not choose stray osds as async_recovery_targets

Without this change, we might accept stray osds as async_recovery_targets,
and need to ensure that they get a chance to become part of the acting set
after recovery is over.

However, when choose_acting() is called in the Recovered state, we set
restrict_to_up_acting=true, which does not allow them to get back to the
acting set.

Therefore, similar to backfill, do not allow stray osds to become
async_recovery_targets.

Signed-off-by: Neha Ojha <nojha@redhat.com>

mds,osd,mon,msg: use intrusive_ptr for holding Connection::priv

See-also: http://tracker.ceph.com/issues/20924
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22010 from b-ranto/wip-expose-avgcount

Expose avgcount to the python modules

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reveiwed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22155 from tspmelo/fix-jest-coverage

mgr/dashboard: Fix Jest conflict with coverage files

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

librbd: utilize the journal disabled policy when removing images

Fixes: http://tracker.ceph.com/issues/23512
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/prometheus: Mention the long running avg types

Signed-off-by: Boris Ranto <branto@redhat.com>

prometheus: Expose sum/count pairs for avgs

This patch exposes the long running avgs as sum/count pairs in the
prometheus exporter module.

Signed-off-by: Boris Ranto <branto@redhat.com>

mgr_module: Deal with long running avgs properly

We have recently started exposing long running avgs with the mgr python
modules but that only covers the total sum, not the avgcount.

With this patch, the counts for long running avgs are exposed to the
python modules allowing them to deal with the new data.

Signed-off-by: Boris Ranto <branto@redhat.com>

mgr: Expose avgcount for long running avgs

This provides a basic interface to export avg count for long running
averages.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #21711 from majianpeng/osd-reduce-safetimer

osd: combine recovery/scrub/snap sleep timer into one.

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22085 from ifed01/wip-ifed-bs-tool-fix

tools/ceph-bluestore-tool: avoid mon/config access when calling global…

Reviewed-by: Sage Weil <sage@redhat.com>

rgw: update ObjectCacheInfo::time_added on overwrite

Fixes: http://tracker.ceph.com/issues/24346
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22323 from tspmelo/fix-formatter-unit-test

mgr/dashboard: Fix formatter service unit test

Reviewed-by: Ricardo Marques <rimarques@suse.com>