common/blkdev: fix non-linux build for get_device_id

Signed-off-by: Sage Weil <sage@redhat.com>

common/blkdev: do not include vendor in fallback mode

The 'vendor' file usually contains a string like "ATA", not an actual
vendor.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: allow open-ended life expectancy

If you leave off the second part of the life expectancy interval, it means
"more than".  e.g., Setting only the first time to today + one month would
mean "more than one month".

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: use prettified durations for life expectancy

Signed-off-by: Sage Weil <sage@redhat.com>

common/ceph_time: very lame timespan_str() helper

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: "predicted failure" -> "life expectancy", and as a range

1- Change terminology from "predicted failure" to "life expectancy", which
has a more intuitive meaning.

2- Change from an expected time stamp to a range.  Any expectancy has an
inherent uncertainty, so a single time is not meaningful.  Instead, express
as a range, e.g. "4-6 weeks from now", where we record the min and max
date as well as when the prediction was made.

This is still a bit awkward, but I'm not sure what is better.  It will
always be a bit awkward to express uncertainty since in precise terms it
is probably a 90% confidence interval or something.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: detailed output for 'ceph device ...'

For JSON ls* commands, dump the full device metadata.

For plain commands, dump tables with useful information.

This is nicer for humans and jq-users, less friendly for those who want
to write a simple bash loop.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonState: skip blank device_ids

Signed-off-by: Sage Weil <sage@redhat.com>

osd: skip blank device ids

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: track and report device names too

If you're looking at a specific device by its unique id, it's also nice
to know what device name (sdb etc) it is and on which host.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ActivePyModule: expose 'devices' to python modules

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: move dump/print to DeviceState

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: implement 'device {set,rm}-predicted-failure' commands

Record predicted device failure time.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: load persistent device metadata on mgr stat

Prune DeviceState when there is no persistent metadata and no daemon
references.  Load persistent metadata from config-key.

Signed-off-by: Sage Weil <sage@redhat.com>

include/utime: add parse() method

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: tolerate racing config-key change during load_store()

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: implement 'device {ls,ls-by-daemon,ls-by-host,info}' commands

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonState: populate DeviceState structures

Track device <-> daemon (and server) mappings.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: parse out devids from daemon metadata

Look for the "device_ids" key in the metadata and parse out a device
list.

Signed-off-by: Sage Weil <sage@redhat.com>

common/blkdev: add modelines

Signed-off-by: Sage Weil <sage@redhat.com>

osd: include device_ids in metadata

Signed-off-by: Sage Weil <sage@redhat.com>

common/blkdev: add get_device_id() helper.

Signed-off-by: Yaarit Hatuka yaarithatuka@gmail.com

Merge pull request #22399 from alfredodeza/wip-rm24417

ceph-volume tests do not include admin keyring in OSD nodes

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #22401 from ricardoasmarques/wip-prettify-package-json

mgr/dashboard: Prettify package.json

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21826 from tspmelo/rxjs6

mgr/dashboard: Update RxJS to version 6

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22122 from tspmelo/wip-osd-scrub

mgr/dashboard: Add scrub action to the OSDs table

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22392 from tchaikov/wip-cmake-boost-1.67

cmake: bump up the required boost version to 1.67

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22405 from neha-ojha/wip-24365

qa/tasks/cbt.py: changes to run on bionic

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tasks/cbt.py: changes to run on bionic

Signed-off-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: Prettify package.json

Signed-off-by: Ricardo Marques <rimarques@suse.com>

ceph-volume tests do not include admin keyring in OSD nodes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22393 from wido/ceph-volume-bluestore-tool

ceph-volume lvm.activate Do not search for a MON configuration

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PR #22359 into master

* refs/pull/22359/head:
osdc/objecter: remove unused method finish_op

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: bump up the required boost version to 1.67

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Add scrub action to the OSD table

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add OSD scrub modal component

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add scrub methods to OSD controller and service

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Extract 'srt_to_bool' method

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21247 from MahatiC/wip-rbd-datapool

rbd: add data pool support to trash purge

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22389 from MahatiC/nit-cleanup

doc: cleanup rendering syntax

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22384 from ukernel/wip-mds-session-ref

mds: add reference when setting Connection::priv to existing session

Merge pull request #22346 from dzafman/wip-scrub-omap

osd: Handle omap and data digests independently

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: cleanup

cleanup format

Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>

ceph-volume lvm.activate Do not search for a MON configuration

This system might not have one nor can we be sure that we have
a client.admin keyring on the system.

Just prime the directory and have the OSD then use it's own cephx
key to communicate with the MONs.

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #22328 from tspmelo/wip-tslint-unused

mgr/dashboard: Add 'no-unused-variable' rule to tslint

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22350 from theanalyst/doc-mimic-index-fix

doc: mimic is stable!

Reviewed-by: Kefu Chai <kchai@redhat.com>

mds: fix MDSRank::send_message_client_counted()

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: add reference when setting Connection::priv to existing session

the bug was introduced by commit 72883956c26 "use intrusive_ptr for
holding Connection::priv"

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mgr/dashboard: Add 'no-unused-variable' rule to tslint

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update RxJS to version 6

I used rxjs-tslint tool to automatically update most of the necessary code.

I added a new package, rxjs-compat, to maintain the compatibility
of external plugins that are still using RxJS v5.X.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21718 from majianpeng/admin-socket-cleanup

common/admin_socket: add new api unregister_commands(AdminSocketHook …

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21913 from tspmelo/wip-cephfs-table

mgr/dashboard: Add Filesystems list component

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22196 from thinkercui/bugfix

osd: read object attrs failed at EC recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22336 from majianpeng/bluestore-handle-kvdb-error

os/bluestore: firstly delete db then delete bluefs if open db met error.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #22306 from liewegas/wip-msgr-addrvec

msg: addr -> addrvec (part 1)

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #22342 from dillaman/wip-23516

librbd: commit IO as safe when complete if writeback cache is disabled

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22348 from dillaman/wip-24169

rbd-mirror: ensure daemon can cleanly exit if pool is deleted

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21812 from thmour/thmour_ceph-volume

ceph-volume: sort and align `lvm list` output

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22367 from tchaikov/wip-cmake-rgw

cmake: rgw_common should depend on tracing headers

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #22364 from tchaikov/wip-doc

 doc: fix the links in releases/schedule.rst

Reviewed-by: Sage Weil <sage@redhat.com>

osd/OSDMap: switch to addrvec for public_addr

Signed-off-by: Sage Weil <sage@redhat.com>

mon/Session: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: remove get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: avoid get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t contains() and is_same_host()

Signed-off-by: Sage Weil <sage@redhat.com>

osd: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

client: use get_addrs()

These are legacy interfaces that should return an IP only.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t::front()

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t::probably_equals()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addrvec_t::get_ports()

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: change 'osd find' output to list all addrs

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: remove get_inst()

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: use get_addrs

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MRoute: remove unused entity_inst_t dest

This is unused in luminous and mimic, too, although the code was
never removed.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: remove dead try_send_message()

No more users!

Signed-off-by: Sage Weil <sage@redhat.com>

mon: remove dead send_command()

No users.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: remove dead 'osd scrub' implementation from mon

This is now handled by the mgr (as of luminous).

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MOSDMarkMeDown: use addrvec

Signed-off-by: Sage Weil <sage@redhat.com>

cmake: rgw_common should depend on tracing headers

see also: 4c0d3531

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22324 from cbodley/wip-rgw-cache-time-added

rgw: update ObjectCacheInfo::time_added on overwrite

Merge pull request #22327 from dillaman/wip-23512

librbd: utilize the journal disabled policy when removing images

Reviewed-by: Mykola Golub <mgolub@suse.com>

osd: retry to read object attrs at EC recovery

In EC recovery read, if the object's attrs read failed or with errors, we erase the attrs we have read and
try to read it again from left shards. This will make the primary osd get the object's attrs correct and
avoid assert.

Signed-off-by: xiaofei cui <cuixiaofei@sangfor.com>

os/bluestore: firstly delete db then delete bluefs if open db met error.

When create Rocksdb but failed, met the followong error messages:

2018-05-31 22:30:15.017 7facd3f4f240 -1 rocksdb: do_open Failed to create rocksdb column family: L
2018-05-31 22:30:15.017 7facd3f4f240 -1 bluestore(/home/jianpeng/ceph/build/dev/osd0) _open_db erroring opening db:
*** Caught signal (Segmentation fault) **
 in thread 7facd3f4f240 thread_name:ceph-osd
 ceph version 14.0.0-109-g9d317e5d23 (9d317e5d23eb78deec1a048aff5c2a31997c8345) nautilus (dev)
 1: (()+0x910ee0) [0x555727d3aee0]
 2: (()+0x12890) [0x7facc9a8d890]
 3: (pthread_mutex_lock()+0) [0x7facc9a84fa0]
 4: (BlockDevice::queue_reap_ioc(IOContext*)+0x28) [0x555727ce4fa8]
 5: (BlueFS::_close_writer(BlueFS::FileWriter*)+0x62) [0x555727ce9d52]
 6: (BlueRocksWritableFile::~BlueRocksWritableFile()+0x49) [0x555727d0e659]
 7: (rocksdb::WritableFileWriter::Close()+0x1e7) [0x555727ef0c77]
 8: (rocksdb::log::Writer::~Writer()+0x4d) [0x555727dfbb1d]
 9: (rocksdb::DBImpl::CloseHelper()+0x342) [0x555727d6d632]
 10: (rocksdb::DBImpl::~DBImpl()+0x4dd) [0x555727d7cb6d]
 11: (rocksdb::DBImpl::~DBImpl()+0x11) [0x555727d7cc11]
 12: (RocksDBStore::~RocksDBStore()+0xc1) [0x555727c8a6f1]
 13: (RocksDBStore::~RocksDBStore()+0x9) [0x555727c8aa29]
 14: (BlueStore::_open_db(bool, bool)+0x2416) [0x555727c204b6]
 15: (BlueStore::mkfs()+0x805) [0x555727c50105]
 16: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x1b0) [0x5557277ffa70]
 17: (main()+0x4222) [0x5557276ebb92]
 18: (__libc_start_main()+0xe7) [0x7facc8948b97]
 19: (_start()+0x2a) [0x5557277b0a9a]
2018-05-31 22:30:15.789 7facd3f4f240 -1 *** Caught signal (Segmentation fault) **
 in thread 7facd3f4f240 thread_name:ceph-osd

This because the error handle for this case firstly delete bluefs then
db. Like close path, it should firstly delete db than delete bluefs.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

doc: fix some redirected links reported by "build-doc linkcheck"

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix the links in releases/schedule.rst

they were broken in e37e49b2

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21864 from LenzGr/documentation-fixes

doc: fixed restful mgr module SSL configuration commands

Reveiwed-by: John Spray <john.spray@redhat.com>

Merge pull request #22295 from jan--f/cmake-fix-ceph-test-build

cmake: fix cython target in test/CMakeFile.txt

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22317 from MahatiC/wip-doc-zipkin

doc: update blkin changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22332 from TommyLike/bug/fix_typo

test/encoding: Fix typo in encoding/types.h file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22337 from tchaikov/wip-24361

osd,mon: enable level_compaction_dynamic_level_bytes for rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22285 from ricardoasmarques/wip-error-handling

mgr/dashboard: Log frontend errors + @UiController

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22335 from majianpeng/bluestore-cleanup

os/bluestore: remove useless condtion.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

osdc/objecter: remove unused method finish_op

Signed-off-by: wumingqiao <wumingqiao@inspur.com>

Merge pull request #21689 from ovh/bp-reduce-bs-bl-rebuilds

os/bluestore/BlueFS: reduce bufferlist rebuilds during WAL writes

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #21920 from cbodley/wip-async-librados

librados: use ceph::async::Completion for asio bindings

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22056 from jiangxiyxy/Branch_master

mon: change monitor compact command to run asynchronously

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd-mirror: attempt to re-acquire leader lock if watcher recovered

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ensure managed lock can shut down if stuck waiting for register

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: dump long image directory upon rbd-mirror test failure

Signed-off-by: Jason Dillaman <dillaman@redhat.com>