osd/OSDMap: don't print hb addrs

Nobody cares!

Signed-off-by: Sage Weil <sage@redhat.com>

msg/DispatchQueue: myaddr -> myaddrs

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: myaddr -> myaddrs

Signed-off-by: Sage Weil <sage@redhat.com>

msg: make set_addr_unknowns take an addrvec

Nicely collapse the wonky logic in OSD into the helper!

Signed-off-by: Sage Weil <sage@redhat.com>

mon/LogMonitor: myaddr -> myaddrs

Signed-off-by: Sage Weil <sage@redhat.com>

librados: myaddr -> myaddrs

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogClient: myaddr -> myaddrs

Signed-off-by: Sage Weil <sage@redhat.com>

client: myaddr -> myaddrs

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: is_blacklisted() for addrvecs

Signed-off-by: Sage Weil <sage@redhat.com>

osd: populate metadata with all addrs

Signed-off-by: Sage Weil <sage@redhat.com>

mds: addr -> addrvec

Update MDSMap to use entity_addrvec_t instead of entity_addr_t.  Adjust
the various mon and mgr pieces that touch it accordingly.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: get_myaddrs()

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: addrvec blacklist helper

Signed-off-by: Sage Weil <sage@redhat.com>

mds: use new pick_addresses

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-osd: use new bindv()

Rely on the new and improved pick_addresses() to produce an addrvec of
addresses we should bind to.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: bind to multiple addresses

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: (legacy) handshake using legacy addr

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: fix some debug prefixes

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: multiple listening ServerSockets

No functional change yet.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: make cluster addrs addrvecs too

This will allow a single-step upgrade from msgr1 to msgr2: as each
nautilus OSD starts up, it will bind to both msgr1 and msgr2 addrs
(assuming it starts up after the mons have been upgraded).  Peer OSDs will
use the msgr2 address if they can, or the legacy one if they can't.
Once require_osd_release >= nautilus OSDs will stop binding to msgr1
ports at all.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Messenger: my_addr -> my_addrs

Minimal changes to SimpleMessenger and AsyncMessenger to keep things
working, assuming we ony have a single addr in the addrvec.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Connection: peer_addr -> peer_addrs

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: hash<> for entity_addrvec_t

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: use new bindv() and pick_addresses()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Messenger: new bindv() that takes an addrvec

Signed-off-by: Sage Weil <sage@redhat.com>

common/pick_address: fill in ipv4/6 and msgr1/msgr2 via config options

This allows users to simply pass in CLUSTER or PUBLIC and the function
fills in the rest.  (Or, if the caller is particular, they can pass in
the other flags too.)

Signed-off-by: Sage Weil <sage@redhat.com>

common/pick_addresses: new addrvec-based pick_addresses()

Flags for everything we would want.  The intent is to return an addrvec
of addrs that we can directly bind to.

Signed-off-by: Sage Weil <sage@redhat.com>

common/pick_addresses: add filtering by ipv4 and ipv6

Signed-off-by: Sage Weil <sage@redhat.com>

CEPH_MON_PORT -> CEPH_MON_PORT_LEGACY; define CEPH_MON_PORT_IANA

Signed-off-by: Sage Weil <sage@redhat.com>

osd: kill osd_heartbeat_addr option

This serves no (good) purpose: we should always use the public addr/
network.

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: add addrvec option type

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #22165 into master

* refs/pull/22165/head:
qa: add one-off clusters to qa/cephfs/clusters
qa: allocate more space for VM disk
qa/cephfs/clusters/*: bigger cinder volumses

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22828 from ceph/wip-conf-error-message

ceph-volume: provide a nice errror message when missing ceph.conf

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22548 from ovh/bp-cls-otp-build

cls: build cls_otp only WITH_RADOSGW

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22764 from ceph/wip-rm24504

ceph-volume: refuse to zap mapper devices

Reviewed-by: Alfredo Deza

ceph-volume: always ignore a missing ceph conf in main.py

Now that we have a nice error message when a ceph.conf is missing
and we try to use values from it, maintaining a list of commands that
don't need ceph.conf isn't as helpful. We had actually missed 'simple
trigger' when we first implemented this causing all our luminous tests
for simple to fail when we backported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #22808 from jcsp/wip-mgr-doc-initial-modules

doc/dashboard: don't advise mgr_initial_modules

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

doc/dashboard: don't advise mgr_initial_modules

This setting tends to confuse people, as it's only respected
on the very first startup of the cluster.  Instead, mention
it (with appropriate caveats) on the general mgr admin
page.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #22802 from bradfitz/its

doc: fix some it's -> its typos

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: fix some it's -> its typos

Signed-off-by: Brad Fitzpatrick <brad@danga.com>

Merge pull request #22761 from fullerdj/wip-djf-24686

osd/filestore: Change default filestore_merge_threshold to -10

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph-volume: show a nice error message when ceph.conf is not loaded

If the ceph configuration file is not loaded correctly and then values
from it are used then an undescript error message is shown, e.g.

AttributeError: 'property' object has no attribute 'get'

With this change that same error condition shows the message:

RuntimeError: No valid ceph configuration file was loaded.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #21772 from oritwas/wip-rgw-test-bi-list

rgw: test bi list

Merge pull request #22769 from idryomov/wip-rbdmap-remote-fs-pre

systemd/rbdmap.service: order us before remote-fs-pre.target

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #22760 from s0nea/wip-dashboard-config-option-flags

mgr/dashboard: add supported flag information to config options documentation

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22772 from joscollin/wip-doc-dashboard-ref

doc: s/doc/ref for dashboard urls

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22776 from tchaikov/wip-thread-setaffinity

crimson/thread: pin thread pool to given CPU

Reviewed-by: Casey Bodley <cbodley@redhat.com>

crimson/thread: pin thread pool to given CPU

to take the full advantage of seastar reactor, we need to confine the
alien threads to given CPU core(s). in current setting, it's assumed
that alien threads do not perform CPU-bound tasks, so a single core
would suffice. we can always change the interface to pass a cpu_set_t
or a std::bitset if one CPU core is not enough.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #22111 into master

* refs/pull/22111/head:
cephfs-journal-tool: check fsid when import purge queue

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22718 into master

* refs/pull/22718/head:
qa/workunits/suites/ffsb.sh: fix strncpy bug in ffsb

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add one-off clusters to qa/cephfs/clusters

To consistently handle openstack configuration.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: allocate more space for VM disk

Otherwise daemons may run out of space due to logging.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa/cephfs/clusters/*: bigger cinder volumses

Fixes: http://tracker.ceph.com/issues/24238
Signed-off-by: Sage Weil <sage@redhat.com>

osd/filestore: Change default filestore_merge_threshold to -1

Performance evaluations of medium to large size Ceph clusters have
demonstrated negligible performance impact from unnecessarily deep
directory hierarchies but significant performance impact from filestore
split and merge activity. Disable merges by default.

Fixes: http://tracker.ceph.com/issues/24686
Signed-off-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #22736 from tchaikov/wip-seastar-lockless-shared-lru

common/shared_cache: add lockless SharedLRU

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: s/doc/ref for dashboard urls

Replaced :doc: with :ref: as per https://github.com/ceph/ceph/pull/22750#discussion_r198895040.

Signed-off-by: Jos Collin <jcollin@redhat.com>

systemd/rbdmap.service: order us before remote-fs-pre.target

If "/usr/bin/rbdmap unmap-all" notices a file system mounted on top of
an rbd device, it will call umount, interfering with systemd shutdown
logic.  Make sure we aren't invoked until all _netdev mounts are dealt
with by systemd.

Fixes: http://tracker.ceph.com/issues/24713
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #22771 from tchaikov/wip-test-librados-ns

test/librados: fix LibRadosList.ListObjectsNS

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

lvm zap: refuse to zap mapper devices

Fixes: http://tracker.ceph.com/issues/24504
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

adds the util.disk.is_mapper_device method

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #22086 from ifed01/wip-ifed-new-bluefs-pcounter

os/bluestore: introduce new BlueFS perf counter to track the amount of

Reviewed-by: Sage Weil <sage@redhat.com>

test/librados: fix LibRadosList.ListObjectsNS

rados_ioctx_get_namespace() returns the length of namespace string on
success, not 0.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21634 from smithfarm/wip-bci-complain-dupe

scripts: backport-create-issue: complain about duplicates and support mimic

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: add flag info to config options documentation page

Add information about the supported flags (introduced by
https://github.com/ceph/ceph/pull/22595) to the config options
documentation page.

Fixes: https://tracker.ceph.com/issues/24605
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #22756 from wjwithagen/wjw-fix-missing-vector

mgr: `#include <vector>` for clang

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22752 from tchaikov/wip-freebsd-w/o-rdma

cmake,do_freebsd.sh: disable rdma features

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Haodong Tang <haodong.tang@intel.com>

Merge pull request #22743 from rzarzynski/wip-common-drop-static_assert.h

common: drop static_assert.h as it looks unused.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22720 from rzarzynski/wip-build-static-ceph-common-in-osd

cmake: link ceph-osd with common statically

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22727 from dzafman/wip-21664

qa/standalone/scrub: When possible show side-by-side diff in addition to regular diff

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22763 from dzafman/wip-remove-sudo

qa: Don't use sudo when moving logs

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22441 from ErwanAliasr1/evelu-makecheck

Improving make check reliability

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

qa: Don't use sudo when moving logs

Caused by: f0964beac5b8b54bc44a11bfb50bdea1f2ae8825

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #22565 from tchaikov/wip-seastar-thread-pool

crimson/thread: add thread pool

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cmake: add crimson static library

libcrimson consolidates the object library of crimson_{net,thread} and
seastar_{buffer,net,thread}_objs. by offering a static library glueing
seastar,  we can link against these libraries and its dependencies in a
simpler way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/thread: add a thread pool impl

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22007 from dalgaaf/wip-da-SCA-20180514

rgw, common: Fixes SCA issues

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: extract the RDMA libraries interface properties out

Signed-off-by: Kefu Chai <kchai@redhat.com>

do_freebsd.sh: disable rdma features

FreeBSD does have libibiverb and librdmacm, but we disable them anyway
for minimal dependency.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: consolidate WITH_{VERBS,RDMACM}

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22759 from ricardoasmarques/wip-fix-rbd-features-style

mgr/dashboard: Fix RBD features style

Reviewed-by: Tiago Melo <tmelo@suse.com>

systemd/rbdmap.service: remove a dependency on local-fs.target

We don't require anything outside of rootfs.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #22750 from joscollin/wip-dashboard-intro

doc: updated the overview and glossary for dashboard

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Fix RBD features style

Signed-off-by: Ricardo Marques <rimarques@suse.com>

doc: Updated dashboard doc references

Signed-off-by: Jos Collin <jcollin@redhat.com>

doc: updated the overview and glossary for dashboard

* Updated the overview of dashboard to avoid many paragraphs,
some outdated statements, etc.
* Updated glossary, glossary references and urls
* Dropped unnecessary term *module* in possible places
* Unified the new dashboard name as *Ceph Manager Dashboard*

Signed-off-by: Jos Collin <jcollin@redhat.com>

mgr: fix Clangs/FreeBSD missing vector

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #22702 from dzafman/wip-standalone-log

qa: For teuthology copy logs to teuthology expected location

Reviewed-by: Neha Ojha <nojha@redhat.com>

cmake: remove stale comments

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22726 from yuriw/wip-yuriw-crontab

qa/tests - added mimic-p2p suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #22560 into master

* refs/pull/22560/head:
qa/fs/upgrade/snaps: whitelist inconsistent rstat
mds: track snapshot count in rstat
mds: add MAY_SNAPSHOT in MDSAuthCaps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22719 into master

* refs/pull/22719/head:
qa: fix symlinks indirectly pointing at qa to .qa
qa: add .qa helper link

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22631 from tianshan/fix_notify_thread

rgw: fix meta and data notify thread miss stop cr manager

Reviewed-by: Casey Bodley <cbodley@redhat.com>

common: drop static_assert.h as it looks unused.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #22632 from tianshan/fix_index_zones_trace

rgw: fix index complete miss zones_trace set

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22660 from cfanz/wip-rgw-fix-24603

rgw-multisite: fix endless loop in RGWBucketShardIncrementalSyncCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #22725 into master

* refs/pull/22725/head:
qa/workunits/suites/blogbench.sh: use correct dir name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #22656 into master

* refs/pull/22656/head:
doc: update experimental features - snapshots
doc: highlight the flags

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22698 from cbodley/wip-async-shared-mutex

common: reintroduce async SharedMutex

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #22724 from ceph/rm-23941

ceph-volume: error on commands that need ceph.conf to operate

Reviewed-by: Alfredo Deza <adeza@redhat.com>

build: link ceph-osd with common statically. Enforce IE TLS model.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>