common/config: Add description to (near)full ratio settings

For many users it is not clear that these only apply to initial creation
of the cluster and that afterwards these commands need to be used:

  ceph osd set-nearfull-ratio <float[0.0-1.0]>
  ceph osd set-full-ratio <float[0.0-1.0]>

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #21644 from p-na/grafana-proxy

mgr/dashboard: Grafana proxy backend

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #21880 from votdev/upgrade_swimlane_datatable_angular_6

mgr/dashboard: Upgrade Swimlane's data-table

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22386 from tspmelo/fix-24281

mgr/dashboard: Changed default value of decimal point to 1

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22404 from ricardoasmarques/wip-optimize-rbd-edit

mgr/dashboard: Do not fetch pool list on RBD edit

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22409 from wido/mgr-dashboard-port

mgr/dashboard: Listen on port 8443 by default and not 8080

Reviewed-by: Kai Wagner <kwagner@suse.com>

mgr/dashboard: Add Grafana proxy

Add the ability to connect the dashboard with a Grafana instance to
display graphs from Grafana incorporated into the dashboard.

Possible configuration options:

$ ceph dashboard set-grafana-api-host <host> # Default: 'localhost'
$ ceph dashboard set-grafana-api-port <port> # Default: 3000
$ ceph dashboard set-grafana-api-scheme <scheme> # Default: 'http'

The configuration can be checked using a GET request to the API endpoint
`/api/grafana/status`. It returns a JSON response in the following
format:

    {
"available": False,
"message": "<error message>",
    }

The message is not returned if Grafana is available.

The Grafana proxy is reachable through
`http(s)://<host>/api/grafana/proxy`.

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #22365 from tone-zhang/doc_spdk

doc: update the description for SPDK in bluestore-config-ref.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22356 from tone-zhang/wip-bug-24371

spdk: fix ceph-osd crash when activate SPDK

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22329 from bstillwell/transaction_code_comments_corrections

os: Minor fixes in comments describing a transaction

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22371 into master

* refs/pull/22371/head:
doc/conf.py: fix man page build vs governance.rst
doc/governance: adjust title
doc/governance: fix link
doc/governance: edits and add user committee
doc/governance.rst: a few notes on ceph project governance

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

doc/conf.py: fix man page build vs governance.rst

Signed-off-by: Sage Weil <sage@redhat.com>

doc/governance: adjust title

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22408 from wumingqiao/wip-pgscan-cleanup

messages/MOSDPGScan: fix initialization of query_epoch

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #22420 from b-ranto/wip-mgr-fix-avg

mgr: expose avg data for long running avgs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22396 from liewegas/wip-24396

osd/PrimaryLogPG: fix on_local_recover crash on stray clone

Reviewed-by: David Zafman <dzafman@redhat.com>

osd/PrimaryLogPG: fix on_local_recover crash on stray clone

If there is a stray clone (one that does not appear in the SnapSet) and
we do any sort of recovery on it the OSD will crash.  Log an error instead
but continue.

This addresses a problem where a cluster has both (1) an unexpected clone
and (2) the clone is not present on all replicas.  Doing repair on that
PG will both not fix the unexpected clone and also cause the remaining
OSDs to crash trying to recover it.

Include a test.

Fixes: https://tracker.ceph.com/issues/24396
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21999 from myoungwon/wip-manifest-unset-op

osd, librados: add unset-manifest op

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22395 from liewegas/wip-24408

osd,mds: make 'config rm ...' idempotent

Reviewed-by: Erwan Velu <erwan@redhat.com>

Merge pull request #21775 from Synesis-LLC/fix-23964

librados: fix buffer overflow for aio_exec python binding

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22417 from tspmelo/fix-lint-error

mgr/dashboard: fix lint error

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Listen on port 7000 by default and not 8080

Port 8080 is a common alternative HTTP port used for web traffic.

The dashboard however uses SSL (which can not be turned off) and for
that purpose we should either use 8443.

Historically the dashboard would run on port 7000 so to keep this backwards
compatibility the dashboard should bind again on TCP 7000.

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #22402 from ricardoasmarques/wip-stacktrace-optional

mgr/dashboard: Stacktrace is optional on 'js-error' endpoint

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

doc/governance: fix link

Signed-off-by: Sage Weil <sage@redhat.com>

doc/governance: edits and add user committee

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Upgrade Swimlane's data-table

Upgrade to latest version 13.0.1 which now supports Angular 6. Check https://github.com/swimlane/ngx-datatable/blob/13.0.1/docs/changelog.md for more information.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #22399 from alfredodeza/wip-rm24417

ceph-volume tests do not include admin keyring in OSD nodes

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

mgr: expose avg data for long running avgs

While exposing the long running avgs, the order of regular and long
running avg data got mixed up resulting in exposing empty data for both
since there is no regular data for long running avgs and vice versa.
This commit fixes this issue by changing the order of the if/else
statement exposing regular data for regular counters and avg data for
long running avgs.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #22401 from ricardoasmarques/wip-prettify-package-json

mgr/dashboard: Prettify package.json

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: fix lint error

Error was introduced in 78dc50443cf64168098f1fb5bc43b7d6125621b8.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21826 from tspmelo/rxjs6

mgr/dashboard: Update RxJS to version 6

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22122 from tspmelo/wip-osd-scrub

mgr/dashboard: Add scrub action to the OSDs table

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Changed default value of decimal point to 1

Changed the default decimals point to 1 in 'formater_number' method, this
is used in the 'dimlessPipe' and 'dimlessBinaryPipe'.
This value can still be changed when needed, using one of the parameters.

I have also removed the 'truncate' method.

Fixes: https://tracker.ceph.com/issues/24281
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22392 from tchaikov/wip-cmake-boost-1.67

cmake: bump up the required boost version to 1.67

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Hide `required` for read-only fields on RBD edit

Signed-off-by: Ricardo Marques <rimarques@suse.com>

messages/MOSDPGScan: fix initialization of query_epoch

Signed-off-by: wumingqiao <wumingqiao@inspur.com>

Merge pull request #22405 from neha-ojha/wip-24365

qa/tasks/cbt.py: changes to run on bionic

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tasks/cbt.py: changes to run on bionic

Signed-off-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: Do not fetch pool list on RBD edit

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Stacktrace is optional on 'js-error' endpoint

Signed-off-by: Ricardo Marques <rimarques@suse.com>

osd,mds,common: make 'config rm ...' idempotent

Fixes: http://tracker.ceph.com/issues/24408
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Prettify package.json

Signed-off-by: Ricardo Marques <rimarques@suse.com>

ceph-volume tests do not include admin keyring in OSD nodes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22393 from wido/ceph-volume-bluestore-tool

ceph-volume lvm.activate Do not search for a MON configuration

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

common/config: fix rm_val error codes

EINVAL if the config option is invalid.
ENOENT if the config option wasn't set.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #22359 into master

* refs/pull/22359/head:
osdc/objecter: remove unused method finish_op

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: bump up the required boost version to 1.67

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Add scrub action to the OSD table

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add OSD scrub modal component

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add scrub methods to OSD controller and service

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Extract 'srt_to_bool' method

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21247 from MahatiC/wip-rbd-datapool

rbd: add data pool support to trash purge

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22389 from MahatiC/nit-cleanup

doc: cleanup rendering syntax

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22384 from ukernel/wip-mds-session-ref

mds: add reference when setting Connection::priv to existing session

Merge pull request #22346 from dzafman/wip-scrub-omap

osd: Handle omap and data digests independently

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: cleanup

cleanup format

Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>

ceph-volume lvm.activate Do not search for a MON configuration

This system might not have one nor can we be sure that we have
a client.admin keyring on the system.

Just prime the directory and have the OSD then use it's own cephx
key to communicate with the MONs.

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #22328 from tspmelo/wip-tslint-unused

mgr/dashboard: Add 'no-unused-variable' rule to tslint

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22350 from theanalyst/doc-mimic-index-fix

doc: mimic is stable!

Reviewed-by: Kefu Chai <kchai@redhat.com>

mds: fix MDSRank::send_message_client_counted()

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: add reference when setting Connection::priv to existing session

the bug was introduced by commit 72883956c26 "use intrusive_ptr for
holding Connection::priv"

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

osd: use null if on_finisher is not needed

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

osd: set has_reference flag when need_reference is set

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

src/test: add a unit test for unset-manifest

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

doc: update the description for SPDK in bluestore-config-ref.rst

Add more information to show the device initialization in SPDK.

Signed-off-by: tone-zhang <tone.zhang@arm.com>

mgr/dashboard: Add 'no-unused-variable' rule to tslint

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update RxJS to version 6

I used rxjs-tslint tool to automatically update most of the necessary code.

I added a new package, rxjs-compat, to maintain the compatibility
of external plugins that are still using RxJS v5.X.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21718 from majianpeng/admin-socket-cleanup

common/admin_socket: add new api unregister_commands(AdminSocketHook …

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21913 from tspmelo/wip-cephfs-table

mgr/dashboard: Add Filesystems list component

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22196 from thinkercui/bugfix

osd: read object attrs failed at EC recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22336 from majianpeng/bluestore-handle-kvdb-error

os/bluestore: firstly delete db then delete bluefs if open db met error.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #22306 from liewegas/wip-msgr-addrvec

msg: addr -> addrvec (part 1)

Reviewed-by: Ricardo Dias <rdias@suse.com>

spdk: fix ceph-osd crash when activate SPDK

When activate SPDK in Ceph, observe a Ceph crash.

It happened in the case 1) there is a pending IOContext in
NVMEDevice and 2) the lower block device has been recreated and
queue_t then points to an NVMEDevice that no longer exists.

This patch addresses the problem by deleting and zeroing queue_t
when NVMEDevice is closed.

Fixes: http://tracker.ceph.com/issues/24371
Signed-off-by: tone-zhang <tone.zhang@arm.com>

Merge pull request #22342 from dillaman/wip-23516

librbd: commit IO as safe when complete if writeback cache is disabled

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22348 from dillaman/wip-24169

rbd-mirror: ensure daemon can cleanly exit if pool is deleted

Reviewed-by: Mykola Golub <mgolub@suse.com>

doc/governance.rst: a few notes on ceph project governance

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21812 from thmour/thmour_ceph-volume

ceph-volume: sort and align `lvm list` output

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22367 from tchaikov/wip-cmake-rgw

cmake: rgw_common should depend on tracing headers

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #22364 from tchaikov/wip-doc

 doc: fix the links in releases/schedule.rst

Reviewed-by: Sage Weil <sage@redhat.com>

osd/OSDMap: switch to addrvec for public_addr

Signed-off-by: Sage Weil <sage@redhat.com>

mon/Session: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: remove get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: avoid get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t contains() and is_same_host()

Signed-off-by: Sage Weil <sage@redhat.com>

osd: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

client: use get_addrs()

These are legacy interfaces that should return an IP only.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t::front()

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: avoid OSDMap::get_addr()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: entity_addrvec_t::probably_equals()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addrvec_t::get_ports()

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: change 'osd find' output to list all addrs

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: remove get_inst()

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: use get_addrs

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MRoute: remove unused entity_inst_t dest

This is unused in luminous and mimic, too, although the code was
never removed.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: remove dead try_send_message()

No more users!

Signed-off-by: Sage Weil <sage@redhat.com>

mon: remove dead send_command()

No users.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: remove dead 'osd scrub' implementation from mon

This is now handled by the mgr (as of luminous).

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MOSDMarkMeDown: use addrvec

Signed-off-by: Sage Weil <sage@redhat.com>

cmake: rgw_common should depend on tracing headers

see also: 4c0d3531

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22324 from cbodley/wip-rgw-cache-time-added

rgw: update ObjectCacheInfo::time_added on overwrite