Merge pull request #21649 from dillaman/wip-23872

osdc/Objecter: prevent double-invocation of linger op callback

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21656 from neha-ojha/wip-integrate-cosbench

qa/tasks: run cosbench using the CBT task

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tasks: run cosbench using the CBT task

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #21676 from liewegas/wip-23871

osd/PrimaryLogPG: clear data digest on WRITEFULL if skip_data_digest

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21686 from jdurgin/wip-ec-backend-comment

osd/ECBackend: update misleading comment about EIO handling

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #21653 from liewegas/wip-23364

osd/PrimaryLogPG: avoid infinite loop when flush collides with write lock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21222 from bmflynn/pybind-alignment

pybind/rados: add alignment getter to IoCtx

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ECBackend: update misleading comment about EIO handling

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21061 from jan--f/jan-prometheus-file-sd-command

pybind/mgr/prometheus: add file_sd_config command

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #21646 from liewegas/wip-rados-api-tests

qa/suites/rados/verify/tasks/rados_api_tests: whitelist OBJECT_MISPLACED

Merge pull request #19509 from cooboos/wip-cephdencoder-additem

ceph-dencoder: add missing begin_iter & end_iter item for RGWObjManifest

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #20835 from croit/fix-mon-crash-on-bad-snap-rm

mon: handle bad snapshot removal reqs gracefully

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21093 from guzhongyan/mgr-ov

mgr/DaemonServer: add overrides value to 'config show'

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21585 from xxhdx1985126/wip-23807

msgr/simple: set Pipe::out_seq to in_seq of the connecting side

Reviewed-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: clear data digest on WRITEFULL if skip_data_digest

If we detect we are on bluestore and should skip the data digest, we
need to clear any old digest on WRITEFULL.  This aligns us with the
WRITE behavior, which is also either an update or a clear (but never
neither!).

Fixes: https://tracker.ceph.com/issues/23871
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21667 from ricardoasmarques/wip-fix-delete-modal

mgr/dashboard: Fix missing $event on deletion modal

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21584 from oritwas/wip-rgw-resharding-versioning

rgw: Fix infinite call for bi list when resharding a bucket

mgr/dashboard: Fix missing $event on deletion modal

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #21658 from tchaikov/wip-msg-many-failure

qa: reduce "mon client hunt interval max multiple" to 2 for all clients

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #19491 from yaarith/wip-mgr-module-hello

pybind/mgr: new 'hello world' mgr module skeleton

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21529 from ricardoasmarques/wip-fix-rbd-form

mgr/dashboard: Display RBD form errors on submission

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21591 from zhouyuan/fix_wo_mgr_build

cmake: fix building without mgr module

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21526 from ricardoasmarques/wip-rbd-snapshot-clone

mgr/dashboard: RBD copy, RBD flatten and snapshot clone (frontend)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21558 from votdev/cd_error_panel

mgr/dashboard: Add 'cd-error-panel' component to display error messages

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #21612 from Songweibin/wip-pglog-cleanup

osd/PGLog: fix sanity check against **complete-to** iter

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #21608 from cbodley/wip-dpp

dout: DoutPrefixProvider operates directly on stream

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20480 from smithfarm/wip-23035

build/ops: run-make-check.sh: handle Python 2 absence

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21629 from liewegas/wip-23840

os/bluestore/bluefs_types: make block_mask 64-bit

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

qa: reduce "mon client hunt interval max multiple" to 2 for all clients

because with high failure rate, we need to connect to mon more
frequently if the connection fails.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21639 from guzhongyan/pglog_clean

osd/PrimaryLogPG: Fix log messages

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/PrimaryLogPG: avoid infinite loop when flush collides with write lock

We try to take a write lock with fop->op.  If we fail, fop->op is put
on the lock's waiting list.  Requeuing it again will simply kick off
processing of another instance of the same op, which will again fail
to take the lock, leading to an infinite loop that can't terminate
because requeue_op is doing a push_front and preventing other PG
messages that might release the lock.

Do the same write lock attempt on any dup_ops so that they too will
end up on the wait list.

It looks like this broke waaay back in commit d700d99f76e0a29bfb419bc85d19ef1950b62a9a,
a 2014 refactor of the OpContext behavior.

Fixes: https://tracker.ceph.com/issues/23664
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21645 from alfredodeza/wip-rm23451

ceph-volume include physical devices associated with an LV when listing

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #21650 from ricardoasmarques/wip-install-deps-tumbleweed

install-deps: Add support for 'opensuse-tumbleweed'

Reviewed-by: Nathan Cutler <ncutler@suse.com>

install-deps: Add support for 'opensuse-tumbleweed'

Signed-off-by: Ricardo Marques <rimarques@suse.com>

osdc/Objecter: prevent double-invocation of linger op callback

Fixes: http://tracker.ceph.com/issues/23872
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21619 from liewegas/wip-merge-bits

mon/OSDMonitor: require --yes-i-really-mean-it for force-create-pg

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21628 from liewegas/wip-23646

osd/PrimaryLogPG: defer evict if head *or* object intersect scrub interval

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #21338 from mogeb/iostat-output

mgr/iostat: print output as a table

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reveiwed-by: John Spray <john.spray@redhat.com>

Merge pull request #21613 from xiexingguo/wip-fix-delete-priority

osd/PG: fix out of order priority for PG deletion

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21623 from liewegas/wip-21977

osd: publish osdmap to OSDService before starting wq threads

Reviewed-by: David Zafman <dzafman@redhat.com>

qa/suites/rados/verify/tasks/rados_api_tests: whitelist OBJECT_MISPLACED

The api tests do some splits, which can move data.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/ceph-volume include information for devices in listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

pybind/mgr: add 'hello world' mgr module skeleton

This simple 'hello' mgr module prints 'hello world' to stdout and to out/mgr.x.log; for documentation purposes. Also added /doc/mgr/hello.rst.

Signed-off-by: Yaarit Hatuka <yaarithatuka@gmail.com>

ceph-volume tests verify associated pvs reporting on listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests update to include devices on listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.listing include associated physical devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #21626 from cbodley/wip-23846

mgrc: free MMgrClose in handle_mgr_close

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21541 from dillaman/wip-18753

qa: introduce rbd-mirror thrasher to existing tests

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21632 from tchaikov/wip-rocksdb-no-aligned-alloc

rocksdb: do not use aligned_alloc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21479 from cbodley/wip-23547

rgw: add buffering filter to compression for fetch_remote_obj

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

pybind/mgr/prometheus: add file_sd_config command

This command returns a valid list of static targets pointing to the mgr
prometheus modules for a cluster. The output can be stored in a
file and the file be listed under the file_sd_config stanza in the
prometheus configuration.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #21576 from cbodley/wip-23506

rgw: fix error handling for GET with ?torrent

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #21569 from cbodley/wip-23803

rgw: aws4 auth supports PutBucketRequestPayment

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #20212 from tianshan/fix_period_pull

rgw: do not reflect period if not current

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20796 from theanalyst/rgw/es-cs-fix

rgw: es module: set compression type correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21500 from yehudasa/wip-23779

rgw: encoding fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21534 from cbodley/wip-23480

rgw: move all pool creation into rgw_init_ioctx

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21551 from zhangsw/fix-rgw-userinfo-stats-bug

rgw: admin rest api shouldn't return error when getting user's stats if the user hasn't create any bucket.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21640 from votdev/config_set_help_txt

mgr/dashboard: Adapt help text if server_addr is not set

Reviewed-by: John Spray <john.spray@redhat.com>

Use bootstrap alert.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add cd-error-panel component to display error messages.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Adapt help text if server_addr is not set

After 'ceph config-key set' is deprecated the help text must be adapted.

Signed-off-by: Volker Theile <vtheile@suse.com>

osd/PrimaryLogPG: Fix log messages

output the content, rathor than the address of obc makes sense,
just like what _copy_some() did.

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #21636 from rjfd/wip-dashboard-fix-pool-py3

mgr/dashboard: pool: fix python3 dict_keys error

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21618 from votdev/loading_panel

mgr/dashboard: Add 'cd-loading-panel' component

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #21633 from ukernel/wip-mds-stat-rstat

mds: fix CEPH_STAT_RSTAT definition

mgr/dashboard: pool: fix python3 dict_keys error

Signed-off-by: Ricardo Dias <rdias@suse.com>

rocksdb: do not use aligned_alloc

this change picks up the change to "use posix_memalign() for
aligned_alloc()" in ceph/rocksdb.

Fixes: http://tracker.ceph.com/issues/23653
Signed-off-by: Kefu Chai <kchai@redhat.com>

mds: fix CEPH_STAT_RSTAT definition

(CEPH_CAP_GWREXTEND << 16) is (1<<22), which causes gaps in caps bits.
CEPH_CAP_FILE_WREXTEND is an unused bit, reuse it for CEPH_STAT_RSTAT.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #21273 from jdurgin/wip-23195

osd/ECBackend: only check required shards when finishing recovery reads

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21620 from jcsp/wip-mgr-balancer-options

mgr/balancer: fix OPTIONS definition

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore/bluefs_types: make block_mask 64-bit

Since block_size is 32 bits, we return 0x00000000fffff000 (or similar).
This breaks things if you get a file (e.g., bluefs metadata log file)
over 4GB.

Fixes: https://tracker.ceph.com/issues/23840
Signed-off-by: Sage Weil <sage@redhat.com>

ceph-volume api.lvm include lv_uuid on pvs listings

Signed-off-by: Alfredo Deza <adeza@redhat.com>

osd/PrimaryLogPG: defer evict if head *or* object intersect scrub interval

Consider a scenario like:
- scrub [3:2525d100:::earlier:head,3:2525d12f:::foo:200]
 - we see 3:2525d12f:::foo:100 and include it in scrub map
- scrub [3:2525d12f:::foo:200, 3:2525dfff:::later:head]
- some op(s) that cause scrub to be preempted
- agent_work wants to evict 3:2525d12f:::foo:100
  - write_blocked_by_scrub sees scrub is preempted, returns false
  - 3:2525d12f:::foo:100 is removed, :head SnapSet is updated
- scrub rescrubs [3:2525d12f:::foo:200, 3:2525dfff:::later:head]
  - includes (updated) :head SnapSet
  - issues error like "3:2525d12f:::foo:100 is an unexpected clone"

Fix the problem by checking if anything part of the object-to-evict and
its head touch the scrub range; if so, back off.  Do not let eviction
preempt scrub; we can come back and do it later.

Fixes: http://tracker.ceph.com/issues/23646
Signed-off-by: Sage Weil <sage@redhat.com>

osd: publish osdmap to OSDService before starting wq threads

The worker threads may request the map from the service (notably,
advance_pg() does this).

Fixes: http://tracker.ceph.com/issues/21977
Signed-off-by: Sage Weil <sage@redhat.com>

mgrc: free MMgrClose in handle_mgr_close

Fixes: http://tracker.ceph.com/issues/23846
Signed-off-by: Casey Bodley <cbodley@redhat.com>

pybind/rados: add alignment getter to IoCtx

Adds alignment method to retrieve the pool alignment bytes for EC pools
to the existing IoCtx class.

Signed-off-by: Bruce Flynn <bmflynn@gmail.com>

Merge pull request #21606 from ceph/fix-build-CEPH_CLS_API

rados/objclass.h: fix build define CEPH_CLS_API in all cases

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21621 from theanalyst/doc/releases/12.2.5

doc: v12.2.5 luminous release notes

Reviewed-By: Sage Weil <sage@redhat.com>

mgr/dashboard: Introduce 'cd-loading-panel' component

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Remove RBD "Default Features" checkbox

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add RBD default features REST API endpoint

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add support for RBD snapshot copy (frontend)

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add support for RBD snapshot copy

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: RBD flatten

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: RBD copy (frontend)

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: RBD snapshot clone (frontend)

Signed-off-by: Ricardo Marques <rimarques@suse.com>

doc: v12.2.5 luminous release notes

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

Merge pull request #21595 from votdev/load_on_init

mgr/dashboard: Load the datatable content on component initialization

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/iostat: print output as a table

Signed-off-by: Mohamad Gebai <mgebai@suse.com>

Merge pull request #20899 from Devp00l/deletion-dialog

mgr/dashboard: Adds reusable deletion dialog

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/balancer: fix OPTIONS definition

I typo'd a couple of these, which was causing test failures
when the access to the correct names got an exception.

Signed-off-by: John Spray <john.spray@redhat.com>

mon/OSDMonitor: require --yes-i-really-mean-it for force-create-pg

This command declares data loss permanent.  Be sure the user really means
it.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/SnapMapper: print debug line on check failure

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Display RBD form error on submission

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Support for template forms on 'cd-submit-button'

If 'formDir' is provided, 'cd-submit-button' will mark
it as submitted after submission.

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Replaces delete-confirmation-modal

This replaces usage of "delete-confirmation-modal" with the use of
"delete-modal".

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Change deletion link to modal only

Due to CSS problems the link solution wasn't the best way, now it will
represent only the modal content. The downside of this solution is that
it put's the burden on the developer to use it the right way and import
a view things to get it working. But on the upside CSS styles will work
as expected.

The unit test example was updated accordingly this way it should be easy
to understand how it can be implemented the right way.

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Changes deletion button to link

This change was made because a link can be placed anywhere instead of a
button element.

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Adds reusable deletion dialog

You can now simply use a deletion dialog without having to import a lot
of different things from ngx-bootstrap. Its easy to extend the dialog
by a detail description.

Signed-off-by: Tiago Melo <tmelo@suse.com>
Signed-off-by: Stephan Müller <smueller@suse.com>