doc/ceph-volume include information for devices in listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 009d49a4f5cf8c97a156b710342d4786b734747b)

ceph-volume tests verify associated pvs reporting on listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f57cc594c282cf06a75b2278ca1c2b2d38f87aa3)

ceph-volume tests update to include devices on listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 08dea1b7d1d92d893c2f0a5e6e750d838d9cdbe4)

ceph-volume lvm.listing include associated physical devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit be1e6c2ae9a6e6a77fcb5297badb1aa58548180c)

ceph-volume api.lvm include lv_uuid on pvs listings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 12a7a48c53b16f52a76ad74773aaa67f8bfc7c52)

12.2.5

Merge pull request #21565 from alfredodeza/backport-wip-ceph-volume

luminous: ceph-volume: Nits noticed while studying code

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #21574 from alfredodeza/backport-cv-doc-updates

luminous: ceph-volume update man page for prepare/activate flags

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc/man/ceph-volume update flags for activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a454abc863817cd393e01e8c135174c1e1839be2)

doc/man/ceph-volume update flags for prepare

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ae96b9f0a283a65ddf6887461f3515a5a756572b)

Merge pull request #21492 from b-ranto/wip-mgr-rgw-luminous

[luminous] Backport recent prometheus rgw changes

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Remove unused and wrongly-described _lsblk_type

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit a97419f5d2b23131242a3602f2fa5f91ac7bc74d)

Fix mixing of format() and %-interpolation

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 9e9a07c9ae41cc72e587dfd6551fd83389cc35e6)

Spelling/typos

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 15ef400bb35e70cbb27144d595759d31a075aa37)

Merge pull request #21538 from alfredodeza/backport-rm23678

luminous: ceph-volume: allow skipping systemd interactions on activate/create

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

mgr/dashboard: performance counter browsers

Signed-off-by: Rubab-Syed <rubab.syed21@gmail.com>
(cherry picked from commit 25ecfbd9cfb620463de1545ca0be50001c805460)
Conflicts: Removed dashboard changes

mgr: expose have_mon_connection to python modules

So that they can warn their end users if the data
we have is probably stale.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5633328e28c0f7a5122229737c2ca132174a4349)

ceph-volume lvm.activate add --no-systemd flag to skip starting/enabling services

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 75685e33bfc192f3a3b16ad57b8cd40313bd8bc6)

ceph-volume lvm.common add --no-systemd flag to skip enabling/starting services

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 832da49ca8f1a57807f8a892cd734aa9b37216a6)

ceph-volume tests functional python tests for --no-systemd in activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8848d037086f889a880f5ea1d3bb2254e16faf7b)

Merge pull request #21533 from alfredodeza/backport-rm23067

luminous: ceph-volume Implement an 'activate all' to help with dense servers or migrating OSDs

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #21506 from liewegas/wip-rados-test-luminous

qa/workunits/rados/test-upgrade-to-mimic.sh: fix tee output

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

pybind/mgr/prometheus: fix creation of osd_metadata metric

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 05a37d4a61c427ed532138ab0c13a9dd4d0f65fb)

mgr/prometheus: fix and complete PG state names

...and reformat into a flat list in the same
order as found in osd_types.cc so that it's
easy to cross check.

Fixes: https://tracker.ceph.com/issues/23584
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 48acc6c7981ef6cbb962883cd36e2666a16d9c23)

mgr/prometheus: skip OSD output if missing from CRUSH devices

Apparently this occurs as a transient state while adding
OSDs.

Fixes: https://tracker.ceph.com/issues/23166
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit e8e5bffbb0842cc9c41bb64e7e9b5c6e2898fc0c)

ceph-volume lvm.activate error if no bluestore OSDs are found

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b1920d6e731f194c4829810d849dba6b087ce3a4)

ceph-volume tests/functional add 'activate all' for xenial/filestore/dmcrypt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d6524297ac423ffc7ea82dc9598d27b4cc61a0ae)

ceph-volume tests/functional add 'activate all' for xenial/bluestore/dmcrypt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 452081e57e5de9bb3a1391f22ffcac1bb712f2e9)

ceph-volume tests/functional add 'activate all' for centos7/filestore/dmcrypt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5513f4de4ee0b4802bc1c0d1af754dd2d80486ed)

ceph-volume tests/functional add 'activate all' for centos7/bluestore/dmcrypt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 16421c9058011e115cf043de4cf279cbbd0ac0ce)

ceph-volume tests/functional add 'activate all' in filestore playbook

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ef09f3996b8a165112afc4e6912a119015c798be)

ceph-volume tests/functional add 'activate all' in bluestore playbook

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 797518a54603ad8fc9b57a66eab7cf7e55ea0299)

ceph-volume tests ensure activate all behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2d8f48983d8674376a57ad98db16700bc00f8e29)

ceph-volume lvm.activate warn when no osds are found with --all

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 967938a0cf4353e99eb55815cde4ee90d6d4b79b)

ceph-volume tests add process.call unit tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6975e221d9f0c1b17531ca6fed87882b514e6d95)

ceph-volume lvm.activate add 'activate all' functionality

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fb0008ea8babe0065c08d3377bdabd36f3c2fa33)

ceph-volume process.call add conditional verbosity on failure

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 50c9d5331b0c42d568baa47a2d52f199b3b4d818)

ceph-volume systemd.systemctl add helpers for daemon status

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2e8249f9e233d408fc45b772e45995c00f5180d7)

ceph-volume lvm.listing create a helper that lists osds

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cee7612ce400ca4bc3e6e5dbcc1a158178edafe5)

ceph-volume lvv.activate allo to configure the CLI with --all

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4f305eb661553bebc1a86d18c11f596fa6206828)

Merge pull request #21509 from ceph/backport-rm23757

luminous: ceph-volume: allow parallel creates

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #21505 from ceph/backport-rm23581

luminous: ceph-volume: automatic VDO detection

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #21519 from ceph/evelu-luminous-readonly

luminous: ceph-volume: Using --readonly for {vg|pv|lv}s commands

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: Using --readonly for {vg|pv|lv}s commands

The actual code is detecting {vg|pv|lv}s by running the usual {vg|pv|lv}s commands.
Those calls expect lvmetad to be aware of the actual state of them.
This works actually pretty well in most of the cases.

When ceph-volume is run from a container,
this code also works from the container itself but don't on the host.

On the host side, running {vg|pv|lv}s commands reports nothing,
making ceph-volume reporting "No valid Ceph devices found"
The root cause is lvmetad not receiving the udev notification and so,
{vg|pv|lv}s commands reports the 'known' state instead of the 'real' state.

This is a major issue as it means that it exists cases or maybe races where
ceph-volume reports "No valid Ceph devices found" while the disk
actually have some Ceph data & metadata on them.
This could be interpreted like disks are free/available while they are not.
This will mislead users or configuration tools trying to understand the
current state of a node.

In July 2015, as per https://www.redhat.com/archives/lvm-devel/2015-July/msg00086.html,
a new option called "--readonly" have been added to lvm2.
One of the most interesting part of it is : "disable using lvmetad so VGs are read from disk"

In our case, that feature is really interesting as it means that
everytime ceph-volume calls a {vg|pv|lv}s command, it will read the
metadata from the disks instead of considering the lvmetad status.

This patch change all the {vg|pv|lv}s call to use --readonly.
It solves the bug exposed here and doesn't affect the traditional use-case.
The main benefit of this patch is to avoid a false report of a disk not having metadata.

(cherry picked from commit 7be00cd19297ec80b4a629e2fc1e79c64205402c)
Fixes: https://tracker.ceph.com/issues/23693
Signed-off-by: Erwan Velu <erwan@redhat.com>

ceph-volume: allow parallel creates

Currently the create code decides the vg_name "ceph-$cluster_fsid" as
the primary vg_name and creates a new name if this already exists.

If this code is run N times in parallel, the script will try to
create N times the vg with the name "ceph-$cluster_fsid" and it
will fail to create the N osds successfully.

Creating vgs with names like "ceph-$uuid4" lets our scripts to run
without any problems.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
(cherry picked from commit 201817f27ecfe195216284d9e4c344038cb654b6)

qa/workunits/rados/test-upgrade-to-mimic.sh: fix tee output

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-volume util.prepare preserve order on incoming mount flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89f8f82402f9eda7cf76ebe860e50dd8e1047d76)

ceph-volume tests update (preserved) order on mount flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1a92ecbede09f084ab34ef556de46b435fa3e9d1)

doc/dev/ceph-volume add VDO tag information

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit eeff3463543c12874ca4dfc3c3852af539d318f2)

ceph-volume tests ensure non-duplicate flags in mount calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 61bed1e615a111f1c978d474fda390ba980bfb85)

ceph-volume util.prepare allow and dedupe extra flags for mounting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 81c20e6b58b77671ffaa0f8030f18747e1a23569)

ceph-volume lvm.prepare detect vdo from device for mounting filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6b87580d4fe8e86de958ac258971cb2f90a70455)

ceph-volume lvm.activate detect vdo from tags for mounting filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0deef5f53c61b4215817008c04589a4f4b5cc3b3)

ceph-volume util.prepare allow extra ad-hoc mount flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d815c4f45429fd73d2506bd874656de438fe74fd)

ceph-volume lvm.prepare capture vdo-ness on LVM tags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c6a79ef21ccc61a7634c05db5a705fc3ef46fbb9)

ceph-volume tests for is_vdo detection

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 75f532597aa5ad5b84e993dc60c60d4a147af191)

ceph-volume api.lvm helpers to detect vdo from a dm path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5b97d8e54ce3dcf2a56872cdf0cbc0139ca0a723)

Merge pull request #21426 from ivancich/wip-fix-existing-mclock-cost-luminous

osd: remove cost from mclock op queues; cost not handled well in dmcl…

prometheus: Implement rgw_metadata metric

This maps rgw id to its hostname and ceph version running on the node.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit c7cbc24803da9474c9d1d15a52da8d0eed2d49af)

Merge pull request #21397 from dzafman/wip-23654

luminous: Special scrub handling of hinfo_key errors

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: Change shard digests to hex like object info digests

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 22ddc6da5fd03e65a7baf96748a8893cc258d977)

test: Luminous backport specific changes

osd-scrub-repair.sh:
    Remove legacy_snaps and redirect_target to keep test more in sync with master
    We don't have omap_digest bluestore handling
        Additional cases of omap_digest_mismatch_oi changes
        omap_digest still set
        ROBJ10 output still applies
osd-scrub-snaps.sh:
    Add head_exists in snapset
    Add snapset for object with head_mismatch

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #21216 from smithfarm/wip-23545-luminous

luminous: journal: Message too long error when appending journal

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21471 from liewegas/wip-librbd-upgrade-luminous

test/pybind/test_rbd: allow v1 images for testing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/pybind/test_rbd: allow v1 images for testing

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21452 from yuriw/wip-fix-p2p-luminous

qa/tests - added tag: v12.2.2 to be used by client.1 ...

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests - added tag: v12.2.2 to be used by client.1 under workload_luminous to avoid failures.

added workunits/rados/test-upgrade-to-mimic.sh, and add --gtest_filter=-CReadOpsTest.Exec to api_c_read_operations

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #21328 from tchaikov/wip-11332-luminous

luminous: mon: update PaxosService::cached_first_committed in PaxosService::may…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21365 from k0ste/luminous_backports

luminous: mgr: prometheus: fix PG state names

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #21394 from ifed01/wip-ifed-backport2

luminous:  BlueStore.cc: _balance_bluefs_freespace: assert(0 == "allocate failed, wtf");

Reviewed-by: Sage Weil <sage@redhat.com>

test: Use jq in a compatible way and for easier diff analysis

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 458babe7ee2e5fc0c4519a94249cbc1bd7cec9b6)

osd: remove cost from mclock op queues; cost not handled well in dmclock library

The current version of the dmclock library does not handle operation
cost well. Therefore cost should not be passed into the library when
enqueuing operations; instead 0 should be passed in.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #21388 from theanalyst/wip-luminous-usage-log

luminous: usage log fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21389 from theanalyst/wip-luminous-23687

Wip luminous 23687

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21393 from theanalyst/wip-luminous-recalc

luminous: rgw recalculate stats option added

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21390 from theanalyst/wip-luminous-23690

rgw: fix mutlisite read-write issues

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21233 from smithfarm/wip-jinja-mgr-luminous

luminous: build/ops: deb: move python-jinja2 dependency to mgr

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: James Page <james.page@ubuntu.com>

Merge pull request #21283 from liewegas/wip-pgpool-enc-luminous

luminous: osd/osd_types: fix pg_pool_t encoding for hammer

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21307 from smithfarm/wip-23606-luminous

luminous: tests: ENGINE Error in 'start' listener <bound  in rados

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #21330 from tchaikov/wip-23610-luminous

luminous: osd: resend osd_pgtemp if it's not acked

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21352 from pdvian/wip-23634-luminous

luminous: doc: outline upgrade procedure for mds cluster

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

os/bluestore: do not assert on insufficient allocation during bluefs
rebalance

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 64abc7ba39fab57cc9e19463219e8a70d284eda0)

test/store_test: add test case to for excessive fragmentation

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit eb0443a1034b42d006d2a1d2bd5cb91881f07df7)

os/bluestore: fix lack of extent demotion in StupidAllocator::init_rm_free

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 89aeea84f9ad24b2f9ca5ea3ee1dd9146e7eb664)

unittest_interval_set: test with flat_map<> too

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 70ddb0c18b2969c26dbe855e1573f3f028d9f113)

include/interval_set: cbegin/cend -> begin/end

c{begin,end} don't exist for btree_map<>.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7bbb7958df45b36556a0e469fe55f9236d036972)

include/btree_interval_set: remove it!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3dafe9d33eaadf8fbd3bed6cca5a4b2519b33a3f)

os/bluestore: use interval_set<...,btree_map<...>>

This avoid the code duplication, yay!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 762dd42971a9ced62ea5617f7cd6b0e0fd2ef74c)

include/btree_map: clean up wrapper header around btree_map

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 78d5afde4334e422cb8b915148ca3fc9f819570a)

os/bluestore/BitMapAllocator: drop useless include

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 79f9c86dfde7356915ec790f0cffca77579552e6)

include/interval_set: tolerate maps that invalidate iterator on change

These changes picked out of the diff between the original
btree_interval_set.h and interval_set.h (sadly I had it rolled into the
initial commit so it was tedious to identify these).

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 87aed2026a8f689540a92eeeea6b4054b6c69acd)

include/interval_set: templating map used

Default to std::map for backward compat.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d254259059c47c714526976a6144eec3b52e9bcd)

common/interval_set: override subset_of for given range

E.g.:
subset_of([5~10,20~5], 0, 100)  -> [5~10,20~5]
subset_of([5~10,20~5], 5, 25)   -> [5~10,20~5]
subset_of([5~10,20~5], 1, 10)   -> [5~5]
subset_of([5~10,20~5], 8, 24)   -> [8~7, 20~4]

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit f90bd4b957938c1ce050b3f5783ead37b297ddc2)

inverval_set: optimize subset_of with sequential search

Optimize subset_of to use sequential search when it
performs better than the lower_bound method, for set
size ratios smaller than 10. This is analogous to
intersection_of behavior since commit 825470fcf919.

The subset_of method can be used in some cases as a
less-expensive alternative to the intersection_of
method, since subset_of can return early if any element
of the smaller set is not contained in the larger set,
and intersection_of has the added burden of storing
the intersecting elements.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit 5c3775815a433a028618b239f5a4bebacff3dc4b)

interval_set: optimize intersection_of

Iterate over all elements of the smaller set, and use find_inc to
locate elements from the larger set in logarithmic time. This greatly
improves performance when one set is much larger than the other:

     2 +-+--+----+----+----+----+----+----+----+----+--+-+
P      +*                                                +
E      |*                                                |
R  1.8 +*                                                +
F      | *                                               |
O      | *                                               |
R  1.6 + *                                               +
M      | *                                               |
A      | *                                               |
N  1.4 + *                                               +
C      |  *                                              |
E      |  *                                              |
   1.2 +   *                                             +
R      |    *                                            |
A      |     *                                           |
T    1 +      ***                                        +
I      |         ******                                  |
O      +               ***********************************
   0.8 +-+--+----+----+----+----+----+----+----+----+--+-+
       0   0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9   1

                          SET SIZE RATIO

The above plot compares performance of the new intersection_size_asym
function to the existing intersection_of function. The performance of
intersection_size_asym gets worse as the set size ratio approaches 1.
For set size ratios where the performance ratio is greater than 1, the
performance of intersection_size_asym is superior. Therefore, this
patch only uses intersection_size_asym when the set size ratio is less
than or equal to 0.1 (code uses the reciprocal which is 10).

The plot was generated using benchmark results produced by the
following program:

#include <iostream>
#include <sys/timeb.h>
#include "include/interval_set.h"

int main()
{
  const int interval_count = 100000;
  const int interval_distance = 4;
  const int interval_size = 2;
  const int sample_count = 8;
  const int max_offset = interval_count * interval_distance;
  interval_set<int> a, b, intersection;

  for (int i = 0; i < max_offset; i+=interval_distance) {
    a.insert(i, interval_size);
  }

  for (int m = 1; m < 100; m++) {
    float ratio = 1 / float(m);

    for (int i = 0; i < max_offset; i+=interval_distance*m) {
      b.insert(i, interval_size);
    }

    struct timeb start, end;
    int ms = 0;
    for (int i = 0; i < sample_count; i++) {
      ftime(&start);
      intersection.intersection_of(a, b);
      ftime(&end);
      ms += (int) (1000.0 * (end.time - start.time)
        + (end.millitm - start.millitm));
      intersection.clear();
    }
    b.clear();

    std::cout << ratio << "\t" << ms << std::endl << std::flush;
  }
}

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit 825470fcf9190c94422716454dbf11b24350a748)

rgw: rgw-rados, rgw-admin add an option to recalculate user stats

Adds a method in rgw-rados to reset user stats calling the earlier implemented
cls user reset stats.
In rgw-admin we add an option called --reset-stats that invokes this method.

Fixes: http://tracker.ceph.com/issues/23335
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 0b72dd7d3beea47b5c479eeb979b73c4cd70f7bc)
(cherry picked from commit 0bd635b2133c7536f0f3e561579e4aab9cf72d59)

cls/user: implement reset user stats

This is an implementation of reset user stats, that recalculates the user stats
purely based on the values of bucket entries in user.buckets object. This is
helpful in cases when user stats has been improperly set in case of manual
resharding etc.

Luminous only additions:
use global scope for encode in cls_user_ops

Fixes: http://tracker.ceph.com/issues/23335
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 63f9127dd685890d938a612739320c942a96f910)

rgw: radosgw-admin abort early for user stats for empty uids

Otherwise we'll call sync stats with an empty argument which results in creation
of an empty ".buckets" object

Fixes: http://tracker.ceph.com/issues/23322
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit aa3a408d4a66296cbf273551920f8ac9b381b2dd)
(cherry picked from commit 99dec6115bb6d8ef1cdaf658fc359a301d31017d)

rgw: Fix multisite Synchronization failed when read and write delete at the same time

This case is first write objA,then write and delete objA at the same
time,write early than delete.
When del objA, use information which stat  of first write objA, so the
op should del the first write data.However when try to del objA, objA
header is second write, so osd "do_xattr_cmp_str" has found idtag change
and return -125(canceled),after rgw client receive the ret -125 , it
will still do "complete_del", then do cls_obj_complete_del to write
bilog。"complete_op" in cls_rgw module  will  write bilog with second
write mtime and second ".ver.epoch". Finally, del op append behind the
second write in bilog. And the slave rgw will merge write op and del op
as del op, and del data,but master rgw complete second write and cancel
del.
This logic is problematic,  so bilog recording the del op should use
cancel op. And squash_map should skip the cancel op.

Fixes: http://tracker.ceph.com/issues/22804
Signed-off-by: Niu Pengju <pengju.niu@xtaotech.com>
(cherry picked from commit a8fd508915a0acf7768fe6ba7f7414e71813f1d7)

rgw: warn user about on going resharding when canceling it

Fixes: http://tracker.ceph.com/issues/21619
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit f4dae60d4cc1ae4099c8d5da785f32a7c39f5a79)