]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agolibrbd: discard should wait for in-flight cache writeback to complete 23594/head
Jason Dillaman [Wed, 4 Apr 2018 15:48:56 +0000 (11:48 -0400)]
librbd: discard should wait for in-flight cache writeback to complete

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(derived from commit 0e04de4bb9c08755bd0915776bd538ce8f7c2b5b)

6 years agoMerge pull request #23680 from k0ste/luminous_backports
Yuri Weinstein [Fri, 14 Sep 2018 16:47:36 +0000 (09:47 -0700)]
Merge pull request #23680 from k0ste/luminous_backports

luminous: osd: backport 'bench' and stdout changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #23740 from k0ste/luminous_backports2
Yuri Weinstein [Fri, 14 Sep 2018 16:46:50 +0000 (09:46 -0700)]
Merge pull request #23740 from k0ste/luminous_backports2

luminous: mgr/balancer: deepcopy best plan - otherwise we get latest

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #23791 from tchaikov/wip-23370-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:46:18 +0000 (09:46 -0700)]
Merge pull request #23791 from tchaikov/wip-23370-luminous

luminous: mgr/MgrClient: guard send_pgstats() with lock

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #23862 from smithfarm/wip-25177-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:45:41 +0000 (09:45 -0700)]
Merge pull request #23862 from smithfarm/wip-25177-luminous

luminous: osd,mon: increase mon_max_pg_per_osd to 250

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #23864 from smithfarm/wip-25203-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:45:09 +0000 (09:45 -0700)]
Merge pull request #23864 from smithfarm/wip-25203-luminous

luminous: rados python bindings use prval from stack

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23871 from smithfarm/wip-32106-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:44:25 +0000 (09:44 -0700)]
Merge pull request #23871 from smithfarm/wip-32106-luminous

luminous: object errors found in be_select_auth_object() aren't logged the same

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #23901 from pdvian/wip-32078-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:43:52 +0000 (09:43 -0700)]
Merge pull request #23901 from pdvian/wip-32078-luminous

luminous: test: Use pids instead of jobspecs which were wrong

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #23943 from batrick/i35716
Yuri Weinstein [Fri, 14 Sep 2018 16:43:12 +0000 (09:43 -0700)]
Merge pull request #23943 from batrick/i35716

luminous: msg: "challenging authorizer" messages appear at debug_ms=0

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #23969 from zmedico/backport-lumious-17121-17239-17410-17820
Yuri Weinstein [Fri, 14 Sep 2018 16:42:50 +0000 (09:42 -0700)]
Merge pull request #23969 from zmedico/backport-lumious-17121-17239-17410-17820

luminous: PGPool::update optimizations

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23499 from pdvian/wip-26844-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:40:50 +0000 (09:40 -0700)]
Merge pull request #23499 from pdvian/wip-26844-luminous

luminous: rgw_file: deep stat handling

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
6 years agoMerge pull request #23522 from pdvian/wip-26846-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:41:51 +0000 (08:41 -0700)]
Merge pull request #23522 from pdvian/wip-26846-luminous

luminous: rgw: Limit the number of lifecycle rules on one bucket.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
6 years agoMerge pull request #23545 from pdvian/wip-26848-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:41:18 +0000 (08:41 -0700)]
Merge pull request #23545 from pdvian/wip-26848-luminous

luminous: rgw: The delete markers generated by object expiration should have owner

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
6 years agoMerge pull request #23665 from pdvian/wip-26922-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:40:43 +0000 (08:40 -0700)]
Merge pull request #23665 from pdvian/wip-26922-luminous

luminous: rgw: Fix log level of gc_iterate_entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #23974 from pdvian/wip-35069-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:40:17 +0000 (08:40 -0700)]
Merge pull request #23974 from pdvian/wip-35069-luminous

cls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24034 from smithfarm/wip-25087-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:39:12 +0000 (08:39 -0700)]
Merge pull request #24034 from smithfarm/wip-25087-luminous

luminous: change default rgw_thread_pool_size to 512

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24046 from pdvian/wip-35707-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:38:42 +0000 (08:38 -0700)]
Merge pull request #24046 from pdvian/wip-35707-luminous

luminous: rgw: raise default rgw_curl_low_speed_time to 300 seconds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24082 from alfredodeza/luminous-rm35970
Andrew Schoen [Thu, 13 Sep 2018 18:01:28 +0000 (13:01 -0500)]
Merge pull request #24082 from alfredodeza/luminous-rm35970

luminous: ceph-volume fix zap not working with LVs

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24080 from alfredodeza/luminous-rm35913
Alfredo Deza [Thu, 13 Sep 2018 17:59:27 +0000 (13:59 -0400)]
Merge pull request #24080 from alfredodeza/luminous-rm35913

luminous: ceph-volume batch: allow --osds-per-device, default it to 1

Reviewed-by: Andrew Schoen aschoen@redhat.com
6 years agoceph-volume tests.functional comment why not use --destroy when zapping lvs in filestore 24082/head
Alfredo Deza [Thu, 13 Sep 2018 14:03:06 +0000 (10:03 -0400)]
ceph-volume tests.functional comment why not use --destroy when zapping lvs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 11858c74a1e704d63185f0ef4b6cd6aa02f17e52)

6 years agoceph-volume tests.functional use --destroy when zapping lvs in bluestore
Alfredo Deza [Thu, 13 Sep 2018 14:00:37 +0000 (10:00 -0400)]
ceph-volume tests.functional use --destroy when zapping lvs in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 87af815691a7f43d14067c24e308d60f39d58051)

6 years agoceph-volume lvm.zap use remove_pv only for physical devices
Alfredo Deza [Thu, 13 Sep 2018 13:34:54 +0000 (09:34 -0400)]
ceph-volume lvm.zap use remove_pv only for physical devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 036683fde799ab1cfad7d6192981515d6b3419cc)

6 years agoceph-volume tests.lvm.strategies initial filestore validation 24080/head
Alfredo Deza [Wed, 12 Sep 2018 13:44:32 +0000 (09:44 -0400)]
ceph-volume tests.lvm.strategies initial filestore validation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d1d68eade56ba3f898cd3bfd29aa1ee3bf5460ec)

6 years agoceph-volume lvm.tests conftest fixtures for custom vgs
Alfredo Deza [Tue, 11 Sep 2018 20:53:54 +0000 (16:53 -0400)]
ceph-volume lvm.tests conftest fixtures for custom vgs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f14db432317c15b29b187d1e1a4bd3a729a7043)

6 years agoceph-volume tests.devices.lvm add bluestore strategy cases
Alfredo Deza [Tue, 11 Sep 2018 20:53:05 +0000 (16:53 -0400)]
ceph-volume tests.devices.lvm add bluestore strategy cases

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2ced05a2bc6ba01cf8946f43d5839890a28cab20)

6 years agoceph-volume tests verify strategy validators for minimum size
Alfredo Deza [Tue, 11 Sep 2018 16:57:21 +0000 (12:57 -0400)]
ceph-volume tests verify strategy validators for minimum size

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9167f76c2a7107c82b781cbe5f51e24ee72e4db4)

6 years agoceph-volume lvm.batch.filestore allow osds-per-device
Alfredo Deza [Tue, 11 Sep 2018 16:12:48 +0000 (12:12 -0400)]
ceph-volume lvm.batch.filestore allow osds-per-device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 66947537b542da82185ea4d79bfaedaae3117918)

6 years agoceph-volume lvm.batch.validators account for osds per device, include collocation
Alfredo Deza [Tue, 11 Sep 2018 16:11:01 +0000 (12:11 -0400)]
ceph-volume lvm.batch.validators account for osds per device, include collocation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 04a40eda0ac089c07654d9ab07958126f0fc8f84)

6 years agoceph-volume lvm.batch.bluestore allow osds-per-device with MixedType
Alfredo Deza [Mon, 10 Sep 2018 21:57:44 +0000 (17:57 -0400)]
ceph-volume lvm.batch.bluestore allow osds-per-device with MixedType

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 028ef60df768c24a9573a044a451164dcb037677)

6 years agoceph-volume lvm.batch.bluestore allow osds-per-device with SingleType
Alfredo Deza [Mon, 10 Sep 2018 21:01:57 +0000 (17:01 -0400)]
ceph-volume lvm.batch.bluestore allow osds-per-device with SingleType

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8c0e707110e284626fe4161097a3121170a3f09d)

6 years agoceph-volume lvm.batch add --osds-per-device flag, defaulting to 1
Alfredo Deza [Mon, 10 Sep 2018 20:12:59 +0000 (16:12 -0400)]
ceph-volume lvm.batch add --osds-per-device flag, defaulting to 1

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe7c5e0129172a4b2a216e7c49e67d8ead570b39)

6 years agoMerge pull request #24075 from alfredodeza/luminous-rm34535
Andrew Schoen [Thu, 13 Sep 2018 12:51:31 +0000 (07:51 -0500)]
Merge pull request #24075 from alfredodeza/luminous-rm34535

luminous: ceph-volume batch carve out lvs for bluestore

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.util verify Disk objects don't change state with divisions 24075/head
Alfredo Deza [Mon, 10 Sep 2018 16:28:57 +0000 (12:28 -0400)]
ceph-volume tests.util verify Disk objects don't change state with divisions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9ca526336221ea2ad32e7daf3e5e46b07291a1d8)

6 years agoceph-volume util.disk fix an issue where Disk objects would mutate on div operations
Alfredo Deza [Mon, 10 Sep 2018 16:18:49 +0000 (12:18 -0400)]
ceph-volume util.disk fix an issue where Disk objects would mutate on div operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7d1a3c74240e67621e993a304eaf566d4e2789d1)

6 years agoceph-volume util.prepare add a helper to get block.db sizes from ceph.conf
Alfredo Deza [Mon, 10 Sep 2018 16:09:50 +0000 (12:09 -0400)]
ceph-volume util.prepare add a helper to get block.db sizes from ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 860068ec5daca6d7087cf1ce0594a2fe36e91193)

6 years agoceph-volume lvm.batch.bluestore add TODOs for custom fast/slow devices
Alfredo Deza [Mon, 10 Sep 2018 16:09:15 +0000 (12:09 -0400)]
ceph-volume lvm.batch.bluestore add TODOs for custom fast/slow devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0eb0bdfc55d8ceb27cd12bc3a7fb807a7e65c788)

6 years ago-f ceph-volume lvm.batch.bluestore validation and reporting with VG reuse
Alfredo Deza [Mon, 10 Sep 2018 16:08:35 +0000 (12:08 -0400)]
-f  ceph-volume lvm.batch.bluestore validation and reporting with VG reuse

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d117d66903fa740108c9948b499150b02523fdf4)

6 years agoceph-volume lvm.batch fix error reporting, Device objects aren't strings
Alfredo Deza [Fri, 7 Sep 2018 18:11:33 +0000 (14:11 -0400)]
ceph-volume lvm.batch fix error reporting, Device objects aren't strings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 267a386cc270081b2cd284bf78bf94b52be113dc)

6 years agoceph-volume lvm.batch.bluestore validation and reporting with VG reuse
Alfredo Deza [Fri, 7 Sep 2018 18:06:13 +0000 (14:06 -0400)]
ceph-volume lvm.batch.bluestore validation and reporting with VG reuse

Reworks the bluestore validation and reporting to account for reusable
VGs from fast devices, and adds validation calls to ensure the new way
to calculate this process will work.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 27f47565a0d43ecee03dea4f6addc739e3e78655)

6 years agoceph-volume lvm.batch.filestore capture SizeAllocationErrors
Alfredo Deza [Fri, 7 Sep 2018 18:04:58 +0000 (14:04 -0400)]
ceph-volume lvm.batch.filestore capture SizeAllocationErrors

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 71fcd35c3d44795f92b17e9aedb7ca868b3750d3)

6 years agoceph-volume lvm.batch make sure data devices don't have existing LVs on bluestore
Alfredo Deza [Thu, 6 Sep 2018 17:03:58 +0000 (13:03 -0400)]
ceph-volume lvm.batch make sure data devices don't have existing LVs on bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d2ea49a4e64820fc3cd95c2a6ca8869309638e00)

6 years agoMerge pull request #23216 from b-ranto/wip-mgr-prom-bp-luminous
Yuri Weinstein [Wed, 12 Sep 2018 21:56:35 +0000 (14:56 -0700)]
Merge pull request #23216 from b-ranto/wip-mgr-prom-bp-luminous

luminous: Sync the prometheus module

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #23461 from pdvian/wip-26838-luminous
Yuri Weinstein [Wed, 12 Sep 2018 21:54:37 +0000 (14:54 -0700)]
Merge pull request #23461 from pdvian/wip-26838-luminous

luminous: mgrc: enable disabling stats via mgr_stats_threshold

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #23566 from pdvian/wip-26908-luminous
Yuri Weinstein [Wed, 12 Sep 2018 21:53:52 +0000 (14:53 -0700)]
Merge pull request #23566 from pdvian/wip-26908-luminous

luminous: kv/KeyValueDB: return const char* from MergeOperator::name()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23579 from VictorDenisov/backport_26934
Yuri Weinstein [Wed, 12 Sep 2018 21:53:19 +0000 (14:53 -0700)]
Merge pull request #23579 from VictorDenisov/backport_26934

luminous: osd/OSDMap: fix apply upmap segfault

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #24039 from ceph/wip-s3a-fix-mirror
vasukulkarni [Wed, 12 Sep 2018 18:51:46 +0000 (11:51 -0700)]
Merge pull request #24039 from ceph/wip-s3a-fix-mirror

luminous: qa/tasks:  s3a fix mirror

6 years agoMerge pull request #23211 from neha-ojha/wip-24988-luminous
Yuri Weinstein [Wed, 12 Sep 2018 15:24:09 +0000 (08:24 -0700)]
Merge pull request #23211 from neha-ojha/wip-24988-luminous

luminous: osd: Limit pg log length during recovery/backfill so that we don't run out of memory

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agorgw: raise default rgw_curl_low_speed_time to 300 seconds 24046/head
Casey Bodley [Mon, 27 Aug 2018 18:45:35 +0000 (14:45 -0400)]
rgw: raise default rgw_curl_low_speed_time to 300 seconds

this timeout is mainly to avoid waiting forever when a sync connection
goes away without seeing a close. a 30-second timeout is much more
likely to catch other delays like loaded osds - and timeout/retry will
increase that load

Fixes: http://tracker.ceph.com/issues/27989
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2fe357657158f6454865b9c37314fa8cfdfc4771)

6 years agoMerge pull request #23761 from k0ste/luminous_backports3
Nathan Cutler [Tue, 11 Sep 2018 20:08:13 +0000 (22:08 +0200)]
Merge pull request #23761 from k0ste/luminous_backports3

luminous: docs: radosgw: ldap-auth: fixed option name 'rgw_ldap_searchfilter'

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoqa/tasks: update mirror link for maven, the original mirror no longer exists 24039/head
Vasu Kulkarni [Thu, 6 Sep 2018 00:08:24 +0000 (17:08 -0700)]
qa/tasks: update mirror link for maven, the original mirror no longer exists

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 13e100259e3658fcfd2123474cb8ea8498ac3fb6)

6 years agoMerge pull request #23621 from joscollin/wip-26917-luminous
Nathan Cutler [Tue, 11 Sep 2018 16:26:09 +0000 (18:26 +0200)]
Merge pull request #23621 from joscollin/wip-26917-luminous

luminous: doc: Fix broken urls

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoqa/rgw: override valgrind --max-threads for radosgw 24034/head
Casey Bodley [Wed, 1 Aug 2018 17:01:30 +0000 (13:01 -0400)]
qa/rgw: override valgrind --max-threads for radosgw

radosgw now uses 512 frontend threads by default, and valgrind won't
start with its default --max-threads=500

Fixes: http://tracker.ceph.com/issues/25214
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e617990efe4c39f592cff1541ea55fbcf9f2d6f2)

Conflicts:
qa/suites/rgw/tempest/tasks/rgw_tempest.yaml
qa/suites/rgw/verify/tasks/0-install.yaml

6 years agorgw: change default rgw_thread_pool_size to 512
Douglas Fuller [Fri, 15 Jun 2018 16:13:34 +0000 (12:13 -0400)]
rgw: change default rgw_thread_pool_size to 512

This value is commonly tuned up and demonstrates better performance on
large systems.

Fixes: http://tracker.ceph.com/issues/24544
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit 28b0c1054fbc0d390d12fc32540c66b1e8c0bae4)

6 years agoMerge pull request #23967 from alfredodeza/luminous-rm35535
Andrew Schoen [Mon, 10 Sep 2018 14:29:22 +0000 (09:29 -0500)]
Merge pull request #23967 from alfredodeza/luminous-rm35535

luminous: ceph-volume: batch tests for mixed-type of devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoosd/PrimaryLogPG: avoid dereferencing invalid complete_to 23211/head
xie xingguo [Mon, 3 Sep 2018 07:37:36 +0000 (15:37 +0800)]
osd/PrimaryLogPG: avoid dereferencing invalid complete_to

For the auto-repair (EIO caused) case, we will not reinitialize
**complete_to** (because last_complete is equal to last_update!)
and hence there is chance that **complete_to** should aleady
point to **log.end()** before we call recover_got.

We could simply drop it here as we (already) logged the **complete_to**
iterator change in a more compatible way a few lines below.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 69a2cc35840939436da09691ca62476d7f599de4)

6 years agocls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder 23974/head
Vaibhav Bhembre [Thu, 31 May 2018 14:35:19 +0000 (10:35 -0400)]
cls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder

Signed-off-by: Vaibhav Bhembre vaibhav@digitalocean.com
(cherry picked from commit a503cec93853235f2f94dbc53eb307618f5a437a)

6 years agoosd/PGPool::update: optimize with subset_of 23969/head
Zac Medico [Wed, 27 Sep 2017 01:33:15 +0000 (18:33 -0700)]
osd/PGPool::update: optimize with subset_of

Replace expensive inverval_set intersection_of and operator==
calls with a single subset_of call. I borrowed this idea from
Piotr Dałek's "osd/PGPool: don't use intermediate interval set"
patch. The following benchmark program demonstrates a 38%
performance increase:

#include <iostream>
#include <chrono>
#include "include/interval_set.h"

#define NANOSECONDS(d) \
    std::chrono::duration_cast<std::chrono::nanoseconds>(d).count()

typedef uint64_t snapid_t;
typedef std::chrono::steady_clock::duration duration;

duration PGPool_update_old(const interval_set<snapid_t> &rs) {
  std::chrono::steady_clock::time_point start, end;
  interval_set<snapid_t> newly_removed_snaps, cached_removed_snaps;

  // initialize state
  cached_removed_snaps = rs;

  // start timed simulation
  start = std::chrono::steady_clock::now();

  {
    newly_removed_snaps = cached_removed_snaps;
    interval_set<snapid_t> intersection;
    intersection.intersection_of(newly_removed_snaps, cached_removed_snaps);

    assert(intersection == cached_removed_snaps);
    cached_removed_snaps.swap(newly_removed_snaps);
    newly_removed_snaps = cached_removed_snaps;
    newly_removed_snaps.subtract(intersection);
  }

  // end timed simulation
  end = std::chrono::steady_clock::now();

  return end - start;
}

duration PGPool_update_new(const interval_set<snapid_t> &rs) {
  std::chrono::steady_clock::time_point start, end;
  interval_set<snapid_t> newly_removed_snaps, cached_removed_snaps;

  // initialize state
  cached_removed_snaps = rs;

  // start timed simulation
  start = std::chrono::steady_clock::now();

  {
    newly_removed_snaps = cached_removed_snaps;

    assert(cached_removed_snaps.subset_of(newly_removed_snaps));
    interval_set<snapid_t> removed_snaps = newly_removed_snaps;
    newly_removed_snaps.subtract(cached_removed_snaps);
    cached_removed_snaps.swap(removed_snaps);
  }

  // end timed simulation
  end = std::chrono::steady_clock::now();

  return end - start;
}

int main(int argc, char *argv[])
{
  assert(argc == 3);
  const int sample_count = std::stoi(argv[1]);
  const int interval_count = std::stoi(argv[2]);
  const int interval_distance = 4;
  const int interval_size = 2;
  const int max_offset = interval_count * interval_distance;
  interval_set<snapid_t> removed_snaps;

  for (int i = 0; i < max_offset; i += interval_distance)
    removed_snaps.insert(i, interval_size);

  duration old_delta(0), new_delta(0);

  for (int i = 0; i < sample_count; ++i) {
    old_delta += PGPool_update_old(removed_snaps);
    new_delta += PGPool_update_new(removed_snaps);
  }

  float ratio = float(NANOSECONDS(old_delta)) / NANOSECONDS(new_delta);

  std::cout << ratio << std::endl;
}

Suggested-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit 18cbba4d2d7e26153afb01a3d904d5fb81cd4568)

6 years agoPGPool::update: optimize removed_snaps comparison when possible
Zac Medico [Thu, 31 Aug 2017 20:36:30 +0000 (13:36 -0700)]
PGPool::update: optimize removed_snaps comparison when possible

In self/unmanaged snaps mode, optimize removed_snaps comparison
for cases where removed_snaps has not changed. This exploits the
fact that remove_unmanaged_snap adds a dummy removed snapshot
to the end of removed_snaps, allowing for inexpensive detection
of changes. In cases where removed_snaps is very large, this
optimization improves performance dramatically.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit caf6803b13d9dbd4540da366b018d721fcfc371a)

6 years agoPGPool::update: avoid expensive union_of
Zac Medico [Thu, 24 Aug 2017 16:40:45 +0000 (09:40 -0700)]
PGPool::update: avoid expensive union_of

Achieve the same result using a swap (constant complexity),
and an assignment (linear complexity).

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit bf20b661cb6a6a33d9e93fb44ad1470f9f8245d2)

6 years agoPGPool::update: optimize with interval_set.swap
Zac Medico [Mon, 21 Aug 2017 20:30:53 +0000 (13:30 -0700)]
PGPool::update: optimize with interval_set.swap

Use constant complexity swap instead of linear complexity
assignment.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit ae64179a63351ac3966e70a074776188ddba5b84)

6 years agoceph-volume lvm.batch use 'ceph' as the cluster name with filestore 23967/head
Alfredo Deza [Thu, 6 Sep 2018 19:19:01 +0000 (15:19 -0400)]
ceph-volume lvm.batch use 'ceph' as the cluster name with filestore

Custom cluster names are currently broken on ceph-volume, should get
addressed with http://tracker.ceph.com/issues/27210 which is out of
scope for these changes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c1481799a254eceb05258313aafd532e062ea0bc)

6 years agoceph-volume tests/functional update filestore xenial test vars
Alfredo Deza [Thu, 6 Sep 2018 15:23:30 +0000 (11:23 -0400)]
ceph-volume tests/functional update filestore xenial test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a096a016ccd0acc86ce816aba8ce95f6e3b68c11)

6 years agoceph-volume tests/functional update bluestore xenial test vars
Alfredo Deza [Thu, 6 Sep 2018 15:23:20 +0000 (11:23 -0400)]
ceph-volume tests/functional update bluestore xenial test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89e52dd19792072919bef6ba2f2ab50d7b0ff4ed)

6 years agoceph-volume tests/functional update filestore centos7 test vars
Alfredo Deza [Thu, 6 Sep 2018 15:18:20 +0000 (11:18 -0400)]
ceph-volume tests/functional update filestore centos7 test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a5ec54207a02c4bb85dc8226890def18da45e2ff)

6 years agoceph-volume tests/functional update bluestore centos7 test vars
Alfredo Deza [Thu, 6 Sep 2018 15:18:05 +0000 (11:18 -0400)]
ceph-volume tests/functional update bluestore centos7 test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2549d5737283b39ecf5b0af98c73ffeb18033c36)

6 years agoceph-volume lvm.api use double -f flags when calling pvremove
Alfredo Deza [Thu, 6 Sep 2018 14:55:31 +0000 (10:55 -0400)]
ceph-volume lvm.api use double -f flags when calling pvremove

Fairly destructive, just like everything else when zapping a device.
This is required in the case of double UUIDs detected, something that
surfaced when testing with a loop device to create an nvme (the loop
device ends up with the same UUID as the nvme).

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c1cbc407c095e0bc3e25a8ff1519b4909ec519cc)

6 years agoceph-volume tests/functional remove duplicate test playbooks
Alfredo Deza [Thu, 6 Sep 2018 14:26:05 +0000 (10:26 -0400)]
ceph-volume tests/functional remove duplicate test playbooks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a008fe1385e4440f510eab97470a5d4e56a69402)

6 years agoceph-volume tests/functional setup playbook for nvme devices
Alfredo Deza [Thu, 6 Sep 2018 13:38:00 +0000 (09:38 -0400)]
ceph-volume tests/functional setup playbook for nvme devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 71db509df0c4a14598010568f8bb078f6b2841ab)

6 years agoceph-volume tests/functional single (test) playbook to zap and redeploy batch operations
Alfredo Deza [Thu, 6 Sep 2018 13:34:11 +0000 (09:34 -0400)]
ceph-volume tests/functional single (test) playbook to zap and redeploy batch operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 23330c0107a9f9ee436338c112b1852fd36e34a3)

6 years agoceph-volume tests/functional create a noop playbook to allow symlinking always to...
Alfredo Deza [Thu, 6 Sep 2018 13:33:21 +0000 (09:33 -0400)]
ceph-volume tests/functional create a noop playbook to allow symlinking always to setup.yml

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 860e66083fe769714a10076fa101b94507e47169)

6 years agoceph-volume tests/functional add mixed-type scenarios to tox.ini
Alfredo Deza [Thu, 6 Sep 2018 13:32:00 +0000 (09:32 -0400)]
ceph-volume tests/functional add mixed-type scenarios to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0d5b90bb1fbf30c6e33dbf7e10a3d0ba053297ae)

6 years agoceph-volume tests/functional add CentOS7 bluestore mixed-type scenarios
Alfredo Deza [Wed, 5 Sep 2018 12:17:54 +0000 (08:17 -0400)]
ceph-volume tests/functional add CentOS7 bluestore mixed-type scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 799ce4508d9ed166ef6447daa21e0564262fbafe)

6 years agoceph-volume tests/functional add CentOS7 filestore mixed-type scenarios
Alfredo Deza [Wed, 5 Sep 2018 12:17:38 +0000 (08:17 -0400)]
ceph-volume tests/functional add CentOS7 filestore mixed-type scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89bbcff659eed8f2cfa26fb93a0c12c7ff83e83b)

6 years agomsg: lower verbosity on normal event 23943/head
Patrick Donnelly [Sun, 2 Sep 2018 19:51:46 +0000 (12:51 -0700)]
msg: lower verbosity on normal event

Fixes: http://tracker.ceph.com/issues/35251
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 351831834d4f9c66395539d7015bcdb017c8dba6)

Conflicts:
src/msg/async/AsyncConnection.cc
src/msg/simple/Pipe.cc

6 years agoMerge pull request #23913 from yuriw/wip-35541-luminous
Nathan Cutler [Wed, 5 Sep 2018 18:28:17 +0000 (20:28 +0200)]
Merge pull request #23913 from yuriw/wip-35541-luminous

qa/tests:  added v12.2.8 to the mix

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoqa/tests: added v12.2.8 to the mix 23913/head
Yuri Weinstein [Tue, 4 Sep 2018 16:05:42 +0000 (09:05 -0700)]
qa/tests:  added v12.2.8 to the mix

Fixes: http://tracker.ceph.com/issues/35541
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agotest: Use pids instead of jobspecs which were wrong 23901/head
David Zafman [Thu, 16 Aug 2018 21:30:09 +0000 (14:30 -0700)]
test: Use pids instead of jobspecs which were wrong

Fixes: http://tracker.ceph.com/issues/27056
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bc3317031057b5737710a095ce593135b62f9302)

6 years agotest: Update tests for error message changes 23871/head
David Zafman [Thu, 16 Aug 2018 22:31:53 +0000 (15:31 -0700)]
test: Update tests for error message changes

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 687f63e59970433bfb53c17d4128a3d042a0c9c6)

Conflicts:
qa/standalone/scrub/osd-scrub-snaps.sh

- luminous has two additional errors

6 years agoosd: Use colon to separate error message from other info
David Zafman [Thu, 16 Aug 2018 22:24:11 +0000 (15:24 -0700)]
osd: Use colon to separate error message from other info

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bdd32cc84787892b3005f2644b5ad9f8ac7e772d)

6 years agoqa: Add new message to whitelist for scrub/repair tests
David Zafman [Mon, 6 Aug 2018 17:37:04 +0000 (10:37 -0700)]
qa: Add new message to whitelist for scrub/repair tests

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b40784290f3b600941e0d97372fddee5c85ec20c)

6 years agotest: Verify cluster logging of scrub error messages
David Zafman [Fri, 3 Aug 2018 20:47:30 +0000 (13:47 -0700)]
test: Verify cluster logging of scrub error messages

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 58c4d322032d04b0d55f627ce42531ca3edab3e9)

6 years agoosd: Log shard errors in be_select_auth_object() as other errors
David Zafman [Wed, 1 Aug 2018 22:13:40 +0000 (15:13 -0700)]
osd: Log shard errors in be_select_auth_object() as other errors

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit cadf727258ea8b3a52ae9122b42a0b4f6914e177)

Conflicts:
src/osd/PGBackend.cc

- luminous still has oi_prio

6 years agoosd: Move object error check out of be_select_auth_object()
David Zafman [Wed, 1 Aug 2018 00:53:17 +0000 (17:53 -0700)]
osd: Move object error check out of be_select_auth_object()

Fixes: http://tracker.ceph.com/issues/25108
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c789fd1899f697a994a28e45a6f9262c6721d77d)

Conflicts:
src/osd/PGBackend.cc

- luminous still has auth_prio and different way of decoding bliter

6 years agopybind/rados/rados: do not pass prval from stack 23864/head
Sage Weil [Mon, 30 Jul 2018 19:18:07 +0000 (14:18 -0500)]
pybind/rados/rados: do not pass prval from stack

The prval is a pointer to an int to write the final completion code of
the rados op.  This can't be on the stack since we immediately leave the
current scope after preparing the op (looong before we do the rados op).

We keep the tuple return value to avoid breaking users of this API
(devicehealth module, gnocchi at a minimum).

Fixes: http://tracker.ceph.com/issues/25175
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8e36f18cdeaa2088a6ce4aaad61b76283e777270)

6 years agoosd,mon: increase mon_max_pg_per_osd to 250 23862/head
Neha Ojha [Thu, 26 Jul 2018 00:25:35 +0000 (17:25 -0700)]
osd,mon: increase mon_max_pg_per_osd to 250

https://bugzilla.redhat.com/show_bug.cgi?id=1603615 indicates
a case when pg calc conflicts with mon_max_pg_per_osd, and does not
allow pool creation when this limit is 200. Hence, increase this limit
to avoid this.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f7911de61bca7c7f1ef007eb229ee845923c9751)

6 years agoosd/PrimaryLogPG.cc: limit trimming at can_rollback_to
Neha Ojha [Thu, 16 Aug 2018 18:48:19 +0000 (11:48 -0700)]
osd/PrimaryLogPG.cc: limit trimming at can_rollback_to

This change is motivated by the failures seen in the multimds suite,
where we hit assert(s <= can_rollback_to), while trimming the log in ec
pools.

This is due to the fact that we had removed limits on the trim_to value to
address https://tracker.ceph.com/issues/23979.

But, seems that this could be dangerous for ec pools. So, keep the
can_rollback_to limit, while calculating the trim_to value.

Fixes: http://tracker.ceph.com/issues/21416
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4b5c6b88d444e2173e716fe4890717873c8dc8e5)

6 years ago12.2.8 v12.2.8
Jenkins Build Slave User [Thu, 30 Aug 2018 17:24:39 +0000 (17:24 +0000)]
12.2.8

6 years agoMerge pull request #23825 from alfredodeza/luminous-rm34309
Andrew Schoen [Thu, 30 Aug 2018 14:52:41 +0000 (09:52 -0500)]
Merge pull request #23825 from alfredodeza/luminous-rm34309

luminous: ceph-volume: update batch documentation to explain filestore strategies

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agodoc/man/ceph-volume uncomment flags that are now fully supported in batch 23825/head
Alfredo Deza [Tue, 28 Aug 2018 20:34:25 +0000 (13:34 -0700)]
doc/man/ceph-volume uncomment flags that are now fully supported in batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f8db9faf9836328b920133c5c9c47a6ffacea443)

6 years agodoc/ceph-volume add filestore docs for batch
Alfredo Deza [Tue, 28 Aug 2018 20:32:19 +0000 (13:32 -0700)]
doc/ceph-volume add filestore docs for batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 188f4ae80fefcdd8d95d719a88d8422b2c120f0c)

6 years agodoc/ceph-volume uncomment section of newly supported batch operations
Alfredo Deza [Tue, 28 Aug 2018 20:22:20 +0000 (13:22 -0700)]
doc/ceph-volume uncomment section of newly supported batch operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 02652343c8501a144c5ae2c012f335e8d0f236cf)

6 years agoMerge pull request #23813 from alfredodeza/luminous-rm34311
Alfredo Deza [Wed, 29 Aug 2018 22:23:28 +0000 (15:23 -0700)]
Merge pull request #23813 from alfredodeza/luminous-rm34311

luminous: ceph-volume: tests.functional inherit SSH_ARGS from ansible

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.functional use Debian ansible os family in deploy.yml 23813/head
Alfredo Deza [Wed, 29 Aug 2018 17:32:55 +0000 (10:32 -0700)]
ceph-volume tests.functional use Debian ansible os family in deploy.yml

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7a076f5e4f58bc3b346cde306ba949aabbb9b743)

6 years agoceph-volume lvm.batch use the abspath attribute for Device objects in bluestore
Alfredo Deza [Wed, 29 Aug 2018 16:56:21 +0000 (09:56 -0700)]
ceph-volume lvm.batch use the abspath attribute for Device objects in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b2a62c2b13d3e079f8020865b7c41f0f6c4b5d71)

6 years agoceph-volume lvm.batch remove usage of non-existent method
Alfredo Deza [Wed, 29 Aug 2018 16:55:31 +0000 (09:55 -0700)]
ceph-volume lvm.batch remove usage of non-existent method

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6f3094449f46d10260c0a505b39571c35235292e)

6 years agoceph-volume tests.functional inherit SSH_ARGS from ansible with synchronize module
Alfredo Deza [Tue, 28 Aug 2018 22:34:58 +0000 (15:34 -0700)]
ceph-volume tests.functional inherit SSH_ARGS from ansible with synchronize module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8e6269336f6e177f415a51e9fedd36244014215a)

6 years agoMerge pull request #23811 from alfredodeza/luminous-rm34310
Andrew Schoen [Wed, 29 Aug 2018 19:45:05 +0000 (14:45 -0500)]
Merge pull request #23811 from alfredodeza/luminous-rm34310

luminous: ceph-volume lvm.batch remove non-existent sys_api property

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume lvm.batch remove non-existent sys_api property 23811/head
Alfredo Deza [Tue, 28 Aug 2018 21:28:08 +0000 (14:28 -0700)]
ceph-volume lvm.batch remove non-existent sys_api property

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a76fccba5ff0058d0dd1ba47f48b34dba27dd51f)

6 years agoMerge pull request #23809 from alfredodeza/luminous-ceph-volume-testing
Andrew Schoen [Wed, 29 Aug 2018 19:28:03 +0000 (14:28 -0500)]
Merge pull request #23809 from alfredodeza/luminous-ceph-volume-testing

luminous: ceph-volume: run tests without waiting on ceph repos

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: retry when 'vagrant up' fails 23809/head
Andrew Schoen [Wed, 22 Aug 2018 20:37:46 +0000 (16:37 -0400)]
ceph-volume: retry when 'vagrant up' fails

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 2c88e70877dcc2a04a69be4ab9c4834d5ced6198)