]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoMerge pull request #27827 from ashishkumsingh/wip-39496-nautilus
Yuri Weinstein [Thu, 30 May 2019 20:25:01 +0000 (13:25 -0700)]
Merge pull request #27827 from ashishkumsingh/wip-39496-nautilus

nautilus: rgw: admin: handle delete_at attr in object stat output

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27904 from pdvian/wip-39228-nautilus
Yuri Weinstein [Thu, 30 May 2019 20:24:30 +0000 (13:24 -0700)]
Merge pull request #27904 from pdvian/wip-39228-nautilus

nautilus: rgw_file: save etag and acl info in setattr

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
6 years agoMerge pull request #27890 from Devp00l/wip-39534-nautilus
Yuri Weinstein [Thu, 30 May 2019 19:24:41 +0000 (12:24 -0700)]
Merge pull request #27890 from Devp00l/wip-39534-nautilus

 nautilus: mgr/dashboard: RBD snapshot name suggestion with local time suffix

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge pull request #28021 from Devp00l/wip-39558-nautilus
Yuri Weinstein [Thu, 30 May 2019 19:24:11 +0000 (12:24 -0700)]
Merge pull request #28021 from Devp00l/wip-39558-nautilus

 nautilus: mgr/dashboard: KV-table transforms dates through pipe

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge pull request #28091 from pdvian/wip-39423-nautilus
Yuri Weinstein [Wed, 29 May 2019 22:10:20 +0000 (15:10 -0700)]
Merge pull request #28091 from pdvian/wip-39423-nautilus

nautilus: test/librbd: drop 'ceph_test_librbd_api' target

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #28120 from dillaman/wip-38869-nautilus
Yuri Weinstein [Wed, 29 May 2019 22:09:37 +0000 (15:09 -0700)]
Merge pull request #28120 from dillaman/wip-38869-nautilus

nautilus: librbd: make flush be queued by QOS throttler

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28121 from dillaman/wip-39428-nautilus
Yuri Weinstein [Wed, 29 May 2019 22:08:52 +0000 (15:08 -0700)]
Merge pull request #28121 from dillaman/wip-39428-nautilus

nautilus: librbd: async open/close should free ImageCtx before issuing callback

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28122 from dillaman/wip-39462-nautilus
Yuri Weinstein [Wed, 29 May 2019 22:08:14 +0000 (15:08 -0700)]
Merge pull request #28122 from dillaman/wip-39462-nautilus

nautilus: rbd-mirror: clear out bufferlist prior to listing mirror images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28203 from dzafman/wip-39699
Yuri Weinstein [Wed, 29 May 2019 22:07:24 +0000 (15:07 -0700)]
Merge pull request #28203 from dzafman/wip-39699

nautilus: osd: Don't include user changeable flag in snaptrim related assert

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #28258 from pdvian/wip-39672-nautilus
Yuri Weinstein [Wed, 29 May 2019 22:06:25 +0000 (15:06 -0700)]
Merge pull request #28258 from pdvian/wip-39672-nautilus

nautilus: os/bluestore: fix missing discard in BlueStore::_kv_sync_thread

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #27746 from LenzGr/wip-39345-nautilus
Yuri Weinstein [Wed, 29 May 2019 21:59:52 +0000 (14:59 -0700)]
Merge pull request #27746 from LenzGr/wip-39345-nautilus

nautilus: mgr/dashboard: code documentation

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agoMerge pull request #27760 from ricardoasmarques/wip-39454-nautilus
Yuri Weinstein [Wed, 29 May 2019 21:58:49 +0000 (14:58 -0700)]
Merge pull request #27760 from ricardoasmarques/wip-39454-nautilus

nautilus: mgr/dashboard: Validate if any client belongs to more than one group

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
6 years agoMerge pull request #27868 from LenzGr/wip-39467-nautilus
Yuri Weinstein [Wed, 29 May 2019 21:57:39 +0000 (14:57 -0700)]
Merge pull request #27868 from LenzGr/wip-39467-nautilus

nautilus: mgr/dashboard: Admin resource not honored

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Wido den Hollander <wido@widodh.nl>
6 years agoMerge pull request #28011 from ricardoasmarques/wip-39616-nautilus
Yuri Weinstein [Wed, 29 May 2019 21:54:56 +0000 (14:54 -0700)]
Merge pull request #28011 from ricardoasmarques/wip-39616-nautilus

nautilus: mgr/dashboard: iSCSI should allow exporting an RBD image with Journaling enabled

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge pull request #28026 from ricardoasmarques/wip-39631-nautilus
Yuri Weinstein [Wed, 29 May 2019 21:51:44 +0000 (14:51 -0700)]
Merge pull request #28026 from ricardoasmarques/wip-39631-nautilus

nautilus: mgr/dashboard: iSCSI form does not support IPv6

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
6 years agoMerge pull request #27920 from sebastian-philipp/wip-39536-nautilus
Nathan Cutler [Wed, 29 May 2019 10:16:42 +0000 (12:16 +0200)]
Merge pull request #27920 from sebastian-philipp/wip-39536-nautilus

nautilus: mgr/test_orchestrator: AttributeError: 'TestWriteCompletion' object has no attribute 'id'

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #28035 from pdvian/wip-39375-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:10:26 +0000 (09:10 -0700)]
Merge pull request #28035 from pdvian/wip-39375-nautilus

nautilus: osd/OSD.cc: make osd bench description consistent with parameters

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #28072 from pdvian/wip-39421-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:09:57 +0000 (09:09 -0700)]
Merge pull request #28072 from pdvian/wip-39421-nautilus

nautilus: osd/OSDMap: Replace get_out_osds with get_out_existing_osds

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #28088 from dzafman/wip-39721
Yuri Weinstein [Tue, 28 May 2019 16:09:23 +0000 (09:09 -0700)]
Merge pull request #28088 from dzafman/wip-39721

nautilus: osd: Include dups in copy_after() and copy_up_to()

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #28160 from pdvian/wip-39512-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:08:51 +0000 (09:08 -0700)]
Merge pull request #28160 from pdvian/wip-39512-nautilus

nautilus: osd/PG: do not use approx_missing_objects pre-nautilus

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #28164 from pdvian/wip-39514-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:08:17 +0000 (09:08 -0700)]
Merge pull request #28164 from pdvian/wip-39514-nautilus

nautilus: osd: take heartbeat_lock when calling heartbeat()

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #28205 from pdvian/wip-39519-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:07:47 +0000 (09:07 -0700)]
Merge pull request #28205 from pdvian/wip-39519-nautilus

nautilus: osd: Don't evict after a flush if intersecting scrub range

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #28219 from pdvian/wip-39539-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:07:15 +0000 (09:07 -0700)]
Merge pull request #28219 from pdvian/wip-39539-nautilus

nautilus: osd/PG: fix last_complete re-calculation on splitting

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27632 from pdvian/wip-39043-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:06:08 +0000 (09:06 -0700)]
Merge pull request #27632 from pdvian/wip-39043-nautilus

nautilus: osd/PGLog: preserve original_crt to check rollbackability

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
6 years agoMerge pull request #27699 from liewegas/wip-39335-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:05:00 +0000 (09:05 -0700)]
Merge pull request #27699 from liewegas/wip-39335-nautilus

nautilus: mgr/BaseMgrModule: run MonCommandCompletion on the finisher

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #27711 from pdvian/wip-39184-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:04:34 +0000 (09:04 -0700)]
Merge pull request #27711 from pdvian/wip-39184-nautilus

nautilus: osd: Primary won't automatically repair replica on pulling error

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #27744 from ashishkumsingh/wip-39432-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:04:00 +0000 (09:04 -0700)]
Merge pull request #27744 from ashishkumsingh/wip-39432-nautilus

nautilus: osd/PG: discover missing objects when an OSD peers and PG is degraded

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27771 from smithfarm/wip-39419-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:03:28 +0000 (09:03 -0700)]
Merge pull request #27771 from smithfarm/wip-39419-nautilus

nautilus: mon/Elector: format mon_release correctly

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27839 from pdvian/wip-39219-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:02:55 +0000 (09:02 -0700)]
Merge pull request #27839 from pdvian/wip-39219-nautilus

nautilus: osd: FAILED ceph_assert(attrs || !pg_log.get_missing().is_missing(soid) || (it_objects != pg_log.get_log().objects.end() && it_objects->second->op == pg_log_entry_t::LOST_REVERT)) in PrimaryLogPG::get_object_context()

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27876 from liewegas/wip-autoscaler-fixes-nautilus
Yuri Weinstein [Tue, 28 May 2019 16:02:28 +0000 (09:02 -0700)]
Merge pull request #27876 from liewegas/wip-autoscaler-fixes-nautilus

nautilus: mgr: fix pgp_num adjustments

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoos/bluestore: fix missing discard in BlueStore::_kv_sync_thread 28258/head
Tang Junhui [Mon, 29 Apr 2019 08:31:55 +0000 (16:31 +0800)]
os/bluestore: fix missing discard in BlueStore::_kv_sync_thread

Discard comand should be issued before space released.

Signed-off-by: Junhui Tang <tangjunhui@sangfor.com.cn>
Fixes: https://tracker.ceph.com/issues/39621
(cherry picked from commit f6205d2eb2b8297b914afbaf54a8311f087eec90)

6 years agoMerge pull request #28215 from smithfarm/wip-40003-nautilus
Nathan Cutler [Mon, 27 May 2019 13:54:57 +0000 (15:54 +0200)]
Merge pull request #28215 from smithfarm/wip-40003-nautilus

nautilus: do_cmake.sh: source not found

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge pull request #28062 from bk201/wip-39664-nautilus
Lenz Grimmer [Mon, 27 May 2019 13:01:08 +0000 (15:01 +0200)]
Merge pull request #28062 from bk201/wip-39664-nautilus

nautilus: mgr/dashboard: incorrect help message for minimum blob size

Reviewed-by: Laura Paduano <lpaduano@suse.com>
6 years agoosd/PG: fix last_complete re-calculation on splitting 28219/head
xie xingguo [Sat, 20 Apr 2019 08:34:12 +0000 (16:34 +0800)]
osd/PG: fix last_complete re-calculation on splitting

We add hard-limit for pg_logs now, which means we might keep trimming
old log entries irrespective of pg's current missing_set. This as a
result can cause the last_complete pointer moving far ahead of the real
on-disk version (the oldest need of missing_set, for instance) the
corresponding pg should have on splitting:

```
2019-04-19 06:41:52.559247 7efd4725c700 10 osd.2 271 Splitting pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 les/251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 unknown NOTIFY m=518 mbc={}] into 5.16
2019-04-19 06:41:52.561413 7efd4725c700 10 osd.2 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 c/f 251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 lcod 0'0 unknown NOTIFY m=261 mbc={}] release_backoffs [MIN,MAX)
```

For the above example, parent's last_complete cursor changed from **0'0** to
**238'300** directly due to the effort of trying to catch up the oldest
log entry changing when splitting was done. However, back into v12.2.9 primary
would still reference shard's last_complete field when trying to figure out all
possible locations of a currently missing object (see PG::MissingLoc::add_source_info):

```c++
  if (oinfo.last_complete < need) {
    if (omissing.is_missing(soid)) {
      ldout(pg->cct, 10) << "search_for_missing " << soid << " " << need
                         << " also missing on osd." << fromosd << dendl;
      continue;
    }
  }
```

Hence a wrongly calculated last_complete could then make primary mis-consider
that a specific shard might have the authoritative object it currently
looking for:

```
2019-04-19 06:41:52.904163 7fd4cfb5a700 10 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}] proc_replica_log for osd.2: 5.6( v 270'943 lc 238'30
0 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/c/f 251/224/0 250/271/229) log((249'563,270'943], crt=270'943) missing(261 may_include_del
etes = 1)
2019-04-19 06:41:52.904645 7fd4cfb5a700 20 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}]  after missing 5:624c3a7a:::benchmark_data_smithi190
_39968_object1382:head need 226'110 have 0'0
2019-04-19 06:41:53.567820 7fd4d035b700 10 osd.5 pg_epoch: 272 pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=271/272 n=471 ec=223/223 lis/c 250/223 les/c/f
251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 unknown m=16 u=13 mbc={255={(1+0)=220,(2+0)=28}}] search_for_missing 5:624c3a
7a:::benchmark_data_smithi190_39968_object1382:head 226'110 is on osd.2
```

note that ```5:624c3a7a:::benchmark_data_smithi190_39968_object1382:head 226'110```
was actually missing on both primary and shard osd.2 whereas primary insisted that
object should exist on shard osd.2!

https://github.com/ceph/ceph/pull/26175 posted an indirect fix
for the above problem by ignoring last_complete when checking the missing set,
but it should generally make more sense to fill in the last_complete field correctly
whenever possible.
Hence coming this additional fix.

Fixes: http://tracker.ceph.com/issues/26958
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit aad5d47be64ed7feba79f540ec1debc45625a74f)

6 years agoMerge pull request #27937 from pdvian/wip-39288-nautilus
Yuri Weinstein [Wed, 22 May 2019 19:49:12 +0000 (12:49 -0700)]
Merge pull request #27937 from pdvian/wip-39288-nautilus

nautilus: [rbd-mirror] image replayer should periodically flush IO and commit positions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27989 from smithfarm/wip-39590-nautilus
Nathan Cutler [Wed, 22 May 2019 16:15:59 +0000 (18:15 +0200)]
Merge pull request #27989 from smithfarm/wip-39590-nautilus

nautilus: qa/tasks/rbd_fio: fixed missing delimiter between 'cd' and 'configure'

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27882 from pdvian/wip-39224-nautilus
Yuri Weinstein [Wed, 22 May 2019 16:15:09 +0000 (09:15 -0700)]
Merge pull request #27882 from pdvian/wip-39224-nautilus

nautlius: librbd: do not allow to deep copy migrating image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27903 from pdvian/wip-39226-nautilus
Yuri Weinstein [Wed, 22 May 2019 16:14:39 +0000 (09:14 -0700)]
Merge pull request #27903 from pdvian/wip-39226-nautilus

nautilus: librbd: support EC data pool images sparsify

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #27988 from smithfarm/wip-39541-nautilus
Yuri Weinstein [Wed, 22 May 2019 16:13:25 +0000 (09:13 -0700)]
Merge pull request #27988 from smithfarm/wip-39541-nautilus

nautilus: tests: rbd: qemu-iotests tests fail under latest Ubuntu kernel

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #28019 from pdvian/wip-39315-nautilus
Yuri Weinstein [Wed, 22 May 2019 16:11:32 +0000 (09:11 -0700)]
Merge pull request #28019 from pdvian/wip-39315-nautilus

nautilus: krbd: fix rbd map hang due to udev return subsystem unordered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #28128 from dillaman/wip-39729-nautilus
Yuri Weinstein [Wed, 22 May 2019 16:10:56 +0000 (09:10 -0700)]
Merge pull request #28128 from dillaman/wip-39729-nautilus

nautilus: qa/workunits/rbd: use https protocol for devstack git operations

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28132 from dillaman/wip-39195-nautilus
Yuri Weinstein [Wed, 22 May 2019 16:10:18 +0000 (09:10 -0700)]
Merge pull request #28132 from dillaman/wip-39195-nautilus

nautilus: librbd: fixed several race conditions related to copyup

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agodo_cmake.sh: use bash 28215/head
Nathan Cutler [Mon, 20 May 2019 15:01:40 +0000 (17:01 +0200)]
do_cmake.sh: use bash

Fixes: http://tracker.ceph.com/issues/39981
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit ac8d7c5fd4045d79ab6caacc0b72d64866fcc5c8)

6 years agoosd: Don't evict after a flush if intersecting scrub range 28205/head
David Zafman [Tue, 26 Mar 2019 22:53:10 +0000 (15:53 -0700)]
osd: Don't evict after a flush if intersecting scrub range

Fixes: http://tracker.ceph.com/issues/38840
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 459cbb5a6ca3b521b20f36f328f25f398d0ef1c4)

6 years agoosd: Don't include user changeable flag in snaptrim related assert 28203/head
David Zafman [Tue, 30 Apr 2019 03:20:18 +0000 (20:20 -0700)]
osd: Don't include user changeable flag in snaptrim related assert

Caused by: a53ba7314c53e75d1e0b8a0edd29181db3c93863

Fixes: http://tracker.ceph.com/issues/38124
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 63c2060c2d9eef49dacf2ba240ccfbead9791f43)

6 years agoMerge pull request #27752 from liewegas/wip-devices-doc-nautilus
Nathan Cutler [Tue, 21 May 2019 08:11:02 +0000 (10:11 +0200)]
Merge pull request #27752 from liewegas/wip-devices-doc-nautilus

nautilus: doc/rados/operations/devices: document device prediction

Reviewed-by: Sébastien Han <seb@redhat.com>
6 years agoMerge pull request #27488 from sebastian-philipp/wip-39169-nautilus
Nathan Cutler [Mon, 20 May 2019 12:31:48 +0000 (14:31 +0200)]
Merge pull request #27488 from sebastian-philipp/wip-39169-nautilus

nautilus: doc/mgr/orchestrator_cli: Rook orch supports mon update

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #28163 from smithfarm/wip-39962-nautilus
Lenz Grimmer [Mon, 20 May 2019 10:56:19 +0000 (12:56 +0200)]
Merge pull request #28163 from smithfarm/wip-39962-nautilus

nautilus: mgr/dashboard: openssl exception when verifying certificates of HTTPS requests

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoosd: take heartbeat_lock when calling heartbeat() 28164/head
Sage Weil [Tue, 23 Apr 2019 18:03:42 +0000 (13:03 -0500)]
osd: take heartbeat_lock when calling heartbeat()

Fixes: http://tracker.ceph.com/issues/39439
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7863d7c6c49347db97a6bca36f63fc35741ac0e5)

6 years agosystemd: ceph-mgr: set MemoryDenyWriteExecute to false 28163/head
Ricardo Dias [Wed, 8 May 2019 13:57:07 +0000 (14:57 +0100)]
systemd: ceph-mgr: set MemoryDenyWriteExecute to false

Fixes: http://tracker.ceph.com/issues/39628
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 1d7506fdce4924fb30bbabc65e26dafa06aab24f)

6 years agoosd/PG: do not use approx_missing_objects pre-nautilus 28160/head
Neha Ojha [Thu, 25 Apr 2019 02:15:27 +0000 (19:15 -0700)]
osd/PG: do not use approx_missing_objects pre-nautilus

We changed async recovery cost calculation in nautilus to also take
into account approx_missing_objects in ab241bf7e927cda2d0ed1698383d18dc4a4b601c
This commit depends on https://github.com/ceph/ceph/pull/23663, hence
wasn't backported to mimic.

Mimic only uses the difference in length of logs as the cost. Due to this,
the same OSD might have different costs in a mixed mimic and nautilus(or above)
cluster. This can lead to choose_acting() cycling between OSDs, when trying
to select the acting set and async_recovery_targets.

Fixes: https://tracker.ceph.com/issues/39441
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4c617ecf1cf6f25cca42e5d59fa162dbc60f4de8)

Conflicts:
src/osd/PG.cc : Resolved in choose_async_recovery_ec and
choose_async_recovery_replicated

6 years agolibrbd: copyup read stats were incorrectly tied to child 28132/head
Jason Dillaman [Wed, 24 Apr 2019 15:37:41 +0000 (11:37 -0400)]
librbd: copyup read stats were incorrectly tied to child

The parent IO stats should be updated when reading from the parent
image for a copyup.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9b615c30753ad35636c875696ed951e807fd694c)

6 years agolibrbd: force a copyup object map update if migration completes
Jason Dillaman [Mon, 8 Apr 2019 17:42:16 +0000 (13:42 -0400)]
librbd: force a copyup object map update if migration completes

If a live-migration copyup was in-progress when the copyup completes,
force the object map to be updated.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e18e413b602208b38f432f4ba5bcb187fe102cb1)

6 years agolibrbd: cleaned up snap/parent lock release in read-from-parent path
Jason Dillaman [Thu, 4 Apr 2019 15:23:58 +0000 (11:23 -0400)]
librbd: cleaned up snap/parent lock release in read-from-parent path

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b7f6ba710d2855ea8f223d205a55a1e425167afd)

6 years agotest/librbd: initial unit test for copyup state machine
Jason Dillaman [Tue, 2 Apr 2019 18:10:42 +0000 (14:10 -0400)]
test/librbd: initial unit test for copyup state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ea4901f539c7404c07e72f5c42fc8abcf840dafa)

6 years agolibrbd: copyup should restart delayed ops against the same object
Jason Dillaman [Thu, 4 Apr 2019 01:35:40 +0000 (21:35 -0400)]
librbd: copyup should restart delayed ops against the same object

This avoids the potential for a race condition where an in-flight
copyup is removed from the in-flight copyup list and a subsequent
IO against the same object causes a second in-flight copyup.

Fixes: http://tracker.ceph.com/issues/39021
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 31ed673a7484c8067dd928cabd5bcb923a35dbaf)

6 years agolibrbd: merge copyup object map update states
Jason Dillaman [Tue, 2 Apr 2019 15:51:36 +0000 (11:51 -0400)]
librbd: merge copyup object map update states

The object map HEAD and HEAD/snapshot update states have been
simplified and merged into a single state. This also fixes
several potential race conditions and an issue where CoR might
incorrectly mark the HEAD object has exists+dirty.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5b90dbe0ae4be3dccfc3fbc0c8d5eb1ed7bb3168)

6 years agolibrbd: refactor copyup state machine to match current code style
Jason Dillaman [Fri, 29 Mar 2019 18:17:15 +0000 (14:17 -0400)]
librbd: refactor copyup state machine to match current code style

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 711a070fa086b499eeba9a4c62005456f406689b)

6 years agolibrbd: properly hold snap/parent locks during IO
Jason Dillaman [Fri, 29 Mar 2019 17:23:37 +0000 (13:23 -0400)]
librbd: properly hold snap/parent locks during IO

The ImageCtx::parent pointer was dereferenced without holding the lock
which could lead to a crash. The ImageCtx::migration_info structure
was also dereferenced without holding a lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6dad1acee3f0a4b4b3c88e25c0cfafe21bc066c4)

6 years agolibrbd: empty IO requests should complete in new context
Jason Dillaman [Tue, 2 Apr 2019 19:33:00 +0000 (15:33 -0400)]
librbd: empty IO requests should complete in new context

This will avoid the possibility of a lock being held by the caller
and having the IO immediately complete back to the calling state
machine.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c9ff6cf1945b248b38e83484aa4a22c206da846b)

6 years agolibrbd: deep-copy object copy should register an in-flight op
Jason Dillaman [Mon, 1 Apr 2019 18:48:47 +0000 (14:48 -0400)]
librbd: deep-copy object copy should register an in-flight op

When handling live migrations, the source image is the parent
image of the destination image. To prevent the parent image from
being closed while a request is in-flight, the object copy
state machine now registers an async operation with the source
image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 89cfa6c8d328b9294c7f7efd53d82edcec59e66d)

6 years agolibrbd: deep-copy object copy should delay computing object existence
Jason Dillaman [Fri, 29 Mar 2019 20:33:21 +0000 (16:33 -0400)]
librbd: deep-copy object copy should delay computing object existence

This prevent a potential recursive lock on the destination image's
snap_lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0c37066797737cd7c677b2649dae1e6422869afb)

6 years agoqa/workunits/rbd: use https protocol for devstack git operations 28128/head
Jason Dillaman [Thu, 9 May 2019 19:48:30 +0000 (15:48 -0400)]
qa/workunits/rbd: use https protocol for devstack git operations

Fixes: http://tracker.ceph.com/issues/39656
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fb4f9a8a08edd18b1a23e1be4c1285d0ec4d1de6)

6 years agorbd-mirror: clear out bufferlist prior to listing mirror images 28122/head
Jason Dillaman [Tue, 23 Apr 2019 12:24:52 +0000 (08:24 -0400)]
rbd-mirror: clear out bufferlist prior to listing mirror images

The second call to list mirrored images will fail deep within the
msgr code due to a "bad crc in data" error.

Fixes: http://tracker.ceph.com/issues/39407
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7c5dcf63f3b15692fd3f5a19c1562b195cd9ec47)

6 years agorbd: fixed memory leak in 'rbd mirror pool' multi-image actions 28121/head
Jason Dillaman [Thu, 18 Apr 2019 17:20:17 +0000 (13:20 -0400)]
rbd: fixed memory leak in 'rbd mirror pool' multi-image actions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ba52f4297f4ad96e8b4db5d24e2e73e4b303fda9)

6 years agolibrbd: async open/close should free ImageCtx before issuing callback
Jason Dillaman [Thu, 18 Apr 2019 17:17:56 +0000 (13:17 -0400)]
librbd: async open/close should free ImageCtx before issuing callback

The destructor for ImageCtx attempts to access librados::IoCtx objects
that might have been destroyed immediately after the async open (failure)
and close actions completed.

Fixes: http://tracker.ceph.com/issues/39031
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit be419a1d6fd1e176ab4fdebfaa17726bb63d6ac8)

6 years agolibrbd: when requeueing QOS throttled request queue it back 28120/head
Mykola Golub [Fri, 15 Mar 2019 15:17:42 +0000 (15:17 +0000)]
librbd: when requeueing QOS throttled request queue it back

to preserve order of throttled requests, which is important for flush.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 633a167ff04b1f9af6acde41f279c525b3ce87f3)

6 years agocommon: add "requeue_back" WorkQueue method
Mykola Golub [Fri, 15 Mar 2019 15:01:26 +0000 (15:01 +0000)]
common: add "requeue_back" WorkQueue method

and rename "requeue" method to "requeue_front" for clarity.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 060ef69eb4ce8473ea5ecc796027729dff883269)

6 years agolibrbd: make flush be queued by QOS throttler
Mykola Golub [Thu, 14 Mar 2019 18:46:23 +0000 (18:46 +0000)]
librbd: make flush be queued by QOS throttler

So when it is eventually processed we sure there is no pending
requests in the throttler queue it needs to wait to complete.

Fixes: https://tracker.ceph.com/issues/38706
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit a30276eadbb29d36baf5532327786da720856db8)

6 years agocommon: allow "0" cost items to be queued by TokenBucketThrottle
Mykola Golub [Thu, 14 Mar 2019 18:30:31 +0000 (18:30 +0000)]
common: allow "0" cost items to be queued by TokenBucketThrottle

They will be queued if there are already blockers.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 4f7780a92c482562974923360e6b2d11bd8ffa58)

6 years agoMerge pull request #27740 from ifed01/wip-ifed-fix-39446
Yuri Weinstein [Wed, 15 May 2019 16:05:06 +0000 (09:05 -0700)]
Merge pull request #27740 from ifed01/wip-ifed-fix-39446

nautilus: os/bluestore: fix out-of-bound access in bmap allocator.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agotest/librbd: drop 'ceph_test_librbd_api' target 28091/head
Jason Dillaman [Fri, 19 Apr 2019 18:15:49 +0000 (14:15 -0400)]
test/librbd: drop 'ceph_test_librbd_api' target

This was used for API backwards compatibility testing, but now that
the C++ API will not remain stable, it serves no purpose.

Fixes: http://tracker.ceph.com/issues/39072
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5b0a611a48bffb4d11d5a1cda21c5eaefb3576fb)

6 years agoosd: Fix merge_log_dups() not remove dup corresponding to log.tail 28088/head
David Zafman [Thu, 9 May 2019 17:38:36 +0000 (10:38 -0700)]
osd: Fix merge_log_dups() not remove dup corresponding to log.tail

The log tail is not inclusive of the log, so not an overlap
Fix unittest_pglog to verify proper operation

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f526c6d55c8866712c10b64491fe9371f64cf4d5)

6 years agotest: unittest_pglog: Additional test cases for trim, copy_up_to and copy_after
David Zafman [Tue, 7 May 2019 04:19:59 +0000 (21:19 -0700)]
test: unittest_pglog: Additional test cases for trim, copy_up_to and copy_after

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d0d244d222c7709a4a0688954816fd42e471b3c9)

6 years agotest: unittest_pglog: Remove irrelevant config values
David Zafman [Tue, 7 May 2019 02:57:55 +0000 (19:57 -0700)]
test: unittest_pglog: Remove irrelevant config values

osd_min_pg_log_entries and osd_max_pg_log_entries are not involved
in the log handling code being tested.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a008bd8c18ee99f24db41c8e2044cd49c175f4b9)

6 years agotest: Adding standalone test of log copy handling
David Zafman [Fri, 3 May 2019 20:33:39 +0000 (13:33 -0700)]
test: Adding standalone test of log copy handling

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 7959159e830b39c69807c832bf3e01581a314d0a)

6 years agoosd: Include dups in copy_after() and copy_up_to()
David Zafman [Thu, 2 May 2019 02:47:55 +0000 (19:47 -0700)]
osd: Include dups in copy_after() and copy_up_to()

Client saw out of order results in a test with a very small
pg_log because a backfill/recovery sent a small number of
log entries for duplicate checks.

Since these copy operations are equivalent to both a copy
and trim (an argument controls how many log entries
transfer), we need to include any new dups with enough
existing dups up to the configured maximum.

Fixes: http://tracker.ceph.com/issues/39304
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 00f279db37fc648584c4c741e0453b1d4f4320e4)

Conflicts:
src/osd/PeeringState.cc contents in src/osd/PG.cc
src/osd/osd_types.cc (trivial)

6 years agoosd: Fix off by 1 error in trim handling of osd_pg_log_dups_tracked
David Zafman [Thu, 2 May 2019 02:44:04 +0000 (19:44 -0700)]
osd: Fix off by 1 error in trim handling of osd_pg_log_dups_tracked

Move log message out of loop so it doesn't repeat for no reason

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b698e34fc8e931623671b501855099050859a18a)

6 years agoosd: Remove unused pg_log_t::copy_range()
David Zafman [Tue, 30 Apr 2019 23:09:47 +0000 (16:09 -0700)]
osd: Remove unused pg_log_t::copy_range()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d1ad129dd4b71640ae2e74d793e45146dca8bf26)

6 years agoosd/OSDMap: Replace get_out_osds with get_out_existing_osds 28072/head
Brad Hubbard [Wed, 17 Apr 2019 06:13:27 +0000 (16:13 +1000)]
osd/OSDMap: Replace get_out_osds with get_out_existing_osds

Fixes: http://tracker.ceph.com/issues/39154
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit adfb6a595363e2c085c6eac3001fcaa2dce007aa)

Conflicts:
src/osd/OSDMap.h : Resolved for get_out_existing_osds

6 years agoMerge pull request #27803 from pdvian/wip-39205-nautilus
Yuri Weinstein [Sat, 11 May 2019 15:43:00 +0000 (08:43 -0700)]
Merge pull request #27803 from pdvian/wip-39205-nautilus

nautilus: osd: shutdown recovery_request_timer earlier

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #28065 from yuriw/wip-yuriw-fix-distros-nautilus
Yuri Weinstein [Fri, 10 May 2019 19:06:49 +0000 (12:06 -0700)]
Merge pull request #28065 from yuriw/wip-yuriw-fix-distros-nautilus

qa/tests: cleaned up supported distro for nautilus

6 years agoqa/tests: cleaned up supported disto for nautilus 28065/head
Yuri Weinstein [Fri, 10 May 2019 16:10:08 +0000 (09:10 -0700)]
qa/tests: cleaned up supported disto for nautilus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #27608 from liewegas/wip-38157-nautilus
Yuri Weinstein [Fri, 10 May 2019 15:20:46 +0000 (08:20 -0700)]
Merge pull request #27608 from liewegas/wip-38157-nautilus

nautilus: mgr/progress: behave if pgs disappear (due to a racing pg merge)

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge pull request #27654 from neha-ojha/wip-38881-nautilus
Yuri Weinstein [Fri, 10 May 2019 15:19:38 +0000 (08:19 -0700)]
Merge pull request #27654 from neha-ojha/wip-38881-nautilus

nautilus: osd/PG: skip rollforward when !transaction_applied during append_log()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #27854 from smithfarm/wip-39504-nautilus
Yuri Weinstein [Fri, 10 May 2019 15:18:47 +0000 (08:18 -0700)]
Merge pull request #27854 from smithfarm/wip-39504-nautilus

nautilus: core: Give recovery for inactive PGs a higher priority

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #27857 from ashishkumsingh/wip-39465-nautilus
Yuri Weinstein [Fri, 10 May 2019 15:18:13 +0000 (08:18 -0700)]
Merge pull request #27857 from ashishkumsingh/wip-39465-nautilus

nautilus: msg: output peer address when detecting bad CRCs

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #27939 from pdvian/wip-39308-nautilus
Yuri Weinstein [Fri, 10 May 2019 15:17:32 +0000 (08:17 -0700)]
Merge pull request #27939 from pdvian/wip-39308-nautilus

nautilus: mgr/ActivePyModules: handle_command - fix broken lock

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agomgr/dashboard: fix help message for minimum blob size in pool form 28062/head
Kiefer Chang [Wed, 8 May 2019 04:15:43 +0000 (12:15 +0800)]
mgr/dashboard: fix help message for minimum blob size in pool form

A value less than maximum blob size is required.

Fixes: http://tracker.ceph.com/issues/39624
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit c8c4ed692a2b6ac8bf2aa9337ed7d683187ff882)

6 years agoMerge pull request #27642 from LenzGr/nautilus-documentation
Ernesto Puerta [Fri, 10 May 2019 13:33:06 +0000 (15:33 +0200)]
Merge pull request #27642 from LenzGr/nautilus-documentation

nautilus: doc: Backported dashboard documentation changes

6 years agoMerge pull request #27936 from dillaman/wip-39450-nautilus
Yuri Weinstein [Thu, 9 May 2019 17:07:44 +0000 (10:07 -0700)]
Merge pull request #27936 from dillaman/wip-39450-nautilus

nautilus: librbd: re-add support for nautilus clients talking to jewel clusters

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28046 from neha-ojha/wip-28036-nautilus
Neha Ojha [Thu, 9 May 2019 15:26:33 +0000 (08:26 -0700)]
Merge pull request #28046 from neha-ojha/wip-28036-nautilus

nautilus: doc/dev/osd_internals/async_recovery: update cost calculation

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agodoc/dev/osd_internals/async_recovery: update cost calculation 28046/head
Neha Ojha [Thu, 9 May 2019 03:19:32 +0000 (20:19 -0700)]
doc/dev/osd_internals/async_recovery: update cost calculation

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 984073e44d096a3322d90b85da42c593fff94c6d)

6 years agoMerge pull request #28024 from ricardoasmarques/wip-39630-nautilus
Lenz Grimmer [Thu, 9 May 2019 10:33:00 +0000 (12:33 +0200)]
Merge pull request #28024 from ricardoasmarques/wip-39630-nautilus

nautilus: mgr/dashboard: iSCSI GET requests should not be logged

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agoMerge pull request #27942 from LenzGr/wip-39524-nautilus
Lenz Grimmer [Thu, 9 May 2019 07:52:56 +0000 (09:52 +0200)]
Merge pull request #27942 from LenzGr/wip-39524-nautilus

nautilus: mgr/dashboard: make auth token work with UTC times only

6 years agoosd/OSD.cc: make osd bench description consistent with parameters 28035/head
Neha Ojha [Tue, 16 Apr 2019 01:47:34 +0000 (18:47 -0700)]
osd/OSD.cc: make osd bench description consistent with parameters

Fixes: http://tracker.ceph.com/issues/39006
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4b240cf86b1e739eb0c255e2f7f1018ea2bc97aa)

6 years agomgr/dashboard: Fix iSCSI form when using IPv6 28026/head
Ricardo Marques [Fri, 3 May 2019 10:23:12 +0000 (11:23 +0100)]
mgr/dashboard: Fix iSCSI form when using IPv6

Fixes: https://tracker.ceph.com/issues/39578
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit f243f7a53170dd84a49348eb06fc566d163c6a47)

6 years agomgr/dashboard: iSCSI GET requests should not be logged 28024/head
Ricardo Marques [Fri, 26 Apr 2019 10:13:20 +0000 (11:13 +0100)]
mgr/dashboard: iSCSI GET requests should not be logged

Fixes: https://tracker.ceph.com/issues/39492
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 93d74cf9b9926c1638f15da6c8e6d3fd68d61831)

6 years agomgr/dashboard: KV-table transforms dates through pipe 28021/head
Stephan Müller [Tue, 16 Apr 2019 13:48:12 +0000 (15:48 +0200)]
mgr/dashboard: KV-table transforms dates through pipe

The key value table that is mostly used for showing details of objects,
will now transform any date with 'cdDatePipe'.

Through this the start and end date of an alert will be shown in local
time instead of UTC time.

Fixes: https://tracker.ceph.com/issues/39296
Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit 3a2732ab6489e8c84d53911f32607e29e9c5eac7)

6 years agoMerge pull request #27758 from ricardoasmarques/wip-39452-nautilus
Lenz Grimmer [Wed, 8 May 2019 11:55:54 +0000 (13:55 +0200)]
Merge pull request #27758 from ricardoasmarques/wip-39452-nautilus

nautilus: mgr/dashboard: iSCSI form is showing a warning

Reviewed-by: Tiago Melo <tmelo@suse.com>