]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agocommon: fix typo in rados bench write JSON output 22407/head
Sandor Zeestraten [Mon, 21 May 2018 08:26:53 +0000 (10:26 +0200)]
common: fix typo in rados bench write JSON output

Fixes typo in `max_latency` key in JSON output from rados bench write.

Fixes: http://tracker.ceph.com/issues/24199
Signed-off-by: Sandor Zeestraten <sandor@zeestrataca.com>
(cherry picked from commit 1491e08e422d6ad5e612205accd0134203cdc563)

Conflicts:
src/common/obj_bencher.cc : Resolved in write_bench

7 years agoMerge pull request #20877 from theanalyst/jewel-23338
Yuri Weinstein [Fri, 25 May 2018 15:11:55 +0000 (08:11 -0700)]
Merge pull request #20877 from theanalyst/jewel-23338

jewel: rgw: radosgw-admin: add an option to reset user stats

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #21212 from smithfarm/wip-22939-jewel
Yuri Weinstein [Fri, 25 May 2018 15:11:15 +0000 (08:11 -0700)]
Merge pull request #21212 from smithfarm/wip-22939-jewel

jewel: rgw: system user can't delete bucket completely

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #21294 from joke-lee/jewel
Yuri Weinstein [Fri, 25 May 2018 15:10:45 +0000 (08:10 -0700)]
Merge pull request #21294 from joke-lee/jewel

jewel: rgw: copying part without http header x-amz-copy-source-range will be mistaken for copying object

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #17818 from smithfarm/wip-upgrade-rh-74
Yuri Weinstein [Thu, 24 May 2018 20:42:10 +0000 (13:42 -0700)]
Merge pull request #17818 from smithfarm/wip-upgrade-rh-74

jewel: tests: make upgrade tests use supported distros

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21194 from dzafman/wip-23267-jewel
Yuri Weinstein [Thu, 24 May 2018 20:40:44 +0000 (13:40 -0700)]
Merge pull request #21194 from dzafman/wip-23267-jewel

jewel: scrub errors not cleared on replicas can cause inconsistent pg state when replica takes over primary

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #21747 from vumrao/wip-vumrao-rhbz1568897
Yuri Weinstein [Thu, 24 May 2018 20:39:43 +0000 (13:39 -0700)]
Merge pull request #21747 from vumrao/wip-vumrao-rhbz1568897

jewel: OSD: print log when osd want to kill self

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22050 from tchaikov/jewel-22530
Yuri Weinstein [Thu, 24 May 2018 20:37:54 +0000 (13:37 -0700)]
Merge pull request #22050 from tchaikov/jewel-22530

jewel: mon/OSDMonitor.cc: fix expected_num_objects interpret error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22188 from tchaikov/jewel-23966
Yuri Weinstein [Thu, 24 May 2018 20:37:03 +0000 (13:37 -0700)]
Merge pull request #22188 from tchaikov/jewel-23966

jewel: Deleting a pool with active notify linger ops can result in seg fault

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22189 from theanalyst/wip-jewel-24244
Yuri Weinstein [Thu, 24 May 2018 20:35:57 +0000 (13:35 -0700)]
Merge pull request #22189 from theanalyst/wip-jewel-24244

mon/MDSMonitor: no_reply on MMDSLoadTargets

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21162 from smithfarm/wip-22378-jewel
Yuri Weinstein [Thu, 24 May 2018 20:28:48 +0000 (13:28 -0700)]
Merge pull request #21162 from smithfarm/wip-22378-jewel

jewel: ceph-fuse: failure to remount in startup test does not handle client_die_on_failed_remount properly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21734 from joscollin/wip-23932-jewel
Yuri Weinstein [Thu, 24 May 2018 20:27:59 +0000 (13:27 -0700)]
Merge pull request #21734 from joscollin/wip-23932-jewel

jewel: client: avoid second lock on client_lock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agomon/MDSMonitor: no_reply on MMDSLoadTargets 22189/head
Sage Weil [Wed, 2 May 2018 19:48:31 +0000 (14:48 -0500)]
mon/MDSMonitor: no_reply on MMDSLoadTargets

If we don't note that we don't reply then we don't close out the routed
mon request and the op will appear as slow on the forwarding mon.

Fixes: http://tracker.ceph.com/issues/23769
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b462b59065424520170956581e72e16481b16f0a)

7 years agolibrados: fix potential race condition if notify immediately fails 22188/head
Jason Dillaman [Mon, 7 May 2018 17:23:43 +0000 (10:23 -0700)]
librados: fix potential race condition if notify immediately fails

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cc06fbab08ef5b7f1b18f2af333c019ad86de03d)

7 years agoosdc: guard op->on_notify_finish with lock
Kefu Chai [Sun, 6 May 2018 03:11:16 +0000 (11:11 +0800)]
osdc: guard op->on_notify_finish with lock

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c16371965cc785f2e2b87d8c6684854fa2de65e1)

7 years agoosdc: invoke notify finish context on linger commit failure
Jason Dillaman [Wed, 2 May 2018 16:52:42 +0000 (12:52 -0400)]
osdc: invoke notify finish context on linger commit failure

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fcd414f6324136ce0695fcf30a60eec6ff338a98)

7 years agoMerge pull request #21503 from smithfarm/wip-23783-jewel
Kefu Chai [Wed, 23 May 2018 09:43:26 +0000 (17:43 +0800)]
Merge pull request #21503 from smithfarm/wip-23783-jewel

jewel: table of contents doesn't render for luminous/jewel docs

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22166 from tchaikov/jewel-javah
Kefu Chai [Wed, 23 May 2018 09:06:42 +0000 (17:06 +0800)]
Merge pull request #22166 from tchaikov/jewel-javah

jewel: makefile: use javac for creating JNI headers

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agomakefile: bump java source option from 5 to 6 22166/head
Kefu Chai [Wed, 23 May 2018 02:48:44 +0000 (10:48 +0800)]
makefile: bump java source option from 5 to 6

this kills the errors like

error: Source option 5 is no longer supported. Use 6 or later.
error: Target option 1.5 is no longer supported. Use 1.6 or later.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
    src/java/Makefile.am: this change is not cherry-picked from
master. as we've moved to java 8 in master. see df27ed3a .

7 years agomakefile: use javac for creating JNI headers
Kefu Chai [Wed, 23 May 2018 02:45:39 +0000 (10:45 +0800)]
makefile: use javac for creating JNI headers

JDK 1.10 does not offer javah anymore, so we need to use "javac -h"
instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit edc504b9d446cc1739f3e8e087914cfe66d6cac3)
Conflicts: configure.ac:
           src/java/Makefile.am: we don't use cmake back in jewel, so
port the cmake changes to autotools.

7 years agoMerge pull request #22130 from ceph/wip-yuriw-clients-fix-jewel
Josh Durgin [Mon, 21 May 2018 22:54:27 +0000 (15:54 -0700)]
Merge pull request #22130 from ceph/wip-yuriw-clients-fix-jewel

qa/test: added supported distro

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoqa/test: added supported ditro 22090/head 22130/head
Yuri Weinstein [Fri, 18 May 2018 15:39:45 +0000 (08:39 -0700)]
qa/test: added supported ditro

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #21210 from pdvian/wip-23303-jewel
Nathan Cutler [Fri, 18 May 2018 11:01:32 +0000 (13:01 +0200)]
Merge pull request #21210 from pdvian/wip-23303-jewel

jewel: rgw: add radosgw-admin sync error trim to trim sync error log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agomon/OSDMonitor.cc: fix expected_num_objects interpret error 22050/head
Yang Honggang [Mon, 22 Jan 2018 02:00:17 +0000 (21:00 -0500)]
mon/OSDMonitor.cc: fix expected_num_objects interpret error

Fixes: http://tracker.ceph.com/issues/22530
Signed-off-by: Yang Honggang <joseph.yang@xtaotech.com>
(cherry picked from commit c24f2baec968ff66e792991d829591505ffba3ae)

Conflicts:
qa/standalone/mon/osd-pool-create.sh: jewel does not have
qa/standalone tests by then. and in jewel, "ruleset" is not renamed to
"rule" yet.
src/mon/OSDMonitor.cc
src/pybind/mgr/localpool/module.py: trivial resolution

7 years agoMerge pull request #21190 from smithfarm/wip-21033-jewel
Nathan Cutler [Thu, 17 May 2018 09:19:03 +0000 (11:19 +0200)]
Merge pull request #21190 from smithfarm/wip-21033-jewel

jewel: build/ops: rpm: bump epoch ahead of ceph-common in RHEL base

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoMerge pull request #21213 from smithfarm/wip-23026-jewel
Nathan Cutler [Thu, 17 May 2018 09:16:19 +0000 (11:16 +0200)]
Merge pull request #21213 from smithfarm/wip-23026-jewel

jewel: rgw: data sync of versioned objects, note updating bi marker

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoOSD: print log when osd want to kill self 21747/head
Haomai Wang [Tue, 24 May 2016 11:47:32 +0000 (19:47 +0800)]
OSD: print log when osd want to kill self

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 9bc7d9667076473deda4a9214eb74955458eb36f)

7 years agoclient: avoid second lock on client_lock 21734/head
Supriti Singh [Fri, 20 Apr 2018 07:48:53 +0000 (09:48 +0200)]
client: avoid second lock on client_lock

Avoid a second nested lock on client lock. As a result, drop unused function ll_get_inodeno().
Fixes: https://tracker.ceph.com/issues/23815
Signed-off-by: Supriti Singh <supriti.singh@suse.com>
(cherry picked from commit 7be8d559a11507255f847ce6279d23abaabde70a)

 Conflicts:
src/client/Client.cc (dropped function call ll_get_inodeno(in))

7 years agorgw: rgw-rados, rgw-admin add an option to recalculate user stats 20877/head
Abhishek Lekshmanan [Mon, 12 Mar 2018 18:14:46 +0000 (19:14 +0100)]
rgw: rgw-rados, rgw-admin add an option to recalculate user stats

Adds a method in rgw-rados to reset user stats calling the earlier implemented
cls user reset stats.
In rgw-admin we add an option called --reset-stats that invokes this method.

Fixes: http://tracker.ceph.com/issues/23335
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 0b72dd7d3beea47b5c479eeb979b73c4cd70f7bc)

 Conflicts:
  src/rgw/rgw_admin.cc
  whitespace conflict
  src/rgw_rados.cc
  switch to jewel version of get_obj_ref (where raw version isn't available)

jewel only admin fixes

7 years agocls/user: implement reset user stats
Abhishek Lekshmanan [Mon, 12 Mar 2018 19:17:36 +0000 (20:17 +0100)]
cls/user: implement reset user stats

This is an implementation of reset user stats, that recalculates the user stats
purely based on the values of bucket entries in user.buckets object. This is
helpful in cases when user stats has been improperly set in case of manual
resharding etc.

Fixes: http://tracker.ceph.com/issues/23335
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 63f9127dd685890d938a612739320c942a96f910)

* conflicts:
cls_user.cc:
jewel only cls fixes, truncated is not a part of omap get vals, so iterating
over the range by checking the output of omap_get_vals and continuing.
cls_user_ops.h:
explicit ::encode/::decode

7 years agodoc update template call for toctree 21503/head
Alfredo Deza [Fri, 18 Aug 2017 16:05:03 +0000 (12:05 -0400)]
doc update template call for toctree

Up until version 1.1.3 of Sphinx, this call would show hidden toctree
entries. Most of Ceph's toctree's are hidden, so when the version of
Sphinx was bumped to >1.6, the sidebar stopped showing entries

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9bb3397a5876ded89853f11cd35b3ca5e8aeec5c)

7 years agoosd: Fix stale scrub stats when a primary takes over 21194/head
David Zafman [Wed, 28 Mar 2018 17:21:39 +0000 (10:21 -0700)]
osd: Fix stale scrub stats when a primary takes over

Fixes: http://tracker.ceph.com/issues/23267
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 5cfb8241f482ed53c63bb97262425b2acb733d7d)

Conflicts:
src/osd/PG.cc (trivial)
test/Makefile.am (Add to make check for Jewel)
test/osd/CMakeLists.txt (Add to make check for Jewel)
src/test/osd/osd-scrub-test.sh (from qa/standalone/scrub/osd-scrub-test.sh)
  Different location of ceph-helpers.sh
  No manager so remove run_mgr
  Remove default rbd pool
  Use 4 OSDs because remapped PG doesn't go clean

7 years agoclient: quit on failed remount during dentry invalidate test 21162/head
Patrick Donnelly [Tue, 28 Nov 2017 23:01:32 +0000 (15:01 -0800)]
client: quit on failed remount during dentry invalidate test

Fixes: http://tracker.ceph.com/issues/22269
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5b2b1d14468c290c56ee6c95ea557c99464e0098)

Conflicts:
PendingReleaseNotes - put the release note under 10.2.11
src/ceph_fuse.cc - changed order of client_try_dentry_invalidate and
    can_invalidate_dentries assignments to match master, and use jewel
            form of g_conf for accessing config option values
src/client/Client.cc - code being replaced by _do_remount() calls is
    different in jewel, and use jewel convention for accessing config
            option values
src/common/legacy_config_opts.h - file does not exist in jewel
src/common/options.cc - file does not exit in jewel; made the changes
            manually in src/common/config_opts.h

7 years agoMerge pull request #21185 from smithfarm/wip-23158-jewel
Nathan Cutler [Tue, 10 Apr 2018 07:07:22 +0000 (09:07 +0200)]
Merge pull request #21185 from smithfarm/wip-23158-jewel

jewel: mds: underwater dentry check in CDir::_omap_fetched is racy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #18010 from jdurgin/wip-rocksdb-jewel
Josh Durgin [Tue, 10 Apr 2018 05:29:35 +0000 (22:29 -0700)]
Merge pull request #18010 from jdurgin/wip-rocksdb-jewel

jewel: core: enable rocksdb for filestore

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #21199 from smithfarm/wip-22403-jewel
Josh Durgin [Tue, 10 Apr 2018 01:43:40 +0000 (18:43 -0700)]
Merge pull request #21199 from smithfarm/wip-22403-jewel

jewel: osd: replica read can trigger cache promotion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21200 from ovh/bp-jewel-backport-22448
Josh Durgin [Tue, 10 Apr 2018 01:40:53 +0000 (18:40 -0700)]
Merge pull request #21200 from ovh/bp-jewel-backport-22448

jewel: osd/PrimaryLogPG: dump snap_trimq size

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21197 from smithfarm/wip-23521-jewel
Josh Durgin [Mon, 9 Apr 2018 23:44:49 +0000 (16:44 -0700)]
Merge pull request #21197 from smithfarm/wip-23521-jewel

jewel: ceph_authtool: add mode option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20381 from pdvian/wip-22941-jewel
Josh Durgin [Mon, 9 Apr 2018 23:38:12 +0000 (16:38 -0700)]
Merge pull request #20381 from pdvian/wip-22941-jewel

jewel: core: librados: Double free in rados_getxattrs_next

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agorgw: If use 'copy part' without 'x-amz-copy-source-range', 21294/head
yuliyang [Mon, 9 Apr 2018 05:12:42 +0000 (13:12 +0800)]
rgw: If use 'copy part' without 'x-amz-copy-source-range',

it will copy an entire source object

Signed-off-by: Malcolm Lee <fengxueyu35@126.com>
(cherry picked from commit 753b3a7)

Conflicts:
 1. in original commit, copy_source is std::string, jewel
    version is const char *.
 2. in original commit, url_decode take only one param,
    jewel version take 2 params.
 3. in original commit, get_obj_state take 6 params, jewel
    version takes 4 params.

7 years agoMerge pull request #21203 from smithfarm/wip-22394-jewel
Nathan Cutler [Sun, 8 Apr 2018 20:03:00 +0000 (22:03 +0200)]
Merge pull request #21203 from smithfarm/wip-22394-jewel

jewel: librbd: cannot copy all image-metas if we have more than 64 key/value pairs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21098 from smithfarm/wip-23348-jewel
Nathan Cutler [Sun, 8 Apr 2018 19:54:40 +0000 (21:54 +0200)]
Merge pull request #21098 from smithfarm/wip-23348-jewel

jewel: rgw: inefficient buffer usage for PUTs

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Marcus Watts <mwatts@redhat.com>
7 years agoMerge pull request #21215 from smithfarm/wip-23546-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:37:17 +0000 (18:37 +0200)]
Merge pull request #21215 from smithfarm/wip-23546-jewel

jewel: journal: Message too long error when appending journal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21189 from smithfarm/wip-23356-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:35:18 +0000 (18:35 +0200)]
Merge pull request #21189 from smithfarm/wip-23356-jewel

jewel: cephfs: client: prevent fallback to remount when dentry_invalidate_cb is true but root->dir is NULL

Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #21232 from pdvian/wip-23543-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:26:06 +0000 (18:26 +0200)]
Merge pull request #21232 from pdvian/wip-23543-jewel

jewel: rbd-nbd: fix ebusy when do map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21228 from dillaman/wip-22396-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:25:25 +0000 (18:25 +0200)]
Merge pull request #21228 from dillaman/wip-22396-jewel

jewel: librbd: cannot clone all image-metas if we have more than 64 key/value pairs

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21227 from dillaman/wip-21442-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:24:20 +0000 (18:24 +0200)]
Merge pull request #21227 from dillaman/wip-21442-jewel

jewel: rbd: clean up warnings when mirror commands used on non-setup pool

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21225 from dillaman/wip-20637-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:23:30 +0000 (18:23 +0200)]
Merge pull request #21225 from dillaman/wip-20637-jewel

jewel: rbd-mirror: ignore permission errors on rbd_mirroring object

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21224 from dillaman/wip-21689-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:22:47 +0000 (18:22 +0200)]
Merge pull request #21224 from dillaman/wip-21689-jewel

jewel: librbd: list_children should not attempt to refresh image

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21223 from dillaman/wip-21915-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:21:37 +0000 (18:21 +0200)]
Merge pull request #21223 from dillaman/wip-21915-jewel

jewel: rbd-mirror: strip environment/CLI overrides for remote cluster

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21220 from dillaman/wip-21867-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:20:25 +0000 (18:20 +0200)]
Merge pull request #21220 from dillaman/wip-21867-jewel

jewel: librbd: object map batch update might cause OSD suicide timeout

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21219 from dillaman/wip-23305-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:19:24 +0000 (18:19 +0200)]
Merge pull request #21219 from dillaman/wip-23305-jewel

jewel: librbd: create+truncate for whole-object layered discards

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21207 from smithfarm/wip-23525-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:18:04 +0000 (18:18 +0200)]
Merge pull request #21207 from smithfarm/wip-23525-jewel

jewel: rbd: is_qemu_running in qemu_rebuild_object_map.sh and qemu_dynamic_features.sh may return false positive

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21206 from smithfarm/wip-23012-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:16:41 +0000 (18:16 +0200)]
Merge pull request #21206 from smithfarm/wip-23012-jewel

jewel: rbd: [journal] allocating a new tag after acquiring the lock should use on-disk committed position

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21205 from smithfarm/wip-22578-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:15:10 +0000 (18:15 +0200)]
Merge pull request #21205 from smithfarm/wip-22578-jewel

jewel: rbd: rbd-mirror split brain test case can have a false-positive failure until teuthology

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21125 from smithfarm/wip-23508-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:12:20 +0000 (18:12 +0200)]
Merge pull request #21125 from smithfarm/wip-23508-jewel

jewel: tests: test_admin_socket.sh may fail on wait_for_clean

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #20627 from pdvian/wip-23153-jewel
Nathan Cutler [Sun, 8 Apr 2018 15:28:49 +0000 (17:28 +0200)]
Merge pull request #20627 from pdvian/wip-23153-jewel

jewel: rbd: TestLibRBD.RenameViaLockOwner may still fail with -ENOENT

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agorgw: fix memory fragmentation problem reading data from client. 21098/head
Marcus Watts [Mon, 5 Mar 2018 21:10:29 +0000 (16:10 -0500)]
rgw: fix memory fragmentation problem reading data from client.

mg_read returns 0 on EOF.  In some versions of civetweb, mg_read can
return "short" reads.  The logic in the rest of ceph depends on a read
always filling its buffer when possible.  So loop here and fill the
buffer.  Looping to fill the buffer means we must also track when
mg_read returns EOF, because after that further calls to mg_read will return -1.

Fixes: https://tracker.ceph.com/issues/23207
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit d9a150b1f1f9ec3952555186722c9b13d7448e2d)

Conflicts:
    src/rgw/rgw_civetweb.cc:
        - master RGWCivetWeb::RGWCivetWeb() becomes RGWMongoose::RGWMongoose() in jewel,
          with slightly different initializer list; adapt mg_read() call for jewel
        - master RGWCivetWeb::read_data() becomes RGWMongoose::read_data() in jewel,
          and inside this function we return -EIO in jewel, instead of throwing an exception

7 years agoMerge pull request #18304 from smithfarm/wip-21546-jewel
Nathan Cutler [Fri, 6 Apr 2018 14:47:02 +0000 (16:47 +0200)]
Merge pull request #18304 from smithfarm/wip-21546-jewel

jewel: rgw: file write error

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #21100 from cbodley/wip-23311
Nathan Cutler [Fri, 6 Apr 2018 13:26:25 +0000 (15:26 +0200)]
Merge pull request #21100 from cbodley/wip-23311

jewel: rgw: s3website error handler uses original object name

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #21073 from linuxbox2/wip-rgw-tcmalloc
Nathan Cutler [Fri, 6 Apr 2018 13:25:03 +0000 (15:25 +0200)]
Merge pull request #21073 from linuxbox2/wip-rgw-tcmalloc

jewel: rgw: tcmalloc

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #20800 from theanalyst/jewel-fe-fixes
Nathan Cutler [Fri, 6 Apr 2018 13:23:44 +0000 (15:23 +0200)]
Merge pull request #20800 from theanalyst/jewel-fe-fixes

jewel: rgw:  make init env methods return an error

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20639 from mkogan1/wip-rgw-t23171
Nathan Cutler [Fri, 6 Apr 2018 13:21:54 +0000 (15:21 +0200)]
Merge pull request #20639 from mkogan1/wip-rgw-t23171

jewel: rgw: core dump, recursive lock of RGWKeystoneTokenCache

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #20421 from oritwas/wip-rgw-jewel-22124
Nathan Cutler [Fri, 6 Apr 2018 13:20:34 +0000 (15:20 +0200)]
Merge pull request #20421 from oritwas/wip-rgw-jewel-22124

jewel: rgw: bucket resharding should not update bucket ACL or user stats

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
7 years agoMerge pull request #20076 from smithfarm/wip-22774-jewel
Nathan Cutler [Fri, 6 Apr 2018 13:18:33 +0000 (15:18 +0200)]
Merge pull request #20076 from smithfarm/wip-22774-jewel

jewel: rgw: file deadlock on lru evicting

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #19887 from pdvian/wip-22636-jewel
Nathan Cutler [Fri, 6 Apr 2018 13:16:09 +0000 (15:16 +0200)]
Merge pull request #19887 from pdvian/wip-22636-jewel

jewel: rgw: add xml output header in RGWCopyObj_ObjStore_S3 response msg

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18207 from smithfarm/wip-21205-jewel
Nathan Cutler [Fri, 6 Apr 2018 13:13:41 +0000 (15:13 +0200)]
Merge pull request #18207 from smithfarm/wip-21205-jewel

jewel: rgw: bi list entry count incremented on error, distorting error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #21264 from smithfarm/wip-rgw-drop-btrfs-jewel
Nathan Cutler [Fri, 6 Apr 2018 06:10:30 +0000 (08:10 +0200)]
Merge pull request #21264 from smithfarm/wip-rgw-drop-btrfs-jewel

jewel: tests: rip out vestiges of btrfs from rgw suite

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #21184 from smithfarm/wip-21786-jewel
Nathan Cutler [Fri, 6 Apr 2018 01:56:33 +0000 (03:56 +0200)]
Merge pull request #21184 from smithfarm/wip-21786-jewel

jewel: osd: OSDMap cache assert on shutdown

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
7 years agoqa: rgw/multifs: test on XFS, drop BTRFS 21264/head
Nathan Cutler [Fri, 6 Apr 2018 01:42:22 +0000 (03:42 +0200)]
qa: rgw/multifs: test on XFS, drop BTRFS

objectstore/filestore-xfs.yaml was overriding fs/btrfs.yaml anyway,
which was confusing.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
7 years agoqa: rgw/verify: test on XFS, drop BTRFS
Nathan Cutler [Fri, 6 Apr 2018 01:41:45 +0000 (03:41 +0200)]
qa: rgw/verify: test on XFS, drop BTRFS

Signed-off-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #21156 from smithfarm/wip-22031-jewel
Nathan Cutler [Fri, 6 Apr 2018 01:12:27 +0000 (03:12 +0200)]
Merge pull request #21156 from smithfarm/wip-22031-jewel

jewel: mds: FAILED assert(get_version() < pv) in CDir::mark_dirty

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21172 from smithfarm/wip-22384-jewel
Nathan Cutler [Thu, 5 Apr 2018 19:39:14 +0000 (21:39 +0200)]
Merge pull request #21172 from smithfarm/wip-22384-jewel

jewel: tests: src/test/libcephfs/test.cc:376: Expected: (len) > (0), actual: -34 vs 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21163 from smithfarm/wip-22380-jewel
Nathan Cutler [Thu, 5 Apr 2018 18:39:00 +0000 (20:39 +0200)]
Merge pull request #21163 from smithfarm/wip-22380-jewel

jewel: cephfs: client reconnect gather race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20111 from pdvian/wip-22764-jewel
Nathan Cutler [Thu, 5 Apr 2018 18:37:44 +0000 (20:37 +0200)]
Merge pull request #20111 from pdvian/wip-22764-jewel

jewel: cephfs-journal-tool: add "set pool_id" option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21152 from smithfarm/wip-21489-jewel
Nathan Cutler [Thu, 5 Apr 2018 18:36:41 +0000 (20:36 +0200)]
Merge pull request #21152 from smithfarm/wip-21489-jewel

jewel: tests: failures from pjd fstest

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21175 from smithfarm/wip-22970-jewel
Nathan Cutler [Thu, 5 Apr 2018 16:26:05 +0000 (18:26 +0200)]
Merge pull request #21175 from smithfarm/wip-22970-jewel

jewel: mds: session reference leak

Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #21208 from ukernel/jewel-23529
Nathan Cutler [Thu, 5 Apr 2018 13:21:38 +0000 (15:21 +0200)]
Merge pull request #21208 from ukernel/jewel-23529

jewel: cephfs: fix tmap_upgrade crash

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21084 from pdvian/wip-23413-jewel
Nathan Cutler [Wed, 4 Apr 2018 19:24:40 +0000 (21:24 +0200)]
Merge pull request #21084 from pdvian/wip-23413-jewel

jewel: log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20622 from dzafman/wip-info-soid
Nathan Cutler [Wed, 4 Apr 2018 19:23:07 +0000 (21:23 +0200)]
Merge pull request #20622 from dzafman/wip-info-soid

jewel: follow-on: osd: be_select_auth_object() sanity check oi soid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21158 from smithfarm/wip-21784-jewel
Nathan Cutler [Wed, 4 Apr 2018 18:24:14 +0000 (20:24 +0200)]
Merge pull request #21158 from smithfarm/wip-21784-jewel

jewel: cli/crushtools/build.t sometimes fails in jenkins' make check run

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21135 from smithfarm/wip-23523-jewel
Nathan Cutler [Wed, 4 Apr 2018 18:22:18 +0000 (20:22 +0200)]
Merge pull request #21135 from smithfarm/wip-23523-jewel

jewel: tests: unittest_pglog timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
7 years agoMerge pull request #20999 from pdvian/wip-23411-jewel
Nathan Cutler [Wed, 4 Apr 2018 18:19:11 +0000 (20:19 +0200)]
Merge pull request #20999 from pdvian/wip-23411-jewel

jewel: legal: remove doc license ambiguity

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #20882 from dzafman/wip-23307
Nathan Cutler [Wed, 4 Apr 2018 18:18:10 +0000 (20:18 +0200)]
Merge pull request #20882 from dzafman/wip-23307

jewel: ceph-objectstore-tool command to trim the pg log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20763 from tchaikov/wip-23185-jewel
Nathan Cutler [Wed, 4 Apr 2018 18:16:50 +0000 (20:16 +0200)]
Merge pull request #20763 from tchaikov/wip-23185-jewel

jewel: ceph.in: bypass codec when writing raw binary data

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoPendingReleaseNotes: note about changing omap backend to rocksdb 18010/head
Josh Durgin [Sat, 3 Mar 2018 02:37:15 +0000 (21:37 -0500)]
PendingReleaseNotes: note about changing omap backend to rocksdb

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
7 years agoos/filestore: disable rocksdb compression
Sage Weil [Fri, 3 Nov 2017 15:03:22 +0000 (10:03 -0500)]
os/filestore: disable rocksdb compression

Experience working with customer escalations suggests that disabling
compression improves performance, and the storage overhead is generally
not a concern for the metadata and omap data we are storing.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b878ead071b328e5fe7309a2368383e67679e9f7)

Conflicts:
src/common/options.cc

Config options moved files since jewel.

7 years agocommon/options: enable multiple rocksdb compaction threads for filestore
Josh Durgin [Tue, 10 Oct 2017 23:54:31 +0000 (19:54 -0400)]
common/options: enable multiple rocksdb compaction threads for filestore

One of the major benefits of rocksdb over leveldb is multithreaded
compaction. The default of 1 thread does not provide much benefit, and
is insufficient for heavy rgw workloads.

For high-write and delete omap workloads I've seen up to 8 compaction
threads be used.  There's little overhead to having a higher max than
are needed, so set the default to 8.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 023fa810aa6b3af305e9027e3f717e54d1bb2712)

Conflicts:
src/common/options.cc (moved to config_opts.h)

7 years agotools/ceph-kvstore-tool: clean up destination store object
Josh Durgin [Tue, 26 Sep 2017 02:33:31 +0000 (19:33 -0700)]
tools/ceph-kvstore-tool: clean up destination store object

If left alive, background threads in the db may crash

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 718fc15831a2cb4eb91ed2aa2f694b47329f9189)

7 years agotools/ceph-kvstore-tool: add destination store type to store-copy
Josh Durgin [Tue, 26 Sep 2017 02:32:39 +0000 (19:32 -0700)]
tools/ceph-kvstore-tool: add destination store type to store-copy

This allows e.g. copying from leveldb into rocksdb

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit a65a7f0430a32a7748b54f4babecba0f58a5e42f)

7 years agoMakefile-rocksdb.am: update for new rocksdb version
Josh Durgin [Thu, 28 Sep 2017 21:31:43 +0000 (17:31 -0400)]
Makefile-rocksdb.am: update for new rocksdb version

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
7 years agoosd: switch filestore to default to rocksdb
Neha Ojha [Wed, 26 Apr 2017 17:20:35 +0000 (10:20 -0700)]
osd: switch filestore to default to rocksdb

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ba96197e6d4614377c80cf16a1b29b6e6ac70f90)

Conflicts:
src/common/config_opts.h (trivial)

7 years agorocksdb: remove experimental status
Sage Weil [Thu, 6 Oct 2016 17:56:43 +0000 (13:56 -0400)]
rocksdb: remove experimental status

We've been testing rocksdb on mons in QA for months now
with no problems.  We've also observed no problems with
its use in bluestore.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 68138893648e052a6549fecf23b0b58af921dfc7)

7 years agoinclude/assert: define ceph_abort_msg(cct, msg)
Sage Weil [Mon, 27 Jun 2016 18:34:31 +0000 (11:34 -0700)]
include/assert: define ceph_abort_msg(cct, msg)

Abort with an error string.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 23d6438f1253b593dd9a0e909e2ccbd045e168b8)

7 years agokv/RocksDBStore: abort if rocksdb EIO, don't return incorrect result
Haomai Wang [Fri, 23 Jun 2017 01:36:20 +0000 (09:36 +0800)]
kv/RocksDBStore: abort if rocksdb EIO, don't return incorrect result

If the underlying disk is missing, the current logic won't check the actual
reason why Get/Set failed, it will result to client get a empty key/value
pair which is not expected. The correct logic should be abort
right now. Otherwise, it will leads to undefined behavior.

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 8c60b55d7d49bc6fa1d8d076c025fc83283c0776)

Conflicts:
src/kv/RocksDBStore.cc (trivial)

7 years agoos/kv: beautify kvdb statistics
Varada Kari [Thu, 22 Dec 2016 09:49:38 +0000 (15:19 +0530)]
os/kv: beautify kvdb statistics

Improves the presentation of rocksdb dump statistics

Signed-off-by: Varada Kari <varada.kari@sandisk.com>
(cherry picked from commit 8e1d4ac92e2f9cd5f7438a99459bc063adf2542b)

7 years agosrc/kv/RocksDBStore.cc:fix typos
xianxiaxiao [Wed, 14 Dec 2016 02:32:16 +0000 (02:32 +0000)]
src/kv/RocksDBStore.cc:fix typos

Signed-off-by: xianxiaxiao <xianxia.xiao@kylin-cloud.com>
(cherry picked from commit b26555a0422423a9fef0034aa9313b3a4c1308ef)

7 years agokv/RocksDBStore: dump extended stats
Varada Kari [Fri, 2 Dec 2016 18:19:48 +0000 (13:19 -0500)]
kv/RocksDBStore: dump extended stats

Signed-off-by: Varada Kari <varada.kari@sandisk.com>
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 9fda9ce0b0fbf795469b50f6e9ebcb9bd0bc32a4)

Conflicts:
src/common/config_opts.h
src/kv/RocksDBStore.cc
src/kv/RocksDBStore.h
(trivial resolution)

7 years agokv/RocksDBStore: no reason to make write_batch using new/delete
Haomai Wang [Wed, 21 Sep 2016 08:20:48 +0000 (16:20 +0800)]
kv/RocksDBStore: no reason to make write_batch using new/delete

discards new/delete for bat

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 79a6611c0aa2c5b16435fad53268dd29a8541188)

Conflicts:
src/kv/RocksDBStore.cc (trivial)

7 years agoLogging rocksdb transaction in ceph log
Somnath Roy [Thu, 8 Sep 2016 18:25:45 +0000 (14:25 -0400)]
Logging rocksdb transaction in ceph log

The rocksdb transaction is now been logged into ceph log based
on debug level. If transaction is failed, the error code,
error string and the entire transaction is dumped in the log.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
(cherry picked from commit 46f1f9e5ca2f4e34393dfa1952fa4283a859d869)