]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agomds: change libmon to .la archive 21367/head
Kefu Chai [Thu, 12 Apr 2018 03:58:15 +0000 (11:58 +0800)]
mds: change libmon to .la archive

before this change, it is a plain .a archive. because MDSAuthCaps.cc is
included by both libmon.a and libmds.la, automake complains that

src/Makefile.am: error: object 'mds/MDSAuthCaps.$(OBJEXT)' created both
with libtool and without

so we need to consolidate them. as most of the convenience libraries are
are created using libtool. to be more consistent, i am chaning libmon.a
to libmon.la.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
   src/Makefile-env.am
   src/mds/Makefile-server.am: this change is not cherry-picked from
master. as we switched from autotools to cmake since luminous.

7 years agodoc: mention stricter caps checking in PendingReleaseNotes
Nathan Cutler [Thu, 12 Apr 2018 02:39:19 +0000 (04:39 +0200)]
doc: mention stricter caps checking in PendingReleaseNotes

Signed-off-by: Nathan Cutler <ncutler@suse.com>
7 years agoqa/workunits/cephtool: test with valid auth syntax
Sage Weil [Mon, 9 Apr 2018 21:50:15 +0000 (16:50 -0500)]
qa/workunits/cephtool: test with valid auth syntax

'allow' is not a valid mon cap; use 'allow *'.  Add some negative tests
to ensure the validation is working.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 497a63251b0d7b8b953bb01d0edb838b8b35919a)

qa/workunits/rest/test.py: update this change also, as the rest proxy
was removed in master.

7 years agomon/AuthMonitor: check caps validity for all cap-related commands
Sage Weil [Mon, 9 Apr 2018 21:40:37 +0000 (16:40 -0500)]
mon/AuthMonitor: check caps validity for all cap-related commands

Add a validity check for 'auth add' and 'fs authorize'.  'auth caps' and
'auth get-or-create[-key]' already had the check.

Fixes: http://tracker.ceph.com/issues/22525
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fa588730bfea17eb1ca556e20ffcfd0b7628d093)

Conflicts:
src/mon/AuthMonitor.cc (no "fs autorize" in jewel)

7 years agomon: expand cap validity check for mgr, osd, mds
lijing [Fri, 5 Jan 2018 09:44:57 +0000 (17:44 +0800)]
mon: expand cap validity check for mgr, osd, mds

Also strenghen the check to declare an unknown cap type as invalid.

Note that this means that in a mixed-version cluster, an older mon
would apply the syntax check for the older caps, even if the (say) OSDs
are newer and could parse something different.  This is judged to be fine:
it is not unreasonable to ask for the mons *and* daemons to be upgraded
before using the new cap syntax or feature.

Signed-off-by: Jing Li <lijing@gohighsec.com>
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 113fa941e3a4feaed0891d1fb7d321ade3e1bca7)

Conflicts:
src/mon/AuthMonitor.cc (no mgr in jewel)
src/mon/CMakeLists.txt (no cmake in jewel; backported the changes
                               manually to src/mon/Makefile.am)

7 years agoMerge pull request #17818 from smithfarm/wip-upgrade-rh-74
Yuri Weinstein [Thu, 24 May 2018 20:42:10 +0000 (13:42 -0700)]
Merge pull request #17818 from smithfarm/wip-upgrade-rh-74

jewel: tests: make upgrade tests use supported distros

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21194 from dzafman/wip-23267-jewel
Yuri Weinstein [Thu, 24 May 2018 20:40:44 +0000 (13:40 -0700)]
Merge pull request #21194 from dzafman/wip-23267-jewel

jewel: scrub errors not cleared on replicas can cause inconsistent pg state when replica takes over primary

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #21747 from vumrao/wip-vumrao-rhbz1568897
Yuri Weinstein [Thu, 24 May 2018 20:39:43 +0000 (13:39 -0700)]
Merge pull request #21747 from vumrao/wip-vumrao-rhbz1568897

jewel: OSD: print log when osd want to kill self

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22050 from tchaikov/jewel-22530
Yuri Weinstein [Thu, 24 May 2018 20:37:54 +0000 (13:37 -0700)]
Merge pull request #22050 from tchaikov/jewel-22530

jewel: mon/OSDMonitor.cc: fix expected_num_objects interpret error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22188 from tchaikov/jewel-23966
Yuri Weinstein [Thu, 24 May 2018 20:37:03 +0000 (13:37 -0700)]
Merge pull request #22188 from tchaikov/jewel-23966

jewel: Deleting a pool with active notify linger ops can result in seg fault

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22189 from theanalyst/wip-jewel-24244
Yuri Weinstein [Thu, 24 May 2018 20:35:57 +0000 (13:35 -0700)]
Merge pull request #22189 from theanalyst/wip-jewel-24244

mon/MDSMonitor: no_reply on MMDSLoadTargets

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21162 from smithfarm/wip-22378-jewel
Yuri Weinstein [Thu, 24 May 2018 20:28:48 +0000 (13:28 -0700)]
Merge pull request #21162 from smithfarm/wip-22378-jewel

jewel: ceph-fuse: failure to remount in startup test does not handle client_die_on_failed_remount properly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21734 from joscollin/wip-23932-jewel
Yuri Weinstein [Thu, 24 May 2018 20:27:59 +0000 (13:27 -0700)]
Merge pull request #21734 from joscollin/wip-23932-jewel

jewel: client: avoid second lock on client_lock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agomon/MDSMonitor: no_reply on MMDSLoadTargets 22189/head
Sage Weil [Wed, 2 May 2018 19:48:31 +0000 (14:48 -0500)]
mon/MDSMonitor: no_reply on MMDSLoadTargets

If we don't note that we don't reply then we don't close out the routed
mon request and the op will appear as slow on the forwarding mon.

Fixes: http://tracker.ceph.com/issues/23769
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b462b59065424520170956581e72e16481b16f0a)

7 years agolibrados: fix potential race condition if notify immediately fails 22188/head
Jason Dillaman [Mon, 7 May 2018 17:23:43 +0000 (10:23 -0700)]
librados: fix potential race condition if notify immediately fails

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cc06fbab08ef5b7f1b18f2af333c019ad86de03d)

7 years agoosdc: guard op->on_notify_finish with lock
Kefu Chai [Sun, 6 May 2018 03:11:16 +0000 (11:11 +0800)]
osdc: guard op->on_notify_finish with lock

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c16371965cc785f2e2b87d8c6684854fa2de65e1)

7 years agoosdc: invoke notify finish context on linger commit failure
Jason Dillaman [Wed, 2 May 2018 16:52:42 +0000 (12:52 -0400)]
osdc: invoke notify finish context on linger commit failure

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fcd414f6324136ce0695fcf30a60eec6ff338a98)

7 years agoMerge pull request #21503 from smithfarm/wip-23783-jewel
Kefu Chai [Wed, 23 May 2018 09:43:26 +0000 (17:43 +0800)]
Merge pull request #21503 from smithfarm/wip-23783-jewel

jewel: table of contents doesn't render for luminous/jewel docs

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22166 from tchaikov/jewel-javah
Kefu Chai [Wed, 23 May 2018 09:06:42 +0000 (17:06 +0800)]
Merge pull request #22166 from tchaikov/jewel-javah

jewel: makefile: use javac for creating JNI headers

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agomakefile: bump java source option from 5 to 6 22166/head
Kefu Chai [Wed, 23 May 2018 02:48:44 +0000 (10:48 +0800)]
makefile: bump java source option from 5 to 6

this kills the errors like

error: Source option 5 is no longer supported. Use 6 or later.
error: Target option 1.5 is no longer supported. Use 1.6 or later.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
    src/java/Makefile.am: this change is not cherry-picked from
master. as we've moved to java 8 in master. see df27ed3a .

7 years agomakefile: use javac for creating JNI headers
Kefu Chai [Wed, 23 May 2018 02:45:39 +0000 (10:45 +0800)]
makefile: use javac for creating JNI headers

JDK 1.10 does not offer javah anymore, so we need to use "javac -h"
instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit edc504b9d446cc1739f3e8e087914cfe66d6cac3)
Conflicts: configure.ac:
           src/java/Makefile.am: we don't use cmake back in jewel, so
port the cmake changes to autotools.

7 years agoMerge pull request #22130 from ceph/wip-yuriw-clients-fix-jewel
Josh Durgin [Mon, 21 May 2018 22:54:27 +0000 (15:54 -0700)]
Merge pull request #22130 from ceph/wip-yuriw-clients-fix-jewel

qa/test: added supported distro

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoqa/test: added supported ditro 22090/head 22130/head
Yuri Weinstein [Fri, 18 May 2018 15:39:45 +0000 (08:39 -0700)]
qa/test: added supported ditro

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #21210 from pdvian/wip-23303-jewel
Nathan Cutler [Fri, 18 May 2018 11:01:32 +0000 (13:01 +0200)]
Merge pull request #21210 from pdvian/wip-23303-jewel

jewel: rgw: add radosgw-admin sync error trim to trim sync error log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agomon/OSDMonitor.cc: fix expected_num_objects interpret error 22050/head
Yang Honggang [Mon, 22 Jan 2018 02:00:17 +0000 (21:00 -0500)]
mon/OSDMonitor.cc: fix expected_num_objects interpret error

Fixes: http://tracker.ceph.com/issues/22530
Signed-off-by: Yang Honggang <joseph.yang@xtaotech.com>
(cherry picked from commit c24f2baec968ff66e792991d829591505ffba3ae)

Conflicts:
qa/standalone/mon/osd-pool-create.sh: jewel does not have
qa/standalone tests by then. and in jewel, "ruleset" is not renamed to
"rule" yet.
src/mon/OSDMonitor.cc
src/pybind/mgr/localpool/module.py: trivial resolution

7 years agoMerge pull request #21190 from smithfarm/wip-21033-jewel
Nathan Cutler [Thu, 17 May 2018 09:19:03 +0000 (11:19 +0200)]
Merge pull request #21190 from smithfarm/wip-21033-jewel

jewel: build/ops: rpm: bump epoch ahead of ceph-common in RHEL base

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoMerge pull request #21213 from smithfarm/wip-23026-jewel
Nathan Cutler [Thu, 17 May 2018 09:16:19 +0000 (11:16 +0200)]
Merge pull request #21213 from smithfarm/wip-23026-jewel

jewel: rgw: data sync of versioned objects, note updating bi marker

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoOSD: print log when osd want to kill self 21747/head
Haomai Wang [Tue, 24 May 2016 11:47:32 +0000 (19:47 +0800)]
OSD: print log when osd want to kill self

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 9bc7d9667076473deda4a9214eb74955458eb36f)

7 years agoclient: avoid second lock on client_lock 21734/head
Supriti Singh [Fri, 20 Apr 2018 07:48:53 +0000 (09:48 +0200)]
client: avoid second lock on client_lock

Avoid a second nested lock on client lock. As a result, drop unused function ll_get_inodeno().
Fixes: https://tracker.ceph.com/issues/23815
Signed-off-by: Supriti Singh <supriti.singh@suse.com>
(cherry picked from commit 7be8d559a11507255f847ce6279d23abaabde70a)

 Conflicts:
src/client/Client.cc (dropped function call ll_get_inodeno(in))

7 years agodoc update template call for toctree 21503/head
Alfredo Deza [Fri, 18 Aug 2017 16:05:03 +0000 (12:05 -0400)]
doc update template call for toctree

Up until version 1.1.3 of Sphinx, this call would show hidden toctree
entries. Most of Ceph's toctree's are hidden, so when the version of
Sphinx was bumped to >1.6, the sidebar stopped showing entries

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9bb3397a5876ded89853f11cd35b3ca5e8aeec5c)

7 years agoosd: Fix stale scrub stats when a primary takes over 21194/head
David Zafman [Wed, 28 Mar 2018 17:21:39 +0000 (10:21 -0700)]
osd: Fix stale scrub stats when a primary takes over

Fixes: http://tracker.ceph.com/issues/23267
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 5cfb8241f482ed53c63bb97262425b2acb733d7d)

Conflicts:
src/osd/PG.cc (trivial)
test/Makefile.am (Add to make check for Jewel)
test/osd/CMakeLists.txt (Add to make check for Jewel)
src/test/osd/osd-scrub-test.sh (from qa/standalone/scrub/osd-scrub-test.sh)
  Different location of ceph-helpers.sh
  No manager so remove run_mgr
  Remove default rbd pool
  Use 4 OSDs because remapped PG doesn't go clean

7 years agoclient: quit on failed remount during dentry invalidate test 21162/head
Patrick Donnelly [Tue, 28 Nov 2017 23:01:32 +0000 (15:01 -0800)]
client: quit on failed remount during dentry invalidate test

Fixes: http://tracker.ceph.com/issues/22269
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5b2b1d14468c290c56ee6c95ea557c99464e0098)

Conflicts:
PendingReleaseNotes - put the release note under 10.2.11
src/ceph_fuse.cc - changed order of client_try_dentry_invalidate and
    can_invalidate_dentries assignments to match master, and use jewel
            form of g_conf for accessing config option values
src/client/Client.cc - code being replaced by _do_remount() calls is
    different in jewel, and use jewel convention for accessing config
            option values
src/common/legacy_config_opts.h - file does not exist in jewel
src/common/options.cc - file does not exit in jewel; made the changes
            manually in src/common/config_opts.h

7 years agoMerge pull request #21185 from smithfarm/wip-23158-jewel
Nathan Cutler [Tue, 10 Apr 2018 07:07:22 +0000 (09:07 +0200)]
Merge pull request #21185 from smithfarm/wip-23158-jewel

jewel: mds: underwater dentry check in CDir::_omap_fetched is racy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #18010 from jdurgin/wip-rocksdb-jewel
Josh Durgin [Tue, 10 Apr 2018 05:29:35 +0000 (22:29 -0700)]
Merge pull request #18010 from jdurgin/wip-rocksdb-jewel

jewel: core: enable rocksdb for filestore

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #21199 from smithfarm/wip-22403-jewel
Josh Durgin [Tue, 10 Apr 2018 01:43:40 +0000 (18:43 -0700)]
Merge pull request #21199 from smithfarm/wip-22403-jewel

jewel: osd: replica read can trigger cache promotion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21200 from ovh/bp-jewel-backport-22448
Josh Durgin [Tue, 10 Apr 2018 01:40:53 +0000 (18:40 -0700)]
Merge pull request #21200 from ovh/bp-jewel-backport-22448

jewel: osd/PrimaryLogPG: dump snap_trimq size

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21197 from smithfarm/wip-23521-jewel
Josh Durgin [Mon, 9 Apr 2018 23:44:49 +0000 (16:44 -0700)]
Merge pull request #21197 from smithfarm/wip-23521-jewel

jewel: ceph_authtool: add mode option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20381 from pdvian/wip-22941-jewel
Josh Durgin [Mon, 9 Apr 2018 23:38:12 +0000 (16:38 -0700)]
Merge pull request #20381 from pdvian/wip-22941-jewel

jewel: core: librados: Double free in rados_getxattrs_next

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21203 from smithfarm/wip-22394-jewel
Nathan Cutler [Sun, 8 Apr 2018 20:03:00 +0000 (22:03 +0200)]
Merge pull request #21203 from smithfarm/wip-22394-jewel

jewel: librbd: cannot copy all image-metas if we have more than 64 key/value pairs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21098 from smithfarm/wip-23348-jewel
Nathan Cutler [Sun, 8 Apr 2018 19:54:40 +0000 (21:54 +0200)]
Merge pull request #21098 from smithfarm/wip-23348-jewel

jewel: rgw: inefficient buffer usage for PUTs

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Marcus Watts <mwatts@redhat.com>
7 years agoMerge pull request #21215 from smithfarm/wip-23546-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:37:17 +0000 (18:37 +0200)]
Merge pull request #21215 from smithfarm/wip-23546-jewel

jewel: journal: Message too long error when appending journal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21189 from smithfarm/wip-23356-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:35:18 +0000 (18:35 +0200)]
Merge pull request #21189 from smithfarm/wip-23356-jewel

jewel: cephfs: client: prevent fallback to remount when dentry_invalidate_cb is true but root->dir is NULL

Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #21232 from pdvian/wip-23543-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:26:06 +0000 (18:26 +0200)]
Merge pull request #21232 from pdvian/wip-23543-jewel

jewel: rbd-nbd: fix ebusy when do map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21228 from dillaman/wip-22396-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:25:25 +0000 (18:25 +0200)]
Merge pull request #21228 from dillaman/wip-22396-jewel

jewel: librbd: cannot clone all image-metas if we have more than 64 key/value pairs

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21227 from dillaman/wip-21442-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:24:20 +0000 (18:24 +0200)]
Merge pull request #21227 from dillaman/wip-21442-jewel

jewel: rbd: clean up warnings when mirror commands used on non-setup pool

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21225 from dillaman/wip-20637-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:23:30 +0000 (18:23 +0200)]
Merge pull request #21225 from dillaman/wip-20637-jewel

jewel: rbd-mirror: ignore permission errors on rbd_mirroring object

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21224 from dillaman/wip-21689-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:22:47 +0000 (18:22 +0200)]
Merge pull request #21224 from dillaman/wip-21689-jewel

jewel: librbd: list_children should not attempt to refresh image

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21223 from dillaman/wip-21915-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:21:37 +0000 (18:21 +0200)]
Merge pull request #21223 from dillaman/wip-21915-jewel

jewel: rbd-mirror: strip environment/CLI overrides for remote cluster

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21220 from dillaman/wip-21867-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:20:25 +0000 (18:20 +0200)]
Merge pull request #21220 from dillaman/wip-21867-jewel

jewel: librbd: object map batch update might cause OSD suicide timeout

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21219 from dillaman/wip-23305-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:19:24 +0000 (18:19 +0200)]
Merge pull request #21219 from dillaman/wip-23305-jewel

jewel: librbd: create+truncate for whole-object layered discards

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21207 from smithfarm/wip-23525-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:18:04 +0000 (18:18 +0200)]
Merge pull request #21207 from smithfarm/wip-23525-jewel

jewel: rbd: is_qemu_running in qemu_rebuild_object_map.sh and qemu_dynamic_features.sh may return false positive

Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #21206 from smithfarm/wip-23012-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:16:41 +0000 (18:16 +0200)]
Merge pull request #21206 from smithfarm/wip-23012-jewel

jewel: rbd: [journal] allocating a new tag after acquiring the lock should use on-disk committed position

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21205 from smithfarm/wip-22578-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:15:10 +0000 (18:15 +0200)]
Merge pull request #21205 from smithfarm/wip-22578-jewel

jewel: rbd: rbd-mirror split brain test case can have a false-positive failure until teuthology

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21125 from smithfarm/wip-23508-jewel
Nathan Cutler [Sun, 8 Apr 2018 16:12:20 +0000 (18:12 +0200)]
Merge pull request #21125 from smithfarm/wip-23508-jewel

jewel: tests: test_admin_socket.sh may fail on wait_for_clean

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #20627 from pdvian/wip-23153-jewel
Nathan Cutler [Sun, 8 Apr 2018 15:28:49 +0000 (17:28 +0200)]
Merge pull request #20627 from pdvian/wip-23153-jewel

jewel: rbd: TestLibRBD.RenameViaLockOwner may still fail with -ENOENT

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agorgw: fix memory fragmentation problem reading data from client. 21098/head
Marcus Watts [Mon, 5 Mar 2018 21:10:29 +0000 (16:10 -0500)]
rgw: fix memory fragmentation problem reading data from client.

mg_read returns 0 on EOF.  In some versions of civetweb, mg_read can
return "short" reads.  The logic in the rest of ceph depends on a read
always filling its buffer when possible.  So loop here and fill the
buffer.  Looping to fill the buffer means we must also track when
mg_read returns EOF, because after that further calls to mg_read will return -1.

Fixes: https://tracker.ceph.com/issues/23207
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit d9a150b1f1f9ec3952555186722c9b13d7448e2d)

Conflicts:
    src/rgw/rgw_civetweb.cc:
        - master RGWCivetWeb::RGWCivetWeb() becomes RGWMongoose::RGWMongoose() in jewel,
          with slightly different initializer list; adapt mg_read() call for jewel
        - master RGWCivetWeb::read_data() becomes RGWMongoose::read_data() in jewel,
          and inside this function we return -EIO in jewel, instead of throwing an exception

7 years agoMerge pull request #18304 from smithfarm/wip-21546-jewel
Nathan Cutler [Fri, 6 Apr 2018 14:47:02 +0000 (16:47 +0200)]
Merge pull request #18304 from smithfarm/wip-21546-jewel

jewel: rgw: file write error

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #21100 from cbodley/wip-23311
Nathan Cutler [Fri, 6 Apr 2018 13:26:25 +0000 (15:26 +0200)]
Merge pull request #21100 from cbodley/wip-23311

jewel: rgw: s3website error handler uses original object name

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #21073 from linuxbox2/wip-rgw-tcmalloc
Nathan Cutler [Fri, 6 Apr 2018 13:25:03 +0000 (15:25 +0200)]
Merge pull request #21073 from linuxbox2/wip-rgw-tcmalloc

jewel: rgw: tcmalloc

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #20800 from theanalyst/jewel-fe-fixes
Nathan Cutler [Fri, 6 Apr 2018 13:23:44 +0000 (15:23 +0200)]
Merge pull request #20800 from theanalyst/jewel-fe-fixes

jewel: rgw:  make init env methods return an error

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20639 from mkogan1/wip-rgw-t23171
Nathan Cutler [Fri, 6 Apr 2018 13:21:54 +0000 (15:21 +0200)]
Merge pull request #20639 from mkogan1/wip-rgw-t23171

jewel: rgw: core dump, recursive lock of RGWKeystoneTokenCache

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #20421 from oritwas/wip-rgw-jewel-22124
Nathan Cutler [Fri, 6 Apr 2018 13:20:34 +0000 (15:20 +0200)]
Merge pull request #20421 from oritwas/wip-rgw-jewel-22124

jewel: rgw: bucket resharding should not update bucket ACL or user stats

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
7 years agoMerge pull request #20076 from smithfarm/wip-22774-jewel
Nathan Cutler [Fri, 6 Apr 2018 13:18:33 +0000 (15:18 +0200)]
Merge pull request #20076 from smithfarm/wip-22774-jewel

jewel: rgw: file deadlock on lru evicting

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #19887 from pdvian/wip-22636-jewel
Nathan Cutler [Fri, 6 Apr 2018 13:16:09 +0000 (15:16 +0200)]
Merge pull request #19887 from pdvian/wip-22636-jewel

jewel: rgw: add xml output header in RGWCopyObj_ObjStore_S3 response msg

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18207 from smithfarm/wip-21205-jewel
Nathan Cutler [Fri, 6 Apr 2018 13:13:41 +0000 (15:13 +0200)]
Merge pull request #18207 from smithfarm/wip-21205-jewel

jewel: rgw: bi list entry count incremented on error, distorting error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #21264 from smithfarm/wip-rgw-drop-btrfs-jewel
Nathan Cutler [Fri, 6 Apr 2018 06:10:30 +0000 (08:10 +0200)]
Merge pull request #21264 from smithfarm/wip-rgw-drop-btrfs-jewel

jewel: tests: rip out vestiges of btrfs from rgw suite

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #21184 from smithfarm/wip-21786-jewel
Nathan Cutler [Fri, 6 Apr 2018 01:56:33 +0000 (03:56 +0200)]
Merge pull request #21184 from smithfarm/wip-21786-jewel

jewel: osd: OSDMap cache assert on shutdown

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
7 years agoqa: rgw/multifs: test on XFS, drop BTRFS 21264/head
Nathan Cutler [Fri, 6 Apr 2018 01:42:22 +0000 (03:42 +0200)]
qa: rgw/multifs: test on XFS, drop BTRFS

objectstore/filestore-xfs.yaml was overriding fs/btrfs.yaml anyway,
which was confusing.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
7 years agoqa: rgw/verify: test on XFS, drop BTRFS
Nathan Cutler [Fri, 6 Apr 2018 01:41:45 +0000 (03:41 +0200)]
qa: rgw/verify: test on XFS, drop BTRFS

Signed-off-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #21156 from smithfarm/wip-22031-jewel
Nathan Cutler [Fri, 6 Apr 2018 01:12:27 +0000 (03:12 +0200)]
Merge pull request #21156 from smithfarm/wip-22031-jewel

jewel: mds: FAILED assert(get_version() < pv) in CDir::mark_dirty

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21172 from smithfarm/wip-22384-jewel
Nathan Cutler [Thu, 5 Apr 2018 19:39:14 +0000 (21:39 +0200)]
Merge pull request #21172 from smithfarm/wip-22384-jewel

jewel: tests: src/test/libcephfs/test.cc:376: Expected: (len) > (0), actual: -34 vs 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21163 from smithfarm/wip-22380-jewel
Nathan Cutler [Thu, 5 Apr 2018 18:39:00 +0000 (20:39 +0200)]
Merge pull request #21163 from smithfarm/wip-22380-jewel

jewel: cephfs: client reconnect gather race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20111 from pdvian/wip-22764-jewel
Nathan Cutler [Thu, 5 Apr 2018 18:37:44 +0000 (20:37 +0200)]
Merge pull request #20111 from pdvian/wip-22764-jewel

jewel: cephfs-journal-tool: add "set pool_id" option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21152 from smithfarm/wip-21489-jewel
Nathan Cutler [Thu, 5 Apr 2018 18:36:41 +0000 (20:36 +0200)]
Merge pull request #21152 from smithfarm/wip-21489-jewel

jewel: tests: failures from pjd fstest

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21175 from smithfarm/wip-22970-jewel
Nathan Cutler [Thu, 5 Apr 2018 16:26:05 +0000 (18:26 +0200)]
Merge pull request #21175 from smithfarm/wip-22970-jewel

jewel: mds: session reference leak

Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #21208 from ukernel/jewel-23529
Nathan Cutler [Thu, 5 Apr 2018 13:21:38 +0000 (15:21 +0200)]
Merge pull request #21208 from ukernel/jewel-23529

jewel: cephfs: fix tmap_upgrade crash

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21084 from pdvian/wip-23413-jewel
Nathan Cutler [Wed, 4 Apr 2018 19:24:40 +0000 (21:24 +0200)]
Merge pull request #21084 from pdvian/wip-23413-jewel

jewel: log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20622 from dzafman/wip-info-soid
Nathan Cutler [Wed, 4 Apr 2018 19:23:07 +0000 (21:23 +0200)]
Merge pull request #20622 from dzafman/wip-info-soid

jewel: follow-on: osd: be_select_auth_object() sanity check oi soid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21158 from smithfarm/wip-21784-jewel
Nathan Cutler [Wed, 4 Apr 2018 18:24:14 +0000 (20:24 +0200)]
Merge pull request #21158 from smithfarm/wip-21784-jewel

jewel: cli/crushtools/build.t sometimes fails in jenkins' make check run

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21135 from smithfarm/wip-23523-jewel
Nathan Cutler [Wed, 4 Apr 2018 18:22:18 +0000 (20:22 +0200)]
Merge pull request #21135 from smithfarm/wip-23523-jewel

jewel: tests: unittest_pglog timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
7 years agoMerge pull request #20999 from pdvian/wip-23411-jewel
Nathan Cutler [Wed, 4 Apr 2018 18:19:11 +0000 (20:19 +0200)]
Merge pull request #20999 from pdvian/wip-23411-jewel

jewel: legal: remove doc license ambiguity

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #20882 from dzafman/wip-23307
Nathan Cutler [Wed, 4 Apr 2018 18:18:10 +0000 (20:18 +0200)]
Merge pull request #20882 from dzafman/wip-23307

jewel: ceph-objectstore-tool command to trim the pg log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20763 from tchaikov/wip-23185-jewel
Nathan Cutler [Wed, 4 Apr 2018 18:16:50 +0000 (20:16 +0200)]
Merge pull request #20763 from tchaikov/wip-23185-jewel

jewel: ceph.in: bypass codec when writing raw binary data

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoPendingReleaseNotes: note about changing omap backend to rocksdb 18010/head
Josh Durgin [Sat, 3 Mar 2018 02:37:15 +0000 (21:37 -0500)]
PendingReleaseNotes: note about changing omap backend to rocksdb

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
7 years agoos/filestore: disable rocksdb compression
Sage Weil [Fri, 3 Nov 2017 15:03:22 +0000 (10:03 -0500)]
os/filestore: disable rocksdb compression

Experience working with customer escalations suggests that disabling
compression improves performance, and the storage overhead is generally
not a concern for the metadata and omap data we are storing.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b878ead071b328e5fe7309a2368383e67679e9f7)

Conflicts:
src/common/options.cc

Config options moved files since jewel.

7 years agocommon/options: enable multiple rocksdb compaction threads for filestore
Josh Durgin [Tue, 10 Oct 2017 23:54:31 +0000 (19:54 -0400)]
common/options: enable multiple rocksdb compaction threads for filestore

One of the major benefits of rocksdb over leveldb is multithreaded
compaction. The default of 1 thread does not provide much benefit, and
is insufficient for heavy rgw workloads.

For high-write and delete omap workloads I've seen up to 8 compaction
threads be used.  There's little overhead to having a higher max than
are needed, so set the default to 8.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 023fa810aa6b3af305e9027e3f717e54d1bb2712)

Conflicts:
src/common/options.cc (moved to config_opts.h)

7 years agotools/ceph-kvstore-tool: clean up destination store object
Josh Durgin [Tue, 26 Sep 2017 02:33:31 +0000 (19:33 -0700)]
tools/ceph-kvstore-tool: clean up destination store object

If left alive, background threads in the db may crash

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 718fc15831a2cb4eb91ed2aa2f694b47329f9189)

7 years agotools/ceph-kvstore-tool: add destination store type to store-copy
Josh Durgin [Tue, 26 Sep 2017 02:32:39 +0000 (19:32 -0700)]
tools/ceph-kvstore-tool: add destination store type to store-copy

This allows e.g. copying from leveldb into rocksdb

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit a65a7f0430a32a7748b54f4babecba0f58a5e42f)

7 years agoMakefile-rocksdb.am: update for new rocksdb version
Josh Durgin [Thu, 28 Sep 2017 21:31:43 +0000 (17:31 -0400)]
Makefile-rocksdb.am: update for new rocksdb version

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
7 years agoosd: switch filestore to default to rocksdb
Neha Ojha [Wed, 26 Apr 2017 17:20:35 +0000 (10:20 -0700)]
osd: switch filestore to default to rocksdb

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ba96197e6d4614377c80cf16a1b29b6e6ac70f90)

Conflicts:
src/common/config_opts.h (trivial)

7 years agorocksdb: remove experimental status
Sage Weil [Thu, 6 Oct 2016 17:56:43 +0000 (13:56 -0400)]
rocksdb: remove experimental status

We've been testing rocksdb on mons in QA for months now
with no problems.  We've also observed no problems with
its use in bluestore.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 68138893648e052a6549fecf23b0b58af921dfc7)

7 years agoinclude/assert: define ceph_abort_msg(cct, msg)
Sage Weil [Mon, 27 Jun 2016 18:34:31 +0000 (11:34 -0700)]
include/assert: define ceph_abort_msg(cct, msg)

Abort with an error string.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 23d6438f1253b593dd9a0e909e2ccbd045e168b8)

7 years agokv/RocksDBStore: abort if rocksdb EIO, don't return incorrect result
Haomai Wang [Fri, 23 Jun 2017 01:36:20 +0000 (09:36 +0800)]
kv/RocksDBStore: abort if rocksdb EIO, don't return incorrect result

If the underlying disk is missing, the current logic won't check the actual
reason why Get/Set failed, it will result to client get a empty key/value
pair which is not expected. The correct logic should be abort
right now. Otherwise, it will leads to undefined behavior.

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 8c60b55d7d49bc6fa1d8d076c025fc83283c0776)

Conflicts:
src/kv/RocksDBStore.cc (trivial)

7 years agoos/kv: beautify kvdb statistics
Varada Kari [Thu, 22 Dec 2016 09:49:38 +0000 (15:19 +0530)]
os/kv: beautify kvdb statistics

Improves the presentation of rocksdb dump statistics

Signed-off-by: Varada Kari <varada.kari@sandisk.com>
(cherry picked from commit 8e1d4ac92e2f9cd5f7438a99459bc063adf2542b)

7 years agosrc/kv/RocksDBStore.cc:fix typos
xianxiaxiao [Wed, 14 Dec 2016 02:32:16 +0000 (02:32 +0000)]
src/kv/RocksDBStore.cc:fix typos

Signed-off-by: xianxiaxiao <xianxia.xiao@kylin-cloud.com>
(cherry picked from commit b26555a0422423a9fef0034aa9313b3a4c1308ef)

7 years agokv/RocksDBStore: dump extended stats
Varada Kari [Fri, 2 Dec 2016 18:19:48 +0000 (13:19 -0500)]
kv/RocksDBStore: dump extended stats

Signed-off-by: Varada Kari <varada.kari@sandisk.com>
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 9fda9ce0b0fbf795469b50f6e9ebcb9bd0bc32a4)

Conflicts:
src/common/config_opts.h
src/kv/RocksDBStore.cc
src/kv/RocksDBStore.h
(trivial resolution)

7 years agokv/RocksDBStore: no reason to make write_batch using new/delete
Haomai Wang [Wed, 21 Sep 2016 08:20:48 +0000 (16:20 +0800)]
kv/RocksDBStore: no reason to make write_batch using new/delete

discards new/delete for bat

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 79a6611c0aa2c5b16435fad53268dd29a8541188)

Conflicts:
src/kv/RocksDBStore.cc (trivial)

7 years agoLogging rocksdb transaction in ceph log
Somnath Roy [Thu, 8 Sep 2016 18:25:45 +0000 (14:25 -0400)]
Logging rocksdb transaction in ceph log

The rocksdb transaction is now been logged into ceph log based
on debug level. If transaction is failed, the error code,
error string and the entire transaction is dumped in the log.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
(cherry picked from commit 46f1f9e5ca2f4e34393dfa1952fa4283a859d869)

7 years agoRocksDBStore: use Get for multiget
Haomai Wang [Thu, 2 Jun 2016 15:01:01 +0000 (23:01 +0800)]
RocksDBStore: use Get for multiget

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit a44f4f7238585996f83f7e35e01587671fe675d4)

7 years agokv/RocksDBStore: convert get() to use rocksdb Get() instead of iterator
Jianjian Huo [Wed, 25 May 2016 23:38:10 +0000 (16:38 -0700)]
kv/RocksDBStore: convert get() to use rocksdb Get() instead of iterator

For RocksDB, iterator will bypass row cache.
We don't want this to happen for get(), after row cache is enabled.

Signed-off-by: Jianjian Huo <jianjian.huo@ssi.samsung.com>
(cherry picked from commit f6ac5579190549291f6385775070c305dc8fba8b)

Conflicts:
src/kv/RocksDBStore.cc (trivial)