]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
9 years agoqa/workunits: remove 'mds setmap' from workunits 8123/head
Sage Weil [Tue, 15 Mar 2016 13:44:13 +0000 (09:44 -0400)]
qa/workunits: remove 'mds setmap' from workunits

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6a80e2df8b31802a39ef6f89136ed357dce50c20)

9 years agoMerge pull request #7911 from dzafman/wip-meta-list-hammer
Loic Dachary [Fri, 11 Mar 2016 03:05:42 +0000 (10:05 +0700)]
Merge pull request #7911 from dzafman/wip-meta-list-hammer

hammer: tools, test: Add ceph-objectstore-tool to operate on the meta collection

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7903 from branch-predictor/bp-hammer-force-bench-smoke
Loic Dachary [Fri, 11 Mar 2016 03:03:43 +0000 (10:03 +0700)]
Merge pull request #7903 from branch-predictor/bp-hammer-force-bench-smoke

hammer: tools/rados: add bench smoke tests

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7876 from SUSE/wip-14946-hammer
Loic Dachary [Fri, 11 Mar 2016 03:02:18 +0000 (10:02 +0700)]
Merge pull request #7876 from SUSE/wip-14946-hammer

hammer: redhat-lsb-core dependency was dropped, but is still needed

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7817 from asheplyakov/hammer-bug-14873
Loic Dachary [Fri, 11 Mar 2016 03:00:48 +0000 (10:00 +0700)]
Merge pull request #7817 from asheplyakov/hammer-bug-14873

hammer: tools: fix race condition in seq/rand bench (part 2)

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7896 from asheplyakov/hammer-bug-14873-p1
Loic Dachary [Fri, 11 Mar 2016 03:00:18 +0000 (10:00 +0700)]
Merge pull request #7896 from asheplyakov/hammer-bug-14873-p1

hammer: tools: fix race condition in seq/rand bench (part 1)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
9 years agoMerge pull request #6604 from ukernel/hammer-13268
Loic Dachary [Tue, 8 Mar 2016 03:00:15 +0000 (10:00 +0700)]
Merge pull request #6604 from ukernel/hammer-13268

hammer: client: added permission check based on getgrouplist

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #6551 from ceph/hammer-client-inoderef
Loic Dachary [Tue, 8 Mar 2016 02:58:30 +0000 (09:58 +0700)]
Merge pull request #6551 from ceph/hammer-client-inoderef

hammer: client inoderef

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #7797 from SUSE/wip-14861-hammer
Loic Dachary [Fri, 4 Mar 2016 03:25:00 +0000 (10:25 +0700)]
Merge pull request #7797 from SUSE/wip-14861-hammer

hammer: ceph init script unconditionally sources /lib/lsb/init-functions

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
9 years agotools, test: Add ceph-objectstore-tool to operate on the meta collection 7911/head
David Zafman [Thu, 4 Jun 2015 17:37:42 +0000 (10:37 -0700)]
tools, test: Add ceph-objectstore-tool to operate on the meta collection

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 31fa89f24997c3e9b4f8795ea28938ab65493d10)

Conflicts:
src/tools/ceph_objectstore_tool.cc
is_pg() still needs snap argument

9 years agocommon/obj_bencher.cc: make verify error fatal 7903/head
Piotr Dałek [Thu, 3 Mar 2016 10:30:53 +0000 (11:30 +0100)]
common/obj_bencher.cc: make verify error fatal

When run without "--no-verify", all verification errors are noted,
but they are not forwarded/reported anywhere else but to cerr, which
will cause automated testing to ignore them. Make seq_read_bench and
rand_read_bench return -EIO on any verification error which will,
in turn, return it back to caller.

Fixes: #14974
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
(cherry picked from commit 658bba85c488a38e25bc598637258c263f711631)

9 years agotest/test_rados_tool.sh: force rados bench rand and seq
Piotr Dałek [Thu, 3 Mar 2016 13:40:44 +0000 (14:40 +0100)]
test/test_rados_tool.sh: force rados bench rand and seq

This forces rados bench rand and seq to execute, so if there's
a regression in any of above, it won't go unnoticed.

Fixes: #14974
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
9 years agoMerge pull request #7672 from dachary/wip-14783-hammer
Sage Weil [Thu, 3 Mar 2016 13:35:37 +0000 (08:35 -0500)]
Merge pull request #7672 from dachary/wip-14783-hammer

hammer: test_bit_vector.cc uses magic numbers against #defines that vary

9 years agohammer: tools: fix race condition in seq/rand bench (part 2) 7817/head
Alexey Sheplyakov [Fri, 26 Feb 2016 12:01:11 +0000 (15:01 +0300)]
hammer: tools: fix race condition in seq/rand bench (part 2)

Commit c2c6d02591519dfd15ddcb397ac440322a964deb which is intended to
cherry-pick 9bcf5f065c4ed4b10d8f98961d1f99493bcdc9b8 incorrectly resolved
a conflict by adding code where it should have been removed. The incorrect
conflict resolution can been displayed with

   commit=c2c6d02591519dfd15ddcb397ac440322a964deb
   picked_from=9bcf5f065c4ed4b10d8f98961d1f99493bcdc9b8
   diff -u --ignore-matching-lines '^[^+-]' <(git show $picked_from) <(git show $commit)

--- /dev/fd/63    2016-03-03 14:09:51.354329129 +0700
+++ /dev/fd/62    2016-03-03 14:09:51.358329122 +0700
@@ -76,20 +79,18 @@
      ++data.in_flight;
 -    if (!no_verify) {
 -      snprintf(data.object_contents, data.object_size, "I'm the %16dth object!", current_index);
--      lock.Unlock();
+       lock.Unlock();
 -      if (memcmp(data.object_contents, cur_contents->c_str(), data.object_size) != 0) {
 -        cerr << name[slot] << " is not correct!" << std::endl;
 -        ++errors;
 -      }
--    } else {
--        lock.Unlock();
--    }
--
-+    lock.Unlock();
-     name[slot] = newName;
-   }
-
-@@ -789,11 +791,14 @@ int ObjBencher::rand_read_bench(int seconds_to_run, int num_objects, int concurr
++    if (memcmp(data.object_contents, cur_contents->c_str(), data.object_size) != 0) {
++      cerr << name[slot] << " is not correct!" << std::endl;
++      ++errors;
+     } else {
+       lock.Unlock();
+     }
+@@ -776,11 +785,14 @@ int ObjBencher::rand_read_bench(int seconds_to_run, int num_objects, int concurr
        }
        lc.cond.Wait(lock);
      }

http://tracker.ceph.com/issues/14873 Fixes: #14873

Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>
9 years agohammer: tools: fix race condition in seq/rand bench (part 1) 7896/head
Alexey Sheplyakov [Thu, 3 Mar 2016 12:30:23 +0000 (15:30 +0300)]
hammer: tools: fix race condition in seq/rand bench (part 1)

src/common/obj_bencher.cc:601: the lock should be taken before calling completion_ret,
not after. Also note that if r < 0 the lock will be unlocked twice in a row.
As a result rados bench seq fails with assertion in Mutex::Unlock().

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>
(cherry picked from commit 0c8faf7c9982c564002771c3a41362a833ace9bb)

Conflicts:
src/common/obj_bencher.cc
src/common/obj_bencher.h
  Pick only the lock related part to unbreak seq bench. The failure due
  to the missing (or wrong sized) objects can be easily worked around, and
  the changes required to fix this problem are way too intrusive for hammer.

http://tracker.ceph.com/issues/14873 Related: #14873

9 years agoclient: use fuse_req_getgroups() to get group list 6604/head
Yan, Zheng [Tue, 29 Sep 2015 07:24:41 +0000 (15:24 +0800)]
client: use fuse_req_getgroups() to get group list

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 0eb6d0ba1b65ac0aaffc72a82f07857cc65a6b88)

9 years agoclient: use thread local data to track fuse request
Yan, Zheng [Tue, 29 Sep 2015 07:11:24 +0000 (15:11 +0800)]
client: use thread local data to track fuse request

When handling an operation, libcephfs code may want to access fuse
request for extra information. by tracking fuse request in thread
local data, we can avoid adding extra parameter to Client::ll_foo
functions.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit a9f91d7d70893cdc5f832a26a5be298941d00315)

9 years agoclient/Client.cc: remove only once used variable
Danny Al-Gaaf [Tue, 18 Aug 2015 10:34:01 +0000 (12:34 +0200)]
client/Client.cc: remove only once used variable

Fix for:
[src/client/Client.cc:4555]: (style) The scope of the variable
 'initial_group_count' can be reduced.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit a29dd45dd89f59ff15018f541601ac5ede162174)

9 years agoclient/Client.cc: fix realloc memory leak
Danny Al-Gaaf [Wed, 12 Aug 2015 16:38:38 +0000 (18:38 +0200)]
client/Client.cc: fix realloc memory leak

Fix handling of realloc. If realloc() fails it returns NULL, assigning
the return value of realloc() directly to the pointer without checking
for the result will lead to a memory leak.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 4f98dab99c35663de89a06e2dfdbd874f56aed41)

9 years agoclient: added permission check based on getgrouplist
blinke [Mon, 10 Aug 2015 15:43:10 +0000 (17:43 +0200)]
client: added permission check based on getgrouplist

Fixes: #13268
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit f04c8da5432174874ca97d11a5b2fef56f95d73d)

9 years agoconfigure.ac: added autoconf check for getgrouplist
blinke [Mon, 10 Aug 2015 13:31:44 +0000 (15:31 +0200)]
configure.ac: added autoconf check for getgrouplist

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 16b59c6cd68d532a47e8c0e7b1f088b41433f048)

9 years agoinit-ceph: check if /lib/lsb/init-functions exists 7797/head
Yan, Zheng [Thu, 23 Jul 2015 07:07:45 +0000 (15:07 +0800)]
init-ceph: check if /lib/lsb/init-functions exists

On OSX/FreeBSD, /lib/lsb/init-functions does not exist

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit bb1fa7f3993dfcd1243b2a9a0bb432c269df215f)

9 years agoMerge pull request #7671 from tchaikov/wip-14583-hammer
Loic Dachary [Thu, 3 Mar 2016 05:05:58 +0000 (12:05 +0700)]
Merge pull request #7671 from tchaikov/wip-14583-hammer

hammer: global/pidfile: do not start two daemons with a single pid-file

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7702 from ceph/wip-14077-hammer
Loic Dachary [Thu, 3 Mar 2016 04:51:33 +0000 (11:51 +0700)]
Merge pull request #7702 from ceph/wip-14077-hammer

hammer: scrub bogus results when missing a clone

Reviewed-by: Martin Palma <martin@palma.bz>
Reviewed-by: Wei-Chung Cheng <freeze.vicente.cheng@gmail.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agopackaging: lsb_release build and runtime dependency 7876/head
Nathan Cutler [Fri, 26 Feb 2016 17:30:49 +0000 (18:30 +0100)]
packaging: lsb_release build and runtime dependency

The lsb_release executable is being run in multiple places, not least in
src/common/util.cc, which calls it via shell in the collect_sys_info() code
path.

This patch addresses this issue on SUSE- and Debian-derivatives, as well
as reinstating the dependency for RHEL/Fedora after it was dropped in
15600572265bed397fbd80bdd2b7d83a0e9bd918.

http://tracker.ceph.com/issues/14906 Fixes: #14906

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit b87f4efb5689a8db77e06baf4f469afcd9df3677)

Conflicts:
ceph.spec.in
             The jewel specfile has diverged considerably from hammer:
             systemd, package split, etc. This is more of a hand backport
             than a cherry-pick.

9 years agoglobal: do not start two daemons with a single pid-file (part 2) 7671/head
Loic Dachary [Mon, 1 Feb 2016 12:32:13 +0000 (19:32 +0700)]
global: do not start two daemons with a single pid-file (part 2)

Fixes the following bugs:

* the fd is open(O_WRONLY) and cannot be read from, safe_read
  always fails and never removes the pid file.

* pidfile_open(g_conf) is close(STDOUT_FILENO) and there is a risk that
  pidfile_open gets STDOUT_FILENO only to have it closed and redirected
  to /dev/null.

* Before writing the file, ftruncate it so that overriding a file
  containing the pid 1234 with the pid 89 does not end up being
  a file with 8934.

* Before reading the file, lseek back to offset 0 otherwise it
  will read nothing.

* tests_pidfile was missing an argument when failing
  TEST_without_pidfile and killed all process with ceph in their name,
  leading to chaos and no useful error message.

* lstat(fd) cannot possibly return a result different from the one
  obtained right after the file was open, stat(path) must be used
  instead.

In addition to fixing the bugs above, refactor the pidfile.cc
implementation to:

* be systematic about error reporting (using cerr for when removing
  the pidfile because derr is not available at this point and derr
  when creating the pidfile).

* replace pidfile_open / pidfile_write with just pidfile_write since
  there never is a case when they are not used together.

More test cases are added to test_pidfile to verify the bugs above are
fixed.

http://tracker.ceph.com/issues/13422 Fixes: #13422

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 9828d49d6f3ccfc78d496153d263ea39b1722d4b)

Conflicts:
src/global/global_init.cc
      - the `flag` argument of `global_init_prefork()` is not used, so
        it was removed in master. but the cleanup commit was not
        cherry-picked to hammer, thus the conflict. we can just keep it
        around in hammer to minimize the code churn, although it may
        stand in the way of future backports.)
      - s/nullptr/NULL/ as hammer does not support c++11.

9 years agoMerge backport pull request #6545 into wip-14077-hammer 7702/head
David Zafman [Thu, 25 Feb 2016 22:25:48 +0000 (14:25 -0800)]
Merge backport pull request #6545 into wip-14077-hammer

A commit skipped because remove_coll() still needed in Hammer

9 years agoMerge backport pull request #5783 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:25:35 +0000 (14:25 -0800)]
Merge backport pull request #5783 into wip-14077-hammer

9 years agoMerge backport pull request #5031 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:25:17 +0000 (14:25 -0800)]
Merge backport pull request #5031 into wip-14077-hammer

9 years agoMerge backport pull request #5842 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:24:58 +0000 (14:24 -0800)]
Merge backport pull request #5842 into wip-14077-hammer

9 years agoMerge backport pull request #5127 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:24:31 +0000 (14:24 -0800)]
Merge backport pull request #5127 into wip-14077-hammer

9 years agoMerge backport 6 commits from pull request #5197 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:23:14 +0000 (14:23 -0800)]
Merge backport 6 commits from pull request #5197 into wip-14077-hammer

Excludes the ceph-objectstore-tool meta-list op

9 years agoMerge backport branch 'wip-temp' into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:22:26 +0000 (14:22 -0800)]
Merge backport branch 'wip-temp' into wip-14077-hammer

9 years agoMerge backport pull request #4932 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:21:20 +0000 (14:21 -0800)]
Merge backport pull request #4932 into wip-14077-hammer

9 years agoMerge backport pull request #4915 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:20:41 +0000 (14:20 -0800)]
Merge backport pull request #4915 into wip-14077-hammer

9 years agoMerge backport 1 commit from pull request #4863 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:20:00 +0000 (14:20 -0800)]
Merge backport 1 commit from pull request #4863 into wip-14077-hammer

9 years agoMerge backport 8 commits from pull request #4784 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:18:38 +0000 (14:18 -0800)]
Merge backport 8 commits from pull request #4784 into wip-14077-hammer

Only excludes actual fix for #10794

9 years agoMerge backport 41 commits from pull request #4473 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:15:05 +0000 (14:15 -0800)]
Merge backport 41 commits from pull request #4473 into wip-14077-hammer

9 years agoMerge backport branch 'wip-journal-header' of git://github.com/XinzeChi/ceph into...
David Zafman [Thu, 25 Feb 2016 22:13:37 +0000 (14:13 -0800)]
Merge backport branch 'wip-journal-header' of git://github.com/XinzeChi/ceph into wip-14077-hammer

Includes a change to the code in the original merge commit itself

9 years agoMerge backport 1 commit of pull request #3686 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:13:02 +0000 (14:13 -0800)]
Merge backport 1 commit of pull request #3686 into wip-14077-hammer

9 years agoceph-objectstore-tool: Add dry-run checking to ops missing it
David Zafman [Fri, 6 Nov 2015 03:06:17 +0000 (19:06 -0800)]
ceph-objectstore-tool: Add dry-run checking to ops missing it

Update test to check most operations

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e89f0e1c5e8a378809bec7c705ea2c23235d4b28)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotest: Remove redundant test output
David Zafman [Wed, 11 Nov 2015 23:26:52 +0000 (15:26 -0800)]
test: Remove redundant test output

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e6f1bdcd310bd6417f0f695fba3430fd0edba43d)

9 years agotest: osd-scrub-snaps.sh uses ceph-helpers.sh and added to make check
David Zafman [Thu, 29 Oct 2015 04:34:44 +0000 (21:34 -0700)]
test: osd-scrub-snaps.sh uses ceph-helpers.sh and added to make check

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d712737be1aadb38ecc3882200999d79d748cdb8)

ceph-helpers.sh in different location in Hammer

9 years agotest: Verify replicated PG beyond just data after vstart
David Zafman [Wed, 11 Nov 2015 02:20:46 +0000 (18:20 -0800)]
test: Verify replicated PG beyond just data after vstart

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9a29c5983cd392e4b479b6b685fb0c3797c10cd2)

9 years agoosd: Use boost::optional instead of snap 0 for "all_clones"
David Zafman [Mon, 26 Oct 2015 18:36:43 +0000 (11:36 -0700)]
osd: Use boost::optional instead of snap 0 for "all_clones"

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 310bf784b7cd8d75f2ab7ee431bc764f5b312aa2)

9 years agotest: Fix verify() used after import to also check xattr and omap
David Zafman [Wed, 11 Nov 2015 01:57:59 +0000 (17:57 -0800)]
test: Fix verify() used after import to also check xattr and omap

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 92d00f22805599c519806c6ff6164847b5c33931)

9 years agoosd, test: When head missing a snapset, clones not an error
David Zafman [Mon, 26 Oct 2015 01:57:18 +0000 (18:57 -0700)]
osd, test: When head missing a snapset, clones not an error

Fix log message for this case and make into "info"

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f508ddc6e9b40fe5c1c54e5faa569f2bc7b78c4b)

9 years agotest: Add test cases for xattr and omap ceph-objectstore-tool operations
David Zafman [Tue, 10 Nov 2015 04:02:21 +0000 (20:02 -0800)]
test: Add test cases for xattr and omap ceph-objectstore-tool operations

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit cbaed6a8c737f77e68a0eb7f1fca32ec4155f045)

9 years agoosd, test: Keep missing count and log number of missing clones
David Zafman [Sat, 24 Oct 2015 23:23:13 +0000 (16:23 -0700)]
osd, test: Keep missing count and log number of missing clones

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2e0bb0ae9498fd9a0c49f1b81bf0c86f7a7cac42)

Conflicts:
src/osd/ReplicatedPG.h (trivial)

9 years agorados: Minor output changes for consistency across operations
David Zafman [Wed, 11 Nov 2015 16:28:01 +0000 (08:28 -0800)]
rados: Minor output changes for consistency across operations

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 016ed34ce85fa111f54857a8ca1002ae32b52b8e)

9 years agotest: Eliminate check for bogus "obj13/head snaps empty" error
David Zafman [Tue, 20 Oct 2015 18:14:42 +0000 (11:14 -0700)]
test: Eliminate check for bogus "obj13/head snaps empty" error

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 8227b4b75c24b632e637decbe4b818606e8589ee)

9 years agoceph-objectstore-tool: Add new remove-clone-metadata object op
David Zafman [Thu, 15 Oct 2015 03:34:28 +0000 (20:34 -0700)]
ceph-objectstore-tool: Add new remove-clone-metadata object op

Fixes: #12740
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c6d283f0f16347a13418d8a0ed136913e1161893)

No Sequencer in transactions in Hammer

9 years agoosd: Fix trim_object() to not crash on corrupt snapset
David Zafman [Thu, 15 Oct 2015 03:42:48 +0000 (20:42 -0700)]
osd: Fix trim_object() to not crash on corrupt snapset

Add test case to check for trim_object() osd crash

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit eb0ca424815e94c78a2d09dbf787d102172f4ddf)

Conflicts:
src/osd/ReplicatedPG.cc (trivial)

9 years agoceph-objectstore-tool: Improve object spec error handling
David Zafman [Wed, 14 Oct 2015 20:12:17 +0000 (13:12 -0700)]
ceph-objectstore-tool: Improve object spec error handling

Add test cases

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit caf2d5970b0a3e3ce4ef941d1955b7821a0fc916)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)

9 years agoceph-objectstore-tool: Add undocumented clear-snapset command for testing
David Zafman [Thu, 8 Oct 2015 00:25:44 +0000 (17:25 -0700)]
ceph-objectstore-tool: Add undocumented clear-snapset command for testing

Add test cases using new feature to corrupt SnapSet

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b6302acdd344997aabdb426104de9f24e9990437)

No Sequencer in transactions in Hammer

9 years agoceph-objectstore-tool: Add set-size command for objects
David Zafman [Thu, 15 Oct 2015 04:59:50 +0000 (21:59 -0700)]
ceph-objectstore-tool: Add set-size command for objects

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 138a33b8946e6fea1e09527b8bc45b7aa4bee1d6)

No Sequencer in transactions in Hammer

9 years agoceph-objectstore-tool: Enhanced dump command replaces dump-info
David Zafman [Thu, 15 Oct 2015 05:04:05 +0000 (22:04 -0700)]
ceph-objectstore-tool: Enhanced dump command replaces dump-info

Show object stat information from objectstore
Show SnapSet if present
Add --head for search by object name

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1688debf7892d36c773c12d7d0b1b9b4cddc5f98)

9 years agotest: Add some clones to ceph-objectstore-tool test
David Zafman [Fri, 2 Oct 2015 20:39:29 +0000 (13:39 -0700)]
test: Add some clones to ceph-objectstore-tool test

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b4ba3e66548a5dee58553df071b916c85d701caf)

9 years agoceph-objectstore-tool: For corrupt objectstores, don't abort listing on errors
David Zafman [Fri, 2 Oct 2015 02:28:54 +0000 (19:28 -0700)]
ceph-objectstore-tool: For corrupt objectstores, don't abort listing on errors

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d276d3286504ea702ae561b91dff4121c99feb9a)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agoceph-objectstore-tool: Improve some error messages
David Zafman [Tue, 13 Oct 2015 20:02:40 +0000 (13:02 -0700)]
ceph-objectstore-tool: Improve some error messages

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b0c884ba8b51eedf9def093c61f0f06398101ecc)

9 years agoceph-objectstore-tool: White space fixes
David Zafman [Thu, 3 Sep 2015 21:22:20 +0000 (14:22 -0700)]
ceph-objectstore-tool: White space fixes

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0564f398269d920d90f6d6ecb008ebcd6609d611)

9 years agotools/rados: Improve xattr import handling so future internal xattrs ignored
David Zafman [Tue, 13 Oct 2015 22:33:26 +0000 (15:33 -0700)]
tools/rados: Improve xattr import handling so future internal xattrs ignored

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9222f56403cf52ec161a38fc437a3f04c24d1625)

Conflicts:
src/tools/rados/RadosImport.cc (Code in ceph_objectstore_tool.cc in Hammer)

9 years agotest: Test scrubbing of snapshot problems
David Zafman [Wed, 30 Sep 2015 02:57:43 +0000 (19:57 -0700)]
test: Test scrubbing of snapshot problems

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e5ad33ec07ee9fffe862d84955a566261d101713)

9 years agoosd: Don't crash if OI_ATTR attribute is missing or corrupt
David Zafman [Mon, 28 Sep 2015 17:59:42 +0000 (10:59 -0700)]
osd: Don't crash if OI_ATTR attribute is missing or corrupt

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e0b39650028205e9d9e314b75c444cc8cf055a31)

9 years agoosd: Additional _scrub() check for snapset inconsistency
David Zafman [Wed, 14 Oct 2015 01:02:32 +0000 (18:02 -0700)]
osd: Additional _scrub() check for snapset inconsistency

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9e48e1801ea1a10ce495b3947da06676c2f5f72f)

9 years agoosd: Better SnapSet scrub checking (find issues instead of asserting)
David Zafman [Fri, 4 Sep 2015 02:43:35 +0000 (19:43 -0700)]
osd: Better SnapSet scrub checking (find issues instead of asserting)

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3b381caaad20c683a330e8b7a4e1c017abcb60df)

9 years agoosd: Make the _scrub routine produce good output and detect errors properly
David Zafman [Wed, 26 Aug 2015 20:58:09 +0000 (13:58 -0700)]
osd: Make the _scrub routine produce good output and detect errors properly

Catch decode errors so osd doesn't crash on corrupt OI_ATTR or SS_ATTR
Use boost::optional<> to make current state clearer
Create next_clone as needed using head/curclone
Add equivalent logic after getting to end of scrubmap.objects

Fixes: #12738
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a23036c6fd7de5d1dbc2bd30c967c0be51d94ca5)

Conflicts:
src/osd/ReplicatedPG.cc (no num_objects_pinned in hammer)
src/osd/ReplicatedPG.h (no get_temp_recovery_object() in hammer)

9 years agoosd: Fix log message name of ceph-objectstore-tool
David Zafman [Thu, 1 Oct 2015 00:17:15 +0000 (17:17 -0700)]
osd: Fix log message name of ceph-objectstore-tool

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 545e4b2c582625f8b93502266ca59ee40e0d0e5f)

9 years agoceph-objectstore-tool: add mark-complete operation
Mykola Golub [Tue, 16 Jun 2015 08:57:08 +0000 (11:57 +0300)]
ceph-objectstore-tool: add mark-complete operation

It is supposed to be used as a last resort to fix a cluster that has
PGs in 'incomplete' state, using the following procedure:

1) stop the osd that is primary for the incomplete PG;
2) run:
  ceph-objectstore-tool --data-path ... --journal-path ... --pgid $PGID --op mark-complete
3) start the osd.

Fixes: #10098
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 6907778d767ba08bb80c495785056ed122b023fe)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotest: Fix failure test to find message anywhere in stderr
David Zafman [Tue, 8 Sep 2015 19:33:44 +0000 (12:33 -0700)]
test: Fix failure test to find message anywhere in stderr

Consolidate test_failure() and test_failure_tty()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 95bd3c2c54312417b132ddb91c89fdbe63b01fd0)

9 years agotest: add test for {get,set}-inc-osdmap commands.
Kefu Chai [Fri, 17 Jul 2015 06:46:16 +0000 (14:46 +0800)]
test: add test for {get,set}-inc-osdmap commands.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 818de1a2c89272d98610328a958fea06d1cc41cc)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)

9 years agotest: Add debug argument to the ceph-objectstore-tool test
David Zafman [Tue, 23 Jun 2015 23:51:27 +0000 (16:51 -0700)]
test: Add debug argument to the ceph-objectstore-tool test

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a3b14ede9e829ec85c49980ecbff11f584ca70ae)

9 years agorados: Fix usage for "notify" command
David Zafman [Tue, 8 Sep 2015 20:41:31 +0000 (13:41 -0700)]
rados: Fix usage for "notify" command

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b968fb3bcefb0cdafb1e84d7c679b61a201a940f)

9 years agotest: add test for {get,set}-osdmap commands
Kefu Chai [Wed, 8 Jul 2015 07:51:55 +0000 (15:51 +0800)]
test: add test for {get,set}-osdmap commands

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0257c150fe552300c0d9f5a22a27aaa49fb6cc64)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)

9 years agotools, test: Some ceph-objectstore-tool error handling fixes
David Zafman [Fri, 26 Jun 2015 00:15:39 +0000 (17:15 -0700)]
tools, test: Some ceph-objectstore-tool error handling fixes

Improve various error messages generated with invalid syntax
Add test cases for most of these error messages

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 127a5f191d3d2442edd3a3fa820006bca3a08c93)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)

9 years agotools/ceph-objectstore-tool: add get-inc-osdmap command
Kefu Chai [Tue, 14 Jul 2015 02:33:47 +0000 (10:33 +0800)]
tools/ceph-objectstore-tool: add get-inc-osdmap command

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7801bb20d23ce83fc4093c23331bcb75e43c878f)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotools: Check for valid --op earlier so we can get a better error message
David Zafman [Tue, 23 Jun 2015 03:11:10 +0000 (20:11 -0700)]
tools: Check for valid --op earlier so we can get a better error message

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d8460714fab61d0a3092b3e7b290c0c19ad58beb)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)
src/tools/ceph_objectstore_tool.cc (meta-list op not in Hammer)

9 years agotools/ceph-objectstore-tool: add set-inc-osdmap command
Kefu Chai [Sat, 11 Jul 2015 08:27:47 +0000 (16:27 +0800)]
tools/ceph-objectstore-tool: add set-inc-osdmap command

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 95344d07c3a1a9a6b1246e5eca3f7daeb062dd06)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotools: Fix newlines in output of --op list
David Zafman [Thu, 11 Jun 2015 00:04:57 +0000 (17:04 -0700)]
tools: Fix newlines in output of --op list

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 6aec38b51f0c14684912b5ee28fb5b1375cb8f8a)

9 years agotools/ceph-objectstore-tool: add "get-osdmap" command
Kefu Chai [Fri, 3 Jul 2015 15:09:54 +0000 (23:09 +0800)]
tools/ceph-objectstore-tool: add "get-osdmap" command

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f0c130d6b1bc2a8a0cce6d78a6e69995fe00a427)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotools: Fix dump-super which doesn't require pgid
David Zafman [Tue, 23 Jun 2015 01:38:01 +0000 (18:38 -0700)]
tools: Fix dump-super which doesn't require pgid

A valid pgid had to be specified in order for superblock to be displayed,
but it was otherwise ignored.
Fix message that didn't include dump-super

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fd1772e0ac4cb801d6f3b056645eeb4d3d6ef001)

9 years agotools/ceph-objectstore-tool: add "set-osdmap" command
Kefu Chai [Thu, 2 Jul 2015 11:14:55 +0000 (19:14 +0800)]
tools/ceph-objectstore-tool: add "set-osdmap" command

Fixes: #12193
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3e30c1746fb8d90b04e4776849069db0b7737c87)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotools: Check and specify commands that require the pgid specification
David Zafman [Tue, 23 Jun 2015 02:16:22 +0000 (19:16 -0700)]
tools: Check and specify commands that require the pgid specification

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 5534bc864813d96a0d20791d79151c4f6695a013)

9 years agohobject_t: modify operator<<
Sage Weil [Fri, 12 Dec 2014 00:25:49 +0000 (16:25 -0800)]
hobject_t: modify operator<<

Put the most significant fields to the left so that it matches the sort
order.  Also use unambiguous separator when the nspace is present
(like we do with the key).

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ecdc8f697acf1627b35a83663c219a3bb0ada453)

9 years agotest, tools: Improve ceph-objectstore-tool import error handling and add tests
David Zafman [Thu, 11 Jun 2015 19:57:10 +0000 (12:57 -0700)]
test, tools: Improve ceph-objectstore-tool import error handling and add tests

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ddc4d52782a04d99a4293c9fc278894dfe328515)

9 years agotools: For ec pools list objects in all shards if the pgid doesn't specify
David Zafman [Wed, 10 Jun 2015 01:14:09 +0000 (18:14 -0700)]
tools: For ec pools list objects in all shards if the pgid doesn't specify

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1aa9655f10b0fcc42cb5114681b8b24a2ccb68f4)

9 years agotools: clean up errors in ceph-objectstore-tool
John Spray [Mon, 20 Apr 2015 16:28:15 +0000 (17:28 +0100)]
tools: clean up errors in ceph-objectstore-tool

We use negative error codes everywhere else, should
use them here too.  Also add a couple of strategically
placed prints.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 8e5594da29e782b6efd046ed03f313599dafa1c5)

Conflicts:
src/tools/RadosDump.cc (Excluded, not in Hammer)

9 years agotest/ceph-objectstore-tool: Don't need stderr noise
David Zafman [Tue, 2 Jun 2015 00:22:18 +0000 (17:22 -0700)]
test/ceph-objectstore-tool: Don't need stderr noise

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d7858c66454093dd6f2fab60e33e5b2472bcd80a)

9 years agoosd: Show number of divergent_priors in log message
David Zafman [Thu, 9 Apr 2015 23:26:07 +0000 (16:26 -0700)]
osd: Show number of divergent_priors in log message

Report dirty_divergent_priors as true or false
Use dout(5) so tests can verify divergent_priors created

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 67da8feb0ff8659f1ff812be5a866ef5e97facf2)

9 years agoosd, tools: Always filter temp objects since not being exported
David Zafman [Tue, 7 Jul 2015 01:14:06 +0000 (18:14 -0700)]
osd, tools: Always filter temp objects since not being exported

Can't use object_locator_to_pg() directly with a temp object
because the pool is negative.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 557c6530a515cffacbaec439cabff8a079edbf5b)

9 years agotest/ceph-objectstore-tool: Show command that should have failed
David Zafman [Tue, 2 Jun 2015 00:15:21 +0000 (17:15 -0700)]
test/ceph-objectstore-tool: Show command that should have failed

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 23551deea66673c35d7183d18b63b5a8438ebf00)

9 years agotest: Add config changes to all tests to avoid order dependency
David Zafman [Tue, 12 May 2015 22:28:07 +0000 (15:28 -0700)]
test: Add config changes to all tests to avoid order dependency

ReplayCorrupt was crashing because journal_ignore_corruption wasn't set true
Improve ReplayCorrupt by checking setting of bool corrupt and use ASSERT_FALSE()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9016269a1c248da69a0e85df4ca3ba95011710c5)

9 years agotools: Don't export temporary objects until we have persistent-temp objects
David Zafman [Mon, 22 Jun 2015 18:15:58 +0000 (11:15 -0700)]
tools: Don't export temporary objects until we have persistent-temp objects

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 4fcf5dd370e8cfac75fadbf709a785053eb26ad9)

Remove "assert(!i->hobj.is_meta());" since is_meta() is not in Hammer

9 years agotest/ceph_objectstore_tool: Improve dump-journal testing
David Zafman [Sat, 23 May 2015 00:41:40 +0000 (17:41 -0700)]
test/ceph_objectstore_tool: Improve dump-journal testing

Fix false failure when journal has no entries
Test dump-journal on all osds in 2 places

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ce76f88777ec026521cce465095d6376efd460a8)

9 years agoceph-objectstore-tool: Allow --pgid specified on import (for post split)
David Zafman [Fri, 22 May 2015 18:52:00 +0000 (11:52 -0700)]
ceph-objectstore-tool: Allow --pgid specified on import (for post split)

Revamp split import testing to use --pgid option post split

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 90202a70f8c32120625c99ef5131778b3c9f6303)

9 years agoceph-objectstore-tool: Invalidate pg stats when objects were skipped during pg import
David Zafman [Fri, 22 May 2015 17:53:48 +0000 (10:53 -0700)]
ceph-objectstore-tool: Invalidate pg stats when objects were skipped during pg import

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 51e27984ec131fd21bbc26411ae1540b9047eb37)

9 years agoceph-objectstore-tool: Add dump-super to show OSDSuperblock in format specified
David Zafman [Tue, 19 May 2015 03:22:03 +0000 (20:22 -0700)]
ceph-objectstore-tool: Add dump-super to show OSDSuperblock in format specified

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 99d7663f870abfa82711dbf0451334f7d6c883d8)

9 years agomds, include: Fix dump() numeric char array to include additional alpha chars
David Zafman [Tue, 19 May 2015 03:25:23 +0000 (20:25 -0700)]
mds, include: Fix dump() numeric char array to include additional alpha chars

Caused by: c87f55b5a8b3e003726e1be22b9cb6894ef2f37d

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f1520061bba1e052a9be3ae11161122c1f6c3425)

9 years agoceph-objectstore-tool: Add dump-journal as not requiring --pgid in usage
David Zafman [Tue, 19 May 2015 03:08:14 +0000 (20:08 -0700)]
ceph-objectstore-tool: Add dump-journal as not requiring --pgid in usage

Additional comment describes --pgid requirement checking

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bd2f62445be7153a9600bcf4399e8fc56f1efd3f)

9 years agotest: ceph_test_filejournal: Conform to test infrastructure requirements
David Zafman [Fri, 6 Mar 2015 00:37:46 +0000 (16:37 -0800)]
test: ceph_test_filejournal: Conform to test infrastructure requirements

Only call RUN_ALL_TESTS() once

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e30efe6c15780a6239b9970297df1f2c1918c9b1)