]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
9 years agotest/pybind/test_ceph_argparse: fix reweight-by-utilization tests
Sage Weil [Thu, 10 Mar 2016 13:50:57 +0000 (08:50 -0500)]
test/pybind/test_ceph_argparse: fix reweight-by-utilization tests

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d0f87131962b26cef9e31fa11651771856c41164)

9 years agoman/8/ceph.rst: remove invalid option for reweight-by-*
Kefu Chai [Thu, 10 Mar 2016 13:41:15 +0000 (21:41 +0800)]
man/8/ceph.rst: remove invalid option for reweight-by-*

--yes-i-really-mean-it is not valid option for reweight-by-*

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 797f1d41d98738411079a7a16d8b1d5352359d33)

9 years agomon: remove range=100 from reweight-by-* commands
Sage Weil [Wed, 9 Mar 2016 18:25:55 +0000 (13:25 -0500)]
mon: remove range=100 from reweight-by-* commands

Specifying the range makes it skip instead of error when you give
it a bad value.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 661891aa28ca0600a62e25f6d10ce7fa94e32cce)

9 years agomon: make max_osds an optional arg
Sage Weil [Thu, 3 Mar 2016 18:46:12 +0000 (13:46 -0500)]
mon: make max_osds an optional arg

We keep the config option with the default (4), but let you
specify the max # of osds to reweight as a third optional arg.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 651f05b6e6bec41b54c0a8f8ca6641ce3aa5dff3)

9 years agomon: make reweight max_change default configurable
Sage Weil [Thu, 3 Mar 2016 18:38:57 +0000 (13:38 -0500)]
mon: make reweight max_change default configurable

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8a9429d85bc99874dc9871b43b2d0c7ad1086803)

9 years agomon/OSDMonitor: fix indentation
Sage Weil [Thu, 3 Mar 2016 18:38:19 +0000 (13:38 -0500)]
mon/OSDMonitor: fix indentation

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 47b650c1497b8eaf37cfcb7e3b9fda12d8cf2d7a)

9 years agoqa/workunits/cephtool/test.sh: test reweight-by-x commands
Sage Weil [Wed, 2 Mar 2016 22:34:48 +0000 (17:34 -0500)]
qa/workunits/cephtool/test.sh: test reweight-by-x commands

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 28b33a5b57e3f898808bd8ccc5e314667f84912a)

9 years agoosd/MonCommand: add/fix up 'osd [test-]reweight-by-{pg,utilization}'
Sage Weil [Thu, 10 Mar 2016 13:28:59 +0000 (08:28 -0500)]
osd/MonCommand: add/fix up 'osd [test-]reweight-by-{pg,utilization}'

- show before/after pg placement stats
- add test- variants that don't do anything
- only allow --no-increasing on the -utilization versions (where
  it won't conflict with the optional pool list and confuse the
  arg parsing)

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1de7e47833f2d14c6915d7d071fe0005408c8da0)

- use non-C++11 sort

# Conflicts:
# src/mon/OSDMonitor.cc
# src/mon/OSDMonitor.h

9 years agomon: add 'osd utilization' command
Sage Weil [Wed, 2 Mar 2016 19:00:15 +0000 (14:00 -0500)]
mon: add 'osd utilization' command

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5b757eff868dbce2393beadd9fe2522372288cc0)

9 years agoosd/OSDMap: add summarize_mapping_stats
Sage Weil [Thu, 10 Mar 2016 13:30:45 +0000 (08:30 -0500)]
osd/OSDMap: add summarize_mapping_stats

Calculate some basic stats about the PG distribution.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ea9abe53d0e777b7dc3b22af71639f77c4de08c8)

- remove c++11 auto and range-based for loops

9 years agomon: make reweight-by-* max_change an argument
Sage Weil [Thu, 3 Mar 2016 18:49:03 +0000 (13:49 -0500)]
mon: make reweight-by-* max_change an argument

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a70eaabcfc1cc1dfb9b6ba2d7f71b3d76496eb55)

9 years agoosd: add mon_reweight_max_osds to limit reweight-by-* commands
Dan van der Ster [Fri, 26 Feb 2016 21:29:46 +0000 (22:29 +0100)]
osd: add mon_reweight_max_osds to limit reweight-by-* commands

Add configurable mon_reweight_max_osds which limits the number
of OSDs modified each time reweight-by-* is called (by default
to 4 OSDs).

Also change the order in which we look at OSDs to go from most
to least utilized.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 3aa0ccefddc1790df5b808bfccd5da4857951220)

9 years agoosd: add mon_reweight_max_change option which limits reweight-by-*
Dan van der Ster [Fri, 26 Feb 2016 21:28:46 +0000 (22:28 +0100)]
osd: add mon_reweight_max_change option which limits reweight-by-*

Add a configurable mon_reweight_max_change which limits be default
the reweight-by-* commands to only change an OSDs weight by 0.05.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 6f0a1fb69da84ee18bdb9b151772cef8f32413c5)

9 years agotest: add simple test for new reweight-by-* options
Dan van der Ster [Fri, 26 Feb 2016 20:58:00 +0000 (21:58 +0100)]
test: add simple test for new reweight-by-* options

Test validity of --no-increasing and --yes-i-really-mean-it.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 1a6ad5053e7fc4f8942855a17ce4540e340e9d4f)

9 years agoosd: add sure and no-increasing options to reweight-by-*
Dan van der Ster [Fri, 26 Feb 2016 20:52:41 +0000 (21:52 +0100)]
osd: add sure and no-increasing options to reweight-by-*

Add a --no-increasing option to reweight-by-* which can be used to only decrease
OSD weights without increasing any. This is useful for example if you need to
urgently lower the weight of nearly full OSDs.

Also add a --yes-i-really-mean-it confirmation to reweight-by-*.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit ddf5c2b62316bb69839f220fe2527d1467863421)

9 years agoMerge pull request #6604 from ukernel/hammer-13268
Loic Dachary [Tue, 8 Mar 2016 03:00:15 +0000 (10:00 +0700)]
Merge pull request #6604 from ukernel/hammer-13268

hammer: client: added permission check based on getgrouplist

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #6551 from ceph/hammer-client-inoderef
Loic Dachary [Tue, 8 Mar 2016 02:58:30 +0000 (09:58 +0700)]
Merge pull request #6551 from ceph/hammer-client-inoderef

hammer: client inoderef

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #7797 from SUSE/wip-14861-hammer
Loic Dachary [Fri, 4 Mar 2016 03:25:00 +0000 (10:25 +0700)]
Merge pull request #7797 from SUSE/wip-14861-hammer

hammer: ceph init script unconditionally sources /lib/lsb/init-functions

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
9 years agoMerge pull request #7672 from dachary/wip-14783-hammer
Sage Weil [Thu, 3 Mar 2016 13:35:37 +0000 (08:35 -0500)]
Merge pull request #7672 from dachary/wip-14783-hammer

hammer: test_bit_vector.cc uses magic numbers against #defines that vary

9 years agoclient: use fuse_req_getgroups() to get group list 6604/head
Yan, Zheng [Tue, 29 Sep 2015 07:24:41 +0000 (15:24 +0800)]
client: use fuse_req_getgroups() to get group list

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 0eb6d0ba1b65ac0aaffc72a82f07857cc65a6b88)

9 years agoclient: use thread local data to track fuse request
Yan, Zheng [Tue, 29 Sep 2015 07:11:24 +0000 (15:11 +0800)]
client: use thread local data to track fuse request

When handling an operation, libcephfs code may want to access fuse
request for extra information. by tracking fuse request in thread
local data, we can avoid adding extra parameter to Client::ll_foo
functions.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit a9f91d7d70893cdc5f832a26a5be298941d00315)

9 years agoclient/Client.cc: remove only once used variable
Danny Al-Gaaf [Tue, 18 Aug 2015 10:34:01 +0000 (12:34 +0200)]
client/Client.cc: remove only once used variable

Fix for:
[src/client/Client.cc:4555]: (style) The scope of the variable
 'initial_group_count' can be reduced.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit a29dd45dd89f59ff15018f541601ac5ede162174)

9 years agoclient/Client.cc: fix realloc memory leak
Danny Al-Gaaf [Wed, 12 Aug 2015 16:38:38 +0000 (18:38 +0200)]
client/Client.cc: fix realloc memory leak

Fix handling of realloc. If realloc() fails it returns NULL, assigning
the return value of realloc() directly to the pointer without checking
for the result will lead to a memory leak.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 4f98dab99c35663de89a06e2dfdbd874f56aed41)

9 years agoclient: added permission check based on getgrouplist
blinke [Mon, 10 Aug 2015 15:43:10 +0000 (17:43 +0200)]
client: added permission check based on getgrouplist

Fixes: #13268
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit f04c8da5432174874ca97d11a5b2fef56f95d73d)

9 years agoconfigure.ac: added autoconf check for getgrouplist
blinke [Mon, 10 Aug 2015 13:31:44 +0000 (15:31 +0200)]
configure.ac: added autoconf check for getgrouplist

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 16b59c6cd68d532a47e8c0e7b1f088b41433f048)

9 years agoinit-ceph: check if /lib/lsb/init-functions exists 7797/head
Yan, Zheng [Thu, 23 Jul 2015 07:07:45 +0000 (15:07 +0800)]
init-ceph: check if /lib/lsb/init-functions exists

On OSX/FreeBSD, /lib/lsb/init-functions does not exist

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit bb1fa7f3993dfcd1243b2a9a0bb432c269df215f)

9 years agoMerge pull request #7671 from tchaikov/wip-14583-hammer
Loic Dachary [Thu, 3 Mar 2016 05:05:58 +0000 (12:05 +0700)]
Merge pull request #7671 from tchaikov/wip-14583-hammer

hammer: global/pidfile: do not start two daemons with a single pid-file

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7702 from ceph/wip-14077-hammer
Loic Dachary [Thu, 3 Mar 2016 04:51:33 +0000 (11:51 +0700)]
Merge pull request #7702 from ceph/wip-14077-hammer

hammer: scrub bogus results when missing a clone

Reviewed-by: Martin Palma <martin@palma.bz>
Reviewed-by: Wei-Chung Cheng <freeze.vicente.cheng@gmail.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoglobal: do not start two daemons with a single pid-file (part 2) 7671/head
Loic Dachary [Mon, 1 Feb 2016 12:32:13 +0000 (19:32 +0700)]
global: do not start two daemons with a single pid-file (part 2)

Fixes the following bugs:

* the fd is open(O_WRONLY) and cannot be read from, safe_read
  always fails and never removes the pid file.

* pidfile_open(g_conf) is close(STDOUT_FILENO) and there is a risk that
  pidfile_open gets STDOUT_FILENO only to have it closed and redirected
  to /dev/null.

* Before writing the file, ftruncate it so that overriding a file
  containing the pid 1234 with the pid 89 does not end up being
  a file with 8934.

* Before reading the file, lseek back to offset 0 otherwise it
  will read nothing.

* tests_pidfile was missing an argument when failing
  TEST_without_pidfile and killed all process with ceph in their name,
  leading to chaos and no useful error message.

* lstat(fd) cannot possibly return a result different from the one
  obtained right after the file was open, stat(path) must be used
  instead.

In addition to fixing the bugs above, refactor the pidfile.cc
implementation to:

* be systematic about error reporting (using cerr for when removing
  the pidfile because derr is not available at this point and derr
  when creating the pidfile).

* replace pidfile_open / pidfile_write with just pidfile_write since
  there never is a case when they are not used together.

More test cases are added to test_pidfile to verify the bugs above are
fixed.

http://tracker.ceph.com/issues/13422 Fixes: #13422

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 9828d49d6f3ccfc78d496153d263ea39b1722d4b)

Conflicts:
src/global/global_init.cc
      - the `flag` argument of `global_init_prefork()` is not used, so
        it was removed in master. but the cleanup commit was not
        cherry-picked to hammer, thus the conflict. we can just keep it
        around in hammer to minimize the code churn, although it may
        stand in the way of future backports.)
      - s/nullptr/NULL/ as hammer does not support c++11.

9 years agoMerge backport pull request #6545 into wip-14077-hammer 7702/head
David Zafman [Thu, 25 Feb 2016 22:25:48 +0000 (14:25 -0800)]
Merge backport pull request #6545 into wip-14077-hammer

A commit skipped because remove_coll() still needed in Hammer

9 years agoMerge backport pull request #5783 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:25:35 +0000 (14:25 -0800)]
Merge backport pull request #5783 into wip-14077-hammer

9 years agoMerge backport pull request #5031 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:25:17 +0000 (14:25 -0800)]
Merge backport pull request #5031 into wip-14077-hammer

9 years agoMerge backport pull request #5842 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:24:58 +0000 (14:24 -0800)]
Merge backport pull request #5842 into wip-14077-hammer

9 years agoMerge backport pull request #5127 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:24:31 +0000 (14:24 -0800)]
Merge backport pull request #5127 into wip-14077-hammer

9 years agoMerge backport 6 commits from pull request #5197 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:23:14 +0000 (14:23 -0800)]
Merge backport 6 commits from pull request #5197 into wip-14077-hammer

Excludes the ceph-objectstore-tool meta-list op

9 years agoMerge backport branch 'wip-temp' into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:22:26 +0000 (14:22 -0800)]
Merge backport branch 'wip-temp' into wip-14077-hammer

9 years agoMerge backport pull request #4932 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:21:20 +0000 (14:21 -0800)]
Merge backport pull request #4932 into wip-14077-hammer

9 years agoMerge backport pull request #4915 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:20:41 +0000 (14:20 -0800)]
Merge backport pull request #4915 into wip-14077-hammer

9 years agoMerge backport 1 commit from pull request #4863 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:20:00 +0000 (14:20 -0800)]
Merge backport 1 commit from pull request #4863 into wip-14077-hammer

9 years agoMerge backport 8 commits from pull request #4784 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:18:38 +0000 (14:18 -0800)]
Merge backport 8 commits from pull request #4784 into wip-14077-hammer

Only excludes actual fix for #10794

9 years agoMerge backport 41 commits from pull request #4473 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:15:05 +0000 (14:15 -0800)]
Merge backport 41 commits from pull request #4473 into wip-14077-hammer

9 years agoMerge backport branch 'wip-journal-header' of git://github.com/XinzeChi/ceph into...
David Zafman [Thu, 25 Feb 2016 22:13:37 +0000 (14:13 -0800)]
Merge backport branch 'wip-journal-header' of git://github.com/XinzeChi/ceph into wip-14077-hammer

Includes a change to the code in the original merge commit itself

9 years agoMerge backport 1 commit of pull request #3686 into wip-14077-hammer
David Zafman [Thu, 25 Feb 2016 22:13:02 +0000 (14:13 -0800)]
Merge backport 1 commit of pull request #3686 into wip-14077-hammer

9 years agoceph-objectstore-tool: Add dry-run checking to ops missing it
David Zafman [Fri, 6 Nov 2015 03:06:17 +0000 (19:06 -0800)]
ceph-objectstore-tool: Add dry-run checking to ops missing it

Update test to check most operations

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e89f0e1c5e8a378809bec7c705ea2c23235d4b28)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotest: Remove redundant test output
David Zafman [Wed, 11 Nov 2015 23:26:52 +0000 (15:26 -0800)]
test: Remove redundant test output

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e6f1bdcd310bd6417f0f695fba3430fd0edba43d)

9 years agotest: osd-scrub-snaps.sh uses ceph-helpers.sh and added to make check
David Zafman [Thu, 29 Oct 2015 04:34:44 +0000 (21:34 -0700)]
test: osd-scrub-snaps.sh uses ceph-helpers.sh and added to make check

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d712737be1aadb38ecc3882200999d79d748cdb8)

ceph-helpers.sh in different location in Hammer

9 years agotest: Verify replicated PG beyond just data after vstart
David Zafman [Wed, 11 Nov 2015 02:20:46 +0000 (18:20 -0800)]
test: Verify replicated PG beyond just data after vstart

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9a29c5983cd392e4b479b6b685fb0c3797c10cd2)

9 years agoosd: Use boost::optional instead of snap 0 for "all_clones"
David Zafman [Mon, 26 Oct 2015 18:36:43 +0000 (11:36 -0700)]
osd: Use boost::optional instead of snap 0 for "all_clones"

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 310bf784b7cd8d75f2ab7ee431bc764f5b312aa2)

9 years agotest: Fix verify() used after import to also check xattr and omap
David Zafman [Wed, 11 Nov 2015 01:57:59 +0000 (17:57 -0800)]
test: Fix verify() used after import to also check xattr and omap

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 92d00f22805599c519806c6ff6164847b5c33931)

9 years agoosd, test: When head missing a snapset, clones not an error
David Zafman [Mon, 26 Oct 2015 01:57:18 +0000 (18:57 -0700)]
osd, test: When head missing a snapset, clones not an error

Fix log message for this case and make into "info"

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f508ddc6e9b40fe5c1c54e5faa569f2bc7b78c4b)

9 years agotest: Add test cases for xattr and omap ceph-objectstore-tool operations
David Zafman [Tue, 10 Nov 2015 04:02:21 +0000 (20:02 -0800)]
test: Add test cases for xattr and omap ceph-objectstore-tool operations

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit cbaed6a8c737f77e68a0eb7f1fca32ec4155f045)

9 years agoosd, test: Keep missing count and log number of missing clones
David Zafman [Sat, 24 Oct 2015 23:23:13 +0000 (16:23 -0700)]
osd, test: Keep missing count and log number of missing clones

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2e0bb0ae9498fd9a0c49f1b81bf0c86f7a7cac42)

Conflicts:
src/osd/ReplicatedPG.h (trivial)

9 years agorados: Minor output changes for consistency across operations
David Zafman [Wed, 11 Nov 2015 16:28:01 +0000 (08:28 -0800)]
rados: Minor output changes for consistency across operations

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 016ed34ce85fa111f54857a8ca1002ae32b52b8e)

9 years agotest: Eliminate check for bogus "obj13/head snaps empty" error
David Zafman [Tue, 20 Oct 2015 18:14:42 +0000 (11:14 -0700)]
test: Eliminate check for bogus "obj13/head snaps empty" error

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 8227b4b75c24b632e637decbe4b818606e8589ee)

9 years agoceph-objectstore-tool: Add new remove-clone-metadata object op
David Zafman [Thu, 15 Oct 2015 03:34:28 +0000 (20:34 -0700)]
ceph-objectstore-tool: Add new remove-clone-metadata object op

Fixes: #12740
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c6d283f0f16347a13418d8a0ed136913e1161893)

No Sequencer in transactions in Hammer

9 years agoosd: Fix trim_object() to not crash on corrupt snapset
David Zafman [Thu, 15 Oct 2015 03:42:48 +0000 (20:42 -0700)]
osd: Fix trim_object() to not crash on corrupt snapset

Add test case to check for trim_object() osd crash

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit eb0ca424815e94c78a2d09dbf787d102172f4ddf)

Conflicts:
src/osd/ReplicatedPG.cc (trivial)

9 years agoceph-objectstore-tool: Improve object spec error handling
David Zafman [Wed, 14 Oct 2015 20:12:17 +0000 (13:12 -0700)]
ceph-objectstore-tool: Improve object spec error handling

Add test cases

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit caf2d5970b0a3e3ce4ef941d1955b7821a0fc916)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)

9 years agoceph-objectstore-tool: Add undocumented clear-snapset command for testing
David Zafman [Thu, 8 Oct 2015 00:25:44 +0000 (17:25 -0700)]
ceph-objectstore-tool: Add undocumented clear-snapset command for testing

Add test cases using new feature to corrupt SnapSet

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b6302acdd344997aabdb426104de9f24e9990437)

No Sequencer in transactions in Hammer

9 years agoceph-objectstore-tool: Add set-size command for objects
David Zafman [Thu, 15 Oct 2015 04:59:50 +0000 (21:59 -0700)]
ceph-objectstore-tool: Add set-size command for objects

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 138a33b8946e6fea1e09527b8bc45b7aa4bee1d6)

No Sequencer in transactions in Hammer

9 years agoceph-objectstore-tool: Enhanced dump command replaces dump-info
David Zafman [Thu, 15 Oct 2015 05:04:05 +0000 (22:04 -0700)]
ceph-objectstore-tool: Enhanced dump command replaces dump-info

Show object stat information from objectstore
Show SnapSet if present
Add --head for search by object name

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1688debf7892d36c773c12d7d0b1b9b4cddc5f98)

9 years agotest: Add some clones to ceph-objectstore-tool test
David Zafman [Fri, 2 Oct 2015 20:39:29 +0000 (13:39 -0700)]
test: Add some clones to ceph-objectstore-tool test

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b4ba3e66548a5dee58553df071b916c85d701caf)

9 years agoceph-objectstore-tool: For corrupt objectstores, don't abort listing on errors
David Zafman [Fri, 2 Oct 2015 02:28:54 +0000 (19:28 -0700)]
ceph-objectstore-tool: For corrupt objectstores, don't abort listing on errors

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d276d3286504ea702ae561b91dff4121c99feb9a)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agoceph-objectstore-tool: Improve some error messages
David Zafman [Tue, 13 Oct 2015 20:02:40 +0000 (13:02 -0700)]
ceph-objectstore-tool: Improve some error messages

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b0c884ba8b51eedf9def093c61f0f06398101ecc)

9 years agoceph-objectstore-tool: White space fixes
David Zafman [Thu, 3 Sep 2015 21:22:20 +0000 (14:22 -0700)]
ceph-objectstore-tool: White space fixes

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0564f398269d920d90f6d6ecb008ebcd6609d611)

9 years agotools/rados: Improve xattr import handling so future internal xattrs ignored
David Zafman [Tue, 13 Oct 2015 22:33:26 +0000 (15:33 -0700)]
tools/rados: Improve xattr import handling so future internal xattrs ignored

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9222f56403cf52ec161a38fc437a3f04c24d1625)

Conflicts:
src/tools/rados/RadosImport.cc (Code in ceph_objectstore_tool.cc in Hammer)

9 years agotest: Test scrubbing of snapshot problems
David Zafman [Wed, 30 Sep 2015 02:57:43 +0000 (19:57 -0700)]
test: Test scrubbing of snapshot problems

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e5ad33ec07ee9fffe862d84955a566261d101713)

9 years agoosd: Don't crash if OI_ATTR attribute is missing or corrupt
David Zafman [Mon, 28 Sep 2015 17:59:42 +0000 (10:59 -0700)]
osd: Don't crash if OI_ATTR attribute is missing or corrupt

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e0b39650028205e9d9e314b75c444cc8cf055a31)

9 years agoosd: Additional _scrub() check for snapset inconsistency
David Zafman [Wed, 14 Oct 2015 01:02:32 +0000 (18:02 -0700)]
osd: Additional _scrub() check for snapset inconsistency

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9e48e1801ea1a10ce495b3947da06676c2f5f72f)

9 years agoosd: Better SnapSet scrub checking (find issues instead of asserting)
David Zafman [Fri, 4 Sep 2015 02:43:35 +0000 (19:43 -0700)]
osd: Better SnapSet scrub checking (find issues instead of asserting)

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3b381caaad20c683a330e8b7a4e1c017abcb60df)

9 years agoosd: Make the _scrub routine produce good output and detect errors properly
David Zafman [Wed, 26 Aug 2015 20:58:09 +0000 (13:58 -0700)]
osd: Make the _scrub routine produce good output and detect errors properly

Catch decode errors so osd doesn't crash on corrupt OI_ATTR or SS_ATTR
Use boost::optional<> to make current state clearer
Create next_clone as needed using head/curclone
Add equivalent logic after getting to end of scrubmap.objects

Fixes: #12738
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a23036c6fd7de5d1dbc2bd30c967c0be51d94ca5)

Conflicts:
src/osd/ReplicatedPG.cc (no num_objects_pinned in hammer)
src/osd/ReplicatedPG.h (no get_temp_recovery_object() in hammer)

9 years agoosd: Fix log message name of ceph-objectstore-tool
David Zafman [Thu, 1 Oct 2015 00:17:15 +0000 (17:17 -0700)]
osd: Fix log message name of ceph-objectstore-tool

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 545e4b2c582625f8b93502266ca59ee40e0d0e5f)

9 years agoceph-objectstore-tool: add mark-complete operation
Mykola Golub [Tue, 16 Jun 2015 08:57:08 +0000 (11:57 +0300)]
ceph-objectstore-tool: add mark-complete operation

It is supposed to be used as a last resort to fix a cluster that has
PGs in 'incomplete' state, using the following procedure:

1) stop the osd that is primary for the incomplete PG;
2) run:
  ceph-objectstore-tool --data-path ... --journal-path ... --pgid $PGID --op mark-complete
3) start the osd.

Fixes: #10098
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 6907778d767ba08bb80c495785056ed122b023fe)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotest: Fix failure test to find message anywhere in stderr
David Zafman [Tue, 8 Sep 2015 19:33:44 +0000 (12:33 -0700)]
test: Fix failure test to find message anywhere in stderr

Consolidate test_failure() and test_failure_tty()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 95bd3c2c54312417b132ddb91c89fdbe63b01fd0)

9 years agotest: add test for {get,set}-inc-osdmap commands.
Kefu Chai [Fri, 17 Jul 2015 06:46:16 +0000 (14:46 +0800)]
test: add test for {get,set}-inc-osdmap commands.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 818de1a2c89272d98610328a958fea06d1cc41cc)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)

9 years agotest: Add debug argument to the ceph-objectstore-tool test
David Zafman [Tue, 23 Jun 2015 23:51:27 +0000 (16:51 -0700)]
test: Add debug argument to the ceph-objectstore-tool test

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a3b14ede9e829ec85c49980ecbff11f584ca70ae)

9 years agorados: Fix usage for "notify" command
David Zafman [Tue, 8 Sep 2015 20:41:31 +0000 (13:41 -0700)]
rados: Fix usage for "notify" command

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b968fb3bcefb0cdafb1e84d7c679b61a201a940f)

9 years agotest: add test for {get,set}-osdmap commands
Kefu Chai [Wed, 8 Jul 2015 07:51:55 +0000 (15:51 +0800)]
test: add test for {get,set}-osdmap commands

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0257c150fe552300c0d9f5a22a27aaa49fb6cc64)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)

9 years agotools, test: Some ceph-objectstore-tool error handling fixes
David Zafman [Fri, 26 Jun 2015 00:15:39 +0000 (17:15 -0700)]
tools, test: Some ceph-objectstore-tool error handling fixes

Improve various error messages generated with invalid syntax
Add test cases for most of these error messages

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 127a5f191d3d2442edd3a3fa820006bca3a08c93)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)

9 years agotools/ceph-objectstore-tool: add get-inc-osdmap command
Kefu Chai [Tue, 14 Jul 2015 02:33:47 +0000 (10:33 +0800)]
tools/ceph-objectstore-tool: add get-inc-osdmap command

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7801bb20d23ce83fc4093c23331bcb75e43c878f)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotools: Check for valid --op earlier so we can get a better error message
David Zafman [Tue, 23 Jun 2015 03:11:10 +0000 (20:11 -0700)]
tools: Check for valid --op earlier so we can get a better error message

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d8460714fab61d0a3092b3e7b290c0c19ad58beb)

Conflicts:
src/test/ceph_objectstore_tool.py (trivial)
src/tools/ceph_objectstore_tool.cc (meta-list op not in Hammer)

9 years agotools/ceph-objectstore-tool: add set-inc-osdmap command
Kefu Chai [Sat, 11 Jul 2015 08:27:47 +0000 (16:27 +0800)]
tools/ceph-objectstore-tool: add set-inc-osdmap command

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 95344d07c3a1a9a6b1246e5eca3f7daeb062dd06)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotools: Fix newlines in output of --op list
David Zafman [Thu, 11 Jun 2015 00:04:57 +0000 (17:04 -0700)]
tools: Fix newlines in output of --op list

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 6aec38b51f0c14684912b5ee28fb5b1375cb8f8a)

9 years agotools/ceph-objectstore-tool: add "get-osdmap" command
Kefu Chai [Fri, 3 Jul 2015 15:09:54 +0000 (23:09 +0800)]
tools/ceph-objectstore-tool: add "get-osdmap" command

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f0c130d6b1bc2a8a0cce6d78a6e69995fe00a427)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotools: Fix dump-super which doesn't require pgid
David Zafman [Tue, 23 Jun 2015 01:38:01 +0000 (18:38 -0700)]
tools: Fix dump-super which doesn't require pgid

A valid pgid had to be specified in order for superblock to be displayed,
but it was otherwise ignored.
Fix message that didn't include dump-super

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fd1772e0ac4cb801d6f3b056645eeb4d3d6ef001)

9 years agotools/ceph-objectstore-tool: add "set-osdmap" command
Kefu Chai [Thu, 2 Jul 2015 11:14:55 +0000 (19:14 +0800)]
tools/ceph-objectstore-tool: add "set-osdmap" command

Fixes: #12193
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3e30c1746fb8d90b04e4776849069db0b7737c87)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

9 years agotools: Check and specify commands that require the pgid specification
David Zafman [Tue, 23 Jun 2015 02:16:22 +0000 (19:16 -0700)]
tools: Check and specify commands that require the pgid specification

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 5534bc864813d96a0d20791d79151c4f6695a013)

9 years agohobject_t: modify operator<<
Sage Weil [Fri, 12 Dec 2014 00:25:49 +0000 (16:25 -0800)]
hobject_t: modify operator<<

Put the most significant fields to the left so that it matches the sort
order.  Also use unambiguous separator when the nspace is present
(like we do with the key).

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ecdc8f697acf1627b35a83663c219a3bb0ada453)

9 years agotest, tools: Improve ceph-objectstore-tool import error handling and add tests
David Zafman [Thu, 11 Jun 2015 19:57:10 +0000 (12:57 -0700)]
test, tools: Improve ceph-objectstore-tool import error handling and add tests

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ddc4d52782a04d99a4293c9fc278894dfe328515)

9 years agotools: For ec pools list objects in all shards if the pgid doesn't specify
David Zafman [Wed, 10 Jun 2015 01:14:09 +0000 (18:14 -0700)]
tools: For ec pools list objects in all shards if the pgid doesn't specify

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1aa9655f10b0fcc42cb5114681b8b24a2ccb68f4)

9 years agotools: clean up errors in ceph-objectstore-tool
John Spray [Mon, 20 Apr 2015 16:28:15 +0000 (17:28 +0100)]
tools: clean up errors in ceph-objectstore-tool

We use negative error codes everywhere else, should
use them here too.  Also add a couple of strategically
placed prints.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 8e5594da29e782b6efd046ed03f313599dafa1c5)

Conflicts:
src/tools/RadosDump.cc (Excluded, not in Hammer)

9 years agotest/ceph-objectstore-tool: Don't need stderr noise
David Zafman [Tue, 2 Jun 2015 00:22:18 +0000 (17:22 -0700)]
test/ceph-objectstore-tool: Don't need stderr noise

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d7858c66454093dd6f2fab60e33e5b2472bcd80a)

9 years agoosd: Show number of divergent_priors in log message
David Zafman [Thu, 9 Apr 2015 23:26:07 +0000 (16:26 -0700)]
osd: Show number of divergent_priors in log message

Report dirty_divergent_priors as true or false
Use dout(5) so tests can verify divergent_priors created

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 67da8feb0ff8659f1ff812be5a866ef5e97facf2)

9 years agoosd, tools: Always filter temp objects since not being exported
David Zafman [Tue, 7 Jul 2015 01:14:06 +0000 (18:14 -0700)]
osd, tools: Always filter temp objects since not being exported

Can't use object_locator_to_pg() directly with a temp object
because the pool is negative.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 557c6530a515cffacbaec439cabff8a079edbf5b)

9 years agotest/ceph-objectstore-tool: Show command that should have failed
David Zafman [Tue, 2 Jun 2015 00:15:21 +0000 (17:15 -0700)]
test/ceph-objectstore-tool: Show command that should have failed

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 23551deea66673c35d7183d18b63b5a8438ebf00)

9 years agotest: Add config changes to all tests to avoid order dependency
David Zafman [Tue, 12 May 2015 22:28:07 +0000 (15:28 -0700)]
test: Add config changes to all tests to avoid order dependency

ReplayCorrupt was crashing because journal_ignore_corruption wasn't set true
Improve ReplayCorrupt by checking setting of bool corrupt and use ASSERT_FALSE()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9016269a1c248da69a0e85df4ca3ba95011710c5)

9 years agotools: Don't export temporary objects until we have persistent-temp objects
David Zafman [Mon, 22 Jun 2015 18:15:58 +0000 (11:15 -0700)]
tools: Don't export temporary objects until we have persistent-temp objects

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 4fcf5dd370e8cfac75fadbf709a785053eb26ad9)

Remove "assert(!i->hobj.is_meta());" since is_meta() is not in Hammer

9 years agotest/ceph_objectstore_tool: Improve dump-journal testing
David Zafman [Sat, 23 May 2015 00:41:40 +0000 (17:41 -0700)]
test/ceph_objectstore_tool: Improve dump-journal testing

Fix false failure when journal has no entries
Test dump-journal on all osds in 2 places

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ce76f88777ec026521cce465095d6376efd460a8)

9 years agoceph-objectstore-tool: Allow --pgid specified on import (for post split)
David Zafman [Fri, 22 May 2015 18:52:00 +0000 (11:52 -0700)]
ceph-objectstore-tool: Allow --pgid specified on import (for post split)

Revamp split import testing to use --pgid option post split

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 90202a70f8c32120625c99ef5131778b3c9f6303)

9 years agoceph-objectstore-tool: Invalidate pg stats when objects were skipped during pg import
David Zafman [Fri, 22 May 2015 17:53:48 +0000 (10:53 -0700)]
ceph-objectstore-tool: Invalidate pg stats when objects were skipped during pg import

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 51e27984ec131fd21bbc26411ae1540b9047eb37)

9 years agoceph-objectstore-tool: Add dump-super to show OSDSuperblock in format specified
David Zafman [Tue, 19 May 2015 03:22:03 +0000 (20:22 -0700)]
ceph-objectstore-tool: Add dump-super to show OSDSuperblock in format specified

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 99d7663f870abfa82711dbf0451334f7d6c883d8)