]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agomds: fix check of underwater dentries 21185/head
Yan, Zheng [Sat, 17 Feb 2018 01:37:48 +0000 (09:37 +0800)]
mds: fix check of underwater dentries

Underwater dentry is dentry that is dirty in our cache from journal
replay, but had already been flushed to disk before the mds failed.
To decide if an dentry is underwater, original code compares dirty
dentry's version to on-disk dirfrag's version. This method is racy
because CDir::log_mark_dirty() can increase dirfrag's version without
adding log event. After mds failover, version of dirfrag from journal
replay can be less than on-disk dirfrag's version. So newly dirtied
dentry can be equal to or less than the on-disk dirfrag's version.

The race can cause incorrect fragstat/rstat

Fixes: http://tracker.ceph.com/issues/23032
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 9d271696b7735ab5b7384cdd386d6ac3eaafe437)

Conflicts:
src/mds/CDir.cc - conditional being replaced by "if (!dn)" is different
  in jewel; refrain from changing "touching wanted dn"
                          conditional
src/mds/CDir.h - undef_inodes pointer, which is being deleted, has
                         different type in jewel

7 years agoMerge pull request #20508 from smithfarm/wip-23066-jewel
Nathan Cutler [Thu, 29 Mar 2018 10:27:00 +0000 (12:27 +0200)]
Merge pull request #20508 from smithfarm/wip-23066-jewel

jewel: ceph.restart + ceph_manager.wait_for_clean is racy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #18743 from shinobu-x/wip-21923-jewel
Nathan Cutler [Thu, 29 Mar 2018 09:16:22 +0000 (11:16 +0200)]
Merge pull request #18743 from shinobu-x/wip-21923-jewel

jewel: core: Objecter::C_ObjectOperation_sparse_read throws/catches exceptions on -ENOENT

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #20435 from pdvian/wip-22968-jewel
Nathan Cutler [Wed, 28 Mar 2018 20:49:06 +0000 (22:49 +0200)]
Merge pull request #20435 from pdvian/wip-22968-jewel

jewel: cephfs: osdc/Journaler: make sure flush() writes enough data

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20335 from pdvian/wip-22865-jewel
Nathan Cutler [Wed, 28 Mar 2018 20:48:21 +0000 (22:48 +0200)]
Merge pull request #20335 from pdvian/wip-22865-jewel

jewel: mds: fix scrub crash

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20333 from pdvian/wip-22863-jewel
Nathan Cutler [Wed, 28 Mar 2018 20:47:32 +0000 (22:47 +0200)]
Merge pull request #20333 from pdvian/wip-22863-jewel

jewel: cephfs-journal-tool: move shutdown to the deconstructor of MDSUtility

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20312 from pdvian/wip-22861-jewel
Nathan Cutler [Wed, 28 Mar 2018 20:46:46 +0000 (22:46 +0200)]
Merge pull request #20312 from pdvian/wip-22861-jewel

jewel: cephfs: osdc: "FAILED assert(bh->last_write_tid > tid)" in powercycle-wip-yuri-master-1.19.18-distro-basic-smithi

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #19993 from pdvian/wip-22700-jewel
Nathan Cutler [Wed, 28 Mar 2018 20:45:54 +0000 (22:45 +0200)]
Merge pull request #19993 from pdvian/wip-22700-jewel

jewel: cephfs: fuse client: ::rmdir() uses a deleted memory structure of dentry leads …

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #19961 from smithfarm/wip-22695-jewel
Nathan Cutler [Wed, 28 Mar 2018 20:44:59 +0000 (22:44 +0200)]
Merge pull request #19961 from smithfarm/wip-22695-jewel

jewel: mds: fix dump last_sent

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #19141 from shinobu-x/wip-22241-jewel
Nathan Cutler [Wed, 28 Mar 2018 20:44:09 +0000 (22:44 +0200)]
Merge pull request #19141 from shinobu-x/wip-22241-jewel

jewel: cephfs: Processes stuck waiting for write with ceph-fuse

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20271 from smithfarm/wip-20823-jewel
Nathan Cutler [Wed, 28 Mar 2018 15:00:00 +0000 (17:00 +0200)]
Merge pull request #20271 from smithfarm/wip-20823-jewel

jewel: cephfs: client::mkdirs not handle well when two clients send mkdir request for a same dir

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20982 from dzafman/wip-compact
David Zafman [Tue, 27 Mar 2018 15:48:18 +0000 (08:48 -0700)]
Merge pull request #20982 from dzafman/wip-compact

jewel: Add compact operation to ceph-osdomap-tool

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21014 from dzafman/wip-remove-bc-jewel
Nathan Cutler [Fri, 23 Mar 2018 14:10:12 +0000 (15:10 +0100)]
Merge pull request #21014 from dzafman/wip-remove-bc-jewel

jewel: test: Replace bc command with printf command

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agotest: Replace bc command with printf command 21014/head
David Zafman [Thu, 22 Mar 2018 22:24:03 +0000 (15:24 -0700)]
test: Replace bc command with printf command

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 293ac9895f86b1258434d0dc539a7d4b7046cb6d)

7 years agoMerge pull request #20479 from pdvian/wip-23021-jewel
Nathan Cutler [Thu, 22 Mar 2018 14:45:40 +0000 (15:45 +0100)]
Merge pull request #20479 from pdvian/wip-23021-jewel

jewel: rgw: fix the max-uploads parameter not work

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20561 from smithfarm/wip-22894-jewel
Nathan Cutler [Thu, 22 Mar 2018 14:43:52 +0000 (15:43 +0100)]
Merge pull request #20561 from smithfarm/wip-22894-jewel

jewel: rgw: ECANCELED in rgw_get_system_obj() leads to infinite loop

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20496 from pdvian/wip-23023-jewel
Nathan Cutler [Thu, 22 Mar 2018 14:43:04 +0000 (15:43 +0100)]
Merge pull request #20496 from pdvian/wip-23023-jewel

jewel: rgw: upldate the max-buckets when the quota is uploaded

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20749 from theanalyst/wip-jewel-curl-ssl
Nathan Cutler [Thu, 22 Mar 2018 12:50:06 +0000 (13:50 +0100)]
Merge pull request #20749 from theanalyst/wip-jewel-curl-ssl

jewel: rgw: libcurl & ssl fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #19635 from linuxbox2/wip-rgw-jewel-20101
Nathan Cutler [Thu, 22 Mar 2018 12:48:55 +0000 (13:48 +0100)]
Merge pull request #19635 from linuxbox2/wip-rgw-jewel-20101

jewel: rgw: S3 POST policy should not require Content-Type

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #19488 from dplyakin/jewel-get-website-error-code
Nathan Cutler [Thu, 22 Mar 2018 12:45:26 +0000 (13:45 +0100)]
Merge pull request #19488 from dplyakin/jewel-get-website-error-code

jewel: rgw: fix GET website response error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20293 from smithfarm/wip-22572-jewel
Nathan Cutler [Wed, 21 Mar 2018 20:14:22 +0000 (21:14 +0100)]
Merge pull request #20293 from smithfarm/wip-22572-jewel

jewel: rgw: stale bucket index entry remains after object deletion

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20292 from smithfarm/wip-22018-jewel
Nathan Cutler [Wed, 21 Mar 2018 20:12:54 +0000 (21:12 +0100)]
Merge pull request #20292 from smithfarm/wip-22018-jewel

jewel: rgw: segmentation fault when starting radosgw after reverting .rgw.root

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #20291 from smithfarm/wip-21954-jewel
Nathan Cutler [Wed, 21 Mar 2018 20:10:13 +0000 (21:10 +0100)]
Merge pull request #20291 from smithfarm/wip-21954-jewel

jewel: rgw: list bucket which enable versioning get wrong result when user marker

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20269 from smithfarm/wip-21951-jewel
Nathan Cutler [Wed, 21 Mar 2018 20:08:14 +0000 (21:08 +0100)]
Merge pull request #20269 from smithfarm/wip-21951-jewel

jewel: rgw: multisite: data sync status advances despite failure in RGWListBucketIndexesCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20262 from smithfarm/wip-21950-jewel
Nathan Cutler [Wed, 21 Mar 2018 20:01:05 +0000 (21:01 +0100)]
Merge pull request #20262 from smithfarm/wip-21950-jewel

jewel: rgw: null instance mtime incorrect when enable versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20179 from shinobu-x/wip-21655-jewel
Nathan Cutler [Wed, 21 Mar 2018 19:59:30 +0000 (20:59 +0100)]
Merge pull request #20179 from shinobu-x/wip-21655-jewel

jewel: rgw: add ability to sync user stats from admin api

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #19057 from shinobu-x/wip-22188-jewel
Nathan Cutler [Wed, 21 Mar 2018 19:33:12 +0000 (20:33 +0100)]
Merge pull request #19057 from shinobu-x/wip-22188-jewel

jewel: rgw: add cors header rule check in cors option request

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18772 from smithfarm/wip-22028-jewel
Nathan Cutler [Wed, 21 Mar 2018 19:31:14 +0000 (20:31 +0100)]
Merge pull request #18772 from smithfarm/wip-22028-jewel

jewel: rgw: boto3 v4 SignatureDoesNotMatch failure due to sorting of sse-kms headers

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoceph-osdomap-tool: Add compact comand option 20982/head
David Zafman [Thu, 19 Oct 2017 19:56:58 +0000 (12:56 -0700)]
ceph-osdomap-tool: Add compact comand option

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b1f397f4fc3313ee0325722156759bfe32286c6b)

7 years agorgw: rgw_admin use raii for curl init/teardown 20749/head
Abhishek Lekshmanan [Mon, 5 Mar 2018 16:59:33 +0000 (17:59 +0100)]
rgw: rgw_admin use raii for curl init/teardown

Since a lot of rgw-admin commands can exit early

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 4667937e4cece63751d5406d75c140418910a823)

7 years agorgw: call setup and release curl handles in setup_curl
Abhishek Lekshmanan [Mon, 5 Mar 2018 15:30:15 +0000 (16:30 +0100)]
rgw: call setup and release curl handles in setup_curl

Since rgw admin can also use it which will fail otherwise.

Fixes: http://tracker.ceph.com/issues/23203
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 08c4c05f5d939976c9e1607b9338f0a9b287708d)

7 years agorgw: rgw_admin also inits curl
Abhishek Lekshmanan [Fri, 2 Mar 2018 09:41:51 +0000 (10:41 +0100)]
rgw: rgw_admin also inits curl

since we use http manager which in turn uses curl and uses curl multi
interfaces. While curl is initialized at the first call of curl_easy_init() this
method isn't guaranteed to be safe when multiple threads may call the function
since curl_global_init isn't reentrant. Calling curl_global_init via
rgw::curl::setup_curl which additionally sets up ssl interfaces etc. when
openssl is used as curl's ssl backend. Similarly moving rgw target link to
accomodate this change.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 112ba0b4dae5972b520aec39fe99275a6dfa899d)

 Conflicts:
  src/rgw/CMakeLists.txt
  beast submodule related changes which are not a part of this patchset
  src/rgw/rgw_admin.cc
  trivial whitespace conflicts #

7 years agorgw: make frontend map optional for ssl init
Abhishek Lekshmanan [Tue, 27 Feb 2018 22:11:30 +0000 (23:11 +0100)]
rgw: make frontend map optional for ssl init

Since rgw admin also needs to init curl globally where we don't have a frontend
map

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 55dde3df50a9735b81c6b31f5683ebda1dabd5e2)

7 years agorgw: setup locks for libopenssl
Abhishek Lekshmanan [Fri, 2 Mar 2018 16:41:42 +0000 (17:41 +0100)]
rgw: setup locks for libopenssl

openssl <= 1.02 requires explicit callbacks for locking which libcurl doesn't
set. This causes random segmentation faults when openssl uses its global
structures across multiple threads. Providing a simple mutex lock/unlock
functions as a callback. We determine whether openssl is used for libcurl via
curl-config utility which should be installed as a part of our curl development
headers package. We also additionally check that the openssl version is < 1.1.0
which alleviates the need for these callbacks. In this patchset we have done the
following:

- move all curl related global init functionality under rgw::curl namespace
  since libcurl may need to set up various ssl libraries etc during its init
- introduce WITH_CURL_OPENSSL in cmake
  this checks the backend curl is deployed with using curl-config. Since curl
  devel is expected to be installed anyway, this binary should be available and
  can help identify the ssl backend curl was compiled with.
- we only setup the locks if beast/civetweb aren't terminated with ssl, since
  these libraries setup the locks anyway and we want to prevent double
  initialization of openssl. Also we pass in ~CURL_GLOBAL_SSL making curl not
  initialize openssl if civetwb/beast is initializing them. Unfortunately this
  flag is a noop from curl >= 7.57 wherein both the libraries will end up
  initializing openssl anyway, which might override certain settings like error
  strings if using openssl < 1.1

https://curl.haxx.se/libcurl/c/threadsafe.html
https://www.openssl.org/docs/man1.0.2/crypto/threads.html#DESCRIPTION

Fixes: http://tracker.ceph.com/issues/22951
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Jesse Williamson <jwilliamson@suse.com>
(cherry picked from commit 4e30277e4b38694e6f8d61925736cd7c2cff515f)

 Conflicts:
  - CMakeLists.txt
  libcryptopp drop in master conflict
  - src/rgw/rgw_main.cc
  conflict as we removed the ac related headers in Luminous

Conflicts:
  CMake isn't supported in jewel, while the CMake changes were cherried,
  CMakelists for rgw does not exist in jewel and this commit doesn't fix that.
  So though CMake changes are backported, cmake build is untested and expected
  to be broken
src/include/config-h.in.cmake
  src/rgw/rgw_main.cc

7 years agojewel: build/rgw: configure if curl is linked against openssl
Abhishek Lekshmanan [Tue, 6 Mar 2018 12:47:28 +0000 (13:47 +0100)]
jewel: build/rgw: configure if curl is linked against openssl

Since the Luminous/master change was cmake only, making an explicit autoconf
test.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
7 years agorgw: reuse CURL* for keystone.
Marcus Watts [Tue, 27 Feb 2018 07:53:37 +0000 (02:53 -0500)]
rgw: reuse CURL* for keystone.

When using keystone with https (the recommended setting),
it is inefficient to start up a new SSL connection for
each and every operation.  Keeping a CURL* structure around
should reduce the cost of doing this.  This logic tries
to do so, but it also tries to free them fairly aggressively
(5-10 seconds).  This should still greatly reduce load on
keystone at peak times while not tying up excess resources.

Fixes: https://tracker.ceph.com/issues/23162
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit 0c5cee1977eac232045571f4850e64be93271aec)

 Conflicts:
  src/rgw/rgw_common.h
  trivial conflict as lowercase_dash_http_attr isnt a part of L

7 years agobuild/link against curl w/ openssl not nss or gnutls. (debian)
Marcus Watts [Mon, 26 Feb 2018 00:33:34 +0000 (19:33 -0500)]
build/link against curl w/ openssl not nss or gnutls. (debian)

curl + gnutls has some big significant performance hits at least
when doing keystone validation.  nss has long-term memory growth
issues.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit cb12e1afd5beedf1b8c0b85e24e412e34c2fc427)

7 years agoMerge pull request #20061 from cbodley/wip-jewel-rgw-log-trim
Nathan Cutler [Fri, 23 Feb 2018 16:22:13 +0000 (17:22 +0100)]
Merge pull request #20061 from cbodley/wip-jewel-rgw-log-trim

jewel: rgw: automated trimming of datalog and mdlog

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
7 years agorgw: fix ECANCELED handling in rgw_get_system_obj() 20561/head
Yehuda Sadeh [Tue, 22 Nov 2016 18:33:12 +0000 (10:33 -0800)]
rgw: fix ECANCELED handling in rgw_get_system_obj()

Fixes: http://tracker.ceph.com/issues/17996
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit e67a3b5b5c63ee3a34f06bdbe91d3256c6660a89)

Conflicts:
    src/rgw/rgw_rados.cc: used ctx.invalidate(obj) instead of ctx.raw.invalidate(obj)
        in RGWRados::SystemObject::invalidate_state()

7 years agoMerge pull request #20557 from smithfarm/wip-really-disable-pidfile-test
Nathan Cutler [Fri, 23 Feb 2018 13:17:06 +0000 (14:17 +0100)]
Merge pull request #20557 from smithfarm/wip-really-disable-pidfile-test

test/CMakeLists: disable test_pidfile.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoClient: mkdirs bugs when two clients send mkdir request for a same dir 20271/head
dongdong [Wed, 12 Jul 2017 03:30:32 +0000 (11:30 +0800)]
Client: mkdirs bugs when two clients send mkdir request for a same dir

Fixes: http://tracker.ceph.com/issues/20592
Signed-off-by: dongdong tao <tdd21151186@gmail.com>
(cherry picked from commit 80bdd3fb4afd262ee3073b4e8377da9238bdaef2)

Conflicts:
        src/client/Client.cc (jewel uses "uid, gid" instead of "perms")

7 years agotest/CMakeLists: disable test_pidfile.sh 20557/head
Sage Weil [Thu, 10 Aug 2017 19:41:38 +0000 (15:41 -0400)]
test/CMakeLists: disable test_pidfile.sh

Too flaky, see http://tracker.ceph.com/issues/20975

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ff3de2304497544033837bb8d0c809a9e54a3e6e)

Conflicts:
    src/test/Makefile.am: applied change manually because jewel uses autotools

7 years agoMerge pull request #20344 from pdvian/wip-22866-jewel
Nathan Cutler [Thu, 22 Feb 2018 18:05:41 +0000 (19:05 +0100)]
Merge pull request #20344 from pdvian/wip-22866-jewel

jewel: mon/OSDMonitor: fix dividing by zero in OSDUtilizationDumper

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #20463 from dzafman/wip-23006
Nathan Cutler [Thu, 22 Feb 2018 16:06:55 +0000 (17:06 +0100)]
Merge pull request #20463 from dzafman/wip-23006

test: Adjust for Jewel quirk caused of differences with master

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20446 from smithfarm/wip-23010-jewel
Nathan Cutler [Thu, 22 Feb 2018 16:05:55 +0000 (17:05 +0100)]
Merge pull request #20446 from smithfarm/wip-23010-jewel

jewel: Filestore rocksdb compaction readahead option not set by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20289 from smithfarm/wip-21239-jewel
Nathan Cutler [Thu, 22 Feb 2018 16:03:00 +0000 (17:03 +0100)]
Merge pull request #20289 from smithfarm/wip-21239-jewel

jewel: test_health_warnings.sh can fail

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #20284 from dzafman/wip-22912
Nathan Cutler [Thu, 22 Feb 2018 16:00:18 +0000 (17:00 +0100)]
Merge pull request #20284 from dzafman/wip-22912

jewel: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #19611 from ukernel/jewel-22494
Nathan Cutler [Thu, 22 Feb 2018 15:58:26 +0000 (16:58 +0100)]
Merge pull request #19611 from ukernel/jewel-22494

jewel: include/fs_types: fix unsigned integer overflow

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #20520 from smithfarm/wip-disable-test-pidfile
Kefu Chai [Thu, 22 Feb 2018 09:18:28 +0000 (17:18 +0800)]
Merge pull request #20520 from smithfarm/wip-disable-test-pidfile

test/CMakeLists: disable test-pidfile.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agotest/CMakeLists: disable test-pidfile.sh 20520/head
Sage Weil [Thu, 31 Aug 2017 13:11:56 +0000 (09:11 -0400)]
test/CMakeLists: disable test-pidfile.sh

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c4936f4493d34341aab8f01488b45dc981d4ef3f)

7 years agoqa/tasks/ceph.py: fix timing of wait-for-* and osd markdown 20508/head
Sage Weil [Fri, 17 Feb 2017 22:17:18 +0000 (17:17 -0500)]
qa/tasks/ceph.py: fix timing of wait-for-* and osd markdown

Mark down osds, *then* wait for them to come up or for the cluster to be
healthy!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 86c0d07e32205e2b6aa417a0e4ae03f0084a1888)

7 years agotasks/ceph.restart osd fix
Warren Usui [Wed, 25 May 2016 23:59:05 +0000 (16:59 -0700)]
tasks/ceph.restart osd fix

ceph.restart should mark restarted osds down in order to avoid a
race condition with ceph_manager.wait_for_clean

Fixes: http://tracker.ceph.com/issues/15778
Signed-off-by: Warren Usui <wusui@redhat.com>
(manual cherry pick of 1b7552c9cb331978cb0bfd4d7dc4dcde4186c176)

Conflicts:
    qa/tasks/ceph.py (original commit was in ceph/ceph-qa-suite.git)

7 years agoMerge pull request #17707 from liewegas/wip-17445
Nathan Cutler [Tue, 20 Feb 2018 09:45:20 +0000 (10:45 +0100)]
Merge pull request #17707 from liewegas/wip-17445

jewel: osd: also check the exsistence of clone obc for "CEPH_SNAPDIR" requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #17883 from andyli029/wip-jewel-signal-handler-typo
Nathan Cutler [Tue, 20 Feb 2018 09:44:29 +0000 (10:44 +0100)]
Merge pull request #17883 from andyli029/wip-jewel-signal-handler-typo

jewel: core: global/signal_handler.cc: fix typo

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #19906 from jdurgin/jewel-filestore-split
Nathan Cutler [Tue, 20 Feb 2018 09:42:43 +0000 (10:42 +0100)]
Merge pull request #19906 from jdurgin/jewel-filestore-split

jewel: HashIndex: randomize split threshold by a configurable amount

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #20331 from dzafman/wip-21036
Nathan Cutler [Tue, 20 Feb 2018 09:41:55 +0000 (10:41 +0100)]
Merge pull request #20331 from dzafman/wip-21036

jewel: snapset xattr corruption propagated from primary to other shards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #18294 from smithfarm/wip-21796-jewel
Nathan Cutler [Tue, 20 Feb 2018 09:41:07 +0000 (10:41 +0100)]
Merge pull request #18294 from smithfarm/wip-21796-jewel

jewel: Ubuntu amd64 client can not discover the ubuntu arm64 ceph cluster

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #18850 from vumrao/vumrao-jewel-rocksdb-cache-size
Nathan Cutler [Tue, 20 Feb 2018 09:40:30 +0000 (10:40 +0100)]
Merge pull request #18850 from vumrao/vumrao-jewel-rocksdb-cache-size

jewel: common/config: set rocksdb_cache_size to OPT_U64

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #19330 from ovh/bp-21862-jewel
Nathan Cutler [Tue, 20 Feb 2018 09:39:47 +0000 (10:39 +0100)]
Merge pull request #19330 from ovh/bp-21862-jewel

jewel: tools/ceph-conf: dump parsed config in plain text or as json

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #19927 from jdurgin/wip-jewel-omap-recovery
Nathan Cutler [Tue, 20 Feb 2018 09:37:34 +0000 (10:37 +0100)]
Merge pull request #19927 from jdurgin/wip-jewel-omap-recovery

jewel: config: lower default omap entries recovered at once

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #19978 from pdvian/wip-22693-jewel
Nathan Cutler [Tue, 20 Feb 2018 09:36:40 +0000 (10:36 +0100)]
Merge pull request #19978 from pdvian/wip-22693-jewel

jewel: common: compute SimpleLRU's size with contents.size() instead of lru

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #20108 from tchaikov/wip-18004-jewel
Nathan Cutler [Tue, 20 Feb 2018 09:35:18 +0000 (10:35 +0100)]
Merge pull request #20108 from tchaikov/wip-18004-jewel

jewel: osd: update heartbeat peers when a new OSD is added

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Pan Liu <wanjun.lp@alibaba-inc.com>
7 years agoMerge pull request #20146 from smithfarm/wip-22818-jewel
Nathan Cutler [Tue, 20 Feb 2018 09:32:37 +0000 (10:32 +0100)]
Merge pull request #20146 from smithfarm/wip-22818-jewel

jewel: repair_test fails due to race with osd start

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #20306 from tchaikov/wip-pr-16661-jewel
Nathan Cutler [Tue, 20 Feb 2018 09:31:39 +0000 (10:31 +0100)]
Merge pull request #20306 from tchaikov/wip-pr-16661-jewel

jewel: throttle: Minimal destructor fix for Luminous

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
7 years agorgw: upldate the max-buckets when the quota is uploaded 20496/head
zhaokun [Tue, 23 Jan 2018 02:33:34 +0000 (10:33 +0800)]
rgw: upldate the max-buckets when the quota is uploaded

Fixes:https://tracker.ceph.com/issues/22745

Signed-off-by: zhaokun <develop.hikdata.com>
(cherry picked from commit 0f3dd4841ee98351fe94dfc1d4dbb37ffcbb4fbb)

7 years agorgw: fix the max-uploads parameter not work 20479/head
liaoxin01 [Mon, 29 Jan 2018 06:40:01 +0000 (14:40 +0800)]
rgw: fix the max-uploads parameter not work

fix http://tracker.ceph.com/issues/22825

Signed-off-by: Xin Liao <liaoxinbit@gmail.com>
(cherry picked from commit 2ff9ea1568c4f5bd2bccbaa63eee3b58ebb9ca52)

7 years agotest: Adjust for Jewel quirk caused of differences with master 20463/head
David Zafman [Fri, 16 Feb 2018 16:48:12 +0000 (08:48 -0800)]
test: Adjust for Jewel quirk caused of differences with master

Fixes: http://tracker.ceph.com/issues/23006
Signed-off-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #18780 from shinobu-x/wip-21053-jewel
Boris Ranto [Fri, 16 Feb 2018 05:40:22 +0000 (06:40 +0100)]
Merge pull request #18780 from shinobu-x/wip-21053-jewel

jewel: RHEL 7.3 Selinux denials at OSD start

Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoMerge pull request #20143 from smithfarm/wip-21872-jewel
Nathan Cutler [Thu, 15 Feb 2018 15:39:12 +0000 (16:39 +0100)]
Merge pull request #20143 from smithfarm/wip-21872-jewel

jewel: ObjectStore/StoreTest.FiemapHoles/3 fails with kstore

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agocommon/options.cc: Set Filestore rocksdb compaction readahead option. 20446/head
Mark Nelson [Thu, 21 Sep 2017 22:21:16 +0000 (17:21 -0500)]
common/options.cc: Set Filestore rocksdb compaction readahead option.

fixes: http://tracker.ceph.com/issues/21505

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 66567e573836c570040c8d8148c34b0cf9dc9ce2)

Conflicts:
src/common/options.cc (master commit made the change in
           src/common/options.cc, which doesn't exist in jewel)

7 years agoosdc/Journaler: make sure flush() writes enough data 20435/head
Yan, Zheng [Mon, 29 Jan 2018 02:59:00 +0000 (10:59 +0800)]
osdc/Journaler: make sure flush() writes enough data

If _do_flush() fails to write all data because of prezere, _do_flush()
should be called again when prezero finishes. Otherwise flush waiters
may sleep forever.

Fixes: http://tracker.ceph.com/issues/22824
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 94a2076b9de9e8ffb73f3d5c873bdb87a0429e65)

7 years agomon/OSDMonitor: fix deviding by zero in OSDUtilizationDumper 20344/head
Mingxin Liu [Mon, 20 Feb 2017 06:36:02 +0000 (14:36 +0800)]
mon/OSDMonitor: fix deviding by zero in OSDUtilizationDumper

Signed-off-by: Mingxin Liu <mingxin@xsky.com>
(cherry picked from commit 9fe103233eaee965849c00c66c1b352f31932e01)

7 years agoMerge pull request #19907 from pdvian/wip-22590-jewel
Nathan Cutler [Tue, 6 Feb 2018 23:16:33 +0000 (00:16 +0100)]
Merge pull request #19907 from pdvian/wip-22590-jewel

jewel: cephfs: ceph.in: pass RADOS inst to LibCephFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20067 from smithfarm/wip-22762-jewel
Nathan Cutler [Tue, 6 Feb 2018 14:38:56 +0000 (15:38 +0100)]
Merge pull request #20067 from smithfarm/wip-22762-jewel

jewel: mds: unbalanced auth_pin/auth_unpin in RecoveryQueue code

Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #20074 from smithfarm/wip-22772-jewel
Nathan Cutler [Tue, 6 Feb 2018 14:35:34 +0000 (15:35 +0100)]
Merge pull request #20074 from smithfarm/wip-22772-jewel

jewel: rgw: user creation can overwrite existing user even if different uid is given

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20057 from linuxbox2/jewel-rgw-latchfix
Nathan Cutler [Tue, 6 Feb 2018 14:34:31 +0000 (15:34 +0100)]
Merge pull request #20057 from linuxbox2/jewel-rgw-latchfix

jewel: rgw: rgw_file: alternate fix deadlock on lru eviction

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #20039 from oritwas/wip-rgw-jewel-22703
Nathan Cutler [Tue, 6 Feb 2018 14:33:15 +0000 (15:33 +0100)]
Merge pull request #20039 from oritwas/wip-rgw-jewel-22703

jewel: rgw: resharding needs to set back the bucket ACL after link

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
7 years agoMerge pull request #19908 from pdvian/wip-22582-jewel
Nathan Cutler [Tue, 6 Feb 2018 14:31:33 +0000 (15:31 +0100)]
Merge pull request #19908 from pdvian/wip-22582-jewel

jewel: rgw: dont log EBUSY errors in 'sync error list'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #19783 from ceph/wip-rgw-22182
Nathan Cutler [Tue, 6 Feb 2018 14:30:18 +0000 (15:30 +0100)]
Merge pull request #19783 from ceph/wip-rgw-22182

jewel: rgw: segfaults after running radosgw-admin data sync init

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #18305 from smithfarm/wip-21447-jewel
Nathan Cutler [Tue, 6 Feb 2018 14:26:56 +0000 (15:26 +0100)]
Merge pull request #18305 from smithfarm/wip-21447-jewel

jewel: rgw: multisite: Get bucket location which is located in another zonegroup, will return 301 Moved Permanently

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #18121 from linuxbox2/jewel-quota-multipart
Nathan Cutler [Tue, 6 Feb 2018 14:25:26 +0000 (15:25 +0100)]
Merge pull request #18121 from linuxbox2/jewel-quota-multipart

jewel: RGW: Multipart upload may double the quota

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #18116 from linuxbox2/jewel-mpserializer
Nathan Cutler [Tue, 6 Feb 2018 14:15:59 +0000 (15:15 +0100)]
Merge pull request #18116 from linuxbox2/jewel-mpserializer

jewel: rgw: release cls lock if taken in RGWCompleteMultipart

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #17731 from ceph/wip-rgw-jewel-15998
Nathan Cutler [Tue, 6 Feb 2018 14:14:20 +0000 (15:14 +0100)]
Merge pull request #17731 from ceph/wip-rgw-jewel-15998

jewel: rgw: fix marker encoding problem

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Marcus Watts <mwatts@redhat.com>
7 years agoMerge pull request #17151 from smithfarm/wip-20480-jewel
Nathan Cutler [Tue, 6 Feb 2018 14:12:42 +0000 (15:12 +0100)]
Merge pull request #17151 from smithfarm/wip-20480-jewel

jewel: rgw: log includes zero byte sometimes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #17149 from smithfarm/wip-20399-jewel
Nathan Cutler [Tue, 6 Feb 2018 14:10:34 +0000 (15:10 +0100)]
Merge pull request #17149 from smithfarm/wip-20399-jewel

jewel: rgw: rgw_file: recursive lane lock can occur in LRU drain

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
7 years agoMerge pull request #16763 from linuxbox2/mb-jewel-rgw-delparts
Nathan Cutler [Tue, 6 Feb 2018 13:53:55 +0000 (14:53 +0100)]
Merge pull request #16763 from linuxbox2/mb-jewel-rgw-delparts

jewel: rgw: fix the bug that part's index can't be removed after completing

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #16708 from linuxbox2/wip-jewel-15912-15673
Nathan Cutler [Tue, 6 Feb 2018 13:51:07 +0000 (14:51 +0100)]
Merge pull request #16708 from linuxbox2/wip-jewel-15912-15673

jewel: rgw: Fix duplicate tag removal during GC, cls/refcount: store and use list of retired tags)

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #19975 from pdvian/wip-22689-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:48:06 +0000 (14:48 +0100)]
Merge pull request #19975 from pdvian/wip-22689-jewel

jewel: client: release revoking Fc after invalidate cache

Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #18084 from batrick/i17935
Nathan Cutler [Tue, 6 Feb 2018 13:45:42 +0000 (14:45 +0100)]
Merge pull request #18084 from batrick/i17935

jewel: ceph_volume_client: fix setting caps for IDs

Reviewed-by: Ramana Raja <rraja@redhat.com>
7 years agoMerge pull request #17925 from batrick/i21519
Nathan Cutler [Tue, 6 Feb 2018 13:42:32 +0000 (14:42 +0100)]
Merge pull request #17925 from batrick/i21519

jewel: client: set client_try_dentry_invalidate to false by default

Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #17188 from smithfarm/wip-21067-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:40:50 +0000 (14:40 +0100)]
Merge pull request #17188 from smithfarm/wip-21067-jewel

jewel: mds: fix integer overflow

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20287 from smithfarm/wip-22191-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:38:04 +0000 (14:38 +0100)]
Merge pull request #20287 from smithfarm/wip-22191-jewel

jewel: class rbd.Image discard----OSError: [errno 2147483648] error discarding region

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #20286 from smithfarm/wip-22186-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:37:10 +0000 (14:37 +0100)]
Merge pull request #20286 from smithfarm/wip-22186-jewel

jewel: abort in listing mapped nbd devices when running in a container

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #19801 from smithfarm/wip-21642-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:36:13 +0000 (14:36 +0100)]
Merge pull request #19801 from smithfarm/wip-21642-jewel

jewel: rbd ls -l crashes with SIGABRT

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
7 years agoMerge pull request #20282 from smithfarm/wip-21971-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:33:16 +0000 (14:33 +0100)]
Merge pull request #20282 from smithfarm/wip-21971-jewel

jewel: [journal] tags are not being expired if no other clients are registered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #20281 from smithfarm/wip-21290-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:32:10 +0000 (14:32 +0100)]
Merge pull request #20281 from smithfarm/wip-21290-jewel

jewel: [rbd] image-meta list does not return all entries

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #20280 from smithfarm/wip-21266-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:30:55 +0000 (14:30 +0100)]
Merge pull request #20280 from smithfarm/wip-21266-jewel

jewel: [cli] rename of non-existent image results in seg fault

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #19855 from pdvian/wip-22594-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:28:25 +0000 (14:28 +0100)]
Merge pull request #19855 from pdvian/wip-22594-jewel

jewel: rbd: librbd: filter out potential race with image rename

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #19644 from pdvian/wip-22498-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:26:20 +0000 (14:26 +0100)]
Merge pull request #19644 from pdvian/wip-22498-jewel

jewel: rbd-mirror: cluster watcher should ensure it has latest OSD map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #19186 from pdvian/wip-22209-jewel
Nathan Cutler [Tue, 6 Feb 2018 13:25:12 +0000 (14:25 +0100)]
Merge pull request #19186 from pdvian/wip-22209-jewel

jewel: rbd: disk usage on empty pool no longer returns an error message

Reviewed-by: Jason Dillaman <dillaman@redhat.com>