]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agocls: fix missing __le32 definition 35502/head
Willem Jan Withagen [Tue, 9 Jun 2020 15:38:32 +0000 (15:38 +0000)]
cls: fix missing __le32 definition

On platforms that do not the __le* definitions it throws a compiler error.

fixes: https://github.com/ceph/ceph/pull/35186
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
5 years agoMerge pull request #35187 from CodingSpiderFox/patch-3
zdover23 [Tue, 9 Jun 2020 13:10:05 +0000 (23:10 +1000)]
Merge pull request #35187 from CodingSpiderFox/patch-3

doc: Introduce the concept of a pool before talking about it.

5 years agoMerge pull request #32705 from petrutlucian94/windows.3
Kefu Chai [Tue, 9 Jun 2020 12:21:10 +0000 (20:21 +0800)]
Merge pull request #32705 from petrutlucian94/windows.3

include,common: Windows support [part 3]

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #35422 from adamemerson/wip-45878
Kefu Chai [Tue, 9 Jun 2020 12:17:26 +0000 (20:17 +0800)]
Merge pull request #35422 from adamemerson/wip-45878

osdc: Fix crash on OSD failure

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33720 from yunfeiguan/wip-44409
Kefu Chai [Tue, 9 Jun 2020 12:13:04 +0000 (20:13 +0800)]
Merge pull request #33720 from yunfeiguan/wip-44409

log: fix timestap precision of log can't set to millisecond.

Reviewed-by: Adam Emersen <aemerson@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #34667 from aclamk/wip-rocksdb-reshard
Kefu Chai [Tue, 9 Jun 2020 12:11:32 +0000 (20:11 +0800)]
Merge pull request #34667 from aclamk/wip-rocksdb-reshard

Resharding tool for sharded rocksdb

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #35186 from liewegas/wip-cas
Kefu Chai [Tue, 9 Jun 2020 12:10:43 +0000 (20:10 +0800)]
Merge pull request #35186 from liewegas/wip-cas

cls_cas: misc cleanup, and dynamic chunk_refs_t

Reviewed-by: Samuel Just <sjust@redhat.com>
5 years agoMerge pull request #34633 from sebastian-philipp/cephadm-total-scheduler
Sebastian Wagner [Tue, 9 Jun 2020 08:10:51 +0000 (10:10 +0200)]
Merge pull request #34633 from sebastian-philipp/cephadm-total-scheduler

mgr/cephadm: make scheduling make sense

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #35460 from agayev/fix-cmake
Kefu Chai [Tue, 9 Jun 2020 04:13:14 +0000 (12:13 +0800)]
Merge pull request #35460 from agayev/fix-cmake

blk: Add missing PUBLIC keyword to target_link_libraries command.

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #35095 from tchaikov/wip-kefu-onode-v1
Kefu Chai [Tue, 9 Jun 2020 04:02:12 +0000 (12:02 +0800)]
Merge pull request #35095 from tchaikov/wip-kefu-onode-v1

crimson/os: add onode node implementation

Reviewed-by: Samuel Just <sjust@redhat.com>
5 years agocrimson/os: add OnodeBlock 35095/head
Kefu Chai [Sat, 16 May 2020 08:59:29 +0000 (16:59 +0800)]
crimson/os: add OnodeBlock

used for representing the an node of onode tree

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocmake: add crimson::gtest target
Kefu Chai [Sat, 16 May 2020 10:42:59 +0000 (18:42 +0800)]
cmake: add crimson::gtest target

instead re-compiling gtest_seastar.cc, better off creating a static
library and link against it instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson: add node implementation for onode tree
Kefu Chai [Fri, 15 May 2020 16:38:41 +0000 (00:38 +0800)]
crimson: add node implementation for onode tree

* node_t and friends are representations of in-mem and on-disk onode
  block/page/node. it might have different names in different contexts.
  but normally, they are the same thing and are loaded as a logical
  cached extent.
* delta_t holds the mutations against a given node onode. they are
  applied to memory once they are committed to disk as part of the
  log record.

regarding to the name of "simple-fltree", it is a short for simple
Flexible Layout Fixed Tree, which was designed by Samuel Just. My
implementation is a naive implementation of this design. But it
tries to be extensive and should be able to adapt to different policies
for rebalancing nodes when overflow/underflow. But there could be
other variant of flexible layout tree implementations in future,
so i added "simple" prefix.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson/os: extract onode.h out
Kefu Chai [Fri, 15 May 2020 10:52:14 +0000 (18:52 +0800)]
crimson/os: extract onode.h out

so the concrete onode manager implementation can include it without
including the onode_manager.h interface as a whole

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson/os/seastore: add ObjectStore and OnodeManager stubs
Samuel Just [Mon, 4 May 2020 23:57:33 +0000 (16:57 -0700)]
crimson/os/seastore: add ObjectStore and OnodeManager stubs

Signed-off-by: Samuel Just <sjust@redhat.com>
5 years agoMerge pull request #33878 from theanalyst/rgw/cleanup/user-policy-append
J. Eric Ivancich [Mon, 8 Jun 2020 23:52:18 +0000 (19:52 -0400)]
Merge pull request #33878 from theanalyst/rgw/cleanup/user-policy-append

rgw: append user policies to existing policies

Reviewed-by: Pritha Srivastava <prsivas@redhat.com>
Reviewed-by: Adam Emersen <aemerson@redhat.com>
5 years agoMerge pull request #33521 from linuxbox2/wip-getset-ver-nosuch
J. Eric Ivancich [Mon, 8 Jun 2020 23:50:47 +0000 (19:50 -0400)]
Merge pull request #33521 from linuxbox2/wip-getset-ver-nosuch

rgw: fail when get/set-bucket-versioning attempted on a non-existent …

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
5 years agoMerge pull request #33511 from theanalyst/rgw/append-gc-fixes
J. Eric Ivancich [Mon, 8 Jun 2020 23:50:04 +0000 (19:50 -0400)]
Merge pull request #33511 from theanalyst/rgw/append-gc-fixes

rgw: append obj: prevent tail from being GC'ed

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #33492 from prasakri/multisite-sync-check
J. Eric Ivancich [Mon, 8 Jun 2020 23:49:46 +0000 (19:49 -0400)]
Merge pull request #33492 from prasakri/multisite-sync-check

RGW:Multisite: Verify if the synced object is identical to source

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge PR #35470 into master
Patrick Donnelly [Mon, 8 Jun 2020 19:07:04 +0000 (12:07 -0700)]
Merge PR #35470 into master

* refs/pull/35470/head:
mds: fix compile error with CephContext declaration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #35344 into master
Patrick Donnelly [Mon, 8 Jun 2020 18:25:34 +0000 (11:25 -0700)]
Merge PR #35344 into master

* refs/pull/35344/head:
vstart: Support deployment of ganesha daemon by cephadm with NFS option.
mgr/test_orchestrator: List nfs daemon

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #35298 from tspmelo/wip-ng-progressbar
Lenz Grimmer [Mon, 8 Jun 2020 16:16:18 +0000 (18:16 +0200)]
Merge pull request #35298 from tspmelo/wip-ng-progressbar

mgr/dashboard: Use ng-bootstrap for Progressbar

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #33083 from mkogan1/wip-rgw-add-beast-access-logs
Casey Bodley [Mon, 8 Jun 2020 15:36:44 +0000 (11:36 -0400)]
Merge pull request #33083 from mkogan1/wip-rgw-add-beast-access-logs

rgw: add access log to the beast frontend

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agosrc: Rename "zns" directory to "zoned". 35460/head
Abutalib Aghayev [Mon, 8 Jun 2020 14:04:58 +0000 (10:04 -0400)]
src: Rename "zns" directory to "zoned".

"zoned" is a more appropriate name for both HDDs and SSDs that support the zone
interface.  ZNS is an NVMe standard for zoned SSDs only.  Zoned HDDs, on the
other hand, use a SATA standard called ZBC/ZAC.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
5 years agosrc: Add missing PRIVATE keyword to target_link_libraries command.
Abutalib Aghayev [Mon, 8 Jun 2020 14:01:19 +0000 (10:01 -0400)]
src: Add missing PRIVATE keyword to target_link_libraries command.

This fixes the following cmake error that happens if you have libzbc installed:

CMake Error at src/blk/CMakeLists.txt:40 (target_link_libraries):
  The keyword signature for target_link_libraries has already been used with
  the target "blk".  All uses of target_link_libraries with a target must be
  either all-keyword or all-plain.

  The uses of the keyword signature are here:

   * src/blk/CMakeLists.txt:32 (target_link_libraries)

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
5 years agoMerge pull request #35379 from votdev/issue_45856_show_host_labels
Lenz Grimmer [Mon, 8 Jun 2020 12:49:56 +0000 (14:49 +0200)]
Merge pull request #35379 from votdev/issue_45856_show_host_labels

mgr/dashboard: Show labels in hosts page

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agomds: fix compile error with CephContext declaration 35470/head
Willem Jan Withagen [Mon, 8 Jun 2020 12:30:37 +0000 (12:30 +0000)]
mds: fix compile error with CephContext declaration

SimilarTo: https://github.com/ceph/ceph/pull/35209
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
5 years agorgw: add access log to the beast frontend 33083/head
Mark Kogan [Tue, 18 Feb 2020 10:59:35 +0000 (12:59 +0200)]
rgw: add access log to the beast frontend

Add to the Beast frontend an access log line similar to CivetWeb.
attempting to adhere as much as possible to the Apache Combined Log
Format.

Fixes: https://tracker.ceph.com/issues/45920
rgw: use beast message for access log
(cherry picked from commit 44ec38933cbe6cc864c56332cc99502d86568fdc)

Co-authored-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Mark Kogan <mkogan@redhat.com>
5 years agoMerge pull request #35381 from tspmelo/wip-fix-cephfs-directories
Lenz Grimmer [Mon, 8 Jun 2020 09:51:14 +0000 (11:51 +0200)]
Merge pull request #35381 from tspmelo/wip-fix-cephfs-directories

mgr/dashboard: Fix CephFS's Directories not showing

Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #35373 from bk201/wip-45870
Lenz Grimmer [Mon, 8 Jun 2020 09:45:14 +0000 (11:45 +0200)]
Merge pull request #35373 from bk201/wip-45870

mgr/dashboard: remove space after service name in the Hosts List table

Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #35457 from tchaikov/wip-rpm-unitdir
Kefu Chai [Mon, 8 Jun 2020 08:36:38 +0000 (16:36 +0800)]
Merge pull request #35457 from tchaikov/wip-rpm-unitdir

test/crimson: avoid using variadic future

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: avoid using variadic future 35457/head
Kefu Chai [Sun, 7 Jun 2020 04:16:55 +0000 (12:16 +0800)]
test/crimson: avoid using variadic future

see also 3cee25ba41069e9e9b4855f99be20910759440ca

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #35405 from ivancich/wip-fix-truncated-bucket-listing
J. Eric Ivancich [Sat, 6 Jun 2020 20:31:22 +0000 (16:31 -0400)]
Merge pull request #35405 from ivancich/wip-fix-truncated-bucket-listing

rgw: bucket list/stats truncates for user w/ >1000 buckets

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
5 years agoMerge pull request #33762 from clwluvw/wildcard-subuser
J. Eric Ivancich [Sat, 6 Jun 2020 19:11:25 +0000 (15:11 -0400)]
Merge pull request #33762 from clwluvw/wildcard-subuser

rgw: Add support wildcard subuser for bucket policy

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
5 years agoMerge pull request #32393 from clwluvw/opa-access-key
J. Eric Ivancich [Sat, 6 Jun 2020 19:10:55 +0000 (15:10 -0400)]
Merge pull request #32393 from clwluvw/opa-access-key

rgw: Add subuser to OPA request

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #35203 from linuxbox2/wip-lc-perfctr
J. Eric Ivancich [Sat, 6 Jun 2020 19:06:00 +0000 (15:06 -0400)]
Merge pull request #35203 from linuxbox2/wip-lc-perfctr

rgwlc: add lifecycle perf counters

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #35073 from cbodley/wip-45382
J. Eric Ivancich [Sat, 6 Jun 2020 19:04:25 +0000 (15:04 -0400)]
Merge pull request #35073 from cbodley/wip-45382

rgw: add quota enforcement to CopyObj

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Or Friedmann <ofriedma@redhat.com>
5 years agoMerge pull request #34041 from mdw-at-linuxbox/wip-rgw-swstat-loops-2
J. Eric Ivancich [Sat, 6 Jun 2020 18:59:00 +0000 (14:59 -0400)]
Merge pull request #34041 from mdw-at-linuxbox/wip-rgw-swstat-loops-2

rgw: fix loop problem with swift stat on account.

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agorgw: convert rgw_user_max_buckets to new config options 35405/head
J. Eric Ivancich [Thu, 4 Jun 2020 22:34:27 +0000 (18:34 -0400)]
rgw: convert rgw_user_max_buckets to new config options

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agorgw: clean up code around buckets listing
J. Eric Ivancich [Thu, 4 Jun 2020 16:41:26 +0000 (12:41 -0400)]
rgw: clean up code around buckets listing

This commit adds some const correctness, avoids some string/structure
copying, and reformats some code.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agorgw: bucket list/stats truncates for user w/ >1000 buckets
J. Eric Ivancich [Thu, 4 Jun 2020 16:41:06 +0000 (12:41 -0400)]
rgw: bucket list/stats truncates for user w/ >1000 buckets

The is_truncated flag was not being handled properly and therefore
limited the output on these radosgw-admin commands to 1000 buckets.

Fixes: https://tracker.ceph.com/issues/45630
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #35380 from sebastian-philipp/cephadm-logs-daemon-id
Sebastian Wagner [Fri, 5 Jun 2020 21:18:59 +0000 (23:18 +0200)]
Merge pull request #35380 from sebastian-philipp/cephadm-logs-daemon-id

cephadm: More robust way to deduce the systemd unit

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #35411 from sebastian-philipp/adopt-9cd9ad020d93b0b420924fec55da30...
Sebastian Wagner [Fri, 5 Jun 2020 21:18:08 +0000 (23:18 +0200)]
Merge pull request #35411 from sebastian-philipp/adopt-9cd9ad020d93b0b420924fec55da307aff8bd422

qa/cephadm: update adopt corpus

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #35372 from mgfritch/orch-service-spec-mypy
Sebastian Wagner [Fri, 5 Jun 2020 21:17:40 +0000 (23:17 +0200)]
Merge pull request #35372 from mgfritch/orch-service-spec-mypy

python-common: add ServiceSpec type checking

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoosdc: Fix crash on OSD failure 35422/head
Adam C. Emerson [Fri, 5 Jun 2020 19:18:35 +0000 (15:18 -0400)]
osdc: Fix crash on OSD failure

OSDSession::lock should not use lockdep, as we violate its rules in a
place that's safe. Revert it to std::mutex.

Thanks to Or Friedmann <ofriedma@redhat.com> for finding the problem.

Fixes: https://tracker.ceph.com/issues/45878
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
5 years agoMerge pull request #34275 from pritha-srivastava/wip-rgw-sts-oidc-provider
Matt Benjamin [Fri, 5 Jun 2020 18:08:39 +0000 (14:08 -0400)]
Merge pull request #34275 from pritha-srivastava/wip-rgw-sts-oidc-provider

rgw: adds code for creating and managing oidc provider entities in rgw and for offline validation of OpenID Connect Access and ID Token.

5 years agorgw: adding code for policy evaluation for ops 34275/head
Pritha Srivastava [Sat, 18 Apr 2020 17:08:47 +0000 (22:38 +0530)]
rgw: adding code for policy evaluation for ops
like getbucketversioning, putbucketversioning etc

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: Correcting pool name for oidc provider for older
Pritha Srivastava [Fri, 17 Apr 2020 03:49:49 +0000 (09:19 +0530)]
rgw: Correcting pool name for oidc provider for older
versions of osd.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: fix for make failure in test folder.
Pritha Srivastava [Mon, 13 Apr 2020 13:17:29 +0000 (18:47 +0530)]
rgw: fix for make failure in test folder.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: adds code for role session to be used an ARN.
Pritha Srivastava [Tue, 31 Mar 2020 10:07:20 +0000 (15:37 +0530)]
rgw: adds code for role session to be used an ARN.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: adding new iam ops for iam oidc provider
Pritha Srivastava [Mon, 30 Mar 2020 10:43:31 +0000 (16:13 +0530)]
rgw: adding new iam ops for iam oidc provider
to correct unittest faliure.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: modifying documentation to include explanation of
Pritha Srivastava [Sun, 29 Mar 2020 17:46:30 +0000 (23:16 +0530)]
rgw: modifying documentation to include explanation of
open id connect provider related REST APIs and removing
references to token introspection for validating incoming
web token.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: adds a commom method to remove prefix(www, http and https)
Pritha Srivastava [Sun, 29 Mar 2020 10:20:32 +0000 (15:50 +0530)]
rgw: adds a commom method to remove prefix(www, http and https)
from a url.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: adds code to check for client id and cert thumbprint
Pritha Srivastava [Sun, 29 Mar 2020 04:56:03 +0000 (10:26 +0530)]
rgw: adds code to check for client id and cert thumbprint
while validating incoming web token in AssumeRoleWithWebIdentity.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: Added code for offline OpenId Connect/ OAuth 2.0 tokens
Pritha Srivastava [Thu, 26 Mar 2020 12:53:58 +0000 (18:23 +0530)]
rgw: Added code for offline OpenId Connect/ OAuth 2.0 tokens
if the token is a JWT.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: Adding jwt-cpp header only library for JWT validation.
Pritha Srivastava [Thu, 26 Mar 2020 12:52:01 +0000 (18:22 +0530)]
rgw: Adding jwt-cpp header only library for JWT validation.
https://github.com/Thalhammer/jwt-cpp

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agorgw: adds REST APIs for OpenID connect providers manipulation.
Pritha Srivastava [Sat, 28 Mar 2020 02:47:54 +0000 (08:17 +0530)]
rgw: adds REST APIs for OpenID connect providers manipulation.

This commit implements:
1. CreateOpenIDConnectProvider
2. DeleteOpenIDConnectProvider
3. ListOpenIDConnectProviders
4. GetOpenIDConnectProviders

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
5 years agoMerge pull request #35340 from mgfritch/cephadm-srv-id-host
Sebastian Wagner [Fri, 5 Jun 2020 15:05:50 +0000 (17:05 +0200)]
Merge pull request #35340 from mgfritch/cephadm-srv-id-host

mgr/orch: allow '.' char in hostname

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #35413 from sebastian-philipp/upgrade-empty-realm
Casey Bodley [Fri, 5 Jun 2020 15:00:19 +0000 (11:00 -0400)]
Merge pull request #35413 from sebastian-philipp/upgrade-empty-realm

doc/cephadm: Adopt single-site RGW

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agomgr/dashboard: Use ng-bootstrap for Progressbar 35298/head
Tiago Melo [Thu, 28 May 2020 19:12:59 +0000 (19:12 +0000)]
mgr/dashboard: Use ng-bootstrap for Progressbar

Fixes: https://tracker.ceph.com/issues/45752
Signed-off-by: Tiago Melo <tmelo@suse.com>
5 years agocephadm: More robust way to deduce the systemd unit 35380/head
Sebastian Wagner [Thu, 4 Jun 2020 09:58:53 +0000 (11:58 +0200)]
cephadm: More robust way to deduce the systemd unit

Generate a better error message when using wrong args.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #35365 from sebastian-philipp/doc-orch-unmanaged
Sebastian Wagner [Fri, 5 Jun 2020 13:47:29 +0000 (15:47 +0200)]
Merge pull request #35365 from sebastian-philipp/doc-orch-unmanaged

doc/mgr: add docs for unmanaged=True

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #35311 from tspmelo/wip-ng-tooltip
Lenz Grimmer [Fri, 5 Jun 2020 13:26:44 +0000 (15:26 +0200)]
Merge pull request #35311 from tspmelo/wip-ng-tooltip

mgr/dashboard: Use ng-bootstrap for Tooltip

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agovstart: Support deployment of ganesha daemon by cephadm with NFS option. 35344/head
Varsha Rao [Mon, 1 Jun 2020 15:32:53 +0000 (21:02 +0530)]
vstart: Support deployment of ganesha daemon by cephadm with NFS option.

NFS Ganesha cluster can be created in two ways with vstart
cluster:

1) With test_orchestrator
NFS=1 ../src/vstart.sh
This type of deployment can have multiple ganesha daemons on random port.

2) With Cephadm
NFS=1 ../src/vstart.sh --cephadm
It can deploy only single ganesha daemon with vstart on default ganesha port.

Both can create multiple cephfs exports.

Fixes: https://tracker.ceph.com/issues/45830
Signed-off-by: Varsha Rao <varao@redhat.com>
5 years agomgr/test_orchestrator: List nfs daemon
Varsha Rao [Mon, 1 Jun 2020 15:27:45 +0000 (20:57 +0530)]
mgr/test_orchestrator: List nfs daemon

This patch adds dummy data for nfs ganesha daemon and adds nfs to list of
supported service type.

Signed-off-by: Varsha Rao <varao@redhat.com>
5 years agodoc/cephadm: Adopt single-site RGW 35413/head
Sebastian Wagner [Fri, 5 Jun 2020 10:55:15 +0000 (12:55 +0200)]
doc/cephadm: Adopt single-site RGW

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoqa/cephadm: update adopt corpus 35411/head
Sebastian Wagner [Fri, 5 Jun 2020 10:17:03 +0000 (12:17 +0200)]
qa/cephadm: update adopt corpus

to 9cd9ad020d93b0b420924fec55da307aff8bd422

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agomgr/dashboard: Use ng-bootstrap for Tooltip 35311/head
Tiago Melo [Fri, 29 May 2020 09:55:29 +0000 (09:55 +0000)]
mgr/dashboard: Use ng-bootstrap for Tooltip

Fixes: https://tracker.ceph.com/issues/45754
Signed-off-by: Tiago Melo <tmelo@suse.com>
5 years agodoc/mgr: add docs for unmanaged=True 35365/head
Sebastian Wagner [Wed, 3 Jun 2020 13:25:27 +0000 (15:25 +0200)]
doc/mgr: add docs for unmanaged=True

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #35297 from tspmelo/wip-alert
Lenz Grimmer [Fri, 5 Jun 2020 08:49:22 +0000 (10:49 +0200)]
Merge pull request #35297 from tspmelo/wip-alert

mgr/dashboard: Use ng-bootstrap for Alert

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #35375 from sebastian-philipp/doc-cephadm-volker-error
Sebastian Wagner [Fri, 5 Jun 2020 07:46:35 +0000 (09:46 +0200)]
Merge pull request #35375 from sebastian-philipp/doc-cephadm-volker-error

doc/cephadm: Add alternative error for missing CIDR network

Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agomgr/dashboard: remove space after service name in the Hosts List table 35373/head
Kiefer Chang [Thu, 4 Jun 2020 06:05:06 +0000 (14:05 +0800)]
mgr/dashboard: remove space after service name in the Hosts List table

Fixes: https://tracker.ceph.com/issues/45870
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
5 years agomgr/orch: use keyword rather than positional arg 35372/head
Michael Fritch [Thu, 4 Jun 2020 16:39:51 +0000 (10:39 -0600)]
mgr/orch: use keyword rather than positional arg

use keyword `service_id` from the orch cli

Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge PR #35263 into master
Patrick Donnelly [Thu, 4 Jun 2020 23:08:58 +0000 (16:08 -0700)]
Merge PR #35263 into master

* refs/pull/35263/head:
test_cephfs_shell: use StringIO instead of BytesIO

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agotest_cephfs_shell: use StringIO instead of BytesIO 35263/head
Rishabh Dave [Fri, 8 May 2020 17:58:27 +0000 (23:28 +0530)]
test_cephfs_shell: use StringIO instead of BytesIO

Code in qa/ uses both StringIO and BytesIO. Let's use StringIO
exclusively (unless necessary) for uniformity. The reason for using
StringIO over BytesIO is that tests mostly need stdout as string than
as bytes and StringIO is used more frequently used in qa/ code at this
point.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agoMerge pull request #35271 from bk201/refactor-template-cephadm
Sebastian Wagner [Thu, 4 Jun 2020 17:50:58 +0000 (19:50 +0200)]
Merge pull request #35271 from bk201/refactor-template-cephadm

mgr/cephadm: use jinja2 to render templates

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #33646 from ofriedma/wip-mkogan-ofriedma-add-cache-api
Casey Bodley [Thu, 4 Jun 2020 16:11:11 +0000 (12:11 -0400)]
Merge pull request #33646 from ofriedma/wip-mkogan-ofriedma-add-cache-api

rgw: Adding data cache and CDN capabilities

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #35018 from sebastian-philipp/cephadm-rm-container
Sebastian Wagner [Thu, 4 Jun 2020 15:59:48 +0000 (17:59 +0200)]
Merge pull request #35018 from sebastian-philipp/cephadm-rm-container

cephadm: Manually remove containers

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agocommon: stub certain modules and functions on Windows 32705/head
Lucian Petrut [Fri, 18 Oct 2019 15:51:30 +0000 (15:51 +0000)]
common: stub certain modules and functions on Windows

We're going to stub certain modules and functions for Windows builds.
Some of them will be implemented in subsequent commits, some are platform
specific and will be skipped.

Modules:
* module.c
* subprocess.cc
* blkdev.cc
* dns_resolve.cc
* numa.cc
* syslog.h
* statvfs.h
* libblk
* neorados

Functions:
x signal functions
x fork
x chown
* socketpair_cloexec
* dump_open_fds
* run_cmd
* is_symlink
* CInode::d_type
* nbd commands

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
5 years agoinclude/buffer: Extend win32 platform check
Lucian Petrut [Tue, 8 Oct 2019 11:59:41 +0000 (14:59 +0300)]
include/buffer: Extend win32 platform check

<sys/mman.h> is unavailable on Windows. There's a check
for __CYGWIN__ that will have to be extended to look for
_WIN32.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
5 years agoinclude/compat: Add win32 compatibility layer
Lucian Petrut [Tue, 8 Oct 2019 10:49:31 +0000 (13:49 +0300)]
include/compat: Add win32 compatibility layer

Windows provides socket and other related structures that are mostly
following the Posix standards, but using different headers.

Rather than adding lots of platform checks and require future commits
to do so as well, we're adding headers with the same names, which
in turn include the required Windows headers.

In a few cases, some functions declared by unistd.h are actually
defined by different Windows headers, so we'll need additional includes.

One thing to note here is that boost requires us to include aio.hpp
before the Windows socket headers.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
5 years agoinclude/compat: Add missing win32 type and constant definitions
Lucian Petrut [Mon, 4 Nov 2019 08:49:31 +0000 (08:49 +0000)]
include/compat: Add missing win32 type and constant definitions

The following types and constants are not defined on Windows, for
which reason we're going to include them in compat.h.

* uid_t, gid_t
* sigset_t
* HOST_NAME_MAX
* O_CLOEXEC, which we're going to define as a no-op. By default,
  handles are not inherited by subprocesses.
* SHUT_* flags
* loff_t
* __STRING
* EBADE, ENODATA, ESHUTDOWN, ESTALE, EREMOTEIO
* suseconds_t
* SIGKILL, SIGINT - we're mostly going to stub signal handling
on Windows

Also, we'll have to make sure that pipe_cloexec uses C style linking,
which would otherwise be affected as the include order changes.

timeval.tv_sec is defined as long instead of time_t on Windows[1], which
we'll have to take into account.

[1] https://docs.microsoft.com/en-us/windows/win32/api/winsock/ns-winsock-timeval

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
5 years agomgr/dashboard: Use ng-bootstrap for Alert 35297/head
Tiago Melo [Thu, 28 May 2020 18:32:55 +0000 (18:32 +0000)]
mgr/dashboard: Use ng-bootstrap for Alert

Fixes: https://tracker.ceph.com/issues/45751
Signed-off-by: Tiago Melo <tmelo@suse.com>
5 years agoMerge pull request #35176 from aclamk/wip-prio-cache-name
Kefu Chai [Thu, 4 Jun 2020 15:01:46 +0000 (23:01 +0800)]
Merge pull request #35176 from aclamk/wip-prio-cache-name

common: PriorityCache's perf counters can have custom name

Reviewed-by: Mark Nelson <mnelson@redhat.com>
5 years agoMerge pull request #35299 from tspmelo/wip-ng-popover
Lenz Grimmer [Thu, 4 Jun 2020 14:55:48 +0000 (16:55 +0200)]
Merge pull request #35299 from tspmelo/wip-ng-popover

mgr/dashboard: Use ng-bootstrap for Popover

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agomgr/dashboard: Show labels in hosts page 35379/head
Volker Theile [Thu, 4 Jun 2020 08:58:08 +0000 (10:58 +0200)]
mgr/dashboard: Show labels in hosts page

Fixes: https://tracker.ceph.com/issues/45856
Signed-off-by: Volker Theile <vtheile@suse.com>
5 years agomgr/dashboard: Use ng-bootstrap for Popover 35299/head
Tiago Melo [Thu, 28 May 2020 19:19:19 +0000 (19:19 +0000)]
mgr/dashboard: Use ng-bootstrap for Popover

Fixes: https://tracker.ceph.com/issues/45753
Signed-off-by: Tiago Melo <tmelo@suse.com>
5 years agoMerge pull request #35348 from dillaman/wip-45714
Mykola Golub [Thu, 4 Jun 2020 10:57:01 +0000 (13:57 +0300)]
Merge pull request #35348 from dillaman/wip-45714

rbd-mirror: stop local journal replayer first during shut down

Reviewed-by: Mykola Golub <mgolub@suse.com>
5 years agoMerge pull request #35339 from dillaman/wip-45803
Mykola Golub [Thu, 4 Jun 2020 10:56:10 +0000 (13:56 +0300)]
Merge pull request #35339 from dillaman/wip-45803

rbd-mirror: don't hold (stale) copy of local image journal pointer

Reviewed-by: Mykola Golub <mgolub@suse.com>
5 years agomgr/dashboard: Fix CephFS's Directories not showing 35381/head
Tiago Melo [Thu, 4 Jun 2020 10:29:51 +0000 (10:29 +0000)]
mgr/dashboard: Fix CephFS's Directories not showing

This is a regretion introduced in fdd8ec8ca9e38bc0cf9a7b31f232b0322b46782a.

We no longer need to check if a tab is selected to load the content.

In that commit we removed the logic that updated the conditional variable,
but  missed removing its use.

Fixes: https://tracker.ceph.com/issues/45877
Signed-off-by: Tiago Melo <tmelo@suse.com>
5 years agoMerge pull request #35290 from tspmelo/wip-navs
Lenz Grimmer [Thu, 4 Jun 2020 09:17:34 +0000 (11:17 +0200)]
Merge pull request #35290 from tspmelo/wip-navs

mgr/dashboard: Migrate Tabs from ngx-bootstrap to ng-bootstrap

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #34934 from tspmelo/wip-testbed-inject
Lenz Grimmer [Thu, 4 Jun 2020 09:16:47 +0000 (11:16 +0200)]
Merge pull request #34934 from tspmelo/wip-testbed-inject

mgr/dashboard: Use TestBed.inject instead of TestBed.get

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agomgr/cephadm: refactor templating in services 35271/head
Kiefer Chang [Fri, 29 May 2020 09:23:01 +0000 (17:23 +0800)]
mgr/cephadm: refactor templating in services

Using Jinja2 to render configs for services:
- Grafana
- Alertmanager
- Prometheus
- NFS
- iSCSI

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
5 years agomgr/cephadm: add template engine Jinja2
Kiefer Chang [Fri, 29 May 2020 09:22:07 +0000 (17:22 +0800)]
mgr/cephadm: add template engine Jinja2

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
5 years agoMerge pull request #35359 from ceph/docs_fix_all_available_devices
Sebastian Wagner [Thu, 4 Jun 2020 08:06:52 +0000 (10:06 +0200)]
Merge pull request #35359 from ceph/docs_fix_all_available_devices

doc/cephadm: align to all-available-devices flag

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agodoc/cephadm: Add alternative error for missing CIDR network 35375/head
Sebastian Wagner [Thu, 4 Jun 2020 07:53:10 +0000 (09:53 +0200)]
doc/cephadm: Add alternative error for missing CIDR network

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agomgr/cephadm: use keyword rather than positional arg
Michael Fritch [Wed, 3 Jun 2020 20:50:40 +0000 (14:50 -0600)]
mgr/cephadm: use keyword rather than positional arg

use keyword `service_id` during the ServiceSpec tests

cephadm/tests/test_cephadm.py:51: in <module>
    class TestCephadm(object):
cephadm/tests/test_cephadm.py:508: in TestCephadm
    ???
../../python-common/ceph/deployment/service_spec.py:606: in __init__
    assert service_type == 'iscsi'
E   AssertionError

Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agomgr/cephadm: fix RGWSpec mypy errors
Michael Fritch [Wed, 3 Jun 2020 20:44:30 +0000 (14:44 -0600)]
mgr/cephadm: fix RGWSpec mypy errors

cephadm/module.py:34: note: In module imported here:
cephadm/services/cephadmservice.py: note: In member "config" of class "RgwService":
cephadm/services/cephadmservice.py:167: error: Incompatible types in assignment (expression has type "List[str]", variable has type "str")

Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agopython-common: fix pytest failure
Michael Fritch [Wed, 3 Jun 2020 20:52:44 +0000 (14:52 -0600)]
python-common: fix pytest failure

523: error: Unsupported operand types for + ("str" and "None")
523: note: Right operand is of type "Optional[str]"

Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agopython-common: add ServiceSpec type checking
Michael Fritch [Wed, 3 Jun 2020 17:08:29 +0000 (11:08 -0600)]
python-common: add ServiceSpec type checking

- add missing type checking to NFSServiceSpec and IscsiServiceSpec
- clean-up type checking for ServiceSpec and RGWSpec

Signed-off-by: Michael Fritch <mfritch@suse.com>