Sage Weil [Mon, 2 Mar 2020 16:30:03 +0000 (10:30 -0600)]
Merge PR #33523 into master
* refs/pull/33523/head:
mgr/orch: ServiceSpec: drop 'count'
mgr/rook: use spec.placement.count (instead of spec.count)
mgr/cephadm: make HostAssignment make sense
mgr/orch: PlacementSpec: do not combine all_hosts with anything else
mgr/orch: use PlacementSpec.from_strings() for all CLI commands
Kefu Chai [Sat, 29 Feb 2020 06:51:28 +0000 (14:51 +0800)]
include/cpp-btree: use the same type when allocate/deallocate
btree_set<> by default uses `std::allocator<Key>`, and btree_map by
default uses `std::allocator<std::pair<Key, Value>>`.
before this change, btree uses the allocator directly for allocating n
elements where element is `Key` or `std::pair<Key, Value>` respectively,
while "n" is actually supposed to be the number of bytes used by each
node which is being allocated.
but, what we need to allocate is actually a "node_type" for holding
multiple slots, and each slot holds an element. in addition to the
slots, a node also keeps track of metadata for btree itself. in short,
what we allocate now is (in bytes):
alignof(sizeof(node_type)) * sizeof(element)
but what we should allocate is (in bytes):
alignof(sizeof(node_type))
in this change:
* always rebind the allocator to the correct aligned type with given
alignment
* extract the allocator related helpers into a template class
Sage Weil [Tue, 25 Feb 2020 00:29:12 +0000 (18:29 -0600)]
mgr/cephadm: make HostAssignment make sense
- if hosts are passed, use those.
- if all_hosts=true, use all hosts.
Otherwise, build a set of hosts based on the labels--either explicit or
implied. If there's no label, use all hosts.
If there is a count, use a subset of candidate hosts. If there was no
label and there is no count, fail.
If count and hosts are both provided, then we either (1) use the hosts
as the candidate list and place among them (if len(hosts) >= count), or
(2) ensure that any result includes *at least* the provided hosts.
Sage Weil [Sun, 1 Mar 2020 20:18:39 +0000 (14:18 -0600)]
Merge PR #33553 into master
* refs/pull/33553/head:
mgr/cephadm: orch ls: include specs with no daemons
mgr/cephadm: orch ls: show spec size
mgr/orch: remove unused fields in RGWSpec
mgr/orch: fix ServiceSpec fields
mgr/cephadm: simplify spec apply
pybind/mgr/mgr_module: revert PersistentStoreDict seperator
mgr/cephadm: apply services after refreshing inventory
mgr/cephadm: catch and log exceptions from apply
mgr/orch: no extra whitespace in stored json specs
mgr/cephadm: drop daemon_type arg to _apply_service
mgr/cephadm: use _apply() helper for all apply_ methods
mgr/cephadm: replace PersistentStoreDict with SpecStore
mgr/cephadm: do not remove service spec when removing a daemon
mgr/cephadm: rename completion variables&cleanup
mgr/cephadm: leverage service specs
Sage Weil [Sun, 1 Mar 2020 13:48:17 +0000 (07:48 -0600)]
mgr/orch: fix ServiceSpec fields
- service_type is required. Make it the first position arg to the ctor.
- service_id is the id *only* and optional.
- service_name() is the full service name (no change)
The old 'name' was previously used as the id only, so it was poorly named,
and optional, but in this series was changed to include the type, breaking
naming for a bunch of things (e.g., daemons called mds.mds.fsname.xyz).
Sage Weil [Sun, 1 Mar 2020 03:09:57 +0000 (21:09 -0600)]
mgr/cephadm: simplify spec apply
- Teach _apply_service how to pick the create (and config) functions, so
that we don't need any weird wrappers in the callers.
- Replace trigger_deploy() and _apply_services() with a simpler
_apply_all_services()
- Drop all of the per-type _apply_foo() methods.
Joshua Schmid [Wed, 26 Feb 2020 13:26:42 +0000 (14:26 +0100)]
mgr/cephadm: leverage service specs
Fixes: https://tracker.ceph.com/issues/44205
This does a couple of things:
* Change the way apply_$service() works:
Instead of triggering the deployment mechanism it will rather
transform the already passed ServiceSpec into a json representation
and save it in a persistent mon_store section.
These locations will be periodically checked in the serve() thread.
This works since all the apply_$service_type functions are idempotent.
* Allow to save a config-like specification in the mon_store.
`ceph orch apply -i <service_spec_file.yaml>`
will read the specified services and save them in the mon store
section like mentioned above. The same serve() mechanism like above
also applies to deployment.
liupengs [Sun, 17 Nov 2019 15:03:07 +0000 (23:03 +0800)]
msg/async/rdma: fix bug event center is blocked by rdma construct connection for transport ib sync msg
We construct a tcp connection to transport ib sync msg, if the
remote node is shutdown (shutdown by accident), the net.connect will be blocked until timeout
is reached, which cause the event center be blocked.
This bug may cause mon probe timeout and osd not reply, and so on.
Casey Bodley [Fri, 28 Feb 2020 17:49:58 +0000 (12:49 -0500)]
rgw: bucket_list_ordered loops until it gets a unique candidate
when we detect a duplicate common prefix, we need to loop until we get
the next unique candidate. we must add a new candidate for each shard,
or we won't visit it again and would miss later entries
Casey Bodley [Fri, 28 Feb 2020 16:06:32 +0000 (11:06 -0500)]
rgw: bucket_list_ordered advances past duplicate common prefixes
we may see the same common prefix from more than one shard. when we
detect a duplicate, we need to advance past it. otherwise, we may make
the wrong decision about is_truncated because the shards with
duplicates won't be at_end()
Chunmei Liu [Fri, 28 Feb 2020 01:50:15 +0000 (17:50 -0800)]
crimson: complete context delete in alien world
This way, we allocate and release only the OnCommit Context on the bluestore
side. Similarly, the crimson_wrapper context containing the passed in
contexts is allocated and released only on the crimson side.
Further, update do_transaction to also call on_applied and on_applied_sync
for completeness.
Chunmei Liu [Wed, 22 Jan 2020 07:25:02 +0000 (23:25 -0800)]
crimson:add alien blue store
adds an alien store wrapper for ceph blue store, which will
create a thread pool, so crimson-osd can call bluestore API
which is running in Posix thread from seastar thread.
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
Chunmei Liu [Wed, 22 Jan 2020 07:03:56 +0000 (23:03 -0800)]
os/bluestore: add BlueStore::omap_get_values()
used by crimson::alienstore. as crimson is not using
`ObjectStore::get_omap_iterator()` for implementing
CEPH_OSD_OP_OMAPGETVALS op. instead, it's using a paged variant of
`ObjectStore::omap_get_values()`, which only exists in
CyanStore before this change.
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
Chunmei Liu [Fri, 17 Jan 2020 23:09:10 +0000 (15:09 -0800)]
crimson:common add TOPNSPC namespace for ceph and crimson
some code coexist in crimson seastar environment and posix environment,
so add namespace to avoid same function conflict, for example add namespace
for CephContext, since the new namespace for classic ceph-osd,
need modify all files declare use CephContext by including "common_fwd.h"
which defined the namespace for each environment.
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
David Zafman [Fri, 28 Feb 2020 01:18:05 +0000 (17:18 -0800)]
test: Fix failing ceph_objectstore_tool.py test
The -N option to vstart.sh was removed, use -k
Old hinfo_key binary happen to be utf-8 decodable, now it
throws an exception trying to decode it. Use new
option to ceph-objectstore-tool to treat stdout as a terminal
and convert binary data to base64.
projects / ceph.git / log