Kefu Chai [Sat, 29 Feb 2020 06:51:28 +0000 (14:51 +0800)]
include/cpp-btree: use the same type when allocate/deallocate
btree_set<> by default uses `std::allocator<Key>`, and btree_map by
default uses `std::allocator<std::pair<Key, Value>>`.
before this change, btree uses the allocator directly for allocating n
elements where element is `Key` or `std::pair<Key, Value>` respectively,
while "n" is actually supposed to be the number of bytes used by each
node which is being allocated.
but, what we need to allocate is actually a "node_type" for holding
multiple slots, and each slot holds an element. in addition to the
slots, a node also keeps track of metadata for btree itself. in short,
what we allocate now is (in bytes):
alignof(sizeof(node_type)) * sizeof(element)
but what we should allocate is (in bytes):
alignof(sizeof(node_type))
in this change:
* always rebind the allocator to the correct aligned type with given
alignment
* extract the allocator related helpers into a template class
Sage Weil [Sun, 1 Mar 2020 20:18:39 +0000 (14:18 -0600)]
Merge PR #33553 into master
* refs/pull/33553/head:
mgr/cephadm: orch ls: include specs with no daemons
mgr/cephadm: orch ls: show spec size
mgr/orch: remove unused fields in RGWSpec
mgr/orch: fix ServiceSpec fields
mgr/cephadm: simplify spec apply
pybind/mgr/mgr_module: revert PersistentStoreDict seperator
mgr/cephadm: apply services after refreshing inventory
mgr/cephadm: catch and log exceptions from apply
mgr/orch: no extra whitespace in stored json specs
mgr/cephadm: drop daemon_type arg to _apply_service
mgr/cephadm: use _apply() helper for all apply_ methods
mgr/cephadm: replace PersistentStoreDict with SpecStore
mgr/cephadm: do not remove service spec when removing a daemon
mgr/cephadm: rename completion variables&cleanup
mgr/cephadm: leverage service specs
Sage Weil [Sun, 1 Mar 2020 13:48:17 +0000 (07:48 -0600)]
mgr/orch: fix ServiceSpec fields
- service_type is required. Make it the first position arg to the ctor.
- service_id is the id *only* and optional.
- service_name() is the full service name (no change)
The old 'name' was previously used as the id only, so it was poorly named,
and optional, but in this series was changed to include the type, breaking
naming for a bunch of things (e.g., daemons called mds.mds.fsname.xyz).
Sage Weil [Sun, 1 Mar 2020 03:09:57 +0000 (21:09 -0600)]
mgr/cephadm: simplify spec apply
- Teach _apply_service how to pick the create (and config) functions, so
that we don't need any weird wrappers in the callers.
- Replace trigger_deploy() and _apply_services() with a simpler
_apply_all_services()
- Drop all of the per-type _apply_foo() methods.
Joshua Schmid [Wed, 26 Feb 2020 13:26:42 +0000 (14:26 +0100)]
mgr/cephadm: leverage service specs
Fixes: https://tracker.ceph.com/issues/44205
This does a couple of things:
* Change the way apply_$service() works:
Instead of triggering the deployment mechanism it will rather
transform the already passed ServiceSpec into a json representation
and save it in a persistent mon_store section.
These locations will be periodically checked in the serve() thread.
This works since all the apply_$service_type functions are idempotent.
* Allow to save a config-like specification in the mon_store.
`ceph orch apply -i <service_spec_file.yaml>`
will read the specified services and save them in the mon store
section like mentioned above. The same serve() mechanism like above
also applies to deployment.
liupengs [Sun, 17 Nov 2019 15:03:07 +0000 (23:03 +0800)]
msg/async/rdma: fix bug event center is blocked by rdma construct connection for transport ib sync msg
We construct a tcp connection to transport ib sync msg, if the
remote node is shutdown (shutdown by accident), the net.connect will be blocked until timeout
is reached, which cause the event center be blocked.
This bug may cause mon probe timeout and osd not reply, and so on.
Chunmei Liu [Fri, 28 Feb 2020 01:50:15 +0000 (17:50 -0800)]
crimson: complete context delete in alien world
This way, we allocate and release only the OnCommit Context on the bluestore
side. Similarly, the crimson_wrapper context containing the passed in
contexts is allocated and released only on the crimson side.
Further, update do_transaction to also call on_applied and on_applied_sync
for completeness.
Chunmei Liu [Wed, 22 Jan 2020 07:25:02 +0000 (23:25 -0800)]
crimson:add alien blue store
adds an alien store wrapper for ceph blue store, which will
create a thread pool, so crimson-osd can call bluestore API
which is running in Posix thread from seastar thread.
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
Chunmei Liu [Wed, 22 Jan 2020 07:03:56 +0000 (23:03 -0800)]
os/bluestore: add BlueStore::omap_get_values()
used by crimson::alienstore. as crimson is not using
`ObjectStore::get_omap_iterator()` for implementing
CEPH_OSD_OP_OMAPGETVALS op. instead, it's using a paged variant of
`ObjectStore::omap_get_values()`, which only exists in
CyanStore before this change.
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
Chunmei Liu [Fri, 17 Jan 2020 23:09:10 +0000 (15:09 -0800)]
crimson:common add TOPNSPC namespace for ceph and crimson
some code coexist in crimson seastar environment and posix environment,
so add namespace to avoid same function conflict, for example add namespace
for CephContext, since the new namespace for classic ceph-osd,
need modify all files declare use CephContext by including "common_fwd.h"
which defined the namespace for each environment.
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
David Zafman [Fri, 28 Feb 2020 01:18:05 +0000 (17:18 -0800)]
test: Fix failing ceph_objectstore_tool.py test
The -N option to vstart.sh was removed, use -k
Old hinfo_key binary happen to be utf-8 decodable, now it
throws an exception trying to decode it. Use new
option to ceph-objectstore-tool to treat stdout as a terminal
and convert binary data to base64.
Sage Weil [Fri, 28 Feb 2020 01:46:20 +0000 (19:46 -0600)]
Merge PR #33488 into master
* refs/pull/33488/head:
doc/mgr/cephadm: document how to see the logs
ceph: -W as shorthand for --watch-channel
pybind/mgr_module: make mgr log message prefix a bit less verbose
mgr/cephadm: silence remoto logger info messages
mgr/cephadm: add misc info messages for major events
mgr/cephadm: adjust logging
mgr/cephadm: log to cluster channel by default
pybind/mgr_module: infrastructure to allow logging to cluster log
mgr/mgr_module: log to mgr log even when log_to_file
mon: accept cephadm channel to 'ceph log last'
ceph: allow watching channels aside from cluster and audit
mon/LogMonitor: fix debug output for logmonitor output file parsing
mgr: allow logging to any channel
David Zafman [Fri, 10 Jan 2020 23:42:19 +0000 (23:42 +0000)]
mon mgr osd: Add dump_osd_times interface for python
The dump_osd_times interface is here for future use of a manager module.
The osd_stats and pg_dump python interfaces don't return network ping
information.
User facing ping time information is formatted 3 decimal places
Use dump_float() (low overhead) for network ping times
used by python dump_osd_times and dump commands like
"ceph --formet=json pg dump osds" which for now yields output like
>>>> "1min": 0.61599999999999999,