common,rbd,rgw,osd: extract config values into ConfigValues
this change introduce three classes: ConfigValues, ConfigProxy and
ConfigReader. in seastar port of OSD, each CPU shard will hold its own
reference of configuration, and upon changes of settings, each
shard will be updated with the new setting in async. so this forces us
to be able to keep two set of configuration at the same time. so we
need to extract the changeable part of md_config_t out. so we can
replace the old one with new one on demand, and let different shards
share the same unchanged part, amon the other things, the Options map
and the lookup tables. that's why we need ConfigValues. we will add
a policy template for this class, so we can specialize for Seastar
implementation to allow different ConfigProxy instances to point
md_config_impl<> to different ConfigValues.
because the observer interface is still using md_config_t, to minimise
the impact of this change, handle_conf_change() and
handle_subsys_change() are not changed. but as it accepts a `const
md_config_t`, which cannot be used to create/reference the ConfigProxy
holding it, we need to introduce ConfigReader for reading the updated
setting from md_config_t in a simpler way, without exposing the
internal "values" member variable.
Sage Weil [Sun, 8 Jul 2018 16:00:12 +0000 (11:00 -0500)]
mon/OSDMonitor: add 'osd repeer <pgid>' command
Selecting force peering on a single PG. In reality this probably induces
*2* interval changes.
Note that in the case of a single OSD cluster we can't actually force a
repeer on a single PG because the pg_temp code is pretty robust about
filtering out redundant or meaningless changes, so we can't pg_temp our
way into a new interval if there are no other OSDs to switch to and the
code also prevents an empty pg_temp.
Sage Weil [Mon, 9 Jul 2018 18:26:39 +0000 (13:26 -0500)]
global/global_init: fix stdout/stderr/stdin closing for daemonization
The global_init_postfork/prefork helpers close stdout/stdin/stderr on
fork and reopen /dev/null in their place. This ensures that if later
code writes to those descriptors (e.g., a stray cout or cerr usage) the
output/input will go nowhere instead of interfering with some other open
fd.
However, with the use of preforker, there are other threads running when
these helpers are run, which means we can race with, say, filestore
opening an object file and end up sending log output there.
Fix by atomically replacing the fds with the dup2(2) syscall, which
will implicitly close and reopen the target fd in an atomic fashion. This
behavior is present on both Linux and FreeBSD.
Fixes: http://tracker.ceph.com/issues/23492 Signed-off-by: Sage Weil <sage@redhat.com>
Since run-make-check.sh already ensures that ccache is installed,
it makes sense to let everyone benefit from the ccache
tweaks introduced by 4cb5a590537a9caaf61db42ce8ea123d2ab961f3
Note 1: The previous solution using "date" would cause build tools to reset
their timestamps after 24 hours, on subsequent runs of run-make-check.sh.
In order to maximize ccache effectiveness, this commit sets SOURCE_DATE_EPOCH
to a fixed value: the number of seconds elapsed since the Unix epoch as at
January 1, 2000 (chosen to commemorate Y2K armageddon).
Note 2: this commit introduces "set -e". This was actually in effect
before, via "source install-deps.sh". Better to make it explicit.
9b80b14783ef895390b4153320078661627f373d extended that check to also
include RECOVERY_DELETES. The Mimic release notes do not mention these
flags as prerequisites for an upgrade beyond Luminous. That creates an
obvious issue for users who skipped Luminous in production, and now
want to upgrade from Jewel to Mimic in, say, a weekend.
Update the release notes to include those flags as prerequisites for a
Luminous to Mimic upgrade, explain how users can make sure that they
are set, and also give users a one-liner to fix up their PGs in a
pinch, if they need to.
Igor Fedotov [Thu, 5 Jul 2018 11:27:12 +0000 (14:27 +0300)]
os/bluestore: fix incomplete faulty range marking when doing compression
GC.
Under some scenarios GC might process an extent range where some inner extents are left untouched by GC (as there is no need for that). Hence GC doesn't invaliate these inner extents with fault_range call. If untouched extents are mapped to unloaded shards it results in subsequent assertion on o->extent_map.dirty_range() call.
The solution is to invalidate the whole extent range when doing GC.
Fixes: https://tracker.ceph.com/issues/23540 Signed-off-by: Igor Fedotov <ifedotov@suse.com>
deb/rpm: add python-six as build-time and run-time dependency
python-six is used by dashboard's controllers. please note,
${python:Depends} is able to fill the installation dependencies for
debian package. so python-six is not added to Requires of ceph-mgr.
Sage Weil [Wed, 4 Jul 2018 19:20:27 +0000 (14:20 -0500)]
Merge PR #22534 into master
* refs/pull/22534/head:
msg/Messenger: use safe_item_history<> for my_addrs
msg/simple: fix set_myaddrs()
msg/Messenger: use mutable_item_history<> for my_addrs
common/item_history: container to support lockless reads, infrequent updates
msg/simple/Accepter: fix my_addr == my_addrs.v[0] constraint
msg/async: track target_addr for each connection
msg/async: mark accepted connections with addr type (legacy or msgr2)
msg/async: mark AsyncConnection with msgr2 flag
msg/async: track connections by addrvec
mon/Session: inst -> name and addrs
osd/OSDMap: don't print hb addrs
msg/DispatchQueue: myaddr -> myaddrs
mgr: myaddr -> myaddrs
msg: make set_addr_unknowns take an addrvec
mon/LogMonitor: myaddr -> myaddrs
librados: myaddr -> myaddrs
common/LogClient: myaddr -> myaddrs
client: myaddr -> myaddrs
osd/OSDMap: is_blacklisted() for addrvecs
osd: populate metadata with all addrs
mds: addr -> addrvec
mon/MonClient: get_myaddrs()
mon/OSDMonitor: addrvec blacklist helper
mds: use new pick_addresses
ceph-osd: use new bindv()
msg/async: bind to multiple addresses
msg/async: (legacy) handshake using legacy addr
msg/async: fix some debug prefixes
msg/async: multiple listening ServerSockets
osd/OSDMap: make cluster addrs addrvecs too
msg/Messenger: my_addr -> my_addrs
msg/Connection: peer_addr -> peer_addrs
msg/msg_types: hash<> for entity_addrvec_t
mgr/DaemonServer: use new bindv() and pick_addresses()
msg/Messenger: new bindv() that takes an addrvec
common/pick_address: fill in ipv4/6 and msgr1/msgr2 via config options
common/pick_addresses: new addrvec-based pick_addresses()
common/pick_addresses: add filtering by ipv4 and ipv6
CEPH_MON_PORT -> CEPH_MON_PORT_LEGACY; define CEPH_MON_PORT_IANA
osd: kill osd_heartbeat_addr option
common/options: add addrvec option type
Sage Weil [Wed, 4 Jul 2018 19:19:04 +0000 (14:19 -0500)]
osd/PrimaryLogPG: rebuild attrs from clients
Ensure that buffers coming in via client ops are rebuilt before populating
the PGTransaction. This ensures that we don't pin the raw buffers for
the entire incoming message in memory.
In the past we've addressed this issue at the ObjectStore layer, but we
did not consider the attr_cache in ObjectContext. Rebuilding the buffers
at this point will sanitize any incoming attribute before it reaches
either PGBackend or ObjectContext object_cache or the ObjectStore
implementation.
Fixes: http://tracker.ceph.com/issues/24768 Signed-off-by: Sage Weil <sage@redhat.com>
run-make-check.sh: Adding ccache tuning for the CI
When run-make-check is run by the CI, some tuning can be performed to
speedup the build.
This commit :
- Detect jenkins by searching JENKINS_HOME env variable
- Defines the SOURCE_DATE_EPOCH to enforce a stable date across builds
- Ask cmake not to use the git versioning which adds useless entropy for a temporary build (ENABLE_GIT_VERSION=OFF)
- Define the ccache slopiness to increase efficiency
- Increase the ccache size to save multiple builds to maximise cache hit between PRs
- Print ccache statistics to evaluate ccache efficiency
projects / ceph.git / log