log: use non-blocking atomic writes to stderr fifos

log: use non-blocking atomic writes to stderr fifos

Buggy container runtimes are known to do partial reads on container's
stderr and emit torn logs [1]. This is due to write(2) allowing partial
data transfer. One way to avoid this when stderr is a FIFO (which is the
case for container runtimes) is to change the file flags to set
O_NONBLOCK which alters the behavior of write(2) when the write size is
less than PIPE_BUF. This makes a write guaranteed to be atomic.

The small complexity added is to poll on stderr in this case whenever a
write(2) fails with EAGAIN. This does not block the log flush thread in
any new way. It would always block when the stderr FIFO was full.

[1] https://github.com/containers/conmon/issues/242

Fixes: cef04bc4ed0a3b1c1e9873fbbdd023b3bdf8af03
Fixes: https://tracker.ceph.com/issues/57923
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1996599
Related: https://bugzilla.redhat.com/show_bug.cgi?id=2108228
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 2551130bcc12c55ddaa879b4f5b77c81890b51b2)

Conflicts:
	src/log/Log.cc: code moved