osd: add an option to prioritize heartbeat traffic

osd: add an option to prioritize heartbeat traffic

By default every hardware queue of a network interface is assigned a
pfifo_fast QDisc. When network congestion occurs, the data traffic may
starve out the heartbeat traffic.

To make sure that heartbeat packets are always transmitted(dequeued) first,
Setting the SO_PRIORITY as 6 for the sockets that are used to transmit
heartbeat messages. The length of heartbeat messages are small. And an
OSD Daemon doesn't ping its peers very often. So the heartbeat traffic
is not likely to starve out the data traffic.

Using fq_codel instead of pfifo_fast is another good choice to avoid
bufferbloat. It's not available until Linux 3.5 though.

Signed-off-by: Jian Wen <wenjianhn@gmail.com>