Following sequence of events can happen when exporting inodes:
- client sends open file request to mds.0
- mds.0 handles the request and sends inode stat back to the client
- mds.0 export the inode to mds.1
- mds.1 sends cap import message to the client
- mds.0 sends cap export message to the client
- client receives the cap import message from mds.1, but the client
still doesn't have corresponding inode in the cache. So the client
releases the imported caps.
- client receives the open file reply from mds.0
- client receives the cap export message from mds.0.
After the end of these events, the client doesn't have any cap for
the opened file.
To fix the message ordering issue, this patch introduces a new session
operation FLUSHMSG. Before exporting caps, we send a FLUSHMSG seesion
message to client and wait for the acknowledgment. When receiveing the
FLUSHMSG_ACK message from client, we are sure that clients have received
all messages sent previously.