source: web/old/remctl-2.14/docs/protocol-v3 @ f6f3e91

web
Last change on this file since f6f3e91 was f6f3e91, checked in by Jessica B. Hamrick <jhamrick@…>, 15 years ago

Preserve directory hierarchy (not sure what happened to it)

  • Property mode set to 100644
File size: 6.1 KB
Line 
1                     remctl Streaming Protocol Draft
2
3Introduction
4
5    This is a draft of what would become version three of the remctl
6    protocol.  It adds optional support for bidirectional streaming,
7    allowing the server and client to exchange arbitrary unsequenced data
8    while a command is running with coordinated termination of the
9    command.
10
11    This draft should not be used for implementation yet.  The details of
12    the protocol may change substantially before it is added to remctl.
13
14    Client library API changes are not discussed in this draft, only
15    protocol issues.
16
17Streaming Overview
18
19    A streaming command is a client-initiated operation.  The client
20    requests streaming when it sends the command.  The server either
21    accepts that request or rejects it.  If a particular command requires
22    streaming, the server may reject a non-streaming version of that
23    command.
24
25    Once the server accepts the streaming command, it runs the command,
26    passing as arguments the arguments provided in the initial command
27    token as before.  From that point on, all command output is
28    immediately sent back to the client, and the client may also send
29    data at any time which is passed as input to the command.  This
30    continues without ordering and sequencing until one side or the other
31    wants to start shutting things down.
32
33    The server can indicate the end of an output stream with a token
34    dedicated to that purpose.  This communicates an EOF on that stream to
35    the client without terminating the command processing.  The client can
36    similarly tell the server that it is done providing input tokens by
37    sending an end of stream token to the server, after which it is not
38    permitted to send any more input tokens until the server indicates the
39    command is finished.
40
41    When the command is over, the server sends a token indicating that
42    fact.  This token will also include the exit status of the command.
43    This token implies end of stream on all output streams.  The server is
44    solely responsible for determining when the command is over.  The most
45    the client can do is indicate the end of the input stream, but it must
46    still wait for the server to indicate the end of the command.
47
48    If the server indicates the end of the command before the client
49    closes the input stream, the server will continue to consume and
50    discard input tokens from the client until the client sends an end of
51    stream token.
52
53    There are therefore two ways for a streaming command to end:
54
55    * Client sends end of stream token.  Server sends end of streaming
56      command token, possibly preceeded by an arbitrary number of
57      streaming output or end of stream tokens.
58
59    * Server sends end of streaming command token.  Client sends end of
60      stream token, possibly preceeded by an arbitrary number of streaming
61      input tokens.
62
63    The streaming command is not over until the server has sent an end of
64    command token and the client has sent an end of input stream token.
65    Only after both of those tokens have been sent do both sides return to
66    normal processing of remctl commands.
67
68New Tokens
69
70  MESSAGE_COMMAND_STREAM
71
72    Identical to MESSAGE_COMMAND but starts a streaming command instead of
73    a regular command.  A separate token is used for this purpose to avoid
74    changing the format of the MESSAGE_COMMAND token to add an additional
75    flag.
76
77  MESSAGE_STREAM_DATA
78
79    Used by both the client and the server to send data during a streaming
80    command.  The format is the same as a MESSAGE_OUTPUT token.  For
81    tokens from the client, the stream is required to be 1.  Other streams
82    are reserved for future versions of the protocol.
83
84  MESSAGE_STREAM_END
85
86    Indicates an end of data on a stream, equivalent to an EOF condition.
87    The only content of this token is the stream number.  After this token
88    is sent, no further MESSAGE_STREAM_DATA tokens with that stream number
89    will be sent as part of this command.  When sent by the client, this
90    indicates the end of streaming data from the client and promises that
91    the client will send no further data until the server sends the end of
92    command token.
93
94  MESSAGE_COMMAND_END
95
96    Indicates the end of a streaming command.  The format is identical to
97    the MESSAGE_STATUS token.  A separate token is used rather than
98    reusing the MESSAGE_STATUS token only because the expected client
99    response is different (the client may need to send a final
100    MESSAGE_STREAM_END token) and it may be useful for the client to
101    easily distinguish.
102
103Implementation Issues
104
105    The remctl server will be responsible for translating the network
106    packets from the client into data on standard input for the command
107    and output from the command into network packets.  To do so safely
108    without making assumptions about ordering of data, it will need to
109    poll the running command and the client network connection and do
110    non-blocking reads and writes.  However, it may need to block the
111    client until the command can process data, or block the command while
112    waiting for client data.
113
114    I think the best way of handling this is to add buffering code to the
115    server with an upper limit on the allowed amount of data buffering.
116    The remctl server will then accept data from either the command or the
117    client up to the buffer size even if it can't yet write that data to
118    its destination.  Once the buffer is full, the server will stop
119    polling that connection until the buffer has been drained, forcing the
120    other end of that connection to block.
121
122    One challenge for such buffering is that the amount of data contained
123    in a single GSS-API token may vary, and ideally the server should not
124    partially consume that data.  If the server accepts a token, it should
125    read the entire token.  This is not strictly necessary, but I think
126    partially consuming an output token would significantly increase the
127    risk of deadlock.
128
129Limitations
130
131    This protocol does not address deadlock and instead requires that the
132    client and server be aware of deadlock issues and find ways of
133    avoiding deadlock within the implementation of a streaming command.
Note: See TracBrowser for help on using the repository browser.