Difference between revisions of "WebIssues/Protocol"

From MiMec
Jump to: navigation, search
(Protocol Versions)
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
This section describes the protocol used for communication between the WebIssues client and server. This is work in progress, for now only the backward compatibility problem is described.
+
This is a description of the design principles of the protocol used for communication between the WebIssues client and server.
  
The documentation of the protocol is available in [http://doc.mimec.org/webissues/server/protocol.html Appendix A] of the WebIssues Server manual.
+
== Design ==
  
== Backward Compatibility ==
+
The WebIssues protocol is designed to support communication between the desktop client and the server in a simple and efficient way. It contains only a minimal set of commands and uses a very simplified syntax to minimize the overhead and make implementation and testing easy.
  
The general principle is that a client should work with a newer version of the server. Otherwise each upgrade of the server would require upgrading all client workstations and that can be pain.
+
The protocol is based on HTTP request-response communication model. The request is a multipart/form-data message which is commonly used by web browsers to submit form data with attached files. It consists of a mandatory 'command' field and an optional 'file' field. Such format is very natural and easy to handle by the PHP-based server application. The command consists of one or more keywords and a number of arguments separated by spaces. The response is a plain text message which consists of zero or more records, each in a separate line, with a keyword and a number of fields separated by spaces.
  
Because of this, the changes to the communication protocol should be backward compatible. It means that the syntax and semantic of existing commands cannot be changed, but new commands can be added. This is very much like binary compatibility of shared libraries.
+
No additional tags or other kind of markup is necessary, because the protocol is well defined and the meaning of each field is known to both the client and the server. Traditional protocols based on XML, such as XML-RPC or SOAP, and other alternatives like JSON or OGDL, are not efficient for transferring large amounts of data because they contain too much bloat. The WebIssues protocol does not depend on any specific language or technology and can be easily parsed by any kind of application.
  
=== Server Version ===
+
Commands defined in the protocol can be divided in two groups: commands for retrieving data from the database and commands for performing modifications in the database. The first group of commands is used to synchronize data between the client and the server (see [[../Architecture|Architecture]] for more information) and they simply return all data from one or more tables (filtered according to user permissions). Modification commands are used to perform add/update/delete operations requested by the user. They check user permissions, ensure validity and integrity of data and usually return no information (except for the identifier of created item).
  
The client should check the version of the server it's connecting to. If the version of the server is newer than the latest known version, the client will simply not take advantage of the new functions that are available. If the version of the server is older, the client will disable functions that are not supported by the server. The server's version is returned in the <tt>X-WebIssues-Server</tt> header (available since version 0.8.3).
+
== Incremental Updates ==
  
=== Protocol Version ===
+
Some lists such as the list of projects and folders or list of users contain relatively small amount of data so they can be efficiently updated as a whole. On the other hand, the list of issues in a folder (including attribute values) and the details of an issue (including comments and changes) can be very large, so it would be very inefficient to always retrieve all data. This is why a mechanism of incremental updates was created.
  
From time to time there is a need to make changes which are not backward compatible. That's for example the case if we decide that deleting or moving issues between folders should be possible. For that purpose the <tt>X-WebIssues-Version</tt> header is used (currently it's 0.8).
+
Each operation related to an issue - including adding a new issue, comment or attachment and changing the value of an attribute - is related with a unique incrementally growing value called the 'stamp'. Identifiers of issues, comments, attachments and changes are directly related to their stamps. In addition, each folder and issue holds a 'last modification stamp' which allows to quickly find which items were modified since the last update. The LIST ISSUES and GET DETAILS commands allows passing a stamp value and only items created or modified after the given stamp are returned.
  
The client will never work with a server using a newer version of the protocol. It may or may not have a compatibility mode for older versions, depending on the need and effort required to maintain such compatibility. The server doesn't need to know anything about the client and it will always use only one version of the protocol. It's the client's responsibility to check and adjust to the server's version.
+
The mechanism of incremental updates was extended in version 1.0 to allow e.g. deleting and moving issues between folders. Such operations are tracked by the server in form of 'stubs', which also have their stamps, and returned to the client if necessary.
  
To reduce the impact of changing the protocol version, this should be done as rarely as possible. Multiple changes should be merged together and they should be only done in the development version until a production version is released. Also the previous version of both the client and server should remain maintained for some time (bug fixes etc.).
+
== Protocol Versions ==
 +
 
 +
The version of the protocol is identified by the X-WebIssues-Version header. It is generally the same as the two first digits of the server version, for example 0.8 or 1.0. The client can communicate with a server using the same version of the protocol. It may also support older versions of the protocol (for example, version 1.1 of the client might support version 1.0 of the server). It should also be possible to install two versions of the client, which support different servers (for example, 0.8 and 1.0), side by side.
 +
 
 +
Patch versions are insignificant, so client 1.0.2 will work with server 1.0.1 and vice-versa, because the same version of the protocol is used. Because of this, backward incompatible changes in the protocol are prohibited between patch versions.
 +
 
 +
It is however possible to extend the protocol without changing the protocol version, as long as the syntax and semantics of existing commands remains unchanged. It is also possible to enable or disable some functionality depending on the configuration of the server. The client should detect the capabilities of the server and use the additional features only if possible.
 +
 
 +
During development, the protocol can change between each version of the server, and the protocol version is marked as alpha or beta. Development versions of the server can only be used with the same version of the client. The stable version of the client will not allow to connect to a development version of the server.

Latest revision as of 20:37, 12 December 2011

This is a description of the design principles of the protocol used for communication between the WebIssues client and server.

Design

The WebIssues protocol is designed to support communication between the desktop client and the server in a simple and efficient way. It contains only a minimal set of commands and uses a very simplified syntax to minimize the overhead and make implementation and testing easy.

The protocol is based on HTTP request-response communication model. The request is a multipart/form-data message which is commonly used by web browsers to submit form data with attached files. It consists of a mandatory 'command' field and an optional 'file' field. Such format is very natural and easy to handle by the PHP-based server application. The command consists of one or more keywords and a number of arguments separated by spaces. The response is a plain text message which consists of zero or more records, each in a separate line, with a keyword and a number of fields separated by spaces.

No additional tags or other kind of markup is necessary, because the protocol is well defined and the meaning of each field is known to both the client and the server. Traditional protocols based on XML, such as XML-RPC or SOAP, and other alternatives like JSON or OGDL, are not efficient for transferring large amounts of data because they contain too much bloat. The WebIssues protocol does not depend on any specific language or technology and can be easily parsed by any kind of application.

Commands defined in the protocol can be divided in two groups: commands for retrieving data from the database and commands for performing modifications in the database. The first group of commands is used to synchronize data between the client and the server (see Architecture for more information) and they simply return all data from one or more tables (filtered according to user permissions). Modification commands are used to perform add/update/delete operations requested by the user. They check user permissions, ensure validity and integrity of data and usually return no information (except for the identifier of created item).

Incremental Updates

Some lists such as the list of projects and folders or list of users contain relatively small amount of data so they can be efficiently updated as a whole. On the other hand, the list of issues in a folder (including attribute values) and the details of an issue (including comments and changes) can be very large, so it would be very inefficient to always retrieve all data. This is why a mechanism of incremental updates was created.

Each operation related to an issue - including adding a new issue, comment or attachment and changing the value of an attribute - is related with a unique incrementally growing value called the 'stamp'. Identifiers of issues, comments, attachments and changes are directly related to their stamps. In addition, each folder and issue holds a 'last modification stamp' which allows to quickly find which items were modified since the last update. The LIST ISSUES and GET DETAILS commands allows passing a stamp value and only items created or modified after the given stamp are returned.

The mechanism of incremental updates was extended in version 1.0 to allow e.g. deleting and moving issues between folders. Such operations are tracked by the server in form of 'stubs', which also have their stamps, and returned to the client if necessary.

Protocol Versions

The version of the protocol is identified by the X-WebIssues-Version header. It is generally the same as the two first digits of the server version, for example 0.8 or 1.0. The client can communicate with a server using the same version of the protocol. It may also support older versions of the protocol (for example, version 1.1 of the client might support version 1.0 of the server). It should also be possible to install two versions of the client, which support different servers (for example, 0.8 and 1.0), side by side.

Patch versions are insignificant, so client 1.0.2 will work with server 1.0.1 and vice-versa, because the same version of the protocol is used. Because of this, backward incompatible changes in the protocol are prohibited between patch versions.

It is however possible to extend the protocol without changing the protocol version, as long as the syntax and semantics of existing commands remains unchanged. It is also possible to enable or disable some functionality depending on the configuration of the server. The client should detect the capabilities of the server and use the additional features only if possible.

During development, the protocol can change between each version of the server, and the protocol version is marked as alpha or beta. Development versions of the server can only be used with the same version of the client. The stable version of the client will not allow to connect to a development version of the server.