Difference between revisions of "WebIssues/Protocol"

From MiMec
Jump to: navigation, search
(→‎Protocol Versions)
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
This section describes the design principles of the protocol used for communication between the WebIssues client and server.
+
This is a description of the design principles of the protocol used for communication between the WebIssues client and server.
−
 
 
−
The full specification of the protocol is available in [http://doc.mimec.org/webissues/server/protocol.html Appendix A] of the WebIssues Server manual.
 
  
 
== Design ==
 
== Design ==
  
−
The WebIssues protocol is designed in order to support the client application in a most simple and efficient way. It contains only the minimal set of commands which are required by the client. It uses a very simplified syntax which minimizes the overhead and makes debugging easy.
+
The WebIssues protocol is designed to support communication between the desktop client and the server in a simple and efficient way. It contains only a minimal set of commands and uses a very simplified syntax to minimize the overhead and make implementation and testing easy.
−
 
 
−
One of the most common question regarding WebIssues is "why doesn't is use XML as the communication protocol?". First of all, XML is not a protocol, but merely a way to represent data. There are many protocols based on XML, such as XML-RPC or SOAP, there are also many alternatives like JSON or OGDL. None of them is "better" than others. The main disadvantage of XML is that it adds a lot of bloat - the tags significantly increase the amount of space and are just syntactic sugar. The WebIssues protocol was designed in such way that the data take the least amount of space possible but are still readable.
 
−
 
 
−
Also remember that the WebIssues system uses a 'rich client' architecture. It means that all logic is implemented on the client side. The server only provides commands for synchronizing data (for example, updating the list of issues) and for performing simple, atomic operations. Technologies such as Web Services are better suited for systems where most of the logic is implemented on the server side and the client only presents the results.
 
−
 
 
−
Embedding requests in a <tt>multipart/form-data</tt> message makes it possible to debug the server using a simple web page containing a form with one input field and file upload field. It also makes handling input data and file uploads trivial through built-in mechanisms of PHP.
 
−
 
 
−
== Backward Compatibility ==
 
−
 
 
−
The general principle of WebIssues is that a client should work with both an older and a newer version of the server. Otherwise each upgrade of the server would require upgrading all client workstations and that can be pain.
 
  
−
Compatibility between the client and the server is very much like binary compatibility of shared libraries. New versions of a library can add new functions, but they cannot change the signature or the semantic of existing functions. So the most important principle of the WebIssues Protocol is that all commands and their results have a fixed syntax and semantics.
+
The protocol is based on HTTP request-response communication model. The request is a multipart/form-data message which is commonly used by web browsers to submit form data with attached files. It consists of a mandatory 'command' field and an optional 'file' field. Such format is very natural and easy to handle by the PHP-based server application. The command consists of one or more keywords and a number of arguments separated by spaces. The response is a plain text message which consists of zero or more records, each in a separate line, with a keyword and a number of fields separated by spaces.
  
−
A newer version of the server can only add new commands, but it must not break compatibility with older clients. Backward compatibility affects not only the commands, but also other aspects of the protocol. For example, a new type of attribute cannot be added, because older clients would not know how to handle it and would report an error. A new preference field can be added and will be ignored by older clients, but changing the syntax or semantics of an existing field is not allowed.
+
No additional tags or other kind of markup is necessary, because the protocol is well defined and the meaning of each field is known to both the client and the server. Traditional protocols based on XML, such as XML-RPC or SOAP, and other alternatives like JSON or OGDL, are not efficient for transferring large amounts of data because they contain too much bloat. The WebIssues protocol does not depend on any specific language or technology and can be easily parsed by any kind of application.
  
−
=== Server Version ===
+
Commands defined in the protocol can be divided in two groups: commands for retrieving data from the database and commands for performing modifications in the database. The first group of commands is used to synchronize data between the client and the server (see [[../Architecture|Architecture]] for more information) and they simply return all data from one or more tables (filtered according to user permissions). Modification commands are used to perform add/update/delete operations requested by the user. They check user permissions, ensure validity and integrity of data and usually return no information (except for the identifier of created item).
  
−
It is the client's responsibility to check the version of the server it's connecting to in order to determine which functions are available. The client should not use functions that are not supported by the server. The server's version is returned in the <tt>X-WebIssues-Server</tt> header (available since version 0.8.3).
+
== Incremental Updates ==
  
−
The availability of some functions may depend on the configuration of the server. In order to determine which optional functions are enabled on a particular server, the <tt>LIST FEATURES</tt> command will be introduced in version 0.8.4. The client should not use functions which are not enabled.
+
Some lists such as the list of projects and folders or list of users contain relatively small amount of data so they can be efficiently updated as a whole. On the other hand, the list of issues in a folder (including attribute values) and the details of an issue (including comments and changes) can be very large, so it would be very inefficient to always retrieve all data. This is why a mechanism of incremental updates was created.
  
−
=== Protocol Version ===
+
Each operation related to an issue - including adding a new issue, comment or attachment and changing the value of an attribute - is related with a unique incrementally growing value called the 'stamp'. Identifiers of issues, comments, attachments and changes are directly related to their stamps. In addition, each folder and issue holds a 'last modification stamp' which allows to quickly find which items were modified since the last update. The LIST ISSUES and GET DETAILS commands allows passing a stamp value and only items created or modified after the given stamp are returned.
  
−
From time to time there is a need to make changes which are not backward compatible. That's for example the case if we decide that deleting or moving issues between folders should be possible. For that purpose the <tt>X-WebIssues-Version</tt> header is used (currently it's 0.8).
+
The mechanism of incremental updates was extended in version 1.0 to allow e.g. deleting and moving issues between folders. Such operations are tracked by the server in form of 'stubs', which also have their stamps, and returned to the client if necessary.
  
−
In general both the server and the client must use the same version of the protocol. The client will never work with a server using a newer version of the protocol. It may or may not have a compatibility mode for older versions, depending on the need and effort. The server doesn't need to know anything about the client; it's the client's responsibility to check the server's version.
+
== Protocol Versions ==
  
−
The protocol version has never changed for the last three years. To reduce the impact, it will only be changed once a significant number of changes is gathered together and tested in the development version. Once this happens, the previous versions of both the client and server will remain maintained for some time until most users migrate to a new version.
+
The version of the protocol is identified by the X-WebIssues-Version header. It is generally the same as the two first digits of the server version, for example 0.8 or 1.0. The client can communicate with a server using the same version of the protocol. It may also support older versions of the protocol (for example, version 1.1 of the client might support version 1.0 of the server). It should also be possible to install two versions of the client, which support different servers (for example, 0.8 and 1.0), side by side.
  
−
=== Development Versions ===
+
Patch versions are insignificant, so client 1.0.2 will work with server 1.0.1 and vice-versa, because the same version of the protocol is used. Because of this, backward incompatible changes in the protocol are prohibited between patch versions.
  
−
The backward compatibility rules do not apply to development versions of the server. They can provide preliminary implementation of some functions which may be changed until the final version is released. Also the database schema can be changed until the final version. Because of that, beta version of the server should not be used for production.
+
It is however possible to extend the protocol without changing the protocol version, as long as the syntax and semantics of existing commands remains unchanged. It is also possible to enable or disable some functionality depending on the configuration of the server. The client should detect the capabilities of the server and use the additional features only if possible.
  
−
In order to help avoid problems and overhead caused with changes in the protocol and schema in the development phase, a development version will have several limitations. First of all, upgrading the database schema will only be supported for official versions. It will not be possible to upgrade the database from a development version (although it would be possible to back up data, create a fresh installation and restore the data). Also the stable version of the client will not allow to connect to an older beta version of the server.
+
During development, the protocol can change between each version of the server, and the protocol version is marked as alpha or beta. Development versions of the server can only be used with the same version of the client. The stable version of the client will not allow to connect to a development version of the server.

Latest revision as of 20:37, 12 December 2011

This is a description of the design principles of the protocol used for communication between the WebIssues client and server.

Design

The WebIssues protocol is designed to support communication between the desktop client and the server in a simple and efficient way. It contains only a minimal set of commands and uses a very simplified syntax to minimize the overhead and make implementation and testing easy.

The protocol is based on HTTP request-response communication model. The request is a multipart/form-data message which is commonly used by web browsers to submit form data with attached files. It consists of a mandatory 'command' field and an optional 'file' field. Such format is very natural and easy to handle by the PHP-based server application. The command consists of one or more keywords and a number of arguments separated by spaces. The response is a plain text message which consists of zero or more records, each in a separate line, with a keyword and a number of fields separated by spaces.

No additional tags or other kind of markup is necessary, because the protocol is well defined and the meaning of each field is known to both the client and the server. Traditional protocols based on XML, such as XML-RPC or SOAP, and other alternatives like JSON or OGDL, are not efficient for transferring large amounts of data because they contain too much bloat. The WebIssues protocol does not depend on any specific language or technology and can be easily parsed by any kind of application.

Commands defined in the protocol can be divided in two groups: commands for retrieving data from the database and commands for performing modifications in the database. The first group of commands is used to synchronize data between the client and the server (see Architecture for more information) and they simply return all data from one or more tables (filtered according to user permissions). Modification commands are used to perform add/update/delete operations requested by the user. They check user permissions, ensure validity and integrity of data and usually return no information (except for the identifier of created item).

Incremental Updates

Some lists such as the list of projects and folders or list of users contain relatively small amount of data so they can be efficiently updated as a whole. On the other hand, the list of issues in a folder (including attribute values) and the details of an issue (including comments and changes) can be very large, so it would be very inefficient to always retrieve all data. This is why a mechanism of incremental updates was created.

Each operation related to an issue - including adding a new issue, comment or attachment and changing the value of an attribute - is related with a unique incrementally growing value called the 'stamp'. Identifiers of issues, comments, attachments and changes are directly related to their stamps. In addition, each folder and issue holds a 'last modification stamp' which allows to quickly find which items were modified since the last update. The LIST ISSUES and GET DETAILS commands allows passing a stamp value and only items created or modified after the given stamp are returned.

The mechanism of incremental updates was extended in version 1.0 to allow e.g. deleting and moving issues between folders. Such operations are tracked by the server in form of 'stubs', which also have their stamps, and returned to the client if necessary.

Protocol Versions

The version of the protocol is identified by the X-WebIssues-Version header. It is generally the same as the two first digits of the server version, for example 0.8 or 1.0. The client can communicate with a server using the same version of the protocol. It may also support older versions of the protocol (for example, version 1.1 of the client might support version 1.0 of the server). It should also be possible to install two versions of the client, which support different servers (for example, 0.8 and 1.0), side by side.

Patch versions are insignificant, so client 1.0.2 will work with server 1.0.1 and vice-versa, because the same version of the protocol is used. Because of this, backward incompatible changes in the protocol are prohibited between patch versions.

It is however possible to extend the protocol without changing the protocol version, as long as the syntax and semantics of existing commands remains unchanged. It is also possible to enable or disable some functionality depending on the configuration of the server. The client should detect the capabilities of the server and use the additional features only if possible.

During development, the protocol can change between each version of the server, and the protocol version is marked as alpha or beta. Development versions of the server can only be used with the same version of the client. The stable version of the client will not allow to connect to a development version of the server.

Retrieved from "https://wiki.mimec.org/index.php?title=WebIssues/Protocol&oldid=315"