Asynchronous input and output

Asynchronous read and asynchronous write help you process inbound data and write outbound responses faster than synchronous read and synchronous write. Asynchronous read and asynchronous write can improve the throughput of your Open Liberty server, particularly if large amounts of post data or response data must be processed.

Asynchronous read

Consider a scenario where a client request includes two packets of data that are sent one second apart by the client. In synchronous read, the application must occupy a thread for the entire time that the client takes to send the data. The thread starts when the first packet is received and then waits for one second to receive the second packet, so the thread is occupied for one second. In a server with 10 threads, you can process 10 inbound requests per second.

In asynchronous read, the application relinquishes the thread after the first packet is received from the client, and the application is redispatched when the second packet arrives. Assume that each inbound request takes 0.1 seconds to process so that the inbound request occupies a thread for 0.2 seconds. In this case, the same server with 10 threads can process 50 inbound requests per second. Ideally, the throughput of your application server isn’t affected by the rate at which data is received from a client. If throughput is affected by the rate at which data is received, a few bad clients can significantly reduce throughput. With asynchronous read, you can have more consistent throughput.

Asynchronous read sample code

The following examples demonstrate the key requirements for implementing asynchronous read in your code.

Register a ReadListener

To take advantage of asynchronous read, your application must provide a ReadListener object that is called when inbound data is received. The ReadListener object reads the data as it’s received and starts the business logic after all data is read. The ReadListener object is registered by a servlet, as shown in the following example:

@Override
protected void doPost(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {

    req.startAsync();
    req.getInputStream().setReadListener(new SampleReadListener(req, res));
}

This example shows a doPost method that includes the two steps that are required by a servlet to read asynchronously. The implementation in the example is for the doPost method because a ReadListener object is useful only when a request includes post data. The first step starts asynchronous processing, and the second step registers a ReadListener object with the ServletInputStream object for the request. After these steps complete, the servlet returns to relinquish the thread. If you need the servlet to perform extra steps, the servlet must perform these extra steps before the ReadListener object is set. Otherwise, the servlet processing and the ReadListener object processing compete with each other.

Read and store inbound data

The ReadListener object implements three methods, onDataAvailable, onAllDataRead, and onError. The onDataAvailable method reads and optionally stores the inbound data. In the following example, after the inbound data is read, the inbound data is stored for processing:

@Override
public void onDataAvailable() throws IOException {

    byte postData[] = new byte[1024];
    int postDataLen;

    ServletInputStream inStream = _req.getInputStream();

    while (inStream.isReady() && !inStream.isFinished()) {
        postDataLen = inStream.read(postData);
        if (postDataLen > 0)
            outData.add(new String(postData, 0, postDataLen));
    }
}

The while loop in this example implements two key requirements:

The isReady() method is called before every read of data. If it returns a false value, all of the currently available data is read, so the thread is relinquished. If the isFinished() method returns a true value, all data is read. A second read of data that’s performed after a call to the isReady() method is effectively a synchronous read and results in an illegal state exception for the method.
The onDataAvailable method isn’t called unless the isReady method returns a false value and all data is read. If the onDataAvailable method returns when the isReady method returns a true value and more data is available to be read, the container doesn’t call the onDataAvailable method again until the isReady method is called and returns a false value. If the onDataAvailable method returns when the isReady method returns a true value and all of the data isn’t read, your application must call the onDataAvailable method again to restart the read.

In this example, the inbound data is saved for later processing, although some applications might process data as they receive it.

Process inbound data

The onAllDataRead method is called by the container after all inbound data is read. The onAllDataRead method performs the business logic based on the inbound data, as shown in the following example:

@Override
public void onAllDataRead() throws IOException {

    for (String outDataString : outData) {
        _res.getOutputStream().print(outDataString);
    }

    _req.getAsyncContext().complete();
}

The data is written back to the client. After all the data is written, the asynchronous request is completed by the AsyncContext.complete method.

Log any errors

The onError method is called if any error occurs when the inbound data is processed. If this method is called, it’s the last method called on the ReadListener object. In this example, the application returns an error message to the client and then completes the asynchronous request:

@Override
public void onError(Throwable arg0) {

    try {
        _res.getOutputStream().println("Exception when processing inbound data : " + arg0);
    } catch (IOException e) {
        // Log an error.
    }

    _asyncContext.complete();
}

Asynchronous write

Asynchronous write is similar to asynchronous read, but asynchronous write is used for sending responses to the client. Assume that a response is sent in two packets. The first packet is sent to the client immediately, but the second packet can be sent only after the client acknowledges that the first packet is received. In synchronous write, a thread is occupied while it waits for the client to acknowledge receipt of data. But in asynchronous write, throughput can be increased because the thread isn’t occupied while it waits for the client to acknowledge receipt of data. Asynchronous write can be less useful than asynchronous read because your servlet and HTTP implementation might effectively perform the work of asynchronous writing. Your application can write as much as it needs, and the underlying implementation might buffer the response and send it asynchronously.

Asynchronous write sample code

The following examples show the key requirements to implement asynchronous write in your code. In most applications, asynchronous read and asynchronous write are combined. In this situation, the ReadListener.onAllDataRead method registers the WriteListener object and provides the response data to the WriteListener object on its constructor.

Register a WriteListener

To use asynchronous write, your application must provide a WriteListener object, which is called when response data can be sent without blocking other processes. The WriteListener object is registered by a servlet, as shown in the following example:

@Override
protected void service(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {

    req.startAsync();
    res.getOutputStream().setWriteListener(new SampleWriteListener(req, res, 200));
}

This example shows a service method that includes the two steps that are required to write asynchronously. The service method is acceptable because a WriteListener object can be used for any inbound method, for example, the doPost method. The first step starts asynchronous processing, and the second step registers a WriteListener object with the ServletOutputStream object for the request. After these steps complete, the servlet returns to relinquish the thread. If you need the servlet to perform extra steps, the servlet must perform these extra steps before the WriteListener object is set. Otherwise, the servlet processing and the WriteListener object processing compete with each other.

Write an outbound response

The WriteListener object implements two methods, onWritePossible and onError. The onWritePossible method is responsible for writing outbound responses:

public void onWritePossible() throws IOException {

    ServletOutputStream outStream = _res.getOutputStream();

    while (outStream.isReady() && _numWritesRemaining > 0) {
        _numWritesDone++;
        _numWritesRemaining--;
        outStream.println(_asyncEvents + "." + _numWritesDone + _outData);
    }

    if (_numWritesRemaining == 0) {
        _req.getAsyncContext().complete();
    } else {
        _asyncEvents++;
    }
}

The onWritePossible method implements three key requirements:

The isReady method is called before data is written. Data that is written a second time after a call to the isReady method is effectively a synchronous write and results in an illegal state exception for the method.
The onWritePossible method doesn’t return unless the isReady method returns a false value or all data is written. The onWritePossible method might return when the isReady method returns a true value and more data must be written. In this case, the container doesn’t call the onWritePossible method again until the isReady method returns a false value.
The AsyncContext.complete method is called to end the asynchronous request after all data is written. An equivalent to the onAllDataRead method of the ReadListener object doesn’t exist for the WriteListener object because only your application knows when all response data is written.

One effect of this second requirement is that all of the response data must be available before the WriteListener object is registered. If the response data isn’t available before the WriteListener object is registered, the method must return when the isReady method is the true value. In this case, some of the response data isn’t yet written. To handle this scenario, the application can call the onWritePossible method, although you must ensure that two threads aren’t running the onWritePossible method at the same time.

The onError method is called if any error occurs when the response data is processed. If this method is called, it’s the last method called on the WriteListener object. In this case, the application generates an error log and then completes the asynchronous request.