Asynchronous Go API idioms
Posted in go on November 14th, 2009 by jb55 – 1 CommentAfter hacking on my Go Twitter API for the past couple of days I’ve started noticing some very cool ways of taking advantage of Go’s concurrency using goroutines and channels, and I thought I’d share them here.
Originally all of my API calls were synchronous. When you called GetStatus(1234) it made an HTTP request to the server and returned when it was finished. This is ok for most applications, but it definitely left much inflexibility in the API. I thought about leaving it like this, arguing that if the client wanted asynchronous calls it could wrap it with its own goroutine. After thinking about it, having the API handle the work of creating a goroutine isn’t too big of a deal and makes less work for the client.
Return channels, not objects!
This inherent property that all potentially blocking API calls be asynchronous has an interesting effect on how you interact with the API:
api := twitter.NewApi();
statusChannel := api.GetStatus(12345); // asynchronous
pubTimelineChannel := api.GetPublicTimeline(); //asynchronous
status := <-api.GetStatus(123456); // synchronous
fmt.Printf("synchronous status: %s, async status: %s\n",
status.GetText(), (<-statusChannel).GetText()) ;
Neat! When the client wants regular synchronous calls, all they have to do is prefix the call with the channel receive operator! What the API does is instead of returning the Status object itself, it returns a buffered channel that receives a Status object. Making the channel buffered has the added benefit of not blocking the goroutine when its sending on the channel, allowing it to destroy itself after receiving the data. When a potentially blocking API call is made, the function creates a 1-sized buffered channel of the return type, launches a goroutine that takes the channel as a parameter, and then returns the channel instantly:
func (self *Api) GetStatus(id int64) chan Status {
response := make(chan Status, 1);
go self.goGetStatus(id, response);
return response;
}Loosening the grip on the receive channel
Awesome! We now have a robust way of getting single Twitter status messages both asynchronously and synchronously, but this isn't flexible enough. Here's why: Say we wanted a clean way of getting N status messages asynchronously. With our current setup we would need to call the function N times and manage N separate channels. Now say you wanted to receive and process the status messages the moment they arrived. To do this you would have to construct a select statement with N cases, this becomes unmanageable with large values of N.
The ideal solution is to let the client create and manage an N-sized buffered channel and pass it into our API. Then they could do something like this:
const nIds = 10;
receiveChannel := make(chan twitter.Status, nIds);
api.SetReceiveChannel(receiveChannel);
var startId int64 = 5641609144;
for i := 0; i < nIds; i++ {
api.GetStatus(startId);
startId++;
}
for i := 0; i < nIds; i++ {
// reads in status messages as they come in
fmt.Printf("Status #%d: %v\n", i, <-receiveChannel);
}Error handling?
A popular idiom used in the standard Go packages is _, ok :=. The basic idea is you return a bool or an os.Error as the second return value of the function to notify of a success or failure. This does not play well with my API, since returning multiple values from a function breaks method chaining and syncronous calls. I currently handle errors with the api function GetLastError which returns a os.Error, and the GetErrorChannel function which returns a channel that receives errors as they occur, which I found useful for tests and logging errors.
That's all for now, let me know what you think.