3 Tenets for Implementing a REST API

In the course of performing my duties at my day job I recently came across the need for our data to be accessible via an API. After researching the various types available, I settled on developing a REST API. The selection process wasn’t the interesting part of this exercise though. Actually implementing a REST API is what was.

REST is a set of principles

If there is one point that you should take away from this post after reading it, it is this: REST is a set of principles and not a specification. Consider for a moment someone you believe to be moral. Traits that may make them moral, such as trustworthiness, decency or honor, are characteristics of their morality; guidelines they follow in their life – a moral code. There is not a law that specifies what morality is, but there is a general sense of what is expected and what is meant when discussing morality. The same holds true for REST. There is not a specific implementation that makes an API RESTful, but instead certain expectations that one should strive to achieve.

The first place you run into “principles versus specification” is with versioning your API. There are really only two ways to implement the concept of versioning in a REST API – either through the syntax of the URI [this also includes using different (sub)domains for different versions] or by the use of HTTP headers. Within the use of HTTP headers, there are two primary options – use of the X-API-Version header or the use of custom media types. But which of these methods are correct? This exact question is discussed several times on the following websites:

After reading these author’s positions, and the ensuing discussions, you are still left with our original question of which method is correct. And there is the “principles versus specification” problem. There isn’t a right or wrong answer about this. There is only opinion and the solution that best serves the needs at hand.

Versioning your REST API

Seeing as that opinion is part of what drives your REST API implementation, and you are reading this blog post, let me take a moment to share with you my opinion on this subject. For me, there are 3 tenets that I have followed in developing my REST API. The two that pertain to versioning are:

  1. Do not use custom HTTP headers
  2. Do not use custom media types

Why these you ask? For the first one, it is because any proxy that is encountered along the transit path may either modify the headers or strip them off and not deliver them to their intended target. If you are relying on the ‘X-API-Version’ header to specify which version should be used, and it is not present in the request, then what do you do? Revert to the oldest version? Serve the latest? Neither of these are good responses, as what the client gets back cannot be relied on to be consistent. How are they to know whether or not this non-standard header is going to make it to your service or not?

For the second one, I do not believe that versioning of your API is as simple as saying you have a different representation of your data. Sure, if you take a look at the following two examples it’s only about a different representation:

Example 1

<customer>
	<name>Joe</name>
	<phone>555-555-1212</phone>
</customer>

Example 2

<customer>
	<name>Joe</name>
	<phone>
		<home>555-555-1212</home>
		<mobile>555-555-2121</mobile>
	</phone>
</customer>

But what happens when the business logic behind the POSTing to a resource does something different than it did before? The structure of the request, and perhaps even the response, remain the same, but the results of your actions are completely different. This is a version change too. Yes, technically, it’s a change to your application and not the API, but to the consuming client your application is your API. If the behavior of your API changes, even if it is really your application that has, then there is need for a new version of your API.

Do not use custom media types

Using custom media types to represent these types of changes just does not cut it. Custom media types can only be effectively (and questionably at that) used to serve a different representation of the response, but not a different behavior of your application. It is for this reason that I do not use either custom HTTP headers or custom media types and instead specify the API version in the URI. Yes, this is controversial to some but, as we have already established, the other methodologies are likely equally controversial. Heck, this approach even has its own problems for me, as I believe that a URI expresses identity, and identity does not change when a new version is introduced. By using the URI to store the version information we give the visual impression that we are representing different, distinct resources when in fact they are the same identity. But in the battle of “principles versus specification”, wherein the specification (or lack of one) does not give direction on this topic, and my belief in the principle that representing a change to the business logic is more important that representing identity distinctness, this is the solution that I use. You are certainly free to disagree and implement your own solution. The following is an overview of how I manage versioning in my API:

  • I version my API with the versioning scheme of [Major].[Minor].[Revision]
  • Anything that encompasses a major change to the way the API works results is an increment of the [Major] value. Let’s say that we have version 1.4.7 of our API. We have become successful and just purchased another company. In the process of integrating our two companies together, we have changed some of our fundamental business processes, whereas now how we process payments has completely changed due to new accounting rules, etc. We modify our API and in order to indicate that there have been significant business rule changes, and not just presentational changes, we change the version of our API to 2.0.0.
  • Whenever there is a change in the way we interact with a REST resource, we increment the [Minor] value. Again, we start with our API version of 1.4.7. Using our previous examples, Example 1 and Example 2, we change our API version to 1.5.0 after implementing the changes reflected in Example 2.
  • The [Revision] value of our API version is incremented whenever we make minor additions to our resources. Using Example 2, let’s say we want to add an additional value in the node to represent a Work number. As long as our change only adds additional data to a resource without making any structure changes, then an increment from version 1.4.7 to 1.4.8 is in need. IMPORTANT: This [Revision] increase is only for when there are no changes to the existing structure! If the consuming application is expecting to find phone numbers under the XML path of “customer > phone” adding an additional one should not cause their code to break (assuming the application is written correctly). However, changing the name of the “Phone” node to “Phones” ABSOLUTELY will cause the the consuming application to break. This is when a [Minor] value change is needed.
  • The [Revision] value is not used for bug fixes. Internally on our development team we have our own [Major].[Minor].[Revision] for the API codebase. While it will likely mirror the public API’s version most of the time, the two are not in lock step with one another. Version 1.4.7 of the API, from a consuming application’s viewpoint, is the same when there was a missing bracket in the response and the same after the required bracket was included in the response. The API version represents the business rules of the application and presentational structure of the requests and responses.

This management style results in URIs that look like these, for example:

  • https://api.awesomesite.com/v1.4.7/article
  • https://api.awesomesite.com/v1.4.7/customer/12/orders

What is involved in maintaining this approach?

Arguably, maintaining the API version in this manner requires a little bit more work. Each version of our API is configured to be hosted as its own application on the server. Our web server of choice is Apache. We use the mod_rewrite engine to set the appropriate ‘DocumentRoot’ value associated with each API version specified in the URI, defaulting to the latest version if the version specified is not recognized. Some of the reasons we do things this way are:

  • It allows us to maintain more than one version of our API simultaneously.
  • Our code does not become convoluted having to keep track of which version of which resource is supposed to be represented at which time.

Does this mean that for each increment of the [Revision] value we have a new installation of our API application that we install on the server and maintain? Yes. But when you think about this for a moment it’s not as bad as it sounds. Let’s say that we again have version 1.4.7 of our API. When we make a change requiring us to increment to version 1.4.8 we will no longer be making any changes to version 1.4.7. It will sit on the server for whatever amount of time we desire to continue supporting its version. When another change occurs, and we’re at version 1.4.9 of our API, we now have two old versions of the API that we no longer have to make changes to.

How long do you need to keep these old versions of the API lying around? Only you can answer this question, but a decent rule of thumb would seem to be that whenever you increment the [Minor] value you would be safe to remove support for the previous versions. Why you ask? Because if you recall from earlier an increment to the [Minor] value is the result of a change in the way we interact with the REST resource. If we have changed the overall structure of our resource, this seems like a good time to quit supporting previous versions of the structure. Not only does this seem like a good idea, but it will also very likely be necessary.

Another reason this is not as bad as it seems is due to this question: How often are you making changes to your API? Once you have put everything through its paces in development and have had the initial launch of your API how often are you going back to your data representation and adding a mobile phone, for example (and from our previous example)? I ask this question in regards to continuously adding structure to your responses that arguably should have been present from the initial launch. Now if you are feverishly adding new features and capabilities to your API then yes, this can become involved. But how else are you going to maintain it? Whether all the code is maintained in one installation of the application or segregated as I have described it is still code that has to be written. I would rather be adding new API functionality to an isolated instance of the codebase than worry about having to maintain backwards compatibility.

The 3rd tenet

The usage of a REST APi is header-driven. How you intend to interact with the data and business objects is conveyed in the use of GET, POST, PUT and DELETE headers. If you need information about the API endpoints, you use OPTION, TRACE and/or ALLOW. You define the format of your request using Content-Type and specify your desired result format using Accept. But just like I don’t use custom HTTP headers because of concern that proxies may strip them off, how can I guarantee that headers in my response actually make it to the calling application, even if they are all standard and do not use custom headers? Call me paranoid, but you can’t. For that reason, I have a third tenet that I have followed in developing my REST API, which is:

  1. Represent the headers in the response payload

Before I show an example of what this looks like, let me provide another reason for taking this step. I am a programmer. As a programmer, I am sometimes developing something that will be consumed and other times I am developing something that is doing the consuming. As the consumptive programmer, I very often run across a situation where I mumble under my breath at the decisions made by the developing programmer in how they arrived at the solution I am attempting to use. Often it is not that more involved to take a few extra steps that will make the life of the consumptive programmer much easier. This third tenant is one such easy extra step. Even if you don’t completely agree with the need for this extra step, or have not yet ran across a situation where it is beneficial, what is really the cost of just doing it anyways? I would proffer that the cost is negligible and that the goodwill earned from your consumptive programmers is worth the effort.

What it all boils down for me is this: I do not know how my API is going to be consumed. Given that REST is a set of principles and not a specification I cannot guarantee that even with the best of intentions that the consuming application will be able to correctly interact with my response headers. Should they be able to access them? Yes. Can they? Who knows. You don’t know if the consuming application is a server-side application written in C# or a browser-based application written in Javascript. Maybe the language or platform the consuming application is written in/on has a limitation of some of the headers not being able to be accessed as desired.

So what does this look like? Consider the following request and response examples:

Request

GET /v1.4.7/countries HTTP/1.1
Host: api.awesomesite.com
Accept: text/xml

Response

Status Code: 200 OK
Date: Wed, 16 Mar 2011 17:31:39 GMT
Vary: Accept
Content-Length: 432
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8"?>
<response>
	<countries>
		<country>
			<id>1</id>
			<name>United States of America</name>
			<regionCode>US</regionCode>
			<callingPrefix>011</callingPrefix>
			<callingCode>1</callingCode>
		</country>
		<country>
			<id>2</id>
			<name>Canada</name>
			<regionCode>CA</regionCode>
			<callingPrefix>011</callingPrefix>
			<callingCode>1</callingCode>
		</country>
	</countries>
</response>

This is a typical GET request for a resource and the response you would normally receive as a result. Following my third tenant of REST API implementation, you would want to modify your response so that instead it looks like this:

Response

Status Code: 200 OK
Date: Wed, 16 Mar 2011 17:31:39 GMT
Vary: Accept
Content-Length: 512
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8"?>
<response>
	<headers>
		<ResponseCode>200</ResponseCode>
		<Vary>Accept</Vary>
	</headers>
	<countries>
		<country>
			<id>1</id>
			<name>United States of America</name>
			<regionCode>US</regionCode>
			<callingPrefix>011</callingPrefix>
			<callingCode>1</callingCode>
		</country>
		<country>
			<id>2</id>
			<name>Canada</name>
			<regionCode>CA</regionCode>
			<callingPrefix>011</callingPrefix>
			<callingCode>1</callingCode>
		</country>
	</countries>
</response>

Notice that in this example I have included an additional “headers” node in my response. This is not part of the “countries” node, therefore it’s inclusion or exclusion in the response body will not impact our data being properly represented. What it has done though is ensure that in any situation in which response headers have gotten stripped, or a consuming application is not able to access them, that the header information is still represented. Which headers you choose to include in this node is completely up to you. I have elected to only include those that I deem the most important in maintaining RESTful communication. Concepts and data such as Response Codes and Vary headers are integral parts of RESTful communication – the date and time of the response are not. The reason I do not include the ‘Content-Type’ header and value in the “headers” node is because this header is used to tell the consuming application what format the response body is in. The consuming application will already have to know how to interpret the response in order to be told what format it is in, therefore it is information that is not needed (at the “headers” node level).

Conclusion

I hope that this post has gotten you thinking about some of the decisions you will have to make when developing your REST API and even if you don’t agree with the decisions I have made, you are at least now better prepared to face them. The one thing to remember, above all else, is that REST is a set of principles and not a specification. As long as you document your REST API and applications are capable of consuming it then no matter how you chose to implement it, it was the right way.

16 thoughts on “3 Tenets for Implementing a REST API

  1. Please don’t advocate your third point about duplicating protocol headers within the body of a representation. Headers are indeed part of a representation and are there to promote visibility – a key part of the uniform interface.

    1. Can you please share with me the position from which you do not like the advocating of the third point? Is it belief that there would never be a situation that the response headers would not be accessible or is it that if there is ever such a situation, effort should be made in helping the offenders adhere to the “standard” rather than spent on work arounds?

      I just finished reading your “Performance of RESTful Apps” post at http://www.subbu.org/blog/2011/03/performance-of-restful-apps and in the “Read Messages Completely” section you talk about some libraries that translate 4xx and 5xx response code into exceptions, and the need in those cases to additionally read the body. If there are libraries that do not correctly translate the 4xx and 5xx response codes, is that not justification for returning them in the response body as well so that the original, intended values can be retrieved?

      1. I have to say that I agree with Sabbu.

        Duplicating the headers in the response has a couple down sides:

        1. It increases bandwidth usage which is huge if your API is consumed on mobile platforms (and it probably will be).

        2. In a pure REST implementation, the body of the response should contain only the Document/Resource that you are accessing. Meta-data about that resource is explicitly why headers exist.

        Admittedly, that second point is academic. But it builds on Subbu’s “uniform interface” comment.

        Look at Twitter for instance. Arguably one of the biggest success stories in APIs. Rate limit information is only provided via the headers unless the return type is explicitly a rate limit error.

        One thing I like to do is contain a “code” property in error responses so that clients that do fishy things with response codes can still get the error number. But beyond that I think the cost outweighs the benefit.

        Of course, if someone wants to add it in their response they can. What I object to is calling it a “Tenet” for implementing REST. If anything, it is anti-REST and closer to SOAP responses.

        1. I agree with your objection to the use of “tenet”. After giving it some thought, I should have likely used a different word or description. This is not to say that I don’t agree with your other points. It’s just that I am still mulling them over :-)

  2. Excellent article. I like that you skipped over REST basics – there’s plenty of information on the basics already.

    I don’t disagree with your versioning, but I feel it may be too aggressive. Yahoo for example seems to only use major versions with their APIs, and will give plenty of notice before deprecating old ones. I don’t mind upgrading my consuming application… unless I’m done with it. If I have to crack it open, change a number, test and redeploy, that could be a drag.

    Perhaps minor versions should be considered beta releases. Of course as you say, this is up to the implementer.

    You kind of touched on identities but really didn’t go into user IDs in the URI. I would have liked to have seen your take on /1.5/countries/MikeWilcox … or is it /1.5/MikeWilcox/countries … or is it neither?

  3. The main problem with putting the version in the uri is that it makes the client’s life harder over time.

    Most non-trivial clients will need to store the uris of resources (think bookmarking). When supporting a new version of the service clients have to deal with the fact that the old stored uris are for a different version than the new uris. This means that once a client has supported a version of the api it has to support it forever.

    You could “solve” this problem in a non-restful way by rewriting the stored uris with the new version number. However, that will probably require quite detailed knowledge of implementation details of the new version. (For example, uris with a particular pattern where renamed. Some uris of another pattern where split to be to separate resources. Etc.) This sort of coupling is one of the things that rest is designed to help prevent.

    I think you can make versions in the uri work, and be restful. The approach you propose above does not meet the criteria, though.

    Oh, and +1 to Subbu’s comment. If you have trouble getting at the http header you need a better http library. Having the server server repeating itself is not the answer.

    1. I agree that putting version information in the URI will make life more difficult for clients, especially over time. I would prefer to never have to use the URI for that, as I believe that the ‘X-API-Version’ header should be all we ever need. While the problem exists though that this header cannot be relied on 100%, we are stuck with the problem of “principles versus specification”.

      An alternate approach would be to pass a parameter in the URI, such as ?version=2.3 which may or may not lessen the problems clients will experience over time. Given the needs we had, and our anticipation that any “major” version changes would be because of changes in concepts, managing versions of the APIs as their own applications seemed a better way to go. I suppose as with all things, time will only tell.

      Thank you very much for your comments. I have enjoyed the challenges to my thought processes they have introduced.

  4. I think another good idea is to have a “latest” version which automatically uses the most recent version. This way applications don’t have to update should a version become obsolete even though nothing is wrong and instead update when the app breaks.

    In regards to the URI+version controversy, you can make the version a query parameter instead. It would be equally weird to also include any authentication pieces as part of the path rather than in the query (if not using headers, which is “meh” to me about the proxy issue).

    I think it’s worth noting that both of these are what eBay implements and I like their versioning scheme of simply incrementing to the next odd number along with the version release notes and version support schedule.

    I also agree with Andrew: I think that once the headers aren’t used for meta-data and instead the the response body is used that it’s becoming SOAP-like and maybe should be SOAP, assuming that the XML is the only response type, so that existing SOAP libraries can be used.

  5. This is a good and mostly well thought out article on how to properly implement a RESTful service. I like that you have stayed away from the basics which we have been told about ad nauseam,

    That being said I take issues with your mentioning the term “business logic” when talking about REST.

    The term “business logic” is meaningless in the context of a RESTful service. REST knows about resources and their various representations and nothing else. If or if not posting a resource to an endpoint triggers additional business logic is beyond what is expressible in a RESTful context.

    Of course nothing keeps you from *emulating* a service e.g AccountService#addTelephoneNumberToExistingAccount by posting the *resource* TelephoneNumberAddition – notice the transition from verb to noun here – to some endpoint. However, in the context of a ROA – Resource Orientied Architecture – as opposed to a SOA this does not constitute a service with some business logic attached to it. It simply means posting a resource.

    Note that this reasoning leads to the same conclusion regarding how to properly implement versioning of a REST service (the term service in itself is a little unfortunate here, yet seems to be prevalent by now). Since REST does only know about resources and their representations, since representations are simply different views of the same resource, since resources are clearly central within a RESTful context, and finally since resources are uniquely defined by their URL, it seems natural to realize versioning in the way you described.

    But I would argue that a scenario where a “service”‘s version is incremented without changing the corresponding resource is beyond the realm of REST. From a RESTful point of view, absolutely nothing has changed, the underlying business logic – which has changed – is totally invisible to REST.

    REST is not just a different way of publishing services. It’s an entirely different view of what a service *is*.

    1. I have absolutely struggled with reconciling that 1) REST represents resources and their representations and with 2) the business application behind it all. From the REST standpoint I can very easily understand what ‘/customer/73/order/12′ represents. Sending a DELETE request to this resource, for example, would cause order #12 for customer #73 to be canceled. And again, from a REST perspective, that makes sense.

      But what does that actually mean? To me, this means that somewhere behind the API, in the application that the API provides access to, changes are going to be made to the data, following pre-determined business rules for doing so. So let’s say that today that means that the order is canceled and a refund check is cut to you and put in the mail.

      Now let’s say that we have changed our business processes and deleting an order no longer automatically causes a refund check to be cut. You have to indicate what you want to do with your refund, either get it as a check or apply it as a credit to your account, by using a different URI. The original ‘/customer/73/order/12′ URI has not changed any as a resource, but it’s behavior has absolutely changed. For me, the consumption of a API is more than just the representation of resources – it’s also a way to interact with a/the application.

      How, then, are the representations of resources and business logic to be reconciled? I ask this question without a concrete answer, because this seems to be a question with the area of REST API development to which there are many varied answers. I am very much interested in yours, and others, though, as I cannot learn and improve without others ideas. Thank you very much for your comment.

      1. I am currently pondering Darrel Miller’s comment that was posted after this one to which I replied, as perhaps that is part of the answer to my above-asked question.

  6. “But what happens when the business logic behind the POSTing to a resource does something different than it did before? The structure of the request, and perhaps even the response, remain the same, but the results of your actions are completely different. This is a version change too.”

    If a change to the implementation logic behind an API could cause a client to break then your API is poorly defined. The API is leaking implementation details.
    When a client makes a request it is merely communicating it’s intent, the API should make no guarantees about how that will be achieved.

    What can change is application workflow, but because REST apis are hypermedia driven, the workflow is explicit in the response representations and can be managed in the same way as the data.

  7. OK, I now realize that I’ve probably made a rather theoretical point here. I’m not at all criticizing your approach here, quite on the contrary. It’s all owing to my belief that the way we talk about some problem will unconsciously shape the solution we come up with.

  8. I think your approach to versioning is wrong because it violates fundamental axiom of the web, namely the
    – opacity axiom http://www.w3.org/TR/webarch/#uri-opacity
    – no aliases axiom http://www.w3.org/TR/webarch/#uri-aliases
    You tacitly acknowledge the second one when you say “a URI expresses identity, and identity does not change when a new version is introduced”.

    Putting versions in the URI violates opacity and forces clients to use out of band information (your URI construction schema), when all they should have to do is follow links.

    Your example of the customer changing from having one phone to two illustrates a data modeling problem: a resource cannot simultaneously have both 1 and 2 phone numbers. What does Joe’s record look like in your database? If you start with the first form and make the change to allow multiples and Joe actually adds a second phone number, then it is incorrect for Joe’s resource to provide the 1st representation. Joe no longer has a “you have exactly one phone number” representation. You could have modeled this differently and included an attribute for primary=”true” and keep Joe’s representation. Then Joe has properties as a resource that aren’t in the original representation, which is absolutely not noteworthy. At least we know it’s the same Joe, because the URI is the same.

    Asking “what happens when the business logic behind the POSTing to a resource does something different than it did before?” reveals the great beauty of REST, at least if you use HATEOAS correctly. The answer is nothing happens. The client is still presented with a representation uses hypermedia to define a menu of application state transitions. In the old media type representation the links must still be valid as well as in the new one. If you can’t do both, you must pick one media type or the other. Putting a version number in the API doesn’t solve anything in this regard and just confuses everything because intermediaries can’t tell you are talking about the same resource. It also means that you will have links to resources that CANNOT have a valid reprentation. Yuck.

    The worst thing about putting versions in the URI is that web you retire old versions you break everybody that stored those links. The web depends on permalinks, use them! How can you possibly expect not to have massive client breakage this way.

    REST is all about media types. Read the last few bullets of Roy’s fabulous post: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

  9. This discussion is very interesting. I have a restful applicaiton that uses xsd defined objects and JAXB beans for input/output. What is the best way to deal with xml validation, and changing schema versions as data to changing restulf apis?

    For instance the schema will define say Customer as having an address and a phone number and the restul api will use the generated bean as input/output for PUT/GET. Now imagine if the structure of the Customer xsd/bean needs to change significantly and I want to support legacy calls and new calls. Ideally I would like one implementation on the server that can recognize the version and produce the correct xml. Maybe that goal is impractical. Maybe I need parallel implementations for the different versions, at least for the translations of my data model into the JAXB beans data model.

    If I have a separate mime type for each schema will schema validation be easy to do? Will I still be able to consume/produce JSON or XML like I do now? If I use versioned uris then each url will be directed to the correct BEAN/Schema processing which can then call my more generic data modeling system.

    Anyway I appreciate your thoughts on this.

    Jon

  10. Have been researching the versioning issue but the one sentence that your article resonates with me is “As the consumptive programmer, I very often run across a situation where I mumble under my breath at the decisions made by the developing programmer in how they arrived at the solution I am attempting to use”. This articles reflects the view of this author based on this principle (which I happen to agree with).

    A lot of people say it is wrong for adding the version to the url. Is ebay wrong, is amazon wrong? I suppose so but in the end you are the one maintaining it.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>