RESTful Web Services

Leonard Richardson, Sam Ruby

Mentioned 48

Shows how to use the REST architectural style to create web sites that can be used by computers as well as machines, providing basic rules for using REST and real-life examples of such Web services.

More on Amazon.com

Mentioned in questions and answers.

What is an idempotent operation?

An idempotent operation can be repeated an arbitrary number of times and the result will be the same as if it had been done only once. In arithmetic, adding zero to a number is idempotent.

Idempotence is talked about a lot in the context of "RESTful" web services. REST seeks to maximally leverage HTTP to give programs access to web content, and is usually set in contrast to SOAP-based web services, which just tunnel remote procedure call style services inside HTTP requests and responses.

REST organizes a web application into "resources" (like a Twitter user, or a Flickr image) and then uses the HTTP verbs of POST, PUT, GET, and DELETE to create, update, read, and delete those resources.

Idempotence plays an important role in REST. If you GET a representation of a REST resource (eg, GET a jpeg image from Flickr), and the operation fails, you can just repeat the GET again and again until the operation succeeds. To the web service, it doesn't matter how many times the image is gotten. Likewise, if you use a RESTful web service to update your Twitter account information, you can PUT the new information as many times as it takes in order to get confirmation from the web service. PUT-ing it a thousand times is the same as PUT-ing it once. Similarly DELETE-ing a REST resource a thousand times is the same as deleting it once. Idempotence thus makes it a lot easier to construct a web service that's resilient to communication errors.

Further reading: RESTful Web Services, by Richardson and Ruby (idempotence is discussed on page 103-104), and Roy Fielding's PhD dissertation on REST. Fielding was one of the authors of HTTP 1.1, RFC-2616, which talks about idempotence in section 9.1.2.

I'm currently returning 401 Unauthorized whenever I encounter a validation failure in my Django/Piston based REST API application. Having had a look at the HTTP Status Code Registry I'm not convinced that this is an appropriate code for a validation failure, what do y'all recommend?

  • 400 Bad Request
  • 401 Unauthorized
  • 403 Forbidden
  • 405 Method Not Allowed
  • 406 Not Acceptable
  • 412 Precondition Failed
  • 417 Expectation Failed
  • 422 Unprocessable Entity
  • 424 Failed Dependency

Update: "Validation failure" above means an application level data validation failure, i.e., incorrectly specified datetime, bogus email address etc.

If "validation failure" means that there is some client error in the request, then use HTTP 400 (Bad Request). For instance if the URI is supposed to have an ISO-8601 date and you find that it's in the wrong format or refers to February 31st, then you would return an HTTP 400. Ditto if you expect well-formed XML in an entity body and it fails to parse.

Note (1/2016): Over the last five years WebDAV's more specific HTTP 422 (Unprocessable Entity) has become a very reasonable alternative to HTTP 400. See for instance its use in JSON API

Richardson and Ruby's RESTful Web Services contains a very helpful appendix on when to use the various HTTP response codes. They say:

400 (“Bad Request”)
Importance: High.
This is the generic client-side error status, used when no other 4xx error code is appropriate. It’s commonly used when the client submits a representation along with a PUT or POST request, and the representation is in the right format, but it doesn’t make any sense. (p. 381)

and:

401 (“Unauthorized”)
Importance: High.
The client tried to operate on a protected resource without providing the proper authentication credentials. It may have provided the wrong credentials, or none at all. The credentials may be a username and password, an API key, or an authentication token—whatever the service in question is expecting. It’s common for a client to make a request for a URI and accept a 401 just so it knows what kind of credentials to send and in what format. [...]

6/2015: Note that @ReWrite and @panteo make the case that HTTP 422 (Unprocessable Entity) from the IETF's WebDAV Proposed Standard is becoming a better choice than HTTP 400.

Keep in mind I have a rudimentary understanding of REST. Let's say I have this URL:

http://api.animals.com/v1/dogs/1/

And now, I want to make the server make the dog bark. Only the server knows how to do this. Let's say I want to have it run on a CRON job that makes the dog bark every 10 minutes for the rest of eternity. What does that call look like? I kind of want to do this:

URL request:

ACTION http://api.animals.com/v1/dogs/1/

In the request body:

{"action":"bark"}

Before you get mad at me for making up my own HTTP method, help me out and give me a better idea on how I should invoke a server-side method in a RESTful way. :)

EDIT FOR CLARIFICATION

Some more clarification around what the "bark" method does. Here are some options that may result in differently structured API calls:

  1. bark just sends an email to dog.email and records nothing.
  2. bark sends an email to dog.email and the increments dog.barkCount by 1.
  3. bark creates a new "bark" record with bark.timestamp recording when the bark occured. It also increments dog.barkCount by 1.
  4. bark runs a system command to pull the latest version of the dog code down from Github. It then sends a text message to dog.owner telling them that the new dog code is in production.

Why aim for a RESTful design?

The RESTful principles bring the features that make web sites easy (for a random human user to "surf" them) to the web services API design, so they are easy for a programmer to use. REST isn't good because it's REST, it's good because it's good. And it is good mostly because it is simple.

The simplicity of plain HTTP (without SOAP envelopes and single-URI overloaded POST services), what some may call "lack of features", is actually its greatest strength. Right off the bat, HTTP asks you to have addressability and statelessness: the two basic design decisions that keep HTTP scalable up to today's mega-sites (and mega-services).

But REST is not the silver bulltet: Sometimes an RPC-style ("Remote Procedure Call" - such as SOAP) may be appropriate, and sometimes other needs take precedence over the virtues of the Web. This is fine. What we don't really like is needless complexity. Too often a programmer or a company brings in RPC-style Services for a job that plain old HTTP could handle just fine. The effect is that HTTP is reduced to a transport protocol for an enormous XML payload that explains what's "really" going on (not the URI or the HTTP method give a clue about it). The resulting service is far too complex, impossible to debug, and won't work unless your clients have the exact setup as the developer intended.

Same way a Java/C# code can be not object-oriented, just using HTTP does not make a design RESTful. One may be caught up in the rush of thinking about his services in terms of actions and remote methods that should be called. No wonder this will mostly end up in a RPC-Style service (or a REST-RPC-hybrid). The first step is to think different. A RESTful design can be achieved in many ways, one way (the simplest, some might say) is to think of your application in terms of resources, not actions:

  • Instead of thinking in terms of actions ("do a search for places on the map"),
  • Think in terms of the results of that action ("the list of places on the map matching a search criteria").

I'll go for examples below. (Other key aspect of REST is the use of HATEOAS - I don't brush it here, but I talk about it quickly at another post.)

About the first design

Let's take a look a the proposed design:

ACTION http://api.animals.com/v1/dogs/1/

First off, we should not consider creating a new HTTP verb (ACTION). Generally speaking, this is undesirable for several reasons:

  • (1) Given only the service URI, how will a "random" programmer know the ACTION verb exists?
  • (2) if the programmer knows it exists, how will he know its semantics? What does that verb mean?
  • (3) what properties (safety, idempotence) should one expect that verb to have?
  • (4) what if the programmer has a very simple client that only handles standard HTTP verbs?
  • (5) ...

Now let's consider using POST (I'll discuss why below, just take my word for it now):

POST /v1/dogs/1/ HTTP/1.1
Host: api.animals.com

{"action":"bark"}

This could be OK... but only if:

  • {"action":"bark"} was a document; and
  • /v1/dogs/1/ was a "document processor" (factory-like) URI. A "document processor" is a URI that you'd just "throw things at" and "forget" about them - the processor may redirect you to a newly created resource after the "throwing". E.g. the URI for posting messages at a message broker service, which, after the posting would redirect you to a URI that shows the status of the message's processing.

I don't know much about your system, but I'd already bet both aren't true:

  • {"action":"bark"} is not a document, it actually is the method you are trying to ninja-sneak into the service; and
  • the /v1/dogs/1/ URI represents a "dog" resource (probably the dog with id==1) and not a document processor.

So all we know now is that the design above is not so RESTful, but what is that exactly? What is so bad about it? Basically, it is bad because that is complex URI with complex meanings. You can't infer anything from it. How would a programmer know a dog have a bark action that can be secretly infused with a POST into it?

Designing your question's API calls

So let's cut to the chase and try to design those barks RESTfully by thinking in terms of resources. Allow me to quote the Restful Web Services book:

A POST request is an attempt to create a new resource from an existing one. The existing resource may be the parent of the new one in a data-structure sense, the way the root of a tree is the parent of all its leaf nodes. Or the existing resource may be a special "factory" resource whose only purpose is to generate other resources. The representation sent along with a POST request describes the initial state of the new resource. As with PUT, a POST request doesn’t need to include a representation at all.

Following the description above we can see that bark can be modeled as a subresource of a dog (since a bark is contained within a dog, that is, a bark is "barked" by a dog).

From that reasoning we already got:

  • The method is POST
  • The resource is /barks, subresource of dog: /v1/dogs/1/barks, representing a bark "factory". That URI is unique for each dog (since it is under /v1/dogs/{id}).

Now each case of your list has a specific behavior.

1. bark just sends an email to dog.email and records nothing.

Firstly, is barking (sending an email) a synchronous or an asynchronous task? Secondly the bark request requires any document (the email, maybe) or is it empty?

1.1 bark sends an email to dog.email and records nothing (as a synchronous task)

This case is simple. A call to the barks factory resource yields a bark (an email sent) right away and the response (if OK or not) is given right away:

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(entity-body is empty - or, if you require a **document**, place it here)

200 OK

As it records (changes) nothing, 200 OK is enough. It shows that everything went as expected.

1.2 bark sends an email to dog.email and records nothing (as an asynchronous task)

In this case, the client must have a way to track the bark task. The bark task then should be a resource with it's own URI.:

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(document body, if needed)

202 Accepted
Location: http://api.animals.com/v1/dogs/1/bark/a65h44

This way, each bark is traceable. The client can then issue a GET to the bark URI to know it's current state. Maybe even use a DELETE to cancel it.

2. bark sends an email to dog.email and the increments dog.barkCount by 1

This one can be trickier, if you want to let the client know the dog resource gets changed:

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(document body, if needed)

303 See Other
Location: http://api.animals.com/v1/dogs/1

In this case, the location header's intent is to let the client know he should take a look at dog. From the HTTP RFC about 303:

This method exists primarily to allow the output of a POST-activated script to redirect the user agent to a selected resource.

If the task is asynchronous, a bark subresource is needed just like the 1.2 situation and the 303 should be returned at a GET .../bark/Y when the task is complete.

3. bark creates a new "bark" record with bark.timestamp recording when the bark occured. It also increments dog.barkCount by 1.

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(document body, if needed)

201 Created
Location: http://api.animals.com/v1/dogs/1/bark/a65h44

In here, the bark is a created due to the request, so the status 201 Created is applied.

If the creation is asynchronous, a 202 Accepted is required (as the HTTP RFC says) instead.

The timestamp saved is a part of bark resource and can be retrieved with a GET to it. The updated dog can be "documented" in that GET dogs/X/bark/Y as well.

4. bark runs a system command to pull the latest version of the dog code down from Github. It then sends a text message to dog.owner telling them that the new dog code is in production.

The wording of this one is complicated, but it pretty much is a simple asynchronous task:

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(document body, if needed)

202 Accepted
Location: http://api.animals.com/v1/dogs/1/bark/a65h44

The client then would issue GETs to /v1/dogs/1/bark/a65h44 to know the current state (if the code was pulled, it the e-mail was sent to the owner and such). Whenever the dog changes, a 303 is appliable.

Wrapping up

Quoting Roy Fielding:

The only thing REST requires of methods is that they be uniformly defined for all resources (i.e., so that intermediaries don’t have to know the resource type in order to understand the meaning of the request).

In the above examples, POST is uniformly designed. It will make the dog "bark". That is not safe (meaning bark has effects on the resources) nor idempotent (each request yields a new bark), which fits the POST verb well.

A programmer would know: a POST to barks yields a bark. The response status codes (also with entity-body and headers when necessary) do the job of explaining what changed and how the client can and should proceed.

Note: The primary sources used were: "Restful Web Services" book, the HTTP RFC and Roy Fielding's blog.


Update:

The original question asked about the design of a URI like:

ACTION http://api.animals.com/v1/dogs/1/?action=bark

Below is the explanation of why it is not a good choice:

How clients tell the server WHAT TO DO with the data is the method information.

  • RESTful web services convey method information in the HTTP method.
  • Typical RPC-Style and SOAP services keep theirs in the entity-body and HTTP header.

WHICH PART of the data [the client wants the server] to operate on is the scoping information.

  • RESTful services use the URI. SOAP/RPC-Style services once again use the entity-body and HTTP headers.

As an example, take Google's URI http://www.google.com/search?q=DOG. There, the method information is GET and the scoping information is /search?q=DOG.

Long story short:

  • In RESTful architectures, the method information goes into the HTTP method.
  • In Resource-Oriented Architectures, the scoping information goes into the URI.

And the rule of thumb:

If the HTTP method doesn’t match the method information, the service isn’t RESTful. If the scoping information isn’t in the URI, the service isn’t resource-oriented.

You can put the "bark" "action" in the URL (or in the entity-body) and use POST. No problem there, it works, and may be the simplest way to do it, but this isn't RESTful.

To keep your service really RESTful, you may have to take a step back and think about what you really want to do here (what effects will it have on the resources).

I can't talk about your specific business needs, but let me give you an example: Consider a RESTful ordering service where orders are at URIs like example.com/order/123.

Now say we want to cancel an order, how are we gonna do it? One may be tempted to think that is a "cancellation" "action" and design it as POST example.com/order/123?do=cancel.

That is not RESTful, as we talked above. Instead, we might PUT a new representation of the order with a canceled element sent to true:

PUT /order/123 HTTP/1.1
Content-Type: application/xml

<order id="123">
    <customer id="89987">...</customer>
    <canceled>true</canceled>
    ...
</order>

And that's it. If the order can't be canceled, a specific status code can be returned. (A subresource design, like POST /order/123/canceled with the entity-body true may, for simplicity, also be available.)

In your specific scenario, you may try something similar. That way, while a dog is barking, for example, a GET at /v1/dogs/1/ could include that information (e.g. <barking>true</barking>). Or... if that's too complicated, loosen up your RESTful requirement and stick with POST.

Update:

I don't want to make the answer too big, but it takes a while to get the hang of exposing an algorithm (an action) as a set of resources. Instead of thinking in terms of actions ("do a search for places on the map"), one needs to think in terms of the results of that action ("the list of places on the map matching a search criteria").

You may find yourself coming back to this step if you find that your design doesn't fit HTTP's uniform interface.

Query variables are scoping information, but do not denote new resources (/post?lang=en is clearly the same resource as /post?lang=jp, just a different representation). Rather, they are used to convey client state (like ?page=10, so that state is not kept in the server; ?lang=en is also an example here) or input parameters to algorithmic resources (/search?q=dogs, /dogs?code=1). Again, not distinct resources.

HTTP verbs' (methods) properties:

Another clear point that shows ?action=something in the URI is not RESTful, are the properties of HTTP verbs:

  • GET and HEAD are safe (and idempotent);
  • PUT and DELETE are idempotent only;
  • POST is neither.

Safety: A GET or HEAD request is a request to read some data, not a request to change any server state. The client can make a GET or HEAD request 10 times and it's the same as making it once, or never making it at all.

Idempotence: An idempotent operation in one that has the same effect whether you apply it once or more than once (in math, multiplying by zero is idempotent). If you DELETE a resource once, deleting again will have the same effect (the resource is GONE already).

POST is neither safe nor idempotent. Making two identical POST requests to a 'factory' resource will probably result in two subordinate resources containing the same information. With overloaded (method in URI or entity-body) POST, all bets are off.

Both these properties were important to the success of the HTTP protocol (over unreliable networks!): how many times have you updated (GET) the page without waiting until it is fully loaded?

Creating an action and placing it in the URL clearly breaks the HTTP methods' contract. Once again, the technology allows you, you can do it, but that is not RESTful design.

Seems like there are two categories of APIs for websites today.

  1. APIs which allow the functionality of the site to be extended like Facebook, Myspace, etc. These APIs seem to be very diverse.

  2. APIs which allow interaction with the existing site functionality like Twitter, Flickr, etc. These all claim to be REST based, but are in reality simply "data over HTTP".

If you were creating a website that allowed both functional extension and outside interaction, what existing APIs would you use as a reference model?

We're doing some research in this area ourselves. Not a lot out there in terms of "gold standard" for website API references.

The most common website APIs referenced are:

Another list here:

http://www.pingable.org/the-top-15-web-apis-for-your-site/

Someone recommended the book Restful Web Services as a good reference on this.

(please feel free to edit the above list to add other high profile websites with APIs)

In my attempt to redesign an existing application using REST architectural style, I came across a problem which I would like to term as "Mediatype Explosion". However, I am not sure if this is really a problem or an inherent benefit of REST. To explain what I mean, take the following example

One tiny part of our application looks like:

collection-of-collections->collections-of-items->items

i.e the top level is a collection of collections and each of these collection is again a collection of items.

Also, each item has 8 attributes which can be read and written individually. Trying to expose the above hierarchy as RESTful resources leaves me with the following media types:

application/vnd.mycompany.collection-of-collections+xml
application/vnd.mycompany.collection-of-items+xml
application/vnd.mycompany.item+xml

Further more, since each item has 8 attributes which can be read and written to individually, it will result in another 8 media types. e.g. one such media type for "value" attribute of an item would be:

application/vnd.mycompany.item_value+xml

As I mentioned earlier, this is just a tiny part of our application and I expect several different collections and items that needs to be exposed in this way.

My questions are:

  1. Am I doing something wrong by having these huge number of media types?
  2. What is the alternative design method to avoid this explosion of media types?

I am also aware that the design above is highly granular, especially exposing individual attributes of the item and having separate media types for each them. However, making it coarse means I will end up transferring unnecessary data over the wire when in reality the client only needs to read or write a single attribute of an item. How would you approach such a design issue?

You're using the media type to convey details of your data that should be stored in the representation itself. So you could have just one media type, say "application/xml", and then your XML representations would look like:

<collection-of-collections>
    <collection-of-items>
        <item>
        </item>
        <item>
        </item>
    </collection-of-items>
    <collection-of-items>
        <item>
        </item>
        <item>
        </item>
    </collection-of-items>
</collection-of-collections>

If you're concerned about sending too much data, substitute JSON for XML. Another way to save on bytes written and read is to use gzip encoding, which cuts things down about 60-70%. Unless you have ultra-high performance needs, one of these approaches ought to work well for you. (For better performance, you could use very terse hand-crafted strings, or even drop down to a custom binary TCP/IP protocol.)

Edit One of your concerns is that:

making [the representation] coarse means I will end up transferring unnecessary data over the wire when in reality the client only needs to read or write a single attribute of an item

In any web service there is quite a lot of overhead in sending messages (each HTTP request might cost several hundred bytes for the start line and request headers and ditto for each HTTP response as in this example). So in general you want to have less granular representations. So you would write your client to ask for these bigger representations and then cache them in some convenient in-memory data structure where your program could read data from them many times (but be sure to honor the HTTP expiration date your server sets). When writing data to the server, you would normally combine a set of changes to your in-memory data structure, and then send the updates as a single HTTP PUT request to the server.

You should grab a copy of Richardson and Ruby's RESTful Web Services, which is a truly excellent book on how to design REST web services and explains things much more clearly than I could. If you're working in Java I highly recommend the RESTlet framework, which very faithfully models the REST concepts. Roy Fielding's USC dissertation defining the REST principles may also be helpful.

I'm looking for a book or any other resource that will help me learn how to create RESTful APIs in Java. Looking on Amazon, I saw that there are several solutions for RESTful Java, but I'm looking for the one that is tailored to a novice.

Looking forward to getting your advices/opinions, thanks!

My recommendation would be this one:

alt text

I don't think I can point to only one resource but I would take a path (which you can customize based on your level of understanding of REST). I'm somebody who would like to get my concepts real clear first and then think about the tools to implement the concepts.

Obviously, I haven't provided a single resource, but something in the lines I've provided would serve well, IMO.

I found REST in Practice to be the best book covering different styles of REST architectures. It's also far more practical in its advice than many other books (I wasn't impressed by RESTful Web Services as I think it lacks focus).

Every tutorial or explanation of REST just goes too complicated too quickly - the learning curve rises so fast after the initial explanation of CRUD and the supposed simplicity over SOAP. Why can't people write decent tutorials anymore!

I'm looking at Restlet - and its not the best, there are things missing in the tutorial and the language/grammar is a bit confusing and unclear. It has took me hours to untangle their First Steps tutorial (with the help of another Java programmer!)

RESTlet Tutorial Comments

Overall I'm not sure exactly who the tutorial was aimed at - because there is a fair degree of assumed knowledge all round, so coming into REST and Restlet framework cold leaves you with a lot of 'catchup work' to do, and re-reading paragraphs over and over again.

  1. We had difficulty working out that the jars had to be in copied into the correct lib folder.

  2. Problems with web.xml creating a HTTP Status 500 error -

The server encountered an internal error () that prevented it from fulfilling this request

, the tutorial says:

"Create a new Servlet Web application as usual, add a "com.firstStepsServlet" package and put the resource and application classes in."

This means that your fully qualified name for your class FirstStepsApplication is com.firstStepsServlet.FirstStepsApplication, so we had to alter web.xml to refer to the correct class e.g:

original:

<param-value>
         firstStepsServlet.FirstStepsApplication
</param-value>

should be:

<param-value>
         com.firstStepsServlet.FirstStepsApplication
</param-value>

Conclusion

I was under the impression that the concepts of REST were supposed to be much simpler than SOAP - but it seems just as bad if not more complicated - don't get it at all! grrrr

Any good links - much appreciated.

It sounds like you could use a solid understanding of the fundamentals of REST, and for that I highly recommend RESTful Web Services by Leonard Richardson and Sam Ruby. I provides a great introduction to REST: what it is and how to implement a (practical) RESTful web service.

Most of the example code in the book is actually Ruby, but it's easy enough to understand even if you're not a Ruby expert. But one thing that should help you specifically is that one of the later chapters of the book contains overviews of several RESTful frameworks, including Restlet. It doesn't really get into any code (it's a 50,000-foot flyover) but I think it'll give you just what you need at this stage.

I'm looking for some resources on taking an existing monolithic Rails 3.0 application (35K LOC) and breaking it apart into an SOA design. Any books, blogs, screencasts, or example applications would be awesome.

The main questions I'm looking to answer are:

  • Is SOA even the right design?
  • Where do I start?
  • What are some common pitfalls I can avoid?
  • What should I be thinking about now vs what can I do later? (ie performance)

Some resources I've seen but not totally sure if they are the right places to start:

Is SOA even the right design?

It depends. Don't you hate these kinds of answers? Breaking up your app to loosely coupled services using messaging or API calls would, by definition, be implementing SOA.

The beauty of it is that you can interchange service implementation without changing their interfaces and allow for independent deployments without having to bring down the whole app. Also, I'd implement the SOA via specialised API controllers that are versioned and expose custom state rather than their whole state which you'd reserve for authenticated users or role-based sessions.

The dilemma, from my experience, is whether to implement synchronous or asynchronous calls. Synchronous calls are obviously easier to program, but may leave your user hanging while they are being executed, and you'd have to handle timeouts for long-running queries. Watch out for database and web server timeouts.

If you implement asynchronous calls, let's say via ActiveMessaging or alike, youd have to handle callbacks or some kind of notifications to bubble up to your user. It also entails setting up primary and secondary message brokers and maybe some JavaScript or pollers to check for status. It's all fun though!

Where do I start?

I'd first see if it's "worth it": after all, SOA is cool, but does introduce multiple points of failure that you do not have currently. If you think your broken up app will result in discrete services that are HA and will serve other projects, I'd start with "the druby book" and "service oriented" as you mentioned.

What are some common pitfalls I can avoid?

I think the biggest concern would be transactions across multiple services and the ability to roll-back the whole operation if a distant service fails. The problems begins if you are in some operation where you call A and it calls B and B calls C and C fails. Who knows that C failed? How will you tell B and A to roll back? Can they? Do they save state? Tough questions for upfront design.

One other issue is that life gets complicated when you throw a workflow on top of your SOA: who's the keeper of the business process? Centralised or distributed? It's all absolutely cool stuff again, but heaviness creeps in, no? But that's life if you must move to SOA.

What should I be thinking about now vs what can I do later? (ie performance)

I'd factor out the obvious generic services that can be used in other apps right now. I would not over-SOA your environment to avoid adding points of failure and keep the ones SOA introduces to a minimum.

I'm building a framework and want developers who build with it to have the ability to allow parts of it to both share data with other sites and allow other sites to add/edit/delete data.

For example, if someone makes a site that has book reviews, authors, quotes, code examples, comments, etc. the developer could make e.g. "book reviews" read-only for other sites and "comments" readable by other sites and writable by certain sites/users. The idea is to use the framework to build applications that can easily be interconnected with other applications.

I envision enabling all interaction with the site via POST and GET which would look something like this:

  • /books.php?category=ruby (returns an XML collection of books about ruby)
  • /books.php?id=23 (returns the XML for a specific book)
  • /books.php?action=add&title=AdvancedRuby&description=....&securityId=923847203487
  • /books.php?action=delete&id=342&securityId=923847203487

Other applications could also "discover and consume" what a certain site has to offer by doing this:

  • /discover.php (returns XML of all public classes and actions available)

Really this is all I need to enable the framework to be a way for developers to quickly create loosely connected sites.

What I want to know is, before I begin implementing this, are there significant/interesting parts of REST that I do not yet understand which I should be building into the framework, e.g.:

  • REST requires GET, POST, PUT and DELETE. Why would I ever need "PUT" and "DELETE"? Am I locking myself out from taking advantage of some standard if I dont' use these?
  • My "discover.php" file functions similarly to a WSDL file in web services. I am surprised in descriptions of REST there seems to be no standardized way of discovering the services that a RESTful service offers, or is there?
  • If a client website tries to e.g. add a book to a server website and does not get any "success" response back, it would simply try again until it got a response. The server website would simply not add the same book twice. This is my understanding of data integrity in REST, is there more to it than this?
  • eventually I want to have multiple sites that have the same rich classes e.g. "BookReview" so that a client site would be able to execute code such as this:

    $bookReview = new BookReview("http://www.example.com/books.php?id=23"); $book->informAuthor("a comment about your book review was posted on our site...");

and the server site would send an e-mail off to the author of that review. Is this type of type interaction a component of the RESTful philosophy or is REST simply the exchange of data via XML, JSON?

Am I locking myself out from taking advantage of some standard if I dont' use these?

You are yourself locking out from the HTTP standard. Of course you can use GET parameters to do the same thing. It's just not REST then, but something RPC-Like.

May I suggest the book RESTful Web Services by Leonard Richardson and Sam Ruby? It's quite fun to read and shows differences between the different approaches.

To answer your questions in a bit more detail: It's up to you to decide which way you go. In theory you can do all the same stuff with both RESTful and RPC-like approaches. With RESTful you use the underlaying HTTP protocol to be the protocol. With RPC you use HTTP just as a means of transportation and hide the work orders somewhere in the transported data. That leads to (unrequired) overhead.

Just look at two of your examples:

  • /books.php?action=add&title=AdvancedRuby&description=....&securityId=923847203487
  • /books.php?action=delete&id=342&securityId=923847203487
    • There's POST and PUT or DELETE, why have action=add and action=delete?
    • There's HTTP authentication. Why invent a - possibly less secure - securityId?
    • BTW: You shouldn't allow changes to data via GET. That's just something that shouldn't be done (another topic, though ;) )

I am creating a new REST service.

What is the standard for passing parameters to REST services. From different REST implementations in Java, you can configure parameters as part of the path or as request parameters. For example,

Path parameters http://www.rest.services.com/item/b

Request parameters http://www.rest.services.com/get?item=b

Does anyone know what the advantages/disadvantages for each method of passing parameters. It seems that passing the parameters as part of the path seems to coincide better with the notion of the REST protocol. That is, a single location signifies a unique response, correct?

Your second example of "request parameters" is not correct because "get" is included as part of the path. GET is the request type, it should not be part of the path.

There are 4 main types of requests:

 GET
 PUT
 POST
 DELETE

GET requests should always be able to be completed without any information in the request body. Additionally, GET requests should be "safe", meaning that no significant data is modified by the request.

Besides the caching concern mentioned above, parameters in the URL path would tend to be required and/or expected because they are also part of your routing, whereas parameters passed in the query string are more variable and don't affect which part of your application the request is routed to. Although could potentially also pass a variable length set of parameters through the url:

GET somedomain.com/states/Virginia,California,Mississippi/

A good book to read as a primer on this topic is "Restful Web Services". Though I will warn you to be prepared to skim over some redundant information.

I just finished reading Restful Web Services and Nobody Understands REST or HTTP and am trying to design an API with a RESTful design.

I've noticed a few patterns in API URI design:

http://api.example.com/users
http://example.com/api/users
http://example.com/users

Assume that these designs properly use Accept and Content-type headers for content negotiations between XHTML, JSON, or any format.

Are these URI's a matter of a pure RESTful implementation vs implicit content negotiation?

My thoughts are that by explicitly using API in the URI is so that a client will expect a data format that is not inherently human pleasing hypermedia and can be more easily consumed without explicitly setting an Accept header. In other words, the API is implying that you expect JSON or XML rather than XHTML.

Is this a matter of separating resource representations logically on the server side?

The only justification I can come up with for why someone would design their URI's with an API subdomain is because, based off my assumption that this is a scaling technique, it should make routing request load easier in a multi-tiered server infrastructure. Maybe situations exist where reverse proxies are stripping the headers? I don't know. Different servers handling different representations?

Maybe a subdomain is used for external consumers only so that the server avoids the overhead from internal usage. Rate limiting?

Am I missing a point?

My proposed design would attempt to follow RESTful practices by setting appropriate headers, using HTTP verbs appropriately and representing resources in a fashion that I feel including 'API' in the URI would be redundant.

Why would someone design a RESTful API with 'API' in the URI?

Or could they? Maybe my problem with not understanding this design is that it doesn't matter as long as it follows some combination of specification which may not lead to a RESTful API implementation but close? There is more than one way to skin a keyboard cat. HATEOAS related?


Update: While researching this topic I have come to the conclusion that it's important to consider ideas from REST but not to treat it as a religion. Thus, whether or not to have 'api' in the URI is more of a design decision than a steadfast rule. If you plan to expose your website's API publicly it would be a good idea to use an api subdomain to help deal with the application's logic. I hope that someone will contribute their insight for others to learn from.

This is my understanding and point of view about your question :

Are these URI's a matter of a pure RESTful implementation vs implicit content negotiation?

No, they are not part of any official documentation (that I know of), and are usually at the developper/team standards discretion. For example, Zend Framework uses some utility classes to detect XHR requests and how responses should be returned. Granted, ZF is not pure RESTful design (or most PHP application as a matter of fact), but it is still possible to write such designs even in PHP.

I have often seen api being used in urls when the returned data expected is indeed not meant to be used as is for display to the end user. However it is usually a design decision and not necessarily an implied standard.

Is this a matter of separating resource representations logically on the server side?

More or less. For example, when performing a PUT request, for example, the interface does not necessarily need to be entirely refreshed, and often a simple response message is good enough. While this response message is part of the view, it is not the view. So, for example, http://domay.com/users/ would return the interface to manage users (or whatever), http://domain.com/users/api would perform operations and return interface updates (without a page reload).

Am I missing a point?

Weell, no. Since it is a design decision to use or not api in the URL, you're not missing anything here. With the proper headers,

http://domain.com/users/api/get
http://domain.com/users/get
http://domain.com/users/

can all be valid RESTful requests.

Why would someone design a RESTful API with 'API' in the URI?

Simply put, to have an URL naming convention. So that, when looking at a request in logs, documentation, etc. One will understand it's purpose.

I am currently struggling with a design issue involving REST. The application I am designing has a requirement to send out events and also support pub/sub style of interaction. I am unable to come up with a design to provide these interaction style without breaking the REST’s “Stateless interaction” constraint. I am not against polling as some people seem to be (polling sucks) , but my application demands a event based and pub/sub style of interaction (polling is not an option for me). So, my question is:

  1. Can I design a RESTful application that supports event based and pub/sub interactions without breaking the REST contraint?
  2. Is REST style suitable for this kind of interaction style?

I'd recommend the Distributed Observer Pattern by Duncan Cragg as a good read (bit difficult to grok but worth the effort).

As others have indicated its likely you'll need to use polling but as you rightly say subscribers could register their own interest (POST to create subscription). If you view the subscription as its own resource, a contract between the publisher and subscriber, then I wouldn't view it as a breaking REST constraints (see State and Statelessness at page 217 of RESTful Web Services for the difference between application and resource state)

I'm about to write an api and thought I'd find some good recommendations on things to look out for, how to write a good api.

I of course did a google search, but aside from this from Dustin Diaz http://www.dustindiaz.com/api-writing-tips/, I haven't really been able to find good recommendations.

What are things that you wish you had done differently when creating an API? What made the biggest difference.

I assuming I'm going to use oauth.

I'm purposely not providing details of the api, as I'm looking for more general recommendations that will hopefully be useful to a larger number of people.

Key points I would recommend you look at:

  • Implement a RESTful interface
  • Offer a variety of data formats (JSON, XML, etc)
  • Make the syntax intuitive and easy to understand
  • Thorough documentation
  • Use proper response codes

Also, here are some links you may find useful:

http://www.slideshare.net/eaton/building-apis-that-rock
http://www.notiondesign.ca/blog/story/how-not-to-build-an-api/
http://blog.apigee.com/category/api_best_practices/
http://blog.isnotworking.com/2007/05/api-design-guidelines.html

Also, this is a great book that may help you get started:

http://www.amazon.com/dp/0596529260/

I'm just getting started with REST and I've been reading this post and the mentioned book about REST response codes. When I look at Play's Controller class however, it seems to be limited to returning

  • 200 - OK
  • 301 - Moved Permanently
  • 302 - Found
  • 304 - Not Modified
  • 400 - Bad Request
  • 401 - Not Authorized
  • 403 - Forbidden
  • 404 - Not Found
  • 5xx

That seems to leave out some potentially useful codes that were mentioned:

  • 201 - Created (good response for successful JSON post?)
  • 202 - Accepted (for queued requests)
  • 204 - No Content (possible response for successful PUT/POST/DELETE)
  • 307 - Temporary Redirect
  • 405 - Method Not Allowed
  • 406 - Not Acceptable
  • 409 - Conflict
  • 410 - Gone
  • 415 - Unsupported Media Type (this seems like the appropriate response for a request for JSON format when no JSON template is defined)

Are those not needed after all? Is Play handling those situations automatically?

Also it appears that one controller can't handle REST requests and normal web page requests for the same resource very well since the web pages are always returned with 200. Am I missing anything there?

Looking at the Play Source code (Play 1.1) at the play.mvc.Http.StatusCode object, Play appears to have the following codes

public static final int OK = 200;
public static final int CREATED = 201;
public static final int ACCEPTED = 202;
public static final int PARTIAL_INFO = 203;
public static final int NO_RESPONSE = 204;
public static final int MOVED = 301;
public static final int FOUND = 302;
public static final int METHOD = 303;
public static final int NOT_MODIFIED = 304;
public static final int BAD_REQUEST = 400;
public static final int UNAUTHORIZED = 401;
public static final int PAYMENT_REQUIERED = 402;
public static final int FORBIDDEN = 403;
public static final int NOT_FOUND = 404;
public static final int INTERNAL_ERROR = 500;
public static final int NOT_IMPLEMENTED = 501;
public static final int OVERLOADED = 502;
public static final int GATEWAY_TIMEOUT = 503;

This would indicate acknowledgement of SOME of the codes your have identified, such as 201, 202, 204. However, the values 307, 405, 406, 409, 410 and 415 are not there.

Also, 201, 202, 204 are acknowledged, but are not referenced anywhere else within the source code. So unless the Netty server, or one of the supplied jar files is managing these for Play (which I am not sure it can do), I can't see how Play can magically handle these situations without knowing the code base.

Looking at the code for renderJSON, it does not appear to set the status code as part of sending the results back (so uses the default 200), so the following hack may work.

public static void myJsonAction() {
    response.status = 201;
    renderJSON(jsonString); // replace with your JSON String
}

I'm developing a simple RESTful API, and utterly in love with how minimalistic it is. But I'm uncertain as to the correct HTTP response codes for various situations:

  1. Incorrectly formed query

  2. Correctly formed query refers to a resource which does not exist

  3. Resource successfully deleted

  4. Resource successfully edited

I'm currently thinking that 1 would be 403 Forbidden; 2 would be 410 Gone; 3 and 4 would be 202 Accepted. Do they sound right?

Get the Richardson & Ruby book - it has a useful appendix on your question and is required reading either way

Is there a specific pattern that developers generally follow? I never really gave it much thought before in my web applications, but the ASP.NET MVC routing engine pretty much forces you to at least take it into consideration.

So far I've liked the controller/action/index structure (e.g. Products/Edit/1), but I'm struggling with more complex urls.

For instance, let's say you have a page that lists all the products a user has in their account. How would you do it? Off the top of my head I can think of the following possibilities for a listing page and an edit page:

  1. User/{user id}/Products/List, User/{user id}/Products/Edit/{product id}
  2. User/{user id}/Products, User/{user id}/Products/{product id}
  3. Products?UserID={user id}, Products/Edit/{product id}

I'm sure there are plenty of others that I'm missing. Any advice?

To add to troethom's comments, RESTful generally also means that, for example, to create a new user you would PUT a representation to /users/newusername

RESTful basically uses the 5 standard HTTP Methods (GET, PUT, POST, DELETE, HEAD) for controlling/accessing content.

Ok, this isn't easy for a web browser, but you can always use overloaded POST (post to /users/username with a representation of a user to change some of the details, etc.

Its a good way of doing things, I'd reccommend reading RESTFul Web services to get a better understanding :D (and it's a darn good book!)

I am creating an iOS application that I need to connect to a database through a web service. I only know basic knowledge about using RESTful web services, I have never written my own before and was wondering if you can give me any advice on where I can find out how to write my own RESTful web service.

In my iOS program I will be sending a part number to the web service the web service will then need to return color and size information on the part. I'm not sure if XML is the best format or is there something better?

I guess my question is twofold here:

  1. Is this something I should be doing with a RESTful web service?
  2. Where can I find tutorials on creating a .NET-based RESTful web service?

Seeing as you have little experience with REST, I would first learn about the concept. It is important to understand what it is (it is not just pretty URLs) fundamentally before proceeding to design your service.

  1. I would start with reading Chapter 5 of Roy Fielding's dissertation (where the term REST originated - read the whole paper if you like).
  2. I would then move on to the excellent RESTful web services.
  3. Finally, I would then read RESTful .NET.

If you gave yourself 2 days you could read, understand and digest all of these resources no problem.

You will probably end up using WCF - you can get the REST Starter Kit from here

I'm developing a REST webservice (Java, Jersey). The people I'm doing this for want to directly access the webservice via Javascript. Some instinct tells me this is not a good idea, but I cannot really explain that instinct. My natural approach would have been to have the webservice do the real logic and database access, but also have some (relatively thin) server-side script layer (e.g. in PHP). Clients would talk to the PHP layer which in turn would talk to the webservice. (The webservice would be pretty local to the apache/PHP server and implicitly trust calls from the script layer. The script layer would take care of session management.) (Btw, I am not talking about just hiding the webservice behind an Apache which simply redirects calls.)

But as I find myself at a lack of words/arguments to explain my instinct, I wonder whether my instinct is right - note that while I have been developing all kinds of software in all kinds of languages and frameworks for like 17 years, this is the first time I develop a webservice.

So my question is basically: what are your opinions? Are there any standard setups? Is my instinct totally wrong? Or partially? ;P

Many thanks,

Max

PS: I might add a few bits of information about the planned usage of the whole application:

  • will be accessed by different kinds of users, partly general public, partly privileged
  • thus, all major OS/browser combinations can be expected as clients
  • however, writing the client is not my responsibility
  • will potentially have very high load/traffic
  • logic of webservice will later be massively expanded for another product which is basically a superset of the functionality of the current project
  • there is a significant likelihood that at some point an API should be exposed which can be used by 3rd party developers - obviously, with some restrictions
  • at some point, the public view of the product should become accessible via smartphones, too (in other words, maybe a customized version of the site to adapt to the smaller display and different input methods)

Firstly I am just extending on what Daff replied above. I am extending Daff's answer from the point of my learning or designing and implementing RESTful WebServices and please note that I am still learning.

When I started learning RESTful WS with Java, Jersey (0.3 IIRC), I had similar questions and the primary cause for that is "Total" mis-conception about RESTful Architecture. The most "Grave" mistake I performed was using JAXB for XML and Jackson for JSON (de)serialization directly from/to the persistence beans. This totally violates the REST principal and hence creating some vital issues in creating a high performance, highly available, scalable web service.

My mistake was, thinking in terms of API a.k.a Service, when we think RESTful WS we should forget "API" and think Resources. We should take great care in interlinking resources. My understanding of this only came after reading this, I suggest it to anyone wanting to create their own web service. My conclusion is what is Resource is to RESTful WS/Architecture what API to a native interface or SOAP Web Service. So I would suggest design your resources with care and understand that there is no limit in how resources your WebService may have.

So here comes how I concluded in implementing systems exposing an "API" through RESTful WS. I create an API which deals communicating with business entities, for example, PersistentBook, which contains either Id of PersistentAuthor or the object itself. All business logic considering persistent entities lie in the API implementation layer.

The web service layer uses the API layer to perform its operations on resources. Web service layer uses persistent entities to generate representations of beans and vice versa, the key feature here would be PersistentBook's representation would have a URI to the PersistentAuthor. If I want to use automated (de)serialization I create another domain layer, e.g. Book, Author etc.

Now as Daff mentioned caching would be inevitable, my checkpoints for them are -

  • Support for 'Cache-Control', 'Last-Modified', 'ETag' response headers and 'If-Modified-Since', 'If-Match-None' request headers are key. Note from my more recent learnings - use 'Vary' header in case of varying representations (content negotiation) based on 'Accept' header.
  • Using a server side caching such as Squid, Varnish in case clients do not use caching. One thing I learnt having all the right header support counts for nothing if clients do support them and in fact increases the cost in terms of computation and badnwidth ;)
  • Use of Content-Encoding.

The resource /user/12345 doesn't exist. Lets say the consumer is trying different ids randomly. There is no authorization. Any user can view any user. In a broader sense, my question is "What should you return if you do a GET on a resource that doesn't exist?"

Should I return an empty user for an id that doesn't exist or should I return an error message with proper status code?

What is the typical/usual/recommended practice?

@Byron is right, return HTTP 404. You want to leverage all of the capabilities of HTTP, and these include response status codes. So if there is a client error, return a 4xx error code, and if your server code has an internal problem, return a 5xx error code, etc.

Richardson and Ruby's RESTful Web Services (O'Reilly) has a good discussion of this, and an appendix with all the most important HTTP error codes and when to use them.

Are there any good patterns/practices used while designing Services. I came across this post today:

When to Use the Decorator Pattern?

Though I didn't completely understand but it really gives a new direction to think about designing services.

Note: This question is not any technology specific.

Here is my list:

  1. Read a book on real-life service design. For a Restful approach i recommend Restful Web Services
  2. Spec it: Designing a service in human code is much easier to discuss and change than implementing it and then discover it's wrong.
  3. Write integration (service tests) in a different language: You can be fooled into thinking that your service is real spiffy by using the same tech on both client and server. Implementing a RESTful service in Java?, then write your service tests in (J)Ruby, creating .NET SOAP service ? then write your service tests in Java.

We have an ecommerce website that displays groups of products by category using a URL format that maps almost exactly to the REST URL format we would like to use for our forthcoming API.

e.g. example.com/products/latest or example.com/products/hats

Is it a valid pattern to use the same URL for visible (HTML) and invisible (JSON) results, and to use the Accept http request header to determine what should be returned.

i.e. if you call example.com/products/latest with Accept: application/json you get just the product data, but if you use text/html you get the full HTML page (header, footer, site chrome etc.)

And if so, is this a good idea - will we run into problems if, for instance, the website needs to change, but the API needs to be stable?

UPDATE: some helpful resources - here is an article[1] by Peter Williams discussing the use of the HTTP Accept header to version APIs, and I have also referenced an SO question[2] that reveals some of the problems of using this approach. Probably better to use a custom HTTP header?

[1] Making the case for using Accept: http://barelyenough.org/blog/2008/05/versioning-rest-web-services/
[2] Problems with jQuery (& IE): Cannot properly set the Accept HTTP header with jQuery
[3] Making the case for using Accept: http://blog.steveklabnik.com/2011/07/03/nobody-understands-rest-or-http.html
[4] Sitting on the fence: http://www.informit.com/articles/article.aspx?p=1566460

I have no experience with this, but Restful Web Services recommends that you version your API via the URL (e.g. api.example.com/v1/products/hats) — I’m not sure that would fit with using the same URLs for the website and the API.

I am developing a web app based on dojo framework.

I decided to use dojox.data.JsonRestStore for communication with the server.

For example, I have this representation of Order (ecommerce) in Json:

{
id: int,
name: str,
date: str,
items: [
    {
        id: int,
        name: str,
        pcs: int,
        price: int,
        total: int

    }, ...
]
}

Order has some basic attributes (id, name, date) and it also contains array of ordered items.

I am not sure if this is good (REST) design and I am wondering if the ordered items should be in a separate resource (and so in separate jsonRestStore).

I think I may have trouble with the current object model when I want to display the Orders' basic attributes in dojo Form and the ordered items in dojo Datagrid.

So my question, is my current approach OK - in way of creating REST client app? Also, what is the correct way to implement my example form with datagrid in dojo?

While your design is RESTful and there is nothing in the REST architecture that requires it, I think most people would agree that keeping your resources separate is the best way to go. If you check out Restful Web Services they describe this as the Resource Oriented Architecture.

I have worked out an example that keeps the orders and items in separate JsonRestStores and will allow you to display the Order object via a Dojo form, and the order's items via a Dojo DataGrid. I think the code is straightforward and I also added some comments in the code to try and clear things up.

I created a simple ASP.NET MVC 3 backend to provide some dummy data for the example. I have posted that code as well, though you are really only interested in the "View' portion of the code, the backend code may also help you figure out or change any of the backend code you have.

View:

<%@ Page Language="C#" Inherits="System.Web.Mvc.ViewPage" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
   <title>JsonRestStore Datagrid Example</title>

<link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/dojo/1.6.1/dojo/resources/dojo.css"/>
<link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/dojo/1.6.1/dijit/themes/tundra/tundra.css"/>
<link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/dojo/1.6.1/dojox/grid/resources/Grid.css"/>
<link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/dojo/1.6.1/dojox/grid/resources/tundraGrid.css"/>

<script type='text/javascript' src='https://ajax.googleapis.com/ajax/libs/dojo/1.6.1/dojo/dojo.xd.js' djConfig="parseOnLoad:true"></script>

<script type="text/javascript">

    dojo.require("dojox.data.JsonRestStore");
    dojo.require("dojox.grid.DataGrid");
    dojo.require("dijit.form.DateTextBox");
    dojo.require("dojox.grid.cells.dijit");
    dojo.require("dojo.date.locale");
    var item_structure = null;

    dojo.ready(function () {
        setItemTableStructure();
        //retrieve a single order
        //expects json to look like
        // {"Id":2,"Name":"Order Name 2","Date":"\/Date(1321135185260)\/","Items":{"Ref":"/Order/2/Items"}}
        order_store = new dojox.data.JsonRestStore({ target: "/Order/2", idAttribute: "Id" });

        order_store.fetch({
            onComplete: function (item) {
                //this method is called after an item is fetched into the store
                //item here is the order json object
                var orderId = new dijit.form.TextBox({ value: order_store.getValue(item, 'Id') }, 'orderId');
                var orderName = new dijit.form.TextBox({ value: order_store.getValue(item, 'Name') }, 'orderName');
                var orderDate = new dijit.form.DateTextBox({ value: parseJsonDate(order_store.getValue(item, 'Date')) }, 'orderDate');
                var items = order_store.getValue(item, 'Items').Ref;

                //make a call to retrieve the items that are contained within a particular order
                //expects a json object that looks like
                //[{"Ref":"/Item/1"},{"Ref":"/Item/2"},{"Ref":"/Item/3"},{"Ref":"/Item/4"},{"Ref":"/Item/5"}]
                var xhrArgs = {
                    url: items,
                    handleAs: "json",
                    load: loadOrder, //main method
                    error: function (error) {
                        console.log(error);
                    }
                }
                var deferred = dojo.xhrGet(xhrArgs);
            }
        });
    });

    //This is the main method
    function loadOrder(data) {
        var itemIds = "";
        dojo.forEach(data, function(item, i){
            itemIds += item.Ref.charAt(item.Ref.length-1);
            itemIds += ",";
        });
        itemIds = itemIds.substring(0, itemIds.length-1);

        //build the backend to accept a comma seperated list of item ids
        //like    /Item/1,2,3
        item_store = new dojox.data.JsonRestStore({target:"/Item/" + itemIds, idAttribute:"Id"});
        items = new dojox.grid.DataGrid({
            name: "items",
            formatter: function(date) {
                        if (date) return dojo.date.locale.format(parseJsonDate(date), {
                            selector: "Date"
                        })
                    },
            structure: item_structure,
            store: item_store
        }, dojo.byId('orderItems'));

        items.startup();
    }

    function setItemTableStructure() {
        item_structure = [
        { 
          field: 'Id'
        },
        {
          field: 'Name'
        },
        {
          field: 'Price'
        },
        {
            field: 'Date',
            type: dojox.grid.cells.DateTextBox,
            widgetProps: {
                selector: "Date"
            },
            formatter: function(v) {
                if (v) return dojo.date.locale.format(parseJsonDate(v), {
                    selector: 'Date'
                })
            }
        }];
    }

    function parseJsonDate(jsonDate) {
        var offset = new Date().getTimezoneOffset() * 60000;
        var parts = /\/Date\((-?\d+)([+-]\d{2})?(\d{2})?.*/.exec(jsonDate);

        if (parts[2] == undefined)
          parts[2] = 0;

        if (parts[3] == undefined)
          parts[3] = 0;

        return new Date(+parts[1] + offset + parts[2]*3600000 + parts[3]*60000);
    }
  </script>
  </head>
  <body>
    <h1>Json Rest Store with DataGrid Example</h1><br />

    <form style='margin: 10px' action='post'>
    <h2>Order</h2>
    <input id='orderId' />
    <input id='orderName' />
    <input id='orderDate' />
    </form>
    <h2>Items</h2>
    <div id='orderItems'></div>
    </body>
 </html>

Models:

using System;

namespace OrdersRestService.Models
{
    public class Item
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public double Price { get; set; }
        public DateTime Date { get; set; }
    }
}

public class Order
{
    public int Id { get; set; }
    public string Name { get; set; }
    public DateTime Date { get; set; }
    public Reference Items { get; set; }
}

public class Reference
{
    public string Ref { get; set; }
}

Controllers:

namespace OrdersRestService.Controllers
{
    public class OrderController : Controller
    {
        public ActionResult Index(string id)
        {
            Order order = new Order
                              {
                                  Id = Convert.ToInt32(id),
                                  Name = "Order Name " + id,
                                  Date = DateTime.Now,
                                  Items = new Reference
                                                {
                                                    Ref = "/Order/" + id + "/Items"
                                                }
                              };

            return Json(order, JsonRequestBehavior.AllowGet);
        }

        public ActionResult Items()
        {
            List<Reference> items = new List<Reference>();

            for (int i = 1; i <= 5; i++)
            {
                Reference r = new Reference();
                r.Ref = "/Item/" + i;
                items.Add(r);
            }

            return Json(items, JsonRequestBehavior.AllowGet);
        }
    }

    public class ItemController : Controller
    {
        public ActionResult Index(string id)
        {
            List<Item> items = new List<Item>();
            foreach (string itemid in id.Split(','))
            {
                Item item = new Item 
                                  {
                                      Id = Convert.ToInt32(itemid),
                                      Name = "Item Name " + itemid,
                                      Date = DateTime.Now,
                                  };
                items.Add(item);
            }

            return Json(items, JsonRequestBehavior.AllowGet);
        }
    }

    public class HomeController : Controller
    {
        public ActionResult Index()
        {
            return View();
        }
    }
}

Global.aspx.cs

public class MvcApplication : System.Web.HttpApplication
{
    public static void RegisterGlobalFilters(GlobalFilterCollection filters)
    {
        filters.Add(new HandleErrorAttribute());
    }

    public static void RegisterRoutes(RouteCollection routes)
    {
        routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

        routes.MapRoute(
            "DefaultWithAction", // Route name
            "{controller}/{id}/{action}", // URL with parameters
            new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
        );
    }

    protected void Application_Start()
    {
        AreaRegistration.RegisterAllAreas();

        RegisterGlobalFilters(GlobalFilters.Filters);
        RegisterRoutes(RouteTable.Routes);
    }
}

I have a PHP-based game that currently works in your typical request/response fashion, generating and outputting pages. I wish to completely separate the presentation and game logic, so I am porting to be API-based (using Phalcon). This will also allow me to make a mobile app that can interact with the same code base.

I'm new to the whole REST API thing so I am trying to follow the guidelines set forth in RESTful Web Services. It is my understanding that the API should work as follows:

POST to /v1/users adds a new user.
GET to /v1/users returns a list of existing users... and so on.

However, the client that is consuming my API (via AJAX most likely) should only issue higher-level commands. For example, I don't want a savvy user making an API call that modifies their experience. Instead, data modification would be the side effect of a game action that takes place based on player input.

For example, the client may say "hey, attack this monster", at which point the server will verify that the player has enough energy, carry out the attack, add the gold and experience to the player, and return the result to the client. These actions are very transactional, with multiple conditions needing to be true and multiple results needing to occur in order for the action to be considered successful.

I picture this functioning somewhere along the lines of:

/v1/attack/player - with the id of the player being a post variable.
/v1/work - with the amount of time being worked as a post variable.

However, this seems like it would fall under an RPC type of architecture, as defined in the book, which I understand as not the same.

  • How do I handle this scenario?
  • Am I correct in thinking a REST API is appropriate for this? It would seem the nature of me needing to call remote procedures is fundamentally against what I'm reading REST is.
  • Assuming I am approaching this with the wrong mindset - is there some other architecture better suited or should I simply continue with the way I mentioned seeing it work?

Any advice on whether I'm headed in the right direction and what I can do better would be very beneficial to me.

Thank you.

I would send a POST /attack and describe the parameters in the body. You could create links for every opponent. So this might work with simple games, but the server won't be able to push data to your browser, unless you use polling.

For realtime games an event based approach with websockets is much better...

I'm in the position where I may be creating a new web service from scratch - without much pre-existing infrastructure to have to contend with. What resources are there that talk about the architectural aspects of deploying a web service? [Clarification: I'm not talking about an Enterprise SOA orientation here - rather setting up one family of services for the public.]

A first list of topics that I'd like to see covered are:

  • SOAP vs. REST
  • JSON vs. XML
  • Relational Database Backed vs. SimpleDB backed vs. ?
  • Scaling
  • Availability
  • Models for restricting access
  • Models for throttling access

What would you recommend?

If you decide to use Microsoft technology (WCF) then you could check out the Microsoft Patterns and Practices group's online library of guidance.

They have a library located here as part of MSDN which deals with Web Service security, Enterprise Buses (obviously not applicable to you scenario) and PAG's own Web Service Software Factory.

Their main page is located here.

Otherwise, assuming you choose WCF it might be worth checking out further reading such as Juval Lowy's book on WCF, although I fear it may cover the implementation more than the theory and design facets.

Do you know roughly what technology platform you'll be working from?

I would recommend Restful Web Services. It's weel written, very complete and vendor agnostic. Also it has a fairly good coverage of both REST (with comparison to SOAP/WS-*), HTTP scaling, resource formats (JSON, XHTML, Atom, XML), security and service modeling.

If you have any specific scaling needs, then you might also want to read Building Scalable Web Sites. It will teach you everything worth knowing about etags, proxies, caching, edge computing and so forth. However if you are just starting out, then the Rest book I mentioned earlier will properly cover most people needs.

So, I'm learning Ruby on Rails. I get the fundamentals of the RESTful approach to application architecture, but I haven't yet built a RESTful application completely.

In a traditional web application, idempotent requests (which don't change any state) are made via HTTP's GET method and all non-idempotent requests are typically made via the POST method. For the application to distinguish between the different actions a POST request might be triggering, there's usually a hidden field in the form that gets included with the POST request, like action=delete or action=add_to_foo_file. This is a fairly simple architecture and is pretty common today, but the REST approach steers us away from this and suggests that this "let's POST everything!" design is considered harmful.

Instead, a RESTful architecture has us manipulating resources via the URIs which uniquely identify them (nouns) and the HTTP request methods (verbs), instead of determining what action to take based on a separate field in the request:

GET    => show the resource
PUT    => update the resource
POST   => create a new resource
DELETE => destroy the resource

So now we come to Ruby on Rails. In Rails, PUT and DELETE are implemented by JavaScript, which changes your links into forms with a hidden field called _method. The form is received from the browser via POST, and then Rails looks for this field and decides whether to route the request to the PUT or DELETE methods of the controller.

Wait, what? That sounds suspiciously like -- even exactly like -- the behavior of a traditional web application, which receives all state-changing requests via POST and inspects specific fields to decide whether to destroy or modify a record.

So how can Rails' approach be said to be RESTful at all, since under the hood, it is just a re-implementation of the "let's POST everything" approach which REST specifically attempts to move away from?

And further, doesn't this approach prevent my Rails app from ever degrading gracefully for people with JS turned off in the browser?

What makes a system RESTful or not isn't so much the verb it uses, but the fact the interactions are driven by the hypermedia (in this case, this includes code-on-demand JavaScript).

The GET/PUT/POST/DELETE mapping you're quoting works fine for CRUD operations, but isn't necessarily appropriate for everything (and isn't necessarily supported by all browsers). This is a reasonable rule of thumb to design a RESTful system, but it's neither sufficient nor necessary. (It's in the same category of ideas as what people insist on when they want "RESTful" URIs: there's no such thing.)

Part of this comes under influence of books like RESTful Web Services. It's a good book, but it came out at a time where WS-*/SOAP was predominant, and where everything was tunnelled through POST. A reaction to SOAP was to make people aware that there were other HTTP verbs they could use.

For REST, what really matters in particular is to respect the notions behind the URI concept, the semantics of each HTTP verb (no side-effects, idempotent requests, ..., where appropriate and if you're using HTTP) and the HATEOS principle. It's an architectural style, don't stress too much about what POST/PUT/DELETE the JavaScript does underneath.

You should certainly read these:

EDIT: (Following comments)

What I should have said was this: If most frameworks turn DELETE http://server.tld/resource/1 into POST http://server.tld/resource/1?_method=DELETE, then what's the advantage of that approach over just using POST http://server.tld/resource/1?my_method=delete in the first place?

  • It makes the overall goal a little bit cleaner/clearer, in terms of API design. REST frameworks can ultimately only be as RESTful as one uses them, since REST is an architectural style, not a protocol or an implementation.
  • In addition, strictly speaking, the URI includes the query component, so http://server.tld/resource/1 and http://server.tld/resource/1?my_method=DELETE could identify different resources in principle (although that wouldn't be a good design choice).
  • It's good for the framework to be able to expose DELETE/PUT as such directly, but have a fallback solution via POST for the clients that don't support it.

    The reason for this is that clients that do support DELETE/PUT will be able to make use of them, and make assumptions about their usage. It's not so much of a problem for the client to treat a request as non-idempotent even if it was idempotent in principle as the opposite. If the client thinks it can send at most 1 request (non-idempotent) when it could have sent N+1 of the requests, this doesn't cause major problems; if the client thinks it can send N+1 times the same request for something that shouldn't be idempotent, this can cause quite a lot of problems. Emulating PUT through POST is OK, you're just not making the most of PUT, emulating POST through PUT would be problematic.

We are evaluating the technology to be used for a web based application and some suggestions are to go with RESTful based services approach. Tech Stack

1) Spring 2) Apache CXF ( JAX-RS)

My questions are

1) How state is managed between requests. For example, a user has been authenticated and now he is making a series of requests lets say going through a paginated report. I would imagine the URL for this will be like

domain.com/reports/customreport/page/1 domain.com/reports/customreport/page/2 etc...

a) Where is the user information & request parameters are stored so that it can be shared between requests. b) Lets say the result is being streamed, where is Rowset is stored?

Is there a complete sample application something similar to Petclinic that can provide the best practices for such an application.

1) The user information is not stored anywhere, the user has to send his credentials (or whatever authentication method you're using) on every single request.

2) Streaming doesn't make much sense in a RESTful API, if you would like to do streaming I'd greatly advice you to look for something like WebSockets (in Java you can easily do this with Jetty)

If you said streaming but you meant paginated results, same as 1, there is no state kept, the client has to send a new request with all the information and the server has to query the database (or go to a cache, or do anything needed) and return the result to the customer.

You should also read more about REST, as your question is quite vague, one good start is the Restful Web Services book or, if you feel adventurous, you can try Roy Fielding dissertation that defined what we call REST today.

We're planning a new API server that will mainly serve JSON responses, with a RESTful interface. We're concerned about scale and availability. Are we on the right track using Restlet with Jetty (or another connector)?

One question we're asking is, is there anything in the Java world like Phusion Passenger? For example, a pre-built solution for keeping server instances up and healthy?

Your question actually is not as much about Restlet as it is about designing a high-scalability, high-availability site. We find that Restlet does scale very well with the right system architecture.

Generally speaking you want to:

  • Run a cluster of web server machines, not just one.
  • Make sure your application is shared nothing, ie, no application state stored in your web servers, if at all possible.
  • Use a load balancer to spread requests to the least loaded web servers.
  • Make sure your JSON responses are cacheable.
  • Add an HTTP reverse proxy cache (eg, Squid) at the border of your site. As the caches between your site and your clients warm up, most of the inbound traffic will be handled by them, and not your web servers.
  • Write your client code to retry requests that fail. This way if a web server dies the next request will be load balanced to a surviving machine.
  • And of course you want to automate your site to bring up crashed web servers, etc. (This is the part that is perhaps better asked on ServerFault.com.)

REST is an architectural style that is ideal for this type of setup.

As @matt mentions you do need to watch out for raw performance, but generally your first concern should be to get the scalable, high availability architecture in place.

Some good sources on this are:

and especially:

Overstock.com runs a highly scaled web site and makes heavy use of Restlet to do it.

A REST resource should be identified by two names. What is the best practice for this?

My ideas:

 .../{id1}-{id2}
 .../{id1}/{id2}

An id can consists of numbers, letters and special characters.

First solution:

A problem occurs if one of the ids contains the character -. In this case the separation character is not unique.

Second solution:

At just .../{id1} will be no resource, is this RESTful?

Edit:

The REST resource represents credentials. Credentials are identified by a provider name and the username.

 .../Credentials/<ProviderName>;<UserName>

I don't want to show at .../Credentials/<ProviderName> all credentials of the provider (it wouldn't fits to my XML structure).

Edit 2:

At .../Credentials all credentials will be showed in XML. A credentials is there represented as a sub element of the XML root element. I want to a create a REST resource structure that is equal to the XML structure. Thus, a sub resource of .../Credentials should be directly a certain credentials (and not a set of credentials like all of a provider).

Your second solution .../{id1}/{id2} says for me that you have a hierarchy relationship between id1 and id2, which seems to not fix very well your resource design since both identifiers reference the same resource, as you say. In the other hand, for you first solution .../{id1}-{id2} you can use ; instead - to avoid implying hierarchy where none exist as cited in Restful Web Services so i will go with this:

.../{id1};{id2}/

Do you know how a resource edit path should looks like on a restful web app?

Can't find any serious reference but the Ruby on Rails way that it's just a convention.

I'm not talking about the put request that is used to update the resource but the path that usually returns a form, or similar, to the user to let him create and submit the final put request that updates the resource.

An example of the previously mentioned Ruby on Rails way would be a get request to:

http://domain.com/resource_name/resource_id/edit

Just wondering if there's some serious reference or explanation that makes this a good approach or not.

Your question is a little vague, I think; but I'll take a stab at it.

Say you have a resource at http://example.com/planets/earth. If you want to edit something about earth, do a PUT to that URI, with the new representation you would like it to have. The same could be said for, say, a user: http://example.com/users/JamesKirk.

The important thing about REST is that the 'edit' path is not seen as a path at all, it is HTTP's methods and how they operate on your resources.

Here's a good book on the subject of restful web services: RESTful Web Services

I'm about to write an application for which I will not develop a user interface. This application works with a database and manipulates the data in the DB.

My question is:

  • What would be the best programming language independent api approach so people can write a web interface, cli interface or desktop app interface towards my application?

I write my app in Python and most data is stored in json format.

Thanks,

Jay

I'm a little uncertain on what exactly you're asking for, but I imagine you're looking for the best API design pattern.

Key points I would recommend you look at:

  • Implement a RESTful interface
  • Offer a variety of data formats (JSON, XML, etc)
  • Make the syntax intuitive and easy to understand
  • Thorough documentation
  • Use proper response codes

Also, here are some links you may find useful:

http://www.slideshare.net/eaton/building-apis-that-rock
http://www.notiondesign.ca/blog/story/how-not-to-build-an-api/
http://blog.apigee.com/category/api_best_practices/
http://blog.isnotworking.com/2007/05/api-design-guidelines.html

Also, this is a great book that may help you get started:

http://www.amazon.com/dp/0596529260/

I'm asked to set up a new webservice which should be easily usable in whatever language (php, .NET, Java, etc.) possible. Of course rolling my own can be done, accepting different content-types (xml / x-www-form-urlencoded (normal post) / json / etc.), but an existing method or mechanism would of course be prefered, cutting down time spent on development for the consumers of the service.

The webservice does accept modifications / sets (it is not only simply data retrieval), but those will most likely be quite a lot less then gets (we estimate about 2.5% sets, 97.5 gets). The term webservice here indicates the protocol should go over HTTP, not being able to implement it totally client sided (javascript in the end-users browser etc.), as it needs specific user authentication.

Both gets and sets are pretty light on the parameter count (usually 1 to 4). Methods like REST (which I'd prefer for only gets), XML-RPC & SOAP (might be a bit overkill, but has the advantage of explicitly defined methods and returns) are the usual suspects.

What in your opinion / experience is the most widely 'spoken' and most easily implementable protocol in different languages (seen from the consumers' viewpoint) which could fullfill this need?

We did a lot of research on this and ended up going with REST. A lot of well-thought-of web services use REST including Amazon S3. I think Gmail also uses REST on the back end.

From my observation, RESTful web service design seems to be becoming more prominent as the advantages of its design and disadvantages of other designs such as SOAP become more obvious through time and experience.

The reason that REST seems to work well is that it is a natural fit for the client-server nature of the web with its separation of safe and idempotent GET requests, idempotent PUT requests, and POST for requests that are neither safe nor idempotent. Its been around for a long time so a properly laid-out RESTful web service can be consumed by a variety of web clients, some that you might not even anticipate. An example of this is that a web crawler will know only to request resources using GET because it is guaranteed not to affect the resource whereas PUT and POST requests have different rules.

After we decided on a RESTful service architecture, I read this book: RESTful Web Services in order to learn the basics. If you decide to read it, know that the book is perhaps twice as long as it needs to be so you'll have to be judicious about what to read and what to skip - cause you'll want to skip the fluff.

i'm new with web service programming and i want to create,using netbeans 6, a restful web service using Jersey over a Grizzly server and then a client javascript in order to use this web service through a browser. So i started learning more on restful web service and i read a lot of guide over the web, then i started learning more on grizzly and jersey by reading jersey user's guide http://jersey.java.net/nonav/documentation/latest/index.html. I succesfully follow the tutorial to create the helloword example resource. So i created all the resources needed for the job and tested successfully with the browser...but i'm still confused: in particular i want know how i can create a static homepage which can be used by the users to select what is the desired resource. Can you give me some tutorial or example?? Thanks to all!

(Moreover i want to learn more on grizzly server and creating jersey restful web service, can someone give me a useful guide or book??)

So, the key to understanding RESTful web services is to understand the HTTP protocol more thoroughly. That's what makes it easier than (and often preferable to) RPC style services epitomized by SOAP. When you pull down a static web page, for example, you can think of it as a limited "web service" which serves only GET requests. In order to make a static web page which "selects resources," you would only need to provide URLs to the resources in question, as long as they're accessed via GET, because that's the same HTTP method used for retrieving web pages (and therefore is the default method for web browsers). If you want to access other types of resources, such as sending POST requests, you can use a form; other than that (with PUT, DELETE, HEAD, OPTIONS, etc.) you'll want to use Javascript or a more programmatic API for accessing the HTTP resources.

There are many good books in this space, and I've found these particularly useful:

The first two approach REST in theory and practice; they are more about the concepts than specific technology. The third addresses the Java standard for RESTful services as defined in JSR 311, of which Jersey is the reference implementation. The last is more of an "enterprisey" book, but it's been useful to me from the approach of designing a system of web services, as opposed to one-off service resources.

I'm not exactly sure what question to ask here since I don't know the vocabulary...

Tomcat servlets (and any server for that matter) work nicely if they are stateless and respond quickly to requests, with state stored in a database. It seems like if I have long-running operations then maybe I want to run some other service in the background, and have the Tomcat handlers communicate with it. Is there a way to run a long-running Java application in the same JVM as Tomcat and interact with it via "regular" Tomcat servlet?


Example: Let's say I want to offer a RESTful number factorization service in HTTP.

Here's a possible scenario (I hope I have the HTTP syntax right, I'm omitting most of the headers):

# comments start with #, > = request, < = response
# 
# first we create a queue
> POST /factorizer/create-queue
> {information here}
< queue=12345B
# then we post some numbers to it
> POST /factorizer/queue/12345B
> 123
> 456
> 678
> 12345678901234567890123456789
< OK
# let's look at the status
> GET /factorizer/queue/12345B/status
< requested=4
< processed=3
# query
> GET /factorizer/queue/12345B/7
< Error: invalid index
> GET /factorizer/queue/12345B/3
< Error: not complete
> GET /factorizer/queue/12345B/0
< 123=3*41
# wait a while
> GET /factorizer/queue/12345B/status
< requested=4
< processed=4
> GET /factorizer/queue/12345B/3
< 12345678901234567890123456789=3*3*3*7*13*31*37*211*241*2161*3607*3803*2906161

I can think of how to write the servlet to handle the queries, but how could I go about implementing a daemon / independently-running service in the same JVM?

edit: In the above example, what I would like to do is to have a background application that runs autonomously, with work queues, to factor prime numbers, and has a Java interface that supports the operations that the Tomcat servlets could use to expose the service to the web. Then I don't have to worry about the web interface or HTTP in my background app, and I don't have to worry about multithreading issues or prime factorization in my servlets.

If you don't absolutely need to be in the same JVM (that is, if you don't need the performance of directly accessing the objects) you could write another Tomcat application and have your other apps communicate with it by HTTP to localhost. In effect you would be writing a web service that happens to run on the same machine. (I don't know how else multiple Tomcat applications can see each other. This is a problem that Enterprise Java Beans solves, but that may be too heavyweight a solution for you.)

If you have only a single Tomcat application that needs to do this, create a worker thread and put it in the application context where all the requests can communicate with it.

With regard to your specific problem, it looks like you are describing something like the Asynchronous Job pattern in the O'Reilly "Restful Web Services" book. This uses the "202 Accepted" status code to indicate that the processing is not complete. See "Asynchronous Operations" in Chapter 8 of the book.

http://www.amazon.com/RESTful-Web-Services-Leonard-Richardson/dp/0596529260/ref=sr_1_1?ie=UTF8&s=books&qid=1255555328&sr=8-1

The title basically says it all, is REST really tied to HTTP or is it protocol independent? Because I've just read few articles about REST and I encountered both opinions so I don't know which one is correct.

Allow me to quote the book "RESTful Web Services" (bold is mine):

The Story of the REST

REST is simple, but it’s well defined and not an excuse for implementing web services as half-assed web sites because “they’re the same.” Unfortunately, until now the main REST reference was chapter five of Roy Fielding’s 2000 Ph.D. dissertation, which is a good read for a Ph.D. dissertation, but leaves most of the real-world questions unanswered. That’s because it presents REST not as an architecture but as a way of judging architectures. The term “RESTful” is like the term “object-oriented.” A language, a framework, or an application may be designed in an object-oriented way, but that doesn’t make its architecture the object-oriented architecture. Even in object-oriented languages like C++ and Ruby, it’s possible to write programs that are not truly object-oriented. HTTP in the abstract does very well on the criteria of REST. (It ought to, since Fielding co-wrote the HTTP standard and wrote his dissertation to describe the architecture of the Web.) But real web sites, web applications, and web services often betray the principles of REST. (...)

Notice in the second bold, he's saying: "HTTP fits REST well", not "REST fits/depends on HTTP".

So, in short: no, the term "REST" is not necessarily tied to HTTP. RESTful web services are just web services that follow a RESTful architecture, hoping to achieve the benefits listed by Fielding in his thesis (such as statelessnes, addressability, etc.).

One way to create RESTful web services is to think of you application in terms of resources (not actions, as the SOAP style uses). Such way of thinking in conjunction with proper usage of HTTP (its methods and status codes) can lead to a REST-enabled architecture as (and with all the benefits) Fielding's thesis enumerates.

Essentially,

I would know to the various components involved to get from start to finish from absolute scratch.

I am about to undertake an Android project where I pretty much need to completely create a back end and a way to interact via HTTP. A RESTful approach seems the way to go but now I am pretty stuck on where to start - things I should take into consideration.

At the moment I see the following being important:

Are there other alternatives I should be considering? Is this the best approach considering I have done no PHP and very little MySQL so far? Does the fact it is mainly Android based mean there are unique options to consider?

If there are any other good resources I should be considering, I would be very grateful!

Cheers

I'm in the same situation. I'm going to develop an Android app with a REST backend (PHP + MySQL). A few things I'm taking into account:

And I would like use Python instead of PHP, but there are less sharing hosting services that support GCI for Python.

I am trying to understand token-based authentication these days, which claims to be a stateless authentication method. And I met the concept of stateless web application.

Below are some threads I read about:

At first, I was thrilled at this idea. But more and more I think stateless is a pseudo-proposition.

For example, suppose we use client-stored token for authentication, how can we make a statistic of online users (suppose there's no log)? Shall we store the token in DB? Doesn't that mean we store state info on server? And even more, is the plain user info such as name, age, etc. in DB also some kind of state info?

I think the real question here is not to make a web app stateless, but to make the web app properly handle the state info such that it won't jeopardize scalability.

That depends on how to interpret the word stateless:

  1. Web app doesn't have state.
  2. Or web app doesn't store state itself.

I prefer 2 because there can always be some inevitable global state (quoted from @deceze's comment to his answer). And no matter we store state info as HTML 5 web storage, or HTTP header, or hidden form fields, or Cookie, the state still exists. Only that it is stored somewhere other than on the server.

Am I missing something great? Could anybody shed some light on this so I can be relieved from this mental struggle?

ADD 1

Just read about the book RESTful Web Services by Leonard Richardson. In chapter 4, at end of the section Statelessness, it classifies the state into Application State and Resource State. So the plain user info and data I mentioned before like images, etc. can be classified as Resource State. And what stateless refers to is Application State. So it doesn't break the code of stateless to store resource state on server.

But the book also mentions the scenario where an application key is used to restrict how many times a user can invoke a web service. It admits that such info cannot be stored on client side. And having to store it on server side breaks the code of stateless and introduce the issue of session affinity. It claims stateless can avoid session affinity issue but doesn't explain how. I really don't see how stateless can handle this scenario. Anyone could shed some light here?

The "state" only really refers to the state between the client and the server. Of course the server will store data, and technically you can see any modification of any data on the server as "altering state". Hence a "stateless" application in this sense makes absolutely no practical sense.

What "stateless" refers to is whether the server is, at any particular time, in a state to allow a particular client to talk to it.

Consider: with a traditional cookie-based login session, the server is only in a state to accept requests from the client for a limited time window; for as long as the current session is valid. The client cannot predict for how long that is. At any time, a request from the client may fail, because some state on the server timed out. In this case, the client needs to reset the server's state by logging in again.

Contrast this with token based authentication. The token must be valid indefinitely. It is essentially a substitution for a username and password. For the sake of discussion, just assume the client sends their username and password with every request. This means every request can be authenticated on its own merits, not requiring the server to be in some particular temporal "state".

The reason why you use tokens instead of usernames and passwords is twofold:

  1. you can authorise multiple clients using the same account, but each with their individually managed credentials
  2. you don't want to be sending the "master password" back and forth with every request

Of course the server will need to keep track of the created tokens and authenticate against some database with each request. That's an irrelevant implementation detail. This does not differ from using session cookies; however, since tokens are valid indefinitely, requests can potentially be cached easier instead of needing to replicate a temporary session store.

One last potential argument that needs preemptive countering: what's the difference between an indefinite session and an indefinite token, and what's the difference when the session ends vs. when the token may be revoked?
When a session ends, it can be reestablished using some other "master credentials" (logging back in). A token can/should only end when actively revoked, which is akin to revoking the authorisation to access the service entirely for the master credentials, and is not something that is part of the regular application flow.


Stateless client/server communication simplifies the client side first of all, since the client can assume at all times to be able to request anything of the server, without needing to know the state of the server ("is my session still active or not?"). It can help scale out the server implementation since only static information about valid tokens needs to be replicated between all servers, instead of a constantly changing pool of valid sessions and their associated data.


Architecturally, your goal should be to have as many stateless components as possible. This will simplify scaling out. For example, if your web server is keeping a local session store, that makes it very hard to scale out your web server to multiple instances behind a load balancer/CDN. One improvement is to centralise the session store to an independent database; now you can have several stateless web servers which know how to get data (including session data) from somewhere and can render templates, but are otherwise completely interchangeable.

However, a session store must be kept in perfect sync across everyone trying to access it, which makes it hard to scale it. Tokens improve this by making the data change less often (only when tokens are added or removed), which means you can use a distributed database or other simpler replication mechanism if you want to have several token stores in possibly multiple locations, and/or makes that data cacheable.

I have successfully made contact with the internet and parsed a few documents via android and now I think my next big questions is how do I create the client that will convert my database into JSON files I can parse from my android app? Ive done some digging and I know it probably involves PHP, but I havent been able to find much specific info.

What I am looking for in a answer are any of the following:

  1. How difficult/complex is creating server end clients like this useually?
  2. Keywords I can research for example something like " you'll need a php server side client do hicky , look into [insert php topics I need to study here]
  3. LINKS LINKS LINKS!!!

You are on the right track. You essentially need a server side application stack that will connect to your mysql database, read out recordsets, and send back HTTP responses that include the JSON payload.

You can do this by simply writing a PHP method that returns the JSON formatted response (this would be fairly easy to get up and running). However if you are looking for a more robust/longterm solution I would invest the time in developing a RESTful web service. RESTful services provide a common interface to access data intensive applications.

As for how to do this, you can check out Phil Sturgeon's blogpost here that describes how to setup a RESTful service in the popular PHP framework, CodeIgniter.

Also if you are not attached to PHP, I would recommend Ruby on Rails as it is designed out of the box to create RESTful web services. DHH, one of the creators of the Ruby on Rails framework, helped write the book on RESTful web services.

I'm looking to implement a RESTful API and I was wondering what parameter is advised when the browser client can't do a PUT/DELETE request to the server. (since some browser apparently doesn't support these requests)

Does it exist a specific parameter name for that? (something like request_method=DELETE on a POST request, that should indicate to the server that the client really want a DELETE but can't indicate it)? or there is no specifications like that?

My searches lead me to the parameters "_method" used by prototype.js. But is this parameters a standard? For example, how jQuery handle it?

Thanks for your help.

What you are essentially asking about is called method overloading - although it's not 100% RESTful, it's an approach which you can fallback to in case that the client doesn't support PUT/DELETE methods. Richardson's/Ruby's book RESTful Web Services mentions this method as well.

You coined the basic principle yourself in your question - the difference when compared to the fully RESTful approach is that you introduce a new parameter which is submitted as part of the request body via POST. This parameter indicates the intended method (e.g. POST/PUT/DELETE) which is being submitted via an (overloaded) POST request. This allows you to overcome the HTTP method limitation of the client while still keeping your URI design clean and RESTful (e.g. without verbs). However you should still aim to be as much RESTful as possible, i.e. keep using GET for read-only requests (thus not impacting caching, etc) and use overloaded post for any write requests. As to how to name the parameter which holds the true HTTP method - I'm not aware of any standardized naming policies myself, so going with what's used by Prototype.js (or with any name that fits well into your naming policy) should be fine.

Regarding jQuery and XmlHttpRequest in general, these should support PUT and DELETE methods. See:

Note that the jQuery documentation warns about support of PUT/DELETE being browser-dependent. The way I understand it is that these methods should be supported when using XHR ... however you should rather test if all your target browsers support these methods before making a decision between POST overloading vs. the full HTTP method stack.

jQuery by default provides access to .get() and .post() methods which are both built on top of .ajax(). For doing PUT/DELETE calls via jQuery you need to use .ajax() method directly and specify the method in type in settings. This however does a true PUT/DELETE request, as jQuery itself does not provide any automatic means how to handle method overloading (so you need to code it yourself so that it fits into your API design).

If you won't be using XHR and need to rely on standard forms, then I'm afraid that you will have to use method overloading because support for PUT/DELETE methods as valid form action values has been dropped from the HTML5 spec (this was also noted in the SO question which you are referring to).

So I just started digging into web tech, and I'm stuck on the concept of REST. Could someone clarify REST by giving me an example of what isn't rest?

So, as far as I can tell, REST requires the server and client to both be in the same state at the end of every request-response HTTP transfer. Does that sound right?

My understanding is that, if a client stores state information locally (which the server does not know about), that service is NOT rest.

Thanks in advance.

Restful Web Services is a good book to read to get what REST is about.

I am looking for best practices on restful web service standards. One of the main things I want to sort out is entity separation in url. What is the right to do, group methods in one url or separate by entityname?

[WebGet(UriTemplate = "countries/{id}")]
Country GetCountry(int id);

[WebInvoke(UriTemplate = "countries/{id}", Method = "POST")]
Country CreateCountry(int id, Country country);
or 
[WebGet(UriTemplate = "resources/lists/countries/{id}")]
Country GetCountry(int id);

[WebInvoke(UriTemplate = "resources/lists/countries/{id}", Method = "POST")]
Country CreateCountry(int id, Country country);

The first option is cleaner. All URIs in a REST API should be resources so 'resources' isn't needed in the URI and I would say 'lists' isn't either since that's just a way of representing a collection of a resource.

Also, if you are going to be implementing REST services in C#, as per the example, I would highly recommend using ASP.NET Web API. It provides a very clean, powerful, and simple way of implementing REST services.

Restful Web Services is a great book that gives basic best practices around restful web service design and has great examples as well. You could also check out the Richardson Maturity Model which describes service restfulness in terms of levels of maturity. For a service to be restful it would have to be at the top maturity level, but the levels give a good guide for taking steps in the right direction.

I have client/server applications and a very simple protocol for communication. More precisely, it's a set of commands a server sends and a set of requests a client can make.

The idea is as follows:

When a command is made by the server, a client has to execute it. When a request is made, server checks permissions and if everything is ok it grants the request.

The application is written in C++ and I got slightly stuck designing the architecture for this sort of communication protocol. Assuming that command/request is a tab-delimited string with first parameter being the name of the message, I was planning to make a MessageManager class which would store all messages in a hash-table and retrieve them when necessary. The problem here is this:

typedef std::vector< std::string > ArgArray;
class Request : public Message
{
public:
   Request( const char *name ) : Message( name ) { }
#ifdef CLIENT
   /** problem here **/
   virtual void make( ... ) = 0;
#elif defined SERVER
   virtual void grant( const Client &c, const ArgArray &params ) const = 0;
protected:
   virtual void checkPermissions( const Client &c, const ArgArray &params ) const = 0;
#endif
};

Because different messages can take different arguments to be constructed I can't really create a complete interface. For example, some messages might need a simple string to be constructed, whereas others might need some numeric data. This complicates things and makes the design a bit untidy... I.e. I have to get round the problem by ommitting make() from interface definition and simply add different make() for each Request I make. Also, if I wish to store pointers to different Requests in one container I cannot use dynamic_cast because Request is not a polymorphic type. An obvious (untidy) solution would use make( int n, ... ) definition and use stdarg.h to extract different arguments but I consider this to be unsafe and confusing for the programmer.

There is obviously a design flaw in my idea. I already have a solution in mind but I am just wondering, how would people at SO tackle this problem? What sort of Object architecture would you use? Is there a simpler approach that could solve this problem? There aren't any specific requirements to this except to keep it as simple as possible and to keep the actual protocol as it is (tab-delimited strings with first parameter indicating which message it is).

In my case the separate system is a web-service (but it could conceivably be anything). My question is what are the best practices when you integrate against a separate system such as a web-service when it comes to data?

Example: Web-service provides a list of products. Products are grouped using categories. You can get all products in a sub-category. You can get a specific product by its id (an integer) or its name (a unique value).

In my application:

I display the list of categories and products - and the user can choose the product and specify an order quantity.

  1. Should I store the name of the category or the id of the category?

  2. Should I store the name of the product or the id of the product?

  3. How should I name the field in the database that stores the data from the web-service (CategoryId or WsCategoryId: so that by convention one knows where the value is coming from?)

Any other best practices?

Any other references?

From your question I understand that the web service's interface looks something like this:

/product/
/product/{ProductId}
/product/{ProductName}
/product/category/{CategoryId}

Since you are asking if you should store CategoryName, I assume that it is unique (same as ProductName).
I also assume that the web service handles cases where products or categories are renamed transparently (i.e. by providing a redirect or any other means which allow you to detect this and handle it accordingly). If it doesn't, do not consider storing names as references to products or categories - always use IDs.

I would provide the same answer to your questions #1 and #2. Even though uniqueness of ProductName and CategoryName will technically allow you to store them in your application as unique identifiers of products and categories, I would opt for storing their IDs instead. The main decision point would be your storage medium. Since you are using a database, and the web service allows you to access objects by unique numerical IDs, database normalization rules should apply - hence you should store IDs.

The above however assumes that you are using a relational database - if you are using a NoSQL database, I assume that storing names instead of IDs would be a viable option as well (at least as far as I can tell with my current understanding of NoSQL solutions, unfortunately I don't have any practical experience with any of them yet).

Regarding question #3 - I would stick with the naming conventions that you already use in your database. There are many different conventions for naming tables and columns out there, so I really doubt that there are any standardized conventions on how to name columns referencing web service objects. I would name them according to your existing naming conventions and in a way that purpose of the columns is clear to everybody who is using the system. Note that if there is a chance that you will be using other web services in the future, you should consider keeping the name of the service in the column name rather than using a generic ws prefix - e.g. AmazonProductId or AmazonCategoryId.

I'll try to point out a few items from my experience, but I would not label them as best practices - just topics to think about.

In my experience, I found it useful to treat data from web services in the same fashion as the data from a database - at least from an application's perspective, where your storage layer would be abstracted from application logic. By this I mean that you would should think about and prepare for similar scenarios regardless if your storage medium is a database or a web service. Same as databases, web services can go down, both can have their data or integrity corrupt, both will require you to sanitize or otherwise process data on input.

Caching of data should be an item which is high on your list - apart from the obvious performance reasons, it can allow you to deal with outages of the web service (to an extend limited by which data you cache).
An example would be that your application displays a list products most frequently purchased products in your application. If your application stores only IDs of products, you will have to do one or more requests to the web service in order to retrieve the names of all products which you need to display in the list. If you cache product names locally or in your database, you will achieve better performance, conserve your resources and you will also have a failsafe scenario in case that the web service goes down.

Referential integrity is one other important aspect to think about when working with web services. As the web service is completely separate from your database, you do not have the option to create foreign keys as you would do in a database-only solution. This means that data changes in the web service (i.e. product updates or deletions) can break the integrity of data in your database.

Regarding references, these depend mostly on the type of web service that you are about to use (you didn't specify which service you will be using). If the service is based on REST principles, I can recommend Restful Web Services by Leonard Richardson and Sam Ruby. Even though it isn't focused on application/service integration as such, it's a great introduction into REST.

Is there a way to host a list using a web service in scheme/racket

If in your question you meant a Scheme/Racket list: Sure, you could make a RESTful API such that POST, GET, PUT, DELETE verbs modified a list. I suppose that could be a lite way to try various RESTful ideas. If you want to do this for real I recommend "RESTful Web Services".

See "Systems Programming in Racket" for how to make a simple HTTP server in Racket from scratch. In step 4, the tutorial is ignoring the request header and assuming the verb is GET. However you could pay attention to the verb in the request header.

You may be able to build up a lite HTTP server that meets your needs, this way. Or, after trying this, you may decide the full server provided in Racket is more suitable for your needs, and better appreciate what it's doing for you. That is described in "Web Applications in Racket"

Ok i am not sure if this question has been asked before on SO.. I am confused with SOAP and REST.. I know that SOAP is formatted in XML and is send over HTTP whereas REST can be send over XML, JSON etc.. Representational state transfer (REST) and Simple Object Access Protocol (SOAP) makes a pretty good point. But does that mean that SOAP cannot be send using JSON????.. I am asking the above question in reference to android. I know this is a pretty stupid question, but i am really confused on this one. Any help is appreciated!!...

Thanks.

Actually SOAP or Simple Object Access Protocol is an envelop format for exchanging WebService request and response information. REST is a special kind of WebService that must be designed with a set of constraints. For example in a RESTful WebService method information must be placed in the HTTP method and scoping information must be placed in the URI. For understanding the architecture of web services and for a good introduction to RESTful web services i highly recommend you to read the following book: http://www.amazon.com/Restful-Web-Services-Leonard-Richardson/dp/0596529260

So I am creating a project with a couple of buddies and I need to create an API. This is my first time creating a big project and creating my own API. I don't know where to start. The project that we're working on will have a website, an app for Android and IOS, and use PostgreSQL. We will be using Javascript and Node.js as well. If there are any tips about creating API's for Android, IOS, and Web please let me know.

I would strongly recommend you read "RESTful Web Services" by Richardson

The book gives a thorough grounding in the "why?" behind organizing resources restfully. Also provides a good outline of different RESTful authentication mechanisms and the pros and cons of each - vital for a mobile app talking to a web-based back-end.

Then, sit back and outline the resources that will be necessary to run your app and have at it!