Introduction to REST Endpoints - Mach-II/Mach-II-Framework GitHub Wiki

1. Background
2. Resources
3. Enter Mach-II REST Endpoints
4. REST Endpoint Defined
5. REST Endpoint CFC Declaration
6. Example REST Endpoint Method Declaration
   • HTTP Methods
7. The Format attribute: auto file extension/content-type detection
8. Example URLs
9. Exception Handling
10. Mach-II Config
    • Optional Parameters
11. What goes into a REST Endpoint Method?
    • The REST Annotations
    • Accessing Custom Metadata Via Annotations
12. Using Endpoint Features
    • Performing Content-Length Checks
    • Building REST Endpoint Specific URLs
    • Using JSONP Support
    • Request Body Parsing Support for Non-Standard HTTP Method Types
    • Accessing Data and Variables Set by the REST Endpoint
13. Providing Authentication (Security) for REST
    • History
    • No authentication
    • Basic HTTP Authentication
    • One Time Use Tokens
14. Frequently Asked Questions

Implementing web services with REST has been called Web-Oriented Architecture (WOA), Resource-Oriented Architecture (ROA), and has become recognized as a leading method to provide shared services to be consumed by a diverse web clients. REST services are much simpler than alternative SOA technologies, but when well designed, they scale to handle the load of any well-designed web application.

There are a large number of outstanding resources that discuss RESTful web services, standards, and show examples. There are also a couple of great books on the subject.

ColdFusion developers can ramp-up quickly to develop scalable REST services by applying their current CFML knowledge and experience and applying REST standards and methods. Then, they can deploy and scale their REST service solutions in the same way they currently manage their other web-apps.

Frameworks in non-CFML technologies have embraced ROA through frameworks like Ruby on Rails and Spring MVC. Other CFML frameworks have also implemented REST to some degree, and CFML engines provides some built-in functionality, but the plumbing provided by modern frameworks is still lacking there.

• A more complete list of RESTful Development/Service resources is here: REST Online Resources.
• Building a RESTFul Service Using Mach-II Endpoints - Doug Smith (approximately 1 hour)

Mach-II 1.9's new REST Endpoints make it simple to expose RESTful services that leverage Mach-II's mature application structure. REST Endpoints have full access to Mach-II's ColdSpring integration, which provides hooks into other features like Caching and Logging. We've worked hard to make it as easy as possible for developers to quickly create intuitive, high-performance REST APIs.

A REST endpoint provides way to map a URI and HTTP method simply and directly into the Mach-II framework. The endpoint does not participate in the typical Mach-II event lifecycle, so it even lighter and faster than normal.

A REST endpoint is a CFC that extends MachII.endpoints.rest.BaseEndpoint. Anyone that has created a Mach-II Listener, Filter, or Plugin will be familiar with this concept, but others -- just hang in there. Simply create a CFC with a declaration like this:

    <cfcomponent
        displayname="ContentServiceEndpoint"
        extends="MachII.endpoints.rest.BaseEndpoint"
        output="false"
        hint="RESTful endpoint for the Content Service.">

Next, create a <cffunction> for every URI/HTTP Method (GET, POST, PUT, DELETE) combination you wish to expose from your application. Here's an example method declaration:

    <cffunction name="getContentItem" access="public" returntype="String" output="false"
        rest:uri="/content/item/{key}"
        rest:method="GET"
        hint="Retrieves an individual content item.">
        <cfargument name="event" type="MachII.framework.Event" required="true">

There are many things to note here. First, notice the two attributes prefixed with the rest: namespace. These custom annotations are the only configuration required for Mach-II to connect that URI and HTTP method to the <cffunction>. The two annotations are required. The rest:uri annotation is an absolute URI, so it starts with a slash, and includes the full path up to, but not including any query string parameters. Tokens can be declared using curly-braces, like "{key}". These tokens allow you to provide any string in that position of the URI, and Mach-II will automatically create an eventArg where the name of the token is the string between the curly braces, and the value is whatever text was entered in that position of the URI. For example, given a rest:uri of:

    /content/item/{key}

Assuming a rest:method of GET, the above function would match and be called by a GET request to the following URIs:

    /content/item/1
    /content/item/my-content
    /content/item/anything_else?option1=something&whatever=else

In each case, the {key} token would automatically generate an event-arg named key, with the value of "1", "my-content", and "anything_else" respectively. All other query string params also become event-args.

NOTE: Keys may include nearly any valid character that would otherwise be a part of the URI, with the exception of ampersand, plus, or semicolon (&, +, ;).

As mentioned above, the other required annotation, rest:method, must be a valid HTTP method: GET, POST, PUT, or DELETE or less used OPTIONS, HEAD, TRACE, or PATCH.

Lots of time REST works on resources in a CRUD fashion, however do not get lulled into a false sense that REST is only about CRUD. REST can be used to perform actions that indirectly operate on resources.

Common Mapping of Verbs to Resources

The REST endpoint will also automatically capture a dot-delimited file extension and return a response with the corresponding Content-Type header. For example, if you use a URI of:

    /content/item/1.json

The response header Content-Type will be set to application/json. In addition, Mach-II sets the eventArg format with the value of the input file extension. Alternatively, you can provide the format as a query string parameter. Any query string parameters are also converted to Mach-II eventArgs, as usual.

A full example URL would look like:

    http://example.com/index.cfm/content/item/my-data.html

If you are using your web server's rewriting capability to remove the index.cfm, you can use the more friendly:

    http://example.com/content/item/my-data.html

It is also possible to pass the endpoint name (defined in the Mach-II config file, shown below) and URI as query string parameters, like so:

    http://example.com/index.cfm?endpoint=content&uri=/content/item/my-data&format=html

TODO: Add information about the onException method here.

• Override onException in endpoint
• returning correct HTTP status code & errors
• use optional param "throw=" to display stack trace in browser instead of REST-friendly error code in headers

Once you have created your endpoint CFC, you need to add just a few lines to your mach-ii.xml file within the <endpoints></endpoints> block like this:

    <mach-ii>

        <!-- Other Mach-II sections -->

        <endpoints>
            <endpoint name="content" type="components.endpoints.ContentServiceEndpoint" />
        </endpoints>

        <!-- Other Mach-II sections -->

    </mach-ii>

With that declaration, the ContentServiceEndpoint will be loaded when the Mach-II application is loaded, and all of its RESTful methods will be exposed and available.

A REST Endpoint method should be relatively small, delegating to a service or model class that you connect with ColdSpring using Mach-II's ColdspringProperty and the depends attribute.

Here's a full REST endpoint example with a few of methods.

TODO: More here - using depends attribute to wire in service components

    <cfcomponent
        displayname="ContentServiceEndpoint"
        extends="MachII.endpoints.rest.BaseEndpoint"
        output="false"
        depends="contentService"
        hint="RESTful endpoint for the Content Service.">

        <!---
        PUBLIC ENDPOINT METHODS
        --->
        <cffunction name="getContentItem" access="public" returntype="String" output="false"
            rest:uri="/content/item/{key}"
            rest:method="GET"
            hint="Retrieves an individual content item.">
            <cfargument name="event" type="MachII.framework.Event" required="true">

            <cfset var key = event.getArg("key", "") />
            <cfset var stcResult = getContentService().getContentItem(key, event.getArgs()) />

            <cfreturn stcResult.body />
        </cffunction>

        <cffunction name="createContentItem" access="public" returntype="String" output="false"
            rest:uri="/content/item"
            rest:method="POST"
            hint="Create a new Content Item.">
            <cfargument name="event" type="MachII.framework.Event" required="true">

            <cfset var item = event.getArg('item', '') />
            <cfset var stcResult = getContentService().createContentItem(UrlDecode(item), event.getArgs()) />

            <cfset addHttpHeaderByStatusCode(201) />
            <cfset addHttpHeaderByName("Location","/content/item/#stcResult.key#") />

            <cfreturn "" />
        </cffunction>

        <cffunction name="updateContentItem" access="public" returntype="String" output="false"
            rest:uri="/content/item/{key}"
            rest:method="PUT"
            hint="Update Content Item.">
            <cfargument name="event" type="MachII.framework.Event" required="true">

            <cfset var key = event.getArg('key', '') />
            <cfset var stcResult = getContentService().updateContentItem(key, event.getArgs()) />

            <cfset addHttpHeaderByName("Location","/content/item/#key#") />

            <cfreturn "" />
        </cffunction>

        <cffunction name="deleteContentItem" access="public" returntype="String" output="false"
            rest:uri="/content/item/{key}"
            rest:method="DELETE"
            hint="Delete a Content Item.">
            <cfargument name="event" type="MachII.framework.Event" required="true">

            <cfset var key = event.getArg("key", "") />
            <cfset var stcResult = getContentService().deleteContentItem(key, event.getArgs()) />

            <cfreturn stcResult.body />
        </cffunction>

        <cffunction name="onException" access="public" returntype="void" output="false"
            hint="Override abstract onException method to report the exceptions thrown by the content service.">
            <cfargument name="event" type="MachII.framework.Event" required="true">
            <cfargument name="cfcatch" type="any" required="true"
                hint="The cfcatch object generated by an exception.">

            <cfset var stcHeaders = StructNew() />
            <cfset var strHeaderName = "" />
            <cfset var key = event.getArg("key", "") />

            <cfswitch expression="#arguments.cfcatch.type#">
                <cfcase value="contentitem.load.failed">
                    <cfset addHttpHeaderByStatusCode(404) />
                    <cfset stcHeaders["service.error.name"] = "service.content.error.item.notFound" />
                    <cfset stcHeaders["service.error.detail"] = "Content item #key# could not be found." />
                </cfcase>
                <cfcase value="contentitem.deleted">
                    <cfset addHttpHeaderByStatusCode(410) />
                    <cfset stcHeaders["service.error.name"] = "service.content.error.item.deleted" />
                    <cfset stcHeaders["service.error.detail"] = "Content item #key# has been deleted." />
                </cfcase>
                <cfcase value="service.content.key.exists">
                    <cfset addHttpHeaderByStatusCode(409) />
                    <cfset stcHeaders["service.error.name"] = "service.content.error.item.exists" />
                    <cfset stcHeaders["service.error.detail"] = "Content item with the key #key# already exists." />
                </cfcase>
                <cfcase value="service.content.item.moved">
                    <cfset addHttpHeaderByStatusCode(301) />
                    <cfset stcHeaders["service.error.name"] = "service.content.error.item.moved" />
                    <cfset stcHeaders["service.error.detail"] = "Content item #key# has been moved." />
                    <cfset stcHeaders["Location"] = "/content/item/#arguments.cfcatch.ExtendedInfo#" />
                </cfcase>
                <cfdefaultcase>
                    <cfset addHttpHeaderByStatusCode(500) />
                    <cfset stcHeaders["service.error.name"] = "service.content.error.unknown" />
                    <cfset stcHeaders["service.error.detail"] = "Exception: #arguments.cfcatch.message#." />
                </cfdefaultcase>
            </cfswitch>

            <!--- Create cfheader entries for each header in the struct, and add them to the log. --->
            <cfloop collection="#stcHeaders#" item="strHeaderName">
                <cfset addHttpHeaderByName(strHeaderName, stcHeaders[strHeaderName]) />
            </cfloop>

            <cfset getLog().error("Service Error Headers", stcHeaders) />
            <cfset getLog().error("Service Error cfcatch detail", arguments.cfcatch) />

        </cffunction>
    </cfcomponent>

There are several annotation based options to configure your REST endpoint methods. Annotations provide syntactical metadata and are embedded in your REST endpoint source code that influence the runtime the behavior of your endpoint.

It is easy to access custom metadata for an endpoint method. You can get all the information you need by getting the restUri object out of the event object inside your endpoint method:

    <cffunction name="onAuthentication" access="public" returntype="void" output="false" rest:someCustom="temp"
        hint="Runs authentication.">
        <cfargument name="event" type="MachII.framework.Event" required="true" />

        <cfset var restUri = arguments.event.getArg("restUri") />
    <cfset var someCustom = restUri.getUriMetadataParameter("someCustom", "defaultValue") />
    </cffunction>

This will allow you to set custom metadata in the rest namespace in the method.

By default, the REST endpoint performs content-length header checks automatically on all PUT and POST requests:

• Check that Content-Length header is present
• Check that byte size of the document body of the request is the same byte size indicated in the Content-Length header

The Content-Length header indicates the size of the document body in bytes and not the length of the document body. In normal ASCII, each character is one byte however once other encodings are used (i.e. UTF-8) characters in extended regions of the character encoding may use two or more bytes per character. This is why we check the byte size of the document and not Len(documentBody) instead.

You can turn off this feature by setting the follow code in the pseudo-constructor of your CFC that implements the REST endpoint:

    <cfset variables.enforceContentLengthDefault = false />

The REST endpoint provides the ability to build URLs specific to endpoints. There are two access the buildEndpointUrl() method -- internally your REST endpoint CFC and externally from other places in the framework. This method accepts either named arguments or positional arguments.

Internal

Calls from within your REST endpoint differ than calling buildEndpointUrl() from elsewhere in your Mach-II application.

    <!--- Positional arguments --->
    <cfset endpointURL = buildEndpointUrl("method=methodName", arg1, arg2, arg3) />
    <!--- Named arguments --->
    <cfset endpointURL = buildEndpointUrl("method=methodName", id="1", arg2="foobar") />

External

Calling buildEndpointUrl() outside of an endpoint, the endpoint name is required.

    <!--- Positional arguments --->
    <cfset endpointURL = buildEndpointUrl("endpointName", "method=methodName", arg1, arg2, arg3) />
    <!--- Named arguments --->
    <cfset endpointURL = buildEndpointUrl("endpointName", "method=methodName", id="1", arg2="foobar") />

For any endpoint that returns JSON formatted data, you can optionally supply a callback function name via the 'jsonp' argument. This will cause the endpoint to wrap the response body and change the content type to 'application/javascript'. For example:

    /content/item/my_content_item.json?jsonp=myfunction

Tomcat and JRun by default do not parse the request body to form variables (i.e. application/x-www-form-urlencoded), which are put in the Event object, for HTTP methods other than POST. The most common complaint is that some REST clients send data that is form encoded for PUT and DELETE HTTP method however Tomcat and JRun servlet containers do not parse the request body.

Jetty does parse the request body for PUT. The HTTP specification does not indicate that form encoded request bodies are invalid however the Java Servlet specification implies that it is only required for POST HTTP method requests.

In order to support form encoded request bodies, you can turn on this feature in the REST endpoint by listing the alternate HTTP methods that you want the REST endpoint to parse the request body. By default, we do not parse the request body unless indicated.

Simply add following to the properties section of your REST endpoint to turn on request body parsing. All servlets automatically parse the request body for the POST HTTP method.

    <cfset variables.parseRequestBodyParametersMethods = "PUT,DELETE" />

The base REST endpoint preProcess() method sets many event args that may be useful to you. All of these can be accessed by using event.getArg("_X") in your methods. Most args that are set by Mach-II are prefixed by _ to aid in reducing the possibility of collisions in the event args with values set by developers.

Request Args

These args are set when the REST endpoint handles your response:

Response Args

This is an area where the phrase "it depends" applies heavily. There is no RFC specification for authentication for REST service layers so the industry offered a multitude of themes and variations on the themes. The early standard was to "roll your own" authentication schema. This lead to a total fragmentation of authenticating to RESTful services that just about all REST APIs authenticate in a totally unique way. Over time it appears that a few themes are "winning" over the cacophony of subtle variations.

Internal API and behind a firewall

This is common for REST APIs that are used internally in a company where access to it locked to certain network IP address or you are behind a firewall.

Public and Open API

This is probably atypical as most public APIs should provide at least basic authentication so "heavy hitters" of the API can be tracked and shut down if they are abusing your API.

Here is an example of adding basic HTTP access authorization to a REST endpoint. In the "configure" method, you can see we are instantiating a basic auth module and passing in the "realm" (part of the basic authentication standard) and the path to the Apache-style credential file (the documentation explain more about this). Later on in the code, we use the "onAuthenticate" point in the endpoint request lifecycle. This method is called between the "preProcess" and the "handleRequest" point cuts. One thing to note is in the REST endpoint we can mark REST methods with rest:authenticate and use that annotation in our "onAuthenticate" method to determine if authentication is needed. Some REST APIs have public methods without authentication and some methods with authentication. We can also defined rest:authenticate="true" on the cfcomponentto indicate a global default which is available by callinggetAutenticationDefault()`.

The REST endpoint will programmatically call onAuthenticate() if one or more of following is true:

• The REST method has rest:authenticate="true" metadata on the method definition.
• The REST CFC has rest:authenticate="true" metadata on the CFC. This is a global flag and sets that all REST methods must authentication unless otherwise stated differently on the CFC REST method directly.

    <cffunction name="getContentItem" access="public" returntype="String" output="false"
        rest:uri="/content/item/{key}"
        rest:method="GET"
        rest:authenticate="true"
        hint="Retrieves an individual content item.">

In the end, the basic HTTP access authentication module takes care of checking the headers, decoding the auth, checking the credential file and ultimately setting other header if the authentication request fails. This module returns a boolean so you can customize the exception output if the authentication fails. Getting back to the code below if the authentication fails we throw an exception. By default, the base REST endpoint handles all exceptions except in this case we are defining special logic in the onException() point-cut to provide the correct data back to the client.

    <!---
    INITIALIZATION / CONFIGURATION
    --->
    <cffunction name="configure" access="public" returntype="void" output="false"
        hint="Configures the API endpoint.">

        <cfset variables.authentication = CreateObject("component", "MachII.security.http.basic.Authentication").init("Dashboard API", getParameter("apiCredentialFilePath")) />

        <cfset super.configure() />
    </cffunction>

    <!---
    PUBLIC METHODS - REQUEST
    --->
    <cffunction name="onAuthenticate" access="public" returntype="void" output="true"
        hint="Runs when an REST method is marked with rest:authenticate='true'.">
        <cfargument name="event" type="MachII.framework.Event" required="true" />

        <cfset var restUri = arguments.event.getArg("restUri") />

        <!--- Authenticate the request via HTTP basic authentication --->
        <cfif variables.authentication.authenticate(getHTTPRequestData().headers)>
            <cfthrow type="MachII.dashboard.endpoints.notAuthorized"
                message="Bad credentials." />
        </cfif>
    </cffunction>

    <cffunction name="onException" access="public" returntype="void" output="true"
        hint="Runs when an exception occurs in the endpoint.">
        <cfargument name="event" type="MachII.framework.Event" required="true" />
        <cfargument name="exception" type="MachII.util.Exception" required="true"
            hint="The Exception that was thrown/caught by the endpoint request processor." />

        <!--- Handle notFound --->
        <cfif arguments.exception.getType() EQ "MachII.dashboard.endpoints.notAuthorized">
            <cfset addHTTPHeaderByStatus(401) />
            <cfset addHTTPHeaderByName("machii.endpoint.error", arguments.exception.getMessage()) />
            <cfsetting enablecfoutputonly="false" />

            <cfoutput>
                <cfinclude template="/MachII/security/http/basic/defaultUnauthorized.cfm" />
            </cfoutput><cfsetting enablecfoutputonly="true" />

            <!--- Default exception handling --->
        <cfelse>
            <cfset super.onException(arguments.event, arguments.exception) />
        </cfif>
    </cffunction>

The debate on one-time use authorization tokens is rather lengthy discussion. There are many pros / cons and in most circumstances one-time use tokens are not worth the added complexity. Some factors to consider are:

• Each method call requires two requests therefore doubling the required server resources and network latency to process requests
• Added complexity to authenticate over basic HTTP authentication. All decent HTTP clients implement basic HTTP authentication however an one-time use token schemas has not been standardized and therefore all clients would have to implement a non-standard authentication schema.

Below are questions that were asked during the Q&A session in a Building RESTful Service Layers presentation by Team Mach-II member Doug Smith (October 2010). There were some great questions so we decided to convert the questions in to FAQs for the wiki.

Where do you do your authentication/authorization? In the endpoint, or in another layer?

You would typically perform authentication in your REST endpoint. This is because your authentication process is going to be specific to your REST endpoint implementation and not to your service layer directly. It's not really the concern of your service layer to authenticate calls to it but really the responsibility of your REST endpoint to make sure the client is authenticated correctly before calling your service layer. This implementation strategy makes your service layer more flexible as it's not tied to a specific authentication schema and allows you to swap in a new authentication layer in your REST endpoint in the future.

So, for authentication would you call authenticate method from all the secure methods?

Doug touched this in the presentation if you watched it. Endpoint have a special request lifecycle methods available. You could easily do perform global authentication in the preProcess method otherwise for URI specific authentication you need to do that in your URI method. By default, the base REST endpoint takes care of certain things in the preProcess like parsing the path info, deciding on the method to call, the type of response format, etc.

Because it's relatively new feature in the BER, Doug didn't mention that Mach-II 1.9 is shipping with a basic HTTP access authentication module. By default, it reads an Apache authentication file for all credentials but this module is easily extensible and you can swap in other credential checking schemas like a database by extending this CFC.

So you mention it is scalable, but how do you make it scale?

REST calls act just like any other web application. A client is a client over HTTP whether it is a web browser or you are programmatically calling it from some web development language. It is proper form that REST calls are stateless. This means that all calls should sent authentication credentials with each request (if required) and that the server does not maintain a "state" (like a CFML session scope) for the client. This allows your REST endpoint to reside in a cluster of servers without having to tie calls from clients to specific servers.

Doug mentioned that they have rolled out a REST API at DaveRamsey.com using the Mach-II 1.9 M2 release (BER). I believe he said they are handling about 800k requests per day with it.

Do you know how the Mach-II Dashboard handles reloading of endpoints in a clustered environment?

Currently the Dashboard handles applications on a per server basis. For those who don't use Mach-II, the dashboard is a development tool build as a Mach-II module that can help you develop and manage your application. I'll leave the laundry list of features out of this blog post, but check out the Dashboard documentation with screen shots for more information. Getting back to the question. We're build a REST API for the Dashboard so I expect in the future we'll see some awesome cluster support where you can send messages to all your clustered servers.

When structuring xml responses, any tips on avoiding bloat in your service layer?

Try to think of formatting your response in a way that is useful to the client instead of scratching your own desires in how you feel data should be organized. Don't be tied to your database model as they may not be useful to your client. At DaveRamsey.com, Doug Smith explained they have had great luck using JSON instead of XML because it has less "bloat" to in the response (i.e. smaller). It's not uncommon to offer responses in multiple response formats (XML, JSON, plain text, etc.) as a client that consumes your REST service may be able to work with XML over JSON easier or vice-versa.

Was the response type json or html handled by base REST API component or you had that in your service?

Yes, the base endpoint looks for certain "file extensions" and automatically sets the correct mime type in the response to the client. By default, can set the default response type if the "file extension" is left off the URI. The type of the response is in the event.getArg('format') event arg.

Have you ever done REST as a wrapper for SOAP in cases where you want to offer the API in both flavors to the world?

Doug answered that they haven't done that at DaveRamsey.com, but they have thought about creating REST wrappers for common SOAP services they have to interact with on a daily basis to make their development lives better.

There is no WSDL equivalent for REST is there?

Yes, there it's called WADL which is in its infancy stage (Fall 2010). Jason York (Team Mach-II and DaveRamsey.com) is already looking into creating a WadlDocumentationGenerator.