The Future of JavaScript is X

June 4, 2015 1 comment

JavaScript in 2015

JavaScript has gotten way better than it was originally. The Good Parts actually make JavaScript a pleasure to program (and much closer to Python, IMHO). Firebug and other in-browser debuggers make it possible develop large-scale projects without losing your mind.

Browser improvements and Node.js have expanded the speed and reach of JavaScript. Jeff Atwood postulates that

any application that can be written in JavaScript, will eventually be written in JavaScript

Node.js is especially compelling, because it opens up the idea that all software might eventually run with JavaScript, be it in the browser or on the server. Write once, run anywhere? Maybe. But maybe not.

The Problem with JavaScript

JavaScript in the browser has one huge gaping hole: It must be sent in the clear. For a browser to run it, this code must be valid JavaScript. Sure, you can encode it and decode it on the fly, but the decoder has to be written in JavaScript. This means that any sufficiently-diligent programmer can reverse-engineer any JavaScript code. There is no way to hide implementation details using JavaScript.

Web APIs to the rescue?

Enter Web APIs. Here, we separate out business logic and other implementation details from the front-end JavaScript. We place them in back-end server code, be it Node.js, Python, Ruby or COBOL (yikes!). No browser ever receives the code that runs on the server, so that’s great, right?

Actually, no. Web APIs do not protect a sufficiently-diligent programmer from documenting your process. In fact, many Web APIs are publicly available, which encourages use. And that’s good. But it does not protect your process.

In reality, Web APIs just complicate things. If your front-end has to call your back-end, then you must implement logic to parse/emit data structures that can be sent across the wire. If you’re using Node.js cleverly, you can implement that once and use it in both places. (Browserify is your helper, here.) But if you are in a heterogeneous environment, you may have servers running Python, .Net, J2EE (shivver!) and Perl. Each of these must also implement parse/emit logic. (“Write once, run anywhere” just went out the window!) Keeping all these in-sync (esp. with versioning in the URL!) is a nightmare.

Chapter 5 REST to the rescue?

Chapter 5 REST would help here, esp. with anarchic scalability. But many developers don’t understand Chapter 5 REST, and fewer are willing to collaborate at that higher level. This is why Web APIs have taken off: it’s low barrier to entry, and you don’t actually need to think about how your own application might be a subset of a broader (global) problem space. (And that’s all I’m going to say about Chapter 5 REST in this article.)

Browser plug-ins to the rescue?

Flash used to be the de-facto way to get closed content to the browser. It’s old news by now that Apple iOS doesn’t run Flash. Microsoft at one time wanted to do the same with its browser…but now they support Flash by default. I’m not sure how I feel about that.

You could install another binary plug-in, but installing browser plug-ins is an administrator’s nightmare. Enterprise security policies may also allow BYOD, but then you have to install the plug-in on devices outside your enterprise control. Not fun.

The original browser plug-in was Java. I don’t write many Java applets these days. Do you?

JavaScript’s replacement

So, the killer of JavaScript is its readability (both machine and human). If you want to protect your implementation details, then you must expose as little JavaScript as possible to the browser. Two relatively new technologies (2011 and 2004) have emerged that enable extremely-thin JavaScript browser clients: websockets and Canvas.

Wait, wait! Couldn’t we do this with AJAX and images?

Well, maybe. But not well enough.

First, AJAX really doesn’t allow the server to send to the browser directly. Several tricks (including HTTP Long Polling and others) have sort-of made that possible. But it hasn’t been really good enough to support massive server-initiated redrawing.

Second, messing with images messes with the DOM. Anytime you mess with the DOM, the browser tends to get really slow. Canvas allows much faster updates that do not impact the DOM.

Websockets and Canvas to the rescue!

It is almost dumb-simple to setup a websocket connection in a browser with JavaScript. The hard part is building the parse/emit logic and then the code that processes incoming messages. Out-going messages are trivially simple–just a single function call. If you get rid of the hard part, then you’re left with a bare-bones, lightweight client. Here’s how.

Don’t implement application-specific message processing in your browser app. Not one bit. Just implement code that takes the server commands and renders them on the Canvas. You could (if you’re lazy and inside a trusted network with no fear of Bad Actors) let the server send actual JavaScript code and eval it client-side. That would, however, be evil.

If you’re a bit more diligent, you still can build minimal code to process rendering commands on Canvas.

You’ll also need to send back keyboard and mouse events to the server. But that’s a very tiny bit of info, namely the (x, y) coordinate, which mouse button and/or key was pressed. The server can process that and stream more rendering commands back via the websocket.

What about multimedia?

Any technology gets harder with multimedia. The same is true here. I’m not going to focus much on that now. However, take a look at WebRTC for peer-to-peer media connections and streams. Just overlay a Canvas on top of a element, and you’re still able to control the video with the least amount of front-end code. Or, if you have tons of server resources (like a Media Server farm), then you could conceivably proxy all the media streams and issue image updates via websockets to Canvas. (That way lies madness, methinks.)

Isn’t this more like VNC?

Maybe. It’s a spectrum that depends on how lightweight your client code is. On the one end, you could send partial images and blit them where needed (using drawImage)–this would be like VNC. On the other end, you could implement higher-level functionality with additional Canvas methods–this would be more like the X Window System.

What about SVG?

I’m no expert, but you could replace Canvas with SVG. It’s a different beast, but the same principal applies, and implementations fall along the same spectrum from VNC-like to X-like.

Really, now…why would anyone do this?

Companies like to protect their intellectual property. Arguably, a JavaScript front-end is protected by technology (HTTPS) and copyright law, but there are some for whom these protections simply are insufficient. They already use similar technologies like VNC, Remote Desktop Protocol and exported X-Window sessions. This websockets/Canvas solution has one major advantage over those: it will run in the browser without any installation…as long as JavaScript is turned on.


Categories: Projects

The RESTaurant Analogy

October 21, 2014 Leave a comment

The RESTaurant Analogy

Try, try again

Real REST is hard…but only because we bring a lot of preconceptions to it. I’ve tried before to explain it as a telescope (for people who are used to microscopes). I’m hoping this analogy will be easier, because it’s one place where the real world works much like a RESTful client-server interaction.

CAVEAT: This example is a bit contrived. I hope you don’t use HTTP with a real waitress.

ASIDE: Customers, Chefs and Waitresses are fabulous individuals. This article intends nothing but respect for all people.

A visit to a RESTaurant

You walk into a short-order restaurant one morning. The hostess leads you to a table and you sit down. The waitress immediately arrives. Since you’re hungry and already know what you want, you ask the waitress politely for it. She scribbles some notes, smiles and walks away.

You open the daily paper you brought with you and start reading. After a few minutes, the waitress returns with your plate of food. Since you’re so engrossed in reading, she sets the plate down on the table.

When you hear the clink of the plate on the table, you look down and see your plate. It’s exactly what you ordered, so you dig in. Yum!

The same visit, expressed in HTTP

All the examples use HTTP Status Codes. You should keep them handy for reference. In fact, you should memorize them, they are that important.

From the Customer, to the Hostess:

GET /table?for=1 HTTP/1.1
Accept: table

From the Hostess, to the Customer:

302 Found
Content-Type: table
Content-Location: /table/13
"Corner booth, by the jukebox."

The act of the Customer sitting down is similar to this:

GET /table/13 HTTP/1.1
Accept: table

(I’m ignoring the server response here, but it would return a Table Resource.)

From the Customer, to the Waitress:

POST /order/please/ HTTP/1.1
Accept: plate;q=1.0,wait;q=0.9
Content-Type: OrderForFood
"Two eggs over easy, with corned beef hash."

(We’ll go back to those “q” values in a little bit.)

Response from Waitress:

202 Accepted
Content-Type: wait
"I'll be right back, sweetie."

From the Waitress, to the Chef:

POST /plate/with/food/ HTTP/1.1
Accept: plate;q=1.0,wait;q=0.9,slot;q=0.8
Content-Type: OrderForFood
"Two eggs over easy, with corned beef hash."

Response from the Chef:

202 Accepted
Content-Type: slot
Content-Location: /slot/5
"I'll put it on the counter in slot #5."

Now, periodically, the Waitress glances at Slot 5 to see if the plate is ready. (This is “polling”.)

The Waitress checks the Slot like this:

GET /slot/5 HTTP/1.1
Accept: plate

And the Slot responds this way when it’s empty:

204 No Content

The Slot response this way when the order has been cooked and put on a Plate here:

200 OK
Content-Type: plate
"Two eggs over easy, with corned beef hash, grits and toast."

NOTICE: The Plate Resource contained grits and toast, even though the Customer’s OrderForFood Resource didn’t explicity request it. I guess you always get grits and toast at this Restaurant, whether you ask for them or not.

When it is, she goes and gets it and brings it back to the Customer. The Customer hears the clink of the plate on the table and sees his order.

There are two ways to accomplish this: Server-Server Notifications or Client-Server Polling.

Server-Server Notifications

One would be for the Customer to have a “web callback” method that the Waitress uses. (This is sometimes used in a Server-Server environment, or with websockets.)

The Waitress would notify the Customer this way:

POST http://customer.server.tld/hey/you/ HTTP/1.1
Content-Type: plate
"Two eggs over easy, with corned beef hash, grits and toast."

In this case, the Customer would acknowledge receipt:

202 Accepted

Client-Server Polling

The other would be for the Waitress to use the shared Table Resource, which the Customer polls. (This is more likely in the Client-Server environment of the Web these days.) It goes like this:

The Waitress PUTs the Plate on the Table:

POST /table/13 HTTP/1.1
Content-Type: plate
"Two eggs over easy, with corned beef hash, grits and toast."

The Table then has the Plate on it, as it shows in its new Representation:

202 Accepted
Content-Type: table
"Napkins, Utensils, and Plate"

Then, the Customer can see what’s on the Table:

GET /table/13 HTTP/1.1
Accept: table

And the Table’s response is the same, except it’s a “200 OK” status code this time.

Coffee and Content Types

Throughout your meal, you’re probably going to drink lots of coffee. Let’s presume that your Waitress likes to be asked, all the same.

The Customer asks for a mug of coffee:

GET /coffee HTTP/1.1
Accept: mug

The Waitress is able to fulfill your request immediately, thanks to clean mugs stacked nearby and the coffee pot she carries around (almost) all of the time.

200 OK
Content-Type: mug
"full of caffeinated, hot coffee"

“Wait a minute!” you say. “I drink iced coffee.” No problem. Let’s just ask for a glass instead of a mug:

GET /coffee HTTP/1.1
Accept: glass

Well, now, you can’t pour hot coffee into a glass. So, your Waitress (who happens to also carry a pitcher of fresh iced coffee) is also able to fulfill this request immediately:

200 OK
Content-Type: glass
"full of caffeinated, iced coffee"

The point is this: Coffee is coffee. But it comes in many forms.

In REST, this is the same as saying: A Resource is a Resource, but it has many different Representations. You tell the server what Representation you want, and let it do the best it can to fulfill your request.

The reality is that sometimes, a server can’t give you what you want.

That’s kind of like when the Waitress runs out of hot coffee. But she still has iced coffee. And on this particular day, she’s not feeling like making a fresh pot…so, you’re just gonna get iced coffee. Her response then is:

406 Not Acceptable
Content-Type: glass
"full of caffeinated, iced coffee. Just pretend it's hot, sweetie!"

That 406 Not Acceptable status code means, “Sorry, I don’t have what you asked for, but here’s something else. I hope you like it…even though you didn’t ask for it.”

How do we know what a Waitress does?

If you’ve never been to a Restaurant before, then you’ll have to be taught. This is “out-of-band” information–that is to say, you won’t find instructions at the Restaurant about how to interact with a Waitress. The best you’ll get from the Waitress herself is a Menu–which also contains no instructions on how to use it. (This Menu is actually hinting at another REST constraint: Hypertext As The Engine of Application State. But we’re not getting into HATEOAS in this post.)

In REST, there is no “Web Application Description Language,” despite the so-called WADL “standard.” Rather, there are widely-agreed upon specifications for Resources. These specifications are created in the open, with a global audience, because we want input from everyone who knows about these Resources. The specifications themselves are published in a human-friendly document, which is not meant to be parsed by a program. The goal of Resource Specification Documentation is to educate the programmer, who will then write programs that are based on an understanding of how the Resource works.

In short: By eliminating machine-generated libraries, we actually simplify the programming of web clients and servers.

Versioning, or how we even got iced coffee

Back in the day, Waitresses only served hot coffee. So, if you made a generic request for coffee, you always got a mug.

Then, on a hot day, someone decided to dump ice into his coffee mug…voila! Iced coffee was born.

In our Restaurant, the Waitress knows about iced coffee and can respond appropriately if you ask for a “glass” of coffee. But, in some Restaurants, their Waitresses don’t serve iced coffee. The end result could be similar to when our Waitress ran out of hot coffee:

The Customer asks:

GET /coffee HTTP/1.1
Accept: glass

And the Waitress responds:

406 Not Acceptable
Content-Type: mug
"full of caffeinated, hot coffee. Dearie, we don't serve that fancy stuff here!"

In this way, you can always try to ask for newer Representations of a Resource, even if the server might not support them.

This is how REST supports versioning. Older servers will still support the older Representations, just like all (OK, most) Waitresses will be able to pour you a hot mug of coffee. Newer servers will support additional Representations, and it’s safe to ask for the new ones, but be prepared to handle the older ones, just in case.

The “quality” indicator in the “Accept” header allows the Client to put a preference on what it would like to receive in a Response. If, for example, the Customer would prefer iced coffee, but would be OK with hot coffee, or whatever, this is how his Request would look:

GET /coffee HTTP/1.1
Accept: glass;q=1.0,mug;q=0.9,*/*;q=0.1

ASIDE: This feels very backward to me, because English uses the semi-colon (;) and comma (,) in exactly the opposite way.

The Server would then pick the Representation that it can render that matches the highest “q” value. The “*/*” means “whatever”. If it can’t satisfy the “Accept” header, it sends back whatever it can render, with a “406 Not Acceptable.”

This is also how Server Upgrades can happen without forced Client Upgrades. This “anarchic scalability” is what has allowed the Web to grow fast, despite the Browser Wars, and all the incompatibilities of lesser browsers.

A Waitress is a Woman, too

So, my young son understands how to ask a Waitress for an Order. But one day, he may also recognize some special Waitress as an attractive woman, as well as just the friendly person who works at the Restaurant. Since I’m teaching him how to be respectful and responsible, he may some day know how to politely and courteously ask this Woman for a date.

Without trivializing such an important step for a young Man, this is how it might look in HTTP:

GET /date?when=Friday&where=ShoppingMall
Accept: date;q=1.0,maybe;q=0.9

Obviously, he would like to receive a confirmation of his Date request (hence, the “q=1.0”). But, he’d be OK with a Maybe.

Now, let’s imagine that the young Woman knows that she must ask permission from her Parents before accepting a Date. In this case, she would probably respond with:

401 Unauthorized
WWW-Authenticate: "You need to ask my dad first"

If he gets up the courage to ask her Father for permission, and her Father grants it, he can again issue his original request, this time with Authorization:

GET /date?when=Friday&where=ShoppingMall
Accept: date;q=1.0,maybe;q=0.9
Authorization: "Your dad said it was OK to ask you out."

Pause a moment. Consider that my young son has no idea about Dating yet. He can still interact with the Waitress. At the same time, more mature Clients know other additional ways to interact with the same Woman. This is another example of “anarchic scalability”–the same Server can interact with multiple Clients with older and newer functionality.

Just don’t ask a married Woman for a Date. You should expect her to reply with a “400 Bad Request” status, which means “Don’t ask again.” (Not to mention what her Husband will do, if he hears about it.)

I see, said the blind man

People often have two reactions to clear REST analogies:
1. “What? I’m not doing REST already?!”
2. “Oh, I get it! REST is JSON!”

Both are wrong. But it’s hard to blame them, because large companies (Microsoft and IBM) have been peddling SOAP/RPC solutions as “REST” so long, that almost everyone thinks they’re doing Real REST.

It’s not easy peeling off a bandaid. But bear with me, because you’ll feel better once you get that wound opened to the clean air.

A visit to a SOAP factory

The SOAP factory is different from a restaurant. It’s purpose is to make soap. And if you know the right stuff, you can make your own soap any way you like.

ASIDE: I don’t enjoy this part, so I’m going to be brief.

No Waitress, Just Instructions

When you walk into the SOAP factory, no one greets you. Instead, there’s a big sign on the wall entitled “Web Service Description Language.” But you have to know which wall it’s on first.

Once you get to the WSDL wall, you can read it for more instructions. It’s not really like a restaurant Menu, because it doesn’t tell you what you can get out of the SOAP factory. Rather, it’s like a “How To” manual: it tells you how to perform activities at the SOAP factory. Unfortunately, it often gives you no clue about the order in which these steps should be performed.

So, to make some SOAP, you have to…do what? How about that “MixIngredients” instruction? Well, you try that, and you do get back something. But it’s not what you expect. Instead, it’s some strange looking plastic egg. If you crack it open, out falls a ticker-tape message:

    "MixIngredientsResponse": {
        "err": {
            "forgot": "ingredients"
            "code": "5016d6fb854f46a645868f1d4f58d4dc"
        "data": null

Hmm. OK. “So, I need to find some ingredients,” you say. What about that “ListIngredients” instruction? This time, you’re lucky. The egg you get back has inside it this ticker-tape message:

    "ListIngredientsResponse": {
        "err": {
            "code": "0",
        "data": {
            "ingredients": {
                "ingredientsList": [

And, based on this message, you start to hope.

So, after a hard day’s work at the SOAP factory, you’ve built a program that properly uses the SOAP Factory API (Application Programming Interface). Congratulations!

Now, in how many other factories can you use this program? Short answer: None. Zip. Zilch. The SOAP Factory instructions are unique to that particular Server. You could try it somewhere else, but good luck. The sad truth is that WSDLs are not built by a global community of people and are simply not standardized.

Worse, they depend heavily on the implementation. Changes internal to the Server often result in breaking changes to the API. SOAP has no way to deal with Content Negotiation, like REST does. As a result, when the API changes, Clients are forced to upgrade…or die.

So, use your shiny SOAP program today. But don’t expect it to work tomorrow.

SOAP in the Restaurant

Let’s go back to our Restaurant analogy to wrap this up.

Using SOAP on the Web is like going into a Restaurant and telling the Chef how to make the food. Instead of placing your order with the Waitress, you write down an exact list of tasks for the Chef to perform: “Turn on oven to 400 degrees. Open bacon. Place bacon on cookie sheet. Put cookie sheet in oven for 30 minutes. Take cookie sheet out. Put bacon on plate.”

It would be insulting to the Chef, and you’d probably be laughed out of the Restaurant. Nobody orders food this way. So…why are we still trying to build Web Apps using SOAP?


Real REST is hard. But it’s easier than anything else in the long run.

I challenge you, my fellow programmer, as well as the global community of developers: Let’s get together and create Resource Specifications about our specific Domains of Knowledge. Let’s create reusable, understandable documentation that explains what we mean. Let’s stop worrying so much about how it’s done, and focus on solid, reliable information Resources. (And please, stop saying REST in the same breath with JSON and CRUD.)

See you at the RESTaurant.:)

Categories: Projects

REST: A Starting Point for Implementors

March 10, 2014 3 comments


REST is difficult.  For starters, Roy Fielding’s dissertation is not for the faint-of-heart. (But if you enjoy academic reading, go for it!) Then, there’s a lot of noise on the Internet, esp. from vendors and SOAP/RPC Web Service developers that mucks up the water.

This article is for developers who are ready to start building with REST.

The point of REST

I wrote a visual analogy about looking into the right end of the telescope. It is critical to get the right perspective before trying to build with REST. (Please read that before continuing.)

ZapThink nails it (in

REST isn’t supposed to make the Web more like system integration; it’s meant to make system integration more like the Web.

So, if we are trying to make our system integration work more like the Web, what do we do first?

Step 1. Build a Web site

Yes, this sounds like advice from the 1990s. But let’s see what happens:

  1. You figure out what kinds of pages you want on your site.
  2. You figure out how to navigate among them.
  3. You decide what domain name you want and build a home page at the root URL.
  4. You style it with CSS and add interactivity with Javascript.

Now, what do you have? A browsable web representation of your pages that satisfies the Hypermedia constraint for any web browser clients. It’s already REST, simply because HTML-over-HTTP implements a subset of REST.

Step 2. Drop the fancy formatting

Your clients probably don’t need CSS or Javascript. (Unless you’re supplying “code-on-demand”, in which case, why are you reading my humble article?) So, drop all references to CSS, javascript and whatever else is there for visual formatting purposes.

If you have a standard page header and footer, you probably want to drop them, too. Side-bars and other junk that’s not related to the page content? Ax them.

DO keep whatever links are there for navigating to related content–even back to the root page (aka, the “entrypoint”)!

DO keep the HTML tags that provide semantic structure, like <h1>, <div>, <caption>, etc. (You get a lot more of these semantic tags with HTML5.)

When you’re done, you should still be able to browse the web site, but it will look as plain-Jane as HTML 1.0. Good!

Step 3. Document page types as distinct Resources

Typically, web sites will have blog entries, shop items, bio items, and stuff like that. Try to figure out what the real meaningful breakdowns are: those will be your Resources.

Then, Document those Resources online–SOMEWHERE ELSE! It can be a github project, a wiki somewhere, or whatever. It should NOT be embedded within your web site content, although you might link to the Resource specifications somehow. (See the video below for an example of using “class” attributes to link to Resource specifications.)

Your Resource Specification documentation should describe the Resource in human-readable (NOT machine-parsable!) prose. The reader needs to be able to understand what the Resource represents, what kinds of links (link relations) it might have, and a rough idea how various Resources might interact. Don’t worry too much about the last part now, because we’ll refine that in later steps.

Don’t forget: Each Resource gets its own Content-Type. This will be critical later, when you want to create a new version of a Resource, or provide a different Representation format for the Resource.

Step 4. Share and solicit feedback

Actually, do this throughout all the steps. If you haven’t started by the end of Step 3, you really need to get feedback starting now.

You want your friends, fellow developers, and pets to have a say at what’s going on in your mind–as expressed in your Resource specifications.

Listen to their feedback and adjust your Resource specifications accordingly.

If you have a big project with lots of people who might comment, you might consider starting some sort of Request for Comment (RFC) process to manage all the feedback.

Step 5. Remove all HTML FORMs

“What?! Then how do clients figure out what fields they can submit?” They don’t. That’s SOAP thinking.

The HTML FORM really is a UI affordance for web browsers, which interact with users. A web browser can’t be expected to know all the site-specific types of forms that can be POSTed to every website in the world, so the application/x-www-form-encoded Content-Type was created. As a consequence, web browsers AND web frameworks are forced to have a ton of code to parse, format and validate HTML FORM data.

Your clients don’t need this. If a client is going to collect data from a user, then the client will have some sort of UI code to do this. But your client does NOT need to parse arbitrary form definitions from the server. Why? Because you control the Resource types that the client can POST to your REST server.

As you yank out each HTML FORM, create a Resource Specification document that describes the schema, content-type, and whatnot. Update other Resource documentation so that your client developers know where they can PUT or POST these new Resources.

(HINT: Save the HTML FORM snippets. You might want to reuse them later. See Afterthought #2.)

You’re Done!

Here’s what you now have:

  1. Proper REST implementation of Resources…
  2. …satisfying the Hypermedia Constraint.
  3. And MOST IMPORTANTLY, documented Resource Specifications.
  4. BONUS: That you can test in a web browser.

Here’s some more you could do:

  1. Help the client developers write their client code according to the Resource Specifications. Make sure they rely on the link relations, and don’t try to construct URLs on their own.
  2. Twiddle with your server and try to use HTTP PUT, DELETE, PATCH and other verbs more effectively. (But, if you have a read-only REST implementation, these don’t matter at all.)
  3. Specify additional Representations for your Resources. Right now, you have a HTML (XHTML, if you were careful) Representations of your Resources. You could add JSON, XML, binary, whatever Representations; however, make sure that you write just as good Specification documentation for these new Representations!
  4. Tweak your server’s HTTP Response Codes. Probably, your web server isn’t sending *quite* the right HTTP Response Codes, esp. for things like 201 CREATED. You should fix that as soon as you get a chance, because real REST clients are going to be looking for that after POSTing a new Resource.
  5. Play with HTTP Caching. You will need to deal with Expires headers and such, but it should make sense if you’ve done any web caching configuration. There’s a lot of good documentation out there.

Video Break

You’ve read this far! Great! Now take a video break and watch this video. It’s the absolute best presentation of how to implement REST that I’ve ever seen.

(I don’t necessarily agree with the way he uses HTML FORMs. But for this demo, it’s great. And maybe I still don’t *get* how he’s using HTML FORMs.)

Afterthought #1. In a perfect world, Resources come first

Building it up this way helps us to stay RESTful, because we’re forced to be browseable. However, the best way to start using REST is to define Resource specifications first. If you have a way to get everyone to think about the Representations before you start, then great! However, in our mixed-up SOAP-sudsy world, you’ll get a lot of non-REST thinking about “web services”. So, pragmatically, it might help your team to think RESTfully if you start with a web site first.

Afterthought #2. For browser-testability, go back and re-add HTML FORMs

So, your client knows about all the Resources from your Specifications? Great! But if you want to use Selenium or another browser-automation tool for testing, you need to help the browser know what to do.

You can re-add those HTML FORMs that you yanked out in Step 5. But don’t add them directly into your Resources; instead, provide a special link relation to a “create-form” and “edit-form” Resource (which will be an HTML page containing an HTML FORM).

If you need to POST content-types other than application/x-www-form-urlencoded (such as application/json or application/xml), you might need to supply some javascript that will convert the FORM values into the appropriate representation format. Mozilla has created the FormData javascript library to help with exactly that.


Implementing REST has been difficult because…well, we weren’t implementing REST. Once you throw away SOAP and get back to plain old web development ideas, REST becomes much easier.

I hope this helps you get started with REST. Please drop me a comment if it does.:)

Categories: Projects Tags: , , , ,

SOAP vs REST: Two Opposing Perspectives

March 10, 2014 2 comments


There is a lot of confusion about SOAP and REST. SOAP developers mistakenly think they can support REST simply by using a JSON data format. Other developers want to use REST, because they understand the benefits, but simply don’t know where to start (or worse, they start with SOAP and try to turn it into REST).

SOAP and REST are completely different, incompatible perspectives.

SOAP is a Microscope

SOAP (and all RPC) requires one system to look inside another system. Imagine one system looking through a microscope at the guts of another system.

Because WSDL is easily-parseable, many developers rely on their tools (Eclipse, Visual Studio, IntelliJ, etc.) to generate SOAP libraries for accessing “services” on SOAP servers.

Unfortunately, just like a bacteria culture, SOAP services often change dramatically or even die off entirely without warning. It’s a constant struggle for SOAP clients to keep up with the changes, esp. when accessing multiple SOAP services on multiple SOAP servers.

Staphylococcus aureus

REST is a Telescope

REST requires systems to focus on shared Resource specifications that exist external to themselves. Imagine a group of peer systems standing on the same side of a telescope, taking turns looking up at the stars.

There is no need for machine-parsable WADL-like specifications, because Resource specifications are meant to be read by human developers! There should never be any automatically-generated client code.

REST forces developers to think first about the Big Picture: Which Resources are important? Where are they specified? How can all interested parties get together to specify Resources, so that we all have a common specification upon which to base our individual implementations?

The great thing about Resources is that they don’t change! (Or, rather, they don’t change without involving the people who develop with them.) The most famous Resource in the history of the Internet is the HTML web page. It has gone through 5 versions, but each one is discussed ad-nausea via the W3C’s Requests for Comment (RFC).

The other great thing is that Resources are NOT tied directly to implementation details. (Remember, no auto-generated client code?) Each system is free to build whatever database structure necessary to store Resources in the way they see fit. The main thing is that the Resource is an externally-defined target for ALL implementations to shoot for.

And the Hypermedia Constraint (HATEOAS)? It allows us to navigate cyberspace in an unambiguous way. Because the stars don’t move (much), Polynesian sailors could use star charts millennia ago to navigate among the islands in their archipelago. In REST, the Resource links themselves change, but the link relations are the reliable points for navigation.

(ASIDE: “Second star to the right” would be equivalent to a relative URL.)

Image link credit:
Great site! Go see all the cool galactic pix!

Looking through the wrong end of the telescope

In comparison, both these images have dots. If these images were fuzzed-out a lot and turned into black-on-white, they might even look similar. But trying to wedge REST support into a SOAP service is like looking through the wrong end of the telescope.

I hope this article brings a bit of clarity to the SOAP vs REST discussion. Your friendly comments are welcome.

Starting a Django 1.6 Project the Right Way

February 28, 2014 Leave a comment

I recommend the Django Tutorial to every web developer, even if you’ve been programming Django for years. There’s a lot of good stuff in there that we forget or overlook.

Today, I found this article. I’m going to recommend it as a companion piece to the Django Tutorial. Definitely worth the read time!

Categories: Projects

Why not PHP?

September 26, 2013 2 comments

DISCLAIMER: If you use PHP for building God’s Kingdom, then more power to you. Sometimes you have to do your best with the tool you have been handed. (However, if you have the freedom to swap tools, you should consider trying Python for a future project and see if it’s a better tool.)

My former supervisor, while I was in missions full-time, was adamant about NOT using PHP. I recall him mock-spitting on the floor in distain when asked by a potential recruit about using PHP. At the time, I was shocked!–especially because I was worried that this reaction would turn off the potential recruit from joining missions.

The truth is that anyone seeking to join missions should be willing to work with whatever tools are in use. Also, those who are decision makers should steer the team towards using tools that balance easy-to-learn with fit-to-the-task. Django is the sweet spot for web development, IMHO.

So, as a former PHP fan-boy who now maintains a PHP website, I understand a lot of PHP’s problems. (So, we’re migrating it to Django.) I’ve had pretty strong convictions about this decision for a while now, but this article really cements it. If you’re wondering why NOT to learn/continue to use PHP, read this article:

Categories: Projects Tags: , ,

LXC Quick Start

September 6, 2013 Leave a comment

I use virtual machines often, esp. when the Operating System differs from my host. (Like when I’m stuck on a Windows box and want to code on Ubuntu.)

I use hosted virtual private servers when I need to share them with people via the Internet.

I’d like to use Linux Containers (LXC), because they offer a lighter-weight alternative to virtual machines.

Up til today, LXC has eluded me. Today, I figured it out.

Installing LXC

Qasim says, “One of the main focus for Ubuntu LTS was to make LXC dead easy to use, to achieve this. Creating a basic container and starting it on Ubuntu.”

sudo apt-get install lxc

Ubuntu has made this setup sensible defaults, so you can skip a lot of the configuration file tweaking, unless you really want to dive into that much detail.

Creating a Container

sudo lxc-create -t ubuntu -n my-container

It takes a while to create the container the first time. This is because the process has to download and install (in your container) all the packages it needs–as if it were indeed its own separate OS. It’s supposed to be a lot quicker after that, because all the packages get cached.

Changing root password and setting up user accounts

Brian’s video really helped me here. Basically, you can chroot into your container’s filesystem, then run whatever configuration commands you need to run before you start the container.

In this example, I change the root password in my container and create new user unclebob as a sudo-er.

sudo chroot /var/lib/lxc/my-container/rootfs
adduser unclebob
addgroups unclebob sudo

Starting the container

Brian recommends (wisely) to run your container behind screen, so your terminal session doesn’t get permanently sucked into the container’s black hole.

screen -S session-name sudo lxc-start -n my-container

You can escape the black hole by typing Ctrl-A, d. See my post Screen FTW for more details.

Install the LXC Web Panel

Looks nice. I haven’t played with it much, nor will likely do so, since I now have all the command-line tools I need. But it might come in handy, esp. if I end up having a lot of containers and need to mess with their settings quickly.


Categories: Projects

Get every new post delivered to your Inbox.