workhere: Quick environment setup for Linux command-lines, Python and Node

April 13, 2017 Leave a comment

Python’s virtualenv-wrapper is amazing. Use it whenever possible.

Still, there are times when you’re on a system over which you don’t control all the things. For example, I am working in a project that uses make in some odd ways and puts binaries in buried directories. It does, however, use Python’s virtualenv and the Node.js Package Manager yarn.

Per standard practice, the Python virtualenv is named venv. Consequently, the binaries for Python’s virtualenv get buried under venv/bin.

Also per standard practice, the Node.js binaries get buried under node_modules/.bin.

I wanted one command to activate my Python virtualenv and add all those binaries to the path. Here’s what I came up with:

alias workhere='export PATH=venv/bin:node_modules/.bin:$PATH && \
                source venv/bin/activate'

I put that in my .bashrc file, so that it is available to any shell I open.

That way, I simply have to cd /path/to/project and then type workhere.

Problem solved.

Categories: Projects

Why Git

December 15, 2016 Leave a comment

A coworker asked me why my team uses Git. He was interested also in a comparison with Subversion, which his team uses. Here’s my response.

Read more…

Categories: Projects

SSHFS: Hacking Linux files from a Mac

December 14, 2016 Leave a comment

I’m a Linux fanboy. Used to love Microsoft, now…not so much. Used to hate Mac, now…well, I love their hardware, and OSX sure has a pretty face.

Anyway, these days, I spend a ton of time with OSX, even though I prefer developing on a Linux Desktop in a VM. But there are times it would be nice to spin up a new VM and mess with its files using some tools installed on the OSX host.

Today I found out how.

Install SSHFS

We’re just going to use the same trick that is available to Linux users, but we have to jump through some hoops to get it on Mac.

Helio Tejedor’s article has a ton of detail about installation, but I found the most relevant bit is this:

 

The easy way to install SSHFS is navigate to http://osxfuse.github.io and download two files:

  • OSXFUSE 2.7.3
  • SSHFS 2.5.0

Do that, but just pick the latest two versions listing right on their homepage.

Manually test it out

Digital Ocean’s article is, like most Digital Ocean articles, excellent documentation. The section “Mounting the Remote File System” has a detailed explanation about how to mount remote drives using SSHFS. I just needed these two lines for my test VM (at IP address 192.168.56.101).

sudo mkdir /mnt/192.168.56.101
sudo sshfs -o allow_other,defer_permissions user@192.168.56.101:/ /mnt/192.168.56.101

The observant reader will notice that:

  • The user is the name of the SSH account that can connect to the box.
  • I am “cheating” by using the IP address for the folder name…
  • …which may have unexpected results when my VM gets a new DHCP lease.

I’m good with all that. It works!

Script it

Mounting, unmounting, making directories…boring. I scripted this as mountvm, and now I have a simple one-liner to mount/unmount/etc. that mimics ssh.

Here’s the magic sauce, for your discerning palette.

#!/bin/bash
user=$1
host=$2
sudo umount /mnt/$host
sudo mkdir -p /mnt/$host
sudo sshfs -o allow_other,defer_permissions $user@$host:/ /mnt/$host

Merry Christmas!

Categories: Projects

Product Management and Development Ten Commandments

December 8, 2016 Leave a comment

Jim Montagnino shared these principles with me recently. With his permission, I am posting them here.

  1. KISS – Keep It Simple Stupid, Less is always better
  2. Make it “Idiot Proof” – a user or admin should not be able to break it
  3. Always keep in mind our customer’s ability to consume what we can build
  4. Keep security both internal and external in the forefront
  5. Configuration – not coding
  6. Leverage inherent Platform Capabilities whenever possible – even at the expense of functionality (go back to first bullet)
  7. Phased approach – don’t boil the ocean on each release
  8. Keep upgradeability in the forefront
  9. Listen to our customer feedback and stay in tune with the market
  10. Sizzle Sells – consider POC & Special Projects to help drive sales

I have a wild ambition to expound on each point, but I will save that for separate posts.

Thanks, Jim!

Categories: Business, Projects

The Future of JavaScript is X

June 4, 2015 1 comment

JavaScript in 2015

JavaScript has gotten way better than it was originally. The Good Parts actually make JavaScript a pleasure to program (and much closer to Python, IMHO). Firebug and other in-browser debuggers make it possible develop large-scale projects without losing your mind.

Browser improvements and Node.js have expanded the speed and reach of JavaScript. Jeff Atwood postulates that

any application that can be written in JavaScript, will eventually be written in JavaScript

Node.js is especially compelling, because it opens up the idea that all software might eventually run with JavaScript, be it in the browser or on the server. Write once, run anywhere? Maybe. But maybe not.

The Problem with JavaScript

JavaScript in the browser has one huge gaping hole: It must be sent in the clear. For a browser to run it, this code must be valid JavaScript. Sure, you can encode it and decode it on the fly, but the decoder has to be written in JavaScript. This means that any sufficiently-diligent programmer can reverse-engineer any JavaScript code. There is no way to hide implementation details using JavaScript.

Web APIs to the rescue?

Enter Web APIs. Here, we separate out business logic and other implementation details from the front-end JavaScript. We place them in back-end server code, be it Node.js, Python, Ruby or COBOL (yikes!). No browser ever receives the code that runs on the server, so that’s great, right?

Actually, no. Web APIs do not protect a sufficiently-diligent programmer from documenting your process. In fact, many Web APIs are publicly available, which encourages use. And that’s good. But it does not protect your process.

In reality, Web APIs just complicate things. If your front-end has to call your back-end, then you must implement logic to parse/emit data structures that can be sent across the wire. If you’re using Node.js cleverly, you can implement that once and use it in both places. (Browserify is your helper, here.) But if you are in a heterogeneous environment, you may have servers running Python, .Net, J2EE (shivver!) and Perl. Each of these must also implement parse/emit logic. (“Write once, run anywhere” just went out the window!) Keeping all these in-sync (esp. with versioning in the URL!) is a nightmare.

Chapter 5 REST to the rescue?

Chapter 5 REST would help here, esp. with anarchic scalability. But many developers don’t understand Chapter 5 REST, and fewer are willing to collaborate at that higher level. This is why Web APIs have taken off: it’s low barrier to entry, and you don’t actually need to think about how your own application might be a subset of a broader (global) problem space. (And that’s all I’m going to say about Chapter 5 REST in this article.)

Browser plug-ins to the rescue?

Flash used to be the de-facto way to get closed content to the browser. It’s old news by now that Apple iOS doesn’t run Flash. Microsoft at one time wanted to do the same with its browser…but now they support Flash by default. I’m not sure how I feel about that.

You could install another binary plug-in, but installing browser plug-ins is an administrator’s nightmare. Enterprise security policies may also allow BYOD, but then you have to install the plug-in on devices outside your enterprise control. Not fun.

The original browser plug-in was Java. I don’t write many Java applets these days. Do you?

JavaScript’s replacement

So, the killer of JavaScript is its readability (both machine and human). If you want to protect your implementation details, then you must expose as little JavaScript as possible to the browser. Two relatively new technologies (2011 and 2004) have emerged that enable extremely-thin JavaScript browser clients: websockets and Canvas.

Wait, wait! Couldn’t we do this with AJAX and images?

Well, maybe. But not well enough.

First, AJAX really doesn’t allow the server to send to the browser directly. Several tricks (including HTTP Long Polling and others) have sort-of made that possible. But it hasn’t been really good enough to support massive server-initiated redrawing.

Second, messing with images messes with the DOM. Anytime you mess with the DOM, the browser tends to get really slow. Canvas allows much faster updates that do not impact the DOM.

Websockets and Canvas to the rescue!

It is almost dumb-simple to setup a websocket connection in a browser with JavaScript. The hard part is building the parse/emit logic and then the code that processes incoming messages. Out-going messages are trivially simple–just a single function call. If you get rid of the hard part, then you’re left with a bare-bones, lightweight client. Here’s how.

Don’t implement application-specific message processing in your browser app. Not one bit. Just implement code that takes the server commands and renders them on the Canvas. You could (if you’re lazy and inside a trusted network with no fear of Bad Actors) let the server send actual JavaScript code and eval it client-side. That would, however, be evil.

If you’re a bit more diligent, you still can build minimal code to process rendering commands on Canvas.

You’ll also need to send back keyboard and mouse events to the server. But that’s a very tiny bit of info, namely the (x, y) coordinate, which mouse button and/or key was pressed. The server can process that and stream more rendering commands back via the websocket.

What about multimedia?

Any technology gets harder with multimedia. The same is true here. I’m not going to focus much on that now. However, take a look at WebRTC for peer-to-peer media connections and streams. Just overlay a Canvas on top of a element, and you’re still able to control the video with the least amount of front-end code. Or, if you have tons of server resources (like a Media Server farm), then you could conceivably proxy all the media streams and issue image updates via websockets to Canvas. (That way lies madness, methinks.)

Isn’t this more like VNC?

Maybe. It’s a spectrum that depends on how lightweight your client code is. On the one end, you could send partial images and blit them where needed (using drawImage)–this would be like VNC. On the other end, you could implement higher-level functionality with additional Canvas methods–this would be more like the X Window System.

What about SVG?

I’m no expert, but you could replace Canvas with SVG. It’s a different beast, but the same principal applies, and implementations fall along the same spectrum from VNC-like to X-like.

Really, now…why would anyone do this?

Companies like to protect their intellectual property. Arguably, a JavaScript front-end is protected by technology (HTTPS) and copyright law, but there are some for whom these protections simply are insufficient. They already use similar technologies like VNC, Remote Desktop Protocol and exported X-Window sessions. This websockets/Canvas solution has one major advantage over those: it will run in the browser without any installation…as long as JavaScript is turned on.

 

Categories: Projects

The RESTaurant Analogy

October 21, 2014 Leave a comment

The RESTaurant Analogy

Try, try again

Real REST is hard…but only because we bring a lot of preconceptions to it. I’ve tried before to explain it as a telescope (for people who are used to microscopes). I’m hoping this analogy will be easier, because it’s one place where the real world works much like a RESTful client-server interaction.

CAVEAT: This example is a bit contrived. I hope you don’t use HTTP with a real waitress.

ASIDE: Customers, Chefs and Waitresses are fabulous individuals. This article intends nothing but respect for all people.

A visit to a RESTaurant

You walk into a short-order restaurant one morning. The hostess leads you to a table and you sit down. The waitress immediately arrives. Since you’re hungry and already know what you want, you ask the waitress politely for it. She scribbles some notes, smiles and walks away.

You open the daily paper you brought with you and start reading. After a few minutes, the waitress returns with your plate of food. Since you’re so engrossed in reading, she sets the plate down on the table.

When you hear the clink of the plate on the table, you look down and see your plate. It’s exactly what you ordered, so you dig in. Yum!

The same visit, expressed in HTTP

All the examples use HTTP Status Codes. You should keep them handy for reference. In fact, you should memorize them, they are that important.

From the Customer, to the Hostess:

GET /table?for=1 HTTP/1.1
Accept: table

From the Hostess, to the Customer:

302 Found
Content-Type: table
Content-Location: /table/13
"Corner booth, by the jukebox."

The act of the Customer sitting down is similar to this:

GET /table/13 HTTP/1.1
Accept: table

(I’m ignoring the server response here, but it would return a Table Resource.)

From the Customer, to the Waitress:

POST /order/please/ HTTP/1.1
Accept: plate;q=1.0,wait;q=0.9
Content-Type: OrderForFood
"Two eggs over easy, with corned beef hash."

(We’ll go back to those “q” values in a little bit.)

Response from Waitress:

202 Accepted
Content-Type: wait
"I'll be right back, sweetie."

From the Waitress, to the Chef:

POST /plate/with/food/ HTTP/1.1
Accept: plate;q=1.0,wait;q=0.9,slot;q=0.8
Content-Type: OrderForFood
"Two eggs over easy, with corned beef hash."

Response from the Chef:

202 Accepted
Content-Type: slot
Content-Location: /slot/5
"I'll put it on the counter in slot #5."

Now, periodically, the Waitress glances at Slot 5 to see if the plate is ready. (This is “polling”.)

The Waitress checks the Slot like this:

GET /slot/5 HTTP/1.1
Accept: plate

And the Slot responds this way when it’s empty:

204 No Content

The Slot response this way when the order has been cooked and put on a Plate here:

200 OK
Content-Type: plate
"Two eggs over easy, with corned beef hash, grits and toast."

NOTICE: The Plate Resource contained grits and toast, even though the Customer’s OrderForFood Resource didn’t explicity request it. I guess you always get grits and toast at this Restaurant, whether you ask for them or not.

When it is, she goes and gets it and brings it back to the Customer. The Customer hears the clink of the plate on the table and sees his order.

There are two ways to accomplish this: Server-Server Notifications or Client-Server Polling.

Server-Server Notifications

One would be for the Customer to have a “web callback” method that the Waitress uses. (This is sometimes used in a Server-Server environment, or with websockets.)

The Waitress would notify the Customer this way:

POST http://customer.server.tld/hey/you/ HTTP/1.1
Content-Type: plate
"Two eggs over easy, with corned beef hash, grits and toast."

In this case, the Customer would acknowledge receipt:

202 Accepted

Client-Server Polling

The other would be for the Waitress to use the shared Table Resource, which the Customer polls. (This is more likely in the Client-Server environment of the Web these days.) It goes like this:

The Waitress PUTs the Plate on the Table:

POST /table/13 HTTP/1.1
Content-Type: plate
"Two eggs over easy, with corned beef hash, grits and toast."

The Table then has the Plate on it, as it shows in its new Representation:

202 Accepted
Content-Type: table
"Napkins, Utensils, and Plate"

Then, the Customer can see what’s on the Table:

GET /table/13 HTTP/1.1
Accept: table

And the Table’s response is the same, except it’s a “200 OK” status code this time.

Coffee and Content Types

Throughout your meal, you’re probably going to drink lots of coffee. Let’s presume that your Waitress likes to be asked, all the same.

The Customer asks for a mug of coffee:

GET /coffee HTTP/1.1
Accept: mug

The Waitress is able to fulfill your request immediately, thanks to clean mugs stacked nearby and the coffee pot she carries around (almost) all of the time.

200 OK
Content-Type: mug
"full of caffeinated, hot coffee"

“Wait a minute!” you say. “I drink iced coffee.” No problem. Let’s just ask for a glass instead of a mug:

GET /coffee HTTP/1.1
Accept: glass

Well, now, you can’t pour hot coffee into a glass. So, your Waitress (who happens to also carry a pitcher of fresh iced coffee) is also able to fulfill this request immediately:

200 OK
Content-Type: glass
"full of caffeinated, iced coffee"

The point is this: Coffee is coffee. But it comes in many forms.

In REST, this is the same as saying: A Resource is a Resource, but it has many different Representations. You tell the server what Representation you want, and let it do the best it can to fulfill your request.

The reality is that sometimes, a server can’t give you what you want.

That’s kind of like when the Waitress runs out of hot coffee. But she still has iced coffee. And on this particular day, she’s not feeling like making a fresh pot…so, you’re just gonna get iced coffee. Her response then is:

406 Not Acceptable
Content-Type: glass
"full of caffeinated, iced coffee. Just pretend it's hot, sweetie!"

That 406 Not Acceptable status code means, “Sorry, I don’t have what you asked for, but here’s something else. I hope you like it…even though you didn’t ask for it.”

How do we know what a Waitress does?

If you’ve never been to a Restaurant before, then you’ll have to be taught. This is “out-of-band” information–that is to say, you won’t find instructions at the Restaurant about how to interact with a Waitress. The best you’ll get from the Waitress herself is a Menu–which also contains no instructions on how to use it. (This Menu is actually hinting at another REST constraint: Hypertext As The Engine of Application State. But we’re not getting into HATEOAS in this post.)

In REST, there is no “Web Application Description Language,” despite the so-called WADL “standard.” Rather, there are widely-agreed upon specifications for Resources. These specifications are created in the open, with a global audience, because we want input from everyone who knows about these Resources. The specifications themselves are published in a human-friendly document, which is not meant to be parsed by a program. The goal of Resource Specification Documentation is to educate the programmer, who will then write programs that are based on an understanding of how the Resource works.

In short: By eliminating machine-generated libraries, we actually simplify the programming of web clients and servers.

Versioning, or how we even got iced coffee

Back in the day, Waitresses only served hot coffee. So, if you made a generic request for coffee, you always got a mug.

Then, on a hot day, someone decided to dump ice into his coffee mug…voila! Iced coffee was born.

In our Restaurant, the Waitress knows about iced coffee and can respond appropriately if you ask for a “glass” of coffee. But, in some Restaurants, their Waitresses don’t serve iced coffee. The end result could be similar to when our Waitress ran out of hot coffee:

The Customer asks:

GET /coffee HTTP/1.1
Accept: glass

And the Waitress responds:

406 Not Acceptable
Content-Type: mug
"full of caffeinated, hot coffee. Dearie, we don't serve that fancy stuff here!"

In this way, you can always try to ask for newer Representations of a Resource, even if the server might not support them.

This is how REST supports versioning. Older servers will still support the older Representations, just like all (OK, most) Waitresses will be able to pour you a hot mug of coffee. Newer servers will support additional Representations, and it’s safe to ask for the new ones, but be prepared to handle the older ones, just in case.

The “quality” indicator in the “Accept” header allows the Client to put a preference on what it would like to receive in a Response. If, for example, the Customer would prefer iced coffee, but would be OK with hot coffee, or whatever, this is how his Request would look:

GET /coffee HTTP/1.1
Accept: glass;q=1.0,mug;q=0.9,*/*;q=0.1

ASIDE: This feels very backward to me, because English uses the semi-colon (;) and comma (,) in exactly the opposite way.

The Server would then pick the Representation that it can render that matches the highest “q” value. The “*/*” means “whatever”. If it can’t satisfy the “Accept” header, it sends back whatever it can render, with a “406 Not Acceptable.”

This is also how Server Upgrades can happen without forced Client Upgrades. This “anarchic scalability” is what has allowed the Web to grow fast, despite the Browser Wars, and all the incompatibilities of lesser browsers.

A Waitress is a Woman, too

So, my young son understands how to ask a Waitress for an Order. But one day, he may also recognize some special Waitress as an attractive woman, as well as just the friendly person who works at the Restaurant. Since I’m teaching him how to be respectful and responsible, he may some day know how to politely and courteously ask this Woman for a date.

Without trivializing such an important step for a young Man, this is how it might look in HTTP:

GET /date?when=Friday&where=ShoppingMall
Accept: date;q=1.0,maybe;q=0.9

Obviously, he would like to receive a confirmation of his Date request (hence, the “q=1.0”). But, he’d be OK with a Maybe.

Now, let’s imagine that the young Woman knows that she must ask permission from her Parents before accepting a Date. In this case, she would probably respond with:

401 Unauthorized
WWW-Authenticate: "You need to ask my dad first"

If he gets up the courage to ask her Father for permission, and her Father grants it, he can again issue his original request, this time with Authorization:

GET /date?when=Friday&where=ShoppingMall
Accept: date;q=1.0,maybe;q=0.9
Authorization: "Your dad said it was OK to ask you out."

Pause a moment. Consider that my young son has no idea about Dating yet. He can still interact with the Waitress. At the same time, more mature Clients know other additional ways to interact with the same Woman. This is another example of “anarchic scalability”–the same Server can interact with multiple Clients with older and newer functionality.

Just don’t ask a married Woman for a Date. You should expect her to reply with a “400 Bad Request” status, which means “Don’t ask again.” (Not to mention what her Husband will do, if he hears about it.)

I see, said the blind man

People often have two reactions to clear REST analogies:
1. “What? I’m not doing REST already?!”
2. “Oh, I get it! REST is JSON!”

Both are wrong. But it’s hard to blame them, because large companies (Microsoft and IBM) have been peddling SOAP/RPC solutions as “REST” so long, that almost everyone thinks they’re doing Real REST.

It’s not easy peeling off a bandaid. But bear with me, because you’ll feel better once you get that wound opened to the clean air.

A visit to a SOAP factory

The SOAP factory is different from a restaurant. It’s purpose is to make soap. And if you know the right stuff, you can make your own soap any way you like.

ASIDE: I don’t enjoy this part, so I’m going to be brief.

No Waitress, Just Instructions

When you walk into the SOAP factory, no one greets you. Instead, there’s a big sign on the wall entitled “Web Service Description Language.” But you have to know which wall it’s on first.

Once you get to the WSDL wall, you can read it for more instructions. It’s not really like a restaurant Menu, because it doesn’t tell you what you can get out of the SOAP factory. Rather, it’s like a “How To” manual: it tells you how to perform activities at the SOAP factory. Unfortunately, it often gives you no clue about the order in which these steps should be performed.

So, to make some SOAP, you have to…do what? How about that “MixIngredients” instruction? Well, you try that, and you do get back something. But it’s not what you expect. Instead, it’s some strange looking plastic egg. If you crack it open, out falls a ticker-tape message:

{
    "MixIngredientsResponse": {
        "err": {
            "forgot": "ingredients"
            "code": "5016d6fb854f46a645868f1d4f58d4dc"
        },
        "data": null
    }
}

Hmm. OK. “So, I need to find some ingredients,” you say. What about that “ListIngredients” instruction? This time, you’re lucky. The egg you get back has inside it this ticker-tape message:

{
    "ListIngredientsResponse": {
        "err": {
            "code": "0",
        },
        "data": {
            "ingredients": {
                "ingredientsList": [
                    "lye",
                    "scent",
                    "fat",
                    "ash",
                    "bleach"
                ]
            }
        }
    }
}

And, based on this message, you start to hope.

So, after a hard day’s work at the SOAP factory, you’ve built a program that properly uses the SOAP Factory API (Application Programming Interface). Congratulations!

Now, in how many other factories can you use this program? Short answer: None. Zip. Zilch. The SOAP Factory instructions are unique to that particular Server. You could try it somewhere else, but good luck. The sad truth is that WSDLs are not built by a global community of people and are simply not standardized.

Worse, they depend heavily on the implementation. Changes internal to the Server often result in breaking changes to the API. SOAP has no way to deal with Content Negotiation, like REST does. As a result, when the API changes, Clients are forced to upgrade…or die.

So, use your shiny SOAP program today. But don’t expect it to work tomorrow.

SOAP in the Restaurant

Let’s go back to our Restaurant analogy to wrap this up.

Using SOAP on the Web is like going into a Restaurant and telling the Chef how to make the food. Instead of placing your order with the Waitress, you write down an exact list of tasks for the Chef to perform: “Turn on oven to 400 degrees. Open bacon. Place bacon on cookie sheet. Put cookie sheet in oven for 30 minutes. Take cookie sheet out. Put bacon on plate.”

It would be insulting to the Chef, and you’d probably be laughed out of the Restaurant. Nobody orders food this way. So…why are we still trying to build Web Apps using SOAP?

Conclusion

Real REST is hard. But it’s easier than anything else in the long run.

I challenge you, my fellow programmer, as well as the global community of developers: Let’s get together and create Resource Specifications about our specific Domains of Knowledge. Let’s create reusable, understandable documentation that explains what we mean. Let’s stop worrying so much about how it’s done, and focus on solid, reliable information Resources. (And please, stop saying REST in the same breath with JSON and CRUD.)

See you at the RESTaurant. 🙂

Categories: Projects

REST: A Starting Point for Implementors

March 10, 2014 3 comments

Introduction

REST is difficult.  For starters, Roy Fielding’s dissertation is not for the faint-of-heart. (But if you enjoy academic reading, go for it!) Then, there’s a lot of noise on the Internet, esp. from vendors and SOAP/RPC Web Service developers that mucks up the water.

This article is for developers who are ready to start building with REST.

The point of REST

I wrote a visual analogy about looking into the right end of the telescope. It is critical to get the right perspective before trying to build with REST. (Please read that before continuing.)

ZapThink nails it (in http://www.zapthink.com/2012/01/10/the-api-is-dead-long-live-the-api/):

REST isn’t supposed to make the Web more like system integration; it’s meant to make system integration more like the Web.

So, if we are trying to make our system integration work more like the Web, what do we do first?

Step 1. Build a Web site

Yes, this sounds like advice from the 1990s. But let’s see what happens:

  1. You figure out what kinds of pages you want on your site.
  2. You figure out how to navigate among them.
  3. You decide what domain name you want and build a home page at the root URL.
  4. You style it with CSS and add interactivity with Javascript.

Now, what do you have? A browsable web representation of your pages that satisfies the Hypermedia constraint for any web browser clients. It’s already REST, simply because HTML-over-HTTP implements a subset of REST.

Step 2. Drop the fancy formatting

Your clients probably don’t need CSS or Javascript. (Unless you’re supplying “code-on-demand”, in which case, why are you reading my humble article?) So, drop all references to CSS, javascript and whatever else is there for visual formatting purposes.

If you have a standard page header and footer, you probably want to drop them, too. Side-bars and other junk that’s not related to the page content? Ax them.

DO keep whatever links are there for navigating to related content–even back to the root page (aka, the “entrypoint”)!

DO keep the HTML tags that provide semantic structure, like <h1>, <div>, <caption>, etc. (You get a lot more of these semantic tags with HTML5.)

When you’re done, you should still be able to browse the web site, but it will look as plain-Jane as HTML 1.0. Good!

Step 3. Document page types as distinct Resources

Typically, web sites will have blog entries, shop items, bio items, and stuff like that. Try to figure out what the real meaningful breakdowns are: those will be your Resources.

Then, Document those Resources online–SOMEWHERE ELSE! It can be a github project, a wiki somewhere, or whatever. It should NOT be embedded within your web site content, although you might link to the Resource specifications somehow. (See the video below for an example of using “class” attributes to link to Resource specifications.)

Your Resource Specification documentation should describe the Resource in human-readable (NOT machine-parsable!) prose. The reader needs to be able to understand what the Resource represents, what kinds of links (link relations) it might have, and a rough idea how various Resources might interact. Don’t worry too much about the last part now, because we’ll refine that in later steps.

Don’t forget: Each Resource gets its own Content-Type. This will be critical later, when you want to create a new version of a Resource, or provide a different Representation format for the Resource.

Step 4. Share and solicit feedback

Actually, do this throughout all the steps. If you haven’t started by the end of Step 3, you really need to get feedback starting now.

You want your friends, fellow developers, and pets to have a say at what’s going on in your mind–as expressed in your Resource specifications.

Listen to their feedback and adjust your Resource specifications accordingly.

If you have a big project with lots of people who might comment, you might consider starting some sort of Request for Comment (RFC) process to manage all the feedback.

Step 5. Remove all HTML FORMs

“What?! Then how do clients figure out what fields they can submit?” They don’t. That’s SOAP thinking.

The HTML FORM really is a UI affordance for web browsers, which interact with users. A web browser can’t be expected to know all the site-specific types of forms that can be POSTed to every website in the world, so the application/x-www-form-encoded Content-Type was created. As a consequence, web browsers AND web frameworks are forced to have a ton of code to parse, format and validate HTML FORM data.

Your clients don’t need this. If a client is going to collect data from a user, then the client will have some sort of UI code to do this. But your client does NOT need to parse arbitrary form definitions from the server. Why? Because you control the Resource types that the client can POST to your REST server.

As you yank out each HTML FORM, create a Resource Specification document that describes the schema, content-type, and whatnot. Update other Resource documentation so that your client developers know where they can PUT or POST these new Resources.

(HINT: Save the HTML FORM snippets. You might want to reuse them later. See Afterthought #2.)

You’re Done!

Here’s what you now have:

  1. Proper REST implementation of Resources…
  2. …satisfying the Hypermedia Constraint.
  3. And MOST IMPORTANTLY, documented Resource Specifications.
  4. BONUS: That you can test in a web browser.

Here’s some more you could do:

  1. Help the client developers write their client code according to the Resource Specifications. Make sure they rely on the link relations, and don’t try to construct URLs on their own.
  2. Twiddle with your server and try to use HTTP PUT, DELETE, PATCH and other verbs more effectively. (But, if you have a read-only REST implementation, these don’t matter at all.)
  3. Specify additional Representations for your Resources. Right now, you have a HTML (XHTML, if you were careful) Representations of your Resources. You could add JSON, XML, binary, whatever Representations; however, make sure that you write just as good Specification documentation for these new Representations!
  4. Tweak your server’s HTTP Response Codes. Probably, your web server isn’t sending *quite* the right HTTP Response Codes, esp. for things like 201 CREATED. You should fix that as soon as you get a chance, because real REST clients are going to be looking for that after POSTing a new Resource.
  5. Play with HTTP Caching. You will need to deal with Expires headers and such, but it should make sense if you’ve done any web caching configuration. There’s a lot of good documentation out there.

Video Break

You’ve read this far! Great! Now take a video break and watch this video. It’s the absolute best presentation of how to implement REST that I’ve ever seen.

(I don’t necessarily agree with the way he uses HTML FORMs. But for this demo, it’s great. And maybe I still don’t *get* how he’s using HTML FORMs.)

Afterthought #1. In a perfect world, Resources come first

Building it up this way helps us to stay RESTful, because we’re forced to be browseable. However, the best way to start using REST is to define Resource specifications first. If you have a way to get everyone to think about the Representations before you start, then great! However, in our mixed-up SOAP-sudsy world, you’ll get a lot of non-REST thinking about “web services”. So, pragmatically, it might help your team to think RESTfully if you start with a web site first.

Afterthought #2. For browser-testability, go back and re-add HTML FORMs

So, your client knows about all the Resources from your Specifications? Great! But if you want to use Selenium or another browser-automation tool for testing, you need to help the browser know what to do.

You can re-add those HTML FORMs that you yanked out in Step 5. But don’t add them directly into your Resources; instead, provide a special link relation to a “create-form” and “edit-form” Resource (which will be an HTML page containing an HTML FORM). https://tools.ietf.org/html/rfc6861#section-4.1

If you need to POST content-types other than application/x-www-form-urlencoded (such as application/json or application/xml), you might need to supply some javascript that will convert the FORM values into the appropriate representation format. Mozilla has created the FormData javascript library to help with exactly that.

Conclusion

Implementing REST has been difficult because…well, we weren’t implementing REST. Once you throw away SOAP and get back to plain old web development ideas, REST becomes much easier.

I hope this helps you get started with REST. Please drop me a comment if it does. 🙂

Categories: Projects Tags: , , , ,