REST: A Starting Point for Implementors

March 10, 2014 3 comments

Introduction

REST is difficult.  For starters, Roy Fielding’s dissertation is not for the faint-of-heart. (But if you enjoy academic reading, go for it!) Then, there’s a lot of noise on the Internet, esp. from vendors and SOAP/RPC Web Service developers that mucks up the water.

This article is for developers who are ready to start building with REST.

The point of REST

I wrote a visual analogy about looking into the right end of the telescope. It is critical to get the right perspective before trying to build with REST. (Please read that before continuing.)

ZapThink nails it (in http://www.zapthink.com/2012/01/10/the-api-is-dead-long-live-the-api/):

REST isn’t supposed to make the Web more like system integration; it’s meant to make system integration more like the Web.

So, if we are trying to make our system integration work more like the Web, what do we do first?

Step 1. Build a Web site

Yes, this sounds like advice from the 1990s. But let’s see what happens:

  1. You figure out what kinds of pages you want on your site.
  2. You figure out how to navigate among them.
  3. You decide what domain name you want and build a home page at the root URL.
  4. You style it with CSS and add interactivity with Javascript.

Now, what do you have? A browsable web representation of your pages that satisfies the Hypermedia constraint for any web browser clients. It’s already REST, simply because HTML-over-HTTP implements a subset of REST.

Step 2. Drop the fancy formatting

Your clients probably don’t need CSS or Javascript. (Unless you’re supplying “code-on-demand”, in which case, why are you reading my humble article?) So, drop all references to CSS, javascript and whatever else is there for visual formatting purposes.

If you have a standard page header and footer, you probably want to drop them, too. Side-bars and other junk that’s not related to the page content? Ax them.

DO keep whatever links are there for navigating to related content–even back to the root page (aka, the “entrypoint”)!

DO keep the HTML tags that provide semantic structure, like <h1>, <div>, <caption>, etc. (You get a lot more of these semantic tags with HTML5.)

When you’re done, you should still be able to browse the web site, but it will look as plain-Jane as HTML 1.0. Good!

Step 3. Document page types as distinct Resources

Typically, web sites will have blog entries, shop items, bio items, and stuff like that. Try to figure out what the real meaningful breakdowns are: those will be your Resources.

Then, Document those Resources online–SOMEWHERE ELSE! It can be a github project, a wiki somewhere, or whatever. It should NOT be embedded within your web site content, although you might link to the Resource specifications somehow. (See the video below for an example of using “class” attributes to link to Resource specifications.)

Your Resource Specification documentation should describe the Resource in human-readable (NOT machine-parsable!) prose. The reader needs to be able to understand what the Resource represents, what kinds of links (link relations) it might have, and a rough idea how various Resources might interact. Don’t worry too much about the last part now, because we’ll refine that in later steps.

Don’t forget: Each Resource gets its own Content-Type. This will be critical later, when you want to create a new version of a Resource, or provide a different Representation format for the Resource.

Step 4. Share and solicit feedback

Actually, do this throughout all the steps. If you haven’t started by the end of Step 3, you really need to get feedback starting now.

You want your friends, fellow developers, and pets to have a say at what’s going on in your mind–as expressed in your Resource specifications.

Listen to their feedback and adjust your Resource specifications accordingly.

If you have a big project with lots of people who might comment, you might consider starting some sort of Request for Comment (RFC) process to manage all the feedback.

Step 5. Remove all HTML FORMs

“What?! Then how do clients figure out what fields they can submit?” They don’t. That’s SOAP thinking.

The HTML FORM really is a UI affordance for web browsers, which interact with users. A web browser can’t be expected to know all the site-specific types of forms that can be POSTed to every website in the world, so the application/x-www-form-encoded Content-Type was created. As a consequence, web browsers AND web frameworks are forced to have a ton of code to parse, format and validate HTML FORM data.

Your clients don’t need this. If a client is going to collect data from a user, then the client will have some sort of UI code to do this. But your client does NOT need to parse arbitrary form definitions from the server. Why? Because you control the Resource types that the client can POST to your REST server.

As you yank out each HTML FORM, create a Resource Specification document that describes the schema, content-type, and whatnot. Update other Resource documentation so that your client developers know where they can PUT or POST these new Resources.

(HINT: Save the HTML FORM snippets. You might want to reuse them later. See Afterthought #2.)

You’re Done!

Here’s what you now have:

  1. Proper REST implementation of Resources…
  2. …satisfying the Hypermedia Constraint.
  3. And MOST IMPORTANTLY, documented Resource Specifications.
  4. BONUS: That you can test in a web browser.

Here’s some more you could do:

  1. Help the client developers write their client code according to the Resource Specifications. Make sure they rely on the link relations, and don’t try to construct URLs on their own.
  2. Twiddle with your server and try to use HTTP PUT, DELETE, PATCH and other verbs more effectively. (But, if you have a read-only REST implementation, these don’t matter at all.)
  3. Specify additional Representations for your Resources. Right now, you have a HTML (XHTML, if you were careful) Representations of your Resources. You could add JSON, XML, binary, whatever Representations; however, make sure that you write just as good Specification documentation for these new Representations!
  4. Tweak your server’s HTTP Response Codes. Probably, your web server isn’t sending *quite* the right HTTP Response Codes, esp. for things like 201 CREATED. You should fix that as soon as you get a chance, because real REST clients are going to be looking for that after POSTing a new Resource.
  5. Play with HTTP Caching. You will need to deal with Expires headers and such, but it should make sense if you’ve done any web caching configuration. There’s a lot of good documentation out there.

Video Break

You’ve read this far! Great! Now take a video break and watch this video. It’s the absolute best presentation of how to implement REST that I’ve ever seen.

(I don’t necessarily agree with the way he uses HTML FORMs. But for this demo, it’s great. And maybe I still don’t *get* how he’s using HTML FORMs.)

Afterthought #1. In a perfect world, Resources come first

Building it up this way helps us to stay RESTful, because we’re forced to be browseable. However, the best way to start using REST is to define Resource specifications first. If you have a way to get everyone to think about the Representations before you start, then great! However, in our mixed-up SOAP-sudsy world, you’ll get a lot of non-REST thinking about “web services”. So, pragmatically, it might help your team to think RESTfully if you start with a web site first.

Afterthought #2. For browser-testability, go back and re-add HTML FORMs

So, your client knows about all the Resources from your Specifications? Great! But if you want to use Selenium or another browser-automation tool for testing, you need to help the browser know what to do.

You can re-add those HTML FORMs that you yanked out in Step 5. But don’t add them directly into your Resources; instead, provide a special link relation to a “create-form” and “edit-form” Resource (which will be an HTML page containing an HTML FORM). https://tools.ietf.org/html/rfc6861#section-4.1

If you need to POST content-types other than application/x-www-form-urlencoded (such as application/json or application/xml), you might need to supply some javascript that will convert the FORM values into the appropriate representation format. Mozilla has created the FormData javascript library to help with exactly that.

Conclusion

Implementing REST has been difficult because…well, we weren’t implementing REST. Once you throw away SOAP and get back to plain old web development ideas, REST becomes much easier.

I hope this helps you get started with REST. Please drop me a comment if it does. :)

Categories: Projects Tags: , , , ,

SOAP vs REST: Two Opposing Perspectives

March 10, 2014 2 comments

Introduction

There is a lot of confusion about SOAP and REST. SOAP developers mistakenly think they can support REST simply by using a JSON data format. Other developers want to use REST, because they understand the benefits, but simply don’t know where to start (or worse, they start with SOAP and try to turn it into REST).

SOAP and REST are completely different, incompatible perspectives.

SOAP is a Microscope

SOAP (and all RPC) requires one system to look inside another system. Imagine one system looking through a microscope at the guts of another system.

Because WSDL is easily-parseable, many developers rely on their tools (Eclipse, Visual Studio, IntelliJ, etc.) to generate SOAP libraries for accessing “services” on SOAP servers.

Unfortunately, just like a bacteria culture, SOAP services often change dramatically or even die off entirely without warning. It’s a constant struggle for SOAP clients to keep up with the changes, esp. when accessing multiple SOAP services on multiple SOAP servers.

Staphylococcus aureus

REST is a Telescope

REST requires systems to focus on shared Resource specifications that exist external to themselves. Imagine a group of peer systems standing on the same side of a telescope, taking turns looking up at the stars.

There is no need for machine-parsable WADL-like specifications, because Resource specifications are meant to be read by human developers! There should never be any automatically-generated client code.

REST forces developers to think first about the Big Picture: Which Resources are important? Where are they specified? How can all interested parties get together to specify Resources, so that we all have a common specification upon which to base our individual implementations?

The great thing about Resources is that they don’t change! (Or, rather, they don’t change without involving the people who develop with them.) The most famous Resource in the history of the Internet is the HTML web page. It has gone through 5 versions, but each one is discussed ad-nausea via the W3C’s Requests for Comment (RFC).

The other great thing is that Resources are NOT tied directly to implementation details. (Remember, no auto-generated client code?) Each system is free to build whatever database structure necessary to store Resources in the way they see fit. The main thing is that the Resource is an externally-defined target for ALL implementations to shoot for.

And the Hypermedia Constraint (HATEOAS)? It allows us to navigate cyberspace in an unambiguous way. Because the stars don’t move (much), Polynesian sailors could use star charts millennia ago to navigate among the islands in their archipelago. In REST, the Resource links themselves change, but the link relations are the reliable points for navigation.

(ASIDE: “Second star to the right” would be equivalent to a relative URL.)

Image link credit: Phys.org
Great site! Go see all the cool galactic pix!

Looking through the wrong end of the telescope

In comparison, both these images have dots. If these images were fuzzed-out a lot and turned into black-on-white, they might even look similar. But trying to wedge REST support into a SOAP service is like looking through the wrong end of the telescope.

I hope this article brings a bit of clarity to the SOAP vs REST discussion. Your friendly comments are welcome.

Starting a Django 1.6 Project the Right Way

February 28, 2014 Leave a comment

I recommend the Django Tutorial to every web developer, even if you’ve been programming Django for years. There’s a lot of good stuff in there that we forget or overlook.

Today, I found this article. I’m going to recommend it as a companion piece to the Django Tutorial. Definitely worth the read time!

http://www.jeffknupp.com/blog/2013/12/18/starting-a-django-16-project-the-right-way/

Categories: Projects

Why not PHP?

September 26, 2013 2 comments

DISCLAIMER: If you use PHP for building God’s Kingdom, then more power to you. Sometimes you have to do your best with the tool you have been handed. (However, if you have the freedom to swap tools, you should consider trying Python for a future project and see if it’s a better tool.)

My former supervisor, while I was in missions full-time, was adamant about NOT using PHP. I recall him mock-spitting on the floor in distain when asked by a potential recruit about using PHP. At the time, I was shocked!–especially because I was worried that this reaction would turn off the potential recruit from joining missions.

The truth is that anyone seeking to join missions should be willing to work with whatever tools are in use. Also, those who are decision makers should steer the team towards using tools that balance easy-to-learn with fit-to-the-task. Django is the sweet spot for web development, IMHO.

So, as a former PHP fan-boy who now maintains a PHP website, I understand a lot of PHP’s problems. (So, we’re migrating it to Django.) I’ve had pretty strong convictions about this decision for a while now, but this article really cements it. If you’re wondering why NOT to learn/continue to use PHP, read this article:

http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/

Categories: Projects Tags: , ,

LXC Quick Start

September 6, 2013 Leave a comment

I use virtual machines often, esp. when the Operating System differs from my host. (Like when I’m stuck on a Windows box and want to code on Ubuntu.)

I use hosted virtual private servers when I need to share them with people via the Internet.

I’d like to use Linux Containers (LXC), because they offer a lighter-weight alternative to virtual machines.

Up til today, LXC has eluded me. Today, I figured it out.

Installing LXC

Qasim says, “One of the main focus for Ubuntu LTS was to make LXC dead easy to use, to achieve this. Creating a basic container and starting it on Ubuntu.”

sudo apt-get install lxc

Ubuntu has made this setup sensible defaults, so you can skip a lot of the configuration file tweaking, unless you really want to dive into that much detail.

Creating a Container

sudo lxc-create -t ubuntu -n my-container

It takes a while to create the container the first time. This is because the process has to download and install (in your container) all the packages it needs–as if it were indeed its own separate OS. It’s supposed to be a lot quicker after that, because all the packages get cached.

Changing root password and setting up user accounts

Brian’s video really helped me here. Basically, you can chroot into your container’s filesystem, then run whatever configuration commands you need to run before you start the container.

In this example, I change the root password in my container and create new user unclebob as a sudo-er.

sudo chroot /var/lib/lxc/my-container/rootfs
passwd
adduser unclebob
addgroups unclebob sudo
exit

Starting the container

Brian recommends (wisely) to run your container behind screen, so your terminal session doesn’t get permanently sucked into the container’s black hole.

screen -S session-name sudo lxc-start -n my-container

You can escape the black hole by typing Ctrl-A, d. See my post Screen FTW for more details.

Install the LXC Web Panel

Looks nice. I haven’t played with it much, nor will likely do so, since I now have all the command-line tools I need. But it might come in handy, esp. if I end up having a lot of containers and need to mess with their settings quickly.

References

Categories: Projects

Dance of the Planets

July 17, 2013 1 comment

I saw a fun article about planetary orbits at http://ensign.editme.com/t43dances.

The author includes the source code at the bottom of the article, but I don’t have Pascal. So, I ported it to Python. Have fun! :)

[Edit: WordPress doesn't properly format Python code, so I shared it as a Pastie here: http://pastie.org/8151072]

Categories: Projects

Removing OpenDNS nameservers from Linux Mint

July 16, 2013 1 comment

I was shocked today to find out that Linux Mint defaults to OpenDNS nameservers. The technical reason makes sense, but the hush-hush manner in which it was done does not. Sure, they mentioned it in the release announcement, but IMHO it should be much more obvious than that.

I was really shocked when I found that OpenDNS was presenting its certificates when I was trying to access https://dropbox.com. This smacked of a Man-In-The-Middle attack, which lead me to this discussion of OpenDNS in Mint.

The last post there makes in clear that Mint inserts OpenDNS nameservers in the default /etc/resolv.conf and /etc/resolveconf/resolve.conf.d/tail files. So, I commented them out.

Then, just to be safe, I restarted the networking service. Bad idea! My windows reverted to bare X-windows or something strange. After restarting, I restored my “Cinnamon” session and everything came back. (Whew! Now I have to go fix my wife’s box, too.)

So, you do need nameservers, and fallback nameservers. Here’s what I put in those two files, since I’m on Windstream, and I like OpenNIC and Google better than OpenDNS right now.

# Windstream clean DNS servers first.
nameserver 166.102.165.32
nameserver 207.91.5.32

# OpenNIC as first fallback.
nameserver 184.154.13.186
nameserver 216.167.252.195

# Google, as second fallback.
nameserver 8.8.8.8
nameserver 8.8.4.4

I put Google last, because it’s a redirect-type DNS server. If it doesn’t find what you’re looking for, it sends you to a Google search for it. This is normally OK, but not if you’re dealing with sensitive internal server names. Hence, it’s last.

Categories: Projects
Follow

Get every new post delivered to your Inbox.