Web app engine project

[Marek]

PBBGs aren't really websites. They're applications.

In the classical client-server model, the client provides a user interface and persistent state. The server responds to requests from the client, such as when a command is issued, and accesses or modifies the underlying database.

Web browsers do precisely that. They act as the client in the HTTP model. But here's the thing: in an ideal web app model, it's the app itself which should act as the client, not the browser. The app should provide a user interface and it should keep a persistent state. Plain HTML pages are incapable of this.

Instead, in most web apps including PBBGs, the server itself handles the user interface (by outputting HTML and handling commands as POST and GET requests) and keeps a persistent state using various hacks like sessions and cookies. For websites that function as collections of documents (which is most of the web) this works well. But for a web app, it's inefficient and puts a lot of work on the server that could be done by the client.

So here's my new project: a client-side framework that moves the user interface and front controller into a JavaScript engine. The server is then only responsible for concise AJAX requests that fetch data or issue commands. All HTML is rendered by the client and no reloading is necessary.

HTML templates will be used, just like templates on the server. A page controller will dispatch requests according to the URL, just like a server-side controller. So, it will look a lot like a regular web app. The basic "HTML forms" user interface paradigm is conserved (which is where it differs from the approach of projects like GWT).

The page controller will use the URL hash fragment (the part that comes after a hash sign #). When a HTTP request is made, the standard behavior is that the hash does not get sent to the server, and when the hash changes, the browser does not reload the page. When the hash changes (such as when a hash link is clicked) the JavaScript controller will handle it, in the same way that the server would handle a HTTP request.
For example: the url www.example.com/app/#/path/to/resource will send a HTTP request to the path /app/, where the app resides. The engine will be loaded from that page, and it will then take /path/to/resource and handle it accordingly. The hash fragment mechanism ensures that browser history and bookmarking work.

Attached is a rough schematic of the idea. I invite comments and ideas.

[JGadrow]

I like the amount of thought you've put into this post. I've been saying for a very long time that dynamic websites are not really websites at all but applications. I'm not certain how accurate your statement about most of the web being static pages is. I'd have agreed with you 10 or maybe even as few as 5 years ago. However, almost every site I visit now is dynamic in nature. Keep in mind, just because it serves dynamic text does not mean it's not an application. That's precisely what a text editor / word processing application does.

The rest of your post makes logical sense. The only caveat I see is that, obviously, it will not support users that have JavaScript disabled. Granted, that's a small percentage of the general internet populace in terms of actual users. However, it will have an effect on search engine results and the like and would need the ability to also store data in a method that is search-engine friendly where appropriate. You would definitely want your front page, description, forums, etc. to be spiderable without the need for JavaScript.

I wouldn't switch to this model simply because in my personal audience are at least a few individuals who disable their JavaScript and I try to make my sites as accessible (with regards to most recent 2 versions of popular web browsers) as I possibly can. So, do with that as you will.

[Blacklava]

I like the idea and would try that (even if I haven't fully understand the scheme lol)

about SEO I agree it would need at least one html index page but nothing more is ok, backlinks is the only needed thing to be ranked

[dsheroh]

I like the amount of thought you've put into this post. I've been saying for a very long time that dynamic websites are not really websites at all but applications.

...and I continue to disagree, to the extent of maintaining that a "web application" is a software application which uses a website as its primary user interface, although I suspect the disagreement has more to do with us defining the word "website" differently than anything else.  (It is an internet site.  The web is the portion of the internet for which it provides services.  Therefore is is a website, just like a site providing ftp services is an ftp site.  Now, if you want to argue that it should be called an "http site" rather than a "website", I could go along with that, but good luck getting broad acceptance for the term "http site" when "website" is already so widely used and so much more convenient to say.)

BTW, your link from the word "websites" is borked.  I get a generic "topic is missing or off-limits" error page when clicking it.

The rest of your post makes logical sense. The only caveat I see is that, obviously, it will not support users that have JavaScript disabled. Granted, that's a small percentage of the general internet populace in terms of actual users.

Small, but I suspect it may be growing as tools such as the NoScript plugin which allow you to enable/disable active content on a site-by-site basis become more widely-known.  Maybe it's just sampling bias, but I know quite a few people who only enabled javascript globally because they had one site they needed it for and they couldn't enable it for that site without also enabling it for the entire internet.

Which doesn't mean that a dependency on javascript is necessarily a bad design choice, but it does create the additional barrier of convincing users that your site is worth adding to their NoScript whitelist (and that it can be trusted not to abuse that access).

I wouldn't switch to this model simply because in my personal audience are at least a few individuals who disable their JavaScript and I try to make my sites as accessible (with regards to most recent 2 versions of popular web browsers) as I possibly can. So, do with that as you will.

Same here, although I see it more as a matter of graceful degradation rather than accessibility.  I like to build things that will work in as many different contexts and under as many different sets of constraints as is feasible rather than saying "you must run this version of this browser on this OS and have this feature enabled in your browser or else I won't talk to you."

Although it's not an inherent problem, most of the fully-javascript sites I've run across just display a blank white page with javascript disabled.  Unless the user is interested enough to "view source" and see that the whole thing lives within an AJAX container that hasn't loaded, then he's likely to just assume the site doesn't work and leave, never to return.  If your site doesn't work without javascript, don't forget to include something telling people this if they have it disabled.

[Chris]

PBBGs aren't really websites. They're applications.

And because of this attitude most of my competitors make one game in years while I do these in weeks

[JGadrow]

... I suspect the disagreement has more to do with us defining the word "website" differently than anything else.

I agree about the root cause of the disagreement. It really doesn't amount to any real difference except in the way that we reference the software. However, a majority of the definitions found through a quick Google seem to support my interpretation.

BTW, your link from the word "websites" is borked.  I get a generic "topic is missing or off-limits" error page when clicking it.

It's not broken; You're just not allowed to follow it yet. I believe the scripts board is still inaccessible until you're at 50 posts.

Small, but I suspect it may be growing as tools such as the NoScript plugin which allow you to enable/disable active content on a site-by-site basis become more widely-known.

I'm not going to hold my breath on this one. Taken as a general category, users are, almost by definition it would seem, idiots. They couldn't care less that there's mean, nasty JavaScript out there that can pick apart their poor old IE 6 browser and leave them crying to their friend / relative who's "good with computers" to help them fix it. Top that with the fact that there are so many people still surfing the web who have no idea that browsers other than IE even exist! Everytime I see site traffic with a non-IE browser increase for a given domain, I do a little happy dance on the inside. lol

Same here, although I see it more as a matter of graceful degradation rather than accessibility.

Making sure your software works for as many as you can is accessibility. Graceful degradation was just the term adopted to describe the utilization of JavaScript in an accessible manner.

By the way, dsheroh. I'm glad you're a part of this community. You make me actually work to prove my point which I believe is beneficial for the community as a whole to see differing viewpoints and the reasons behind them.

[dsheroh]

BTW, your link from the word "websites" is borked.  I get a generic "topic is missing or off-limits" error page when clicking it.

It's not broken; You're just not allowed to follow it yet. I believe the scripts board is still inaccessible until you're at 50 posts.

Well, then, how fortuitous that this is my 50th post.

(Better error messages... Clearly distinguish between 404 and 403 (or, arguably, 401) conditions... Blah, blah, blah...)

Edit:  Nope, still "either missing or off limits to you", although I suppose there could be something that still needs to be updated by cron before the site recognizes that I've hit 50.

By the way, dsheroh. I'm glad you're a part of this community. You make me actually work to prove my point which I believe is beneficial for the community as a whole to see differing viewpoints and the reasons behind them.

Happy to help, modulo the usual internet annoyance at occasionally having a long argument, only to discover that you've been in agreement all along, but didn't notice because you meant different things when you said "website".  

[Marek]

Progress

I've made a prototype page controller and the concept works, although it's not ready to show off online yet.

At a basic level, the page controller looks for the "hash change" event. This event is fired when a new URL is loaded, and only the hash fragment differs from the current URL. When this happens, the browser does not load a new HTTP request (instead, it attempts to jump to an anchor in the page, or fail silently) The hash change event is issued when:

• Clicking a link
• Using the browser history (which includes pressing back & forward buttons)
• Editing the browser address bar and pressing enter
• Setting the page address from within JavaScript

So all of the above work as expected, and emulate normal browser behaviour.

So, how do we use the hash change event? At the lowest level we could just hook it to callbacks, but I want to add an abstraction where hash paths are coupled with html pages. So, you don't actually give a JS callback to tell the engine what to do, but you give it a routing entry that tells it what page to load.

A routing entry might look like this:
 "When $path is requested, load data from $dataSource and feed it to $template and display the page."
The $dataSource might be an instruction to load an AJAX request or to use a static object or a JavaScript function.

Interaction
The easy, lazy solution for basic interaction: using jQuery, we can magically catch any form submits, feed the data into a JS callback, and handle the form client-side. A form routing table would route forms to callbacks, and depending on what's needed, the callback might make a server request or not.

But there's a lot of questions I haven't thought of yet. Where should custom JS reside and how to run it? Document ready events don't apply so there needs to be a replacement system for that.