Setting up a simple blog part 1

Introduction

This will be the first part of a series on using node and various libraries to set up a blog.

I'm actually setting up my own blog to replace this one. So I expect part 3 onwards to be written on my self hosted blog at raynos.org (Which of course is not ready yet)

I've set up express in the same manner that I did last time.

Github

You can find all the code that is referenced here at the github URL

app.js setup

Let's take a look at my setup.

(function() {

    global._ = require("./public/javascripts/lib/underscore.js");

    var express = require('express'),
        routes = require("./app/app-routes.js"),
        configure = require('./app/app-configure.js');

    var app = module.exports = express.createServer();

    configure.configure(app, __dirname);

    routes.route(app);

    // Only listen on $ node app.js

    if (!module.parent) {
        app.listen(8080);
        console.log("Express server listening on port %d", app.address().port);
    }
   
}());

A few things have changed from last time. I've personally decided to make underscore global for simplicity. I've moved the routes and configuration to their own files. Notice how I need to pass the __dirname of the root app to configure as configure has its own dirname.

for both configure and route I pass in a reference to the app and these modules will augment the app with the correct routes and configuration.

App configure is basically all the default configurations moved to their own file for tidyness.

app-route.js

App route has changed enough to show you.

(function() {

    // Routes

    var route = function(app)  {

        app.get('/', function(req, res){
            res.render('index', getViewData('index'));
        });

        app.get('/about/', function(req, res) {
            res.render('about',getViewData('about'));
        });

    };

    var viewRE = new RegExp("(\/views\/)(.+)(\.ejs)");

    var getViewData = function(view) {
        view = view.replace(viewRE, "$2");
        console.log(view);
        switch(view) {
            case "index":
                return {
                    title: 'A new experience with node.'
                };
            case "about":
                return  {
                    title: 'A new experience with node.'
                };
        }
    };

    _.extend(module.exports, {
        "route": route,
        "getViewData": getViewData
    });

}());

Here we have two routes. one for about and one for index. They both grab the relevant templates and fill them with view data.

You may ask yourself why I've got a method to getViewData instead of hard coding it in. Well it's a simple model concept. These views have static models attached to them. The reason I'm doing it this way is so that I can re use the views and model data on the client.

The getViewData function takes a view string. Which is either a simplistic string as seen above or a more complex "\views\name.ejs" string as seen from the regular expression. I use the regular expression to normalize it to the simpler version then return different static data depending on which view was asked for.

Client - Server communication with now.

Now how would you go about handling these views and data on the client? We're going to use now to make getViewData accessible to the client.

var everyone = require("now").initialize(app);

everyone.now.getViewData = function(view, cb) {
    console.log(arguments);
    cb(routes.getViewData(view));
}

Here we simply tell now that everyone has access to the getViewData function. The function asks for a view and has a cb to handle the view. We then call the callback with our view data.

On the client we simply call this function as such.

now.getViewData(view, function(data) {
    console.log(data);
    new EJS({ url: view }).update('content', data);
});

In this particular case all we are doing is asking EJS which is the templating engine we are using on both the client and the server to render that view with the data we got from the server.

Now notice how easily we re used templates and view data on both the client and the server? It's beautiful isn't it?

main.js on the client

If you remember from last time main.js is the file that requireJS loads for us when it has finished loading itself.

(function() {

    require(
        [
            "/javascripts/lib/underscore.js",
            "/javascripts/lib/jquery.js",
            "/nowjs/now.js",
            "/javascripts/lib/ejs.js"
        ],
        function() { $(go); }
    );

    function go() {
        $("nav a").each(function() {
            var $this = $(this),
                view = $this.data("view"),
                span = $("<span></span>").text($this.text());

            if (view) {
                span.click(function() {
                    now.getViewData(view, function(data) {
                        console.log(data);
                        new EJS({ url: view }).update('content', data);
                    });
                });
                $this.replaceWith(span);
            }
        });
    }

}());

Here were saying that we need underscore, jQuery, now and EJS before we can run our code. Our code which is the function go is itself wrapped in jQuery's on DOM ready handler.

Now let's show you a snippet from our view to give you some context.  Now might also be a good time to mention that this project uses EJS instead of Jade as it's templating engine.

<header id="menu">
  <nav>
     <a href="/" data-view="/views/index.ejs"> Home </a>
     <a href="/about/" data-view="/views/about.ejs"> About </a>
     <a href="/posts/"> Blog </a>
  </nav>
  <hr/>
</header>
<section id="content">
  <%- body %>
</section>

Yes I am using the HTML5 header and nav. Currently I'm only writing and testing in firefox/chrome. I'll write a later post on how to support legacy browsers like IE8.

The important thing to grab from the information above is that where possible we are telling our links that they can be rendered on the client by giving them a HTML5 data-view attribute.

Looking back at the code we are grabbing this view data property and if it exists we are creating a new span element to replace the link. When the span is clicked we grab the view data from the server and get EJS to render that view with said data to the element with id "content".

This is doing a similar thing as we would have done on the server by redirecting to the about URL. The express routing also renders that view with the same data.

Notice how we've upgraded the page with progressive enhancement here. If you have no JavaScript the link works. And if you have JavaScript we get the client to do all the rendering over an AJAX request.

Shamefully this type of set-up does not scale well because EJS doesn't have native support for partials. Partials are part of express, we have to find a full blown view engine for the client rather then a templating engine. I'm still working on that.

test

Setting up a simple login form with node.js

Having spend the majority of the night setting this supposedly "simple" thing up, I'll give you guys the tutorial I needed.

Let's start of with the code. Here's the github link . To get it to work your going to have to install a bunch of things through npm.

• express
• mongoose
• now
• supervisor (Just for kicks)

In case you've never used npm before then go get it.

I personally wanted to set up a good foundation for this rather then get something quick and dirty out so I went for installing express. If you haven't already done this then go over to the express site and watch the screencasts. It gives you a good overview of what express is. 

To start my project I created a fresh express project with

$ express --sessions --css less

This set the express project up with the basic app file, jade templating and less for CSS compilation. Now I agree all of these things are completely over kill for something simple, but if you want to expand and grow on something it's best to have your foundation as large as possible.

The templates : 


index.jade

!= partial('login')
div#data.hidden

If you've seen the screencasts (Make sure to watch them) You can see that our HTML is just a call to our partial view for the login and a hidden data div to be shown when the login is successful.

login.jade

div#logincontainer
    h1 Please login

    label(for='username') Username:
    br
    input#username
    br
    label(for='password') Password:
    br
    input(type='password')#password
    br
    p#error.hidden
    br
    input(type='button', value='Login')#login.button
    input(type='button', value='Register')#register.button

We should be able to see what this is. We have a div container with some labels, inputs and buttons to login in or register. I won't bother to show the LESS but you can look at the github source if your interested.

I should point out that I've added this one line to layout.jade to kick start the client side javascript using require.js

script(type='text/javascript', src='/javascripts/lib/require.js', data-main='/javascripts/main')

require.js is pretty neat in this regard. You simply include the script and set a data-main attribute which will be loaded by require when require itself is ready. This is a really clean way to include javascript in your templates in one line.

Server side js in app.js :

In my own code I've left the auto generated routes code almost untouched.

Routes:

// Routes

app.get('/', function(req, res){
    res.render('index', {
        title: 'Pong Game'
    });
});

To be honest I did actually expect to make a pong game over the night but the learning curve did delay me quite considerably. To re-iterate what the express guys say this simply handles get requests on the main route and gets express to render our index view with the title local passed to it.

Next up is setting up the mongoose, mongo db connection. For this to even remotely work I recommend you download and install mongo db and if your own unix step through the mongo db unix tutorial

Mongoose:

// Mongoose db

    var mongoose = require("mongoose");

    mongoose.connect("mongodb://localhost/ponggame");

    var userSchema = new mongoose.Schema({
        'username': String,
        'password': String
    });
    
    userSchema.pre("save", function(next) {
        var user = this.username, password = this.password;
        if (user.length < 4) {
            next(new Error("User too short"));
        } else if (password.length < 8) {
            next(new Error("Password too short"));
        }
        next();
    });

    mongoose.model('User', userSchema);

    var User = mongoose.model("User");

First we get mongoose to connect to a database with an url and a database name. In this case the url is localhost and the database is ponggame. (I will actually get round to writing a multi player pong game.).

The way mongoose works is that you create schema's and interact through them. So I made a very simple userSchema that has a user name and password string (passwords in plain text, the best security ever). I'll leave implementing real security as a exercise for the user. I was going to try but I don't have any HTTPS certs lying around.

You can use the same concept of middle ware as you would in express, in mongoose. So I've added some validation middle ware to the save method. Ideally you would use the in build validation system but I wanted to test pre. In your middle ware method you can access the particular instance of the schema through this and you just simply do what you want then either call next. Here you can pass through an error as the first parameter and your callback on the save method can check for errors like that.

Of course we need to tell mongoose that this schema is a model and then we also need to ask mongoose for a real model object from our schema.

Of course we haven't actually done anything with our database yet but we're only going to interact with it on our communication layer.

Now:

// Now communication

    var everyone = require("now").initialize(app);

    function findUser(data, cb) {
        var user = {};
        if (data.username !== undefined) {
            user.username = data.username;
        }
        if (data.password !== undefined) {
            user.password = data.password;
        }
        User.find(user).exec(cb);
    }

    everyone.now.login = function (data) {
        var that = this;
        findUser(data, function(err, arr) {
            if (err) {
                that.now.loginFailure(err);
            } else if (arr && arr.length > 0) {
                that.now.loginSuccess(arr[0].doc);
            } else {
                that.now.loginFailure();
            }
        });
    };

    everyone.now.register = function (data) {
        var that = this;
        var user = {
            "username": data.username
        };
        function handleUsersFound(err, arr) {
            console.log(arguments);
            if (err) {
                that.now.registerFailure(err);
            } else if (arr && arr.length > 0) {
                that.now.registerFailure("User is already taken");
            } else {
                var user = new User();
                user.username = data.username;
                user.password = data.password;
                var o = user.save(handleSave);
            }
        }
        function handleSave(err) {
            console.log("save handled");
            if (err) {
                that.now.registerFailure(err);
            } else {
                that.now.registerSuccess();
            }
        }
        findUser(user, handleUsersFound);
    }

Hey look here, We're making all the communication trivial with now. We just have to define some methods on the server and we can call them on the client. Where possible the client will use real web sockets to communicate over TCP with the server.

To start of we initialize the everyone variable with the instance of our server (which is our express app).

I've defined a helper function which takes which tries to find a user in our database. it creates an object literal with a user name and optional password then calls .find on our model to find all users who match the name (and optionally the password). We then call .exec with a callback on our query to run it.

Calling find on the model is building a query and calling exec runs it. The callback takes two parameters, an error parameter and an array of data found in our Model.

Our server side login method is easier to understand we try to find the user with the data passed in from the client. In our callback we check for errors and pass them to the client if they exist. Calling methods that exist on "that.now" is simply calling method defined on the particular client that called the server method. If the arr exist and is non empty we tell the client his login succeeded and if the array is empty well then the user/password combination didn't exist and we just tell the user that the login failed.

Our register method is similar but a bit more complex. we again find the user but this time we only search for the user name. If the query returns a non-empty array then the user already exist and we throw conflict error. Otherwise we create a new user from the model. Set it's values and save it.

We need to have a callback on the save to see whether it was successful. We have some middle ware on the save that does validation and will pass in an error if the validation failed. So if the validation failed tell the client his register failed otherwise tell him he registered successfully.

Client side js in main.js :


Validation and talking to server :

(function() {

    require(["javascripts/lib/jquery.js", "/nowjs/now.js"], function() {
        $(function() {
            var login = $("#login.button"),
                register = $("#register.button"),
                username = $("#username"),
                password = $("#password");

            function validate(user, pwd) {
                if (user.length < 4) {
                    $("p#error").text("User too short").show();
                    return false;
                } else if (pwd.length < 8) {
                    $("p#error").text("Password too short").show();
                    return false;
                }
                return true;
            }

            $.each({"login": login, "register": register}, function(key, val) {
                val.click(function() {
                    var user = username.val();
                    var pwd = password.val();
                    if (validate(user, pwd)) {
                        $("p#error").hide();
                        now[key]({
                            "username": user,
                            "password": pwd
                        });
                    }
                });
            });
        })
    });

}());

So this is the main.js file that require.js loads for us. I have a habit of wrapping everything in its own private function scope so I use a self executing function for that.

Here we ask require to make sure that jQuery and now are loaded (Make sure you npm installed now or that file location won't work) before we do our code execution.

The validate function here looks similar to the one on the server. If the validation was larger/more complex I would be tempted to use now to run the validation from the server on the client. But currently the validation simply checks that the user and password are a sensible length and if there not they set the error text and show it.

The click handlers for login and register were so similar I used $.each to run the same function for both, the only thing that changes is which jQuery object to bind the click function to and which server side function to call on the now object. The click handler simply grabs the user name and password then runs validation and passes this data to the login or register function on the server.

The beauty here is that the entire socket.io communication layer is handled for you under the hood. Be wary of efficiency though. It's best to only define functions and use callbacks with now. Keeping data synchronized is not cheap. Nesting objects in the now name space is not cheap either.

In this particular instance I hide the error message if the validation succeeds. it doesn't feel right to put it there but I haven't set up backbone to handle the views for the login form on the client yet. (That's really overkill).

Methods called when server talks to client : 

// client side callbacks.

            now.registerSuccess = function success() {
                console.log("register sucess");
                $("p#error").text("").hide();
                $("div#logincontainer").hide();
                $("div#data").text("You registered").show();
                setTimeout(function() {
                    $("div#data").text("").hide();
                    $("div#logincontainer").show();
                }, 5000)
            };

            now.registerFailure = function fail(msg) {
                console.log("register failure");
                msg || (msg = "");
                $("p#error").text("Register failed " + msg.toString()).show();
            };

            now.loginSuccess = function success() {
                console.log("log in success");
                $("p#error").text("").hide();
                $("div#logincontainer").hide();
                $("div#data").text("You logged in").show();
            };
            now.loginFailure = function fail(msg) {
                console.log("log in failure");
                msg || (msg = "");
                $("p#error").text("Log in failed " + msg.toString()).show();
            };

Here are the four call backs that the server calls inside it's login/register method to tell the client that it either succeeded or failed. These simply manipulate the DOM pleasantly.

The register success hides errors, the login screen and shows a message in our data div saying you've registered. There's a timeout to reshow the login screen so the user can login in with its registered user.

The login success hides errors, the login screen and shows a message saying you've logged in.

The failure functions sets the msg to it's default value if it's falsy and shows the error.

This really isn't very DRY at all, these could easily have been replaced with a single success / failure function which takes an argument saying what failed/succeeded.

Starting the server with supervisor : 



If you've also installed supervisor then you can use the command line to start your app like so :

$ supervisor app.js

This means that supervisor will restart your app if it crashes. It also does hot reloads on any code changes so you don't have to turn your server on/off manually each time you make a change.

That's all for now. I hope this helps you with a few of the basics of express/mongoose and now. 

Tutorial 0: Set up node and the various tools.

Introduction:

I've recently managed to crash my copy of linux. One of the system updates made the windows ubuntu installer useless. I've tried hacking around to get it to work to no avail. Here's a quick tip: wubi is useless, don't use it. (As a side effect the boot loader is broken and win7 won't start anymore. I'll wipe the drive once my back up external drive arrives on monday and start from scratch again)

I've got a fresh copy of USB boot ubuntu and I have a self imposed deadline for sunday on kicking out a proof of concept game with nodeJS. This leads me to re install all the software I'm going to use to develop with nodeJS. Seeing as I'm nice I'll walk you guys through it.

About nodeJS:

I'm not here to tell you why you would want to use nodeJS because its simply awesome. Enough said.

Linux:
First things first. Your going to need to get a copy of Linux if you haven't got one already. nodeJS is server side technology and to run this server you need it to run on a linux machine. I would recommend the wubi installer but it's horrible.

If I guide you to ubuntu installation you should be able to grab a liveCD and append a partition to your drive to install Linux. Simply boot from the live-CD and install it. It should be "simple". If you get something other then Ubuntu then your on your own (I'm installing Fedora and its much more of a pain)

nodeJS:

Grab a copy of 0.2.5 (in linux of course). Simply double click on it and extract it your location of choice. As with any software the first thing we do is look for a useful readme file, you'll find it in node-v0.2.5/README.

  ./configure
  make
  make install

We assume by now that you've opened Applications -> Accessories -> Terminal or wherever the terminal is in your Linux distribution (I'll keep assuming your using Ubuntu from now on.). An easier method is to open the terminal by (use ctrl+alt+F7 to escape) pressing ALT+CTR+F1.

We assume you know how to use ls & cd to navigate to your /node-v0.2.5 folder. If you've unpackaged it in the default downloads folder it can be found with

  cd Downloads/node-v0.2.5

So since we _read_ the readme file we know what to do:

  ./configure

Make sure your inside the node folder. Now we most likely we hit a error saying we don't have a c++ compiler. Oh dear where do we get one? We use the powers of apt-get

  sudo apt-get install g++

Lets try to configure again? We get past the c++ compiler and now it tells us we dont have openSSL development packages. By the powers of apt-get:

 sudo apt-get install libssl-dev

We run configure again. It tells me that it hasn't found port.h, sys/event.h and kqueue. I don't think we care about that so let's make node

  make

We get a gigantic wall of text telling us what files were running and eventually we have compiled & linked all the c++ files that nodeJS runs on. It should finished with " 'build' finished successfully 3m4.5s" telling us that we succeeded. Of course your installation is either faster or slower then mine. Feel free to brag about that in the comment.

We should make the installation file and then were finished

  sudo make install

I had an issue on my copy where I need to use sudo to get root permission so I can make folders. It might be unnecessary for you but there is no harm.

So we have node installed. Just run `node` within your terminal (dont do it in the virtual ones under ctr+alt+f1-6 it breaks their formatting). It should show you the ">" prompt and let your write JS at it. For example "console.log('show me text!');" will work. It's not very interesting though. Your better off closing it (surely you must know ctrl+C) and running the tests. I would actaully advice running the test for most thing you download.

  make test

Mine throws only two errors. It can't find the curl command and some hash is not equal to some other hash. 137 tests pass so I'm fairly confident my copy of node works.

It would probably be worth while to kick open a text editor and dump the hello world sample into it from the nodeJS website. Save it as a usefuly descriptive name like "helloworld.js". Navigate to the folder (Remember "cd .." moves up a folder)  and 


  node helloworld.js


Now lets look at your server


Getting an awesome HTML5 browser:


Just pick one : Chrome 7, Opera 11, Firefox 4 beta

Enough said. Pick one.

Get a real IDE:

To me there are only two choices : Netbeans & WebStorm. If you actually want to search around yourself feel free to.

I'll walk you through WebStorm. Download and unzip it. Then go to the /bin folder and run "webide.sh". Double click and select run in terminal. It will tell you no JDK was found.

So lets go Applications -> Ubuntu software center and hit "sun java jdk" in the search bar. Pick up "Sun Java(TM) Development Kit (JDK) 6" and wait a while.

6 hours later. 


Ok I'm working on getting Web Storm to recognize the Java development kit. I forgot Linux was _this_ manual. Good luck installing it. I got it working eventually, I'm sure you will as well :)

Oh netbeans is real easy to install. Go with that if you want.

Set up Git Hub:


Using git hub is a great way to store your project online. Its version control and allows it to be shared with the open source community. We are writing nodeJS projects now so let's keep them open source and have the community grow.

Just create an account and then look at the help page. Create a repository for you work and if your using WebStorm you can create a new project directly from the repository you made.

Now we're ready to go I think.

Good luck.