Dec 14, 2011

Mobile Architecture using jQuery, Node.js (+ express) and CouchDB

Apache Couchdb is the chosen one and now we have a complete (sort of) lightweight architecture for mobile apps. For those who think that the first line was an abrupt one, please refer to the previous post.

I now have a working example (a shoddy one) of a web app for entering fantastic details of the fauna you might be a keen observer of. The web app works but is without any validation, without any error handling and bloody hell - without even a feedback. So what the hell is a working app?

  1. Does this look good on a mobile? Does it actually look like it was built for mobile?
  2. Is it lightweight? (Define lightweight)
  3. Is it easy to develop? What are the skillsets required?
  4. Is it scalable? (Think reverse of what you just thought i.e. extremely small to start with not whether it can cater to your prospective 2.5million users after 2 years. Maybe that too.)
  5. Is it secure?

For question 1, I picked up the popular jQuery Mobile Framework after reading a couple of comparisons. It is very easy to use and the online documentation is very well organised and detailed. And the primary reason, it looks great on both iOS and Android. You can actually see a silly web app called Numbers, to help your kid fall asleep while counting sheep. It just uses a couple of jQuery Mobile elements.

The second question is the most difficult one and depends on how lightweight is defined. For some, it could mean less CPU intensive, less disk space or uncomplicated licensing model. For others, it could mean easy to setup, easy to configure and easy to debug. I am no authority on this, but Node.js seems to give a positive reply to many of these. And so does CouchDB. There are other NoSQL options - MongoDB, being a very popular choice. But what impressed me about CouchDB was the fantastic RESTful design it has implemented. And how does it matter? Well, inquiring and manipulating data through a middleware using simple HTTP GET, PUT, DELETE calls gets amazingly simple, while handling the HTTP requests from the front-end, without any extra learning curve.

Question 3 is the easiest to answer - we don’t need anything other than HTML/Javascript. How much easier can it get?

Both Node.js and CouchDB are being touted as the ultimate in scalability and performance. But as I said, let’s see those from the current situation rather than the future. When you are starting with the business idea and expect the volumes to pick up only after a certain period or if you just want to test the waters, do you have to invest heavily upfront in an unpredictable business model? And does it offer the flexibility to change rather than build to last? Solutions such as Node.js and NoSQL are good answers to this predicament. Q4 answered.

These are still very early days to say much about Node.js security. So no idea yet. But we can at least rule out SQL injection :-)

Anyway, too much rambling above. Here is the action (Linux or Mac) -

1. Simple 'untar Node.js source, ./configure, make, sudo make install' routine in Linux/Mac OS X. No major dependency issues other than Python and OpenSSL. (test using 'node -v')

2. Install npm - Node Package Manager using the one line install. It didn’t work due to proxy settings as I was trying with superuser credentials and environment variables exported in normal login are available only by using the -E option. Also, I had to replace sh with bash. Eventually, 'curl npmjs.org/install.h | sudo -E bash' worked. (test using 'npm -v')

3. 'npm install express' - for the Node.js module, express - a web framework on node, which will be our middleware.

4. Installing CouchDB on Mac was easy using Homebrew - 'brew install couchdb'. But on Ubuntu, building with source was a safe bet. There are quite a few dependencies to be handled on Ubuntu. (test using 'couchdb' or 'sudo couchdb')

The above are just hints, search on the net and ample of instructions (detailed) can be found.

Though I went through a few trials and errors, the following is how to make it work.

1. After starting CouchDB, create your database from the command line 'curl -X PUT http://127.0.0.1:5984/acme_fauna' or accessing http://localhost:5984/_utils/ through a browser and selecting the appropriate options.

2. Create the index.html (in a folder called ‘public’) which will be your data entry screen with the jQuery Mobile niceties in.

<!DOCTYPE html>
<html>
<head>
  <title>ACME Fauna Entry</title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="stylesheet" href="jquery.mobile-1.0.min.css" />
  <script type="text/javascript" src="jquery-1.6.4.min.js"></script>
  <script type="text/javascript" src="jquery.mobile-1.0.min.js"></script>
  <script language="JavaScript" type="text/javascript">
  </script>
</head>
<body> 

<div data-role="page" data-theme="b" data-content-theme="b">

  <div data-role="header">
    <h1>ACME Fauna Entry</h1>
  </div><!-- /header -->

  <div data-role="content">
    <form action="http://localhost:3000/fauna" method="post">
      <fieldset>
        <div data-role="fieldcontain" class="ui-hide-label">
          <label for="type" class="select">Type</label>
          <select name="type" id="type">
            <option value="bird">Bird</option>
            <option value="moth">Moth</option>
            <option value="mammal">Mammal</option>
            <option value="reptile">Reptile</option>
          </select>
          <label for="cname">Common Name</label>
          <input type="text" name="cname" id="cname" value="" placeholder="Common Name"></input>
          <label for="sname">Scientific Name</label>
          <input type="text" name="sname" id="sname" value="" placeholder="Scientific Name"></input>
          <label for="subtype">Subtype</label>
          <input type="text" name="subtype" id="subtype" value="" placeholder="Subtype"></input>
          <label for="Description">Description</label>
          <textarea name="desc" id="desc" value="" placeholder="Description"></textarea>
          <input type="submit" id="submit" value="Submit"></input>
          <input type="reset" id="cancel" value="Cancel"></input>
        </div>
      </fieldset>
    </form>
  </div><!-- /content -->

  <div data-role="footer">
    <div data-role="navbar">
      <ul>
      <li><a href="#" style="text-align:center" onclick="List()">List</a></li>
      <li><a href="#" style="text-align:center" onclick="Review()">Review</a></li>
      </ul>
    </div><!-- /navbar -->
  </div><!-- /footer -->

</div><!-- /page -->

</body>
</html>

3. The form in index.html will be submitted through a POST method to a web server (webserver.js) that will be implemented using express on Node.js

var http = require('http');
var express = require('express');

var getcuuid = {
  host: 'localhost',
  port: 5984,
  path: '/_uuids',
  method: 'GET'
};

var createcdoc = {
  host: 'localhost',
  port: 5984,
  path: '',
  method: 'PUT'
};

var app = express.createServer();
app.use(express.bodyParser());
app.use(express.static('public'));
//app.use(express.static(__dirname + '/public'));

app.get('/', function(req, res){
  res.send('Hello World');
});

app.post('/fauna', function(req, res){
  // GET http://localhost:5984/_uuids to get a new UUID for each entry
  var cuuid = '';
  var cureq = http.request(getcuuid, function(cures) {
    cures.setEncoding('utf8');
    cures.on('data', function(cuuiddata) {
      cuuid = JSON.parse(cuuiddata).uuids[0];
      //console.log(cuuid);

      // PUT http://localhost:5984/acme_fauna/<UUID> with data in JSON form i.e. req.body
      createcdoc.path = '/acme_fauna/'+cuuid;
      //console.log(createcdoc.path);
      var ccreq = http.request(createcdoc, function(ccres) {
        ccres.on('data', function(ccdata) {
        //console.log(ccdata);
        });
      });
      //console.log(JSON.stringify(req.body));
      ccreq.write(JSON.stringify(req.body), encoding='utf8');
      ccreq.end();
    });
  });
  cureq.end();
  //res.send(req.body);
  res.redirect('home');
});

app.listen(3000);

4. Run the webserver using 'node <path/to>webserver.js'. This should be run from the folder containing the folder ‘public’ and providing the correct path of webserver.js to the node command.

5. Access the web app through any browser by pointing to http://localhost:3000/

6. Add birds, mammals, moths and insects in the database to your heart’s content.

The diagram is still pending!

Nov 2, 2011

Enterprise Mobile Architecture to start with…

After a few days of research, I am very much inclined to suggest that HTML5/CSS/JavaScript as a mobile development platform is the best way to get started for the enterprise. When you compare the alternate (primary choice) of organisations experimenting with the native platforms like iOS or Android, while being undecided about their choice of mobile platform for overall deployment, is more cost effective method to ride the mobility wave.

Lacking the vision of how enterprise apps on mobile devices will actually look like in the future - remember how alien the web front end concept looked like in the days of client-server architecture (ok, I will add thick in front of that) or how alien the thick client architecture seemed when we coded on the mainframe - the web app strategy is the path with the least risk of cost. In the era of slick iPad dashboards for reports, that may sound a bit old-fashioned, but I will take that in stride.

The first question that a customer requests for an enterprise mobile app is the offline mode. Understandably so, because of the unreliable and costly data network. HTML5 has introduced the usage of Session Storage, Local Storage, AppCache and Indexed Database (unfortunately WebSQL has been deprecated, but it is the only option till iOS supports IndexedDB) and using these, a web app should be able to do everything that a native app could do with the data in offline mode. Similar to a native app, the web app can synchronise with the back-end when it goes online.

Another major HTML5 feature is the introduction of WebSockets. Though it will take effort to bring it into practice immediately (not everybody developing business applications want to read about bi-directional, full-duplex TCP connections etc.), it will form the basis of performance benchmarks for the bandwidth usage of apps (mobile data plans are costly). And maybe save some battery juice (you don’t want users to complain that your app drains the battery).

Of course, the much known Canvas, Video and Geolocation tags will find their way in the enterprise apps, too.

The slick UI that customers expect from an iOS or Android app can be built using HTML/CSS/JavaScript (did you notice the missing 5? yes, it was deliberate). jQuery, Dojo and Sencha are some of the Javascript frameworks that can enable quick development of rich front-ends.

Now, if you combine the above with REST and Node.js on the server side, we are closing in on a really compact and lightweight architecture for the Enterprise Mobile Architecture to start with…

Will come up with the database side observations and a beautiful diagram later!

Oct 17, 2011

Mobile Web Apps - Dashcode

There is a lot of buzz around HTML5. And some of it is about how it is a solution for mobile apps to address the cross-platform requirements in the mobile space (don’t we all know that actually it is just about two platforms… okay more than two due to the Android fragmentation… personally and seriously, I think, Windows+Nokia can be a dark horse). Canvas and local storage are the two biggest game changers that have generated this interest in HTML5. And it makes more sense for enterprises to go for HTML5 rather than native if the app requirements are more forms based than those which require camera, GPS, accelerometer etc.

So there I was on my quest to find out more about tools for developing HTML5 mobile web apps. And it was a rude shock as I hit the bottleneck quite soon – there aren’t any established players here. Adobe has just released a preview version of its new tool called Edge (but which may be more focused on animated web content). And another one is Maquetta (web based visual authoring of HTML5 user interfaces) from The Dojo Foundation.

But I stumbled upon a couple of blogs about Dashcode from Apple, which was present on my laptop for ages but I had not bothered to even check it out for the simple reason that I assumed it was only for developing Apple’s Dashboard widgets.

On further exploring, it was amazing to find that this is a good featured development application to build Mobile Web Apps capable of running on Safari (Mac and iOS)… no wait… it may also work for Webkit based browsers (Chrome and Android mobile). I say good featured (and not great), because using it will have challenges for enterprises. Unlike XCode, which supports Subversion and GitHub out of the box, for version control, I could not find any such arrangement for Dashcode. And like XCode, it is available only for Mac OS X.

I decided to run a crash course for myself using a good book but could only get two books on Books24×7 with Dashcode in the title (a) Creating Mac Widgets with Dashcode - by William H. Murray and Chris H. Pappas and (b) Dashcode For Dummies - by Jesse Feiler. Obviously (a) did not make sense as it was all about widgets. So, decided to be a dummy. Have gone through it with hands-on development for 5 days and I have got a certain grasp on this.

I feel the book has a number of redundant chapters, but it does introduce fundamentals fairly and more importantly, emphasizes the mobile app development. It gives a good direction on how to explore further possibilities but falls short talking about integration aspects with the backend systems (without which an enterprise app will have no use). On the data front, it elaborates on using a JSON and XML local data sources in the form of files within the project. For the local storage, it just talks about creating a new project from Dashcode’s existing templates and exploring the code to understand how that can be used. It is a ‘for Dummies’ book and I think the expectations should be limited anyway.

For the integration aspect, I should forget about Dashcode and look in other directions viz. JavaScript, Web Services and middleware technologies. And I have enlisted a couple of experts for that.

Oct 6, 2011

Steve Jobs

A personal loss. Doesn’t feel real. Difficult to believe that now I am living in a world without him.

Aug 6, 2011

Konkan - Ganapatipule

It was more than a year since I had a vacation and when my friends suggested Ganapatipule and the MTDC resort there, I did not think much before taking the decision. Also, with the good winter weather prevailing this time, the risk of ending up in a sweaty situation was less.

I didn’t have any other references apart from MTDC for the accomodation, so I had a look at the photographs on the Maharashtra Tourism website - unfortunately the photographs do not describe the different types of rooms available (Deluxe, Konkani House, Sea View Cottage, Sea View Deluxe, Super Deluxe etc.) nor does the availability link. Again, luckily for me only Konkani House Non A/c was available for the days I had planned and believe those were the best ones. Bookings can be done online from the Maharashtra Tourism website. One tip though, after booking call the resort on their local number and confirm your arrival and if you have more than one room booked (like I had). This makes sure that you will get a room with nice views and also be together.

I had a hired vehicle, so cannot guide in terms of public transport. But one word of caution, the Konkani houses are located 1 km from the resort entrance. It is a private red muddy road - did not see any autorickshaw coming till there and definitely not advisable to walk down that road with luggage - otherwise you will have a horrible start to the holiday. So, if you plan to take public transport, call the resort ahead for guidance on how to reach the Konkani houses.

Mumbai to Ganpatipule is 6.5 hours by road and you will need at least one break for lunch or dinner if you don’t travel at night. And I would suggest not to travel at night else you will miss the fantastic route through the beautiful district of Ratnagiri e.g. before you reach Chiplun, you can see the following view of the River Vaishishti.

We were running late than intended and it was dark by the time we settled in our rooms. It was pitch dark outside and without moonlight it was difficult to know what was the view from the room. So the first thing that I did in the morning was to discover this - the creek right behind the room. Advaita and I went to explore the surroundings immediately. And bewildered to actually see the crystal clear water and the fish swimming right upto the edge - it was unbelievable.

Though we had decided to wake up as late as possible, all of us were up quite early - maybe the excitement of the holiday did the trick. Being at a holy place, the first item on the agenda was to visit the Ganapati temple. I spoke to the resort staff and they told that the temple was just 15 mins walk from the resort beach. So decided to take that way instead of the car. And let me warn, it was not a good decision to walk on the beach with no shade in the noon with 2 kids in the tow and parents. Fortunately, the walk was just 20-25 mins.

I have never seen a temple right on the beach and that is how Ganpatipule temple is. It is a small temple and though it is one of the famous ones in Maharashtra, it is not at all busy. And the striking thing is that it is not littered with people trying to sell you puja sahitya and memoirs. The temple’s bright orange colour makes is a good backdrop for family photographs. Advaita and Dev had a good time and few puranpolis around the temple. At the entrance of the temple, there is a 3.5-4 ft tall bronze statue of a mouse and devotees whisper their wishes in his ears.


After having lunch at one of the many vegetarian restaurants - pretty basic but homely - we returned to the resort for a good time on the hammock under the shades of the coconut and pine trees.
In the evening (at around 5pm), we drove towards Jaigad - a small fort 10kms from Ganpatipule. The road goes through Malgund - birthplace of a famouse Marathi poet, Kavi Keshavsut - and numerous other small and beautiful villages. Jindal has set up a coal based power plant in Jaigad. That has added one of the very very few industrial parts to the otherwise natural landscape.

The fort itself is in ruins and someone really stupid had tried to concretise the main tower of the fort. Maybe the work stopped after Archaeological Survey of India intercepted it and declared it as a Protected Monument. But spending an hour in the fort was fascinating and one can understand how Babasaheb Purandare dedicated his life to the history of Shivaji Maharaj after visiting various forts in Maharashtra. While in the fort, the imagination runs wild and you start hearing voices from the bygone era. There are still some small houses and a temple in the fort.



The next day we went to Ratnagiri. The drive along the coast from Ganapatipule to Ratnagiri is breath taking - virgin beaches all along the way. The first stop was Thibaw Palace which has a pretty sad story - it is about a Burmese King who was captured by the British and exiled in Ratnagiri. Burma and Ratnagiri could be so different places that one can only imagine how the king and his family may have spent their life here.

After Thibaw Palace, we visited Lokmanya Tilak’s birthplace which is maintained very well and as with historical places, one starts thinking about how the great man grew up here in a small town and went on to be a defining character in India’s struggle for independence - that in an era without TV and Internet.

After having a nice and filling Malvani lunch with couple of glasses full of Solkadi, we were back on the scenic route that justifies Konkan’s description as Maharashtra’s California.




Next Page »