Monday, November 23, 2015

Our Adventures into Creating a Secure MVC Framework - Part 3 - Experimentation

Now that we have defined our design requirements, we needed to determine the tools and approach that we would use to create our framework.

We had created previous versions of iPhora Touch using Dojo and wanted to start from scratch on what tools we would use to create our framework. Everyone that I talked to asked me why I was using Dojo since it is big and slow.  So we started looking at different options.  Do we take one of the existing more bare bones MVC framework like Backbone and take components from them to create our framework or do we start totally from scratch using JavaScript or jQuery.

There are so many frameworks out there it is amazing. You can find many examples of MVC frameworks in action at It was very difficult reading up on each framework, understanding them, and experimenting with them to see if they would work for us. Since we were already familiar with Dojo, we decide to start from there.  We found a couple of articles and frameworks that others had already done. However, as usual documentation was lacking.  What we did find was a very basic framework created by Christian Hoegl called Dojorama, and this served as our starting point. However it was built using version Dojo 1.9 the AMD version of Dojo.

Though we have been using Dojo for many years, we were using the non AMD version of Dojo and to us the new approach was like reading a foreign language. The structure of Dojo AMD and how it is used is totally different than previous versions of Dojo.  You had to type so much more code in order to do the same thing.  It seems ridiculous to us at that time. We struggled to make sense of what Christian was discussing. It seem to be too much work.  So we drop Dojo as a possible approach.

We tried Backbone, but we had to rewrite many of our widgets that we already created in Dojo.  In addition, we had to write more code to integrate with Backbone. Therefore, we dropped Backbone and went back to Dojo. So we spent many weeks learning and understanding the AMD approach used in newer versions of Dojo. Like always, most articles only presented simple examples.  We were struggling to understand the newer approach since our mindset on how to approach a problem was totally different.

As a precursor to further our discussion, we need to talk about Dojo widgets. A Dojo widget is a collection of HTML template and JavaScript. One hot topic of discussion in the Web development in recent years has been the concept of web components.  However, in Dojo this concept is no more than just a Dojo widget. Therefore, the concept of view/controller, it is just a Dojo widget.

A Dojo widget does not need to be a combobox or a field, it is can be as simple or comprehensive as you want it to be.  The class inheritance nature of Dojo widget makes it possible to extend the Dojo widget concept from being just a simple widget to a container widget or even something bigger and much more complex that inherits behavior from parent widgets.  The critical part that Dojo provides is the ability to create classes, superclasses, and class inheritance without having to create your own structure. This is one of the biggest advantages that most developers using Dojo have never utilized. Unless one has created complex Dojo widgets, one probably would not be familiar with these features. The OOP nature of Dojo with classes forces you to follow structured design patterns which makes it much easier for others to follow and manage your development process.

When a Dojo widget starts up it goes through a series of lifecycle methods including:
  • constructor
  • postscript
  • create
  • postMixInProperties
  • buildRendering
  • postCreate
  • startup

This lifecycle methods define how a widget is created, initiated, and displayed. A Dojo widget can inherit the behavior and functionality from superclasses or from other widgets.  In addition to these 7 lifecycle method for creating a widget you also have one method to destroy the widget.
  • destroy
For almost all cases the only one you normally would be using are:
  • constructor
  • postCreate
Most of the time, if you are using a multiple page application, you would never need to call the destroy method since the widget will be destroyed as a you navigate between pages. For a single page application however, the destroy method is critical. What we learned as we will discuss later, the standard Dojo lifecycles was not sufficient for us and we had to create many of our own in order to make everything work the way we needed to.

With this body of knowledge and experience we went back to the Dojorama framework to see if you can utilize any of the concepts presented by Christian to create our framework. Christian treated a web page simply as a Dojo widget.  As one navigates between pages you were displaying a different widget. The Dojo widgets (pages) were containers with simple HTML element inside. He used Javascript within each container to controller the behavior of the elements within each view.  This immediately hit us that the use of Dojo widgets in creating our MVC framework was the right direction.

Since we wanted iPhora Touch to be a multi-pane single page application, the structure need to be far more complex while being more robust.  The HTML components within a Dojo widget page needed to be complex Dojo widgets and they themselves can be containers contain widgets.  So we came up a layered Dojo widget based architecture that had the following structure.

Each layer was a Dojo widget starting from the core application layer all the way up to a widget like a combobox.

Next time:  Creating the Architecture - Dojo Widget Galore

Thursday, November 12, 2015

Our Adventures into Creating a Secure MVC Framework - Part 2 - Design Requirements

When we created iPhora Touch version 2, it was a multiple page application that utilized Dojo 1.53 and Bootstrap 2.32.  The interface worked well on desktops but broke down when used on tablets. It was totally unusable in a phone environment.  Since it was a multiple page solution each page needed to reload Dojo as you jump from page to page. Since we wanted a secured environment, pages could not be cached. However, this slowed down the loading and response of each page.  Since we were using Dojo 1.53 at the time we did not benefit from any form of AMD.

iPhora Touch V2 Community Home Page

We wanted to redesign the interface to work well with mobile devices and desktops, have better performance, but at the same time to be a secured environment.

Though our iPhora Touch solution is a social community-based solution, we designed it to be a closed environment by default and it is assumed that you are not allowed access to anything, which is contrary to how most social collaboration platform are designed.  Social collaboration no matter how any analyst or others spin it, is all about data. Who has access to this data needs to be securely controlled. Being openly social and with no control leads to data breaches. Security is not just about SSL, firewalls, and encryption.

So for any community you may have access to one piece of information at any given state, but in another one you may have no access or limited access.  This holds true not only within a community but all the way down to individual web containers and components part of a community.

So the challenge was to create an infrastructure and interface that met these requirements. Over the course of many months, we looked at many interfaces and did not find anything that we thought was appropriate. I asked my oldest son who is an industrial designer on design, but that was of no avail.  So it was a struggle to find something that would work.

My youngest son is a big user of Pandora and was listening to music when I noticed their application on his iPad.  I thought to myself this is a nice and simple interface, but very powerful. So I borrowed his iPad and played with Pandora for hours trying to understand its functionality. Unfortunately, there was a number of things that I thought was not usable for us, but it was a good starting point, a design inspiration. The multiple pane approach is what we needed, but how in the heck do we implement something like that. Many of the cool things it did we did not need or I was not sure we could even do on a web browser-based interface.

So using my favorite mockup tool, Balsamiq, I moved forward in laying out what the next version of iPhora Touch would might look like.  So we came up with a multiple pane environment that is shown below.  Depending on the state of the application some panes would be hidden and some not.  Many application used a similar approach including applications like Facebook on the phone.

This was the easiest part.  The challenge was to create a multi-pane single page application framework and make it work.

So we came up with our design requirements and they were as follows:
  • The JavaScript library that would be used to drive iPhora Touch need to utilize AMD to reduce the loading process
  • The JavaScript library components needed to only load once during a session and when needed.
  • Each pane needed to have its own independent security environment but at the same time could be dependent on the state of a different pane and controlled by the application developer
  • The content of each pane had to handle multi-state of existence:
    • Fully Cached, where both the view and scope is cached
    • View Cached, where only the view is cached and scope is destroyed after use
    • No Cached, where both the view and scope are destroyed after use.
    • Defined Cached, where the existence of view and scope is programmatically controlled by the application developer or the server
  • No data can be stored locally on the client, but create the illusion that data is local
  • The MVC framework had to handle not only the defined pane structure shown above, but needed to function in any multiple pane layout that one can come up with. For example, it should easily function in a three pane environment shown below.
  • All web container and components had to inhere to iPhora multiple level state environment. What this means is that for any web component there is a number of possible state of existence and the state in which it exist varies depending on the level of user access at any given time within its parent.
  • The MVC framework had to use dynamic routing. Since the state of any content within a pane is dynamic and ever changing, routing needed to be dynamic so static hash tag routing was totally out.
  • Data binding for components needed to be controlled by the application designer rather than being automatically performed by the system.
  • Web containers and components within a pane had to inherit the security of its parent container
  • All web containers and components needed to be generated dynamically and can not pre-exist since it is dependent on what a user is doing as it interacts with the application.
  • The MVC framework had to function with the iPhora RESTful APIs controlled by the Gatekeeper on the server.
  • The framework had to be lightweight and as small as possible
  • It needed to function in all favors of web browsers, both desktop and mobile devices (modern versions)
  • It needed a responsive UI.

Alternative Multi-pane layouts

So these were the design requirements and which there are many.

Coming next time, Part 3 - The Experiments

Friday, November 6, 2015

Our Adventures into Creating a Secure MVC Framework - Part 1 - Introduction

As I was writing this series, it cause me to rethink some approaches and redesign part of our MVC framework. Therefore, it took me a bit longer to start this series than I thought. But here goes.

The concept of Model-View-Controller MVC can be implemented in two ways, using a server-side component framework that generates the Web application interface/view or as a JavaScript framework that gets data from the server and generates the Web application interface/view .

In this series, we will be discussing how we created and implemented a JavaScript MVC framework. If you a Java developer and would like to implement a server-side MVC framework, Vaadin is a good server-side MVC framework to look into.  For developers in the XPages community, Ulrich Krause and Paul Withers has been blogging about Vaadin.

Each approach has its advantages and which approach you use is based on your development skills, tools, and architecture of your web application. If you are a XPages developer and plan to continue implementing web applications strictly on the server-side, I suggest you seriously look into Vaadin and move away from the whole XPages approach.  Vaadin provide you a more standardized and reusable approach.

What is an MVC Framework?
The goal with most MVC framework is to separate the application into 3 primary components that are use to generate the interface used to interact with the data, the model (data), the view (presentation), and the controller (manipulator).

If you are using a server-based approach, everything is done on the server. For a JavaScript-based MVC, the client retrieves the data and templates from the server. The controller facilitates the binding process and interface manipulation to generate what you see on the device. One addition component is the router which facilitates the communication between the client and the server.  For our architecture, the router is a separate component layer.  In some frameworks it is part of the model.

They are many different derivatives of the MVC concept. Some are very basic and some more encompassing. The model can reside both in memory and in local storage. In some approaches, the model structure is a directly reflection of the data on the server and the data is automatically synced. The controller always accesses the model locally whether in memory or local storage. As a result, any changes to the local data propagates back to the server.

MVC frameworks can implement either a one-way or bi-directional binding of the model with the view. Bi-directional binding significantly reduces the need to manage the data. One of the best examples of MVCs utilizing bi-directional binding is Angular. At this time, it is the most popular JavaScript-based MVC framework. Marky Roden has done a number of blogs and presentations at MWLUG and IBM Connect relating to Angular. Angular makes it easy for you with the help of some CSS create some very nice applications. Dojo also provides the ability for this two-way binding process, but it requires must more coding than Angular.

Two-way binding is great, but what happens when you start applying strict security rules on read/write/delete access that dynamically changes during a session. Well, the beautiful bi-directional binding falls apart and everything you do becomes a hack.

Secure MVC framework
As I mentioned in my prologue, our goal was to create a secure MVC framework. So what constitutes a secure MVC framework versus a regular MVC framework? A secure MVC framework has to perform like a regular MVC framework, but must take into account that data and information security trumps over the user interface. At no time can data reside locally on the client. It should only be in memory and if it is no longer needed it should automatically be destroyed. It needs to work in an environment where user access to any particular data at any instance is dynamic, access level is dynamic and may be revoked at any time during a session. It needs to respond accordingly to handle multi access level from the application level all the way up to the widgets level.

In addition, our framework needed to be server technology agnostic and work in a multiple pane environment where user access rights maybe different for each pane. So that was our goal.

Coming Part 2, Design Requirements

CollabSphere 2022 Presentation: COL103 Advanced Automation and Cloud Service Integration for your Notes/Nomad Applications

Our presentation for CollabSphere 2022. Learn how to quickly add workflow automation into Notes/Nomad applications using No-code tools that ...