http://dominointerface.blogspot.com/2014/05/transforming-domino-into-social.html
In my last blog, I described our desire to a create a community-based social collaboration platform designed to meet the needs of small and medium-sized organization. This post is the geeky part of this series. Only being naïve would one have taken on this journey. So the great experimentation began.
So we wanted to transform Domino into a social collaboration platform. But what did that mean in the technical sense. Since data is fluid and mostly unstructured, we needed the entire infrastructure to be fluid and that was the challenge. The challenges create many forks in the road of which many were dead ends.
There was four areas of technical challenges that we had to overcome.
Web Page and Interface
Though XPages technically separates the data from the form, it is intimately tied together and requires you to use the same infrastructure used in Notes client development. This is fine if you want to continue working in that environment. This locked structure does not allow for a fluid development environment. It required you to create views upon views for the front-end which we did not want to do. Instead, we want a loosely coupled structure so that the interface was totally independent of the back-end. A true MVC approach. If we decided at one point to replace Domino, we would not have to redo the entire front-end.
However, we needed a way to generate the Web pages and take advantage of the capabilities of XPages. We were experimenting with a development environment that we created called UXPages a number of years ago. It had the fluidity, but it was used primary for static web pages. Through the UXPages compiler, it took XML like code and converted it to HTML all written in LotusScript might I add. It was modeled after Adobe Flex.
http://dominointerface.blogspot.com/2009/07/introducing-ux-pages-and-dlex-project.html
We decided to expand on the UXPages compiler which eventually became the iPhora Application Designer to handle serious web application development. Rather than an application being a number of XPages and databases, our approach uses a single XPage as a Web container from which all web pages are generated.
Since we did not want the interface to be intimately tied to the back-end structure we decided that we would use JSON Restful services to pull/push the data from/to the server and generate the web pages using JavaScript. The advantages is that we can use any type of toolset to generate the JSON Restful services, we could use SSJS, Java, LotusScript, or and anything else.
A number of different JavaScript frameworks were consider including jQuery. Dojo won out because it incorporated a structured methodology that is especially important for creating widgets. The JSON data that comes from the server had to automatically bind and generate the information on the Web page. So we had to create our own data binding approach with the Dojo widgets similar to Angular.js and Backbone.js which created an entire development fork in itself.
Since we wanted the interface to be responsive, so naively started creating our own responsive CSS framework. However, after hearing about Bootstrap from a blog by Jake Howlett, we decided that Bootstrap was the way to go. We did consider Foundations, but there was so much open source stuff for Bootstrap.
We started creating the web interface using standard Dojo widgets, but discovered Dojo widgets to be slow and bloated and really does not work in the new responsive design paradigm. We experimented with different approaches, but after many tries we decided to build our own Dojo Bootstrap widgets. This gave us the simplicity of Bootstrap widgets while allowing us to have the automatic data binding. But one problem, almost 100% of the stuff was written in jQuery!!! So that started another venture of learning how to create Bootstrap widgets using Dojo.
After many, many attempts, we eventually were able to create an entire framework and toolset to create the desired development environment. Now that we had a fluid front end we needed a fluid back-end.
Database Infrastructure
NoSQL databases are now a big buzz in the IT world. However, Domino is one of the first NoSQL database system. Many newer databases like CouchDB are modeled after Notes and Domino. One difference between the newer NoSQL database and Domino NSF is the integrate of form and data in an NSF database which is unnecessary in the Web paradigm.
From our previous work with Integrated Business Framework (IBF), we knew that the Domino concept of a Notes form would not cut it. The main problem was that for every application you built you needed many forms and many views even if you are using XPages. So the more applications, the more views. And what slows down a Domino server, view indexes. So if we have 10 applications each with many forms and many views, you do the math. So one of the biggest challenges for us was how to create a database structure that would minimize the number views and forms. We struggled with many different designs and approaches until we finally developed a database structure that had only one form which served as the database container and a fixed number of views no matter how many applications were running. The form was a mixture of fields and MIME objects that stored everything as JSON allowing us to handle very large records similar to other NoSQL databases like CouchDB.
One important note. The newer NoSQL databases are fast and cool, but they lack one critical feature, security, especially when it comes to record access. As a domino developer we have one critical tool that no one else has, reader fields. If you are not taking advantage of readers fields to control access, then you need to learn how to effectively use them. With reader fields in place, records in the our database had the added security that we needed.
So now that we had a fluid web interface and a database infrastructure, we needed a controller to manage the security and data. That for next time. The blog is getting too long.
Transforming Domino into a Social Collaboration Platform Part 2B - Experimentation.
In my last blog, I described our desire to a create a community-based social collaboration platform designed to meet the needs of small and medium-sized organization. This post is the geeky part of this series. Only being naïve would one have taken on this journey. So the great experimentation began.
So we wanted to transform Domino into a social collaboration platform. But what did that mean in the technical sense. Since data is fluid and mostly unstructured, we needed the entire infrastructure to be fluid and that was the challenge. The challenges create many forks in the road of which many were dead ends.
There was four areas of technical challenges that we had to overcome.
- Web Interface Generation
- Database Infrastructure
- Application Structure
- Business Process and Workflow Management
Web Page and Interface
Though XPages technically separates the data from the form, it is intimately tied together and requires you to use the same infrastructure used in Notes client development. This is fine if you want to continue working in that environment. This locked structure does not allow for a fluid development environment. It required you to create views upon views for the front-end which we did not want to do. Instead, we want a loosely coupled structure so that the interface was totally independent of the back-end. A true MVC approach. If we decided at one point to replace Domino, we would not have to redo the entire front-end.
However, we needed a way to generate the Web pages and take advantage of the capabilities of XPages. We were experimenting with a development environment that we created called UXPages a number of years ago. It had the fluidity, but it was used primary for static web pages. Through the UXPages compiler, it took XML like code and converted it to HTML all written in LotusScript might I add. It was modeled after Adobe Flex.
http://dominointerface.blogspot.com/2009/07/introducing-ux-pages-and-dlex-project.html
We decided to expand on the UXPages compiler which eventually became the iPhora Application Designer to handle serious web application development. Rather than an application being a number of XPages and databases, our approach uses a single XPage as a Web container from which all web pages are generated.
Since we did not want the interface to be intimately tied to the back-end structure we decided that we would use JSON Restful services to pull/push the data from/to the server and generate the web pages using JavaScript. The advantages is that we can use any type of toolset to generate the JSON Restful services, we could use SSJS, Java, LotusScript, or and anything else.
A number of different JavaScript frameworks were consider including jQuery. Dojo won out because it incorporated a structured methodology that is especially important for creating widgets. The JSON data that comes from the server had to automatically bind and generate the information on the Web page. So we had to create our own data binding approach with the Dojo widgets similar to Angular.js and Backbone.js which created an entire development fork in itself.
Since we wanted the interface to be responsive, so naively started creating our own responsive CSS framework. However, after hearing about Bootstrap from a blog by Jake Howlett, we decided that Bootstrap was the way to go. We did consider Foundations, but there was so much open source stuff for Bootstrap.
We started creating the web interface using standard Dojo widgets, but discovered Dojo widgets to be slow and bloated and really does not work in the new responsive design paradigm. We experimented with different approaches, but after many tries we decided to build our own Dojo Bootstrap widgets. This gave us the simplicity of Bootstrap widgets while allowing us to have the automatic data binding. But one problem, almost 100% of the stuff was written in jQuery!!! So that started another venture of learning how to create Bootstrap widgets using Dojo.
After many, many attempts, we eventually were able to create an entire framework and toolset to create the desired development environment. Now that we had a fluid front end we needed a fluid back-end.
Database Infrastructure
NoSQL databases are now a big buzz in the IT world. However, Domino is one of the first NoSQL database system. Many newer databases like CouchDB are modeled after Notes and Domino. One difference between the newer NoSQL database and Domino NSF is the integrate of form and data in an NSF database which is unnecessary in the Web paradigm.
From our previous work with Integrated Business Framework (IBF), we knew that the Domino concept of a Notes form would not cut it. The main problem was that for every application you built you needed many forms and many views even if you are using XPages. So the more applications, the more views. And what slows down a Domino server, view indexes. So if we have 10 applications each with many forms and many views, you do the math. So one of the biggest challenges for us was how to create a database structure that would minimize the number views and forms. We struggled with many different designs and approaches until we finally developed a database structure that had only one form which served as the database container and a fixed number of views no matter how many applications were running. The form was a mixture of fields and MIME objects that stored everything as JSON allowing us to handle very large records similar to other NoSQL databases like CouchDB.
One important note. The newer NoSQL databases are fast and cool, but they lack one critical feature, security, especially when it comes to record access. As a domino developer we have one critical tool that no one else has, reader fields. If you are not taking advantage of readers fields to control access, then you need to learn how to effectively use them. With reader fields in place, records in the our database had the added security that we needed.
So now that we had a fluid web interface and a database infrastructure, we needed a controller to manage the security and data. That for next time. The blog is getting too long.
Transforming Domino into a Social Collaboration Platform Part 2B - Experimentation.
Comments