Friday, September 16, 2022

The iPhora Journey - Part 7 - Transforming Domino with Microservices

Most of the concepts that we have been talking about in our iPhora Journey are neither new nor revolutionary. Collectively, they merely represent the current state of the art in designing web applications.

In other (i.e., non-Domino) platforms, you would need an array of services each running on a separate server, and you would have to build the integration between the different components and maintain security between all of those components. With virtualization and containers all of the necessary services can now bundled into a single installation, which is helpful, but the integration of those components can still be fraught with issues, especially compatibility between different versions of those services.

With Domino all of this is taken care for you: authentication, directory services, security, web services, and database services are all tightly integrated in a single server,  which is an advantage that no other technology offers. While Domino has its quirks and does not support many of the latest features, hopefully HCL will be addressing those in the near future.  Even as it stands right now, the simple addition of nginx as a proxy to Domino addresses most of the deficiencies.

The iPhora platform is built on Domino, and in doing that we take advantage of the inherent integration of the Domino services. But we also incorporate design methodologies that are usually associated wit non-Domino applications. With this approach, iPhora has the capabilities and features of a modern web application, but all it needs to run is a single server with the most minimal of hardware requirements. We challenge any vendor to be able to do everything that iPhora does on a single box as small as this.

Creating a modern application is not solely about using latest technology to build the application. Well-supported technologies with a long term commitment to maintenance are always preferable to the latest design fad. The important question is what you do with that technology? 

The iPhora concept evolved over the span of 15 years, as we developed applications for organizations, both large  and small, and gained experience with how companies handled their internal business processes. Step by step, we abandoned the more monolithic approach normally used when creating Domino applications, for a more fluid and plug and play architecture. Eventually, we wound up with a service based and loosely coupled architecture, a microservice architecture. It should be noted that this applies to the iPhora components that we developed, not to the underlying Domino services.

Wikipedia defines microservice as:

"A microservice architecture – a variant of the SOA (service-oriented architecture) structural style – arranges an application as a collection of loosely-coupled services. In a microservice architecture, services are fine-grained and the protocols are lightweight. The goal is that teams can bring their services to life independent of others. Loose coupling reduces all types of dependencies and the complexities around it, as service developers do not need to care about the users of the service, they do not force their changes onto users of the service. Therefore it allows organizations developing software to grow fast and big, as well as use off-the-shelf services more easily. Communication requirements are reduced. These benefits come at a cost to maintaining the decoupling. Interfaces need to be designed carefully and treated as a public API."

Using the programming tools available in Domino, we were able to build a microservice architecture that passes data in and out using a common bus and transport schema. Each microservice is an isolated black box with only input and output.

Remember that in a previous blog I mentioned the importance of the JSON format and the ability to quickly process JSON. The reason is that the data transport layer between microservices is all JSON. By having all input and output defined as JSON objects, we achieve a standardized and easily extensible communication layer. This allows microservices to be independent and reusable by multiple services and processes. A typical process can chain together a series of microservices to accomplish a specific and highly complex task, as opposed to writing a custom services to accomplish the same thing.

But how do you connect these different microservices together to create your application? This leads to our next discussion next time, flow-based programming.

Tuesday, September 6, 2022

The iPhora Journey - Part 6 - An Application, Rethinking and Redefining

In our previous articles, we discussed the advantages of using JSON in applications and the importance of being able to effectively process JSON. This leads to the interesting question: what exactly is an application, and the answer to that question may differ depending on who you ask. Most people today would assume you are referring to a phone app, and even then there are wide range of options. A college student might think of apps like Tiktok or WhatsApp, and a person looking for a significant other might automatically think of Hinge or Tinder. All of these people would be able to provide you with multiple examples of apps, and they might be able to list some of the common features of an app, even if they do not understand any of the technical details.

In general, a web app today consists of the following four modules, even if the technical details may differ tremendously from app to app:

  • User interface (UI)
  • Database
  • Business Logic
  • Security

Now, let's consider what an app is in the traditional Notes/Domino sense. Here, an application still consists of the same four modules, but everything is mixed together (actually smeared-together may be a better term), as opposed to being separated. A Domino application consists of one or more Domino databases, and access and security is defined within each database using ACLs.  This is true for both Notes client and Web-based applications created on Domino -- so far, so good.  However, when it comes to the UI and Database modules, Domino uses the concept of a form, which defines both the user interface and the data structure for storage, modules that are typically separated in more modern apps.

Moreover, the business logic is programmed directly into the actual UI components all over the form. Even something as simple and basic as controlling what UI elements are visible (i.e., hide-when formulas) requires that the programming logic is dispersed throughout the form and may be associated with each and every UI component.

This approach can work for stand alone applications or situations involving only a few applications. However, as the number of applications increases, providing the infrastructure to effectively create and manage the applications and user access becomes more and more difficult.  This is very true for Domino applications because the business logic can be implemented in so many different ways and in so many different places.

The complexity of Domino applications tends to increase even further because the data is distributed throughout multiple databases using different data structures. The more different forms an application uses, the more views are needed and the more indexing is required, resulting in the need for more storage space.

So our primary goal was to structure iPhora to follow a modern application architecture, but still provide the extensive data security one expects from Domino. We wanted to be able to build and manage hundreds of applications with the same ease as doing it for one application. Almost all of the development projects that we have been involved with so far began with one critical application that eventually expanded to dozens of related applications, each with different sets of users and different ACLs requirements. For one customer, that one initial application grew into 50 applications and over 100 processes, each with different roles and user access requirements and all accessed and managed within a single instance of iPhora. Yet even as more and more applications were added, no additional Domino databases were needed.

An iPhora installation uses only seven Domino databases, each serving a specific purpose within the iPhora architecture: 

  • API / GateKeeper
  • Attachment Store
  • Business Logic
  • Data Store
  • Logging
  • UI Interface
  • User Profile

In other words, there is no longer a one-to-one relationship between applications and Domino databases. Any number of applications, including widely different types of applications, can be accommodated using this structure.

Since data security is of the upmost importance for iPhora, it was extremely important that users never have direct access to the data, which is contrary to a traditional Domino application. We wanted all of the databases to be totally locked down. Our API consists of LotusScript agents that interact with the data according to strict protocols. In general, the only database that users interact with is the database that holds those agents, and the UI interface. The other databases cannot be opened using Domino URL commands.

Since data is stored as JSON, the data store module utilizes a single Domino form to store all records regardless of the application, although the fields may different from one form to another. Any attachments that are uploaded into a data store document are actually stored in a separate attachment store database which also utilizes only one single Domino form. With DAOS, we can handle 1 TByte of attachments and if more are needed, we just can simply add another attachment store.

For companies that already have data in Domino databases, iPhora has the ability to read and display data from other Domino databases, including databases that are structured in the traditional Domino way. The only requirement is that form structure needs to be defined and added to iPhora.

In conclusion, by re-thinking and re-imagining what an application is in Domino, Domino becomes a scalable application platform on which to build and run secure applications that utilize a modern design framework.  The advantages of a structured and scalable application architecture become apparent as one moves towards utilizing no-code/low-code tools to build applications, something we will consider later in this series. 

How we access and communicate with external services including non-iPhora Domino applications is the subject of our next discussion, Microservices and Naoservices

CollabSphere 2022 Presentation: COL103 Advanced Automation and Cloud Service Integration for your Notes/Nomad Applications

Our presentation for CollabSphere 2022. Learn how to quickly add workflow automation into Notes/Nomad applications using No-code tools that ...