Skip to main content

The iPhora Journey - Part 8 - Flow-based Programming

After my last post in this series -- way back in September 2022, several things happened that prevented any further installments. First came CollabSphere 2022 and then CollabSphere 2023, and organizing international conferences can easily consume all of one's spare time. Throughout this same time period, our product development efforts continued at full speed and are just now coming to fruition, which means it is finally time to continue our blog series. So let's get started...

As developers, most of us create applications through the conscious act of programming, either procedural, as many of us old-timers grew up with, or object-oriented, which we grudgingly had to admit was better. This is true whether we are using Java, LotusScript, C++ or Rust on Domino. (By the way, does anyone remember Pascal? When I was in school, I remember being told it was the language of the future, but for some reason it didn't seem to survive past the MTV era).  But in the last decade, there are some new developments in the programming world that we need to take into account... and no, I'm not talking about AI.

Let's start with the basics:

Based on recent surveys, 60 to 70% of all enterprise applications are associated with implementing business processes, and a simple definition of a "business process" is that it is a series of tasks that need to be performed by an organization to achieve a defined business goal.

In general, if the business process involves interaction with humans we refer to those processes as workflows.

Simple business processes are usually linear and may involve only a few steps and decisions.  However, more complex processes are often non-linear and involve the launching of other processes or sub processes. They may also involve data or data processing requests to external services.

If the business process needs to occur frequently, it is best to automate it. This can be achieved via software programming using any of a multitude of available languages. For example, over the past 30 years, literally thousands of workflow applications have been built by Domino developers and many of them are still in use today.

However, the core business process itself is fluid; it is always changing. This is because the business environment itself changes over time and competitive pressure means that improvements are always necessary. Unfortunately, constant tweaking and adjusting of all process across an entire organization is a tremendously expensive proposition, so much so, that most processes are rarely touched once they are rolled out.

When we first built iPhora Foundation to automate workflows and processes for our customers, it was XML-based and functionally similar to how typical Domino-based workflows operated. But that approach did not allow for the flexibility that we needed to handle non-linear processes, and of course, if any changes or improvements were needed, the customer has to pay our consulting rates to get it done.

But that experience taught us a great deal. We knew we needed to develop a different approach. Non-linear processes involve asynchronous triggering of incoming and outgoing requests. It may involve a mix of user-based and automated actions, as well interaction with external services.

At the same time, the industry has since moved in the direction of no-code programming, where the developers are the business users themselves. The goal of this approach is to make the resulting applications more suitable to business needs of the users and to reduce the costs associated with creating and maintaining applications. There are now hundreds of no-code and low-code solutions on the market, most of them utilizing the concept of flow-based programming. A well-known flow-based tool is Node RED, which is designed for low-code and pro-code developers.

From Wikipedia, flow-based programming is defined as.

"In computer programming, flow-based programming (FBP) is a programming paradigm that defines applications as networks of black box processes, which exchange data across predefined connections by message passing, where the connections are specified externally to the processes. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. FBP is thus naturally component-oriented.

FBP is a particular form of dataflow programming based on bounded buffers, information packets with defined lifetimes, named ports, and separate definition of connections."

You can think of it as creating a flowchart with each node performing a specific business operation. Each node is a black box and should be reusable.

So our goal was to transform iPhora Foundation to utilize a flow-based model of process automation. This turned out to be a much more daunting task than we originally imagined. Most flow-based solutions on the market use a simplified security model. Since many of the processes that our customers needed involved sensitive data, we wanted a higher level of security, where the operation of each node is allowed or disallowed based on workflow roles, which could be evaluated in real time based on a number of criteria.

We needed a framework that would allow us to create different types of nodes with defined business logic and make them available for users to build their processes. But how do you get these different nodes to communicate with one another? To handle this we created a persistent common bus that allowed the nodes to communicate and transfer data between themselves as long as the process was active. We decided that the communication pathway should be in the form of JSON objects. This allowed for a standard data packet structure that could handle any situation from the simple to the complex.  Since we had already developed a LotusScript-based JSON parser, this was a no brainer.

We started with 5 basic nodes:

  • Start
  • Task
  • Notification
  • Conditional Branching
  • Complete

With these nodes, business users can easily automate a basic workflow process. To handle more complex processes, we came up with dozens of additional nodes, including a node that can handle independent threaded processes called ActionStreams. (We will talk more about ActionStreams in a future blog post.)

We are constantly coming up with new nodes. Some of them require a low-code or pro-code developer to design and build, but the resulting node can then be consumed by business users in their own applications, giving them a wealth of business logic that they can include in their designs with no coding on their part.

The other part of the puzzle is the user interface. Our target audience of business users requires an intuitive, visual interface to assemble their processes.  Luckily, we were able identify a great open source solution, mxGraph, that provided the visualization that was needed.  If you have ever used a great little product called to layout a flow diagram, it uses the mxGraph library.

But setting the goals and selecting the tools was the easy part. Creating an effective, easy-to-use interface to our node processing engine was the majority of the battle. After many tries and redesigns, we were successful, and this design is now incorporated into our iPhora AppBuilder solution.

Flow-based programming is a powerful methodology for creating applications.  It forces you to think in terms of reusable independent building blocks, and the result is reduced development time and less need for skilled developers..

Next time, we will take flow-based programming to a different level, The iPhora Journey - Part 9 - Flow-based UI Programming.


Popular posts from this blog

Creating Twitter Bootstrap Widgets - Part II - Let's Assemble

Creating Twitter Bootstrap Widgets - Part I - Anatomy of a Widget Creating Twitter Bootstrap Widgets - Part II - Let's Assemble Creating Twitter Bootstrap Widgets - Part IIIA - Using Dojo To Bring It Together This is two part of my five part series "Creating Twitter Bootstrap Widgets".   As I mentioned in part one of this series, Twitter Bootstrap widgets are built from a collection standard HTML elements, styled, and programmed to function as a single unit. The goal of this series is to teach you how to create a Bootstrap widget that utilizes the Bootstrap CSS and Dojo. The use of Dojo with Bootstrap is very limited with the exception of Kevin Armstrong who did an incredible job with his Dojo Bootstrap, Our example is a combo box that we are building to replace the standard Bootstrap combo box. In part one, we built a widget that looks like a combo box but did not have a drop down menu associated with it to allow the user to make a select

The iPhora Journey - Part 3 - Creating an Integrated UI Framework

The iPhora Journey - Part 1 - Reimagining Domino The iPhora Journey - Part 2 - Domino, the Little Engine that Could The iPhora Journey - Part 3 - Creating an Integrated UI Framework There are many ways to create the user interface (UI) for a web application. The HTML page could be created on the server and then pushed out. It could be static with the data generated on the page by the server with JavaScript, providing a more dynamic experience, or the server could generate new HTML content to update portions of the web page. XPages or PHP are good examples of this. Another method is to have the web page partially generated by the server and have JavaScript build the rest of the HTML by pulling data from the server via an API. This is the approach used in the Single Page Application (SPA) model. In all cases, it is still dependent on the web server technology being using.  As mentioned previously in this blog, XPages is dependent on complete integration between form and document, which e