After my last post in this series -- way back in September 2022, several things happened that prevented any further installments. First came CollabSphere 2022 and then CollabSphere 2023, and organizing international conferences can easily consume all of one's spare time. Throughout this same time period, our product development efforts continued at full speed and are just now coming to fruition, which means it is finally time to continue our blog series. So let's get started...
As developers, most of us create applications through the conscious act of programming, either procedural, as many of us old-timers grew up with, or object-oriented, which we grudgingly had to admit was better. This is true whether we are using Java, LotusScript, C++ or Rust on Domino. (By the way, does anyone remember Pascal? When I was in school, I remember being told it was the language of the future, but for some reason it didn't seem to survive past the MTV era). But in the last decade, there are some new developments in the programming world that we need to take into account... and no, I'm not talking about AI.
Let's start with the basics:
Based on recent surveys, 60 to 70% of all enterprise applications are associated with implementing business processes, and a simple definition of a "business process" is that it is a series of tasks that need to be performed by an organization to achieve a defined business goal.
In general, if the business process involves interaction with humans we refer to those processes as workflows.
Simple business processes are usually linear and may involve only a few steps and decisions. However, more complex processes are often non-linear and involve the launching of other processes or sub processes. They may also involve data or data processing requests to external services.
If the business process needs to occur frequently, it is best to automate it. This can be achieved via software programming using any of a multitude of available languages. For example, over the past 30 years, literally thousands of workflow applications have been built by Domino developers and many of them are still in use today.
However, the core business process itself is fluid; it is always changing. This is because the business environment itself changes over time and competitive pressure means that improvements are always necessary. Unfortunately, constant tweaking and adjusting of all process across an entire organization is a tremendously expensive proposition, so much so, that most processes are rarely touched once they are rolled out.
When we first built iPhora Foundation to automate workflows and processes for our customers, it was XML-based and functionally similar to how typical Domino-based workflows operated. But that approach did not allow for the flexibility that we needed to handle non-linear processes, and of course, if any changes or improvements were needed, the customer has to pay our consulting rates to get it done.
But that experience taught us a great deal. We knew we needed to develop a different approach. Non-linear processes involve asynchronous triggering of incoming and outgoing requests. It may involve a mix of user-based and automated actions, as well interaction with external services.
At the same time, the industry has since moved in the direction of no-code programming, where the developers are the business users themselves. The goal of this approach is to make the resulting applications more suitable to business needs of the users and to reduce the costs associated with creating and maintaining applications. There are now hundreds of no-code and low-code solutions on the market, most of them utilizing the concept of flow-based programming. A well-known flow-based tool is Node RED, which is designed for low-code and pro-code developers.
From Wikipedia, flow-based programming is defined as.
"In computer programming, flow-based programming (FBP) is a programming paradigm that defines applications as networks of black box processes, which exchange data across predefined connections by message passing, where the connections are specified externally to the processes. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. FBP is thus naturally component-oriented.
FBP is a particular form of dataflow programming based on bounded buffers, information packets with defined lifetimes, named ports, and separate definition of connections."
You can think of it as creating a flowchart with each node performing a specific business operation. Each node is a black box and should be reusable.
So our goal was to transform iPhora Foundation to utilize a flow-based model of process automation. This turned out to be a much more daunting task than we originally imagined. Most flow-based solutions on the market use a simplified security model. Since many of the processes that our customers needed involved sensitive data, we wanted a higher level of security, where the operation of each node is allowed or disallowed based on workflow roles, which could be evaluated in real time based on a number of criteria.
We needed a framework that would allow us to create different types of nodes with defined business logic and make them available for users to build their processes. But how do you get these different nodes to communicate with one another? To handle this we created a persistent common bus that allowed the nodes to communicate and transfer data between themselves as long as the process was active. We decided that the communication pathway should be in the form of JSON objects. This allowed for a standard data packet structure that could handle any situation from the simple to the complex. Since we had already developed a LotusScript-based JSON parser, this was a no brainer.
We started with 5 basic nodes:
- Start
- Task
- Notification
- Conditional Branching
- Complete
With these nodes, business users can easily automate a basic workflow process. To handle more complex processes, we came up with dozens of additional nodes, including a node that can handle independent threaded processes called ActionStreams. (We will talk more about ActionStreams in a future blog post.)
We are constantly coming up with new nodes. Some of them require a low-code or pro-code developer to design and build, but the resulting node can then be consumed by business users in their own applications, giving them a wealth of business logic that they can include in their designs with no coding on their part.
The other part of the puzzle is the user interface. Our target audience of business users requires an intuitive, visual interface to assemble their processes. Luckily, we were able identify a great open source solution, mxGraph, that provided the visualization that was needed. If you have ever used a great little product called draw.io to layout a flow diagram, it uses the mxGraph library.
But setting the goals and selecting the tools was the easy part. Creating an effective, easy-to-use interface to our node processing engine was the majority of the battle. After many tries and redesigns, we were successful, and this design is now incorporated into our iPhora AppBuilder solution.
Flow-based programming is a powerful methodology for creating applications. It forces you to think in terms of reusable independent building blocks, and the result is reduced development time and less need for skilled developers..
Next time, we will take flow-based programming to a different level, The iPhora Journey - Part 9 - Flow-based UI Programming.
Comments