In part 3 of this series, we have seen how Services, APIs and Microservices can “break the monolith” and accelerate innovation by making it easier to onboard and decommission “applications”.
In the proposed architecture, a new application’s System of Record (SoR) can be onboarded behind the service layer to achieve consistency with existing SoRs. New services can be added as the application supports reaching new outcomes and finally, the API layer exposes new application specific APIs which can reuse existing services or new services.
Image may be NSFW.
Clik here to view.
In part 2, we explained how functional HTML and in particular the SAM pattern help minimize the number of services and apis that feed your front-end because they are application specific rather than view specific APIs. Further, when you use the SAM pattern you can align your actions with the APIs and the Model’s units of work with your services.
However, there is two last problems with the proposed architecture that need to be addressed.
The first one is, how do you orchestrate the consistency of your information system? A simple “change of Address” outcome can easily spiral into a complex orchestration. The server-side version of the SAM pattern can also help here: I have published a simple library for both Java and JavaScript that allows you to implement robust orchestrations in a language and with semantics familiar to most developers.
The second problem is the proliferation of interfaces between APIs and Services. In a typical IT organization this problem can quickly spin out of control and become a major delivery bottleneck. That problem manifests itself in the design and the mediation between services and APIs. That problem has been around for a good 20 years, and has become excruciatingly painful in the last 5 years as the number of APIs has rapidly increased to support omnichannel applications.
The pundits would advise you to ignore that problem and just bite the bullet. Well, just like most problems in IT, ignoring it generally turns into a deadly trap. I have worked for organizations where even something as simple as an address structure was not standardized across the service interface. Every developer was free to reinvent an address structure as they were designing a new interface (sigh).
I have also worked for organizations that spent an excruciating amount of time creating and maintaining a “Common Information Model”, generally using UML or an industry standard such ACORD XML. To be frank, from the data points I have collected, first hand, using a UML based CIM is worse than not using a CIM at all. My allergy to UML has reach its ultimate stage, and I would never advise using UML for anything other than visualization. This was a great idea on paper, but UML semantics simply make no sense outside the very narrow context of OOP. UML is an M2 technology that people constantly try to use as an M3, as long as this will happen, it will continue generating lots of pain.
So how do you address this problem? Back in 2009, I published a paper on InfoQ detailing a new architecture for “message formats”. After 7 years, we have developed at xgen.io a solution that implements this architecture. I believe this work will change the way we architect applications because it removes the last bottleneck for delivering a true “composite application” paradigm, i.e. an application that relies on Services, APIs and Microservices.
Xtensible Solutions has done some very impressive work on establishing a CIM capability in the enterprise and it really costs me to criticize it, but it would not be fair, recommending using UML and/or XML Schema to manage your CIM. They are the problem, they are the fundamental reason why the concept of a CIM has not delivered the value we thought it would.
I don’t want to do too much theory here, but both UML and XML Schema suffer from two simple, yet critical, flaws. First, you cannot version Classes and Complex Types. Of course, OOP never had such concepts, so it’s not surprising to see them critically missing from these modeling staples. In essence, the problem is as follows, when any artefact is expressed in reference to a particular Class or Type, you cannot specify which version/variant of the Class or Type you are referencing. So when you need to change the Class or Type to accommodate new requirements, you cannot because it would ripple throughout all its dependencies.
Second, these technologies do not allow you to create variants of a type. For instance when you design a schema that supports “Querying”, “Creating” and “Updating” a particular entity, these schemas would be variants of each other. Queries typically have a flexible multiplicity of the type properties. In a “Create” schema, the identity of the object would be removed from the schema (compared to the update schema). The typical fall back design strategy is to create schemas where all elements are optional (the common denominator).
Even Swagger has not addressed that problem and you can see that the “update” schema for the PetShop example makes all elements optional, which simply makes no sense at all:
Image may be NSFW.
Clik here to view.
So we have solved that problem and developed a tool that allows you to create and maintain (as in properly version) an Entity-Relationship Data Model (dubbed the Common Information Model) and create specific schemas as projections of this data model, using variants of the CIM types. This is a major step forward in building entreprise grade back-ends which can easily run up to the hundreds (if not thousands) of operations.
Image may be NSFW.
Clik here to view.
Xtensible Solutions notes a number of problems that can arise when using a CIM:
- Concern 1: the CIM is too large
- Concern 2: the CIM inhibits innovation
- Concern 3: the CIM is too slow
- Concern 4: The CIM is too abstract
- Concern 5: The CIM is not a best practice data model
- Concern 6: The CIM is too hard to implement
The tool that we have developed addresses all of this problems. The CIM can be as large as you need it to be, and contrary to popular belief, using our tools, message formats can be designed in minutes, precisely because they are expressed as a projection of the CIM. Our tool, combined with an Enterprise Data Model accelerates innovation.
The schemas produced by our tool are concrete (query, create, update), they are also “flat” thanks to a particular feature of our projection definitions. There is no longer the need for complex, and yes abstract, hierarchies inherited from industry standards.
Because of our precise versioning strategy we create a new level of best practices in managing an Enterprise Data Model.
Last, but not least, implementing a CIM with our tool starts with a single schema, there is no need for the CIM to be complete before you can create a schema. All you need are the entities needed for that particular schema.
Here is the general architecture supported by the tool:
Image may be NSFW.
Clik here to view.
Here are some “variants” sample showing how we can create concrete messages based on the “Account” entity. For instance an “UpdateAccountDetailRequest” from the Account entity, we make the “Profile” and “ShippingAddress” optional and we “subtract” the credit card information which cannot be updated via this message. As you can see new concrete message formats can be created in minutes based on the CIM:
Image may be NSFW.
Clik here to view.
Please contact us, if you’d like to get access to our tool.
We are very excited about this new capability, when you factor in the progress we have made in the last couple of years with the SAM pattern, Frameworks such as Angular2, as well as enterprise grade middleware such as Node.js, we have, today, all the ingredients to create a new OmniChannel Application Architecture.