Blog

The DI Mess

[My August column on BPMInstitute.org]

BPMS Watch readers know I am a big fan of OMG’s Business Process Modeling Notation (BPMN) 2.0, which has passed its first approval hurdle and is now in the Finalization Task Force stage. A major reason is that for the first time, BPMN has standardized the schema for XML interchange of process models. That means you will be able to create a BPMN model in one tool with confidence you can open it in a different tool. I think that’s what every user expects from a “standard,” but BPMN never had it until v2.0. But there is one part of the standard that the team messed up big time: Diagram Interchange (DI), meaning the graphical layout of the shapes and symbols.

Prior to BPMN 2.0, the de facto interchange format for BPMN models has been XPDL, an existing standard from the Workflow Management Coalition (WfMC) modified to handle BPMN. Each node in XPDL represents both an element in the BPMN semantic model and its corresponding visual element – including position, size, and other graphical details – in the diagram. But BPMN 2.0 separates the semantic model from its diagrammatic representation. That’s in a separate model – the DI model – that sits alongside the semantic model in the XML interchange document.

That’s actually a good thing. It allows multiple diagrams (views, pages, etc.) to represent the same semantic element in different ways, such as different levels of detail. For example, if you have a hierarchical diagram of a complex process model, with a top-level view showing the entire process at very coarse-grained detail and many child-level views showing increasing levels of detail, BPMN 2.0 maintains a single consistent semantic model for the process, referenced by all the various views. That is good for maintaining the integrity of the model. In contrast, with XPDL each view of a particular model element represents a different XML element in the export, so if a subprocess or pool appears on multiple pages of the model, it is up to the tool – or the modeler – to maintain consistency between their definitions.

But OMG did not want to make DI, the graphical model, BPMN-specific. They wanted it to support other types of diagrams as well, particularly UML, something completely different but also owned by OMG. And they didn’t want to define the format in an XML schema document (XSD), the universal standard used by XML tools, SOA, and BPM. They wanted to define it in XMI (XML Model Interchange), OMG’s own standard for interchanging UML models. The problem with DI as specified in the BPMN 2.0 draft is that BPMN-specific diagram structure is not defined in a format usable by XML tools. Instead, it is defined in a separate “library,” which you could think of as a cheat sheet off to the side to remind you of the particular attributes allowed or required for BPMN-specific shapes.

If you think that is a strange way to define what is still nominally a notation (i.e. graphics) standard, you are not alone.

The problem is not that it is impossible to define a BPMN 2.0 model using the graphics as specified by DI, just that it will be too difficult to do so in practice. Unless you can represent DI as a true BPMN-specific schema, tool implementers cannot use the rich array of XML tools available to validate, transform, and test their XML exports. Graphical model interchange is difficult enough in XPDL, where the graphics come pre-attached to their semantic elements. A few BPMN 1.x tool vendors – ITP Commerce, Fujitsu, Global360, eClarus, TIBCO, BizAGI, and Lombardi – have achieved some level of BPMN interoperability using XPDL 2.1, but it requires tweaks to the XPDL using XSLT, the standard language for XML transformation. It would be extremely difficult to create those tweaks with DI – and they no doubt will still be needed with BPMN 2.0 – since tools that create, execute, and debug XSLT all need a proper schema. OMG needs to try again on the DI part.

It is no great feat to turn DI into a proper BPMN-specific schema. I did it myself (BpmnDi.xsd) in a few hours. I also created an XSLT that maps XPDL 2.1 to BPMN 2.0, using BpmnDi as the graphics model. If you are interested in such things, you can download from http://www.bpmnstyle.com on the Tools menu page. If your BPMN tool outputs XPDL 2.1, you can try it out on your own models. [So far only the XPDL "standard"-level conformance is supported.] If you are interested in interoperability between BPMN tools, please make your sentiments known to OMG. These implementation “details” are what the FTF phase is all about.

  1. MohamedMohamed08-06-2009

    Hi,
    I followed your attempt for mapping XPDL-2.1 to BPMN-2.0, but it seems fundamentally incorrect.

    For example, in the 4 examples you provided, representing start event, end event, gateways, etc, as instances of ?Activity? is semantically wrong wrt BPMN-2.0.

    In BPMN-1.x, there is no formal metamodel, so validating that kind of relationships is inconsistent, where it depends on how different vendors understand what these relationships should be.

    Now, with XMI metamodel of BPMN-2.0, these relationships are precisely defined. Therefore, events and gateways should NOT be represented as activity types anymore (as in your examples). In BPMN-2.0, Activity, Event, and Gateway are sub-classes of a concept called ?FlowNode?. In addition, they are disjoint concepts, meaning un-related.

    Thanks,

    Mohamed

  2. brucebruce08-06-2009

    Mohamed,
    It is possible that my mappings are incorrect, but I think you are on the wrong track. The xslt maps xpdl 2.1 to bpmn 2.0, including my improvised bpmndi.xsd to replace di.xsd with the M1 library. The source document for the transform is xpdl 2.1, which various BPMN tools can produce. In xpdl, events and gateways are types of “activities.” XPDL predated BPMN and didn’t start over in v2.0, just filled in the gaps. So the fact they call events and gateways activities is something you might take up with WfMC. When you transform the .xpdl file with the .xslt file, the resulting .bpmn file should conform to the BPMN 2.0 schema and metamodel (except for the modified graphics part, as described). Are you saying that it does not?
    –Bruce

  3. MohamedMohamed08-06-2009

    Bruce,
    My apology, yes, I was on the wrong track looking at *.xpdl files, instead of *.bpmn ones, and they look great.

    However, I still have some minor issues:

    In ?trivial-2.bpmn? example, there is the following subProcess

    Which does not have a corresponding in Visio diagram, can you please explain?

    In ?standardProcess.bpmn? example, ?Participant? and ?Process? named ?Standard Process? have the same id, where I think (could be wrong) they are different concepts. In BPMN-2.0 metamodel, ?Participant? has a reference to ?Process? called ?processRef?, so we could have a nested element in ?participant? referring to ?process?, instead of having them as identical.

    Thanks,

    Mohamed

  4. brucebruce08-06-2009

    trivial-2.vsd has the subprocess expansion on another tab of the Visio file. It is simply the sequence start event, task A1, task A2, end event.

    In all of the .bpmn files, the participant id should be the process id with the suffix ‘-p’. If the id’s are the same, it is a schema error. If participant has a processRef attribute, I agree I should include it in my xslt. XPDL has nothing equivalent to BPMN participant (XPDL participant is what BPMN calls resource.) Also, XPDL assigns an empty (black box) pool a process, where BPMN says it has no process. So my xslt takes XPDL process id for both the BPMN process and (with -p suffix) the participant.
    –Bruce

  5. SteveSteve08-07-2009

    Hi Bruce,

    WRT to pressuring OMG. Anyway we can organize something. I’m not a member but we are a client of Casewise who might have some weight. I also know someone who works with OMG on BPMN.

    Any thoughts ?

  6. brucebruce08-07-2009

    Steve,
    I would say simply that if you are concerned about DI, make your feelings known to the BPMN team at OMG. There is probably a formal procedure but I don’t know what that is. But I’m pretty sure that if you email ivana.trickovic@sap.com, you will get it to the right place. (That doesn’t mean they will act on it…) You should identify yourself and your particular concern.
    –Bruce

  7. Robert ShapiroRobert Shapiro08-08-2009

    While I agree with the general theme of Bruce’s blog, I would like to make a few additional points:

    1: XPDL provides a method for allowing multiple views of the same process model: the attribute ToolId in the graphics element for each model object. The specification does not spell out a systematic way of using this, and it should do a better job to support model portability; but then again, the BPMN2.0 spec doesn’t spell out how to use the multiple views either.
    2: We are working very hard to make portability work for the conformance classes we have developed for XPDL2.1, and certification in a particular portability class will insure that models can be moved without requiring any XSLT tinkering.
    3: BPMN2.0 fails the portability objective for another major reason: it includes so many details and new concepts, even in the ‘core’ elements, that vendors will end up supporting their own subsets and defeat the portability objective. Conformance class subsets are essential in this context. In fact Bruce and I have worked hard to develop a set of portability classes for XPDL2.2, which will include the process modeling capabilities of BPMN2.0.
    4: In my opinion, The finalization Task Force must face the issue of how to provide a BPMN2.0 specification that addresses the needs of business analysts, rather than IT professionals focused on providing all the details for enactment or execution of a process model. I’m willing to bet that if you took a survey of all the current BPMN models you would discover that the vast majority are ‘higher level’. If the FTF does nothing about this problem BPMN2.0 will ultimately not serve the process modeling community at large.
    5: I am on the FTF, along with some colleagues who support this perspective. We will need all the help we can get. If you agree with this please help us. I can be contacted at rshapiro@processanalytica.com.

  8. brucebruce08-08-2009

    Thanks Robert. I am glad you are on the BPMN FTF. I tried to bring the business analyst perspective to the IOS team in the drafting stage and basically got nowhere. It is possible things will change in FTF, but I suspect some other parallel effort is needed to achieve BPMN tool interoperability at the non-executable level, something like WS-I did for web services. I see things like your BPMN Google group, the XPDL 2.2 effort in WfMC, my http://www.bpmncase.com site, etc, as the informal precursors of such an effort.
    –Bruce

  9. MohamedMohamed08-08-2009

    One more feedback about mapping XPDL-2.1 to BPMN-2.0.

    In ?standardProcess.bpmn? example, regarding ?Open Mail? userTask and its 2 associated dataObjects.

    Is userTask missing 2 dataOutput?, then the 2 dataOutputAssociation would be referring to 2 dataOutput in ?sourceRef?, instead of the useTask itself. Something like this:
    <userTask id=”_29749b37-5ab0-4bbb-8b27-9d8d5ffc55dd” name=”Open Mail”>
      <ioSpecification>
        <dataOutput id=”_123″/>
        <dataOutput id=”_456″/> 
     
        <outputSet>
          <dataOutputRefs>_123</dataOutputRefs>
        </outputSet>
     
        <outputSet>
          <dataOutputRefs>_456</dataOutputRefs>
        </outputSet>
      </ioSpecification>
     
      <dataOutputAssociation id=”_ef59debe-c0ea-49c8-8c96-3d625d4f82ff”>
        <sourceRef>_123</sourceRef>
        <targetRef>_6b35b310-ad84-4c8c-854c-1cefd2655736</targetRef>
      </dataOutputAssociation>
      <dataOutputAssociation id=”_dc482d79-eb09-4246-bfc4-002af3a3cee6″>
        <sourceRef>_456</sourceRef>
        <targetRef>_a2370682-3fca-40f5-ad23-a25180931e6a</targetRef>
      </dataOutputAssociation>
    </userTask>
    The same is for ?Process Remittance? & ?Process Application? subProcesses, as it seems that they miss dataInput each.

    Does this make sense or I?m missing something?

    Thanks,

    Mohamed

  10. MohamedMohamed08-09-2009

    My previous comment included script example, but for some reason it does not show up:

    _123
    _6b35b310-ad84-4c8c-854c-1cefd2655736

    _456
    _a2370682-3fca-40f5-ad23-a25180931e6a

  11. brucebruce08-09-2009

    Mohamed,
    I’m glad you are working through the examples! I wish more people did. Your comment touches on another important subject that I have yet to write much about, which is separating the BPMN elements into those required in non-executable models vs. the BPMN team’s “default” of executable models. In BPMN 1.x, data flow was an “artifact,” meaning decoration of the diagram with no process semantics. In BPMN 2.0, it was promoted to first-class semantic element, but that is really to support BPMN as an executable design language.

    In my view, dataInput and dataOutput elements (children of activity or event) relate to executable design, not non-executable modeling. They are children of the ioSpecification element, which is executable design detail. dataOutput allows the dataAssociation to represent the mapping , i.e., transformation, of that data to a stored variable (the data object), and dataInput allows another mapping from the variable to the input of the target activity or event.

    In BPMN 1.x — and in non-executable models generally — the modeler’s intent is not signify XPath or XSLT or Java mapping in data flow; it is just to indicate that this data or document is passed to the next step. While the mapping elements (assignment or transformation) are not required in the xsd, without them dataInput and dataOutput are really redundant. A dataOutputAssociation child of an activity implies the source is the dataOutput; if you do not specify the mapping, you should not have to add the dataOutput element in the serialization. More important, most tools for non-executable modeling don’t provide the equivalent of dataInput or dataOutput, so they would just have to add extra stuff to the export that adds zero value.

    As you correctly indicate, the sourceRef (required) for a dataOutputAssociation out of an activity is, according to the draft xsd, the id of the dataOutput not that of the activity itself. So on that score my serialization violates the draft schema, I concede. In the BPMN drafting, I argued that this forced execution-related elements into non-executable models. This was acknowledged by the team and deferred to FTF. The options we discussed were either allowing the dataOutputAssociation sourceRef to point to the activity itself (instead of its grandchild dataOutput element) or to allow the sourceRef to be optional, since the source is obviously implied. (And of course, all the same issues for data inputs…) I don’t know the current status of this issue in FTF.

  12. Robert ShapiroRobert Shapiro10-10-2009

    The Nov 2009 WfMC meeting will be a Thought Leader Summit in Maidenhead, England, just outside of London. Nov 4: discussion of BPMN 2.0 Model Interchange serialization and conformance classes. Headed by Robert Shapiro who has been leading the effort to define a standard way to exchange BPMN Diagrams. There are some tough decisions to be made about what should and should not be included, and this will be a chance for experts in BPMN modeling to meet and discuss this. See http://wfmc.org/november-member-meeting.html .

  13. scottscott07-06-2010

    Bruce,
    are the examples still available? I have trouble getting them to show up via the links…

    thanks!
    scott

  14. brucebruce07-06-2010

    The links might be broken but in any case, OMG did the right thing in the end, more or less, with DI. There is a real schema. It requires a lot of different namespace prefixes, which is quite annoying for xslt developers, but it was a necessary concession to the metamodel maniacs at OMG. Thanks should be given to Denis Gagne and Robert Shapiro for making it right in the end. I’ll try to dig up and post the final BPMNDI.xsd.

Leave a Reply


− two = 1