kontrawize

Converting a MagicDraw UML Model into Enterprise Architect format

I like UML for modelling, in spite of it being too focussed on object-oriented design.  However, interoperability between UML file formats is woeful, even though there is a common XML format for UML, called XMI.  Unfortunately, each UML tool vendor has found a different, incompatible way of using XMI, it seems.  It really holds back UML tools; we live in a world where people now expect interoperability.  The interoperability between UML tools is vastly less than, for example, the interoperability between OpenOffice and MS Office.

However, the "300lb Gorilla" for many vendors is Eclipse, which has its own built-in UML editor.  While the Eclipse implementation doesn't have the same degree of development and features as the commercial tools (although making great strides), the commercial vendors can't ignore it because so many organisations use Eclipse now as their primary Java IDE.  This means that many UML tools can import and export Eclipse ".uml" and/or ".uml2" files, so the Eclipse format was recently recommended to me as a better way (than XMI hacking) for moving model between different UML tools.

This morning, I tried this, using the ISO 20022:2009 draft profile as the example.  This contains both the ISO 20022 profile (stereotypes) and the ISO 20022 type library.  From MagicDraw, I opened "unifi-profile.mdxml" and then selected "File | Export To | EMF UML2 (v2.x) XMI File".

 

I saved to an empty directory, where it created 13 ".uml" files, including "unifi-profile.uml".  That's one thing about the Eclipse format, you end up with lots of files, not a single "project" or "model" file like other UML tools.

I then started Enterprise Architect (EA) and created a new, empty model.  In the project browser, I right-clicked on the top level "Model" node and selected "Import Model from XMI...".  This opens the "Import Package from XMI" dialog, which contains a button "Import EMX / UML2 Files".  Pressing that button gets you to an alternative dialog which allows you to import your Eclipse model.

However, when I went to import "unifi-profile.uml", I couldn't because EA expected UML2 files to have a ".uml2" extension, while MagicDraw saves them with a ".uml" extension.  Luckily, the solution was as easy as renaming "unifi-profile.uml" to be "unifi-profile.uml2" (note: without renaming any of the other ".uml" files).

Did it work?  Well, partially.  It did import the type library, which is good.  A couple of bogus double-entries were created, but these were easy to find as they didn't have stereotypes like the actual type library members.  EA didn't import the actual profile (stereotypes), which wasn't a big a surprise to me.  EA doesn't yet support the full range of UML 2.x metaclasses (these represent UML constructs - classes, attributes, associations, etc.), while MagicDraw does.  That's a pity, it means I will have to construct the profile by hand, as step towards providing EA support in the Londata UML Message Generator.

Generating XML With Scala

As I mentioned in "Generating XML Schemas from ISO 20022 UML models", I am using Scala to write an implementation of the ISO 20022 method for generating XML Schemas from a UML model.  The code is in two parts; the first part processes a MagicDraw UML model via the MagicDraw Open API (a Java API) and produces a simple XML representation of the ISO 20022 message information that is contained in the UML model.  That simple XML representation can then be processed to produce resources such as the XML Schemas; in a later iteration, I also plan to support generation of ASN.1 specifications for the same messages.  ASN.1 is the way that ISO 20022 messages will be able to be binary encoded rather than XML encoded, for situations where volumes require something more compact than textual XML. At the moment, I have just finished writing the code for generating the XML Schemas from the simple XML representation (having previously written the code to generate the simple XML representation from a MagicDraw UML model).

Back to Scala and XML, a major reason for using Scala rather than Java for this code is that Scala has better built-in support for XML. In spite of some really top XML people having worked at Sun in the last decade, the XML facilities in Java have never been more than an awkward tacked-on piece that continues to make Java developers resistant to using XML.  Scala goes a long way to addressing that problem, and making XML straightforward to work with.

For example, in Scala I can write

val x = <document><h1>Example of inline XML in Scala code</h1></document>

which creates a value "x" of type "scala.xml.Node". Note that I didn't have to put the XML in quotes to convert it into a string.  Scala knows what XML is, so you can use XML directly.

If you need to insert values into the XML, you can do it like this:

val famousText = "To be or not to be"
val speaker = "Hamlet"
val famousPhrase = <phrase speaker={speaker}>{famousText}</phrase>

and the value of "famousPhrase" is

<phrase speaker="Hamlet">To be or not to be</phrase>

Braces are used to indicate where something in the XML should be intepreted as code.  For attributes, if the attribute value is contained in braces, the enclosing quotes are omitted.  Some of you may think that this looks not unlike ASP or JSP, and in some ways it does, but it is neatly integrated into Scala.  These expressions can appear in any Scala code, they aren't forced into separate files as you typically have with ASP and JSP.  Also, the resultant expressions are understood by the code to be XML, not just text with angle brackets.  I can extract values from the XML using a path notation:

val extractedSpeakerAttr = famousPhrase \ "@speaker"
val extractedSpeaker = extractedSpeakerAttr.text

The result of this is that "extractedSpeakerAttr" is an XML node sequence with one node, an attribute; "extractedSpeaker" is a string with the value "Hamlet".

This is why I like Scala for working with XML. It's a full general purpose programming language, like Java/C#/etc., but the code you write for working with XML is short and to the point, comparable to XQuery. For my XML Schema generator, it seems like the best solution. Brevity is still one of the best ways to keep your code maintainable, and to keep the cost of that maintenance down to a minimum.

By the way, if you want to try out these Scala example, just download and install Scala, and type "scala" at the command line. That starts the Scala interpreter; you can type the above code, and it will print out the results. It's an easy way to get a feel for Scala and what it can do.

Generating XML Schemas from ISO 20022 UML models

As I've mentioned in the past, I'm the principal UK expert (yes, the "PUKE") for ISO TC68/WG4, the ISO working group that has produced an updated version of the ISO 20022 standard. ISO 20022 is standard for defining messages based on definitions in a central repository, with a focus on banking and finance. Those of you whose are involved with financial messaging via the SWIFT network will have used ISO 20022 messages whenever you use one of the XML "MX" series messages. WG4 has produced a 2009 version of ISO 20022; it is currently a committee draft which has gone to TC68 for an international vote, so it is not yet an approved version; the current approved version is the original 2004 version.

My particular interest in ISO 20022 is in the part that describes how to generate XML Schemas from UML models. I like the repository based approach where you can share components between related messages, but don't have to implement that sharing at the Schema level, with includes and imports. ISO 20022 has one Schema file per message, and that makes life easier than if you have to get people to set up a hierarchical directory of cross-linked Schemas correctly.

I wrote an XML Schema generator for the current 2004 version of ISO 20022. That generator works with XMI (UML in XML) files produced by Enterprise Architect (from Sparx Systems), and is implemented as a series of XSLT scripts.

For the new 2009 version of ISO 20022, the good people at MagicDraw donated licences for WG4 to use their UML editor, and so MagicDraw is the UML editor that was used to develop the new metamodel and UML profile. For this reason, for my new implementation I decided to target MagicDraw (I might support other UML editors too, later, depends on how many requests I get).

I decided to continue with my previous approach, working from the XMI, this time using a mix of XQuery and XSLT. XMI is a rather generic XML format that can be hard to read and understand, so I consider it error-prone to try and generate XML Schemas from the XMI directly. Instead, I distill the XMI into a much simpler XML format which reflects the structure of the ISO 20022 metamodel, an "XML DSL". I then process that simpler XML file to generate Schemas or anything else. So, the plan was to use an XQuery to generate the simple XML, and then XSLT to generate the Schemas (there are some things that are easier in XSLT than XQuery).

However, it turns out that the latest XMI 2.1 standard is loose around how UML stereotypes (user annotations) are represented. So, after some failed attempts at "reverse engineering" how MagicDraw represents stereotypes in XMI, I decided I needed a different approach. Instead, I am using the MagicDraw Open API to read the contents of an ISO 20022 UML model. This means I need to be using a Java program, or something that runs on a JVM.

Java itself isn't great for XML, in as much as the code tends to end up rather more verbose and unreadable than it need be. I tried Groovy, which has some reasonable support for XML, but moved away from that because the XML support is incomplete. It seems that in a number of languages, the people who implemented the XML library didn't really like XML, so they decided to only implement support for the parts of XML that they thought were worthwhile. For users, that causes nothing but problems.

What I am currently using is Scala, which is convenient because you can just start typing XML as the value of a variable in the middle of a program, and Scala knows what to do. I've now written the code to generate the simple XML format. It hasn't been as easy as it should have been, because Scala support in IDEs is still weak (I use Intellij), and because Scala's integration with Java is good but not as good as it could or should be. For example, Scala avoids the use of null values. It does have some support for them because it needs that to work with Java libraries. However, at times when Java code would give me a null pointer exception, I find my Scala code just hangs, leaving me with a lot of searching to do to find what went wrong and where. Still, the code is more readable than Java, and things can only get better, so I'm planning to stick with Scala for this.

Once I have XML Schema generation done, the next thing to look at is ASN.1. ASN.1 ("abstract syntax notation") is an ISO standard for designing messages that predates XML, and has been most widely used in the telecommunications industry. Note that I wrote in a previous post that ASN.1 was the format used for ISO 15022 messages (more-or-less the predecessor to ISO 20022). That was wrong, I want to point out; somewhere along the line I got the wrong idea, and in spite of my saying it in a number of WG4 meetings, only recently did someone from SWIFT point out that I was mistaken.

Anyway, WG4 has added support in the 2009 version of ISO 20022 for ASN.1 as an alternate message syntax to XML Schema. Now, ASN.1 itself is actually multi-syntax. There are a couple of binary encodings it supports, plus there is a more recent XML encoding. The ISO 20022 profile of ASN.1 has been constructed so that, as closely as possible, an XML message that is valid according to the XML Schemas will also be valid as an XML encoded ASN.1 message, and vice-versa. Since ASN.1 messages also have a binary encoding, this new ASN.1 support means that ISO 20022 will have an official binary encoding as well as the existing XML encoding. It will be interesting to see how the usage of XML versus binary ISO 20022 messages goes over the next few years; banking and finance applications have size and latency challenges for which plain-text XML often isn't suitable.

I should be able to generate the ASN.1 grammars for the messages from the same simple XML that I use to generate the Schemas. Doing so will show that the simple XML genuinely decouples that back-end code generators from the format of the input UML model; that's something I need to have the possibility to support UML editors other than MagicDraw in the future.

If any of this interests you, either ISO 20022 or my work on Schema and other generators for ISO 20022, feel free to get in touch.

OASIS genericode 1.0 (Code List Representation, XML format) is available for second public review

OASIS has released the second committee draft of genericode 1.0, the OASIS Code List Representation XML format, for the second public review. The review runs from today for 15 days. See the "Genericode News" section at

http://www.genericode.org/

for details. As I've mentioned previously, this is based on the XML code list format that I presented at the XML 2004 conference (there is a link to that paper at  http://www.genericode.org/).

Comments are very welcome, preferably posted to the OASIS Code List Representation TC's comments list (codelist-comment@lists.oasis-open.org). If you have any general questions about genericode, you can  mail them to me.

ISO 19312 Securities Data Model (and MDDL)

David Winter posted a comment, noting that he had been unable to find the ISO securities data model in the ISO Web site (that's financial securities – stocks/shares/etc.). The reason that it isn't there is that ISO 19312 is still a work in progress, it hasn't yet been approved (indeed, it hasn't yet started reached the ISO voting stage yet). As such, it isn't yet generally available, although you may find that you can get a copy of the current draft of the data model from your local ISO/standards representatives if you are involved in banking and finance.

David is currently looking at MDDL, which is the best place to be looking in the meantime. Indeed, ISO 19312 will only cover a subset (more or less) of what MDDL covers, at least initially. David seems a little perturbed or surprised that MDDL puts trading terminology (snap, time-series) at the top of its document hierarchy, but in my experience most surprises that people have about standards are usually due to people assuming different requirements to those that were used (or otherwise disagreeing with the requirements that were used). MDDL, for example, was originally designed for distributing financial market data (from stock exchanges and such), so it's not surprising that it talks about snaps (snapshots) and time-series, which are what people look at when they look at market data. MDDL isn't just a financial instrument data model, as ISO 19312 is (but note that ISO 19312 also covers corporate actions – changes to the details of securities because companies have merged, split, changed name, or because of other similar reasons). David considers MDDL to be inverted, so I guess the way he would have done the requirements would have been inverted, in some sense, as compared to the way they were done (and there's often no good or bad in these things, just choices that have to be made, one way or the other).

For anyone who is looking at MDDL as a data model, note that the most versions of MDDL have their data model defined in an XML file (e.g. “mddl-2.5-beta.xml”), which is a simple XML format constrained by a Schema (e.g “mddl-2.5-beta-schema.xsd”). The latest version of MDDL, 3.0-beta, comes with the data model in the form of a UML model (and documentation generated from that model is supplied with the release).

If you feel you must reverse-engineer a data model from the MDDL Schema, please use the “lite” or “base” version of the Schema, as the full Schema implements a number of MDDL-specific “ shorthand” structures which don't make sense if directly reverse-engineered as a data model.

OASIS genericode 1.0 (Code List Representation, XML format) is available for public review

OASIS has released the committee draft of genericode 1.0, the OASIS Code List Representation XML format, for public review.  The review runs until 16 July 2007.  See the "Genericode News" section at

http://www.genericode.org/

for details.  This is based on the XML code list format that I presented at the XML 2004 conference (there is a link to that paper at http://www.genericode.org/).

Comments are very welcome, preferably posted to the OASIS Code List Representation TC's comments list (codelist-comment@lists.oasis-open.org).  If you have any general questions about genericode, you can mail them to me.

Standards and Celtic Music in Dublin

I spent last week in Dublin at the Guinness Storehouse for the 10th UN/CEFACT Forum. UN/CEFACT develops standards for trade facilitation, i.e. making international trade simpler, easier, and cheaper. Electronic messaging standards are a big part of that.

The UN/CEFACT technology stack is broadly equivalent to ISO 20022 (UNIFI), although ISO 20022 is simpler and focussed on banking and finance, whereas UN/CEFACT's strength is commerce. The two areas of business overlap around payments. That is to say, when a large sum of money is paid from one company to another, it is actually done by sending instructions to the two companies' banks. If that sounds simple, well, it is when everything goes right. However, payments are a complex area because of the things that can go wrong, and managing what to do if something does go wrong.

A set of payments messages for ISO 20022 have recently been published, but the content of those messages has been translated into UN/CEFACT message components (Core Components and Business Information Entities, to use the jargon) so that equivalent messages can be created using the UN/CEFACT process and style. We don't know yet which style of message will become the preferred one for payments, but the outcome could affect the direction that messaging takes in banking and finance in the future, say in the next 5 years (certainly not in the next 6 months or even 1 year).

On a lighter topic, on my last night in Dublin I went on the "Musical Pub Crawl". It's a musical evening led by two professional Celtic musicians (drawn from a pool of those who happen to be in Dublin that evening), and takes place mainly in some private rooms in pubs in Dublin (it is a "crawl", so you have to walk from pub to pub a couple of times). The private rooms are needed because they play acoustic music, just the instruments and voices with no amplifiers and such. The evening I was there, we had Anthony as the guitarist, and Eugene playing the tenor (4-string) banjo (an instrument which was brought from America back to Ireland). Both were brilliant musicians, and they taught us about the different styles of Celtic music and some of the history and culture of Celtic music.

At the end of the evening, they asked the people on the tour to sing, and a request for an Australian meant that I was nominated by my UN/CEFACT colleagues. It guess it wasn't the worst rendition of "Waltzing Matilda" ever; I'm still waiting to see the video that someone took on their phone.

All in all, it was one of the best evenings I've had in ages, so if you find yourself in Dublin sometime, I highly recommend it.

UBL 2.0 is now an OASIS standard

UBL 2.0 (Universal Business Language) is now an OASIS standard. To quote the press release: UBL defines a royalty-free library of standard, electronic XML business documents such as purchase orders and invoices. UBL formats in electronic messages enable direct connection into existing business, legal, auditing, and records management practices, eliminating the re-keying of data in existing fax- and paper-based supply chains and providing an entry point into electronic commerce for small- and medium-sized businesses.

Denmark in particular has taken to UBL. The government there has been using it since version 0.7, and expects to save hundreds of millions of US dollars from managing its invoices electronically. See Mikkel Brun's XML 2006 presentation for more about this (note that the presentation and/or slides haven't been posted yet - but they should be soon).

My own contribution to UBL has been rather small - I joined a couple of years ago as I lobbied various groups to converge on a single approach to code lists. UBL 2.0 makes use of the draft genericode format that I created, and the OASIS Code List Representation TC is now working towards creating an approved 1.0 version of genericode (to be followed version 2.0, with 3.0 under discussion). Crane Softwrights already have some free tools to help people working with genericode, and they are also training people in how to use them (as part of their UBL training course).

A lot of people have put in a lot of work for UBL, and I'm glad to see UBL 2.0 make it to the status of OASIS standard. There's a lot more money that can be saved (even within Denmark, let alone in the entire rest of the world) by putting it to good use.

Time to review UBL 2

Version 2.0 of UBL, the Universal Business Language, is now available for public review. UBL's purpose is to provide a standard set of general business XML messages (e.g. purchase orders, invoices) which can eventually replace older standards like EDIFACT which tend to be much more expensive to implement. In particular, with EDIFACT, the tendency has been for large corporations (typically Fortune 500) to use it to set up different bilateral messages for use with their different trading partners. Small business just couldn't afford that, having different messages for each and every company they do business with.

UBL 2.0 uses (in a non-normative fashion) the genericode XML format that I proposed for code lists, which is now one of the key inputs for the new OASIS Code List Representation Technical Committee.

If you are interested in (general) business automation, you should have a look at UBL 2.0. Now is the time when you can provide your feedback, before 2.0 is signed off.

ISO TC68/WG4 & ISO 20022

I spent Monday and Tuesday at the first meeting (in London) of Working Group 4 of Technical Committee 68 of the International Standards Organisation (ISO TC68/WG4), discussing ISO 20022. But what do all these numbers mean?

TC68 is the ISO technical committee that is responsible for standards in the banking & finance area. ISO 20022 is a standard for generating business messages (in XML) based on a business model (in UML) of the processes, participants, and data that are involved in each section of banking & finance.

The foundations for ISO 20022 were laid in 2000, back in the early days of XML. The technology has moved on in the five years since, as has our understanding of how to get the best value out of the technology. The task of WG4 is to review ISO 20022 and see how the implementation can and should change to make better use the technologies and methodologies that are now available. It is still going to be producing W3C XML Schemas for XML messages, and the models will still be in UML. However, the way this happens should improve. WG4 is aiming to make the process more transparent, more repeatable, more objective, and less labour-intensive.

I'm really looking forwards to WG4. This is one of the most important pieces of work going on in the banking & finance area, in my opinion, because it is where we expect to see convergence of the different flavours of XML that currently exist in different banking & finance niches. People want integration, they want to be able to represent all data in a common way, not the least because companies are continually creating new financial products that are based on multiple existing products, and those new products require that the data for the existing products can be brought together in once place in a simple and consistent manner.

If we get it right, we have a lot to gain, and that is what makes it so important.

[ Technorati tags: data architecture, software architecture, XML ]