kontrawize

Never

People say "never say never!" But if you never say never, you can never say "never say never".

Right?

How many parents do you have?

There is an awful lot of information in the world that fits neatly into a tree hierarchy, where every item has a single parent item. That is one of the reasons why XML has proven to be so popular. On the other hand, some data just doesn't fit neatly into single-parent trees, because some items have multiple parents. Where this becomes problematic is when you have data where most items have a single parent, but some items have multiple parents.

Where I see this coming up a lot is in geography. In most cases:

• continents contain countries;
• countries contain cities;
• continents contains cities (combining the previous two rules).

and so people try to define data models with a continent/country/city tree hierarchy. Unfortunately, this example fails because

• Russia spans Europe and Asia;
• Turkey spans Europe and Asia;
• Part of Istanbul is in Europe, the other part is in Asia.

It's not that there are such of lot of these special cases, but enough to make a single-parent tree hierarchy useless for representing this data. Being able to represent 95% of the data, or even 99% of the data, isn't enough here. You need to be able to represent it all.

So be careful about modelling data using single-parent tree hierarchies. If there is any doubt at all about whether each item has one and only one parent item, then allow for a multi-parent hierarchy. Although these are harder to model and harder to process, sometimes you just have to bit that bullet. The world is as it is, after all.

[ Technorati tags: data architecture, software architecture, databases, XML ]

Maven 2.0 beta 1 is available

I've written a number of posts about Apache Maven. I've been using Maven 2.0, even though it was in alpha release, because it has some fundamental changes from Maven 1.x, so I didn't want to invest the time in Maven 1.x. The release of the first beta is great; it certainly makes me feel more comfortable about being an early adopter of 2.0. That said, "alpha 3" did things well enough (just not as many things as Maven 1.x does), so I'm not unhappy by any means. I look forwards to when there will the first full release of Maven 2.0, and I'm guessing that a 2005 release date is a real possibility now. Excellent news.

[ Technorati tags: Java, version management ]

Opera is free!

I'm a fervent user of the Opera Web browser. I've even paid for licences. It's fast, it's slick, but particularly I like the built-in e-mail client, "M2" (which also reads RSS/Atom blog feeds).

The big news with the release of Opera 8.5 is that Opera is now free! So if you haven't tried Opera before, I recommend giving it a try. I don't expect the Firefox users of the world to change over in great numbers, but if you are still using Internet Explorer, you may be seduced.

Data models without data

Tim Bray gave the closing keynote at the XML 2004 conference, and he made a point that keeps coming back to me over and over again. I don't remember his exact words, but to paraphrase him, you can't do a good job of modelling data without looking at the real-life data that you are modelling. The reason these words keep coming back to me is because I'm involve in a lot of modelling work, including standards work, and I rarely see (enough) real-life data entering into the modelling process.

Let's look at how many different ways there are to go about data modelling without looking at the real-life data that you are modelling.

• Writing an XML schema without having written prototypical XML documents;
• Writing a UML class model without having written written prototypical object graphs;
• Writing Java/C#/C++/etc. classes without having written prototypical object graphs;
• Writing a relational database schema without having written protypical data tables.

Of all of these, XML is in the best shape. Some people do write prototypical XML documents before writing a matching schema. Also, if you suggest to people that they should work this way, they usually agree with you. I have to say that since I moved more from Java to XML as the focus of what I do, I feel much more in touch with the data than I ever did.

In principle, UML shouldn't be in such a bad position, because you can create object (instance) diagrams into which you can put real data. That allows you to validate that your UML classes really can represent the data that you are trying to represent. However, I rarely see anybody creating such object diagrams. Some UML tools, including the most widely used, make it hard to create UML object diagrams, which doesn't help.

Databases also shouldn't be so bad. You can mock up some tables and fill them with real data to check that your data schema is going to work. Like XML specialists, database specialists tend to be in touch with real-life data.

I think it is the Java/C#/C++/etc. developers who are in the worst position. They have languages and tools for creating classes, but no matching tools for testing what data looks like when stored in those classes. The closest they usually get is by examing object contents in a debugger, and that gives you a very narrow view of what is in your data, compared to looking at a database table or (better) an XML document. I feel very strongly that this lack of object manipulation tools to match the usual class manipulation tools contributes to poor design of object-oriented class models. I'm sure most object-oriented modellers and designers don't currently perceive a problem with the tools they have, but I'm also sure they aren't comparing their situation what happens in the XML and database worlds.

We all know the importance of regression testing for software. If you don't have XML documents to match your XML schema, you can't do regression testing of the impact of schema changes. So it is common to have such a test suite of XML documents for an important Schema.

However, few UML models contain object models that can be used to provide the same regression testing as the model changes. Few Java/C#/C++/etc. class models have matching libraries of object instances (e.g. serialised objects in a text/XML/binary format) that can be used to provide regression testing of those models. That has to change. If you can't regression test a model to understand the impact of changes, are you really doing your modelling job properly? I don't think so. If regression testing is a necessary part of software development, it must also be a necessary part of data modelling. There is simply no other way.

[ Technorati tags: data architecture, software architecture, databases, Java, XML ]

Core Components

This week I am in Brussels at my first meeting of the UN/CEFACT TMG (Techniques and Methodologies Group), which has been working since 2000 to produce a methodology for creating data models where the components are easy to re-use across companies and across industries. This is nowhere near as easy as it sounds; something as apparently trivial as different naming conventions can stop you from re-using parts of an external data model in your own data model, and that's just one of the issues.

In particular, the meeting I am attending is for the CCTS (Core Components Technical Specification), which is available both as part 8 of ebXML and as part 5 of ISO 15000. If you want to download a copy, I suggest you get it via ebXML, as ISO charges for standards. CCTS is about how you represent data models in a format that is independent of any implemenation or syntax. The modelling approach needs to allow you to just as easily generate database schemas, object-oriented language classes (Java/C#/C++/etc.), W3C XML Schemas, or any other data representation, from the source data model.

CCTS consists of several pieces. One is a data model for data models (aka a meta-model). This is at the same level as UML, but more restrictive, so that there are fewer ways of representing the same thing. This is similar to the approach taken by ISO 20022 (and it has been mooted that ISO and UN/CEFACT will merge these approaches in the future, but I couldn't give you a guess as to when). Another piece of CCTS is a convention for naming, based on part 5 (Naming and identification principles for data elements) of ISO 11179 (Information technology - Specification and standardization of data elements). CCTS names, like ISO 11179 names, are very mechanical in style, and take some getting used to. However, because they are so mechanical, you spend less time creating them, and there are fewer ways of coming up with different names for the same thing; that is a big win for large projects involving lots of naming.

So if you have big data models to build, and if sharing those models, or re-using external models, is important to you, you should definitely take a look at CCTS and the other work of the UN/CEFACT TMG. As standards go, the UN is no less important than ISO, so expect to see goverments picking this stuff up, and companies following so they can keep or win government business. If we can get more re-use out of data models, we have some chance to get more re-use out of software than we do today, i.e. real re-use, signficantly less re-writing. We can't just type faster to get software out quickly and efficiently, we need to simplify the way we build it, and data models are a big part of that.

[ Technorati tags: standards, data architecture, software architecture, databases, Java, XML ]

Maven repositories

I mentioned in a previous post that Apache Maven (a software build management tool) manages version dependencies between software by creating a repository of JAR files in which version information is embedded in the file name of each JAR, e.g. "junit-3.8.1.jar". In particular, I noted that sites which use repository tools that provide good configuration management won't want this kind of solution, because they can control which version of "junit.jar" each user sees in their file system view, and adding explicit version information to the file names just impedes the proper use of this configuration management.

However, looking through some of the development issue logs for Maven 2 (version 2 is currently available as an alpha), what I've managed to pick up is that this explicit versioning of file names is specific to file: (file system) and http: (Web) implementations of Maven repositories. There should be provision to plug-in alternative repositories based on version control systems. With these alternative repositories, the version of "junit.jar" would be selected by looking for the version with the appropriate tag/label, perhaps something like "junit/junit/3.8.1" (most version control systems allow you to tag/label particular versions by name, so you can locate them easily later). There would be no need for filenames to contain version information, and enterprise sites would certainly be happier.

That said, I haven't yet tracked down the specific piece of work in the Maven schedule that will provide this. However, it fits so neatly into the way Maven works now, I hope we will see it sooner rather than later. It will remove the one major objection to Maven that I have encountered to date, the explicit versioning of file names, and that can only be a good thing all round.

[ Technorati tags: Java, version management ]

You need a new resolution!

XML often refers to files using URLs. An XML document will refer to a schema or stylesheet using a URL. However, you don't always want to retrieve these schemas or stylesheets from the Web. For performance and security reasons, it's better to recognise the URL, and then just use your own local copy of the schema or stylesheet.

The OASIS Entity Resolution Technical Committee (which includes yours truly) has been looking at this problem for some years, and has produced the XML Catalog specification. A vote has just opened to make version 1.1 into a full OASIS specification. XML Catalogs are already widely used, so the vote is just to seal their status and encourage an even broader take-up.

I was going to write more, but the TC chair, Lauren Wood, has already written a great piece about it all, so I recommend you just read her article. Keep in mind that when you need to replace a document at some external URL with a local copy of your own, there is a way to do it, and that way is XML Catalog.

[ Technorati tags: XML, standards ]

Unit Tests for XML

Most of my consulting work revolves around XML in some way or another. Much of the code that I write isn't Java/C#/C++/Python, it's XSLT stylesheets. So a lot of the stuff I need to write tests for is XML based.

Now, when I am writing tests for Java, I do what most Java developers do, and use JUnit, which provides a framework for running tests on Java classes (for C#, NUnit provides similar functionality). JUnit will run my tests for me, and display the results graphically, so that I can see how many of my tests passed/failed, and which ones failed. This is a great way to manage your regression testing of existing code, or unit testing of new code.

If I use Ant or Maven to for my Java builds (you can use NAnt for C#), I can get either to run my JUnit tests and produce a report of the results. It's important not to underrate just how important it is to have good reporting tools.

A few years back, when XML databases were coming on to the market, I spent more than a bit of time discussing the pros and cons of them with database administrators. In the XML world, we were excited that XML databases would perform better than relational databases for data that was more tree-like than tabular. However, we rarely got to the subject of performance. The first question was usually something like "so tell me about the reporting tools for this new-fangled XML database of yours". I never got past that argument. Replacing a database might take some effort, but having to replace your reporting tools as well can make the effort 5 times greater. Reporting tools are not something you want to write, but they are something you won't want to live without, particularly if you work on large projects.

Where does all this fit in with XML/XSLT testing? Well, the thing that I want for my XML testing is reporting. I want to be able to run tests, e.g. run some XML through an XSLT stylesheet and compare the output to an expected result. Once those tests are run, I want a report of how many of the tests were successful, which ones were not successful, and why those failing tests were not successful. To do that, I need some kind of framework, like Ant/Maven/Junit (or NAnt/NUnit) give you.

The conclusion that I've recently come to is that the framework I have to use is Ant/Maven/JUnit or NAnt/NUnit, just because there isn't a good alternative around that provides the test result reporting that I'm after. This means that I need to run my XSLT transformations from within Java or C#, and have suitable JUnit/NUnit tests to check the results. It's more code that XML/XSLT developers often want to buy into, but it's a lot less code that writing your own reporting engine in XSLT ...

So I am going to be playing around with how to get this to work best, and seeing what is out there to help me. Right now I have my eye on <XMLUnit/> and Schema Unit Test (SUT) as the two JUnit-for-XML utilities that can help me. I have just signed up with SUT as a developer. They have a JUnit test suite class that can turn a single XML file into a suite of XML examples to test a W3C XML Schema, which is the kind of thing I need for regression testing on Schemas. At the moment, they usually run the JUnit GUI directly, so I will be checking that it runs within Ant and Maven, and tweaking it as required to get it to work. So, expect to hear some more from me about this over the next weeks.

[ Technorati tags: XML, Java ]

Why Skype, why now?

Well, you've surely heard that eBay has bought Skype. It was only recently that I suggested that Skype needed a public perception if it was to compete with Google and others. Well, public perceptions don't come much better than eBay's. What brought this home to me last night was watching a UK comedy programme on TV where the comedian did a routine based on his buyer feedback on eBay. When people can use your service as the basis for a comedy sketch, you know that you've hit the big time.

The price surprised me. I wouldn't have thought that Skype should cost billions of US dollars. However, I'm starting to see where that price comes from. After all, if you are eBay, and you want to add a voice/text chat service, you want a good one. The major competitors to Skype are, arguably, Yahoo, AOL, Microsoft, and perhaps Google. The price of any other those would have been vastly more than eBay payed for Skype, if they could have been purchased at all.

Why didn't eBay try instead to license Skype's technology for what might have been a much smaller sum? After all, Skype's user base probably doesn't add much to eBay's, so that wouldn't be a strong reason for buying the company rather than just the technology. However, Skype's agreements with telcos across the globe to allow calls in and out of Skype are something the eBay would have had to do from scratch if it had bought the technology only, and I suspect that was a significant part of the value equation.

Anyway, Skype is suddenly a big player in all senses. Whether that means I'm going to get a Skype video plug-in for Linux any sooner is another matter.