kontrawize

Getting your XML into MarkLogic Server (with a focus in FpML), part 2

Earlier posts in the series

1. Getting started developing with MarkLogic Server (semi-structured database and app server)
2. Getting your XML into MarkLogic Server (with a focus in FpML)

(and check out the comments on these posts, there are some great ones)

In the previous post on developing with MarkLogic Server, I had got as far as using the simple 'Load' tab in the admin console web page to load FpML 4.7 XML Schemas into MarkLogic. In this post, I want to discuss how you view the contents of the database. As I mentioned earlier, databases can be all too easy a place to lose data, and that is because the tools focus more in specific queries than they do on general browsing. While it may not be possible to just "browse" a database containing a huge number of documents (i.e. a huge amount of data), a development setup often has much less content, and being able to browse the documents (in the manner of browsing files in a file system) is a very useful thing to be able to do.

A good place to start is by downloading the 'CQ' application from the MarkLogic developer site. This allows you to run XQueries on a MarkLogic database, but it also allows you to view what files are in your database. Note that while 'CQ' is a tool I would always recommend installing for development purposes, it isn't a tool you should leave installed on a live production system.

1. Download 'CQ' from 'http://developer.marklogic.com/code/cq'. Assuming you are running MarkLogic Server 4.1, download 'mark-logic-cq-4.1-1.zip'.
2. We will install 'CQ' into the existing 'Docs' database that is part of the default MarkLogic Server setup. This database uses a filesystem directory. Go to your MarkLogic Server installation directory, and find the 'Docs' subdirectory.
3. Extract the 'CQ' ZIP file into the 'Docs' directory, preserving the directory structure. This should create a 'cq' directory inside the 'Docs' directory.

It's that easy, 'CQ' is now installed. Let's test it. First, note what the URL is for your admin console. It will be port 8001; my admin console URLs start with 'http://localhost:8001/'. You probably won't have 'localhost', as by default MarkLogic uses the machine name. Make a note of your admin console domain name, and then use that domain to replace 'localhost' in this URL: 'http://localhost:8000/cq/'. You should now see the 'CQ' page:

Just above the 'Query results' section is a 'content-source' drop-down list. Select 'FpML-Schemas' as the content source, and the click the 'explore' link near the top of the page.

You should see results for 37 files appear in the lower section of the page. The exact URIs will depend on where the FpML XML Schemas were on your file system when you loaded them.

So, now you have a basic way to examine what documents are in your database. You can also browse the documents by running an XQuery in 'CQ'. In the list of 'Queries' at the right of the window, click on number 1 and replace the query with

xquery version "1.0";
(: list all docs in database :)
for $i in collection() return document-uri($i)

Make sure that the content source is 'FpML-Schemas', then press the 'TEXT' button. The URIs for all of the documents in the 'FpML-Schemas' database are listed.

Another way to browse the documents in your database is to create a WebDAV server for your database in MarkLogic Server. That will be the subject of the next post in this series.

Getting your XML into MarkLogic Server (with a focus in FpML)

In my previous blog post, I wrote about getting set up for doing development with MarkLogic Server, which is a database and app server for semi-structured data like XML. This time, I want to focus on getting content into MarkLogic Server. When you start developing with MarkLogic Server, or any similar database, you are likely to want to pre-load some information into it. In my case, I wanted to load FpML Schemas, validation rules and examples (FpML is the Financial Products Markup Language, http://www.fpml.org/). It's taken me longer than I expected to load these files the way that I want them in the database, as I get used to what approaches do and don't work with MarkLogic Server, so I want to discuss what I found by experience (and would welcome comments on anything I've done that could have been done more easily another way).

One thing about semi-structured databases is that when you load a block of information (e.g. a message or document), the content tends to stay together in the database (with some exceptions for huge documents). So, if you load an XML document into an XML database, you can extract the same document at a later time. This differs from what happens when you put semi-structured information into a relational database. In the case of FpML, if you use a tool to create an equivalent relational schema for you, you end up with a couple of thousand database tables, which is a bit of a nightmare for even experienced database developers to work with. Additionally, while the tools are able to "burst" your FpML message and populate the tables with the information, I'm not aware of any simple way to request that the database return to you all of the information from a particular message. That's sometimes an important thing to do — an FpML message typically represents the details of a trade of an OTC derivative, and there are real business reasons for wanting to retrieve all of those trade details together. The SQL required to do it is a significant and expensive piece of work to write by hand. Indeed, many teams don't bother, they store the original message separately to the database, because they know that the relational database is going to make hard work of it. While that kind of archiving is OK for ad-hoc auditing purposes, it doesn't help if one of your use cases is to be able to present screens showing the details for particular trades. Where there are good reasons to keep particular sets of related data items together and not have them shredded into separate tables, a semi-structured database can save a lot of development effort.

Here is what you get with an FpML release (they are free to download, but you have to register). I'm taking FpML 4.7 (2nd recommendation) as my example):

When unzipped, there are top-level 'documents', 'html', 'pdf' and 'xml' directories. We'll mainly be interested with the 'xml' directory. It contains the FpML XML Schemas, and its sub-directories contain FpML examples, including some invalid examples for negative testing. However, the other interesting directory for use with MarkLogic is 'html/validation-rules'. Here there are XQueries (*.xq) which implement extra validation rules for FpML, extra rules that XML Schemas can't check.

Before you import a large number of files into a database like MarkLogic, it's a good idea to think about how you want to organise things. One of my traditional criticisms of databases, and the way people use them, is that it can be too easy to "lose" documents or data, lose it in a sea of other information. With MarkLogic, documents are stored in 'Forests', and one or more forests can be part of a 'Database'. An app server uses particular databases. So app servers use databases, databases use forests, forests contain documents. However, there are some other things to think about too.

Every document in the database has a URI to identify it. These are sometimes proper URLs that start with 'http://...', but they can also be just paths like '/AAA/BBB/CCC' (so they aren't constrained to be proper URIs, in spite of what they are called in the documentation). Documents are organised in a hierarchy of directories, like a file system, but each document can also be part of any number of 'collections'. Collections allow you to group any set of files together, regardless of where they are located in the database. So, when importing files, it's worth spending a bit of time thinking about how you want the document URIs to be structured (i.e. how you want your database's directories to be structured), and also what collections you should create to group related files to each other (just to make it easy to find those files later, as a related group).

Additionally, there are relationships between databases. For each database, you can specify which database contains the XML Schemas for validating the XML (it can be the same database, by the way). Additionally, for each app server you can specify a database for 'modules' (XQueries), although you can also choose to store your XQueries in the file system where MarkLogic Server is installed. What this means is

• you need to decide whether your XML will be store in one or more forests, and in one or more databases;
• you need to decide whether your XML Schemas will be in the same database as your XML files, or a different one;
• you need to decide whether your modules (XQueries) will be stored in a database, or in the file system.

Let look at doing the following:

• creating a new forest and database for the FpML 4.7 XML Schemas;
• importing the FpML 4.7 Schemas into that database.

(In later posts, we will look at importing the other files, and doing some validation.)

In the MarkLogic admin console (usually port 8001), go to the 'Forests' section:

Click on the 'Create' tab, and create a new forest called 'FpML-Schemas'. You only need to fill in the name (for the purposes of this exercise), then you can press 'ok'.

You will now have an empty forest for the FpML XML Schemas. Now go to the 'Databases' section of the console:

Click on the 'Create' tab, and create a new database called 'FpML-Schemas' (it's OK to have the same name as the forest):

After you press 'ok', you will find that you are prompted to select one or more forests for the database:

Click on the 'Database->Forests' link and select 'FpML-Schemas' as the forest for the database (remember, we gave them both the same name):

You now have a database with a forest, ready for loading your FpML XML Schemas. The quickest way to load the FpML XML Schemas is to go to the admin page for the 'FpML-Schemas' database, and click on the 'Load' tab. Put in the path for your FpML 4.7 'xml' directory, and use '*.xsd' as the filter. This will load all of the XML Schemas in the 'xml' directory (but not in any sub-directories).

When you press 'ok', you are shown a page with the files that will be loaded, and the database URI that will be assigned to each. You should find that the URIs are identical to the file paths. Press 'ok' to start loading the Schemas. In a few moments, the FpML XML Schemas are loaded. Press 'ok' again. The status page for the 'FpML-Schemas' database should now show that you have some documents in it:

How do you see which XML Schemas have been loaded? How do you control the URIs that are assigned to them when they are loaded in the database? How do you put them in collections? These are things we will look at in the next installment.

Getting started developing with MarkLogic Server (semi-structured database and app server)

I spend much of my professional life working with XML.  I often design XML structures that are used to represent financial information as it is transmitted between computers, but I also spend quite a bit of time looking at tools to support this usage of XML.

To be honest, I'm often a bit horrified by how people set up their software to work with XML. An all too common setup for a system to read incoming XML is:

1. Compile the XML Schemas into Java or .NET using a Schema binding tool (problem #1 - the XML is your decoupling mechanism, to help isolate your systems and stop them from being too tightly coupled to each other, but when you compile a Schema, you tightly couple your code to the XML that is supposed to be your decoupling mechanism.  If you use Schema compilers and every change to your XML's structure over time causes you headaches, you might need to reconsider your use of a Schema binding tool);
2. Create a staging relational database, and write code to store the information from your Java/.NET objects into that database (problem #2 - it can be a lot of work to create a relational model of a hierarchical document, like an XML message.  It is also a lot of work to write the code to do the storage. Do you really want to spend that time? Also, people often model all of the XML in the staging database, even if it won't all be used. That means the system ends up coupled to all changes in the XML, even those that would otherwise not have impacted the system - not a good plan.);
3. Write some more code to copy the information from the staging database into a separate application database, often with a different structure (so the code that you have to write isn't trivial).

How would I do it differently?  It is much less effort to

1. Store the XML in an XML database (i.e. in a semi-structured database). In particular, use an XML database that can store arbitrary XML, not one that requires you to have a Schema for your XML;
2. Query information out of the XML database as your application requires it (typically using XQuery, although your XML database may also support JDBC/ODBC views of the information, to simplify integration with existing apps).

Notice the difference - no time spent writing staging database schemas, no time spent writing code to populate the staging database.  You simply write the queries that you need to get the information out of the XML database, and you should pull the minimum amount of data possible when doing that so that you minimise the coupling between the application and the XML.

In fact, it's worth noting that using an XQuery (or XPath) to pull the information from your XML database provides the same level of decoupling that SQL and views provides when working with a relational database; that use of a query to provide decoupling is so common now with relational databases that people think of it as always having been that way, but it wasn't.  There were software tools in the 80s that wrote C++ code that was tightly bound to the database schema, and that code had all the same problems as we have now with code generated from XML Schemas.  The relational database folks learned that you always need a SQL query to providing decoupling from the physical storage structure; the XML world has been a bit slower to realise the same thing, but it's an important lesson - use XQuery/XPath/XSLT to decouple yourself from the physical structure of your XML and you take a lot of pain out of your application development.

It can be hard to get people to move away from the relational database mindset, in spite of the extra costs of development.  However, various people in the finance world seem to be looking seriously now at semi-structured databases (which includes XML databases), and the one that gets mentioned most often is MarkLogic Server.  I should say that I know quite a few people now who work for MarkLogic (they've been hiring of late, and picking up some really top class people).

So, I've decided to set up a proper MarkLogic development environment for myself (I've only toyed with it in the past).  You can download a development version from the MarkLogic developer site, where there is also a lot of developer documentation.  It doesn't run on Macs yet, but it does run under Windows, Linux and Solaris.  I couldn't get it to install under Windows 7 (Vista is what is on the supported list), so I installed it under Ubuntu.  Ubuntu isn't officially supported but my friends at MarkLogic told me how to install it; if you want to know, let me know.  It's pretty easy, and it's working well for me.

With MarkLogic Server, your installation can run multiple app servers (HTTP, WebDAV, or XDBC [direct code connection]).  Each app server uses certain databases, and each databases uses one or more 'forests' of documents.  The documents are stored compressed on disk, so it doesn't waste space like some XML databases that store them on disk as uncompressed text.  The documents are automatically indexed, but you can also manually adjust how they are indexed (like any database, there is an art to doing that well).

The default setup has a 'Docs' app server which used a 'Documents' database, and these are what you should use in your initial experiments. The first thing you will want to do is load some documents and try out some queries. This can be a bit daunting, it isn't as obvious as it should be how you do this. I believe MarkLogic are working on improving this, but also it's because MarkLogic Server is a secure, industrial-strength database (used by the U.S. security services among others), and so you have to learn to negotiate the permissions. In general, the quick answer is that for your initial learning with MarkLogic Server, use your admin login for everything (i.e. while you are just experimenting, and don't have anything too valuable stored in the database). I spent too much time trying to set up a non-admin user for development work, but it isn't a good way to get to know MarkLogic, so don't make my mistake.

The 'Docs' app server uses the 'Documents' database. Get started by reading the admin guide and create a WebDAV server and an XDBC server for the 'Documents' database (it's pretty easy, I won't describe it here). This allows tools to connect to the database. In particular, I have two working now:

• MarkLogic has an Eclipse plugin that sets it up for XQuery development (available from the MarkLogic developer site). It runs with the Galileo 3.5 release of Eclipse, not with the latest Helios release (trust me, I tried, it doesn't). This gives you syntax support for XQuery, including a knowledge of MarkLogic-specific XQuery functions (MarkLogic Server extends XQuery in the same way PL/SQL or TransactSQL extend SQL - to turn it from a query language into a full programming language. However you can also run pure standard XQuery 1.0 if you want). You can run your XQueries from within Eclipse just as you would run a piece of code;
• oXygen's XML editor, a favourite of mine, also does a great job with MarkLogic Server. It can validate your XQueries using MarkLogic's validator (so it picks up knowledge of MarkLogic-specific functions, etc.), and it can run those queries on MarkLogic Server and return the results, just as easily as Eclipse.

Either of these tools is a great way of getting up and running with MarkLogic Server. You can also install 'CQ' (http://developer.marklogic.com/code/cq) which is a neat little Web UI for running XQueries. It remembers your queries, which is nice, but it doesn't have the syntax checking and colouring that Eclipse and oXygen provide. CQ is well worth installing, but a tool like Eclipse or oXygen is really necessary to be productive (and that is what this is all about, after all, being productive in development, avoiding the effort and cost of unnecessary development).

To load files in a database, go to the admin console (port 8001) and look at the details for that database. There is a 'Load' tab that is the simplest way to get content in. However, it doesn't do what I want, to load content from a whole directory tree while filtering on file names. I am working on an XQuery for that, I will post it later when it is working. Otherwise you can also use WebDAV to drag-and-drop files and directories into MarkLogic Server. I'm going with the XQuery because it will allow me to set things like permissions as the files are imported, and I think that will prove useful, if a bit more involved.

I will write more in the future. I have outlined what I'm doing now. Please feel free to post questions (I know I've skimmed over a lot of details). I will try to answer any questions, or I'll ask one of my MarkLogic friends if they have an answer.

Interrogate, Report, Act - a layered approach to validation & checking

Yesterday I attended a standards meeting (ISO TC68/WG4, ISO 20022, message modelling for banking & finance).  After the meeting, some of us went to a nearby pub for a drink and a chat.  While we were talking, we touched on the subject of validation, and I mentioned my personal preferred approach - "Interrogate, Report, Act".  It seems to resonate with the others, so I thought it might be worth mentioning here.

One thing I've done a few times for banking clients is to build systems for quality control and release of large sets of XML Schemas.  Just as you check code before a release - compile it, test it, run style checks over it - you also need to do the same kind of checking for XML Schemas.  You test that they are valid Schemas, you regression test them using XML examples, you run style checks over them, and perhaps also generate code or other artefacts from them.  What you do, whether it's for Java/C# or whether it's for XML Schemas, is to check some kind of IT resource using some set of rules.  You can call it validation, but that's just a long word for checking.

If you are checking XML documents using a set of structural/formatting rules defined by an XML Schema, you call it validation.  People are used to validators that check XML and log errors, just as they are used to compilers that check code and log errors.  It's very familiar, it's a common model for how such tools work.  However, I've found it to be a problem when I'm building XML Schema checking frameworks.  Why?

One reason is that, with checks like style checks, today's disallowed style is tomorrow's allowed style, and vice-versa.  As organisations develop their XML Schemas over time, they adjust their style checks.  Type extension or substitution groups might be banned today, they might be allowed tomorrow.  To build a flexible checking framework, you want to be able to change quickly the interpretation or the severity of a particular feature, of a particular style.  This is why I found it best to break things up into the three stages - "Interrogate, Report, Act".

Interrogate

Understand the resource(s) you are checking, the structure, the formatting, the style, whatever you need to understand. Don't make any judgements at this stage - the judgements are the things you need to be able to change, so you don't want to embed them into the interrogations.

Report

Put the results of your interrogations into a common reporting format. If you have XML Schema validation results, Schematron validation results, ad-hoc XQuery/XSLT validation results, and Schema compiler validation results (as an example), they won't all be in the same format. Some might be XML, others will be text. Get them into a common format for reporting. That's important so that you can slice and dice your interrogation results, and display them in a consistent way that gives developers, testers and managers the most appropriate summaries. It's also important so that you can easily add new results without impact on your presentation and drill-down code.

Act

Once all interrogation results are in a common reporting format, make your judgements and perform any consequent actions. Throw exceptions, log errors or warnings, or choose to ignore particular results if that is what the system's users have configured (there can be good reasons for ignoring particular test results for particular resources - but for sanity, make sure you capture the reasons for doing so in the configuration information).

This is the layering of concerns that has worked well for me. I've used it for XML and XML Schemas, but it is a general approach that can be applied to any kind of validation or checking process. By not making too early a judgement of thumbs up or thumbs down, you will have a more flexible checking framework that is more easily configurable and extensible than it would otherwise have been.

New version of ISO 20022:2009 UML to XML Schema generator available

Londata, my company, has made a new version of the Londata UML Message Generator available, version 1.0-preview-3.  This is (I believe) still the only available software that implements the new ISO 20022:2009 methodology for generating XML Schemas from UML models (as opposed to the older 2004 version of the standard).  ISO 20022 is the ISO standard for XML messages for banking and finance, as used on the SWIFT network.  ISO 20022 XML Schemas aren't hand-edited in an XML Schema editor as happens with some standards.  Rather, they are all generated from UML models so that there is consistency across the XML Schemas, and so that human errors are minimised.  The Londata UML Message Generator allows companies to use the same ISO methodology for their in-house XML Schema development and management.

Updated example UML models are now available - MagicDraw and Enterprise Architect example models, and Londata's conversion of the official MagicDraw UML profile into Enteprise Architect format.  Enterprise Architect isn't supported yet by the Londata UML Message Generator.  The Enterprise Architect examples were done to prove the feasibility of supporting Enterprise Architect for ISO 20022 modelling, MagicDraw having been used during the development of the 2009 version of the standard.

If you have any questions or comments, they are very welcome, please send them to [email protected]

Want to meet up in NY, 19-25 June? I have something to show you.

I haven't been to NY for a while, but I will be visiting from 19-25 June.  I will be attending the SIFMA Financial Services Technology Expo from 22-24 June, and also doing some meetings.

As well as visiting the Expo, I will have the latest version of the Londata UML to XML Schema generator, and would be happy to give demos.  The software implements the new 2009 version of the ISO 20022 process for generating XML Schemas from UML.  ISO 20022 is the XML format that SWIFT is increasingly using on its network, replacing the older ISO 15022 messages.  I'm happy to come and discuss why the methodology that works for SWIFT and ISO can work for your company too.

In addition to Londata's product, I will also be able to demo a companion product from Nomos. Nomos's tool reads business rules from your UML models and converts them into Java code that validates your XML messages.  This is validation that you can't do with XML Schemas, and for many companies it is the missing piece that is needed to catch errors that can otherwise lead to major delays in testing and roll-out of new systems.  I've seen first hand that if systems that send XML messages go into UAT with too many errors in the messages they are sending (and believe me, they all too often do), it has a huge impact on the testing schedule of the receiving systems, and nobody can really afford to have new systems and new business revenues delayed by weeks or months just because of inadequate checking on XML messages during the development/test/UAT cycle.  The Londata and Nomos tools together give companies a chance to address that problem and rein in their schedules.

If you would like to meet up while I am in NY, please feel free to get in touch; my e-mail is [email protected].  If 19-25 June doesn't suit, or you aren't in New York, please get in touch and we can set up an online discussion or demo to suit you.

How should XML formatting work?

I've been working with the Scala programming language for some months now, using it for processing XML.  When I generate XML, I need to have it formatted in an appropriate fashion, because

• clients expect the XML for be layed out nicely, and not look higgledy-piggledy, or
• I need to compare different versions of an XML file, and that is easiest if the formatting is consistent across versions.

I've written some thoughts about a general approach to XML formatting on the Scala Incubator Wiki, and I would appreciate it if you could have a look and add comments here if you have your own suggestions.  I wouldn't be surprised if someone is already doing what I am proposing.  If so, I would appreciate feedback on how well it works in practice.

My ultimate aim is to turn the proposal into code that can then be presented to the "scala-xml" mailing list, with the hope of getting it approved for inclusion in a future version of Scala.

Converting a MagicDraw UML Model into Enterprise Architect format

I like UML for modelling, in spite of it being too focussed on object-oriented design.  However, interoperability between UML file formats is woeful, even though there is a common XML format for UML, called XMI.  Unfortunately, each UML tool vendor has found a different, incompatible way of using XMI, it seems.  It really holds back UML tools; we live in a world where people now expect interoperability.  The interoperability between UML tools is vastly less than, for example, the interoperability between OpenOffice and MS Office.

However, the "300lb Gorilla" for many vendors is Eclipse, which has its own built-in UML editor.  While the Eclipse implementation doesn't have the same degree of development and features as the commercial tools (although making great strides), the commercial vendors can't ignore it because so many organisations use Eclipse now as their primary Java IDE.  This means that many UML tools can import and export Eclipse ".uml" and/or ".uml2" files, so the Eclipse format was recently recommended to me as a better way (than XMI hacking) for moving model between different UML tools.

This morning, I tried this, using the ISO 20022:2009 draft profile as the example.  This contains both the ISO 20022 profile (stereotypes) and the ISO 20022 type library.  From MagicDraw, I opened "unifi-profile.mdxml" and then selected "File | Export To | EMF UML2 (v2.x) XMI File".

 

I saved to an empty directory, where it created 13 ".uml" files, including "unifi-profile.uml".  That's one thing about the Eclipse format, you end up with lots of files, not a single "project" or "model" file like other UML tools.

I then started Enterprise Architect (EA) and created a new, empty model.  In the project browser, I right-clicked on the top level "Model" node and selected "Import Model from XMI...".  This opens the "Import Package from XMI" dialog, which contains a button "Import EMX / UML2 Files".  Pressing that button gets you to an alternative dialog which allows you to import your Eclipse model.

However, when I went to import "unifi-profile.uml", I couldn't because EA expected UML2 files to have a ".uml2" extension, while MagicDraw saves them with a ".uml" extension.  Luckily, the solution was as easy as renaming "unifi-profile.uml" to be "unifi-profile.uml2" (note: without renaming any of the other ".uml" files).

Did it work?  Well, partially.  It did import the type library, which is good.  A couple of bogus double-entries were created, but these were easy to find as they didn't have stereotypes like the actual type library members.  EA didn't import the actual profile (stereotypes), which wasn't a big a surprise to me.  EA doesn't yet support the full range of UML 2.x metaclasses (these represent UML constructs - classes, attributes, associations, etc.), while MagicDraw does.  That's a pity, it means I will have to construct the profile by hand, as step towards providing EA support in the Londata UML Message Generator.

Generating XML With Scala

As I mentioned in "Generating XML Schemas from ISO 20022 UML models", I am using Scala to write an implementation of the ISO 20022 method for generating XML Schemas from a UML model.  The code is in two parts; the first part processes a MagicDraw UML model via the MagicDraw Open API (a Java API) and produces a simple XML representation of the ISO 20022 message information that is contained in the UML model.  That simple XML representation can then be processed to produce resources such as the XML Schemas; in a later iteration, I also plan to support generation of ASN.1 specifications for the same messages.  ASN.1 is the way that ISO 20022 messages will be able to be binary encoded rather than XML encoded, for situations where volumes require something more compact than textual XML. At the moment, I have just finished writing the code for generating the XML Schemas from the simple XML representation (having previously written the code to generate the simple XML representation from a MagicDraw UML model).

Back to Scala and XML, a major reason for using Scala rather than Java for this code is that Scala has better built-in support for XML. In spite of some really top XML people having worked at Sun in the last decade, the XML facilities in Java have never been more than an awkward tacked-on piece that continues to make Java developers resistant to using XML.  Scala goes a long way to addressing that problem, and making XML straightforward to work with.

For example, in Scala I can write

val x = <document><h1>Example of inline XML in Scala code</h1></document>

which creates a value "x" of type "scala.xml.Node". Note that I didn't have to put the XML in quotes to convert it into a string.  Scala knows what XML is, so you can use XML directly.

If you need to insert values into the XML, you can do it like this:

val famousText = "To be or not to be"
val speaker = "Hamlet"
val famousPhrase = <phrase speaker={speaker}>{famousText}</phrase>

and the value of "famousPhrase" is

<phrase speaker="Hamlet">To be or not to be</phrase>

Braces are used to indicate where something in the XML should be intepreted as code.  For attributes, if the attribute value is contained in braces, the enclosing quotes are omitted.  Some of you may think that this looks not unlike ASP or JSP, and in some ways it does, but it is neatly integrated into Scala.  These expressions can appear in any Scala code, they aren't forced into separate files as you typically have with ASP and JSP.  Also, the resultant expressions are understood by the code to be XML, not just text with angle brackets.  I can extract values from the XML using a path notation:

val extractedSpeakerAttr = famousPhrase \ "@speaker"
val extractedSpeaker = extractedSpeakerAttr.text

The result of this is that "extractedSpeakerAttr" is an XML node sequence with one node, an attribute; "extractedSpeaker" is a string with the value "Hamlet".

This is why I like Scala for working with XML. It's a full general purpose programming language, like Java/C#/etc., but the code you write for working with XML is short and to the point, comparable to XQuery. For my XML Schema generator, it seems like the best solution. Brevity is still one of the best ways to keep your code maintainable, and to keep the cost of that maintenance down to a minimum.

By the way, if you want to try out these Scala example, just download and install Scala, and type "scala" at the command line. That starts the Scala interpreter; you can type the above code, and it will print out the results. It's an easy way to get a feel for Scala and what it can do.

How XML Could Have Averted The Recession

"How XML Could Have Averted The Recession" is the name of the presentation I am giving at this year's XML Summer School in Oxford (UK), 20-25 September 2009. The title is a bit whimsical; I'm not claiming to know how we actually could have avoided the recent financial crash, but I will be discussing how open standards, open data and timely disclosure might have helped us avert the recent failure, and how it might help us avert future failures.

This will be the 5th XML Summer School I've presented at. You won't find a better set of presenters anywhere in the world, if XML and related technologies are your interest. The Summer School is small enough that you can get to meet all of the presenters socially during the week, and the presenters keep coming back year after year because we all enjoy the fun we have together socially.

So, if you can come, you will be very welcome, and I hope to see some of you there. If you can make it, you won't regret it.