I just got back from a vacation in Denmark – the land where Legos come from. I thought it might be appropriate to spend some time talking about Flavio Zeni at UNDESA calls Legislative Legos – building blocks that can be used to build legislative systems.

Why is this such an important concept? Well, it’s all about managing risk and building a system that can adapt to change in timely manner. Legislative systems can take years and cost many millions of dollars to develop. As the sophistication grows and automation takes root throughout the organization, these systems become extensive mission critical parts of the organization’s fabric. So failure is not an option.

At the same time as legislative systems become so ingrained, technology cycles are shrinking. Rapid changes in technology ensures that the expected lifecycle of any part of a system will be compressed. Wasting 5 years of a technology cycle to perfect a large system chews up a lot of the viable life of a technology as well. Also, as change occurs, the technology waves are being blurred into streams of constant change. So catching a technology wave at the exact right moment becomes impossible.

Imagine for a moment choosing to build a brand new, built from scratch, system. You’re looking at a multi-million dollar proposition that will take at least 3-5 years to come to fruition. The technologies you choose will all be at various stages of maturity. Some technologies might be well established and stable – but they might also be waning and likely to soon be obsolete. There the risk is building a system now that will be obsolete by the time it is deployed. On the other hand, other technologies will be on the bleeding edge. Choosing to use them might maximize the lifecycle of the result, but it does so at the risk of using a technology that might not yet be ready for prime time. When building a large scale system, the likelihood of all the technologies being optimal for adoption at the same time you need them is very remote.

Think about the infrastructure of a city. The road system has all sorts of problems. Traffic jams, potholes, and aging infrastructure abound. Under the streets are a rat’s nest of utilities from different eras. It might be very tempting to want to simply demolish the city and build a new one. But that’s a preposterous idea. Not only would it not be affordable, the inconveniance and risk would be unthinkable. This is something that becomes true of all systems as them become large and ingrained. They are no longer replaceable. Instead they must evolve constantly – in as smooth and efficient a manner as can be achieved.

So how must legislative systems evolve? The must evolve by being built as interoperable modules. These modules must stand on their own and be able to be deployed, updated, and replaced on their own timetable. These modules must be defined in such a way as to maximize the likelihood that expertise gained elsewhere can be harnessed to reduce risk and lower costs. These modules must be sufficiently independent of the rest of the system that should they fail or become obsolete they do not jeopardize everything else. This is all simply good risk management.

So let’s think about Legos for a moment. When I was a kid they were my favorite toy and I was the champ at the Lego competitions in the small town where I grew up. I would spend hours and hours building all sorts of different things out of them. A consistent standard for how the blocks connect is what made this possible. That standard doesn’t dictate how the blocks go together or what you can build with them. Instead, Lego defines a simple protocol that allows the blocks to connect and then they provide enough variety in their blocks to allow your imagination to run wild.

We need the same concept for legislative systems. Instead of large monolithic, likely to either fail or be obsolete, but always be expensive solutions, we need a simple set of protocols that will allow modules to be built and then mixed or matched to solve all the varying requirements that exist for legislative systems.

So what is the legislative analogue of the Lego standard? First we need the basic protocol for how things connect. That is where XML and something like Akoma Ntoso comes in. It defines the most basic parameters of how the information can be connected. Next we need to define what the basic pieces are. We need an editor, a repository, some sort of metadata management, various publishing engines, and so on. Those are the blocks. Not every system will need the same blocks and not every block will fit every application. With enough forethought and enough industry cooperation, blocks can be made to fit together in a variety of ways to solve every need.

I come from the electronic CAD industry. Our early generation systems in the 1980s were large monolithic solutions. As a vendor, we loved the idea because we locked our customers into our solution. Their investment in our system become so large that replacing us was inconceivable. All we had to do was have our professional services people show up at a prospect, convince them to choose our product suite, and then accept all the custom code we would build to cement the relationship forever. This worked great until our product became obsolete. Not only could our customers not get out from under the monstrosity we had built, neither could we. Smaller companies with new innovative ideas started chipping away at our fortress (that’s what we called it) and we couldn’t adapt. When we tried, the result was more pieces stuck to our monolith which only made the problem worse. Monolothic system thinking nearly killed the business and made a lot of customers very angry. Today that same industry is made up of a myriad of much smaller less tightly coupled products, available from a large selection of venders, and bound by common industry standards. The Lego inspired model made the industry much stronger and served the customers far better. The design efficiency that results is what makes thing like smartphones and tablets, updated every few months, even possible.

The same thing must happen for legislative systems. Modular Legislative Legos will allow this field to flourish – to everyone’s benefit. So let’s work together to make this happen.

Way back in 2002, when I first started building an editor for legislation, the problem looked so simple. We figured it was a quick six month project and we would then have a product we could resell to other jurisdictions. I remember when we first made our modest proposal, having it pushed back at us and being told we needed to go and rethink it. California had already learned, the hard way, that the deceptively simple problem of building a legislative editor was actually much more difficult and complex. What I have come to appreciate, over time, is that California’s approach actually makes the problem easier. It’s even harder elsewhere where the drafting rules are less structured and well thought out. I actually lucked out in starting with California, and it still took over three years to get a deployed solution in place – and that was only the beginning.

So what makes building a legislative editor so difficult? First of all, you need to understand what a bill really is. If you’re a software developer, the easiest way to understand it is to think of it as a document to “patch” existing law. Quite confusingly to software types, that process of patching the law results in something called “compiling codes”. So bills are patch documents containing intricate instructions for how to modify the law. What’s so hard about that? Well, for starters, the law has been built up over many decades or centuries and has accumulated all sorts of errors and odd situations that have to be accommodated. How do you correctly refer to something when, quite by accident, two different things have been numbered the same? How do you deal with the situation that arises when the law is defined to change in some complex temporal way? What happens when something is referred to by one number before a particular date, and then by a different number after that date? Quickly all these complexities start accumulating on top of your clean and simple architecture and the hope you’ll ever be able to solve it starts becoming a panic that perhaps you’ve attempted a problem that cannot be solved.

Let’s add another layer of complexity to the problem. The patch documents, which legislative types like to call bills, are also patched – or amended to use the real terminology. So you’ve got to deal with patches of patches. As a bill winds through the process of becoming law, it might be amended many times. Tracking all those amendments is going to be important to a lot of people. Keeping it all straight is crucial to producing an eventual result that patches the law accurately. Those legislative types aren’t very understanding when you get it wrong.

So now that you understand the complexity of the process, lets add some more complexity. Legislative traditions were defined long before there were computers. The procedures were defined for an era where everything was done using pen and paper. Amendments were made using scissors and glue. In fact, scissors and glue were still a part of the amendment process until quite recently in California. Other jurisdiction still have to resort to these procedures even today. Unfortunately, it is simply not possible to replace these outdated traditions with modern computer-based methodologies overnight. Legal traditions evolve on a deliberately conservative path. Good luck trying to throw out the old way just because the new way will be easier for the software. Chances are, you’re going to have to adapt more to the current process than the current process is going to adapt to you.

So what other aspects of legislative bill drafting are complex? I could go on and on about this subject. First of all, you’re not going to be able to convince the government to throw out all the existing laws and start again. You’re just going to have to deal with all the strange ways things were done in the past. If they numbered things with vulgar fractions (i.e. 1/2, 1/3) once-upon-a-time, you’re going to have to deal with that. If they believe that 100.15 reads as “one hundred point fifteen” and comes somewhere between “one hundred point ten” and “one hundred point twenty”, except when it is found between “one hundred point one and one hundred point two”, then you’re going to have to accept that too. Then there is the whole subject of redlining. Don’t be fooled into believing that redlining is the same as track changes in a word processor. It might look the same and be conceptually similar, but chances are there is a lot more hidden meaning in those strikes and inserts than you realize. When you get to the subject of making references or citations, you’ll discover just how much ambiguity there is in the subject. Quite often, a reference to a person is by last name alone. The only way to know which person with that last name is being referred to is to understand the context in terms of location and time. Even then, you might have to deal with ambiguity. When you get to the subject of tables, you’ll want to throw your arms in the air and walk away. Amending tables is an almost intractible problem. If you’re lucky, page and line numbers won’t come up at all. How on earth do you represent page and line numbers in a document that is explicitly designed to not worry about the physical aspects of the published form?

Okay, so you have found your way through all of that and now your editor is ready to go. Whew, that was hard! Oh, but now they want to be able to tear apart and rearrange the document in arbitary ways that make no sense with the XML hierarchy you’ve defined. Of course, you’re expected to be able to piece together the resulting mess to produce a valid document. Well they can’t do that. That’s just the way it is and you’ll tell them that when they tell you they’re not happy with the editor. Alright, that’s not a very good answer. Well, back to the drawing boards on that. Building an editor that both conforms to the software notions of how a document should be structured and how a non-technical users perceives it to be structured is a very difficult problem.

Legislative XML editors can be built. There are a few successful examples to show that it is possible. I’m proud to have muddled through to success with California, but it was very difficult. At the same time, there have been plenty of failures and projects that have gone way over budget. Having a standardized model, such as is promised by Akoma Ntoso and the standards effort within OASIS, should help by allowing this problem to be solved only a few times and then reused many times afterwards. However, given all the variations that exist, even with a standard in place this is going to be a difficult problem to solve for the foreseeable future.

I just read this excellent post by Tom Bruce, et. al. from the Legal Information Institute at the Cornell University Law School.

Tom’s post brought to mind something I have long wrestled with. (Actually so long that it was a key part of my job working on CAD systems in the aerospace industry long ago). I sometimes wonder if having common identifiers isn’t more important than having a common data format. The reason is that being able to unambiguously establish relationships is both very difficult and very useful. In fact, one of the reasons you want a common format is so that you can find and establish these identifiers.

I have used a number of schemes for identifiers over the past ten years. Most of the schemes I have used have involved Uniform Resource Names or URNs. A decade ago we designed a URN mechanism for use in the California Legislature. Our LegisWeb product uses a very similar URN schema based on the lessons learned on the California Project. In more recent years I have experimented with the URN:lex proposal and the URL-based proposal that is within Akoma Ntoso. Both of these proposals are based around FRBR. I can’t say I have found the ideal solution.

I favor URN-based mechanism for a number of reasons. URNs are names – that was their intent as defined by the IETF. They are not meant to imply a location or organizational containment (well mostly they aren’t). In theory, these identifiers can be passed around and resolved to find the most relevent representation when needed. But there is a problem. URNs have never really been accepted. While they conceptually are very valuable, their poor acceptance and lack of supporting tools tends to undermine their value.

Akoma Ntoso takes a different approach. It uses URLs intead of URNs. Rather than using URLs as locations though, they are used as if they are identifiers. It is the duty of the webserver and applications plugged into the webserver to intercept the URLs and treat them as identifiers. In doing this, the webserver provides the same resolution functions that URNs were supposed to offer. My upcoming editor implements this functionality. I have built HTTP handlers that convert URLs into repository queries which retrieve and compose the requested documents. I have it working and it works well – as much as I understand the Akoma Ntoso proposal. I’m still not totally crazy about overloading identifier semantics on top of location semantics though. At least the technology support is better in place.

So what issues have I struggled with?

First of all, none of the proposals seem to adequately address how you deal with portions of documents. There are many issues in the area. The biggest of course is the inherent ambiguity within legislative documents. As Tom mentioned in his post, duplicate numbering often occurs. There are usually perfectly good and valid reasons for this – such as different operational conditions. But sometimes these are simply errors that everyone has to accommodate. Being able to specify the information necessary to resolve the ambiguity is not in any proposal I have seen. Add to that the temporal issues that come with renumbering actions. How do you refer to something that is subject to amendment and renumbering? Do you want a reference to specific wording at a specific point in time, or do you want you reference to track with amendments and renumbering?

At this point people often ask me why a hash identifier followed by a cleverly designed element id won’t work. The first thing you have to realize is that the # means something to the effect of “retrieve the document and then scroll to that Id”. The semantic I am looking for is “retreive the document fragment located at Id”. The importance of the difference becomes obvious when you realize that the client browser holds the “#” part of the request and all the server sees is the document URL, minus the hash and identifier. When your document is a thousand pages long and all you want is a single section, that distinction is quite important. Secondly managing ids across renumbering actions is very messy and introduces as many problems as it solves.

Secondly, the referencing mechanism tends to be documented oriented. Certainly, Akoma Ntoso uses virtual URL identifiers to refer to much more than simple documents, but the whole approach gets cumbersome and hard to explain. (If you want to appreciate this, try and explain XML schema’s namespace URI/URL concept to an uninitiated developer.) What’s more, it’s not clear if a common URL mechanism does enough to establish common enough practices for the effort to be useful. For instance, what if I want to refer to the floor vote after the second reading in the Assembly? Is there a reference to that? In California there is. That’s because the results of that vote are reported as a document. But there is nothing that says this should be the case. I have had the need to interrelate a number of ancilliary documents with the legislation. How to do that in a consistent way is not all that clear cut.

The third problem is user acceptance. The URN:Lex proposal, in particular, looks quite daunting. It uses lots of characters like @, $, ;, While end users can be shielded from some of this, my experience has taught me that even software developers rebel against complexity they can’t understand or appreciate. So far, this has been a struggle.

I’m eagerly awaiting Part 2 of Tom’s post on identifiers. It’s a great subject to explore.