For the last couple of years, C-dot Consultants has been working with a client to develop a plugin that interfaces to the well known (and quite excellent) IBM ClearQuest bug tracking and workflow management system. This interface allows Wiki users to make queries into the ClearQuest database, and present the results of those queries in tabular and graphical form. The interface was developed for the client's internal use, and has been heavily used by a large number of people for quite some time. It has proven to be very reliable. It occurred to us that it might be of some use to other enterprises who use ClearQuest. If you are interested, please contact me.

KontextWork and C-Dot Consultants have worked together to create the WebDAVContrib, a brand new Web DAV implementation for Foswiki and Apache 2.

Web DAV (Web-based Distributed Authoring and Versioning) is a set of extensions to the Hypertext Transfer Protocol (HTTP) that allows users to edit and manage files collaboratively on remote World Wide Web servers.

The new Web DAV module maps Foswiki content – webs, topics and attachments – to a directory structure, and exports it using the Web DAV protocol. It allows users to work with Foswiki content using the tools they are most familiar with, without compromising on Foswiki features. For example attachments can be opened, manipulated and saved right within desktop applications such as Microsoft Office, Open Office, Adobe Photoshop (and many more). Wiki webs can be opened in file browsers such as Windows Explorer, and wiki topics can be edited using many different text editors.

The new plugin is a completely new Perl implementation of Web DAV, written specifically for Apache 2, and has been extensively tested using the standard litmus tests. It works over HTTP and secure HTTPS, fully respects Foswiki permissions, and can be used with all standard Apache authentication methods. A version for TWiki is also available.

The WebDAVContrib is currently undergoing final pre-release testing. Contact Andre Ulrich for availability and pricing.

Here are some of the requirements I've been thinking about:

1. More flexible specification of publish sets, including publishing groups of webs
2. Record keeping (who published what, when, and what they published)

Michael Daum has related, but slightly different requirements. Michael publishes by copying content to a static web where it is picked up by a different CMS. So he has a common requirement to specify publish sets, but has additional requirements such as unpublishing, and automation of approval processes. Anyway, the upshot of the conversation was that we should have some common way of specifying "publish sets". The current method – regexes in the publish topic – is so crude as to be laughable. The obvious approach to specifying publish sets would be to use WebNotify, but as Michael points out that's far too geeky. Another approach would be to think about "specification" and focus on the UI, perhaps build on the SubScribePlugin. Another approach would be to use a wiki app to gather published content into one place, similar to the genwebnotify.pl approach taken to generate the notification mails from the Foswiki:Tasks web. Record keeping means maintaining unforgeable records of a publishing event. That means stamping the published docs with publishing information, including the history in the published data, and publishing to publish-event specific targets (e.g. directories named by the current date).

It would be interesting to hear what other ideas people have had.

We recently developed a TWiki plugin to support execution of Perl scripts that are written in TWiki topics. The scripts are executed on the server, and of course that means we have to do everything possible to ensure those scripts don't open security holes.For years Perl has had the Safe module, a clever package that provides a tightly constrained execution environment for Perl eval statements. Perl compiles all its code to a rich set of high level opcodes, which are then run on a virtual machine. By limiting the set of opcodes that are allowed to be run in the container, the Safe module can be used to create a very secure execution environment.For example, most people would consider the perl 'backtick' operator to be very dangerous, as it allows the caller arbitrary access to the shell. Backtick has a corresponding Perl opcode – called backtick – and to disable it, all we have to do is to remove it from the set of legal opcodes. The Perl developers have even gone so far as to classify the operators according to the usual safety concerns that a caller may have, making it relatively easy to decide which to allow, and which to exclude.Of course there's more to safety than that. We also have to be sure that the code being executed only has access to the namespaces we want it to have access to. The default condition for scripts run in a safe container is that they can only access the namespace of the container. We have to explicitly grant the container access to other namespaces when we create it.Of course there are potential risks with allowing any sort of script execution on your server, but in the case of a web server behind a corporate firewall, those risks are relatively small, and the 'Safe' module helps to make sure that such scripts are well controlled.The new TWiki plugin, called the PerlPlugin, is released under the GPL and is available to all WikiRing consultants for deployment on client sites.

With the recent announcement about the new Google Sites application, a number of former Jot Spot customers have decided to migrate to TWiki. WikiRing partner C-Dot Consultants has been engaged to help, and this post describes our experiences.

Jot Spot stores topic data in an XML database. Within that database, actual topic data is stored as "decorated HTML"; the basic topic content is HTML, within which Jot applications are embedded using Jot Spot's proprietary script, which uses XML tags.

Because of some fairly fundamental architectural differences between Jot Spot and TWiki it's not simple to automate the conversion of Jot Spot applications to TWiki. Fortunately our clients have not invested heavily in developing Jot applications, but have instead chosen to use the applications that Jot Spot provide by default. So the focus of our work has been to:

1. Convert existing Jot Spot topics to TWiki, with minimal formatting loss
2. Map a subset of Jot applications (most notably the Bug Reporter) to TWiki

Fortunately we were able to secure an XML dump of the Jot Spot database. This has alllowed us to perform the conversion without relying on the patchy availability of the Jot Spot site. Conversion of Jot topics has been achieved by leveraging a couple of existing technologies:

• The SAX XML parser from CPAN
• The open-source HTML to TML convertor we wrote for WYSIWYG editor integration into TWiki

SAX allows us to parse the Jot XML, pick out the form fields, and identify the subset of the topics suitable for passing on to the HTML to TML convertor. The HTML to TML convertor is already a proven technology, used every day with the Tiny MCE integration in TWiki, so it is robust and minimises formatting loss.

On the receiving side, we have customised the publicly available TWiki:Plugins.BugsContrib to support the data fields from the reporter in Jot Spot. We have had to develop a number of new reporting screens, something which has been made much simpler by the use of the type="query" search we contributed to TWiki 4.2.

It's cheering to note how similar the structure of a Jot Spot topic is to a TWiki topic. I guess you could call it convergent evolution!