For a long time we've talked about stitching a relational database into Foswiki, mainly to speed up query searches. To keep things simple Foswiki uses a text database, which stores topics in lots of individual files. By default, searches are implemented by searching every one of these files individually for matching text. This isn't as bad as it sounds; searching is surprisingly fast. But it doesn't scale up terribly well, so people have coupled in various full-text search engines, such as Kinosearch, to accelerate text searches.
However there's been very little successful work yet on speeding up query searches – which is a shame, because query searches are heavily used in most wiki applications. The problem has always been that it's difficult to map from a Foswiki search query to any kind of structured search language, such as SQL.
One approach (championed by WikiRing member Sven Dowideit) has been to support a wider range of queries in the text search engine. Using the code I wrote to map query searches to regular expressions, Sven has been able to couple Foswiki to the MongoDB database, with some pretty impressive results. This is already available as the MongoDBPlugin. Other approaches have tried to map Foswiki queries to other database search models, such as XQuery, but with little success.
SQL is still the most common query language, and the RDBMS that support it are among some of the most mature and robust software on the planet. It's important that Foswiki has solutions that fall into that comfort zone. To that end I have implemented a new module, the DBIStoreContrib, that maps from the Foswiki query language to SQL. This allows us to use many of the existing RDBMS, such as MySQL, SQLite and PostgreSQL, to accelerate queries. By caching topics into the database in parallel to the text database, we avoid the risk of data loss that has made people nervous of other wiki solutions that use RDBMS, such as Mediawiki, while still maximising search performance.
It's early days for this module, and there's still lots to do to optimise how it works, so watch this space! If you are interested, and have a development environment, then you can check out the DBIStoreContrib from the Foswiki Subversion trunk.
Foswiki – the very best wiki application platform – is taking another giant leap forward….
What is legally permissible when adding a copyright notice to a derivative work?
In 2008 I stopped contributing to an open source project when it was pwned by a commercial interest. At that time a number of my original works existed in their source code repository, and still exist there pretty much as I left them 2 years ago, when I moved further development to a different repository. All of these works were released under the GPL, and carry my personal copyright notice, or that of a wider group of contributors who had worked on the project up to that date.
Since I left the project, the now-owners have redefined that contributors group, and have taken to retrospectively applying a different copyright to a number of these works. They have made some minor changes (mostly removing my personal branding, but also adding some scraps of new documentation) and added a dated copyright notice of their own that extends their claim back several years, despite the fact that (according to their own public records) no changes have been made until very recently.
Now, all the code and documentation is in their source code repository, so I would have no problem proving that I am the original copyright holder. However by making some very small changes, they have technically created a derivative work, and I don't have any problem with them adding a copyright notice to cover these new changes.
But what I find really objectionable is the dating of that new copyright notice back to a point well before the changes (5 years before, in at least one case).
This is obviously immoral, but the question is, is it actually illegal?
Update: it has come to light that in at least one case (not my code) the new copyright notice has been extended back to a date before the work was first published. I'm not sure whether to laugh or cry!
The WebDAV support for Foswiki took another leap forward this morning, when we finished development of a new FilesysVirtualPlugin which supports different views for topics. This makes Foswiki topic content much more accessible to a wider range of tools.
Some time ago we (C-Dot Consultants and Kontextwork) jointly released the WebDAVContrib and FilesysVirtualPlugin, which together support access to Foswiki data via the WebDAV protocol. More recently we released the WebDAVLinkPlugin, which enhances the integration of WebDAV documents into Foswiki, by allowing links to WebDAV documents to be edited using native applications on the client, such as the Microsoft Office™ suite.
That's great for working with attachments, but what about topics? Up to now, if you wanted to edit a topic, you had to edit the raw Topic Markup Language (TML) that the topic was stored in. Using Microsoft Word to edit TML is like using the space shuttle to commute 2km to work. Life would be much simpler if we could feed Word with a format it understands, like HTML.
We've been able to WYSIWYG edit Foswiki topics in the browser for a long time now; so the technology to convert topic content to other formats and back again exists; all we needed was a way to present this through WebDAV.
What we need to do is to give the WebDAV user a way to select what format they want the topic in. We've done this by allowing the same topic content to be mapped to more than one file entry in a WebDAV directory listing; each file entry is referred to as a "view" (because it's just a different way of looking at the same data).
The FilesysVirtualPlugin is configured with a list of views that it supports; for example, txt, html, json. When it is asked for a list of files in a Foswiki web, it generates a file entry for each of the different views, so the topic MyTopic ends up with the entries:
These file entries can be read and written just like normal files; the FilesysVirtualPlugin takes care of mapping the content back to TML. Because these file entries are all views of the same data, then if you edit MyTopic.txt, then MyTopic.html automatically changes too, and vice-versa.
Besides txt, html, and raw, two meta-data views have been provided, json and perl. These views allow you access to topic meta-data. It's easy to add new views, and we foresee other useful view types, such as xml and yaml coming along later.
WebDAVContrib, FilesysVirtualPlugin and the companion WebDAVLinkPlugin are available from Kontextwork.
It is often claimed that only a few developers moved from TWiki to Foswiki, therefore the first article will look at who are/were active core developers of both projects.
Number of core developers
The analysis focus on core development of both projects, as this is a major indicator for the health of a software project. This is because you need a lot more knowledge and background for developing the core product than for writing an extension with a clearly defined API.
"We have healthy downloads, an active user community, and a very active support community. However, we are a smaller developer community than we used to be."
Source: Blog post by Peter Thoeny (Twiki.net) – 11 Nov 2009
Developers of TWiki and Foswiki
New to core development
2 Core Contributers
32 Core Contributers
Look at the list of developers contributed to the core of both projects since the commercial takeover of Twiki.net. The TWiki.org development community has dropped to almost nothing – only Peter and his employee Sopan is left. Whereby all other TWiki.org core developers moved to Foswiki.org. Together with new developers, Foswiki has almost doubled its core contributors!
Number of core contributors
before commercial takeover
of TWiki.org (2008-10-27)
one year later (2009-10-27)
Yes, lots of users still download the old TWiki code as they don´t know about Foswiki and its progress. Our marketing was very bad as we are better in concentrating on improvements than in talk about it. Sadly even most of our old TWiki user community don´t know about Foswiki and may wonder why there is no progress on TWiki.org.
As the analysis is based on open source, you can easily check the correctness of the data by yourself. For the first analyzation I used the following sources:
One year after the commercial take over of TWiki.org its time look how TWiki and its successor Foswiki performed. A small blog series will cover facts and give you insight into the development status of both projects.
As a Wiki consultancy we regulary got asked about different projects and how good they are. In case of TWiki.org and Foswiki.org customers are very irritated and cannot see the difference. The fact that Foswiki is based on TWiki, but got developed much further within the last year is not obvious to most users. Contrary to the fact that Foswiki is the superior product, TWiki still has the better recognition marketing wise.
In order to help our customers getting an insight into the development of both projects I will do an analysis and compile the results in an understandable way. The results will be published in a small series of articles.
The first analysis will be based on core development of both projects, as this is a major indicator for the health of a software project. This is because you need a lot more knowledge and background for developing the core product than for writing an extension with a clearly defined API.
As the analysis will be based on open source, you can easily check the correctness of the data by yourself. For the first analyzation I will use the following sources:
TWiki.org SVN trunk/core
Foswiki.org SVN trunk/core
Stay tuned and see how Foswiki and TWiki developed over the past year.