On November 18th, the community has voted for a new name for the project that recently forked off TWiki.org and which hit the front pages recently under the working title NextWiki. So welcome the new old player.

This is the start of a great new brand that is about to evolve at foswiki.org The tagline of the project is

The Free and Open Source Wiki

which obviously is reprocessing the trauma of the Hostile takeover of the Open Source Project TWiki but also targets at its best competitors in the market segment, which are not Open Source.

It remains to be seen if this new project will be able to catch up with the field, as it was dropping behind recently, according to the Magic Quadrant for Social Software, 2008. In this paper, Gartner categorizes TWiki as a niche player only whereas it was on par with Socialtext in the Magic Quadrant for Team Collaboration and Social Software, 2007. That might be the result of the long paralysis caused by the governace crisis over the recent years on the TWiki project, which finally culminated in the recent fork 3 weeks ago.

This paralysis obviously has deflagrated as the new project Foswiki shows an impressive amount of activities by all members. Long-time contributors, that went on strike as the trademark issues on TWiki started to manifest, now show an outburst of activities, committing and impressive stream of updates and bugfixes to the new platform. Surely, a vast amount of work is related to rebranding the software by pinching the string "TWiki" out of every place.

The questionnaire on finding the new brand name for the project had an impressive 100 submissions spot on just within a few days. There was a prior survey among the community to find out about its likings for a new name that had even more submissions. Compare this with just a hand full of members that dared to voice up on the old TWiki project during all of its quarrels. That's over now and people obviously have fun again to volunteer.

There was already very positive feedback on the cheers-and-donate mailing list coming from outside. One of which says:

The new website is very appealing and the first impression is not "bitter" but "better", so keep that up! A big cheer for everyone involved in this initiative!!

Still, according to the criteria for a product to be listed on the Magic Quadrant study by Gartner, it seems quite probable that TWiki will drop out on 2009, as it lost its community which created the product before. Regarding its Open Source process, TWiki can be considered dead and with it goes its core engine. Foswiki, on the other side, now continues work on the engine with a lot of verve, but won't show up easily on a forthcoming 2009 Magic Quadrant as it still has to build up its new brand and user base. However, as most authors of extensions have moved to Foswiki and now maintain their work on the new platform, moving from TWiki to Foswiki is a natural choice to keep up with upstream updates and improvements.

TWIKI.NET, the new owner of TWiki.org, won't rest and just watch. They are actively working on interesting applications to address intranet needs build on top of the TWiki engine. The fact that this software can be used to build applications like that is great in its own and TWIKI.NET does score in that field as far as can be seen from outside. But of how much value are such applications in the long run, when the platform they are build on erodes? TWIKI.NET is funded by venture capital paying a bunch of developers in Asia and elsewhere. Time will show if that business plan pays off to cope with all the open road works that they have to face all alone now. Good luck from WikiRing.

Access to the site is only granted if contributors agree to a set of newly installed terms and conditions dictated by TWIKI.NET, a company founded by Peter Thoeny 12 months ago. His power to do so grows out of two sources: (a) he is the sole owner of the trademark on TWiki and (b) he is sponsoring the server hardware and thus had root access.

And now he has triggered the trademark gun and fired the TWiki community. He even repeatedly threatened people on the #twiki IRC channel that "[he has] been advised by one of [his] investors, Wilson Sonsini Goodrich and Rosati, that [they] need to protect [their] trademark". Clearly, their VC people have no picture of the situation other than their own return of investment. Sure, protecting a registered trademark is what it is all about. But threatening the community that has been working on TWiki on a volunteer basis for the recent 10 years that way is a bit strong. Too strong for the TWiki community.

If there was ever any hope to re-establish a relationship of trust and faith to create a win/win situation by combining community & commerce, this is totally gone now. Thoeny installed himself as BDFL (Benevolvent Dictator for Life) again, despite being rejected by a community vote during the TWiki Summit in Berlin last month.

During the TWiki Summit in Berlin 4+5 September 2008, it became clear that Thoeny has sold part of his trademark rights to his venture capital funded company. Part of that deal was that while he remains ownership on the trademark itself, TWIKI.NET gained the sole right to exploit the brand on a commercial basis. This created a completely new situation for the Open Source project and all of its already existing commercial eco-system. As a consequence, TWIKI.NET was asked to grant a perpetual license to the community to secure the legal situation for contributors and commercial stakeholders, a license that would only have formalized the way TWiki has been running for more than 10 years with Thoeny promising to "take care of the brand".

As faith in him as a leader diminished over the years, and the foreboding of a trademark problem increased, the community asked Thoeny to write down the rights he has granted orally before. Which he didn't. Instead he pulled the trademark trigger in a move he calls "relaunching the project" to "weed out" the good and the bad. Trust in Thoeny as a leader diminished last but not least when his role as a community leader became more and more mixed up with his interests as a CTO of TWIKI.NET, up to the point where he obviously showed more interest to cement a genuine marketing advantage for TWIKI.NET.

The rise of his newly created company continually eroded willingness to contribute to TWiki as an Open Source project. People were more and more irritated by the changed rules of the game. The community has been watching the actions of TWIKI.NET with a lot of interest, in the hope that they would add significant value to this very successful project. Unfortunately, they took an approach of recasting the success of the product, created with years of volunteer work, as their own success.

That's where Open Source shows its ugliest face. And there's definitely no beauty in this shock therapy, even though Tom Barton, CEO of TWIKI.NET says: "the beautiful thing about open source is you don't need to recognize the authority of TWiki.net". What an irony to close another very sad chapter. The last one for TWiki.org as we knew it before.

The appearance of TWIKI.NET on the scene forced a governance crisis TWiki was not able to overcome, despite the good progress that was made up to a couple of hours before. On the TWiki Summit in Berlin last month, a democratically elected Interim Board of Directors was founded whose sole agenda was to negotiate the conditions under which this governance crisis could have been overcome.

The plan was to create a TWiki Association consisting of a Board of Directors and a General Assembly following the example of KDE e.V. The board itself would have created so called Task Teams that manage the operational part of the project to a finer granularity.

The members of the Interim Board of Directors were in the process of creating the Articles of Association and were prepared for the next logical move in an ever growing project, organizing it similarly to other projects in the Open Source business. This formal body would also have been an entry path for sponsors and other organizations willing to partner with TWiki as a project. No such thing was available before. The only way outside parties could have made donations was to give them directly to Thoeny and thereby TWIKI.NET.

This was the case when Sun donated server hardware to power the TWiki.org community site. Sun sponsored TWiki as an Open Source project, not TWIKI.NET. However, there was no entity other than Thoeny and TWIKI.NET to handle these opportunities and resources. It now is clear that the access to these server resource has been used against the TWiki community itself by locking it out.

The democratically elected Interim Board of Directors of TWiki has been displaced by the trademark holder of TWiki as a final chord on the governance crisis. Now, Thoeny is sending around emails to high profile contributors individually to invite them to come back subordinate to the governance of TWIKI.NET. He obviously seems to be in hope that people will do so once the situation has settled. Quite far-fetched and not very likely to happen. Those same contributors who implemented the features he is praising aloud as the shiny new TWiki, are far too displeased by his hostile behavior to be willing to go back to business as usual.

TWIKI.NET is striving to repaint their move as a "new opportunity". What they don't see is that they have put their own business case into severe danger. They just lost the horse power for a product that they were selling. They have been signaling to the community that they don't have the manpower for certain developments and were seeking for help, even willing to pay work for hire. Another error. Adding money as an incentive to Open Source is changing the game completely. Before, people volunteered as part of an act of free speech. Add money to it and nobody will work for free anymore. This poisoned the dynamics.

The current situation is that all core developers have left the ship and joined a new undertaking with the working title NextWiki. This is a fork based on the current code in TWiki that will soon be released under a new name. The goals of NextWiki are clear. Basically, the plan is to found an Association as a formal body for the project, including the reorganization of its governance down to all operational questions, as was in progress for the TWiki project.

The result will be a much strengthened new player, much more agile as it just got rid of the reason for TWiki's ongoing paralysis.

There remains a message for TWiki's users: no worries, we continue working, faster and more productive than ever before, embedded in a volunteer-friendly environment. Sure, this fork now introduces a new choice that was not there before. Well, it was there before and it was introduced by TWIKI.NET, not those guys that "asked for a fork". TWiki users already had the choice between TWIKI.NET's product (a rebranded version of an old TWiki release, packaged as a VMware image), or Open Source TWiki, most recent stable version. This choice more or less remains available with the difference that you will get the real thing from a new site, reworked to be real Open Source, backed up by a large and highly motivated community as a guarantor for continuity and innovation.

Update

Due to recent changes in the TWiki eco-system, this shop has stopped selling TWiki accessories. However, we will update the selection as soon as we've got a proper name for the NextWiki fork.

The current code implements some storage backends now. Before we only had a Cache::Memcached based backend which is great on your own servers, but not applicable for little TWiki installations in hosted environments where you are not allowed to launch daemon processes.

What we have right now is

• Cache::Memcached
• Cache::FileCache
• Cache::SizeAwareFileCache
• Cache::MemoryCache
• Cache::SizeAwareMemoryCache

I did not make use of the shared memory variants as they are quite limited in their storage capacity. The Cache::FileCache is said to perform as fast anyway, but does not have those restrictions. For html page caching the first two variants are the most interesting. Basically you can forget Cache::SizeAwareFileCache, and it's only there for completeness. This backend simply performs too much maintenance on its own behalf to be usable for TWiki. The Cache::SizeAwareMemoryCache, however, is comparably fast purging out pages, as it does not suffer from the extensive disk IO that happens in the Cache::SizeAwareFileCache. Both memory variants are not able to share their cache among several processes, and thus are obviously out for html page caching. The current WikiRing blog runs on a Cache::FileCache now. It used the Cache::Memcached backend for quite some time. But as I am low on RAM but not on disk space on this server anyway (anybody out there that likes to sponsor a bigger server or more memory?) I switched to the file-based variant, which performs quite well.

I still owe you performance statistics, hard facts.

Client-side as opposed to server-side caches try to cache as close to the browser as possible, to reduce bandwidth congestion early. Such a cache gets an url, and returns a full html page. The only additional knowledge it has is the time the page is supposed to expire. There are also server-side caches that work along the same lines — called reverse caches.

The central task of a cache is to maintain its integrity (don't return outdated information). Caches tend to sacrifice a bit of integrity to get any caching effects at all. Now, the problem is that the cache and the content source don't talk to each other. If the content source has an update it should notify the cache about it, so that the cache can actively invalidate some of its store. To my knowledge there's no third party cache that implements such an API. Nevertheless, there are third party cache solutions that have very good APIs to integrate them into your own software. Two of them are Memcached and Varnish. While Memcached has bindings for several (scripting) languages, Varnish comes with its own scripting language. Memcached is more of a multi-purpose cache for any kind of storable objects; Varnish is a highly optimized server-side cache and restricted to url based caching only.

Frankly, notifying the cache so that it performs invalidation is no big conceptual problem. The hard bit is when and which parts of the cache shall be invalidated. The task of “dependency tracking” is to record which objects were needed to compute one another. Based on this information, dependencies are “fired” to recursively invalidate cache information. As you can imagine, one page can depend on a lot of other information spread all over the place. For example, the same page may look differently for each user due to user preferences, url parameters or session values. A wiki page may depend on a WikiWord not being defined yet, as it renders a link to it differently when it exists or not. While these case can be detected automatically there are also cases when the engine can't. Imagine a RandomPlugin that generates random blind text. A page using such a feature is basically not cacheable, as content materializes out of nowhere. More realistic examples are plugins that integrate external data sources. The knowledge about the content source is part of an external system. So it is responsible to establish dependencies to the objects that are constructed on its base.

Dependency tracking will be nearly impossible in other cases where there's no a priori knowledge at hand: a page showing query results. The page is assembled using its search results, and thus depends on the found items. Either this page can become invalid because the query does not match an item anymore, or a new item comes into existence that now matches the query. This is bad news as search queries are the most expensive and most valuable operation in any content engine. However, things are not that bad for caching as data does not change so frequently, and we can present the same results as long as we know that no data has changed. Think of it as a 1:n dependency instead of a couple of 1:1 dependencies between different objects.

Yet another type of dependency is content that changes over time, without any external input, e.g. a clock. You really don't want to cache anything here, the same way as it is pointless to cache random numbers on a page using your shiny new RandomPlugin. From the angle of a cache this type of content is pure “dirt”. In fact, while the rest of the page is quite static for a while, some areas in it may be filled with up-to-date information on every request. A way to deal with this is so-called “dirty areas” … which is nothing offensive but just a way to prevent the cache from being trashed with information it can't cache at all. In a way, cached pages that have dirty areas in it are rather similar to very specific templates.

Alright, so far I mused about caching in a more general way motivating the typical problems and outlined what can be done about it. Let me introduce you to the TWiki::Cache that has been implemented recently in one of my next postings.