Services · Blog · Demo
Get on the scene

Foswiki is getting SQL RDBMS support!

2 years, 1 month ago in by Crawford Currie database foswiki plugin searching sql storage
419970_33314492.jpg
For a long time we've talked about stitching a relational database into Foswiki, mainly to speed up query searches. To keep things simple Foswiki uses a text database, which stores topics in lots of individual files. By default, searches are implemented by searching every one of these files individually for matching text. This isn't as bad as it sounds; searching is surprisingly fast. But it doesn't scale up terribly well, so people have coupled in various full-text search engines, such as Kinosearch, to accelerate text searches.
However there's been very little successful work yet on speeding up query searches – which is a shame, because query searches are heavily used in most wiki applications. The problem has always been that it's difficult to map from a Foswiki search query to any kind of structured search language, such as SQL.

One approach (championed by WikiRing member Sven Dowideit) has been to support a wider range of queries in the text search engine. Using the code I wrote to map query searches to regular expressions, Sven has been able to couple Foswiki to the MongoDB database, with some pretty impressive results. This is already available as the MongoDBPlugin. Other approaches have tried to map Foswiki queries to other database search models, such as XQuery, but with little success.

SQL is still the most common query language, and the RDBMS that support it are among some of the most mature and robust software on the planet. It's important that Foswiki has solutions that fall into that comfort zone. To that end I have implemented a new module, the DBIStoreContrib, that maps from the Foswiki query language to SQL. This allows us to use many of the existing RDBMS, such as MySQL, SQLite and PostgreSQL, to accelerate queries. By caching topics into the database in parallel to the text database, we avoid the risk of data loss that has made people nervous of other wiki solutions that use RDBMS, such as Mediawiki, while still maximising search performance.

It's early days for this module, and there's still lots to do to optimise how it works, so watch this space! If you are interested, and have a development environment, then you can check out the DBIStoreContrib from the Foswiki Subversion trunk.

Foswiki – the very best wiki application platform – is taking another giant leap forward….

… reply

The Dark Side of Open Source: pwning a copyright

2 years, 7 months ago in by Crawford Currie community copyright collaboration
What is legally permissible when adding a copyright notice to a derivative work?
In 2008 I stopped contributing to an open source project when it was pwned by a commercial interest. At that time a number of my original works existed in their source code repository, and still exist there pretty much as I left them 2 years ago, when I moved further development to a different repository. All of these works were released under the GPL, and carry my personal copyright notice, or that of a wider group of contributors who had worked on the project up to that date.

Since I left the project, the now-owners have redefined that contributors group, and have taken to retrospectively applying a different copyright to a number of these works. They have made some minor changes (mostly removing my personal branding, but also adding some scraps of new documentation) and added a dated copyright notice of their own that extends their claim back several years, despite the fact that (according to their own public records) no changes have been made until very recently.

Now, all the code and documentation is in their source code repository, so I would have no problem proving that I am the original copyright holder. However by making some very small changes, they have technically created a derivative work, and I don't have any problem with them adding a copyright notice to cover these new changes.

But what I find really objectionable is the dating of that new copyright notice back to a point well before the changes (5 years before, in at least one case).

This is obviously immoral, but the question is, is it actually illegal?

Update: it has come to light that in at least one case (not my code) the new copyright notice has been extended back to a date before the work was first published. I'm not sure whether to laugh or cry!

… reply

WebDAV gets data views

2 years, 8 months ago in by Crawford Currie ajax foswiki webdav wysiwyg
The WebDAV support for Foswiki took another leap forward this morning, when we finished development of a new FilesysVirtualPlugin which supports different views for topics. This makes Foswiki topic content much more accessible to a wider range of tools.
Some time ago we (C-Dot Consultants and Kontextwork) jointly released the WebDAVContrib and FilesysVirtualPlugin, which together support access to Foswiki data via the WebDAV protocol. More recently we released the WebDAVLinkPlugin, which enhances the integration of WebDAV documents into Foswiki, by allowing links to WebDAV documents to be edited using native applications on the client, such as the Microsoft Office™ suite.

That's great for working with attachments, but what about topics? Up to now, if you wanted to edit a topic, you had to edit the raw Topic Markup Language (TML) that the topic was stored in. Using Microsoft Word to edit TML is like using the space shuttle to commute 2km to work. Life would be much simpler if we could feed Word with a format it understands, like HTML.

We've been able to WYSIWYG edit Foswiki topics in the browser for a long time now; so the technology to convert topic content to other formats and back again exists; all we needed was a way to present this through WebDAV.

What we need to do is to give the WebDAV user a way to select what format they want the topic in. We've done this by allowing the same topic content to be mapped to more than one file entry in a WebDAV directory listing; each file entry is referred to as a "view" (because it's just a different way of looking at the same data).

The FilesysVirtualPlugin is configured with a list of views that it supports; for example, txt, html, json. When it is asked for a list of files in a Foswiki web, it generates a file entry for each of the different views, so the topic MyTopic ends up with the entries:

  • MyTopic.txt
  • MyTopic.html
  • MyTopic.json
These file entries can be read and written just like normal files; the FilesysVirtualPlugin takes care of mapping the content back to TML. Because these file entries are all views of the same data, then if you edit MyTopic.txt, then MyTopic.html automatically changes too, and vice-versa.

Besides txt, html, and raw, two meta-data views have been provided, json and perl. These views allow you access to topic meta-data. It's easy to add new views, and we foresee other useful view types, such as xml and yaml coming along later.

WebDAVContrib, FilesysVirtualPlugin and the companion WebDAVLinkPlugin are available from Kontextwork.

… reply

Development of Foswiki and TWiki - get the facts

2
3 years, 1 month ago in by Andre Ulrich community foswiki opensource twiki
It is often claimed that only a few developers moved from TWiki to Foswiki, therefore the first article will look at who are/were active core developers of both projects.

Number of core developers

The analysis focus on core development of both projects, as this is a major indicator for the health of a software project. This is because you need a lot more knowledge and background for developing the core product than for writing an extension with a clearly defined API.

Claim

"We have healthy downloads, an active user community, and a very active support community. However, we are a smaller developer community than we used to be."
Source: Blog post by Peter Thoeny (Twiki.net) – 11 Nov 2009

Fact

Developers of TWiki and Foswiki
Core contributers
PeterThoeny
SopanShewale
CrawfordCurrie CrawfordCurrie
KoenMartens KoenMartens
MichaelDaum MichaelDaum
RafaelAlvarez RafaelAlvarez
AndreUlrich AndreUlrich
TravisBarker TravisBarker
ArthurClemens ArthurClemens
GilmarSantosJr GilmarSantosJr
LynnwoodBrown LynnwoodBrown
OliverKrueger OliverKrueger
ColasNahaboo ColasNahaboo
KennethLavrsen KennethLavrsen
MarkusUeberall MarkusUeberall
SvenDowideit SvenDowideit
AntonioTerceiro AntonioTerceiro
WillNorris WillNorris
MartinCleaver MartinCleaver
New to core development
SebastianKlus SebastianKlus
OlivierRaginel OlivierRaginel
EugenMayer EugenMayer
LarsEik LarsEik
IsaacLin IsaacLin
GeorgeClark GeorgeClark
AndrewJones AndrewJones
New developers
SeanMcCarthy
RobManson
BenBeijer
RaulFRodriguez
MarkSchumann
MichaelTempest
AndrewPantyukhin
DrakeDiedrich
PaulHarvey
2 Core Contributers 32 Core Contributers

Statement

Developer community:

Look at the list of developers contributed to the core of both projects since the commercial takeover of Twiki.net. The TWiki.org development community has dropped to almost nothing – only Peter and his employee Sopan is left. Whereby all other TWiki.org core developers moved to Foswiki.org. Together with new developers, Foswiki has almost doubled its core contributors!

  Number of core contributors
  before commercial takeover
of TWiki.org (2008-10-27)
one year later (2009-10-27)
TWiki.org 18 2
Foswiki.org n/a 32

Healthy Downloads:

Yes, lots of users still download the old TWiki code as they don´t know about Foswiki and its progress. Our marketing was very bad as we are better in concentrating on improvements than in talk about it. Sadly even most of our old TWiki user community don´t know about Foswiki and may wonder why there is no progress on TWiki.org.

Sources

As the analysis is based on open source, you can easily check the correctness of the data by yourself. For the first analyzation I used the following sources:

  • TWiki.org SVN trunk/core
  • Foswiki.org SVN trunk/core

Don't miss the rest of the series:


Development of Foswiki and TWiki - get the facts (Intro)

3 years, 1 month ago in by Andre Ulrich community fork foswiki opensource twiki
One year after the commercial take over of TWiki.org its time look how TWiki and its successor Foswiki performed. A small blog series will cover facts and give you insight into the development status of both projects.

Intro

As a Wiki consultancy we regulary got asked about different projects and how good they are. In case of TWiki.org and Foswiki.org customers are very irritated and cannot see the difference. The fact that Foswiki is based on TWiki, but got developed much further within the last year is not obvious to most users. Contrary to the fact that Foswiki is the superior product, TWiki still has the better recognition marketing wise.

In order to help our customers getting an insight into the development of both projects I will do an analysis and compile the results in an understandable way. The results will be published in a small series of articles.

The first analysis will be based on core development of both projects, as this is a major indicator for the health of a software project. This is because you need a lot more knowledge and background for developing the core product than for writing an extension with a clearly defined API.

As the analysis will be based on open source, you can easily check the correctness of the data by yourself. For the first analyzation I will use the following sources:

  • TWiki.org SVN trunk/core
  • Foswiki.org SVN trunk/core

Stay tuned and see how Foswiki and TWiki developed over the past year.

Continue reading at:


r5 – 29 Mar 2008 – 13:08:59 – MichaelDaum
Copyright © 1999-2012 WikiRing Partnership –