Regex and Link Rewriting:Wikka Documentation

Regex and Link Rewriting

Last edited by YanB:
Removed HomePage link
Sat, 01 Nov 2008 23:23 UTC [diff]

I open this page for discussing a regex issue met during the development of a FetchRemote action.
Basically, the aim of this action is to fetch raw page content from a remote Wikka server and rewrite it before printing it on screen.

1. What is raw content

Raw content is the source code of Wikka pages, containing WikkaSyntax tags. For example, the raw content of a page like WikiEngine is:

WikkaDocumentation
----
===== What is a Wiki? =====

A **wiki** (pronounced "wicky" or "weeky" or "viki") is a website (or other hypertext document 
collection) that allows any user to add content, but also allows that content to be edited by any 
other user while keeping track of the different versions.
In short, a Wiki is one of the most powerful tools for **web-based collaborative editing**.

A WikiEngine is the software used to create and run such websites. For instance, this wiki runs on 
the [[HomePage | WikkaWiki]] engine.


<<More information on Wikis is available on: [[http://en.wikipedia.org/wiki/Wiki | Wikipedia]]<<


----
CategoryDocumentation - CategoryWiki

The new showcode handler allows you to display the raw content of any page by appending /showcode to its name in the URL:
http://docs.wikkawiki.org/WikiEngine/showcode

2. Link rewriting in `FetchRemote`

The FetchRemote action requires parsing a fetched page's raw content and rewriting internal links in a specific way.
Basically there are two kinds of links that have to be rewritten: forced internal links and CamelCase links.
For the action to work properly, forced internal links and camelcase links in the fetched page should be respectively rewritten as follows:

Forced links:

[[WikkaDocumentation | A good link]] => <a href="FetchRemote?page=WikkaDocumentation">A good link</a>

CamelCase links:

WikkaDocumentation => <a href="FetchRemote?page=WikkaDocumentation">WikkaDocumentation</a>

3. Link rewriting through `preg_replace()`

To do so, I use the PHP preg_replace() function. I've almost managed to have both of the above cases correctly parsed using the following patterns:

$forced = "/\[\[([^ \/]+) ([^\]]+)\]\]/";
$camel = "/[^a-z=>\"\[\/\{]([A-Z]+[a-z]+[A-Z][A-Za-z0-9]+)+/";

For matching a forced link I think it may be better to start with the same RE for a forced link that the formatter does:

$forced = '/\[\[(\S*)(\s+(.+))?\]\]/';

- that way you take care of any form of whitespace between the two parts of a foced link; if you want to match only words that are allowed as page names (and not whole URLs), you could replace the \S* in there with the (partial) RE that matches a valid page name - see my hints on that page for how to build up a RE from pattern blocks. --JavaWoman

and rewrite the raw page content ($content) by applying twice the preg_replace() function:

// rewrite forced links
$content = preg_replace($forced, "\"\"<a href='".$this->Href("","","page=\\1")."'>\\2</a>\"\"", $content);

// rewrite camelcase links
$content = preg_replace($camel, "\"\" <a href='".$this->Href("","","page=\\1")."'>\\1</a>\"\"", $content);

All those double quotes are confusing; let me try to get rid of some to make it more readable so I can follow what you're doing! how about:

// rewrite forced links
$content = preg_replace($forced, '""<a href="'.$this->Href('','',"page=\\1").'">'."\\2".'</a>""', $content);

// rewrite camelcase links
$content = preg_replace($camel, '""<a href="'.$this->Href('','',"page=\\1").'">'."\\1".'</a>""', $content);

--JavaWoman

4. Tricky cases

The link rewriting rules above will work fine in most cases. What they still cannot capture is a number of cases in which a WikiWord appears in the context of a forced internal link, like for example:

[[WikkaDocumentation | This is the homepage of the WikkaWiki Documentation Project]]

.
It's clear in this case that WikkaWiki should NOT be rewritten (it's not a link, but part of the anchor text of a link).

If you take for example the rawcontent of WikiEngine displayed above, the preg_replace() patterns I'm using won't handle a link like [[HomePage | WikkaWiki]] properly.

After the first preg_replace() application (forced link rewriting) this code is correctly rendered as:

""<a href='FetchRemote?page=HomePage'>WikkaWiki</a>""

But after the second preg_replace() application (camelcase links rewriting), this will be rendered as:

""<a href='FetchRemote?page=HomePage'>""<a href='FetchRemote?page=WikkaWiki'>WikkaWiki</a>""</a>""

5. Million-dollar question

Now, here comes the big question.
How can I have the camelcase rewriting rule parse and rewrite any camelcase-formatted strings except those that appear in the anchor text of an already rewritten link?

The question is tricky, because whereas in the above example cases like FetchRemote or HomePage that appear in the URI are easily dealt with by excluding camelcase words that are adjacent to characters like ", =, ' etc., a camelcase word within the anchor text can be preceded and followed by other text, like:

""<a href='FetchRemote?page=HomePage'>Here's some text preceding WikkaWiki, which is in turn followed by other text</a>""

How do I exclude WikkaWiki from being rewritten?

Thanks if you had the patience to read this long and boring page.

-- DarTar

Possible solution

OK, I think I have found how you should approach this. (Thanks for distracting me all day with a challenging puzzle. ;-)) Rather than writing all the code for you, I'll give an outline how I would apprach it - but if you need help, let me know.
I think you need a three-step approach, just two preg_replace() calls can't handle it (or at least I can't think my way through it). Here goes:

Instead of the first preg_replace() (forced link), use a preg_replace_callback(); inside the callback function you can then separately treat the link text. What I'd do here is use yet another RE to find all occurrences of a CamelCase string, and enclose each of them with a special pair of "tags" (like £ or ¥).

Rewrite your $camel RE so that a string within those special tags isn't matched; then do your preg_replace() - only "lone" CamelCase words will then be rewritten.

Finally, clean up by simply removing the special "tags" you used to mark the "don't-replace-these" CamelCase words.

--JavaWoman

Brilliant, I'll try to cook up something.. -- DarTar

(later) OK, it works almost fine!
The last thing that I need to know - sorry for the hassle :) - is how to call a global function from within the callback function.
Basically, I have

function MarkCamel {
...
}

$content = preg_replace_callback($forced, 'MarkCamel', $content);

I need to use the global Href() function within MarkCamel. Is this possible? How?
$this->Href() doesn't seem to work :-/
-- DarTar

Off the cuff (I'm about to go to the train to spend the Sinterklaas weekend with my parents...):

Within your function declare: global $wakka; - that makes the object "known" to your function.

To call any function (like Href()) from the object from within an external function, use $wakka->Href() instead of $this->Href() (and similar).

I think that should do it. There's an example lurking somewhere in the Wikka code, I think. Dig a bit if this doesn't do it.
HTH --JavaWoman

CategoryRegex CategoryDevelopmentActions CategoryDevelopmentFormatters CategoryReview

Wikka Documentation : LinkRewriting