Towards the Grand Unification

Today's programming world is quite different from 20 years ago, when C/C++ was still the dominant language(s), and we just need to learn one language(s) to do all our work. Today, there are 3 major paradigms of applications: the oldest command line apps, the desktop GUI apps and the latest web apps. The last one is quite different from the other two as it utilizes many DSLs (domain specific languages). These DSLs made us more expressive, but also gave us all the headaches of a multi-lingual speaker. Isn't it great if we just need to speak one "universal language"?

In the following table, I just list down some of the technologies that a web programmer may encounter today, and many of them are "contributed" by W3C:

Syntax and Serialization	XML, XML Namespace, HTML, XHTML, JSON, MIME Message, XOP
Data Model	hierarchical data model: XML Infoset, XML Schema, XQuery Data Model; metadata data model: RDF, RDF Schema, OWL; object (class-based vs. prototype based) data model; relational data model;
Schema and Pattern	DTD, XML Schema, RELAX NG, XQuery Sequence Type; OWL; RegEx;
Path and Reference Languages	URI, XPath, XPointer, XLink, XInclude; File system path and wildcards;
Query Languages	XQuery, XSLT, XBL, XProc; SPARQL; SQL, PL/SQL;
Scripting Languages	client-side: JavaScript, AJAX, jQuery, ActionScript; server-side: ASP/C#/JSP/Java/Perl/PHP/Python/Ruby, shell scripting;
Style and UI	HTML5, XHTML, CSS, DOM, XSL-FO, XForms, XFrames; SVG, Canvas element, SMIL; XUL, XAML;

As Bill put it:

"XML standards support a broad spectrum of capabilities and they continue to evolve.  However, these capabilities have evolved in uncoordinated, inconsistent and sometimes conflicting ways. In particular, many XML based standards try to define overlapping features, mechanisms and capabilities, that are not specific to their particular intent."

And I don't think W3C is ever going to, nor able to clean up its own mess. So Candle takes up the challenge to create something that is simple and coherent; something at least I'm happy to use myself. And hope that it will be taken up by the Internet community and proves that sometimes design-by-community can do a better job than design-by-committee.

In the following, I'll elaborate a bit more on some of the design principles that I follow while taking on this "grand unification" challenge.

The first important design principle of Candle is To Keep it Simple and Coherent. When I design Candle I strives to eliminate almost every overlapping features that I found in the related standards and technologies. If it is not so, there would be not point of adopting a new language; we might be better-off just stick with the embedded DSLs.

The other principle Candle embraces is to be as declarative as possible. You just describe the results you want, instead of specifying out the detailed steps of how to carry it out. However, there are two kinds of "declarative" languages. One kind is the modeling languages, like UML and Design Patterns, whose primary focus is on the design thoughts instead of the exact logic and executability. The other kind is the runnable declarative languages, like SQL, XQuery, XSLT, SPARQL, XML Schema, RELAX NG, OWL. Candle is definitely the second kind. Actually, I goes far beyond providing just a formal specification, I also go and implement every feature that I define in the language. It's always easier "declaring" something than having it implemented. The devil is in the details.This very down-to-earth approach ensures that Candle does not have the design-by-committee syndrome. And I also used Candle to develop real-world applications and utilities to ensure the features are indeed easy to use and coherent in the application development. (Some of these apps and utilities shall be released in the coming releases).

I think Bill might have a wrong impression of the procedural features in Candle. Candle does support procedural features in order to be a general-purpose programming language. But they are much higher-level than those in the conventional procedural languages. Candle is designed to be as functional as possible and it deliberately restricts the usage of assignment operation. Data construction, evaluation, pattern matching, query and transformation all uses high level functional features and never need assignment. And node update uses CRUD statements rather than assignment operation. And there's the separation-of-side-effects mechanism to preserve the pure functional features. The only case in Candle that needs assignment is in while-loop (Candle differentiates variable initialization and re-assignment). Yes, it is possible to completely eliminate the while-loop and local variable assignment through recursion, but I think most procedural programmers won't be used to that. So this is kept in Candle just as "syntax sugar". I think, no matter how high-level a language is, as long as it is a general purpose language, it'll need procedural features; even in human languages, we still use procedural instructions all the time.

The other pitfall I try to avoid when designing Candle is the pitfall of "good at all things, great at nothing". To avoid that, I ensures that Candle has a focused strength, that is at least as good as any comparable DSLs. That focused strength is hierarchical data processing. Thus XSLT and XQuery are chosen as the basis of Candle, and on top of that I add OOP and other coherent features. The OO features might not be able to compete with a full-fledged OOP language, but that's OK, as long as the unification brings in more benefits than the niche features that we have to give up.

Finally, Candles takes a modularized and layered architecture, so that different features are unified at different levels, and into different modules. Below is the architecture diagram of Candle platform:

A clean architecture is the key to the health of this whole unification. For example, Candle's styling and UI is supported by a Scene module that is quite orthogonal the Script module. This is like how the Gecko layout engine is separated from the SpiderMonkey Javascript engine in Mozilla. And Candle Scene module focuses on supporting just the vector graphics, while leaving the advanced drawing and charting features to a higher-level Charting library.

I'll write more blog articles on the details of the design rationales, considerations and trade-offs made while creating Candle. So do come back and check it out.