Why I invented Candle (II)

In the previous article, I talked about the motivation of inventing Candle from the perspective of a markup-data processing language. In this article, I'll continue the discussion from the perspective of a general-purpose programming language.

We all know that general-purpose programming languages evolve gradually over the years, from machine language to assembly language, then to procedural language. But is the evolution still going on? Since we already call the procedural languages high-level programming languages, are we going to have high-high-level programming languages? With the dominance of OOP languages, many people feel that we are already not far from the programming utopia. But are OOP languages really the best we can have?
While OOP languages, like Java, do mark an era in programming language history, their glory cannot completely shadow the glimmer of new inventions. The evolution of programming languages do go on.

Firstly, we see the rise of scripting languages, like JavaScript, PHP, Python, Ruby and Lua over the last decade. Are scripting languages completely different from typical OOP languages? I don't think so. I'd say 80% of their language features are the same, like all cross-platform, built-in garbage collection, etc. But the 20% differences are enough to tell one from the other. To just name a few:

• Scripting languages are dynamically interpreted without pre-compiling. Java and C# are probably the last programming languages that are distributed as compiled binary files instead of direct source files. Source distribution not only eliminates the trouble of compilation but also makes the maintenance more convenient. In the big company I last worked for, we do encounter problems like missing source code, or having to match the production binary with the right version of source code in our repository. The other important side-effect of source distribution is that it promotes open-source. Probably most if not all projects based on scripting languages are open-sourced. They simply don't have a close-source option, unless they hide their source code at server-side.
• Dynamic script construction and evaluation. Because eval("script") is such a piece of cake for scripting languages, we saw it being used often if not excessively in scripting languages, especially in JavaScript. This do add flexibility and dynamism to the application. But such behavior is seldom seen and difficult in pre-compiled languages like Java. We do see scripting languages get hosted in Java runtime environment, but that's another scripting language, not Java itself.
• Dynamic typed instead of statically typed. Scripting languages, like Python, JavaScript, Lua, are dynamic typed, which eliminates the complicated generic programming in Java and C++. Yes, we have to pay a price - that is less type errors can be detected statically.
• Built-in support of two common collection types, list and map. Collections can be more easily constructed, merged, sliced, filtered in scripting languages than in OOP languages.
• Prototype based instead of class based OOP. This allows methods in the objects to be changed dynamically.

The purpose of going through these points is not to convince you that scripting languages are better than OOP languages, but to highlight to you that programming languages do evolve gradually over the years.

And the evolution has go beyond the scripting languages. Over the years, we also see the wide adoption of several DSLs (domain specific languages), including SQL, RegEx, server-side scripting (JSP/ASP/PHP/etc). And yet more DSLs, like XPath, XSLT, XQuery, RELAX NG, SPARQL, are appearing on the horizon. The wide adoption of these DSLs shows the limitation of today's general-purpose programming languages, both OOP and scripting. While the DSLs give us strength in specific domain, they also give us big headache in general. When they are embedded in the GPLs, there's lack of support of these DSLs from the IDE. They have to be escaped as string; their syntax is not checked; there's no intellisence or symbol management; and they can hardly be debugged, refactored and tested.

So is it possible to have a general-purpose programming language that is as high-level as these DSLs?

Candle is invented to answer this question. The second driving force in inventing Candle is to create a more advanced GPL that have the core features of the DSLs built directly into the language. I won't claim that Candle has completely achieved that, but it has made some major advancement in this direction.

• Built-in hierarchical data processing: node trees can be as easily constructed, selected, queries, transformed and serialized as any objects or primitive data in OOP.
• Set-oriented expressions: the expressions in Candle can handle not only a single item, but also a sequence of items - making the language more expressive and convenient.
• Unified pattern language: Candle unifies several pattern related DSLs (including RegEx, BNF, RELAX NG, XQuery Sequence Type) into one consistent pattern language that can apply to sequence of items, nodes and strings.
• Unification of functional and procedural programming: Some multi-paradigm programming languages, like Python, already have a mixture of features from both worlds. But by introducing a mechanism, I called separation-of-side-effects, Candle unifies two worlds in a more orderly manner. In Candle, routines are divided into functions and methods. Functions are routines without side-effects and methods are routines with side-effects. The rule of separation-of-side-effects is that methods can call functions, but not vice versa. In this way, pure functional islands are well-preserved in the vast sea of procedural program.

As you start programming in Candle, you'll see the seamless integration of these features into one coherent language - no like the mixing of water and oil, but the solution of water and salt. Whether it is really the unified platform for desktop and Internet applications as it claims, you'll have to dive into it to see.