This post was written in late July and had a rest as a draft since then… I thought it needed some rewriting, and also that I needed to at least reach some conclusion on the topic before publishing it… then months passed by, and… oh well, here you are. A published draft. For your eyes only. Even if most of the ideas expressed here have ultimately been left behind.
There is something funnily recursive about my approach to design and its roots into chronic procrastination. This relationship may be driven by what I once said about coding… it’s like playing Lego with fractal bricks.
So where the hell do one stop specifying and/or refining a design, and start doing useful stuff ?
And, one more urgent question for today : what balance do one chose between the quest for more performance and the quest for more expressiveness (as I suspect NeREIDS would need a great deal of both, even if I’m just being guilty of premature optimization at the moment…)
As of now, Quest for performance has driven my recent long period of tooling for going towards independent processing of large chunks of data in one go.
In other words, list processing…
The words speak by themselves… there is one “odd” language amongst all computer languages : LISP, which happens to be a language named after ‘LISt Processing’. Odd, as in, some would say, almost divine, as compared to alternatives. Lisp, as old as it is, may be the holy grail to the Quest for expressiveness.
I’m not a Lisp guru, oh, I’m very very (very) far from it : I barely scratched the subject at school, then never used that language again. Yet I’m quite found of reading Paul Graham, one of the most convincing advocates of the language. In fact I enjoy reading his articles on many more topics than programming, even if (or would it be “because” ?) he may sound quite subversive at times.
Reading him while knowing nothing about Lisp really begins to tickle me. I don’t even have the slightest idea what useful things are possible to build with the language, let alone how to build anything useful.
I’m quite confident that I’ve a fair level of power at my disposal using C++. I’m quite confident that I now understand a lot more of the internals of C and C++ (and imperative coding in general) than a few years ago, and I’m quite confident I’m able to go quite far using it (provided I’ll keep on learning the parts I’m still missing). It’s a comfortable feeling, y’know ? Just keep doing what you do best. But it bugs me to no end to know there’s also a curtain there hiding Lisp(s) (family of) language(s), and to know I’m being oblivious to what’s behind. Could it be the case that I’m just wasting my time, not having seen the land of opportunities there ? Or on the contrary would I waste some precious time trying to look behind that curtain, now that *I’ve* *some* *concrete* *work* *on* *my* *hands* *dammit*, being here on this project with you. I’d lose any credit you could have given to the plea for my ability to deliver NeREIDS, and against my procrastinative nature.
At the same time I just feel I need to evolve on my coding skills, one way or another. Modern hardware and parallel architectures require a drastic shift in paradigms. The (dreamed) scale of NeREIDS, and it’s very nature, require that I envision that piece of work with a fresh eye. Confronting a new problem in a field with the eye of a freshman in said field has historically shown surprising results. No, please, don’t [source-?]-ask me here as in Wikipedia, it suits me to simply believe this for a while. I can’t really say I’m a freshman in Computer Science, but I can confidently assure you that whole parts of the discipline are alien enough to me that I can pretend just the same, without the slightest blink of an eye (except for the shame of secretively feeling like a freshman with a degree).
So, on one side there, you have my recent shift towards performance-optimized, parallel-suited, processing of lists in C++, and on the other side you have my readings about the promised land of Lisp languages where would reign, hand in hand, higher expressiveness and organic building of code, design and thought (possibly at the expense of performance).
In the middle of these forces driving my curiosity lies the need of doing concrete work for coming up with a working NeREIDS implementation. A game in a 3D virtual environment is one highly performance-sensitive application. Using a Lisp interpreter for it would be madness, anyway, right ? And what about integrating graphic libraries with a Lisp ? What about the (almost certain) need for a Lisp-code-to-C++-code interaction in the end, when similar integration tasks have always been a nightmare to me. Besides, I don’t know any Lisp, so why bother…
And yet, that feeling of missing something crucial there is almost aching. Towards which direction should I lean ? More performance or more expressiveness ? Both sides are about lists ! List PROCESSING. FFS !
And this was driving me crazy. Quite literally.
Reading some more, I finally came up with a stupid idea, although it may just work. Oh, I don’t even know if it would be an original one, now, hold on… I happened to read the online book Build your own Lisp. The author has published here a very pleasant “introductory” course to C, where he is teaching the C language by making his “students” (that could be you, by the way, just go take a look) implement, using C, nothing less than a (lightweight) Lisp interpreter.
Now wait a minute, sure I don’t know enough Lisp to think about a way to use it… (if it was a sword, I couldn’t tell its handle from its pointy end). But… I happen to know enough Lisp to do as this author did : implement an interpreter for a home-made Lisp-like language. Then I would learn in which way to use it as it goes.
Err, how could one implement a language when understanding nothing of it ? That’s just the beauty of Lisp. It has a strikingly simple core (so the interpreter code is quite small), and (allegedly) a strikingly infinite power.
Okay, so, embedding a Lisp interpreter of my own into NeREIDS C++ code seems doable and would alleviate the problem of integration and availability of graphic libraries. But what about the performance side of the matter ? Well, I may have found a way to have that part covered, so that both ends of the equation (speed of data-oriented processing in C and expressive power of Lisp) join in some kind of middle-point tailored to my needs. Hopefully getting the best of both worlds there.
If I fail, at least I’d have learned some Lisp along the way. If I moderately succeed… the resulting Lisp interpreter could turn out to be a scripting system for NeREIDS, a work I’ve postponed since then. If I am to fully succeed, then for that matter all of NeREIDS logic could end up being a Lisp expression, and if half of Lisp promises are true, that would be like having nuclear warheads at my disposal when going on with pure C++ would have been like using conventional weaponry.
“Blood, toil, tears and sweat”. I have no proof for Victory, but my guts tell me there is something worth exploring there.
Till next time, fellow readers !
Wait, “readers” does take an ‘s’ when there are two of them, right ?