A scenario where tests help facilitate compatibility and communication

A scenario where tests help facilitate compatibility and communication

Steve Kim

February 25, 2011

Let's start off with some backstory: there was a time when only one Ruby implementation existed in the world: Matz's own Ruby Interpreter (MRI) written purely in C. Some time later, JRuby arrived onto the scene and, while it was still in its infancy, an announcement was made by Evan Phoenix for a project called Rubinius.

Rubinius was not just an ordinary Ruby implementation; it was an implementation with a goal to be written in Ruby as much as possible, inspired and modeled off of Smalltalk-80. Soon thereafter, Brian Ford joined forces with Phoenix to continue Rubinius development under Engine Yard and to work on a specialized framework called MSpec.

This simplistic framework allowed the Rubinius team to create tests of “correct behaviors” found in MRI and to mimic those results in Rubinius. It was a basic idea designed to measure the progress of the project, and it reaped rewards far more than originally anticipated.

RubySpec became the name of the collection of “correct behavior” tests and the executable specification for all implementations in the Ruby ecosystem.

By having all implementations follow the standard definition of Ruby (which in this case were MRI and KRI), programmers would be able to move around to different implementations with ease and confidence.

RubySpec on Communication

As it stands, all Ruby implementations adhere to RubySpec. We now know how RubySpec facilitates compatibility among each implementation, but how does it facilitate communication? RubySpec tests are not only intended to be executed against implementations, but like any other open source project, it fosters contributions.

Implementers and even developers are encouraged to add tests to RubySpec to enhance and increase the value and quality of the test suite. By adding tests, it creates an opportunity for implementers of different implementations to talk to each other. This is what I would call a symbiotic relationship, one where all implementers have a common goal to make their implementations look and feel the same as MRI/KRI.

The Case of Other Languages

Unfortunately, this symbiotic relationship does not appear in other languages such as JavaScript and Smalltalk (at least not yet). JavaScript implementers took the specification from ECMA complied with most of what was in the specification, and added a language feature set on top.

Over time, after revisions and rewrites of their JavaScript engines, it became difficult to distinguish between what was originally part of ECMAScript and what was part of the implementation.

Perhaps that was never the goal of ECMAScript, but the language slowly became fragmented, if ever so slightly, and developers found it challenging to switch between platforms (in this case, browsers).

Smalltalk implementers also took the specification from ParcPlace and proceeded in different directions, never to look back. If you search the words, “Smalltalk” and “Balkanization,” you will come up with some interesting articles on why Smalltalk did not really take off.

While some will argue that both JavaScript and Smalltalk are backed by proprietary vendors and, therefore, not in their best interests, Ruby has proprietary vendors as well. Fortunately, though, nearly all of the implementations are open sourced and all of them use RubySpec.

Companies like Engine Yard, Microsoft, Apple, and Gemstone have created their own implementations and, while only two of them are fully implemented, developers are excited about using them and have confidence that the language they have grown to love will work the same in the new implementation.

What Can Be Done (Or Is It Too Late)?

Thanks to JavaScript developers like Juriy Zaytsev, there is a set of tests that implementers can use to find common ground in the JavaScript.

There is also the Sputnik test suite which determines how well an implementation adheres to ECMA-262, 3rd Edition. Of course, it is up to the implementers to consider trying to find common ground, but having a set of tests will reveal incompatibilities across JavaScript implementations.

For Smalltalk, there has been apparently more converging than diverging in the past decade, but we may never see the light at the end of the tunnel. Now reaching its thirties, finding a common ground may prove difficult.

As of yet, there is no standard test suite that all Smalltalk implementers use and, as a result, communication through tests does not occur. It may be too late because of the complexity involved with each implementation.

The only thing Smalltalk implementers have going for them is whether or not Seaside runs on their implementation, similar to how Ruby implementations use Rails to test their own implementations.

Final Thoughts

As you can see, tests can play a major role in software development. Not only does it give confidence about the code and design, but also allows communication, similar to acceptance tests. This is one particular case where tests can help a language ecosystem thrive and sustain compatibility and communication.

I propose this is one of the reasons why we love Ruby so much. Having common ground allows developers to fiddle with different implementations without worrying about syntactical issues, but at the same time, having the confidence that their code will work much the same way as it did from one implementation to another.