Lisa Source Code: Understanding Clascal

On January 19, Apple and the Computer History Museum released the source code to the Lisa Office System 7/7 version 3.1, including both the complete Office System application suite and the Lisa operating system. (The main components not released were the Workshop environment and its tooling, including the Edit application and the Pascal, COBOL, BASIC, and C compilers and the assembler.) Curious people have started to dig into what’s needed to understand and build it, and I thought I’d share some of what I’ve learned over the past few decades as a Lisa owner and enthusiast.

While Lisa appears to have an underlying procedural API similar to that of the Macintosh Toolbox, the Office System applications were primarily written in the Clascal language—an object-oriented dialect of Pascal designed by Apple with Niklaus Wirth—using the Lisa Application ToolKit so they could share as much code as possible between all of them. This framework is the forerunner of most modern frameworks, including MacApp and the NeXT frameworks, which in turn were huge influences on the Java and .NET frameworks.

One of the interesting things about Clascal is that it doesn’t add much to the Pascal dialect Apple was using at the time: Pascal was originally designed by Wirth to be a teaching language and several constructs useful for systems programming were left out, but soon added back by people who saw Pascal as a nice, straightforward, compact language with simple semantics that’s straightforward to compile. While in the 1990s there was a bitter war fought between the Pascal and C communities for microcomputer development, practically speaking the popular Pascal dialects and C are almost entirely isomorphic; there’s almost nothing in C that’s not similarly simple to express in Pascal, and vice versa.

So beyond standard Pascal, Apple Pascal had a concept of “units” for promoting code modularity: Instead of having to cram an entire program in one file, you could break it up into composable units that specify their “interface” separately from their “implementation.” Sound familiar?

When creating a unit under this model, both the interface and the implementation can go in a single file, but in separate sections. So let’s say you want to create a unit that makes some simple types available along with procedures and functions to operate on them. (In code examples, I’m putting keywords in uppercase since Pascal was historically case-insensitive and it helps to make clear the distinction between language constructs and developer code.)

UNIT Geometry;

INTERFACE

  TYPE
    Point  = RECORD
               h, v: INTEGER;
             END;

  VAR
    ZeroPoint: Point;

  PROCEDURE InitGeometry;
  PROCEDURE SetPoint(var p: Point; h, v: INTEGER);
  FUNCTION EqualPoints(a, b: Point): BOOLEAN;

IMPLEMENTATION

  PROCEDURE InitGeometry
  BEGIN
    SetPoint(ZeroPoint, 0, 0);
  END;

  PROCEDURE SetPoint
  BEGIN
    p.h = h;
    p.v = v;
  END;

  FUNCTION EqualPoints
  BEGIN
    IF a.h = b.h AND a.v = b.v THEN BEGIN
      EqualPoints := TRUE;
    ELSE BEGIN
      EqualPoints := FALSE;
    END
  END;

END.

Reading through this code, what’s the first thing you notice? While InitGeometry would typically be written without parentheses, as is normal for a zero-argument procedure or function in Pascal, functions and procedures that do take arguments and return values are also written without parameter lists but only in the IMPLEMENTATION section.

This is why, in a lot of the Lisa codebase, they would actually be written like this:

  FUNCTION EqualPoints{(a, b: Point): BOOLEAN}
  BEGIN
    IF a.h = b.h AND a.v = b.v THEN BEGIN
      EqualPoints := TRUE;
    ELSE BEGIN
      EqualPoints := FALSE;
    END
  END;

This is because, despite being “wordy,” Pascal also typically tries to minimize repetition and risk of error. So since you’ve already specified the INTERFACE why specify it again, and potentially get it wrong?

What’s interesting about Clascal is that it does the same thing! You define a class and its methods as an interface, and then its implementation doesn’t require repetition. This may sound convenient but in the end it means you don’t see the argument lists and return types at definition sites, so everyone wound up just copying & pasting them into comments next to the definition!

A couple of other things that are interesting about Clascal is that it sticks closer to Smalltalk terminology than most modern systems other than Objective-C (and, marginally, Swift): Instead of this it has SELF and instead of “member functions” it has “methods,” as PARC intended. This makes perfect sense as a bunch of the people who created and used Clascal came from PARC.

So to define a class, you simply use SUBCLASS OF SuperclassName in a TYPE definition section, provide your instance variables as if they were part of a RECORD, and declare its methods using almost-normal PROCEDURE and FUNCTION declarations (not definitions!) that require an OVERRIDE keyword to indicate a subclass override of a superclass method.

So the above code would look like this adapted to Clascal style:

UNIT Geometry;

INTERFACE

  TYPE
    TPoint = SUBCLASS OF TObject
               h, v: INTEGER;
               FUNCTION CREATE(object: TObject, heap: THeap): TPoint;
               PROCEDURE Set(h, v: INTEGER);
               FUNCTION Equals(point: TPoint): BOOLEAN;
             END;

IMPLEMENTATION

  METHODS OF TPoint;

    FUNCTION TPoint.CREATE{(object: TObject, heap: THeap): TPoint};
    BEGIN
      { Create a new object in the heap of this class, if not
        initializing an instance of a subclass. }
      IF object = NIL THEN
        object := NewObject(heap, THISCLASS);
      SELF := TPoint(TObject.CREATE(object, heap));
    END;
    PROCEDURE TPoint.Set{(h, v: INTEGER)};
      SELF.h := h;
      SELF.v := v;
    END;
    FUNCTION TPoint.Equals{(point: TPoint): BOOLEAN};
      Equals := a.h = b.h AND a.v = b.v;
    END;
  END;

END.

In addition to SELF there’s of course SUPERSELF to send messages to your superclass instead. And messages are sent via dot notation, e.g. myPoint.Set(10,20); to send Set to an instance of TPoint. It’s just about the most minimal possible object-oriented addition to Pascal, with one exception: It takes advantage of Lisa’s heap.

Just like Macintosh, Lisa has a Memory Manager whose heap is largely organized in terms of relocatable blocks referenced by handles rather than fixed blocks referenced by pointers. Thus normally in Pascal one would write SELF^^.h := h; to dereference the SELF handle and pointer when accessing the object. However, since Clascal knows SELF and myPoint and so on are objects, it just assumes the dereference—making it hard to get wrong. What I find interesting is that, unlike the Memory Manager on Macintosh, I’ve not seen any references to locking handles so they don’t move during operations. However, since there isn’t any saving and passing around of partially dereferenced handles most of the time, I suspect it isn’t actually necessary!

Honestly, as late-1970s languages go, it isn’t so bad at all. It wouldn’t even be all that difficult for the editor to show this information inline anyway, it’s the sort of thing that can be done fairly easily even in static language development environments from the 1970s.

Mastodon URIs, not URLs

One of the annoying things about Mastodon is that it’s tough to share Mastodon links and have them open in your favorite app instead of in a web browser. This is due to the lack of a shared scheme or a shared server—which makes sense for a distributed/federated system, but doesn’t help its usability.

One thing the community should do is use a URI instead of a URL or a Twitter/AOL-style “handle” to refer to an account: A URI is a Uniform Resource Identifier that is resolved to a URL, which makes it easier to have all links to Mastodon accounts go to the user’s preferred app—and also enable the global namespace that ATP cares about so much.

So instead of saying my Mastodon account is either @eschaton@mastodon.social or https://mastodon.social/@eschaton, I should say that my Mastodon account isacct:eschaton@mastodon.social. That will let the Mastodon app—or any other app that registers as a handler for the acct: URI scheme–resolve the URI into https://mastodon.social/@eschaton.

This acct: scheme is already well-defined in RFC 7565, and is very easy to resolve using the WebFinger (RFC 7033) protocol that Mastodon servers are already required to support as part of their support for federation: You transform it into a query against the specified domain, and that tells you what it should resolve to.

So there you go, a global distributed namespace for Mastodon—and other—accounts, based entirely on Internet standards.

The only improvement I could see is to also support the use of RFC 2782 DNS SRV records with WebFinger. This would stop requiring the server that’s sent the WebFinger query be the one named in the acct: URL. That is, instead of sending the query to whatever mastodon.social normally resolves it, it could be sent to any of a number of servers specifically set up for handling the queries. That would allow domains to run social.example.com etc. while still giving their users example.com usernames, without having to run additional software on the main web servers handling the example.com content.

But that’s just an optimization/evolution. For right here and right now, WebFinger is plenty easy, and so are acct: URIs.

Note: One thing this doesn’t resolve is linking to content on another Mastodon server. Thus one might still want a mastodon: URI scheme that works identically to an acct: URI but also allows reference a post or hashtag. That way you can say “this content from this account” and have the resolution happen such that clients don’t need to “shell out” to a web browser to present it. That’d eliminate another supposed advantage of centralized microblogging systems.

The Web Sucks Now

Web pages should be blazing fast on *25 year old* machines, modulo a little lag from more serious crypto.

Like, I should be able to build *1998 Mozilla* against a modern SSL stack on my Silicon Graphics O2, and browsing the web should be nearly as fast as it was using Netscape Navigator 4 on that system when it shipped.

The sole reason it’s not is the hubris of the people developing those web pages.

GG was a trial run

jorm:

it’s so clear to me that all the bullshit of the past 8 years was stoked so that we’d be too divided to enter this fight

I mean, I think everything. I think Brexit, Gamergate, Q-Anon.

In the end all of this feels like the weaponization of exploits in both our general and specific social cognition: General in that it represents an entire class of exploit, and specific in that it was tailored to the signifiers, mores, and normative style of the Anglosphere.

Within any group there’s a tendency to use similar metaphors and representations, as well as values, rooted in a shared literary traditions, and to use similar forms of argument to persuade (at least at the level of the laity). So if you want to intentionally spread specific ideas among a population, you could do worse than to study just what that populations literary tradition (and therefore metaphors and values) and preferred style of argument are.

The ways you spread an idea among Anglophones and Francophones may be very different, but the way you determine how to spread an idea among each is the same.

Federation Stability and Starfleet

So, the Federation is “fully automated luxury space communism.” Of course there are people who just hang out and don’t contribute, but that’s OK! They don’t need to!

But what do you do with the people who do need to? There are people who use the fact that they’re in a post-scarcity society to research, create, invent, make, that sort of thing. And that enriches their society further!

But what about the people who have a need to “do” but aren’t like that? The real misfits, weirdos, maniacs? Turns out, there’s a place for them, too: Starfleet!

Have an obsessive need for hierarchy? Want to create or work with barely-functional cutting-edge and frankly dangerous experimental technology? Eager for volumes of rules, full of loopholes, and the constant breaking thereof for a “higher purpose?” Then Starfleet’s for you!

It’s a total honeypot! It sure beats having these sorts of people screwing around with the actual society that trillions of people live in, and sometimes they can even do some good!

And that’s why “Lower Decks” is the most realistic Star Trek series. Thanks for coming to my talk at TEDx DS-3!

California Care

Instead of ballot propositions for bullshit like splitting up the state, California really needs a ballot proposition implementing universal healthcare and seizing the existing health care infrastructure for the people.

Treating health care as a for-profit business is a crime against humanity.

Ur-Fascism

Italian author Umberto Eco defined “Ur-Fascism” in an article in the New York Review of Books in June, 1995. It can be distilled into fourteen points.

It’s worth taking a look at these to see just what our society is dealing with when it comes to “alt-right” Neo-Nazis and the ideology they promote.

Thanks to forums poster Improbable Lobster for the succinct write-up that I reproduce here.

  1. The cult of tradition. “One has only to look at the syllabus of every fascist movement to find the major traditionalist thinkers. The Nazi gnosis was nourished by traditionalist, syncretistic, occult elements.”
  2. The rejection of modernism. “The Enlightenment, the Age of Reason, is seen as the beginning of modern depravity. In this sense Ur-Fascism can be defined as irrationalism.”
  3. The cult of action for action’s sake. “Action being beautiful in itself, it must be taken before, or without, any previous reflection. Thinking is a form of emasculation.”
  4. Disagreement is treason. “The critical spirit makes distinctions, and to distinguish is a sign of modernism. In modern culture the scientific community praises disagreement as a way to improve knowledge.”
  5. Fear of difference. “The first appeal of a fascist or prematurely fascist movement is an appeal against the intruders. Thus Ur-Fascism is racist by definition.”
  6. Appeal to social frustration. “One of the most typical features of the historical fascism was the appeal to a frustrated middle class, a class suffering from an economic crisis or feelings of political humiliation, and frightened by the pressure of lower social groups.”
  7. The obsession with a plot. “The followers must feel besieged. The easiest way to solve the plot is the appeal to xenophobia.”
  8. The enemy is both strong and weak. “By a continuous shifting of rhetorical focus, the enemies are at the same time too strong and too weak.”
  9. Pacifism is trafficking with the enemy. “For Ur-Fascism there is no struggle for life but, rather, life is lived for struggle.”
  10. Contempt for the weak. “Elitism is a typical aspect of any reactionary ideology.”
  11. Everybody is educated to become a hero. “In Ur-Fascist ideology, heroism is the norm. This cult of heroism is strictly linked with the cult of death.”
  12. Machismo and weaponry. “Machismo implies both disdain for women and intolerance and condemnation of nonstandard sexual habits, from chastity to homosexuality.”
  13. Selective populism. “There is in our future a TV or Internet populism, in which the emotional response of a selected group of citizens can be presented and accepted as the Voice of the People.”
  14. Ur-Fascism speaks Newspeak. “All the Nazi or Fascist schoolbooks made use of an impoverished vocabulary, and an elementary syntax, in order to limit the instruments for complex and critical reasoning.”

The very straightforward application of this to today’s political landscape in the United States and elsewhere should terrify everyone.

Raspberry Pi vs SPARCstation 20: Fight!

A couple weeks back, I tweeted the following:

Turns out a Raspberry Pi now is about 6 times as fast as a SPARCstation 20 was 20 years ago. And a Pi 2 is more like 15 times as fast.

I was a little low in my numbers, too — they’re more like 7 times and 16 times to 41 times as fast — since I was going from memory!

Here’s how I came up with that.

The BYTE UNIX Benchmark

The standard benchmark for UNIX systems back in the day was the BYTE UNIX Benchmark, a set of benchmarks originally developed at a university and fleshed out substantially by BYTE Magazine so they could evaluate the new servers and workstations that were coming to market.

Even though BYTE itself is no more (RIP) the benchmark lives on: The most recent version was posted on Google Code and had some additional portability and enhancement work done. These days, the most up-to-date version is on GitHub.

What’s useful about this benchmark is that it’s scaled, and UNIX hasn’t changed all that much in itself, so it’s still moderately useful as a way to compare systems with each other.

I’d recently made an offhand comment that the Raspberry Pi, despite feeling “underpowered” by today’s standards, was actually extremely powerful — and that it put a decent workstation from the mid-1990s to shame, the kind of system we tended to be jealous of as college students.

What was a SPARCstation 20?

Back then, Sun was the biggest UNIX workstation vendor, primarily because both their hardware and baseline operating system were good and they offered a ton of flexibility in their product line.

In 1994, Sun introduced a new lineup of SPARCstation systems that had dramatically improved performance compared to their previous models — the original of which was so iconic that it defined the “pizza box” form factor for desktop workstations — and the SPARCstation 20 was one of their flagships.

Here are some specs for the Sun SPARCstation 20 model 61, which shipped in June 1994:

  • One 60 MHz SuperSPARC CPU
  • 1 MB of cache
  • 32MB RAM (expandable to 512MB)
  • 20 MB/second SCSI-2
  • 1152 by 900 8-bit graphics

In 1994, this was quite a substantial system, and it cost $16,195 in its minimum configuration. (That’s $25,580 today!) And if you used one, it felt like it: This thing was wicked fast.

This was also the last system for which the BYTE benchmark was re-indexed, defining this SPARCstation 20 to have a score of 10.0.

The Benchmarks

Actually running the benchmarks under Raspbian Jessie on my Raspberry Pi and Raspberry Pi 2 was trivial, literally just a matter of cloning the git repository and running the script.

Here are the results. Note that the Raspberry Pi 2 has two sets of results, because the BYTE UNIX Benchmark runs once to get “single-CPU” performance numbers and another time to get “multi-CPU” numbers. Its single-CPU numbers are really more like “single process” numbers, however, since the other three cores aren’t actually disabled while the benchmark is run.

System Benchmarks Index Values SS20-61 Result SS20-61 Index RPi Result RPi Index RPi2x1 Result RPi2x1 Index RPi2x4 Result RPi2x4 Index
Dhrystone 2 using register variables 16700.0 10.0 1647374.0 141.2 3000237.2 257.1 1948737.7 1023.9
Double-Precision Whetstone 55.0 10.0 239.6 43.6 435.3 79.1 1729.8 314.5
Execl Throughput 43.0 10.0 167.7 39.0 321.5 74.8 1210.6 281.5
File Copy 1024 bufsize 2000 maxblocks 3960.0 10.0 30363.8 76.7 70026.8 176.8 110940.6 280.2
File Copy 256 bufsize 500 maxblocks 1655.0 10.0 9473.6 57.2 20353.5 123.0 31384.0 189.6
File Copy 4096 bufsize 8000 maxblocks 5800.0 10.0 76219.4 131.4 186926.9 322.3 296346.9 510.9
Pipe Throughput 12440.0 10.0 118393.6 95.2 181562.5 146.0 713070.2 573.2
Pipe-based Context Switching 4000.0 10.0 14539.1 36.3 33809.8 84.5 126241.1 315.6
Process Creation 126.0 10.0 434.6 34.5 1190.8 94.5 2572.9 204.2
Shell Scripts (1 concurrent) 42.4 10.0 354.5 83.6 1087.0 256.4 2395.0 564.9
Shell Scripts (8 concurrent) 6.0 10.0 44.9 74.8 301.0 501.7 317.0 528.3
System Call Overhead 15000.0 10.0 276169.1 184.1 399939.7 266.6 1545514.4 1030.3
System Benchmarks Index Score 10.0 71.9 165.6 417.4

What does this tell us?

A lot can happen in 20 years. Even when it comes to things like I/O throughput, where the Raspberry Pi really falls down compared to other systems — because it attaches to everything via USB — it’s still way faster than a mid-1990s Sun that we all thought was extremely fast.

In particular, according to the indexes, a Raspberry Pi is about seven times as fast as a baseline SPARCstation 20 model 61 — and has substantially more RAM and storage, too. And the Raspberry Pi 2 is sixteen times as fast at single-threaded tasks, and on tasks where all cores can be put to use it’s forty one times faster.

Ideally, this would also mean that even a Raspberry Pi Zero should feel exceptionally fast. However, our software appetite has grown even faster than our appetite for fast hardware, and the feel of systems compared like this can demonstrate that well.

What’s next?

Well, I just got a DragonBoard 410c, which is a quad-core 64-bit ARM board using a Qualcomm CPU, and which doesn’t have any of the major design issues of the Raspberry Pi…

SBCL test failures on ARM

For hacking/prototyping/fun purposes I have a few embedded systems laying around. For example, I have a couple of Raspberry Pi systems, one of the original Raspberry Pi model B boards and one of the new Raspberry Pi 2 model B boards.

And on everything, I have the latest Steel Bank Common Lisp building.

On my Raspberry Pi, which is an armv6 device, I see the following failures in SBCL’s unit tests:

 Failure:            debug.impure.lisp / (TRACE ENCAPSULATE NIL)
 Failure:            debug.impure.lisp / (TRACE-RECURSIVE ENCAPSULATE NIL)
 Expected failure:   packages.impure.lisp / USE-PACKAGE-CONFLICT-SET
 Expected failure:   packages.impure.lisp / IMPORT-SINGLE-CONFLICT
 (62 tests skipped for this combination of platform and features)

On my Raspberry Pi 2, which is an armv7 device, I see the following additional failures:

 Failure:            float.pure.lisp / (SCALE-FLOAT-OVERFLOW BUG-372)
 Failure:            float.pure.lisp / (ADDITION-OVERFLOW BUG-372)
 Failure:            float.pure.lisp / (ADDITION-OVERFLOW BUG-372 TAKE-2)
 Failure:            debug.impure.lisp / (TRACE ENCAPSULATE NIL)
 Failure:            debug.impure.lisp / (TRACE-RECURSIVE ENCAPSULATE NIL)
 Expected failure:   packages.impure.lisp / USE-PACKAGE-CONFLICT-SET
 Expected failure:   packages.impure.lisp / IMPORT-SINGLE-CONFLICT
 (62 tests skipped for this combination of platform and features)

This says to me that, contrary to what some have told me, SBCL probably does need to distinguish the various ARM instruction set variants.

Is anyone actually working on SBCL on ARM?

I also have a DragonBoard 410c on the way, and it might be nice to have a fast Lisp on ARM64, though I suspect that’s a bit further out…