The Web Sucks Now

Web pages should be blazing fast on *25 year old* machines, modulo a little lag from more serious crypto.

Like, I should be able to build *1998 Mozilla* against a modern SSL stack on my Silicon Graphics O2, and browsing the web should be nearly as fast as it was using Netscape Navigator 4 on that system when it shipped.

The sole reason it’s not is the hubris of the people developing those web pages.

GG was a trial run

jorm:

it’s so clear to me that all the bullshit of the past 8 years was stoked so that we’d be too divided to enter this fight

I mean, I think everything. I think Brexit, Gamergate, Q-Anon.

In the end all of this feels like the weaponization of exploits in both our general and specific social cognition: General in that it represents an entire class of exploit, and specific in that it was tailored to the signifiers, mores, and normative style of the Anglosphere.

Within any group there’s a tendency to use similar metaphors and representations, as well as values, rooted in a shared literary traditions, and to use similar forms of argument to persuade (at least at the level of the laity). So if you want to intentionally spread specific ideas among a population, you could do worse than to study just what that populations literary tradition (and therefore metaphors and values) and preferred style of argument are.

The ways you spread an idea among Anglophones and Francophones may be very different, but the way you determine how to spread an idea among each is the same.

Federation Stability and Starfleet

So, the Federation is “fully automated luxury space communism.” Of course there are people who just hang out and don’t contribute, but that’s OK! They don’t need to!

But what do you do with the people who do need to? There are people who use the fact that they’re in a post-scarcity society to research, create, invent, make, that sort of thing. And that enriches their society further!

But what about the people who have a need to “do” but aren’t like that? The real misfits, weirdos, maniacs? Turns out, there’s a place for them, too: Starfleet!

Have an obsessive need for hierarchy? Want to create or work with barely-functional cutting-edge and frankly dangerous experimental technology? Eager for volumes of rules, full of loopholes, and the constant breaking thereof for a “higher purpose?” Then Starfleet’s for you!

It’s a total honeypot! It sure beats having these sorts of people screwing around with the actual society that trillions of people live in, and sometimes they can even do some good!

And that’s why “Lower Decks” is the most realistic Star Trek series. Thanks for coming to my talk at TEDx DS-3!

Raspberry Pi vs SPARCstation 20: Fight!

A couple weeks back, I tweeted the following:

Turns out a Raspberry Pi now is about 6 times as fast as a SPARCstation 20 was 20 years ago. And a Pi 2 is more like 15 times as fast.

I was a little low in my numbers, too — they’re more like 7 times and 16 times to 41 times as fast — since I was going from memory!

Here’s how I came up with that.

The BYTE UNIX Benchmark

The standard benchmark for UNIX systems back in the day was the BYTE UNIX Benchmark, a set of benchmarks originally developed at a university and fleshed out substantially by BYTE Magazine so they could evaluate the new servers and workstations that were coming to market.

Even though BYTE itself is no more (RIP) the benchmark lives on: The most recent version was posted on Google Code and had some additional portability and enhancement work done. These days, the most up-to-date version is on GitHub.

What’s useful about this benchmark is that it’s scaled, and UNIX hasn’t changed all that much in itself, so it’s still moderately useful as a way to compare systems with each other.

I’d recently made an offhand comment that the Raspberry Pi, despite feeling “underpowered” by today’s standards, was actually extremely powerful — and that it put a decent workstation from the mid-1990s to shame, the kind of system we tended to be jealous of as college students.

What was a SPARCstation 20?

Back then, Sun was the biggest UNIX workstation vendor, primarily because both their hardware and baseline operating system were good and they offered a ton of flexibility in their product line.

In 1994, Sun introduced a new lineup of SPARCstation systems that had dramatically improved performance compared to their previous models — the original of which was so iconic that it defined the “pizza box” form factor for desktop workstations — and the SPARCstation 20 was one of their flagships.

Here are some specs for the Sun SPARCstation 20 model 61, which shipped in June 1994:

  • One 60 MHz SuperSPARC CPU
  • 1 MB of cache
  • 32MB RAM (expandable to 512MB)
  • 20 MB/second SCSI-2
  • 1152 by 900 8-bit graphics

In 1994, this was quite a substantial system, and it cost $16,195 in its minimum configuration. (That’s $25,580 today!) And if you used one, it felt like it: This thing was wicked fast.

This was also the last system for which the BYTE benchmark was re-indexed, defining this SPARCstation 20 to have a score of 10.0.

The Benchmarks

Actually running the benchmarks under Raspbian Jessie on my Raspberry Pi and Raspberry Pi 2 was trivial, literally just a matter of cloning the git repository and running the script.

Here are the results. Note that the Raspberry Pi 2 has two sets of results, because the BYTE UNIX Benchmark runs once to get “single-CPU” performance numbers and another time to get “multi-CPU” numbers. Its single-CPU numbers are really more like “single process” numbers, however, since the other three cores aren’t actually disabled while the benchmark is run.

System Benchmarks Index Values SS20-61 Result SS20-61 Index RPi Result RPi Index RPi2x1 Result RPi2x1 Index RPi2x4 Result RPi2x4 Index
Dhrystone 2 using register variables 16700.0 10.0 1647374.0 141.2 3000237.2 257.1 1948737.7 1023.9
Double-Precision Whetstone 55.0 10.0 239.6 43.6 435.3 79.1 1729.8 314.5
Execl Throughput 43.0 10.0 167.7 39.0 321.5 74.8 1210.6 281.5
File Copy 1024 bufsize 2000 maxblocks 3960.0 10.0 30363.8 76.7 70026.8 176.8 110940.6 280.2
File Copy 256 bufsize 500 maxblocks 1655.0 10.0 9473.6 57.2 20353.5 123.0 31384.0 189.6
File Copy 4096 bufsize 8000 maxblocks 5800.0 10.0 76219.4 131.4 186926.9 322.3 296346.9 510.9
Pipe Throughput 12440.0 10.0 118393.6 95.2 181562.5 146.0 713070.2 573.2
Pipe-based Context Switching 4000.0 10.0 14539.1 36.3 33809.8 84.5 126241.1 315.6
Process Creation 126.0 10.0 434.6 34.5 1190.8 94.5 2572.9 204.2
Shell Scripts (1 concurrent) 42.4 10.0 354.5 83.6 1087.0 256.4 2395.0 564.9
Shell Scripts (8 concurrent) 6.0 10.0 44.9 74.8 301.0 501.7 317.0 528.3
System Call Overhead 15000.0 10.0 276169.1 184.1 399939.7 266.6 1545514.4 1030.3
System Benchmarks Index Score 10.0 71.9 165.6 417.4

What does this tell us?

A lot can happen in 20 years. Even when it comes to things like I/O throughput, where the Raspberry Pi really falls down compared to other systems — because it attaches to everything via USB — it’s still way faster than a mid-1990s Sun that we all thought was extremely fast.

In particular, according to the indexes, a Raspberry Pi is about seven times as fast as a baseline SPARCstation 20 model 61 — and has substantially more RAM and storage, too. And the Raspberry Pi 2 is sixteen times as fast at single-threaded tasks, and on tasks where all cores can be put to use it’s forty one times faster.

Ideally, this would also mean that even a Raspberry Pi Zero should feel exceptionally fast. However, our software appetite has grown even faster than our appetite for fast hardware, and the feel of systems compared like this can demonstrate that well.

What’s next?

Well, I just got a DragonBoard 410c, which is a quad-core 64-bit ARM board using a Qualcomm CPU, and which doesn’t have any of the major design issues of the Raspberry Pi…

SBCL test failures on ARM

For hacking/prototyping/fun purposes I have a few embedded systems laying around. For example, I have a couple of Raspberry Pi systems, one of the original Raspberry Pi model B boards and one of the new Raspberry Pi 2 model B boards.

And on everything, I have the latest Steel Bank Common Lisp building.

On my Raspberry Pi, which is an armv6 device, I see the following failures in SBCL’s unit tests:

 Failure:            debug.impure.lisp / (TRACE ENCAPSULATE NIL)
 Failure:            debug.impure.lisp / (TRACE-RECURSIVE ENCAPSULATE NIL)
 Expected failure:   packages.impure.lisp / USE-PACKAGE-CONFLICT-SET
 Expected failure:   packages.impure.lisp / IMPORT-SINGLE-CONFLICT
 (62 tests skipped for this combination of platform and features)

On my Raspberry Pi 2, which is an armv7 device, I see the following additional failures:

 Failure:            float.pure.lisp / (SCALE-FLOAT-OVERFLOW BUG-372)
 Failure:            float.pure.lisp / (ADDITION-OVERFLOW BUG-372)
 Failure:            float.pure.lisp / (ADDITION-OVERFLOW BUG-372 TAKE-2)
 Failure:            debug.impure.lisp / (TRACE ENCAPSULATE NIL)
 Failure:            debug.impure.lisp / (TRACE-RECURSIVE ENCAPSULATE NIL)
 Expected failure:   packages.impure.lisp / USE-PACKAGE-CONFLICT-SET
 Expected failure:   packages.impure.lisp / IMPORT-SINGLE-CONFLICT
 (62 tests skipped for this combination of platform and features)

This says to me that, contrary to what some have told me, SBCL probably does need to distinguish the various ARM instruction set variants.

Is anyone actually working on SBCL on ARM?

I also have a DragonBoard 410c on the way, and it might be nice to have a fast Lisp on ARM64, though I suspect that’s a bit further out…

Sad…

image1384260977.jpg

Cupertino is at one edge of the Santa Clara Valley, one of the best places on the continent to grow fruit.

This display is in our Whole Foods, one of the (if not the) largest stores they have. All of the brands are local and don’t exist any more, because we paved them over in favor of single-family homes and office parks and fucking *lawns*.

It’s really sad, and would have been easy to avoid, too, by building **up** instead of **out**. Instead, there are always new ballot measures in Cupertino to try to limit the “scale” of building – in other words, to prevent building up – and ensure the sprawl stays. And I expect that applies to the rest of the Santa Clara Valley as well, and a huge percentage of the United States as a whole, everywhere a “subdivision” has replaced farmland.

Even if we could build high-rise apartments and offices, it’s not certain that the land we’ve covered with little shitbox houses, chemically-maintained lawns, and asphalt could even be used for agriculture again. The land may well be “used up” and require extensive rehabilitation to even support parkland, much less farming.

Someday people will look back on this as a monumental disaster, an utter failure of urban planning ands demonstration of our society’s lack of any capacity for forethought. Probably once all of humanity is facing permanent food and water shortages, later this century.

When to use NSOperation vs. GCD

Mac OS X has a number of concurrency mechanisms, and that increases with Snow Leopard. In addition to run loops, threads (both Cocoa and POSIX) and operations, Snow Leopard adds **Grand Central Dispatch** (GCD), a very lightweight way to represent units of work and the style of concurrency they need, and have the system figure out how to schedule them.

But wait, don’t we have that already in NSOperation? It shouldn’t surprise you in the least to learn that NSOperation, on Snow Leopard, is built atop GCD. However, there are a number of differences between the two, and for that reason people have started to ask “How should I decide which to use when?”

The straightforward answer is a general guideline for all application development:

> **Always use the highest-level abstraction available to you, and drop down to lower-level abstractions when measurement shows that they are needed.**

In this particular case, it means that when writing Cocoa applications, you should generally be using NSOperation rather than using GCD directly. Not because of a difference in efficiency, but because NSOperation provides a *higher-level abstraction* atop the mechanisms of GCD.

For example, you can set up a dependency between two NSOperations such that the second will only be run after the first is complete — even if they’re run on different queues. You can use KVO to observe the completion (or cancellation) of different operations — and you can create operations that support being cancelled in the first place. You can set a completion block to run after an application has finished. And you can, of course, create operations from blocks using *NSBlockOperation*.

You’ll also fit in better with Cocoa by using NSOperation in your high-level code. If you take a look at new Snow Leopard API on NSNotificationCenter, you’ll see one where you specify the NSOperationQueue on which you wish a notification to run a block.

Ultimately, you spend a little bit of overhead to use NSOperation instead of GCD, but you gain significant additional functionality that will be useful as you start to compose operations together. And that’s the biggest benefit of NSOperation: You can break up your application in terms of units of work that can not only be run on a queue, but also canceled, observed, and depended upon. This lets you easily define your data dependencies and ensure that you aren’t simply running code serially as a side-effect of locking.

Welcome to Snow Leopard!

Last week, Mac OS X 10.6 Snow Leopard was released!

Snow Leopard represents a lot of hard work by a lot of folks at Apple and at seeded third-party developers, and it really shows.

Now that it’s shipped, I can actually talk about some of the especially cool things this release has for developers.