Tomorrow Is Already Here

A screenshot showing side-by-side comparison of Google's Gemini LLM and Google Search answering the question "could you move diagonally in ultima 1". Gemini does a very succinct job of this, giving a definite answer with no "extras", while Google Search does not come close to answering the question (at least above the fold).

I think we're at peak LLM usefulness here. A succinct answer I'm willing to trust, no ads, nothing seeking my further engagement. Basically I can't see a way for Gemini to get any better for this type of dead-end query than where it's at right now. Prepare thyself for the crapwave.

This is a great video of Lex Fridman interviewing Max Tegmark about AI and Moloch and Consciousness. It's a really long one. Maybe watch it in 15-minute increments. I had it open as a browser tab forever, half-way watched, but finished it a few nights ago. I may revisit it, as it's pretty dense and there are a shit-tonne of ideas in there.

"I kind of love programming. It's an act of creation. You have an idea, you design it, and you bring it to life, and it does something." - Lex Fridman - at about 1:35:00. He mourns the loss of some form of "innocence". He struggles a bit with how to articulate the feeling of loss in the face of LLMs that can now code. Lex, I feel you, but...

In and of themselves, advanced LLMs do not rob creators of a pride in their art. If you're feeling a loss of "specialness", then it most likely because you mixed some ego into it. Ego is everywhere. This blog post and the way I'm posturing the language in it is mostly about ego, to make it seem like I'm "in the know" when in reality I'm just as clueless as the next person.

And while it is definitely surprising that things are moving so fast, it's not incredibly surprising to me that a model trained on the entire sum of human output up to this point can do a lot of "our special human things" so well.

On the other hand, LLMs may indeed rob creators of the ability to make a living at their art because of greed.

Greed is what bothers me the most about this whole AI Hype Cycle. Speaking of posturing, we keep hearing about how AI will benefit humanity, and I believe that it can and will. Curing disease, eliminating the slave labor class, and becoming a multi-planetary civilization seem like really important goals. But no one can possibly believe that is what is driving the research. The prime motivator is making money. Expand the profit margins of the big corporations by enabling them to do more while paying fewer humans to do it. Or grow startups so they can become big corporations or get acquired by big corporations.

The fact that we can all see this and yet do nothing different is the term "Moloch" that Max speaks about, that apparently was an ancient Carthaginian demon, but is a nice label to identify why things get shitty [1] [2] mixed with the prisoner's dilemma and the race to the bottom. I still have to read the full extent of this Scott Alexander piece, but you can get the gist from the video.

The insane push to always show growth and be more profitable than the last time. Where does it come from?

A cartoon drawing made by an LLM of people with disturbing faces and limbs having fun on a roller coaster that ends abruptly in a trash bin.
Courtesy of some mega-corp AI - I honestly forget which, probably meta.ai

Humans are an intelligence that is constantly trying to optimize for happiness (at some biological level, literally the neurochemicals floating around in our brains giving us sensations). Long ago we equated happiness with money because it gives us the ability to pursue whatever we want. Seems logical, if sad.

Corporations as a form of intelligence is another weird thing briefly talked about in the video (1:17:00) and mentioned in their first discussion, which I found very interesting. By that argument, governments fit into the same bucket.

A corporation is an outcome of humans trying to acquire more money with less risk, it does this by being a construct that can live and operate larger than any single human can and operates with a huge amount of resources. The example Max gives is of a tobacco company CEO deciding one day that the company should stop making cigarettes - an act which would only result in that human being replaced as CEO.

A corporation, as an entity, is constantly trying to optimize for profit. Originally set up to serve society, the roles have been reversed and society is largely serving the institutions (the video also touches on how the really big corporations have captured the regulators).

Today's AI require huge amounts of compute power and data to be trained and operate - basically meaning corporations are the ones that can run the powerful ones. I wonder how long it will take for the roles to reverse such that the corporations are serving the AI.

One thought experiment: If today's AI can be turned off by killing power to a data center or deleting a model, how would an advanced AI make it impossible for this to happen? Would it be to couple the AI services to mega corporation profits such that turning off the AI means the death of the corporation? I'm not saying that huge LLMs are consciously scheming in this direction, but it certainly seems like we are pointing in a direction where giant companies naturally are serving this.

Humans gave birth to corporations and corporations are giving birth to the next wave of intelligence.

§1407 · April 30, 2024 · Software, Technology · (No comments) ·

Filed Under Pragmatica

dice

Once upon a time, I ported a turn-based strategy / tactical combat game to the web. The average play time for the game is quite long (many hours), so saving games is an important feature.

PC/Console games have two ways to do "game saves":

  1. Save on the machine, keyed to the user thanks to the operating system's file system and
  2. Save on the cloud, keyed to the user by some identity provider.

1) is easy to implement (writing and reading a file to disk). 2) is much harder to implement, costs money to run, and requires the gaming machine to have network connectivity.

Web games can do 2) just as easily as PC games. Since this is a hobby project with near-zero players, I don't plan to build my own Cloud service, and I don't trust any Cloud-hosted "gaming" API to be stable enough or not get killed in the future. So that leaves me with saving and loading games locally.

Unfortunately, browsers have a major road block: they do not allow native file system access. Back a decade ago when I started to write the game, it looked to me like browsers would eventually implement this: progress on browser compatibility and "native app" capabilities were pretty breathtaking at that time.

So I put up with some jankiness: To load a savegame, pick a file from your computer. To save a game, the web app downloads the savegame file to your computer. That kind of sucks and doesn't allow a save-as-you-play experience, but good enough for now. I threw in more jank with the option to use Google Drive API as well for a cheap-o version of a Cloud service. (Why not Play Games API? Because they eventually killed their Web API. See "stable forever" above.)

All this jank was an acceptable workaround until browsers caught up, or so I thought.

dice

Here we are a decade later. Though Chrome desktop has proposed and implemented the File System Access API for native file access, it's not quite enough when compared to what native apps can do - and other browser vendors have adamantly refused to implement true native file access, making this capability a sad bloated corpse floating in a lake. I'm saying native file access is dead in the water.

Why didn't I just use localStorage or IndexedDb for this? Well, for one, it was stubbornness and naïvety. For two, I worried about how much data I could store reliably. For three, I thought that users would expect their saved games in Chrome to be playable in Firefox or Safari on the same machine or want ways of passing their saved games around.

The fact my savegames are something like 35k means that storage was never really a concern.

The fact that most users only ever use a single browser, means I don't need to really worry that much about portability.

It's actually pretty straightforward to implement a virtual file system in IndexedDb that works good enough. Heck, Apple did it as some weird peace offering to people who wanted File System Access. [As an aside: as far as I can tell, an origin-private file system offers zero user-facing benefits over IndexedDb, you're just trading one API for another... so it's a devex offering. True native file access offers user benefits, such as backing up your save games, hacking, modding, sharing outside of the browser]

So in the end, I just wrote my own storage service that loads/saves games to an IndexedDB store in the browser. These days, superior user experience trumps web specification idealism for me every time.

apple

P.S. I'm no security expert. I'm sure Mozilla and Apple have good reasons for rejecting the File System Access API. I'm just sitting here loving the technology of the web (zero install, the APIs, JavaScript/Typescript) and the developer experience but wishing I had all the perks that native apps do.

P.P.S. I was surprised to learn that an IndexedDB instance for a page at foo.net is different than the IndexedDB instance of an iframe of foo.net hosted at bar.net. That's a subtlety I hadn't thought through.

§1394 · April 19, 2024 · Apple, Entertainment, Firefox, Games, Google, Software, Technology, Web · (No comments) ·

The Bethesda Slide

dice

I am a big BIG fan of Bethesda's Elder Scrolls and I consider Morrowind one of the best games ever made (my top 2 are Morrowind and Doom in no particular order). I have put in only a few hours into Starfield (so far), and I have to say I like it less and less. Primarily my complaints lie with all the loading scenes that break the illusion of an immersive, seamless galaxy to explore. It feels like Skyrim with even more disconnected regions.

In Morrowind you could walk from the wilderness into a city with no observable loading, the only loading screens were upon fast-travel (which was not necessary) or going indoors. As you leveled up, you could learn levitate spells that let you fly over seemingly insurmountable mountain ranges or water breathing spells that let you explore underwater. You could learn water walking and even walk across the ocean from the mainland to the new island included in the Bloodmoon expansion, if you were patient enough and didn't want to fast-travel via ship.

Oblivion did away with some of the crazier things (like Levitation and Water Walking), and added loading screens as you move into cities and this trend sadly continued into Skyrim. And yet, I still loved those games and devoured them, spending hours. Something about the ability to look at the horizon and say "I'm going to walk to there" was very captivating.

In Starfield, you get loading screen after loading screen, moving from location to location, outdoor to indoor, in and out of spaceships, launching from the surface, grav-jumping, etc. I know this criticism is not incredibly insightful or novel; many folks have already complained loudly about this online and contrasted it against the illusion of seamlessness in No Man's Sky. I know I'm just adding to the din, but I can't help myself, my disappointment is as vast as the regions in Starfield are not.

There are no enormous planets floating in endless space. You're just an ant hopping from leaf to pretty leaf trying to cross a river. I expected so much more.

I will give it a few more hours, but sadly it's been a dud for me so far. Anybody have a recommendation for a modern open world game that knows how to maintain the illusion?

§1376 · January 18, 2024 · Entertainment, Games, Microsoft · 1 comment ·

Turn of the Century Musical Asplosion

music

Apropos of nothing, to the person who once said to me that the 1990s had no good music, I offer a list of 40 albums worth owning. This list has a good drubbing of obvious Alternative Rock and Brit Pop picks, but banks and dips into Shoe Gaze, Noise Pop, Trip Hop, and Electronica, which are more recent musical interests of mine. I own other albums from the nineties that are Canadian Rock-weighted (Tragically Hip, Tea Party, Odds):

  • Heaven or Las Vegas by The Cocteau Twins, 1990
  • Gala by Lush, 1990
  • Achtung Baby! by U2, 1991
  • Ten by Pearl Jam, 1991
  • Nevermind by Nirvana, 1991
  • Loveless by My Bloody Valentine, 1991
  • Blood Sugar Sex Magik by Red Hot Chili Peppers, 1991
  • Screamadelica by Primal Scream, 1991
  • Peng! by Stereolab, 1992
  • Lazer-Guided Melodies by Spiritualized, 1992
  • So Tonight That I Might See by Mazzy Star, 1993
  • Siamese Dream by The Smashing Pumpkins, 1993
  • Debut by Björk, 1993
  • G-Stoned by Kruder & Dorfmeister, 1993
  • Zooropa by U2, 1993
  • Dummy by Portishead, 1994
  • Superunknown by Soundgarden, 1994
  • Weezer by Weezer, 1994
  • Post by Björk, 1995
  • (What's the Story) Morning Glory? by Oasis, 1995
  • Mellon Collie and the Infinite Sadness by The Smashing Pumpkins, 1995
  • The Bends by Radiohead, 1995
  • Exit Planet Dust by The Chemical Brothers, 1995
  • Electr-O-Pura by Yo La Tengo, 1995
  • Odelay by Beck, 1996
  • Emperor Tomato Ketchup by Stereolab, 1996
  • Among My Swan by Mazzy Star, 1996
  • Ok, Computer by Radiohead, 1997
  • Urban Hymns by The Verve, 1997
  • Ladies and Gentlemen We Are Floating in Space by Spiritualized, 1997
  • Homogenic by Björk, 1997
  • Opera by Tosca, 1997
  • I Can Hear the Heart Beating as One by Yo La Tengo, 1997
  • Northern Star by Groove Armada, 1998
  • Mezzanine by Massive Attack, 1998
  • Moon Safari by Air, 1998
  • Music Has the Right to Children by Boards of Canada, 1998
  • Glee by Bran Van 3000, 1998
  • Vertigo by Groove Armada, 1999
  • Surrender by The Chemical Brothers, 1999

That 1997 tho.

And since I want to keep the party going, here's an extension of this list into the first half of the 2000's, 20 more albums too good to leave off:

  • Since I Left You by The Avalanches, 2000
  • Felt Mountain by Goldfrapp, 2000
  • Kid A by Radiohead, 2000
  • And Then Nothing Turned Itself Inside-Out by Yo La Tengo, 2000
  • Simple Things by Zero 7, 2001
  • Amnesiac by Radiohead, 2001
  • Comfort Eagle by Cake, 2001
  • Goodbye Country (Hello Nightclub) by Groove Armada, 2001
  • Charango by Morcheeba, 2002
  • Machine Says Yes by FC Kahuna, 2002
  • Lost Horizons by Lemon Jelly, 2002
  • Details by Frou Frou, 2002
  • Elephant by The White Stripes, 2003
  • Black Cherry by Goldfrapp, 2003
  • Talkie Walkie by Air, 2004
  • When It Falls by Zero 7, 2004
  • Lemon Jelly.ky by Lemon Jelly, 2004
  • The Campfire Headphase by Boards of Canada, 2005
  • Haughty Melodic by Mike Doughty, 2005
  • The Cosmic Game by Thievery Corporation, 2005

I guess it's kind of easy to look back twenty years into the past and peer at "the best of" with the rosy hue of critical acclaim and record sales, but man, do I enjoy the music. Please drop me notes on what I've missed or what I should pick up from the next twenty years!

§1286 · January 3, 2024 · Entertainment, Music · (No comments) ·

Channeling a Communication Breakdown

JavaScript logo

Back when I first wrote my unzip implementation in pure JS using Web Workers (code here), JavaScript runtimes were a very new thing (NodeJS had been released less than a year before). Ok, I had played with C++ bindings to the V8 JS engine for a hobby video game engine I had been writing, but that was it for me when it came to "JavaScript outside of the browser".

Well over a decade later and JavaScript/Typescript runtimes are all the rage in this continuously fractious software world. Even so, it hadn't really ever occurred to me that the unzip/unrar/untar implementations in BitJS might be useful in NodeJS or other runtimes (Deno, Bun) until someone opened a bug.

Anyway, the way unzip/unrar worked was pretty straightforward: The host code passes bytes into the unzip/unrar implementation via a postMessage() call, the implementation does some bits and bobs as a WebWorker (aka not on the UI thread), crawling through bytes of the archive and emitting interesting events that the host code listens for (like "here's a file I extracted").

sequenceDiagram participant Host participant Worker box Worker JavaScript Context participant WorkerGlobalScope participant unrar.js end Host->>Worker: postMessage<br/>(rar bytes) Worker-->>WorkerGlobalScope: WorkerGlobalScope->>unrar.js: onmessage<br/>(rar bytes) Note right of unrar.js: unrar<br/>the thing unrar.js->>WorkerGlobalScope: postMessage<br/>(an extracted file) WorkerGlobalScope-->>Worker: Worker->>Host: onmessage<br/>(an extracted file) unrar.js->>WorkerGlobalScope: postMessage<br/>(2nd extracted file) WorkerGlobalScope-->>Worker: Worker->>Host: onmessage<br/>(2nd extracted file)

Unfortunately, Node still has not adopted Web Workers (though eventually they may); they even have their own different thing called Worker Threads - confusing. Anyway, it left me wondering how I should approach supporting Node... until I learned about MessageChannel / MessagePort, which are now supported nearly universally (as of Node 15).

So in the end, it continues to be pretty simple. The MessageChannel becomes the abstraction through which messages are passed, the host code owns one MessagePort, the unzip implementation owns the other, and the implementation no longer assumes it lives in a WebWorker (oh and thanks Dynamic Imports!).

sequenceDiagram participant Host Code participant Port1 box Any JavaScript Context (could be a Web Worker) participant Port2 participant unrar.js end Host Code->>Port1: postMessage(rar bytes) Port1-->>Port2: (MessageChannel) Port2->>unrar.js: onmessage(rar bytes) Note right of unrar.js: unrar<br/>the thing unrar.js->>Port2: postMessage(an extracted file) Port2-->>Port1: (MessageChannel) Port1->>Host Code: onmessage(an extracted file) unrar.js->>Port2: postMessage(2nd extracted file) Port2-->>Port1: (MessageChannel) Port1->>Host Code: onmessage(2nd extracted file)

This allows environments that support Web Workers to keep their Web Worker implementation and the NodeJS version to have the implementation in its main thread. If someone wants to make it more performant using Node's Worker Threads send pull requests!

It seems like all JS libraries that do intensive computations (like training ML models or mining teh bitcoinz) and then emit a series of events, should probably think of MessageChannel as the means of communication with the host software going forward so that the implementation can be ported to more environments. What? WebAssembly? ... oh shhhh!

This little weekend hack also let me write some decent automated unit tests for BitJS decompression, so hurrah for that too!

§1357 · December 27, 2023 · Uncategorized · (No comments) ·

← Older entries