One of the greatest and maybe a bit underappreciated achievements of late Thorsten Otto was the full Atari TOS 1.x-3.x source code restoration. He used what he could from Lonny's https://docs.dev-docs.org, dug out and restored all of the ancient build tools, reverse engineered missing pieces and supplied disassembled source code for the rest. The end result was breathtaking, a byte to byte exact build for all language variants, after no less than 25 years of being in the dark.
The missing piece for me always was the TOS 4.x work. Except the now nearly forgotten attempt for the STE from 2015, there hadn't been anything done in this field. Interestingly, those dev-docs.org archives have the best support for it (their history mostly starts at 4.x era) but rely on many more tools and most importantly, on Lattice C 5.5 which Thorsten had no experience with (I think he had built some kind of RE framework around the ancient Alcyon compiler). So it was destined to quietly sit until somebody picks up the gauntlet...
So is this going to be another story starting with Back in ... ? Of course it is!
Back in 2019 I saw a rather cryptic post by no one else than Christian Zietz: he claimed that some Atari source code contains not only the source code files but also their history. Wow, how could that be possible?!
So I took a look at those archives and yes, there were some weird RCS folders. Not knowing what they are for, and with no Claude at hand ;), I started googling lame queries like "RCS to git". Whatever I found at that time - scripts, tools, guides, ... was all useless, nothing worked.
One day in October 2023 I told myself that I'm going to crack this mystery, those weird filename.ext,v files are just text after all. Soon I found out that those are basically compatible with CVS, another ancient version control system. That gave me some toolset to work with but then again, except a few simple queries those damned checkouts just wouldn't work.
So that was time for ... custom tooling, of course. ;) So I analysed the whole RCS/CVS format, created a custom rcs_to_git converter going as far as parsing correct timezones and DST shifts to get all the precious commits into some presentable form. And most importantly, to find out why all of those tools were failing. That didn't take long... they were failing because basically the files were corrupted! The reason is how RCS/CVS works: it stores its latest revision as full text; the previous revision as a diff against the full text; the revision before that as a diff against the latest + previous diff etc. So break one revision and every older revision gets broken as well (considering the era when those tools were created - end of the 70s/beginning of the 80s - I'd assume that they wouldn't be so optimistic about reliability of stored data!)
Funny thing is that later I discovered that Atari engineers themselves had noticed the corruption. But it seems that they didn't have time to repair it (I'm sure that they had server backups somewhere) so they just wrote it down in a readme and carried on with their work, with 5 years of work history lost. ;)
First I was crushed but then I realised that I could actually partially recover it because only one revision was corrupted: all the previous ones still had their changes. And assuming that the revision in question didn't change the entire file, I could still get a lot of useful information from previous commits when repaired (like when they changed a value of some bit or added a function etc). And then of course I had the existing binary builds to disassemble.
Well, it took me about two months to get it into a pretty presentable form but then I ran out of steam: I had several disconnected branches, some components without any history (only their timestamps in a ZIP archive...) and in general I had no idea how to put it all together. So I let it rot on my disk for another 3 years (during which I almost lost it all)
... until now! As you might have guessed, Claude again helped me with the infamous last 10 % of the project. I was pretty sad to see that the analysis, tooling and git repository I so hard worked on in 2023 was recreated by Claude in about two hours (I didn't want to "spoil" it with my results and I used them as a verification step), it even found a better way how to restore the lost / corrupted content. Oh well. So a few days later, after many iterations and corrections, I can announce to you the outcome of my (our?) work: the complete TOS 4.x, MultiTOS (yes, a ROM version of the 4.92 disk image...), GEM.SYS (the legendary MultiTOS AES source code some had wished for back in the 90s...) and Desktop recreation, byte-to-byte exact to the released binaries!
TOS 5.0 ROM (4.92 BIOS/GEMDOS/VDI + MultiTOS 1.08 AES)
The hardest part was sewing it all together because:
- only BIOS, GEMDOS and the MultiTOS AES got their history, then there were just ZIP files dated at random moments between releases (fortunately usually with correct timestamps!)
- BIOS/GEMDOS history stops before TOS 4.04 release and no archive contained newer files either so those two had to be reconstructed for later MultiTOS releases
- MultiTOS AES history starts after TOS 4.02 as a WIP snapshot (so only other sources tell us that Eric R. Smith started working on it around Nov 1992)
- the archives seem to be sourced from different people's machines at different times so it took some time to understand relationship between each other
- And of course single-TOS AES and VDI had to be modified for each release because there was only very few ZIP files to take code from (this is where Claude helped the most and I fear where Thorsten suffered the most)
The most interesting 'trivia' I have found in various comments and documents so far:
- MultiTOS idea was actually born pretty early, in February 1991 Leonard Tramiel tasked Allan Pratt with creation of a multitasking GEMDOS. Allan's initial good-case estimate was May 1991. They shipped 20 months later.
- MultiTOS AES was Derek Mui's child, Slavik Lozben worked on the new VDI, Minna Lai on the window manager ... they all started sometime around the same time as Allan; in mid-1991 it was clear that they were going to use Eric Smith's MiNT.
- Allan Pratt managed to ship Sparrow's TOS 2.07, hired Eric in 1992 and then left the company. :)
- Eric Smith left with the release of MultiTOS 1.08 beta in August 1993, this was the end of development as such for the Falcon line (this is aligned with the end of hardware development).
- GEMDOS was actually built with GCC and its assembly output (!) was converted to Alcyon format for final builds; it seems that Ken Badertscher was the force behind Lattice C transition as he was also the one working on BIOS/GEMDOS after Allan had left.
- The origins of Srealloc(): "This call is brought to you by the ingenious wizards who created a machine with a video mode that uses more than 70% of system memory." :)
- The infamous boot delay was first hard-wired into TOS; only TOS 4.01 changed it to NVRAM.
- Tough debugging sessions: "The only way to debug the AES is to study the codes and try to figure out something from it."
- Uwe Seimet was one of (Multi)TOS beta testers
- After MultiTOS 1.01 the decision to create a full-blown TOS (instead of it being an add-on to MINT.PRG) was made; that meant AES which works in both single-TOS and MiNT. Also AES was converted to Lattice C.
- By June 1993 the work on TOS 5.01 was in progress, leading to the famous RAM TOS 4.92 as a boot floppy.
- The 4.92 had an interesting hardware check: it reads bit 4 of $FF9201 and, when it is clear, reports minor 1 in both _VDO and _MCH cookies. The Sparrow spec and the STE joysticks define only bits 0 to 3 of that byte so it could have been a check for the Microbox as a Falcon030 flavour.
Never released Swedish TOS 4.0x (the resource files couldn't fit into the ROM!)
🌎 https://github.com/mikrosk/tos4x contains everything: build tools, releases (for all available languages and sometimes even as standalone programs), clean separation between Atari's code and my changes. Verified to be byte-exact where a public binary is available.
Enjoy! :-)
Add comment