Category Archives: Productivity

Summer ’22 update

In attempt to increase frequency of status updates on the project after slow progress during the past year, here’s some information about what we’ve been working hard during the summer vacation.

Styled TextEdit updates

One of the most overlooked – but still very important – features has been the Styled TextEdit support. We already had started working on the foundations almost two years ago, and had last year got the drawing to work pretty well (mostly focusing on Marathon’s “computer terminal” screens) – but actual editing features of the styled text have been largely missing. Luckily SimpleText, with its fully-featured styled text editing features, provides a bunch of test cases for a number of them. Below is screenshot of some text styling at work:

Right now, styled text can be finally entered (using various style parameters such as face, font, size), and SimpleText can also open various text documents with styling information. Below is one example, the READ ME file of Prince of Destruction:

Opening a styled text document in SimpleText

But a lot of work needs to be still done, as major features are still unfinished (such as hit-testing of styled text records, which is needed for both mouse click, arrow key and selection support).

“AppleDoubler” tool

Motivation to work on the “AppleDoubler” tool got boost by a question asked in the June on emaculation about our file system choices. We had planned for a long time to make tool for converting native Mac files to the ADF format, but hadn’t taken time to work on that until now. It didn’t actually take more than a couple hours to write (plus some random time to do some tweaks later), but as nice effect we are able now to convert applications for testing blazingly quickly (compared to the manual conversion process earlier). This has led to discovery of a number of bugs which were fixed, and missing special cases of existing features that have been tweaked.

The AppleDoubler tool will be available as soon as we sort out our binary distributions to work a bit better (until now, files have been hosted here on the WP blog, but due to a bunch of issues with executables, and the time-consuming progress of updating downloads page, we will look into more automated alternate solutions, to allow more frequent updates, and also more downloads to be available.)

Software cursor

One longtime “to-do” task has been proper software cursor implementation, which has been delayed mostly because SDL’s cursor API has handled the monochrome cursors quite well for most of the use cases. However, there have been three downsides of this approach:

The “inverted” cursors (such as the I-beam, and Lemming’s cursors) don’t show up properly, as most modern host systems don’t support inversions in cursor graphics. Especially in lemmings this has been tricky in the monochrome mode, as the cursor would appear black on black background, making the game really hard to play
The native cursors don’t show up when doing screenshots of windows, or when capturing in OBS using window capture. This has been mostly prominent on Twitch streams where mouse has been invisible
Color cursor support has been unimplemented, as for proper ‘crsr’ display the mapping of cursor color table needs to be done

The final push for software cursor implementation came when we tried running Super Studio Session editor, which replaces a number of default cursor vectors with its own implementation (which it uses to align the “note” cursor on music sheet at certain spots). We saw that the cursor that application implemented all by itself worked mostly really well, but because of some ShieldCursor bugs, it was leaving trails at certain places. Below are screenshots of Super Studio Session (ignore the “THINK Pascal” window title, we don’t yet have proper general-purpose shell/desktop application yet, so we used the very practical “Transfer” feature of THINK Pascal to launch any application from the file system through Standard File dialog, without needing to do any extra work 😀 )

It took a couple days of work to get the software cursor to work (the work was sped up a lot by a number abstractions we had done earlier, related to VBL task execution etc), and adding color cursor support was not that tough as ‘crsr’ and ‘ppat’ resources share a common heritage, and thus their color table and depth expansion code could be shared. Below are some screenshots of Civilization, showing the color mouse cursor in action:

SoftWindows

As a curiosity, in progress of trying out various applications, we noticed that SoftWindows 1.x (the last 68k-compatible version of that application) seems to work pretty well. As it does have quite high memory requirements (considering the currently implemented 24-bit memory manager by default has only 8 megabytes of RAM), we did have to make a temporary hack to boost the RAM limit in the test configuration to 12 megabytes (leaving space for only 3 slots in the emulated slot address space – while the default configuration allows up to 7). This allowed us to actually boot into Windows 3.1 and run a number of games/apps on it (biggest limiting factor for what worked was that this version SoftWindows only emulates a 286, thus allowing only Standard mode in Windows 3, instead of the Enhanced mode required by a lot of applications). Below are some screenshots of what we tried out:

MIDI Manager research

As we knew that certain Sierra games do advertise MT-32 compatibility, we did some quick snooping around, and learned that at least Space Quest 1 does use Apple’s little-known MIDI Manager to handle the hardware MIDI playback (unlike Thexder, which appears to have its own serial driver for MIDI, choosing to use the serial port directly). This led to some curiosity about the MIDI Manager implementation, for which there is sadly very little documentation – an article about programming it on the always valuable MacTech archive and the universal headers. A couple articles from mid 90’s point to ARPA as source to buy the MIDI documentation and SDK from, which sadly is no longer available.

The work on MIDI Manager support is still ongoing, but we feel it might be relatively easy to write a layer that emulates MIDI Manager’s patching features, and wrap the MIDI driver as a abstract device that could pass the MIDI packets to host OS’ native MIDI interface (in Mac OS X case, the CoreMIDI). All MIDI Manager calls go anyway through the same SoundDispatch trap like a number of Sound Manager calls, so tapping to that interface is really easy. (Another question, though, is how well timing will work, and whether default MIDI synthesizer (or the SC-55 midi hardware) will handle Space Quest’s MIDI output that expects an MT-32…)

Various other updates

As mentioned before, there have been a ton of fixes and updates, but here is mention of a couple ones that might be of interest:

Popup Control CDEF (and TextEdit) fixes: There were a number of bugs in both the Popup Control drawing (and TextEdit setup) code, which manifested quite visually in the HyperCard script editor. Below are screenshots of “before” and “after” to show what kind of effects the fixes had:

Menu Bar Pattern Control Panel: As another curiosity, we tried this one old Control Panel from early 90s, to see what would happen – and seems that whatever magic it uses to draw patterns behind menu bar, it seems to be 100% compatible:

FracSin/FracCos: The After Dark module called “Spotlight” (not to be confused with the OS X search tool) was the first case that needed FracSin/FracCos implementation to work, so we used it as a test case for those traps and got it working pretty well:

Palette fixes: There were a number of bugs & missing features in Palette Manager code (for example, SetEntries had bug that made it fail incorrectly with cProtectErr, which caused a number of issues around the place). Here is Lode Runner: The Legend Returns editor, which had weird color issues before the fix (ignore the palette debugger in bottom right):

Lode Runner: The Legend Returns level editor

BitmapToRegion: This trap was already before used by Escape Velocity, but as that game needs a couple other improvements to reach playable state, Snood worked out pretty well as a use case for this (as it uses it for practically all of its sprites to convert their masks to regions for drawing). After a couple hours, the game seems to be quite happy with the routine (Snood 1.x pictured):

Snood using sprites with masks defined by BitmapToRegion

Default MDEF ‘cicn’ support: While trying out Snood, we noticed that Snood 2 was the first use case of color icons (‘cicn’s) in menu items. As the MDEF already had almost-working handling for this case, we added some temporary code to the Icon Utilities module and got nice color icons in menus now:

More blitter fixes/improvements: There were numerous fixes to basic blitters too, which affected test cases such as 3 in Three, “Spotlight” (when brightness was other than 0%), Shanghai II mahjong, etc). Here’s some screenshots of what we got working with the improvements:

Next step

We will continue working on the missing features as always, trying to reach the goal of a generic runtime release as soon as possible. We also will try to find solution for the binary file distribution in near future, so anybody interested trying the new features out could do that as soon as possible (especially considering the last released binaries are already getting quite old). We will keep you updated on this blog on our progress as always.

Full list of changes since last post

2022-07-31 00:45:08 +0300 • Implement colorizing srcOr rectangle blitter
2022-07-30 23:49:00 +0300 • Implement colorizing srcBic rectangle blitter
2022-07-30 13:34:42 +0300 • Fix missing handle derefence bug
2022-07-30 13:47:58 +0300 • Implement SaveEntries trap for Palette Manager
2022-07-30 04:09:17 +0300 • Disable potentially invalid check in Entry2Index
2022-07-30 03:08:06 +0300 • Stretching subPin blitter + improve call interface
2022-07-30 03:07:01 +0300 • Reinforce zone checks for handle operations
2022-07-30 01:27:53 +0300 • Fix AppendMenu/InsertMenuitem legnth > 127 cases
2022-07-30 01:14:39 +0300 • Implement a rudimentary PlotCIconHandle (for MDEF)
2022-07-30 00:39:04 +0300 • Merge branch 'master' of
2022-07-30 00:38:58 +0300 • Proper implementation for BitmapToRegion QD trap
2022-07-26 23:16:20 +0300 • Add dummy MIDIManager (+ MIDISignIn & MIDIAddPort)
2022-07-26 20:35:26 +0300 • Merge branch 'master' of
2022-07-26 20:35:16 +0300 • Implement PaletteDispatch ResizePalette selector
2022-07-26 18:28:13 +0300 • Fix timer thread race condition issue
2022-07-26 15:53:28 +0300 • Implement color QD opDefHilite PICT opcode
2022-07-26 15:52:42 +0300 • Implement opPnLocHFrac PICT opcode
2022-07-26 15:52:06 +0300 • Fix edge case of shielding cursor in classic mode
2022-07-26 14:51:53 +0300 • Fix GetNew(C)Window autopositioning code
2022-07-26 14:07:53 +0300 • Fix TESetStyleScrap (StyleDataPtr -> TextStylePtr)
2022-07-26 05:35:47 +0300 • Version 9 compressed resource support (+fixes)
2022-07-26 03:05:37 +0300 • Add dummy ReplaceText selector stub for ScriptUtil
2022-07-26 00:46:57 +0300 • Add TESetStyleScrap (needs still some work)
2022-07-25 03:33:36 +0300 • Update line start for redraw when setting heights
2022-07-24 14:12:29 +0300 • Implement styled TESetSelect
2022-07-24 14:11:48 +0300 • Partial implementation of styled TEInsert
2022-07-24 06:08:06 +0300 • Merge branch 'master' of
2022-07-24 06:05:17 +0300 • Work on styled text (TEContinuousStyle selector)
2022-07-24 06:03:22 +0300 • Implement AdMin blend mode in color region blitter
2022-07-23 22:33:53 +0300 • Call vector #4 for unimplemented instructions
2022-07-23 02:07:22 +0300 • AppleDoubler: cleanup tool console output logging
2022-07-23 01:40:40 +0300 • AppleDoubler: add timestamps,buildscript & cleanup
2022-07-21 16:26:26 +0300 • Fix incorrect cProtectErr check in SetEntries
2022-07-21 15:32:55 +0300 • Implement FracSin and FracCos traps
2022-07-19 15:40:00 +0300 • Only mark arc dirty, if drawn on the screen port
2022-07-19 15:02:32 +0300 • Fix some compiler warnings (missing QD includes)
2022-07-19 15:00:12 +0300 • Use bullet (0xA5) as mark in popupUseWFont CDEF 63
2022-07-19 14:58:54 +0300 • Fix one-off error in MBDF hittest (crashed popups)
2022-07-19 06:45:32 +0300 • Fix saving of port state in TextEdit calls
2022-07-19 06:44:33 +0300 • Fix warnings in cursor module
2022-07-19 06:13:22 +0300 • Merge branch 'master' of
2022-07-19 06:12:48 +0300 • Add color QD hilite & fix font state in CDEF 63
2022-07-18 21:14:00 +0300 • Merge branch 'master' of
2022-07-18 21:13:52 +0300 • Cleanup: Refactor cursor code out of Mouse module
2022-07-18 04:12:22 +0300 • Support also function key (F1-F15) key codes
2022-07-17 07:38:02 +0300 • Fix incorrect ShieldCursor offsetPt in CopyBits
2022-07-17 07:27:27 +0300 • Implement DefaultSetCCursor (for 'crsr' support)
2022-07-17 07:26:49 +0300 • Fix MCallSetCCursor (called wrong vector)
2022-07-17 07:26:03 +0300 • Fix handle resizing bug in GetCCursor
2022-07-17 06:15:09 +0300 • Fix cursor left-side clipping
2022-07-17 02:36:52 +0300 • Color QD monochrome software cursor implemented
2022-07-11 23:12:03 +0300 • Use DoVBLTask do update cursor in color QD mode
2022-07-11 23:09:54 +0300 • Implement AddIconToSuite selector in IconDispatch
2022-07-11 21:56:20 +0300 • Implement NewIconSuite selector for IconDispatch
2022-07-11 19:40:19 +0300 • Implement AttackVBL trap
2022-07-11 14:27:11 +0300 • More color vs b&w QD cursor code specialization
2022-07-11 02:43:18 +0300 • Cleanup Cursor module code
2022-07-11 02:42:49 +0300 • Specialize Color QD case in ShieldCursor
2022-07-11 02:35:42 +0300 • Init cursor at startup
2022-07-10 14:29:23 +0300 • Redraw software cursor in SetCursor, if changed
2022-07-10 13:13:16 +0300 • Fix clamping of mouse in DefaultCrsrTask
2022-07-10 04:07:45 +0300 • Merge branch 'master' of
2022-07-10 04:07:02 +0300 • Implement software cursor for "classic" QD mode
2022-07-10 04:05:22 +0300 • Fix DefaultShieldCursor (it did not erase cursor)
2022-07-10 03:58:34 +0300 • Fix CrsrObscure handling in the DefaultCrsrTask
2022-07-10 03:57:44 +0300 • Only setup AllocCursor and SetCCursor in color QD
2022-07-10 03:56:18 +0300 • Increase screen buffer refresh rate temporarily
2022-07-10 03:55:34 +0300 • Fix mouse native window clamping (-1 from bounds)
2022-07-08 02:35:07 +0300 • Merge branch 'master' of
2022-07-08 02:35:00 +0300 • Vectorize SetCCursor (& fix some setup bugs)
2022-07-08 02:17:30 +0300 • Merge branch 'master' of
2022-07-07 20:25:22 +0300 • Fix ShieldCursor bugs
2022-07-05 23:44:59 +0300 • Fix Serial.h
2022-07-05 02:56:01 +0300 • Merge branch 'master' of
2022-07-05 02:55:56 +0300 • AppleDoubler: Fix OSX's weird swap of ':' and '/'
2022-07-04 16:50:27 +0300 • Fix LoadSeg 32K+ offset wraparound (use unsigned)
2022-07-04 16:29:48 +0300 • Merge branch 'master' of
2022-07-04 16:29:41 +0300 • Add dummy serial driver stub (AIn/AOut/Bin/BOut)
2022-07-04 15:04:51 +0300 • Fix UPP call signature of JShieldCursor
2022-07-04 04:28:09 +0300 • Merge branch 'master' of
2022-07-04 04:27:59 +0300 • AppleDoubler: Also encode folder names
2022-07-04 02:16:46 +0300 • Fix byteswap bug in font size check
2022-07-04 02:00:14 +0300 • Fix FOND style lookup: start at 1st match,not last
2022-07-04 01:56:59 +0300 • Add Oregon Trail (1.x) to test app configs
2022-07-03 14:05:54 +0300 • Implement QDDone selector for QDExtensions
2022-07-03 13:56:22 +0300 • Merge branch 'master' of
2022-07-03 13:56:16 +0300 • Add (no-op) LockRange & UnlockRange in FSDispatch
2022-07-03 04:29:30 +0300 • Add first version of AppleDoubler converter tool
2022-07-02 06:13:32 +0300 • Handle NFSGetDirInfo errors in NFSCreateFileHook
2022-07-02 06:12:36 +0300 • Implement DateToSeconds
2022-07-02 04:06:13 +0300 • Add SoftWindows test app to JSON configs
2022-07-02 04:05:42 +0300 • Implement GetGWorldDevice selector in QDExtensions
2022-06-18 02:05:16 +0300 • Add ZPC to test app config JSONs
2022-04-28 04:01:02 +0300 • Fix crash in TECallCharByte utility routine
2022-04-28 04:00:57 +0300 • Add missing header file from previous commit
2022-04-28 03:57:54 +0300 • Tweak naming of some private TE data structures
2022-04-28 01:18:54 +0300 • Fix GetCTable crash of locked 'clut' in 24bit mode
2022-04-23 22:04:03 +0300 • Implement styled TESetStyle
2022-04-23 21:42:07 +0300 • Work a bit on InitScripts (load smRoman 'itlb' 0)
2022-04-23 13:55:03 +0300 • Add Font2Script & implement FontScript+IntlScript
2022-04-18 11:28:33 +0300 • Temporary fail-safe in _Enqueue
2022-04-18 02:06:31 +0300 • Finish faster non-MeasureText path in NPixel2Char
2022-04-17 22:52:23 +0300 • Use NPixel2Char in TextEdit (FindLine etc)
2022-04-17 22:50:50 +0300 • First version of working NPixel2Char in Script Mgr
2022-04-16 21:08:10 +0300 • Implement SndControl (classic SM2)
2022-04-15 05:03:02 +0300 • Prep TEHitTestHook calls to work with NPixel2Char
2022-04-15 04:59:57 +0300 • Handle 0x2C (font) and 0x2E (glyph state) in PICTs
2022-04-13 16:20:10 +0300 • Fix Munger ptr2=nil & len2=0 case (search)
2022-04-13 02:41:36 +0300 • Allow bypass of ampCmd and timbreCmd in snth 4101

TextEdit update: first styled text features added

The progress has been a bit slow in the last month because of us being extra busy at our regular daytime jobs, but we did manage to squeeze some long-awaited changes in. We finally started the work on multistyled textedit support, which is extension added to TextEdit API around System 6.0.4, allowing more than a single style to be applied within text edit records.

As we did not have much time to work on this yet during November, we only have so far roughly these features implemented for style support:

Multistyled TE record creation using TEStyleNew
Partial setting of appending multistyled text using TEStyleInsert
Adjusting existing styles using TESetStyle
Drawing multistyled text within the TEDoText’s teDraw selector
Measuring multistyled text

And the following style properties can be used within the styles:

Style font family
Text face (bold, italic, underline, etc…)
Font size
Text color

From the test applications we have been running, the most important ones requiring styled textedit support have been HyperCard 2.x and Marathon. Below is screenshot of HyperCard 2.0 Home stack, with multistyled text showing right on the welcome screen:

HyperCard 2.0 Home stack with multistyled text

And even more importantly, Marathon terminals can now be opened and operated, which means that we can finally proceed forward in Marathon gameplay to the other levels beyond the first one:

Marathon’s terminal view, with multistyled textedit field used for the text rendering

However, there are still some missing features, of which we need add these most important ones next:

Finalize TEStyleInsert to handle given StScrpHandle data properly
Implement other multistyle TEDoText selectors, such as teFind and teCaret, to allow actual styled text editing
Merge identical adjacent styles after style adjust operations, to avoid excessive style run array growth
Add other missing selectors to TEDispatch where needed
Also implement any other missing toolbox calls used by textedit where needed

With current speed of progress, the most important ones of the remaining parts textedit API should be usable within a couple of weeks. Unless we get too distracted by playing Marathon 🙂

Full list of changes since last post

2020-11-29 23:52:29 +0200 • Added PortChanged (no-op) selector in QDExtensions
2020-11-29 23:27:24 +0200 • Add JSON cmake config for HyperCard 2.2 test app
2020-11-29 23:26:55 +0200 • Add no-op dummy InitDateCache ScriptUtil selector
2020-11-29 23:22:50 +0200 • Implement partially ScriptUtil's SetEnvirons
2020-11-28 04:47:13 +0200 • Fix clamping of negative offset in TEPinScroll
2020-11-28 04:46:09 +0200 • Fix clamping of below bottom in pt-to-line routine
2020-11-28 03:48:03 +0200 • Implement TEGetPoint selector for TEDispatch
2020-11-27 01:30:49 +0200 • Handle styled text in TEDispose
2020-11-27 00:52:53 +0200 • Refactor TE locals, don't cache dereferenced hTE
2020-11-26 05:38:32 +0200 • Other fixes to make TESetStyle to work & add debug
2020-11-26 05:33:34 +0200 • Get correct style run idx for singlebyte edge case
2020-11-26 05:31:53 +0200 • Fix lineheight calc (wrong index & zero new slots)
2020-11-26 05:28:27 +0200 • Add missing TEApplyStyle byteswap (run->startChar)
2020-11-26 05:27:33 +0200 • Return correct style index @ existing style lookup
2020-11-26 05:26:30 +0200 • Set stCount, stHeight and stAscent for new styles
2020-11-26 05:25:31 +0200 • Add viewRect validation to TEDoText (debug mode)
2020-11-26 05:24:38 +0200 • Set stCount to 1 when inserting a new style
2020-11-26 05:23:52 +0200 • Re-deref TEHandle in TEStyleNew after hText alloc
2020-11-26 05:22:51 +0200 • Increment text pointer in TEFindLine for new style
2020-11-26 05:19:32 +0200 • Fix byteswap in numerator check in StdText
2020-11-25 04:28:32 +0200 • Work on TESetStyle (not yet working, but almost)
2020-11-23 03:24:16 +0200 • Start work on TESetStyle (and TEContinuousStyle)
2020-11-22 05:15:36 +0200 • Add style support to TEGetLineForV
2020-11-22 04:47:30 +0200 • TESetSelect styled text support
2020-11-22 04:45:35 +0200 • More TEStyleInsert work (line heights calculated)
2020-11-22 00:57:54 +0200 • TEStyleInsert progress (TEFindLine style support)
2020-11-21 04:53:05 +0200 • TextEdit BIG refactor & start adding TEStyleInsert
2020-11-21 02:46:25 +0200 • Implement GetCPixel and SetCPixel (ResEdit PACK 1)
2020-11-20 15:59:17 +0200 • Fix colorQD colormap bug, 1&2-bit modes flipped fg
2020-11-20 15:37:02 +0200 • Implement GetEntryUsage trap for palette manager
2020-11-20 15:23:33 +0200 • Implement old-style FONT lookup in RealFont
2020-11-20 14:47:15 +0200 • Fix bugs in styled text rendering, it works now
2020-11-20 13:56:34 +0200 • Don't set/read flags for purged/unallocated handle
2020-11-20 12:44:49 +0200 • Implement TEGetHeight (non-styled) for SimCity 2K
2020-11-20 12:28:04 +0200 • Make SWAP macro single-line safe
2020-11-20 05:48:11 +0200 • Fix for what looks like VERY strange bug in Excel…
2020-11-20 05:45:10 +0200 • Add dummy GetPortNameFromPSN selector @ OSDispatch
2020-11-20 05:44:27 +0200 • Add PPC trap dispatcher w/ dummy PPCInit & PPCOpen
2020-11-20 05:20:57 +0200 • Temporarily increase system heap size
2020-11-20 05:20:15 +0200 • Dummy switcher info data for testing Excel 3.0
2020-11-20 04:36:18 +0200 • Prototype styled TextEdit draw & linewidth measure
2020-11-10 22:48:13 +0200 • Implement TEGetStyleHandle for TEDispatch
2020-11-10 22:28:22 +0200 • Implement dummy IsLayer selector for LayerDispatch
2020-11-10 22:09:35 +0200 • Add TEDispatch, and TECustomHook and TEFeatureFlag
2020-11-10 03:22:20 +0200 • Implement TEStyleNew trap handler for TextEdit
2020-11-10 03:20:44 +0200 • Update Scarab of RA test app config to new version
2020-11-08 13:47:35 +0200 • Optional release version logging/assertions toggle
2020-11-08 13:46:56 +0200 • Minor code cleanups (logging, prep dirtyrects etc)
2020-11-08 13:27:22 +0200 • Fix src/mask address calculation in B&W CopyMask
2020-11-08 00:54:19 +0200 • Reinforce save of auxiliary regs in C->68k calls
2020-11-08 00:42:58 +0200 • Fix clamping of teLength in TextEdit's DoDraw proc
2020-11-04 16:17:43 +0200 • Add GWorld rowbytes calculation method selection
2020-10-31 03:26:53 +0200 • Handle wInGoAway also in SystemClick (Close DA)
2020-10-31 03:25:08 +0200 • Add missing change from the slot manager commit

Finding the Finder

Recently, somebody was asking on Emaculation forums, what would happen if Apple’s Finder would be attempted to run in M.A.C.E.? That was an interesting question, which motivated us to try it out.

Mac System Software architecture

To think about what would make Finder work, let’s consider the classic Mac system software as an architecture consisting of roughly three layers (in real Macs):

ROM: Most of Toolbox APIs, ROM resources and stage 0 boot loader.
System: Patches to ROM, extensions, system zone, the so-called “kernel” of old MacOS.
Finder: The “desktop metaphor” front-end running as a regular Mac application.

This categorization is very naïve, does not consider details such as role of hard drive drivers in boot process, etc. Additionally, M.A.C.E. combines roles of ROM and System as a single entity implemented in native code.

The interesting part to highlight is how Finder is basically “just another” application running on the System, and was even replaced by alternate startup applications on the old Macs (such as MiniFinder, MultiFinder, or other applications using the “Set Startup” option). As such, especially on System 6.x and older, the undocumented dependencies between System and Finder might be few in number, if any.

The first run

So, the question was, what would happen if we would try to run it? Well, we got this far on the first attempt, and with a couple minor fixes to progress further:

So, almost there. What we needed was a couple new (missing) routines and some fixes:

Fixed the 0x16 and 0x17 csCodes for .NativeFS driver, which return the logical/physical ICN# data for the volume icon (as seen on the desktop). We actually had that icon already one year ago, but this was the first time anything wanted to access it. 🙂
Added working directory indexing to GetWDInfo, which Finder uses to close all open working directories, presumably left open by misbehaving applications under System 6.x.
Fixed a buffer overflow bug in GetVolInfo (in volume name).
Implemented File locking APIs, which Finder wants to use to maintain the Desktop file.

Getting to the desktop

After a long and tedious session of debugging, it was discovered that due to a stupid bug in GetWDInfo Finder was crashing upon attempting to enumerate volumes. After that fix, we got nice Finder desktop, and a lot of functionality already working:

Operating on the Finder windows added need to implement a couple other missing traps, although as dummy versions for now (SetCatInfo and SetVolInfo, which Finder attempts to use to write metadata about modifications done in the spatial finder windows, icons, etc). Some points:

Finder and system icons appear as documents, but that might be because those don’t yet exist in the System file. Not sure about that yet.
It’s nice to see how a Finder 6.1.8 from Apple’s real System 6.0.7 behaves nicely even though M.A.C.E. System declares being version 7.0. 🙂
For some reason, doing “Get Info” on our Hard Drive turns our custom icon returned by the .NativeFS DRVR to a generic floppy icon.
It’s interesting how Finder considers at the moment our custom Hard Drive as “AppleTalk” device, but that might be because we’re hooking into the ExtFS file manager hooks which *I think* were used by AppleTalk in the real System 6.x. Just another thing which needs more investigation.
The modification and creation dates shown in “Get Info” (and list-style finder windows) use International Utilities date & time formatting functions, which are still placeholders. And as such, they only display mostly the numeric value of date-time instead of the proper, human-readable format.

Making the Desktop file work

At this point, we didn’t yet get the proper application and document icons to appear, so we did a bit more debugging on what was happening with the Desktop file – which Finder uses to keep track of known application creator & types, file comments, etc. It turned out, that we accidentally had all the “hasBundle” (0x2000) bits turned off in applications, which is why they were not detected by Finder.

As this issue was fixed, Finder started to process the applications properly, and quickly needed a some improvements to Resource Manager APIs. Mostly, it wanted to use UniqueID to allocate unique IDs for resources inside the Desktop file’s resource fork (to avoid conflicts). After that, updating Desktop file seems to work quite nicely, resulting in some fancy icons now appearing in the Finder under M.A.C.E.:

For now, running the real Finder remains as a curiosity, but nevertheless works as a great test case for a various number of Toolbox File Manager APIs which still need work (among some other minor bugs). The summary of the major known issues is:

The launching of programs “almost” works. Everything else appears to be fine, but it suffers from the same issue as previously found out using the “Transfer” command in THINK Pascal: The Font Manager data caches do not clear completely, leaving some dangling handles/pointers to the old application zone, which cause crash on attempting to reference them after the zone has been reinitialized for the new application.
Getting back to Finder (or any shell) is not yet possible, as ExitToShell is still hard-coded to terminate the entire emulator, instead of launching LMGetFinderName() application. Getting this to work would also encourage researching the Emscripten and Process Manager compatibility, as all of those features share a common need for a better in-emulator process management.
SetCatInfo/SetVolInfo are still dummy placeholders, so Finder is trying to use them to update metadata, though it is not yet saved.
Folder creation (_DirCreate), File/Folder deletion (_Delete), renaming (_Rename) are not yet implemented, so attempting those operations on Finder will still fail.
The _Eject trap is not yet implemented (and in any case you will not be able to eject the NativeFS drives).
As previously pointed out, Date & Time formatting does not yet work
The “Rebuild Desktop” operation (though using option + cmd keys at startup) does not yet seem to do the actual rebuild on a “fresh” start (i.e. if Desktop file has been wiped). Additionally, attempting that operation while old Desktop file exist will error out because _Delete is unimplemented, and old file cannot be removed out of the way.
Maybe other issues too, those are just the ones found out in the quick testing.

The “generic” M.A.C.E. environment might benefit of this though, as a replacement shell and/or program launcher, that could be optionally used by the end-user.

Full list of changes since last post

2019-12-12 18:19:41 +0200 • Add UniqueID/…1ID, to allow Desktop file updates
2019-12-12 17:49:18 +0200 • Add Disk Initialization Pack2 to keep Finder happy
2019-12-12 17:46:55 +0200 • Fix byteswap bug in NativeFS _Allocate handler
2019-12-12 15:00:12 +0200 • Add missing NativeFS.c changes for previous commit
2019-12-12 04:34:28 +0200 • Add dummy SetCatInfo and SetVolInfo selectors
2019-12-12 04:02:48 +0200 • Fix GetWDInfo bug (ioVRefNum & ioWDVRefNum mixed)
2019-12-09 02:58:47 +0200 • Fix memFullErr check in SetApplLimit
2019-12-08 06:31:06 +0200 • Fix filename overflow in GetVolInfo NativeFS call
2019-12-08 06:30:06 +0200 • Implemented CloseWD
2019-12-08 06:29:03 +0200 • Add WD indexing to GetWDInfo
2019-12-08 06:27:20 +0200 • Add Finder test app to cmake configs
2019-12-08 06:27:01 +0200 • Implement icon csCodes in NativeFS Control call
2019-12-08 06:25:52 +0200 • Make more macros in memory manager MSVC friendly
2019-12-08 06:25:18 +0200 • Add dummy SetFilLock and RstFilLock
2019-12-08 06:23:36 +0200 • Fix FinderName lowmem global
2019-12-06 23:44:06 +0200 • Add Bash Big Blue to cmake config for testing
2019-12-04 21:21:36 +0200 • Fix QD_LOG to respect debug settings
2019-12-04 20:48:17 +0200 • Add Cairo ShootOut! to cmake config for testing
2019-11-30 02:02:05 +0200 • Fix empty VA_ARGS problem in MaceLog macro
2019-11-29 23:55:30 +0000 • Make SWAP and MIN/MAX macros MSVC compatible
2019-11-29 23:54:08 +0000 • Made vararg ellipsis parameters MSVC compatible
2019-11-29 23:52:21 +0000 • Make Memory Manager return macros MSVC compatible
2019-11-29 23:51:27 +0000 • Make ROMScratch macros MSVC compatible
2019-11-29 23:50:08 +0000 • Move SwapPoint to MacQuickDraw.h to make it inline
2019-11-29 23:48:05 +0000 • Change void* to UInt8* in fake ROM (for MSVC)
2019-11-29 23:46:50 +0000 • Make SANE conversion macros MSVC compatible
2019-11-29 23:33:03 +0000 • Merge branch 'master' of
2019-11-29 23:32:43 +0000 • WinAPI EndianXX_Swap macros
2019-11-30 01:30:54 +0200 • Add UInt128 type to EmuMacTypes.h

SoftPC (and the quest for the XT memory map)

Before Macintosh was created, there was a computer architecture called IBM PC. With Intel 8086 processor, it was incompatible with the Motorola 68000 family used in Macintosh computers – and thus, no IBM PC software would work in Macintosh. But there were some really smart people, who created a special application called SoftPC in mid-80’s, which interpreted Intel’s x86 instructions on the fly, and emulated enough PC architecture and BIOS ROM to allow those programs to run on Macintosh. And now, SoftPC finally also works on M.A.C.E.

SoftPC 2.51 architecture

The version of SoftPC we got to run is 2.51, which happened to be compatible with the “Classic”-type 68000 Macintosh computers, one which M.A.C.E. aims to emulate in the Phase 1. That is, it works on 68000 processor, supports the 1-bit “classic” QuickDraw on 512×342 screen, and appears to be compatible with System 6 level Toolbox calls.

The IBM PC architecture which SoftPC 2.51 emulates is IBM XT with 640 kilobytes of RAM, and a CGA monochrome display. It also supports two serial ports (although M.A.C.E. serial port drivers are still unfinished to try them out), a printer port (no printing support for us yet), two hard drives (through disk images), and two physical floppy drives (which we don’t either emulate yet).

The XT memory map

The challenging part of getting SoftPC to work was ingenious way it emulates the PC memory map. As everybody knows, IBM PC has physically 20-bit addressing scheme (through 16-bit segment and offset register pairs, which give 4+16 bits of usable virtual address space, as there is overlap between segments). This 20 bits equals 2^20 = 1048576, or 1 megabyte of address space. What SoftPC does, is that it allocates this entire 1 megabyte area to be linearly accessible throughout the physical Macintosh address space, but it saves memory by allowing this area to have “holes” in it for the non-used areas of the PC memory map. This is basically the layout it creates:

000000-09FFFF: RAM
0A0000-0B7FFF: hole
0B8000-0DDFFF: CGA video memory
0DE000-0EFFFF: hole
0F0000-0F4FFF: ROM BIOS #1
0F5000-0FDFFF: hole
0FE000-100000: ROM BIOS #2

The steps it uses to ensure this linear layout is created is quite unique, and a good challenge to test the memory manager implementation:

For starters, the availability of large enough linear memory space is ensured by doing allocation of 1048576+32 bytes of memory using NewPtr. If this fails, it will know that there won’t be enough memory.
The allocated space is shrunk down to the 640K+32 RAM size using SetPtrSize. (The 32 bytes extra is used for aligning the actual addresses inside allocations to 16-byte boundaries for apparently memory access speed optimization on later Macs).
The three other blocks (CGA memory and two ROM areas) are allocated at correct locations using the following algorithm:
1. The size of “hole” preceding the allocation is calculated. For example, having video memory at B8000 would leave 18000 bytes space between A0000 and B8000.
2. Two pointers are allocated after each other using NewPtr: one for the hole (using calculated hole size), and the actual memory block (+32 bytes for alignment).
3. Now this is where the magic happens: If the latter allocated block’s start would not be within alignment range of the required offset (in CGA case B8000) from start of RAM (the 640K block), the size of hole would be decremented with 16 bytes, the two blocks would be disposed, and allocation re-attempted at step 3-2.
4. After repeating steps 3-2 through 3-3 enough so that required range is reached, the pointer to the allocated hole block is saved, and next one of the three blocks is allocated starting from step 3-1. (It should be noted, that because of a very minor bug in M.A.C.E. memory manager, the allocation size failed to decrease, and this loop would be terminated after 64 attempts, with incorrect block left allocated.)
After all three blocks have been allocated at correct offsets, the holes preceding each of them would be disposed using DisposePtr. This part of process leaves space inside the emulated PC address space, which the Toolbox can use for any regular memory allocations – pointers and handles, even resources. It is not 100% certain, but it may be that SoftPC may optimize performance in such way, that writing to PC address space at these holes might actually even cause native memory to get corrupted, BUT for now we did not try to see if that might happen.

The issue with Memory Manager in step 3-4 was, that as the DisposePtr call on the two blocks actually creates two adjacent “free” blocks, the allocation of new block using NewPtr right after that with only 16 bytes smaller size, would actually not return a physical block with smaller size. This was, because the block allocation routine checks if the space between allocated and next block would be less than minimum allowed block size (for creating new “free” block between allocation and next block), it would actually merge that small bit of space back to the physical allocation. The bug in Memory Manager was, that the when allocating block in free space, it was supposed to always merge the next “free” block after the allocation, and slice the new “free” space from that. A real Mac merges only one extra “free” block after allocation, but in M.A.C.E. for simplicity we fixed this by merging all free space in one run, thus at the same time reducing clutter of having many adjacent free blocks.

With this fix in Memory Manager, SoftPC is now finally able to boot up. Thanks to the Standard File Package implemented earlier this autumn, we could not only configure the hard disk image (C:/), but also set up a “shared” network disk (E:/) through a folder inside emulated Mac file system. This allows easy access of files (and thus DOS applications) from the M.A.C.E. file system, including some games. The shared disk access routines, however, surfaced luckily a couple File Manager bugs, related to the ioNamePtr handling in _GetCatInfo and _GetFileInfo calls, which possibly also improve compatibility of other applications too.

Screenshots

Below are some screenshots of the configuration dialogs of SoftPC (click for larger image):

And here is DIR command output right after booting up:

Also, there was GWBASIC included with the DOS installation, so we can run GWBASIC compatible BASIC programs:

And also some games work nicely. The Monochrome CGA emulation has interesting approach of emulating the 320×200 resolution with 1.5x scaling as 480×300 pixel bitmap display. The four CGA colors are also mapped into distinct vertical stripe patterns, which might also have been used on some old monochrome CGA displays:

SOPWITH.EXE running on SoftPC on M.A.C.E.

Of the two above games, SOPWITH appears to be quite responsive and playable, but ALLEYCAT controls are very laggy. Also Sierra adventure games seem to run:

The Black Cauldron (pictured above) seems to work just fine, although for some reason the function keys were not working, while they did work in GWBASIC earlier (opening of doors with F6, for example). Also, none of the games produced any sound although the SoftPC boot beep works, so it might be that SoftPC may lack actual PC speaker emulation in this particular version (beyond the beep).

Other issue that we also fixed while working on this was a bug in Scrap Manager, which caused the clipboard to misbehave when working in “on-disk” mode (after _UnloadScrap was called). This allows Copy-Paste operations to work nicely in SoftPC, and it also improves clipboard functionality in other applications.

Only remaining issue with SoftPC is that some routine, possibly in SANE and most likely related to converting PC date & time, is causing divide overflow error to popup at random times, which appears to be related to the actual date & time. This needs to be investigated still at later point.

Full list of changes since last post

2019-11-27 00:34:43 +0200 • Fix a Scrap Manager bugs, fixes SoftPC copy+paste 
2019-11-26 23:06:55 +0200 • Fix ioDirFlag to be 4 which appears to be correct 
2019-11-26 23:06:27 +0200 • Don't crash on invalid zone in Handle validation  
2019-11-26 04:01:31 +0200 • Fix register-based 68k->native UPP calls' D0 nuke 
2019-11-26 03:57:20 +0200 • Don't overwrite *ioNamePtr if not indexing files  
2019-11-22 03:41:15 +0200 • Add FREMS selector to SANE's FP68K dispatcher     
2019-11-22 03:40:22 +0200 • Force merge of free blocks at end of BlockFindSize
2019-11-19 22:59:13 +0200 • Fix bug where everything left of a list got erased
2019-11-19 22:55:27 +0200 • Redraw modified cell in LSetCell                  
2019-11-19 01:42:03 +0200 • Add support 23 new native 68k exception handlers

Bug fixing & stabilization

Since the previous update, focus has been mostly on improving the compatibility and stability of existing test applications through debugging, bug fixes and finishing some partially implemented features. This has so far had mostly effect on Excel, THINK Pascal and Fokker Triplane Flight Simulator:

Microsoft Excel 3.0 progress

Seems that Excel was complaining the out of memory error only because it could not locate the ‘PACK’ 6 resource (IntlUtils package). Adding a ‘PACK’ wrapper resource to System file, in similar fashion to the one which was added earlier for ResEdit as ‘PACK’ ID = 0, allowed this check to pass. After that a couple of new IntlUtils selectors were needed, with a couple file manager tweaks, and now Excel is able to start up successfully (albeit with a alert about not finding Excel, which can be passed).

It seems that a lot of tools and features are already functional, including basic worksheet creation, cell selection/modification, shape drawing, basic formatting etc. However, the most important feature, text input to allow entering data to worksheet cells, still requires TEDispatch to be implemented.

THINK Pascal 4.0 progress

One application which surfaced a lot of bugs and issues was THINK Pascal 4.0, here’s a rough summary of the issues found & ‘fixed’:

Reallocation of resources was not working properly in ChangedResource. Resource Manager did not save & restore active resource entry when measuring space available between current and next resource, which caused the new resource allocation in growth case to end up in wrong resource entry in the map (the next one was reallocated instead of current).
Removing name was attempted even if no name existed, causing resource names to get corrupted
RmveResource was not offsetting the types properly when removing the last instance of a given type.
The internal routine RGetResourceCount was trashing the resource entry pointer in resource manager’s stack state, causing some resources to be skipped altogether in iteration.
SetHandleSize was allowing locked blocks to move during the operation, which was explicitly told in Inside Mac documentation would not happen. This bug caused sometimes locked memory to move, with dereferenced pointers becoming invalid, causing crash during pascal code compilation.
A very rough approximation of MoveHHi was added. It works well enough to allow THINK Pascal to move resources out of the way to not prevent heap fragmentation from growing size of locked handles – but still causes sometimes fragmentation enough to surface this issue. However, this is for now enough to get the projects compile roughtly 2 to 3 times in one session before memory fragmentation kicks in and crashes the compiler.
The resource copy phase of THINK Pascal surfaced a bug in CountResources/GetIndResource, where CurMap was used instead of TopMap (CurMap should only be used when operating on 1-deep versions of the calls, Count1Resources/Get1IndResource). With this fix, the resource file copy phase succeeds.
Using “Transfer” command in THINK Pascal can be used to launch another app – in most cases, the application that was just compiled. However, the application zone teardown was – and still is – largely unfinished, having a number of issues. One that was fixed, was that resource manager attempts to release and purge all font caches with allocations in the application zone, which would be invalid upon launching the new program.
GetScrap trap had a minor issue, in which the case of hDest being nil it would crash, instead of just returning the scrap size as it was supposed to. This fixes the clipboard to (almost) work in THINK Pascal.

Now even a bit more complex programs can be compiled using THINK Pascal, for example one of Toni’s old games from as early as 1998. However, there are still minor glitches in various places:

Sometimes clipboard operation does not work on the first attempt.
Replacing existing application with “Build application” & replace dialog does not yet work because _Delete trap is still unfinished.
Sometimes text editing has very tiny minor selection glitches (when doing “Search again” in source code, where previous hit was selected and visible in area which was scrolled only little up, the old selection is left on when highlighting the new one).
Because resource map compaction is (still) unfinished, each time resources are updated and/or written in resource files, the file size grows indefinitely (by size of modified resource).
Compiling only works a couple times (2-3) on large projects, before heap fragmentation crashes the compiler.
The “Go” option does not yet work, because when running applications inside the THINK Pascal IDE, it does not actually launch a separate applications process for it. Instead, it loads the application “on top” of the pascal IDE, and in order to do this, it wraps a lot of its handlers to toolbox routines, and to do this, it manipulates the trap tables directly. This however assumes the toolbox trap table is located at 0xC00 on mac plus (0x75)-type machine, while it is currently located at 0xE00 (later machines). Because some currently used low memory globals overlap this area in non-compatible ways (for example, cursor device), they need to be adjusted a bit to behave differently on “Classic” type environment (like on real Mac Plus), and other way on later (slot-based / color-QD) environment.

Fokker Triplane Flight Simulator

It seems that the only issue causing Fokker Triplane to freeze upon starting new game was actually the lack of low-level interrupt handler emulation. The game attempted to override system VBL handler by adding its own VBL interrupt handler in the Lvl1DT low memory area containing the list of VIA’s Level 1 interrupt handler table, and as it was never called, caused the lock-up. The VBL, and 1-second timer, have now proper handlers in this low memory area, which is checked upon triggering the emulated VBL interrupt. This also includes the familiar performance optimization, where unpatched handlers skip the MixedMode interface with direct method call. It seems that with this change, the game is now fully playable with sound effects (although not thoroughly tested).

Full list of changes since last post

2019-11-16 04:08:45 +0200 • Lowlevel VIA interrupt handler for Fokker Triplane
2019-11-13 03:50:11 +0200 • Tweak SndNewChannel to not crash ResEdit          
2019-11-09 09:03:38 +0200 • Dummy OpenDeny/OpenRDeny (=>paramErr for local FS)
2019-11-09 07:51:53 +0200 • Workaround for mapping wideOpen Rgn in DrawPicture
2019-11-09 07:49:02 +0200 • Move File Manager commands to separate module     
2019-11-09 07:47:37 +0200 • Move SoundDriver test code to separate module     
2019-11-09 02:29:58 +0200 • Add dummy BitMapRgn trap for aWorm                
2019-11-06 05:40:10 +0200 • Respect port clipRgn in CopyBits                  
2019-11-06 04:31:42 +0200 • Fix edge cases in UnpackBits                      
2019-11-04 21:32:35 +0200 • If GetScrap argument hDest = nil, return only size
2019-11-04 11:31:38 +0200 • Clear font caches when closing res file w/ fonts  
2019-11-04 04:16:41 +0200 • Add IUMagString & improve Transliterate for Excel 
2019-11-04 03:09:38 +0200 • Fix Count…/GetIndResource to begin at TopMap    
2019-11-04 02:15:21 +0200 • Add memLockedErr enum missing from previous commit
2019-11-04 02:14:56 +0200 • (HACK) MoveHHi added - works, but not as in specs!
2019-11-04 02:12:08 +0200 • Fix SetHandleSize for locked blocks (don't move)  
2019-11-04 02:06:36 +0200 • Fix tail check in the zone check                  
2019-11-04 02:04:53 +0200 • Fix name allocation in AddResource                
2019-11-04 02:03:14 +0200 • StdFile: No "New folder" button in old dialogs    
2019-11-03 05:53:37 +0200 • More verbose heapcheck & optional free space debug
2019-11-02 18:13:37 +0200 • Fix possible TextEdit calibration memory move bug 
2019-11-02 17:40:30 +0200 • Fix resource map validation for maps with no names
2019-11-01 17:55:35 +0200 • Add dummy cstr2dec and pstr2dec selectors in Pack7
2019-11-01 17:54:15 +0200 • Don't trash ResourceEntry ptr in RGetResourceCount
2019-11-01 02:30:02 +0200 • Dummy IUMagWString & Transliterate(Text) for Excel
2019-11-01 00:43:41 +0200 • Add 'PACK' 3,4,5,6&7 mapping (Excel now starts up)
2019-10-31 21:33:21 +0200 • Resource fork compact placeholder code            
2019-10-31 01:01:48 +0200 • Fix RmveResource offset bug at last item of a type
2019-10-30 21:56:35 +0200 • Don't try to remove name if entry doesn't have one
2019-10-30 21:55:55 +0200 • Add map checks to Resource Manager calls for debug
2019-10-30 04:54:54 +0200 • Fix reallocation of resource in ChangedResource

A LOT of fixes, improvements and new test applications

It’s now approximately one month since last update, and a lot of “under the hood” type progress has been made.

M68K Tester and 68K bug fixes

One major task we had on roadmap was attempting to see how we could integrate Pukka’s emulator core with the amazing M68K Emulator Testsuite (by Gwenolé Beauchesne and Ray Arachelian, archived at this URL: http://web.archive.org/web/20090417202939/http://gwenole.beauchesne.info/en/projects/68ktester ). After holidays, Pukka was able to get the integration done, and we found a bunch of hard-to-find CPU bugs thanks to it:

Even though a lot of bugs were fixed, there are still test cases which do not pass, and will be fixed soon. However, with the fixes already done to the bugs that were found thanks to the test suite, Dark Castle and Beyond Dark Castle appear to be stable in rooms that used to crash before:

New relative mouse mode

A long-time task that has been waiting implementation has been the relative mouse mode, in which M.A.C.E. captures mouse cursor entirely, and instead of using the absolute position given by the host operation system, calculates mouse motion itself using the mouse delta vectors. This both prevents stray clicks outside emulated desktop window, and allows long delta swipes such as required to control helicopter in Apache Strike. As a result of this, Apache Strike can now be played, as previously it would immediately crash into the wall – and Pirates! no longer hangs in the swordfight at beginning – both which depended on certain low-level mouse low memory globals being emulated correctly:

Another fun feature which we added to complement the relative mouse mode as experimenting with game controller input API of SDL, which took about 30 minutes to do and works quite OK as seen on this video:

Apache Strike controlled using USB gamepad

Another issue that was fixed was missed clicks problem with “Tap to Click” MacBooks, which now should generate mouseDown events correctly.

Currently the cursor itself is still displayed using SDL “hardware” cursor API, but with the recent changes adding support for software cursor should not be a big problem – One that might be required to have decent cursor is certain games, where the cursor visibility depends on the cursor having ability to “invert” depending on underlaying graphics (such as the crosshair on black background in lemmings), which the SDL hardware cursor does not support.

Keyboard text input rewritten with proper character mapping

In first iteration of keyboard handler, we only had scancode support with minimal MacRoman mapping (without modifiers), which although was working for the simple use cases, did not allow flexibility required for example to enter upper-case letters.

For second iteration, we attempted to utilize host operating system keyboard layout mapping by using SDL text input API, and although it worked nicely for TextEdit text input, broke command keys and other things (such as Dark Castle & Lode Runner controls) horribly.

For the latest, third iteration, we went back to scancode based input, but instead are simulating the exactly same type KCHR scancode to MacRoman mapping like on real Macs. This allows every key to work properly in text editing, game input and menu shortcut etc, but one downside is that for international layouts we will need to have separate KCHR resources, currently only having U.S. as default layout. However, as “dead keys” are also working, typing special characters such as accents, umlauts, etc is working nicely even on this layout.

ResEdit bug “treasure chest”

ResEdit proved to be a real “treasure chest” of bugs and improvements, as it used literally every possible List Manager routine, and it also surfaced some unexpected bugs in other places such as TextEdit, Dialog Manager, QuickDraw, and other already heavily used toolbox traps.

After Dialog Manager and QuickDraw fixes, DLOG editor works perfectly

Some major improvements found through testing ResEdit:

Most of List Manager calls were implemented, including resizing lists, deleting rows/columns, searching data. Also some bugs were fixed, such as corruption of data offsets in some cases, etc.
Dialog manager edit field handling had a number of bugs, which surfaced in the DLOG editor, and there also were bugs in item hiding/showing which were also manifesting in Civilization.
TextEdit had some bugs in the MixedMode handling of special case calling conventions, which caused crash in the hex editor. After fixing them, not only the hex editor works, but also MacPascal editing bugs were fixed, as it also used custom TEDoText hooks.
The PACK 1 (“BitEdit”) package in ResEdit surface a couple QuickDraw bugs, such as random SeedFill memory corruption (using the paint bucket tool) and StretchBits memory corruption. These also make MacPaint more stable, which was also suffering from the SeedFill bug.

Some of these fixes had quite universal effects, such as improving the previously buggy behaviour of Lists and TextEdit in HyperCard 1.x:

Other improvements

Text scaling and measuring had some bugs, which were fixed so that MacPaint and Macwrite can display text of all sizes, faces and styles.
MacWrite now works, at least kind of…all other edition options are working (styling, page breaks, font sizes, fonts, rulers, etc), except adding newline using enter/return causes text to corrupt.
As the region corruption bug in polygon rendering was fixed, PT-109 appears now the be completely stable.
With “dummy” SetPalette trap implemented, Railroad Tycoon is now able to proceed past first “End of Fiscal Period” report
Scrollbar (CDEF 1) implementation (and related Control Manager TrackControl/DragControl stuff) were finally finished, so the indicator thumb dragging works now.
Prototype of font fractional width support added, only test case for this is curently Photoshop, which enables them. Right now only FOND width tables are supported, but extended width tables could be easily added as soon as some application needs them.

New test applications added

Also, we have now finally two new test app bundles that should be complete enough for trying out. These are ZeroGravity by Duane Blehm, and Blob Manager Demo by Paul DuBois. They are now available for trying out in the Downloads section.

Additionally, the old test application bundles have been updated with the new M.A.C.E. runtime, which includes all the bug fixes and improvements done since their last update. For example, Stunt Copter uses now the new relative mouse mode, and all apps have been updated to support the “Tap to Click” trackpad mode on MacBooks.

Full list of changes since last post

There are too many changes to detail in a single post, so as a new feature below is the version control log of all changes made since last update:

2019-08-19 22:54:49 +0300 • Add SetDialogDefaultItem/CancelItem/TracksCursor 
2019-08-19 22:47:32 +0300 • Tweak TrackControl and LClick delay to 8 ticks 
2019-08-19 18:15:47 +0300 • Save alert default item also in the DialogRecord 
2019-08-19 16:23:03 +0300 • Partial support for default dialog StdFilter proc 
2019-08-19 07:03:12 +0300 • Bumb the M.A.C.E. version number & use current date 
2019-08-19 07:02:52 +0300 • Replace printfs with log, and add a few DEBUG #ifs 
2019-08-19 07:00:17 +0300 • Fix the SDL window title for now by using EnvStartupFile 
2019-08-19 06:59:57 +0300 • Tweak Blob Mgr Demo and ZeroGravity for release 
2019-08-19 06:00:30 +0300 • Work on FOND widthtable and fractional size support 
2019-08-17 03:17:27 +0300 • Work a little on resource map "shadowing" handling 
2019-08-17 03:04:32 +0300 • Fix WaitMouseUp to not return 1 extra "down" state 
2019-08-17 02:23:40 +0300 • Fix SeedFill random memory corruption case 
2019-08-15 23:36:01 +0300 • Merge branch 'master' of  
2019-08-15 23:35:56 +0300 • Fix TextEdit cursor right-side clamping 
2019-08-15 22:55:05 +0300 • add xcode 6 project 
2019-08-15 22:43:06 +0300 • Fix incorrect register usage in TEDoText overrides 
2019-08-15 21:21:29 +0300 • Fix StretchBits stack buffer overrun 
2019-08-14 23:46:30 +0300 • Fix scrollbar redraw bug in List Manager 
2019-08-14 23:25:48 +0300 • SetPalette classic-QD dummy trap 
2019-08-14 21:44:36 +0300 • Fix LSetSelect highlight bugs 
2019-08-14 19:28:32 +0300 • Fix lNoNilHilite flag handling in LSetSelect 
2019-08-13 03:14:24 +0300 • Fix value clamping in Control Manager 
2019-08-13 03:11:51 +0300 • Tweak CloseWD stub to not crash 
2019-08-13 03:11:28 +0300 • Implement LClrCell and proto of LDelRow/LDelColumn 
2019-08-12 21:10:26 +0300 • Fix a couple byteswap bugs in lists (selection) 
2019-08-12 17:39:11 +0300 • Fix one-cell delta calculation in list pageup/down 
2019-08-12 15:03:44 +0300 • Fix maxIndex bug in LNew 
2019-08-12 02:34:18 +0300 • List Manager LSize implementation 
2019-08-11 05:12:39 +0300 • Finished LAutoScroll, another easy one for ResEdit 
2019-08-11 04:56:40 +0300 • Implemented C_LFind, was simpler than I thought… 
2019-08-11 04:46:28 +0300 • Finish LNextCell trap, work on LClick multiselect 
2019-08-11 02:18:20 +0300 • Implement C_LClick scroll indicator handling 
2019-08-11 02:06:55 +0300 • Fix dividend overflow in scrollbar CDEF 1 rounding 
2019-08-11 01:43:52 +0300 • Implement DragControl, finish TrackControl & CDEF1 
2019-08-10 03:41:37 +0300 • Fix caps lock usage in SDL key event handler 
2019-08-10 02:01:37 +0300 • SDL gamecontroller support "hack" to Apache Strike 
2019-08-09 20:45:44 +0300 • Add FSpOpenDF, FSpCreate, FSpDelete, FSpGetInfo, FSpRename & (dummy) _Rename 
2019-08-09 18:08:02 +0300 • Hacked procinfo for UIMagIDString call in LSearch 
2019-08-09 03:28:00 +0300 • Fix buffer overflow in GetVolParms 
2019-08-09 02:42:54 +0300 • Added dummy GetVolParms selector to _FSDispatch 
2019-08-09 02:00:31 +0300 • Add Jigsaw test app 
2019-08-09 02:00:10 +0300 • Implement LSearch, and (dummy stub) IUMagIDString 
2019-08-09 01:58:42 +0300 • Implemented 1-bit NewGWorld in QDExtensions 
2019-08-08 17:10:34 +0300 • Fix allocation of key state buffer 
2019-08-08 02:45:45 +0300 • C_KeyTranslate Deadkey support + fix mac scancodes 
2019-08-07 18:30:41 +0300 • Fix dialog edit field handling 
2019-08-07 12:13:26 +0300 • Fix picture recording of port fgColor/bkColor 
2019-08-07 04:49:10 +0300 • Add git log generator, for fun 🙂 
2019-08-07 04:21:32 +0300 • Tweak type of ScrapSize in low memory 
2019-08-07 04:21:04 +0300 • Fix fromMode handling in Picture Line recording 
2019-08-07 04:19:58 +0300 • Fix picture line recording Point saving 
2019-08-07 04:17:59 +0300 • Fix Hide/Show dialog item bounds calculation 
2019-08-07 04:15:36 +0300 • Prevent editfield citation replacements in dialogs 
2019-08-06 19:43:28 +0300 • Fix Scrap Manager bugs 
2019-08-06 19:41:31 +0300 • Clean up Mouse module a bit 
2019-08-06 19:07:03 +0300 • Fix bug in DefaultShowCursor 
2019-08-06 16:38:47 +0300 • Un-obscure cursor before _MDrawCursor in CrsrTask 
2019-08-06 01:42:37 +0300 • Make phase-shift adjustment configurable 
2019-08-06 00:20:31 +0300 • Move audio phase-shift from RAM to ClassicSound 
2019-08-06 00:17:50 +0300 • Disable VM_MAP and reverse-page mapping 
2019-08-05 20:44:39 +0300 • Disable a couple of mouse debug log calls 
2019-08-05 00:10:21 +0300 • Updated some test apps with relative mouse mode 
2019-08-05 00:09:22 +0300 • Refactor cursor to support relative motion mode 
2019-08-04 00:30:38 +0300 • fix abcd, partially fix sbcd instruction 
2019-08-03 23:20:09 +0300 • fix zero status for subx.b/w/l instruction 
2019-08-03 00:40:07 +0300 • fix add.b, addx.b/w/l and negx.b/w/l status bits 
2019-08-03 00:40:01 +0300 • Merge branch 'master' of  
2019-08-02 00:05:57 +0300 • Fix TEIdle blinker 
2019-08-01 23:10:07 +0300 • Implement LLastClick and tweak trap glue generator 
2019-08-01 16:03:22 +0300 • Disable the extra log debug trace of heap compact 
2019-08-01 16:02:28 +0300 • Reduce logging in MixedMode calls 
2019-08-01 16:01:44 +0300 • Fix size bug in WriteResource 
2019-08-01 16:00:47 +0300 • Fix resProtected flag check in ChangedResource 
2019-08-01 15:58:51 +0300 • Fix dialog edit field disabled check + others 
2019-08-01 05:19:49 +0300 • Add missing lowmem changes for CurActivate bug fix 
2019-08-01 05:19:04 +0300 • Work on KeyTranslate KCHR mapping (fix keyboard) 
2019-07-31 01:21:08 +0300 • Add MacConcentration test app 
2019-07-31 01:20:39 +0300 • Implement FADDD & fix major 64-bit float SANE bug 
2019-07-31 01:08:21 +0300 • fix lsl and lsr instruction for shift > 31 
2019-07-30 22:53:46 +0300 • fixes for m68k tester 
2019-07-30 21:50:19 +0300 • move x68k xcode6 project to x68k root 
2019-07-30 21:48:42 +0300 • add m68k tester 
2019-07-30 21:46:25 +0300 • make kemu lib c++ compatible 
2019-07-29 15:51:21 +0300 • Tweak GunShy config to use version 1.3 
2019-07-28 04:54:27 +0300 • Fix window activation handling 
2019-07-27 04:36:27 +0300 • Fix GetFontInfo for scaled fonts 
2019-07-27 04:22:21 +0300 • Fix bug in fast-case of text blitter 
2019-07-27 02:13:56 +0300 • Set Lo3Bytes lowmem global properly 
2019-07-27 02:12:01 +0300 • Implemented ReadDateTime and (dummy) SetDateTime 
2019-07-27 01:26:28 +0300 • Fix LActivate 
2019-07-27 00:46:46 +0300 • Fix cursor leak 
2019-07-26 18:29:56 +0300 • Tweak InitResources to be (again) re-callable 
2019-07-26 17:32:20 +0300 • Fix UPP ProcInfo of DABeeper callback 
2019-07-26 15:46:55 +0300 • Implemented LActivate selector in List Manager 
2019-07-26 05:45:33 +0300 • Fix bug in resource manager name remove code 
2019-07-26 05:44:44 +0300 • Add Glider4 test app 
2019-07-26 05:44:24 +0300 • Fix possible bug in SetVol NFS handler 
2019-07-26 05:43:07 +0300 • Setup CurrentA5 during boot stage 2 for INITs 
2019-07-26 02:55:49 +0300 • Re-fix PurgeSpace to also return correct D0 
2019-07-26 01:52:59 +0300 • Optimize FramePolygon case 
2019-07-26 01:43:57 +0300 • Actually update the existing ADF header 
2019-07-26 01:36:18 +0300 • Finish SetFileinfo handler in NativeFS 
2019-07-26 01:35:28 +0300 • Fix region corruption bug in polygon drawing 
2019-07-25 23:16:56 +0300 • Fix MapRgn duplicate inversion point bug 
2019-07-25 23:07:21 +0300 • Fix MapPt negativity check 
2019-07-25 22:08:06 +0300 • Fix size of ScriptUtil selector 
2019-07-25 22:02:10 +0300 • Fix edge case bug in UnpackBits 
2019-07-25 21:02:00 +0300 • Fix ADF file create bug, mixed up rsrc and data 
2019-07-25 19:27:32 +0300 • Also reserve space for mapping in the stack after packBuffer

Update on resource fork writing, ResEdit, and resource ‘dcmp’ decompressors

Resource write support status

So, the plan since last update was to do some work on resource write support, and there was indeed some progress. The nature of how resource forks are handled on disk compared to memory is quite interesting; After the resource map is read into memory, the resource fork on disk only needs to preserve resource data, and thus the resource manager has quite free hands on how it can move data around in the resource fork:

When a new resource is added using AddResource, it gets both a resource entry in the in-memory resource map, AND the resource fork gets expanded (to reserve space on disk), and on-disk location for the resource is appended after the current data area on the disk
When ChangedResource is called on existing resource, its size is checked to see if it grows or shrinks. Shrinking is more simple, as its data length just gets shorter, and does not need space allocation. However, if a resource grows, it needs more space allocated to it, so while the resource file is open, the resource manager simply allocates new space at end of resource fork, like it does in AddResource, and leaves the old location as “gap” in the file. This feature actually is why apple warns in the Inside Macintosh documentation, that calling ChangedResource often without calling UpdateResFile may lead to large amount of disk space used, as each new allocation expands the file size by the new allocation length!
When WriteResource is called, a resource with resChanged flag will get contents of its handle written to the location specified by resource entry in the resource map
UpdateResFile does the real “house-keeping” of resource fork, as it will iterate through all resources, removing any gaps in the resource fork, moving data to the proper places (a process called “compating” of resource fork). After this, the resource map is written at end of the resource fork (after the data section), and file size is truncated to minimum required, releasing the extra disk space.

Currently, most of write support is in place, except that resource fork compacting still needs finalization to make it work properly

Extended resources and ‘dcmp’ decompression

Of course a natural way to stress-test resource manager is to try ResEdit, which I think uses literally almost all of resource manager features. However, an interesting surprise was that ResEdit seems to utilize resource compression on a large number of its own resource, which led to the following MacTech article:

http://preserve.mactech.com/articles/mactech/Vol.09/09.01/ResCompression/index.html

Which had nice, although brief, insight on ‘dcmp’ resources and their role in resource decompression. Adding the parsing of extended resource headers was not that difficult, and luckily ResEdit contains its own ‘dcmp’ decompressors, so they could be immediately be tested, resulting in following type of logs:

This snippet of debug log shows ResEdit’s ‘PACK’ resource ID=1 decompression procedure, when Get1IndResource was called. This shows how on-disk compressed data of 19568 bytes gets decompressed into 27200 bytes in memory, using a ‘dcmp’ ID=0 decompressor which is contained within ResEdit as 68k code.

ResEdit

And of course, with the decompression working, and with a lot of fixes to file manager, list manager and various other places, ResEdit magically appears to be starting to work, one bit at a time 🙂 In this test case, we made a copy of ResEdit itself as “Test File” file, which was hard-coded for now as reply to CustomGetFile call to get it open. And this is what we got so far:

ResEdit splash screen, with PICT animation
List of all resource types in Test File
List of ‘ALRT’ resources
‘ALRT’ resource editor
‘ICN#’ editor
List of ‘MENU’ resources
‘MENU’ resource editor
List of ‘PICT’ resources
Info window for a ‘PAT#’ resource
‘STR#’ editor (using template)

Naturally there’s still a lot to fix, but surprisingly many features seem to work out-of-the box. But at least the following problems were immediately noticeable:

The actual resource writing does not yet work properly
List Manager still has some calls which are not properly implemented which leads to a number of assertions, and most likely causes the missing icons in ‘ICN#’ editor.
Font scaling has issues which is why dialog texts appear weird in the preview of ‘ALRT’ editor
A couple IconDispatch/ScriptUtil calls which need to implemented for resource file info dialog (mostly to get Finder labels, and format the file dates)
Proper multi-select support for List Manager
A bunch of other minor issues, such as text missing from a couple places, etc.
Probably other issues in features not yet tested

It’s still a bit too early to start testing the actual resource write support, but it won’t be far away from now after ResEdit is made more stable first.

Package Manager update

One small, but important thing which was fixed in this update is an actual, proper ‘PACK’ resource support. This is because ResEdit uses Apple’s ‘PACK’ ID=1 to implement its “BitEdit” graphic editor (used at least in ‘ICN#’ resource editor). Until this point, all packages could be implemented as regular dispatcher calls, but for the ResEdit to work we actually had to load the ‘PACK’ resource in InitPack, and register it as one of the AppPacks in low memory globals. A quite small change, but one which ResEdit seems to be happy with, for now 🙂

Ps. As another interesting side effect, with bug fixes done to toolbox HyperCard can now both close files properly (allowing us to navigate back to Home stack), and also HyperTalk editing seems to work well enough that changes to the script source actually affect behaviour of the stack 🙂

Second mid-summer update: Post-roadtrip, TextEdit progress, Clipboard…

Okay, so both members of development team have been busy being on vacation trips for the past few weeks, but we’re again back (although still on “kind of” summer break). To celebrate this we decided to each post a picture of what we’ve been up to, but to make this appropriate to topic of this blog, we do this by showing the photos using M.A.C.E running Photoshop 🙂

Road trip vacations

First, Pukka went on a road trip to Eastern Finland with his good old trusty Volvo station wagon for one week:

Toni went on a two-week road trip to Northern Germany, visiting 12 cities in 14 days, including taking this amazing train from Hamburg to Kiel:

TextEdit progress (and Scrap Manager)

During this two weeks of visiting a lot of places, Toni had a couple evenings time to do some work on (non-styled) Text Edit, which now is nearly complete, and is able to run TeachText with almost all text editing features working:

These are the notable features which work now in TextEdit:

Multi-line text display, with scrolling support (viewRect/destRect offsetting)
TEKey text input (still a bit hacky but functional)
TEClick with autoscrolling support, double-click word selection (using TEFindWord), and fExtend support for “shift-key”-type extending of selection, and recognition of left/right side of character for proper caret positioning
TextEdit hook support for nearly all TE Hooks (TEDoText, TERecalc, TEWordBreak, ClikLoop, TEFindWord, TEFindLine, TETrimMeasure, EOLHook, HighHook, etc…) which can be customized by 68K apps
Caret support with TEIdle blinking, TEKey cursor key movement (including multi-line support), and multi-line selection range support
Nearly all “quirks” of regular System 7.x TextEdit implemented as they work on real Mac; including the weird way caret gets positioned when moving across line boundaries using left/right arrow keys if previously clicked on right-hand side of a character, etc…
Dialog Manager edit field support, with System 7-type multi-line text support (i.e. edit fields with one line get extended horizontally 2x size, and scroll both horizontally and vertically, while multi-line edit fields only scroll vertically), “tabbing” between edit fields, and complete TEKey/TEClick support
Maybe some others I’ve forgotten to mention

Additionally, as TeachText wanted to use also Scrap Manager for edit functions, in parallel to implementing TECut/TECopy which use private TextEdit scrap, the implementation of generic Scrap Manager was finalized so that Clipboard file is fully supported (previously only in-memory state of scrap was implemented).

However, TEPaste still needs to be added for a 100% completion of “Edit” menu features for TeachText – and also, styled TextEdit work has not yet been started, which will be required in future to support more complex apps such as SimpleText and HyperCard 2.x.

Next up, on Toolbox API side, work is being done on Resource Manager’s resource fork write support, mostly just because after Scrap Manager was implemented, SuperPaint was able to progress a bit further in it’s startup, but wants to create “SuperPaint Prefs” file using Resource Manager’s functions, so might as well add that next. As of writing this, there’s already support for AddResource, ChangedResource and WriteResource, but we need to add resource fork compacting support to UpdateResFile to actually write the file properly. But we’ll get back to that later.

Bonus pic – for fun 🙂

Meanwhile, here’s a funny pic of what it looks when we run 28 M.A.C.E. instances at once 🙂 All quite responsive even though they’re all in full debug mode, although same can’t be said about the old Mac Pro when it was running them… 😀

28 individual M.A.C.E. applications (in debug mode) running at once 🙂

Mid-summer update: TextEdit progress

We’re still on summer “break”, but there has been a few evenings time to work on some TODO items, lately especially on the TextEdit implementation. It’s still far from complete (the basic, single-style TextEdit – Styled TextEdit is another undertaking on its own!), but there has been some progress that has already some visible effects.

For example, one of the games we’ve been testing emulation with, Scarab of RA, relies EXTREMELY lot on TextEdit, including quest log, inventory (which includes selecting items with text selection highlight included), help/hint, about, and score dialogs. Some of these work in very interesting ways, for example quest log and inventory assemble text by concatenating text from individual segments – for example, “16 ounces of food” line in inventory is assembled one by one from strings “16”, ” “, “ounce”, “s”, ” of food”, and carriage return. The game also appears to use the internal line start offsets to intercept clicks in inventory, highlighting individual lines using text selection by itself.

The game is already approaching playable state, but there are still a few issues – most notably, recalibration of line start offsets does not update last line offsets correctly when removing, causing removed lines to sometimes leave “trashed” text at end of text edit boxes. Here’s some screenshots of how it looks like now:

Scarab of RA gameplay screen, with inventory selection highlighted

Multi-line text highlighting of recent message in quest log

Additionally, a few other games (Lemmings, Simcity) can now proceed past copy protection screens, as we can finally use the VERY hacky TEKey implementation to enter correct answers to the protection questions (SimCity also required a bunch of new selectors to be added to the FP68K SANE dispatcher):

Lemmings can now proceed to gameplay, which appears to work perfectly, including sound & music output using Sound Driver

SimCity (black & white version) can also proceed to game, although sound does not yet work as it still has the weirdly mixed up sound completion routine method signature which crashes the device manager if sound is not disabled

EDIT: Here’s a video of first 15 minutes of Mac Lemmings gameplay:

Lemmings running on M.A.C.E.

There is also some effect on productivity apps:

Text object can now be created in MacDraw

As the “message” window in HyperCard 1.x is now able to accept commands, we can use “set userlevel to 5” command to actually modify the included stacks! Still a lot of things broken there, but yet a lot new functionality unlocked… We haven’t though had time to play with them, but it should soon be much more usable!

EDIT: After minor tweaking with SDL TextInput API, MacPascal can also now accept text input, compile and run Pascal code:

Some test code entered, compiled and run in MacPascal

In general, TextEdit still has a lot of important features unfinished or to be implemented:

No caret display yet (teCaret in TEDoText)
TEClick not yet implemented, thus using mouse for selecting text does not work yet
TEKey is VERY hacky, needs for example arrow key control, backspace support, etc
Dialog Manager keyboard input does not yet handle default items/clipboard commands etc
The keyboard driver is still passing key events through scancodes without real mapping to actual MacRoman characters, preventing modifiers/host operating system keyboard layout from having effect
Line start recalibration has still sometimes issue when removing lines from middle of text
Selections have still some visual glitches, especially when typing text in dialogs
Styled TextEdit support still completely missing…for now
Probably other things too which I can’t remember…

And now back to the summer break…

Some “Music” to entertain in the quiet period

The past few weeks have been a bit quiet for us, as we’ve been both busy doing important work and personal life stuff, so we haven’t had as much time recently to spend on this project. We will continue progress soon when each of us gets more spare time, but meanwhile here’s a bit of entertainment for meanwhile:

MS-BASIC 2.0 running “Music” example program in M.A.C.E.

As mentioned a few weeks ago, the audio output – which previously was not working in MS-BASIC – suddenly started working as Pukka fixed some 68k CPU bugs. At that time, I recorded a short video of the “Music” sample program running with audio included – and as there’s no other news to share at this time, we hope this “filler” video will keep you entertained for the time being!