I’m publicly releasing version 1.2.0 of my REFDoor REF scripting engine door today!
This release contains all kinds of bug fixes, some new commands, and a slew of behind the scenes optimizations and improvements including changes which should greatly improve REFDoor’s performance when running on retro hardware. The most notable under the hood change is the addition of what I’m calling “memory mode” which behaves very similar to the original RTSoft RTReader by reading the entire script into memory for faster parsing and less disk activity. This may seem incredibly draft to the few of you who read my REFDoor 1.0 article and/or its included documentation and recall that the whole reason I developed REFDoor was specifically to see if I could mimic the entirety of the original parser’s behavior without reading the file into memory, but as I have vague aspirations to use REFDoor as the engine behind a future door game project of some sort, I want to implement as many options and improvements as possible.
Here’s the entire WHATSNEW.TXT file (which includes the unreleased 1.0.1 and 1.1.0 versions) for reference:
REFDoor v1.2 What's New? --- ! = Bug Fix + = Additions / Changes - = Removed * = Additional Notes v1.2.0 - Released 02/12/2021 --- + Added "DO CENTERPAD" command which works similarly to DO PAD and DO FRONTPAD but centers the string within the specified length (defaults to 80 if not specified) padding out the string with spaces on either side. Similar to "CENTER" but potentially more useful for things like title strings with different backgrounds, etc. + Added the "IS CHAR" function to the "DO" command. This function will allow you to return the character at the specified position from the provided string. Until now, you haven't been able to access characters from strings in REF. This should help with enable numerous string operations that would have previously only been possible with built-in REF commands. + The "SHOW SCROLL" command now optionally accepts a parameter pointing to an external file to display instead of parsing the "SHOW SCROLL" text inline. + Added a clean up procedure to REFDoor's startup to clean out any orphaned temporary file for the node being launched. In most cases this won't have fixed anything since most routines check for these files and remove them before generating a new one. + Added "memory mode" which loads the entire script into a memory buffer. On some systems this will be dramatically faster than the default line-by-line text parsing and with no IO hit, though on some it may not actually improve performance. Memory mode requires at least 64K of convential memory free and maxes out at around 640K per file. When using commands like "RUN" or "ROUTINE" to call other scripts, memory will be swapped. Memory mode is available by using the "/M" command line parameter. + Added label mapping when using memory mode - when a script is first launched (or reloaded after a swap) the label positions in the script are mapped into memory to make "DO GOTO" calls much faster on slow machines. The traceoff is that slow machines will also see a 1-2 second delay while this is happening. Semi-modern machines won't even notice. + As such, "NOCHECK" command as been made functional. This disables the memory mapping functionality from when it is encountered on. I'd highly recommend NOT using this for anything but very simple scripts (at least on slow machines - it shouldn't make much difference on semi-modern machines!) + Added an alternate version of "SHOW SCROLL" which parses a text file to memory when using memory mode. This version looks identically but allows line scrolling using the up and down arrows, has instant file export, and the find command results in the exact line of the next hit rather than just loading the page. The memory mode "SHOW SCROLL" should result in dramatically improved performance when loading massive amounts of text on slower machines. * If you're running into memory mode / memory usage issues, REFDoor logs some potentially helpful details about memory availability and usage. Use the new /V switch to see them. ! Fixed a display issue with the status bar in "SHOW SCROLL" when viewing a larger number of pages. Changed the entire layout of the status bar to compensate - while its no longer identical to RTReader, it still looks very similar. - The 16 bit DOS compatible executable no longer uses an overlay file. It was used before to leave the maximum amount conventional memory free but due to the way REFDoor is structured, resulted in extremely bad performance on older (386, 486, etc.) machines. I may revisit this in the future as it would allow for a larger buffer to be available when operating in memory mode, however, RTReader worked the same way, so this shouldn't be a massive negative. I would love to know if this change negatively impacts you! ! Fixed a bug related to page up/page down key detection, which particularly impacted the "SHOW SCROLL" commands. ! Fixed a very isolated bug where REFDoor's ANSI parser would process display codes if they were at the beginning of a line right after a CRLF on the previous line, *before* showing the MORE prompt for that line during a @DISPLAY or @DISPLAYFILE command, thus sometimes messing up the MORE prompt's display. This only fixes invalid codes - valid codes can still potentially be parsed this way, causing some odd behavior in rare instances - it's unlikely you'll be displaying files with control codes besides colors this way in the first place. Unfortunately a complete fix would require rewriting the ANSI parser (which I should probably do one of these days.) + Various logging changes including logging exit codes / error levels when a halt is issue, as well as a revised exit code scheme. + Added "SCROLLTOP" which lets you define where the SHOW SCROLL window starts. + Added "SCROLLSIZE" which lets you define how many lines long the SHOW SCROLL window is. + Added "SCROLLFOOTER" which lets you enable or disable the footer bar below your SHOW SCROLL window. + Added "SCROLLHEADER" which lets you enable or disable a header bar above our SHOW SHOW SCROLL window. + Added "SCROLLTITLE" which lets you define a title displayed in the SHOW SCROLL header, if used. + Added "SCROLLCOLOR" which lets you customize the color scheme of your SHOW SCROLL header and footer bars and pop-up windows. + Added "DO READYESNO" which will display a Yes/No lightbar prompt and set the provided string variable to the result. + Added "YESNOMAP" which allows you to define the strings of the yes and no selection prompts on the following two lines, respectively. + Included a silly little WHATSNEW.REF example script. The script's only function is to view the WHATSNEW.TXT file included in this archive, but it demonstrates several new-ish features, such as using SHOW SCROLL to view an external file, customizing SHOW SCROLL, using LIGHTBAR, using CENTER and CENTERPAD, and using DO READYESNO and YESNOMAP. + As mentioned earlier, there is now a /V command line parameter to enable more verbose logging. Generally, REFDoor will only log errors, warnings, and SysOp related events. If using verbose logging, you'll see additional information messages about loading and exiting, memory usage in memory mode, etc. This should improve performance, especially with memory mode on slow machines. + Modified main parser routine to more efficient handle parsing commands, though performance increases probably aren't noticible on anything but VERY old/slow machines. v1.1.0 - Released 01/19/2021 --- ! Added a range check to parsing string variables to prevent an invalid string variable reference from sometimes looping infinitly. - Removed some unnecessary variable / control code parsing passes from a several commands which might slightly improve performance in some cases. These commands were among the first I coded, before variables were parsed when read, meaning the separate parsing pass was necessary. * I added the following by request to assist with making BBS software specific doors. These are really only for specific use cases - DO STRIPALL/STRIPCODE should still be the primary command for stripping user input. + Added "DO STRIPPCBCOLOR" which strips PCBoard style @ color codes. + Added "DO STRIPWCCOLOR" which strips PCBoard style @ color codes. + Added "DO STRIPPIPECOLOR" which strips pipe style color codes. + Added "DO STRIPSETHCOLOR" which strips SethCode style color codes. + Added "DO STRIPWW4COLOR" which strips WWIV style heart color codes. + Added "DO STRIPALLCOLOR" which strips all known styles of color codes. * Likewise, added a few commands to convert BBS specific color codes to REFDoor's "SethCode" style color codes. Note that these only support the standard 16 color palette and do not support blinking or ICE colors. + Added "DO PCBCOLOR2SETH" to convert PCBoard style @ color codes. + Added "DO WCCOLOR2SETH" to convert Wildcat! style @ color codes. + Added "DO WW4COLOR2SETH" to convert WWIV style heart color codes. + Changed the REF parser version to "11". Starting from this release onward the versioning scheme will be based on the major+minor version of REFDoor itself. Since this number will always be more than RTReader and LORD 2's version of "2" this shouldn't break backwards compatibility, but it will let us use the VERSION command for its intended purpose going forward, with the assumption that any minor version will include changes/additions to the REF language. + Added "PAUSEMAP" command. Works exactly the same as MOREMAP but only affects the PAUSE command's pause string. This will let you have separate pause and more prompts, as long ase PAUSEMAP is defined after MOREMAP. + Changes to various SysOp side strings to make the aesthetic a bit more fitting for REFDoor rather than previous Darkness styling. + Futher changes to the local login to make it look a bit more like the local login in post RTReader doors like Planets:TEOS 2.00 and LORD 2. ! Fixed a various typos in the documentation. v1.0.1 - Unreleased --- ! Fixed an issue where I was using a Integer instead of a LongInt for filesize in the REF picker menu. VP defaults to LongInt sizes for integers, but BP/TP uses smaller integer sizes which are too limited for this purpose. + Added new multinode sensitive open text file procedures similar to xDoor's but with remote output and logging of errors. Updated all text file read only and append opens (temporary files, mostly) to use the procedures. Many of our procs that were opening text files weren't opening them safely. ! Fixed a formatting issue where the time left was being set to a different color in Xdoor's updateTime proc than in the statusBar drawing proc. This usually manifested itself after the statusbar was redrawn by commands like DES and PROGNAME. v1.0.0 - Released 06/19/2020 --- Initial release.
For the record, I’m planning on continuing work on REFDoor for as long as I continue to make any use of it. I was originally pretty wishy-washy on whether there would be any future work done on the engine, but I’ve finally decided that any work I do to it, even as part of another door, will be rolled into REFDoor proper and released. Stay tuned!
thanks plash.
You’re welcome, pulie! 😛
I wish I had come across this work earlier… I too have implemented a REF parser, and your REFDoor documentation is vastly better than the mess that comes with LORD 2. 🙂
Once I implement the igm and l2cfg stuff in JavaScript, I plan to do a JSDoor release of my LORD 2 work… if it turns out to be popular (ie: people actually work on new REF files), I’ll likely try adding at least some of your REFDoor extensions as well (I likely won’t ever add the extra colour code stuff though since it’s LORD 2).
If you’re curious about my terrible hacks, you can have a look here:
https://gitlab.synchro.net/main/sbbs/-/blob/master/xtrn/lord2/lord2.js
Awesome! Glad to know someone else out there is hacking around with REF parsing. I’ll definitely keep an eye on your project – LORD2 deserves a breath of life, IMO.
For the record, the only real reason I added pipe color codes was because they were already present in the door kit I use, so I wouldn’t really consider them an attempt to extend REF.
Hey Jack Phlash! I’ve been doing some DOS/Win32 BBS door development, mostly taking a ‘Rumors’ door I wrote for Renegade and swapping out various Pascal-based door kits to get a feel for the functions (DDPLUS, MANNDOOR, D32, XDOOR).
I’m really digging XDOOR v3, but then I just read this article, and it looks like REFDOOR is alive and kicking! Going to give it a shot. Any words of advice in going from Xdoor to Refdoor?
Cheers!
Apologies, I somehow missed this comment!
Well, I’d said that first of all, REFDoor isn’t a door kit in the same way as DDPlus, D32, and XDoor. In fact, it’s made *using* XDoor! Instead, it’s a scripting engine made for making relatively simple doors.
So, with that in mind, my second point would be that REF is a pretty funky language with lots of limitations and oddities. It can be fun to script in it, working around those limitations, of course, but if you already know Pascal or C, for instance, it might be like more work than simply coding your own door in a high level compiled language. That said, it would be simple to code a rumors door using it. In fact, there are several examples of something similar already.
As far as it being “alive and kicking”, 1.3 will be released very soon, as well as another new example script and updates to all of my old ones. It should be out sometime before the end of August. If you *do* decide to play with REFDoor, I’m always open to suggests for expanding its capabilities, and of course, bug reports.