REFDoor 1.2 Release

By | February 12, 2021

I’m publicly releasing version 1.2.0 of my REFDoor REF scripting engine door today!

This release contains all kinds of bug fixes, some new commands, and a slew of behind the scenes optimizations and improvements including changes which should greatly improve REFDoor’s performance when running on retro hardware. The most notable under the hood change is the addition of what I’m calling “memory mode” which behaves very similar to the original RTSoft RTReader by reading the entire script into memory for faster parsing and less disk activity. This may seem incredibly draft to the few of you who read my REFDoor 1.0 article and/or its included documentation and recall that the whole reason I developed REFDoor was specifically to see if I could mimic the entirety of the original parser’s behavior without reading the file into memory, but as I have vague aspirations to use REFDoor as the engine behind a future door game project of some sort, I want to implement as many options and improvements as possible.

Here’s the entire WHATSNEW.TXT file (which includes the unreleased 1.0.1 and 1.1.0 versions) for reference:

REFDoor v1.2 What's New?
---

 ! = Bug Fix
 + = Additions / Changes
 - = Removed
 * = Additional Notes


v1.2.0 - Released 02/12/2021
---

 + Added "DO CENTERPAD" command which works similarly to DO PAD and DO FRONTPAD
   but centers the string within the specified length (defaults to 80 if not
   specified) padding out the string with spaces on either side. Similar to
   "CENTER" but potentially more useful for things like title strings with
   different backgrounds, etc.
 + Added the "IS CHAR" function to the "DO" command. This function will allow
   you to return the character at the specified position from the provided
   string. Until now, you haven't been able to access characters from strings
   in REF. This should help with enable numerous string operations that would
   have previously only been possible with built-in REF commands.
 + The "SHOW SCROLL" command now optionally accepts a parameter pointing to
   an external file to display instead of parsing the "SHOW SCROLL" text
   inline.
 + Added a clean up procedure to REFDoor's startup to clean out any orphaned
   temporary file for the node being launched. In most cases this won't have
   fixed anything since most routines check for these files and remove them
   before generating a new one.
 + Added "memory mode" which loads the entire script into a memory buffer. On
   some systems this will be dramatically faster than the default line-by-line
   text parsing and with no IO hit, though on some it may not actually improve
   performance. Memory mode requires at least 64K of convential memory free
   and maxes out at around 640K per file. When using commands like "RUN" or
   "ROUTINE" to call other scripts, memory will be swapped. Memory mode is
   available by using the "/M" command line parameter.
 + Added label mapping when using memory mode - when a script is first launched
   (or reloaded after a swap) the label positions in the script are mapped into
   memory to make "DO GOTO" calls much faster on slow machines. The traceoff is
   that slow machines will also see a 1-2 second delay while this is happening.
   Semi-modern machines won't even notice.
 + As such, "NOCHECK" command as been made functional. This disables the memory
   mapping functionality from when it is encountered on. I'd highly recommend
   NOT using this for anything but very simple scripts (at least on slow
   machines - it shouldn't make much difference on semi-modern machines!)
 + Added an alternate version of "SHOW SCROLL" which parses a text file to
   memory when using memory mode. This version looks identically but allows
   line scrolling using the up and down arrows, has instant file export, and
   the find command results in the exact line of the next hit rather than just
   loading the page. The memory mode "SHOW SCROLL" should result in
   dramatically improved performance when loading massive amounts of text on
   slower machines.
 * If you're running into memory mode / memory usage issues, REFDoor logs some
   potentially helpful details about memory availability and usage. Use the
   new /V switch to see them.
 ! Fixed a display issue with the status bar in "SHOW SCROLL" when viewing a
   larger number of pages. Changed the entire layout of the status bar to
   compensate - while its no longer identical to RTReader, it still looks
   very similar.
 - The 16 bit DOS compatible executable no longer uses an overlay file. It was
   used before to leave the maximum amount conventional memory free but due to
   the way REFDoor is structured, resulted in extremely bad performance on
   older (386, 486, etc.) machines. I may revisit this in the future as it
   would allow for a larger buffer to be available when operating in memory
   mode, however, RTReader worked the same way, so this shouldn't be a massive
   negative. I would love to know if this change negatively impacts you!
 ! Fixed a bug related to page up/page down key detection, which particularly
   impacted the "SHOW SCROLL" commands.
 ! Fixed a very isolated bug where REFDoor's ANSI parser would process display
   codes if they were at the beginning of a line right after a CRLF on the
   previous line, *before* showing the MORE prompt for that line during a
   @DISPLAY or @DISPLAYFILE command, thus sometimes messing up the MORE
   prompt's display. This only fixes invalid codes - valid codes can still
   potentially be parsed this way, causing some odd behavior in rare instances
   - it's unlikely you'll be displaying files with control codes besides colors
   this way in the first place. Unfortunately a complete fix would require
   rewriting the ANSI parser (which I should probably do one of these days.)
 + Various logging changes including logging exit codes / error levels when
   a halt is issue, as well as a revised exit code scheme.
 + Added "SCROLLTOP" which lets you define where the SHOW SCROLL window starts.
 + Added "SCROLLSIZE" which lets you define how many lines long the SHOW SCROLL
   window is.
 + Added "SCROLLFOOTER" which lets you enable or disable the footer bar below
   your SHOW SCROLL window.
 + Added "SCROLLHEADER" which lets you enable or disable a header bar above
   our SHOW SHOW SCROLL window.
 + Added "SCROLLTITLE" which lets you define a title displayed in the SHOW
   SCROLL header, if used.
 + Added "SCROLLCOLOR" which lets you customize the color scheme of your
   SHOW SCROLL header and footer bars and pop-up windows.
 + Added "DO READYESNO" which will display a Yes/No lightbar prompt and set the
   provided string variable to the result.
 + Added "YESNOMAP" which allows you to define the strings of the yes and no
   selection prompts on the following two lines, respectively.
 + Included a silly little WHATSNEW.REF example script. The script's only
   function is to view the WHATSNEW.TXT file included in this archive, but it
   demonstrates several new-ish features, such as using SHOW SCROLL to view
   an external file, customizing SHOW SCROLL, using LIGHTBAR, using CENTER and
   CENTERPAD, and using DO READYESNO and YESNOMAP.
 + As mentioned earlier, there is now a /V command line parameter to enable
   more verbose logging. Generally, REFDoor will only log errors, warnings, and
   SysOp related events. If using verbose logging, you'll see additional
   information messages about loading and exiting, memory usage in memory
   mode, etc. This should improve performance, especially with memory mode on
   slow machines.
 + Modified main parser routine to more efficient handle parsing commands,
   though performance increases probably aren't noticible on anything but
   VERY old/slow machines.

v1.1.0 - Released 01/19/2021
---

 ! Added a range check to parsing string variables to prevent an invalid string
   variable reference from sometimes looping infinitly.
 - Removed some unnecessary variable / control code parsing passes from a
   several commands which might slightly improve performance in some cases.
   These commands were among the first I coded, before variables were parsed
   when read, meaning the separate parsing pass was necessary.
 * I added the following by request to assist with making BBS software specific
   doors. These are really only for specific use cases - DO STRIPALL/STRIPCODE
   should still be the primary command for stripping user input.
 + Added "DO STRIPPCBCOLOR" which strips PCBoard style @ color codes.
 + Added "DO STRIPWCCOLOR" which strips PCBoard style @ color codes.
 + Added "DO STRIPPIPECOLOR" which strips pipe style color codes.
 + Added "DO STRIPSETHCOLOR" which strips SethCode style color codes.
 + Added "DO STRIPWW4COLOR" which strips WWIV style heart color codes.
 + Added "DO STRIPALLCOLOR" which strips all known styles of color codes.
 * Likewise, added a few commands to convert BBS specific color codes to
   REFDoor's "SethCode" style color codes. Note that these only support the
   standard 16 color palette and do not support blinking or ICE colors.
 + Added "DO PCBCOLOR2SETH" to convert PCBoard style @ color codes.
 + Added "DO WCCOLOR2SETH" to convert Wildcat! style @ color codes.
 + Added "DO WW4COLOR2SETH" to convert WWIV style heart color codes.
 + Changed the REF parser version to "11". Starting from this release onward
   the versioning scheme will be based on the major+minor version of REFDoor
   itself. Since this number will always be more than RTReader and LORD 2's
   version of "2" this shouldn't break backwards compatibility, but it will let
   us use the VERSION command for its intended purpose going forward, with the
   assumption that any minor version will include changes/additions to the REF
   language.
 + Added "PAUSEMAP" command. Works exactly the same as MOREMAP but only affects
   the PAUSE command's pause string. This will let you have separate pause and
   more prompts, as long ase PAUSEMAP is defined after MOREMAP.
 + Changes to various SysOp side strings to make the aesthetic a bit more
   fitting for REFDoor rather than previous Darkness styling.
 + Futher changes to the local login to make it look a bit more like the local
   login in post RTReader doors like Planets:TEOS 2.00 and LORD 2.
 ! Fixed a various typos in the documentation.

v1.0.1 - Unreleased
---
 ! Fixed an issue where I was using a Integer instead of a LongInt for filesize
   in the REF picker menu. VP defaults to LongInt sizes for integers, but BP/TP
   uses smaller integer sizes which are too limited for this purpose.
 + Added new multinode sensitive open text file procedures similar to xDoor's
   but with remote output and logging of errors. Updated all text file read
   only and append opens (temporary files, mostly) to use the procedures. Many
   of our procs that were opening text files weren't opening them safely.
 ! Fixed a formatting issue where the time left was being set to a different
   color in Xdoor's updateTime proc than in the statusBar drawing proc. This
   usually manifested itself after the statusbar was redrawn by commands like
   DES and PROGNAME.

v1.0.0 - Released 06/19/2020
---
 Initial release.

Download v1.2.0 of REFDoor!

For the record, I’m planning on continuing work on REFDoor for as long as I continue to make any use of it. I was originally pretty wishy-washy on whether there would be any future work done on the engine, but I’ve finally decided that any work I do to it, even as part of another door, will be rolled into REFDoor proper and released. Stay tuned!

6 thoughts on “REFDoor 1.2 Release

  1. Stephen Hurd

    I wish I had come across this work earlier… I too have implemented a REF parser, and your REFDoor documentation is vastly better than the mess that comes with LORD 2. 🙂

    Once I implement the igm and l2cfg stuff in JavaScript, I plan to do a JSDoor release of my LORD 2 work… if it turns out to be popular (ie: people actually work on new REF files), I’ll likely try adding at least some of your REFDoor extensions as well (I likely won’t ever add the extra colour code stuff though since it’s LORD 2).

    If you’re curious about my terrible hacks, you can have a look here:
    https://gitlab.synchro.net/main/sbbs/-/blob/master/xtrn/lord2/lord2.js

    Reply
    1. jack phlash Post author

      Awesome! Glad to know someone else out there is hacking around with REF parsing. I’ll definitely keep an eye on your project – LORD2 deserves a breath of life, IMO.

      For the record, the only real reason I added pipe color codes was because they were already present in the door kit I use, so I wouldn’t really consider them an attempt to extend REF.

      Reply
  2. aLPHA64

    Hey Jack Phlash! I’ve been doing some DOS/Win32 BBS door development, mostly taking a ‘Rumors’ door I wrote for Renegade and swapping out various Pascal-based door kits to get a feel for the functions (DDPLUS, MANNDOOR, D32, XDOOR).

    I’m really digging XDOOR v3, but then I just read this article, and it looks like REFDOOR is alive and kicking! Going to give it a shot. Any words of advice in going from Xdoor to Refdoor?

    Cheers!

    Reply
    1. jack phlash Post author

      Apologies, I somehow missed this comment!

      Well, I’d said that first of all, REFDoor isn’t a door kit in the same was as DDPlus, D32, and XDoor. In fact, it’s made *using* XDoor! Instead, it’s a scripting engine made for making relatively simple doors.

      So, with that in mind, my second point would be that REF is a pretty funky language with lots of limitations and oddities. It can be fun to script in it, working around those limitations, of course, but if you already know Pascal or C, for instance, it might be like more work than simply coding your own door in a high level compiled language. That said, it would be simple to code a rumors door using it. In fact, there are several examples of something similar already.

      As far as it being “alive and kicking”, 1.3 will be released very soon, as well as another new example script and updates to all of my old ones. It should be out sometime before the end of August. If you *do* decide to play with REFDoor, I’m always open to suggests for expanding its capabilities, and of course, bug reports.

      Reply

Leave a Reply to jack phlash Cancel reply

Your email address will not be published. Required fields are marked *