From:	SMTP%"carl@SOL1.GPS.CALTECH.EDU" 13-AUG-1994 21:27:17.89
To:	EVERHART
CC:	
Subj:	Re: add lines at fiole start

From: carl@SOL1.GPS.CALTECH.EDU (Carl J Lydick)
X-Newsgroups: comp.os.vms
Subject: Re: add lines at fiole start
Date: 8 Aug 1994 20:51:36 GMT
Organization: HST Wide Field/Planetary Camera
Lines: 78
Distribution: world
Message-Id: <3265so$cml@gap.cco.caltech.edu>
Reply-To: carl@SOL1.GPS.CALTECH.EDU
Nntp-Posting-Host: sol1.gps.caltech.edu
To: Info-VAX@CRVAX.SRI.COM
X-Gateway-Source-Info: USENET

In article <1994Aug8.130135.1@aspen.uml.edu>, welchb@aspen.uml.edu (Brendan Welch, W1LPG) writes:
=1) [Regardless of this particular application] could DEC be pressured
=into creating a PREPEND command to go along with APPEND?

As has been pointed out by several people, for many file organizations/record
types there's no way, in general to do this:  If the stuff being prepended is
not an integral multiple of 512 bytes, you're generally going to have to move
data around in the existing part of the file.  If you do that, then there will
be times when the file is inconsistent.  A system crash at such a time would
quite possibly cause you to lose data (unless you've got jounalling enabled and
DEC also added a new journalling feature).  Appending to a file doesn't cause
this problem to as great an extent:  It never modifies data that's already in
the file, so the file will never be inconsistent.  If the end-of-file mark
isn't updated until the append is completed, then the only recovery operation
needed in the event of a crash in the middle of the append is to execute the
APPEND command again.

By the way, I made a couple of mistakes in an earlier post in this thread:
	1)  I incorrectly gave the value used to indicate a dummy record in a
	    variable-record-length file that doesn't allow records to span
	    block boundaries as %XFF.  It's actually %XFFFF.  Which leads to my
	    second mistake:
	2)  That value can also be used for the same purpose in a
	    variable-record-length file that DOES allow records to span block
	    boundaries.  I don't know that this is documented anywhere, but I
	    just tried it and it works.  That means that, at the cost of some
	    wasted disk space, you could efficiently (time- and I/O-wise)
	    prepend to a variable-record-length file by:
	    1)  Writing the new data to a new file;
	    2)  Writing a dummy record with byte count %XFFFF following the
		real data;
	    3)  Diddling the retrieval pointers for the new file to include the
		blocks from the old file (this step is a precaution to allow
		you to recover in case something crashes during step 4);
	    4)  Diddling the retrieval pointers for the old file to include the
		blocks of the new file;
	    5)  Invalidating the file header for the new file;
	    6)  Marking the new file header as unused in the index file bitmap.
	    7)  Removing the directory entry for the new file;

This is, of course, completely unsupported, at least at the RMS level
(actually, steps 1 and 2 can be accomplished using RMS:  First write the new
file using record I/O;  then close it and reopen it for block I/O.  Read the
last block; using the byte count returned from the read, decide where the first
free byte of the block is, and fill that byte and the next one with %XFFFF.
Then rewrite the entire block). I'm not familiar enough with the ACP/QIO
interface to know whether there's support for this sort of thing there.  If
not, you'd have to open INDEXF.SYS and parse and modify the file headers and
the index file bitmap yourself.  At least you won't have to deal directly with
BITMAP.SYS.

=Obviously one way to do this is to have some dummy records at the beginning
=of every file.   (Admittedly this is no great original idea; the posters
=have sort of talked around this; I just want to mention it explicitly.
=And maybe the original poster's application just can not handle this.)

If the record format's "VARIABLE" or, I think, "VFC", and if the application's
using RMS record I/O, then the method I outlined above would be transparent to
the application:  When RMS sees a record with a byte count of %XFFFF, it
ignores it and skips to the beginning of the next block.  Thus the application
would never see the dummy record.

Now, as to the efficiency of prepending to a file, there *IS* one sort of
sequential file under VMS for which the system already DOES support prepending:
A directory file.  Actually, in this case, the system supports inserting
records in arbitrary locations in the file in terms of other records.  The
record format and attributes of directory files were selected to make this sort
of thing reasonably efficient (variable-length records that can't span blocks). 
And we all know just how efficient adding new records at the beginning of a
directory file is :-(.
--------------------------------------------------------------------------------
Carl J Lydick | INTERnet: CARL@SOL1.GPS.CALTECH.EDU | NSI/HEPnet: SOL1::CARL

Disclaimer:  Hey, I understand VAXen and VMS.  That's what I get paid for.  My
understanding of astronomy is purely at the amateur level (or below).  So
unless what I'm saying is directly related to VAX/VMS, don't hold me or my
organization responsible for it.  If it IS related to VAX/VMS, you can try to
hold me responsible for it, but my organization had nothing to do with it.