Save

Deprecated. Do not use in new code.

(Engine-Level Function)

Description: This threaded function stores data in a circular historical data file at times indicated by a condition and returns the record number last written to disk.
Returns: Numeric
Usage: Steady State only.
Function Groups: File I/O
Threaded: Yes
Related to:  Save | HistorianDeleteRecords | HistorianGetData | HistorianGetInfo | HistorianReadRecords | HistorianWriteRecords | Get | GetHistory | GetLogInfo | SaveHistory | TGet
Format:  Save(NStatus, NByte, NShort, NLong, NFloat, NText, TSize, Records, Buffers, File, Trigger, V1, V2, ...)
Parameters:  
NStatus
Required. Any numeric expression giving the number of status type values to store in the file. Any functions used in this parameter must be able to be executed in a script, since triggering of the Save re-evaluates it as if it were in a script.

Status types must be equal to a 0 or a 1. This value must be greater than or equal to 0.
NByte
Required. Any numeric expression giving the number of values that are only one byte long. Any functions used in this parameter must be able to be executed in a script, since triggering of the Save re-evaluates it as if it were in a script.

Byte types must be in the range 0 to 255 inclusive. This is a subset of the short type but require 50% less file space than short values and 75% less space than either long or float values. This value must be greater than or equal to 0.
NShort
Required. Any numeric expression giving the number of short type values to store in the file. Any functions used in this parameter must be able to be executed in a script, since triggering of the Save re-evaluates it as if it were in a script.

This value must be greater than or equal to 0.
NLong
Required. Any numeric expression giving the number of long type values to store in the file. Any functions used in this parameter must be able to be executed in a script, since triggering of the Save re-evaluates it as if it were in a script.

This value must be greater than or equal to 0.
NFloat
Required. Any numeric expression giving the number of float type values to store in the file. Any functions used in this parameter must be able to be executed in a script, since triggering of the Save re-evaluates it as if it were in a script.
This value must be greater than or equal to 0.
NText
Required. Any numeric expression giving the number of text type values to store in the file. Any functions used in this parameter must be able to be executed in a script, since triggering of the Save re-evaluates it as if it were in a script.

This value must be greater than or equal to 0.
TSize
Required. A short constant giving the number of characters to reserve for text values. Text values that are longer than this number will lose the characters that fall beyond this limit. All text values stored will have the same size.
Records
Required. Any numeric expression giving the number of entries in the file. If the file is to be created full size (to prevent fragmenting of the file as it grows larger), the size should be multiplied by -1.
Because the file is circular in nature, after this number of records have been written, new records will overwrite old ones, beginning with the oldest record in the file.
Buffers
Required. Any numeric expression for the number of records to keep in memory (RAM) before writing to disk. A value of 0 will result in the data being directly written to disk each time Trigger is true. Setting the buffer to large numbers speeds the average data logging rate substantially but requires RAM equal to the number of records specified by Buffers.
When the RAM buffer is full the entire buffer is written to disk. If the buffer size is changed or if the Save is stopped (such as when VTScada stops), any data in the buffer is immediately written to disk.
Also, if a Get is executed, any data in the buffer will be immediately written to disk, so that the user in ensured of getting all current data.
Normally when Save stops, a record with all invalid fields, and the current time and date stamp, is written to the file. This indicates that data are no longer logged, and fields are unknown until Save resumes. If Buffers is negative, this invalid record is not written, and the number of buffered records is -Buffers - 1. This is to remain compatible with prior versions of VTScada.
Be aware that any data held in a RAM buffer will be lost if VTScada is not correctly stopped (for example, a power failure).
File
Required. Any text expression giving the file name for the historical data. Any path name, including any special, remote, and networked drive is allowed. The default extension is ".DAT" and the default path is the current application path (where VTScada was started). If the file name is prefixed with a period, the path will be to the directory the module is contained in.
Trigger
Required. Any resettable function whose transition from false to true indicates the data items V1, V2, ... are to be written to the file. The Save statement resets the Trigger parameter if the Trigger is true (not 0).
V1, V2, ...
Required. A series of expressions which give the values to be stored in the file. The parameters must be in the order: status, byte, short, long, float, and text. The number of each type is given by the NStatus, NByte, NShort, NLong, NFloat and NText parameters.
There must be exactly the number of Vn parameters specified by each of these parameters.
Comments: This function returns the record number last written to disk. Because this function is threaded and runs as a background job, this value can be used to determine when data have actually been written by examining the change in the return value.
This function creates the historical data file when it is entered during configuration. If the file already exists and is of a different format than specified by the statement, the program automatically converts the file to the new format. This conversion will result in data loss if the number of values of any type is reduced. For example, if NStatus is reduced from 16 to 12, the last four status parameters stored in the file before the change will be lost. If a value type number is increased or remains the same, no data will be lost.
When a Save function is triggered, the next buffer is filled. When the number of filled buffers equals the value implied by the parameter Buffers the file is opened, written, and then closed. This allows a file to be logged to a network server, and read by other VTScada applications on the network. The VTScada NetBIOS I/O driver is not required for this type of network communication. All that is required is a network redirector.
If a file is referenced by two or more active Save functions, there will be unpredictable results, with a possibly corrupted file and reduced performance. Do not reference the same file from more than one Save function at the same time. This is not the case with a SaveHistory, Get or TGet statement. Any calls to SaveHistory will result in data being written to the file in the correct location based on its timestamp. When a Get or TGet is executed, the Save statement will write all of the data that it holds in memory to the file prior to the data retrieval. This also occurs if the application is stopped.
The values of the Vn parameters need not be valid for the Save statement to operate. Any invalid values are recorded as such in the file. The data are stored in the file in the order given by the Vn parameters. This order is important for retrieving the data using the Get statement.
Note that the Trigger parameter is reset when it becomes true. This means that data can be saved at regular time intervals by using TimeOut or AbsTime as the trigger. After TimeOut becomes true, the timer will be restarted by the Save statement. Change can also be used in Trigger to cause data to be stored when data values change by a given amount. The starting point for the Change function will be reset after the Trigger becomes true and the data are stored.

The size of the file can be calculated as follows:

Record *
Ceil(Ceil((NStatus + NByte + NShort + NLong + NFloat + NText)/8) +
Ceil(NStatus / 8) + NByte + 2 * NShort + 4 * NLong + 4 * Nfloat +
(TSize + 2) * NText + 8)
+ 25

For example, if the Save statement was as follows :

Save(3, 1, 4, 0, 6, 1, 24, 10000, ...

The file size in bytes would be :

10000 * (Ceil((3 + 1 + 4 + 0 + 6 + 1) / 8) + Ceil(3 / 8) + 1 +
2 * 4 + 4 * 0 + 4 * 6 + (24 + 2) * 1 + 8)
+ 25
= 10000 * (Ceil(15/8) + Ceil(0.375) + 1 + 8 + 0 + 24 + 26 + 8)+ 25
= 10000 * (2 + 1 + 67) + 25
= 700,025 bytes

The size of the files generated by Save must be considered so the available disk space is not exceeded. Note that the files grow as data is recorded up to a maximum of the calculated size. To prevent fragmenting of the file as it grows over time, it may desirable to create the file full size, by making the Records parameter negative.

It is good practice to create several log files of varying frequency. For example, you may want to log data to a file every second for a week. This will be a very large disk file - the minimum size is 6 megabytes! To load a week's data into an array using a Get statement could take several minutes (depending on the computer), during which VTScada will do nothing but process disk information (this is because VTScada is a non-preemptive multitasking system). Also, most screens have only 640 to 1024 pixels across. Displaying 604800 plot points on a VGA screen will be displaying approximately 1000 data points using 1 plot point - not very efficient or easy to read.

The solution is to record two (or more) separate files. One will be a large file recording data every second for a week - this will be used to look at a few minutes at a time. Another file would record hourly averages (or minimums, maximums, etc.) for a week (168 items) - this will be used to look at the entire week. Variations on this strategy, using a variety of log rates will provide the best solution to your data logging problems, both in disk space and processing time.

Another way to decrease logging frequency is using a Trigger based on Change, if the values logged to the file change significantly only rarely.

File Format:The standard file format for Save files is a 38 byte header, followed by identical format records (as specified by the parameters). The header contains the following information:

Bytes

Size

Description

0-1

Int

Number of status values

2-3

Int

Number of bytes values

4-5

Int

Number of short values

6-7

Int

Number of long values

8-9

Int

Number of float values

10-11

Int

Number of text values

12-13

Int

Text size

14-17

Long

Maximum number of records in file

18-21

Long

Index of next record to write

22

Byte

Bits

Meaning

0

File has not wrapped around

1

File has wrapped around

2

File has not wrapped but was created full size

3-5

File format version

6

Checked flag. This bit is set when the file has been checked for inconsistent timestamps. If this file is opened by the Save, SaveHistory, or the ValidateHistory function, the invalid record is updated, and the bit is set.

7

Clean flag. This bit is cleared whenever data is written to the file by the Save, SaveHistory, or ValidateHistory function. When the Save function is stopped, the invalid record is updated, and the bit is set. If the file is opened by the Save, SaveHistory, or ValidateHistory function, the bit is checked, and if not set, the invalid record at the end of the file is corrected

23-24

Int

Length of a record (in bytes)

25

Reserved

26-29

Long

Actual size of header

30-37

Double

A timestamp for determining the latest time for which valid data was written.

The first 8 items in the header are the values in the parameters of the Save statement that created the file. See the Save statement parameters for a more detailed description.

Bytes 18-21 are a long number that indicates where the next record will be written in this file. Record numbers begin at 0. To find the file offset (in bytes) where the next record will be written, multiply this number by the length of a record (in bytes), and add 25. Thus record 0 will be found at offset 25.

Byte 22 is a flag. If 0, the circular file has not been filled (the size of the file continues to grow). If 1, data have been overwritten and the last record index has "wrapped-around" to position 0 at least once.

Bytes 23 to 24 are an integer that specifies the length of a record in bytes.

Note that the size of the file header may be increased by using the SetLogHeader function, which will cause additional bytes to be added to the end of the header. Information may then be written to the header using an FWrite (or similar) function with a starting offset of 25. If the header size is to be expanded and written to, however, it is crucial that the Save statement not be active at the time, and also that the data written to the expanded header not exceed the size by which the header was increased. If either of these conditions is not met, the file will become corrupt and data may be lost.

25  Reserved

26-29 Long actual size of header in this file

30-37 Double a special timestamp value which, in the event that the file is not closed tidily, allows the latest time for valid data to be determined.

In addition, bits 3-5 of byte 22 have the value 1 to indicate the file format version.

Note that the size of the file header may be increased by using the SetLogHeader function, which will cause additional bytes to be added to the end of the header. Information may then be written to the header using an FWrite (or similar) function with a starting offset of 38. If the header size is to be expanded and written to, however, it is crucial that the Save statement not be active at the time, and also that the data written to the expanded header not exceed the size by which the header was increased. If either of these conditions is not met, the file will become corrupt and data may be lost.

Record Format:Each record consists of the following parts:

1. The first 8 bytes of each record are a double precision number in IEEE format, which indicates the time and date when the record was written, as the number of seconds since midnight on 1 January, 1970.

2. Next, 1 byte is allocated for each 8 data items in the record (or fraction thereof). The bits in each byte indicate whether each item is valid (1) or invalid (0).

3. Data items for the record are written sequentially. Each data type is written in a particular Format:1 byte is allocated for each 8 status items or fraction thereof. Each bit contains 1 status value.

Byte items are written as unsigned integers (1 byte each)

Short items are written as signed integers, low byte first (2 bytes each).

Long items are written as signed integers, low byte first (4 bytes each).

Float items are written in single precision IEEE floating point format (4 bytes each).

Text items consist of 2 + TextSize bytes each. The first 2 bytes indicate the number of meaningful data bytes to follow. All bytes beyond that number contain random data. The data bytes are not null-terminated. If, however, TextSize has the value 0, then this indicates that variable length text storage is required. In this case each text entry in the file has a fixed size of 10 bytes and the actual text is stored in a separate file which has the same root filename as the .DAT file, but has a .STR extension.

It should be noted that the Save statement accepts calculated values for its first six parameters. This allows a more efficient generic data logger to be written. The parameters to be logged are treated as if they were in a script so that the changes to these values do not retrigger the Save (thereby saving time and RAM, since this retriggering is always redundant). The Save statement treats assignments in the first six parameters (which is very rare), so that they will only be executed when the Save trigger becomes true. Only expressions that are allowed in scripts are permissible for the first six parameters.

Example:

Save(1 { Number of status values },
     0, 0, 0 { Number of byte, short, long values },
     numFloat { Number of floating point values },
     1 { Number of text values },
     32 { Size of text value },
     1152 { Number of records (4 days worth) },
     50 { Buffer 50 records before writing },
     "G:\DATA\REACTOR" { Path and file name (default .DAT) },
     AbsTime(1, 300, 0) { Save every 5 minutes },
     valveOpen { Status value },
     mixerTemp { First float value },
     inletTemp { Second float value },
     outletTemp { Third float value },
     coreTemp { Fourth float value },
     chemName { Text value });

This stores 1 status, NumFloat float and 1 text value to a file on G: (a shared network drive) in directory DATA called REACTOR.DAT. Only the first 32 characters of the text value chemName are logged. Values are logged every 5 minutes, and written to disk every 250 minutes (50 records). The number of records has been chosen based on the fact that we want to have exactly 4 days worth of data. This means that we need 4 * 24 * 60 / 5 = 1152 records in the file. Because it may also be important to know how large this file is, we can calculate the size as follows:

1152 records *
[ Ceil((1 status value + 4 float values + 1 text value) / 8) +
  Ceil(1 status value / 8) + 4 * 4 float values +
  (32 chars + 2) * 1 text value + 8
]
+ 25
= 1152 * (Ceil(6/8) + Ceil(1/8) + 16 + 34 + 8) + 25
= 1152 * (1 + 1 + 16 + 34 + 8) + 25
= 1152 * (60) + 25
= 69 145 bytes

Latching and Resetting Functions