Clarion Community Help

The TopSpeed file system offers high performance, high security, and excellent compression for the Clarion development environment. Concerns have been raised about the stability of the driver, because of error messages - posted from the driver - when applications are running and accessing TopSpeed files. TopSpeed files are not more susceptible to corruption than other files; however, error detection is higher because the driver engine is extremely sensitive to the smallest anomaly in the data file.

Performance and security standards require the driver to have very aggressive error checking algorithms. The integrity of the data written into the file is a major concern. For this reason, writes to the data file are handled in two steps. First the data record is written to the file and a flush is issued to clear the buffer and write to the physical disk. Only then is the header of the file updated to reflect the new record and a flush is issued again.

The network hardware and software is responsible for physically writing the information to the disk. If a bad bit is transferred and then written into the file, it is usually identified, by the driver, when the record is decompressed. During decompression, the actual record size is compared to the size reported in the header. If either of these was written incorrectly to the file, the two sizes will not match. This detection will cause the driver to report an error or corruption in the file (tpsbt.cpp line XXXX).

A variety of data files have been tested extensively for corruption, and at this time only one problem has been identified with the file driver. This data corruption occurs if the following code is issued:

STREAM(file)

LOGOUT(1,file)

ROLLBACK

This corruption can be prevented with the following change to the code:

STREAM(file)

FLUSH(file)

LOGOUT(1,file)

ROLLBACK

Clarion data files, while not as stringent on error checking as TopSpeed files, will post errors if a bad bit is written into a field containing key information. If the record and key are not matching, then the driver will post and Invalid Key File error (Error 38) or a Key File Be Rebuilt Error (Error 46).

Both TopSpeed and Clarion files operate in a multi-user environment through shared data access in a station to station manner. This means that there is no particular station that works as a server controlling the requests made to the files. This type of data environment requires that the underlying operating system and network requesters work flawlessly to ensure proper passing of data from station to station. If this layer is faulty or unstable, incorrect bits could be sent to the data files causing data corruption.

If consistent file corruption are occurring on applications using TopSpeed or Clarion data files, there are two steps to fixing these problems. First, fix the corrupted data file; second, prevent further corruption.

Fix the corrupted data file.

TopSpeed data files

TPSFix will recover most corrupted TopSpeed files. Make sure that the box for Build Keys is checked so that the utility will build all the keys. In some corruption cases, TPSFix will initially report no errors found in the file. If this is the case, continue to run the Fix utility to the end of the file. It may still find errors when reading through the file. See Chapter 33 in the Programmer's Guide for more information on the TPSFix utility.

If TPSFix is unable to correct the file, a conversion program can be created which will loop through the file in record order and rebuild the keys. Refer to the Users Guide, Chapter 17, Generating Source for File Conversion, for a step by step explanation for creating a simple conversion program. The Source Structure and Target Structure will be the same in this case, you may wish to write the new file out to a new name.

It may be necessary to only rebuild the key files for the .tps file, for the corruption to be fixed. This can be done in a application by issuing the BUILD(file) statement after the file has been OPENed. Before issuing the BUILD() statement, make sure to issue to the FULLBUILD=ON command to the file driver. This insures that all the keys are rebuilt, even if no changes have been made. FULLBUILD is usally set to OFF to optimize the speed of the driver, so it should be set to ON before the BUILD and OFF after the BUILD is completed, such as:

OPEN(file)

SEND(file,'FULLBUILD=ON')

BUILD(file)

SEND(file,'FULLBUILD=OFF')

Clarion data files:

Because the problem is generally a corrupted key, the key can be deleted and then rebuilt. Rebuilding the key can be done within the Database Browser , by selecting the file and then pressing the Browse Table button in the DCT Explorer toolbar. When prompted to rebuild the keys choose yes. This should fix the corrupted key.

If the automatic rebuild does not work, use the conversion program method described for the TopSpeed files above.

Prevent Further Corruption

Once the file has been corrected and replaced on the network, the underlying reason for the error needs to found and fixed. At this time, only one specific circumstance has been identified as causing corruption to TopSpeed and Clarion data files.

Faulty user hardware. Corruption of files have been reported and tracked down to bad network cards, memory chips, and motherboards. While identifying the faulty hardware is difficult, once it has been replaced the corruption to data files is corrected.