Next: Changes in V4.2
Up: CHANGES AND NEW FEATURES
Previous: CHANGES AND NEW FEATURES
The following describes the most significant changes which occurred in HDS
between versions V4.0 and V4.1 (not the current version).
- 1.
- The C source code of HDS now complies with the ANSI C standard, and
UNIX implementations of HDS use a new "makefile". Both of these
changes are designed to make it relatively straightforward to
implement HDS on new platforms.
- 2.
- HDS now contains routines which automatically determine the primitive
data representation used by common types of computer hardware
(e.g. the floating point number format and byte storage
order). This also makes it easier to implement HDS on new platforms,
since changes to the source code are not normally required.
- 3.
- A DECstation implementation of HDS has been added. An implementation
on Silicon Graphics hardware is also in use, although this is not
currently supported by Starlink.
- 4.
- The SUN Sparcstation implementation of HDS now uses file mapping as
its default mode of file access. This can give appreciable performance
gains, largely, it appears, as a result of reduced memory usage which
may allow better buffering of file contents.
- 5.
- The directory in which HDS resides now contains a file called
hds_datestamp which holds information about how the HDS system
was built. This includes details of the primitive data representation
used by the host machine which may sometimes be of use to programmers.
- 6.
- A bug has been fixed which could cause the length of new _CHAR
objects to exceed the length actually requested. Typically, an extra
digit was being appended to the length as a result of an un-terminated
internal C string.
- 7.
- A bug has been fixed which caused HDS not to recognise primitive data
values written in big-endian IEEE double precision format
(i.e. typically written on a SUN) which have the "bad" data value
due to
a previous format conversion error. The result of this bug was that a
data conversion error could result if the data were read on a machine
(typically a VAX) with a more restricted double precision number
range. The correct behaviour is for the bad SUN value to be converted
silently to the corresponding bad VAX value. This now occurs.
- 8.
- A bug has been fixed which could corrupt a mapped slice of a primitive
object because only a single byte of data was being transferred for
the final element. This could happen both when reading and writing
the data, although not in all circumstances.
- 9.
- A problem has been fixed which typically resulted in "bus errors"
on SUN systems when attempting to access primitive data as double
precision values where format conversion was also required. This
resulted from an inability to handle double precision values passed
from Fortran to C, where the value may be badly aligned in memory
(i.e. on a 4-byte rather than an 8-byte boundary). This problem
was not always repeatable, in the sense that it depended on where the
Fortran compiler placed the relevant variable in memory.
- 10.
- A workaround has been installed for a problem sometimes encountered
when writing to a file served by a VMS machine using the Network File
System (NFS) from a SUN. Depending on the file size and the amount of
data written, a failure to extend the file can sometimes occur. This
seems to happen only when the new file size requested by the SUN lies
between the current end of file and the physical file size allocated
by VMS as a result of clustering of disk blocks. This problem seems to
have arisen since the previous version of HDS due to changes to the
VMS UCX software. The workaround involves repeating the file extend
call if it fails on the first attempt.
- 11.
- A bug has been fixed in the queue handling facility used internally by
HDS. This could cause regions of memory to be over-written. It is not
clear what adverse consequences this may have had.
- 12.
- An error in the documentation concerning the order in which character
arguments should appear when passing mapped character values using the
%VAL facility has been corrected (see §
).
- 13.
- The description of the DAT_BASIC routine has
been improved to make it clear that this routine accesses bytes of
primitive data as written, and does not perform conversion
to/from the data representation of the host machine.
- 14.
- Several minor typos in error messages have been corrected.
- 15.
- Use of the routine HDS_START is no longer
necessary in order to start up HDS before using it. The system is now
self-starting, typically when the first routine which accesses a
locator is called. HDS_START has been documented
as obsolete, but its use remains optional.
- 16.
- Use of the routine HDS_STOP at the end of a
program is now optional, since its action will be performed
automatically by an exit handler. Examples showing its use have been
modified appropriately. Note that HDS_STOP is not obsolete, as it
remains the only method of closing down HDS in the middle of a
program. In practice, the need to do this is likely to be limited.
- 17.
- Since HDS_START and
HDS_STOP are now both optional, the routine
HDS_RUN no longer serves a useful purpose and has
been documented as obsolete.
- 18.
- The method by which HDS determines how long a container file should be
held open has been rationalised. This no longer depends on the
existence of a "top-level"
locator. Instead, the concept of "primary"
and "secondary" locators has been introduced (see
) and a container file remains open for as long as
at least one primary locator is associated with it. Any locator may
be designated as primary by means of the new routine
DAT_PRMRY, thus removing the special status of
top-level locators in this context. Routines which previously
incremented the container file "reference
count" now issue primary locators (all other locators are
secondary by default), so that existing behaviour is retained.
This change has been introduced to remove the requirement that all
software using HDS maintain its own table of top-level locators in
order to prevent container files being closed. It also opens the way
for future improvements to the programming interface of HDS, which
should allow objects to be identified by their pathname as well as by
locator.
- 19.
- All routines which annul
locators will now also close the associated
container file if there are no longer any
primary locators associated with it. Any
outstanding secondary locators associated with a closed container file
are now correctly annulled (previously they were simply left
"dangling"). The main implication of this is that
DAT_ANNUL will now close a container file which
is no longer in use, removing the need to do this explicitly.
- 20.
- As a result of the above changes, the HDS_CLOSE
routine is now redundant, and has been documented as obsolete. Its
continued use is not recommended. Its behaviour has always been
flawed, since it decrements the reference count for a container file
regardless of whether the top-level locator it annuls originally
caused this count to be incremented when it was created. Since several
methods exist for generating top-level locators without incrementing
this count, it is possible for a container file to be prematurely
closed if HDS_CLOSE is used.
HDS_CLOSE may still be useful as an emergency measure to close a file
in the presence of a programming error which has left it open.
- 21.
- A new routine DAT_REFCT has been introduced to
return the current reference count for a container file. This makes it
possible to predict when a file will actually be closed.
- 22.
- The temporary limit on the number of simultaneously open container
files imposed in V4.0 of HDS has been removed. HDS now imposes no
restrictions on the number of open files beyond those set by the host
operating system.
- 23.
- On UNIX systems, HDS now reports an error if the name of a container
file which is already in use is given as the name of a new container
file to be created. This avoids the fairly common problem on UNIX,
where the user of an application supplies the same name for both the
input file and the output file, and ends up with no file at all
because the output over-writes the input before it has been read.
- 24.
- When creating new container files on UNIX systems, HDS will now check
before over-writing an existing file to ensure that it is a regular
file and not a directory or FIFO, etc. An error results if it is
not a regular file.
- 25.
- HDS now consistently ignores all leading and trailing white space on
file names passed to it.
- 26.
- On UNIX systems, HDS now uses the full (absolute) pathnames of all
files when referring to them in messages and when returning file names
via routine arguments.
- 27.
- On UNIX systems, HDS will now pass file names which contain
"special" characters (anything except alphanumerics, `.', `/', `-'
and `_') to a shell for interpretation. This means that normal UNIX
shell syntax may be used to construct file names, thus allowing
environment variable expansion, etc. In fact, any shell command
can, in principle, be used to construct a file name. Pattern-matching
characters are also accepted - if more than one file matches, then
the first match is used.
- 28.
- Two new routines HDS_WILD and
HDS_EWILD have been introduced to permit
"wild-card" searching for HDS container files specified using
pattern-matching characters.
- 29.
- A new tuning parameter `SHELL' has been introduced (see
) to allow a choice of which UNIX shell is
used to interpret file names (this includes wild-carding of file names
using HDS_WILD and
HDS_EWILD). By default, the "sh" shell is
used but, if available, the "csh" and "tcsh" shells may also be
selected. It is also possible to turn shell expansion of file names
off.
- 30.
- It is now possible to modify the default tuning profile of HDS by
means of environment variables, which are read and interpreted by HDS
at startup (see §
). For instance, the definition:
setenv HDS_MAP 0
could be used to disable file mapping in favour of I/O in any
application which uses HDS.
- 31.
- A call to HDS_GTUNE will now only return the
value 1 for the `MAP' tuning parameter (corresponding to file mapping
being used as the file access mode) if this mode has previously been
requested and if it is actually implemented on the host machine. This
makes it possible for the caller to determine whether file mapping is
implemented or not.
- 32.
- Three new special values (-1, -2 and -3) may now be given for
the `MAP' tuning parameter in order to select the file access mode
best suited for a given type of access. These options select (in
order) the file access method which gives fastest sequential access,
fastest random (sparse) access, or minimum usage of memory (see
). The value returned by
HDS_GTUNE may be used to determine which file
access mode was actually selected, as this will depend on the host
operating system.
- 33.
- Some of the values returned by the routine
DAT_CLEN describing the number of digits required
to format floating point numbers have been changed to reflect the
recommendations of the IEEE floating point standard.
- 34.
- The HDS_SHOW routine now has a new `DATA' option
which displays details of the primitive data representation in use by
the host machine.
- 35.
- Internal changes have been made which allow HDS "container records"
(which contain information about the components which reside within an
HDS structure) to be reduced in size when components are erased. Some
hysteresis is allowed in this process. This typically gives a slight
performance improvement and saves a little file space.
- 36.
- The NBLOCKS tuning parameter is now actually used at several places
within HDS. Previously its value had no effect.
- 37.
- A new script called hds_dev has been provided on UNIX systems
to create and remove symbolic links to the HDS public include files
within the current directory. This allows the same INCLUDE statements
to be used in Fortran code on both UNIX and VMS systems.
- 38.
- The new error code DAT__WLDIN and the symbolic constant DAT__NOWLD
have been introduced.
- 39.
- This document (SUN/92) has been updated and produced using a larger
font.
Next: Changes in V4.2
Up: CHANGES AND NEW FEATURES
Previous: CHANGES AND NEW FEATURES
HDS Hierarchical Data System
Starlink User Note 92
R.F. Warren-Smith & M.D. Lawden
23rd February 1999
E-mail:rfws@star.rl.ac.uk
Copyright (C) 1999 Central Laboratory of the Research Councils