draft NCSA HDF Calling Interfaces and Utilities Version 3.2 University of Illinois at Urbana-Champaign NCSA HDF Version 3.3 source code and documentation are in the public domain, available without fee for education, research, noncommercial and commercial purposes. Users may distribute the binary or source code to third parties provided that this statement appears on all copies and that no charge is made for such copies. READ ME NOW If you want to see more software like NCSA HDF, you need to send us a letter, email or US mail, telling us what you are doing with NCSA HDF. We need to know: (1) What science you are working on--an abstract of your work would be fine; and (2) How NCSA HDF has helped you, for example, by increasing your productivity or allowing you to do things you could not do before. We encourage you to cite the use of NCSA HDF, and any other NCSA software you have used, in your publications. A bibliography of your work would be extremely helpful. NOTE: This is a new kind of shareware. You share your science and successes with us, and we can get more resources to share more software like NCSA HDF with you. NCSA Contacts Mail user feedback, bugs, Send communications via electronic and software and manual mail to one of the following: suggestions to: Bug Suggestions NCSA Software Tools Group bugs@ncsa.uiuc.edu HDF bugs@ncsavmsa.bitnet 152 Computing Applications Bldg. All Other Communications 605 E. Springfield Ave. softdev@ncsa.uiuc.edu Champaign, IL 61820 softdev@ncsavmsa.bitnet Disclaimer UI MAKES NO REPRESENTATIONS ABOUT THE SUITABILITY OF THE SOFTWARE FOR ANY PURPOSE. IT IS PROVIDED "AS IS" WITHOUT EXPRESS OR IMPLIED WARRANTY. THE UI SHALL NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY THE USER OF THIS SOFTWARE. The software may have been developed under agreements between the UI and the Federal Government which entitle the Government to certain rights. Trademark Acknowledgments Macintosh and Macintosh II are trademarks of Apple Computer Inc. UNIX is a registered trademark of AT&T. CRAY and UNICOS are registered trademarks and CRAY-2 and CFT77 are trademarks of Cray Research Inc. IBM PC is a registered trademark of International Business Machines Corporation. MS-DOS is a registered trademark of Microsoft Corporation. Sun is a registered trademark and Sun Workstation is a trademark of Sun Microsystems Inc. Table of Contents Introduction Overview xiii What Is HDF? xiii Differences Between HDF 3.1 and HDF 3.2 xiv Future Plans xvii Use of This Manual xvii Chapter 1 NCSA HDF Basics Chapter Overview 1.1 What Is Hierarchical Data Format? 1.1 Why Was HDF Created? 1.1 NCSA HDF Application Software 1.3 Getting Started with HDF 1.5 Examples 1.6 FORTRAN and C Language Issues 1.8 Installing HDF 1.11 Transferring HDF Files 1.11 How to Get HDF 1.11 Chapter 2 Storing Raster Images Chapter Overview 2.1 Header Files 2.1 Raster Image Sets 2.1 Reasons to Use Raster Image Sets 2.3 8-Bit Raster Image Sets 2.3 24-Bit Raster Image Sets 2.15 Sample Programs 2.24 Chapter 3 Storing Palettes Chapter Overview 3.1 HDF 8-Bit Palettes 3.1 Writing Palettes to a File 3.2 Reading Palettes from a File 3.4 Other Palette Routines 3.4 Chapter 4 Storing Rectangular Gridded Arrays of Scientific Data Chapter Overview 4.1 Scientific Datasets 4.1 Header File 4.4 Writing Scientific Datasets to a File 4.4 Reading Scientific Datasets from a File 4.22 Other SDS Routines 4.36 How SDS Routines Store and Convert Scientific Data 4.37 Backward Compatibility 4.39 Sample Programs 4.40 Chapter 5 Annotating Data Objects and Files Chapter Overview 5.1 Annotation Tags 5.1 The Annotation Interface 5.4 Writing Annotations for HDF Objects 5.5 Reading Annotations for HDF Objects 5.7 Listing All Labels for a Given Tag 5.10 Writing Annotations for HDF Files 5.13 Reading Annotations for HDF Files 5.16 Getting Annotation Information from a File 5.20 Chapter 6 NCSA HDF Command Line Utilities Chapter Overview 7.1 Introduction 7.1 hdfls: Listing Basic Information about an HDF File 7.2 hdfed: Editing an HDF File 7.3 fptohdf: Converting Floating-Point Data to SDS and/or RIS8 7.12 Utilities for Working with Raster Image Sets 7.16 hdfpack: Compacting an HDF file 7.22 Appendix A NCSA HDF Tags Overview A.1 Tag Types and Descriptions A.1 Appendix B Routine Lists Raster Image Routines E.1 Palette Routines E.2 Annotations Routines E.3 Scientific Dataset Routines E.3 Utility Routines E.5 Introduction Overview This introduction provides an very brief overview of NCSA hierarchical Data Format (HDF), a description of the differences between HDF 3.2 and HDF 3.1, and a description of the organization and use of this manual and notational conventions. What Is HDF? Hierarchical Data Format (HDF) is a multi-object file format for the transfer of graphical and numerical data between machines. The design of this format allows self-definition of data content, and easy extensibility for future enhancements or compatibility with other standard formats. Features of NCSA HDF Version 3.2 include the following: * Facilities for sharing data across machines and systems. The systems now currently supported by NCSA include: Machine Operating Platform Type System1 Sun-4 SUN SunOS Cray-2 UNICOS UNICOS Cray-Y/MP UNICOS UNICOS Convex (IEEE format) CONVEX ConvexOS Convex (native format) CONVEXNATIVE ConvexOS SGI Iris 4D IRIS4 IRIX IBM RS/6000 IBM6000 AIX DECstation 3100/5000 MIPSEL Ultrix Vax VMS VMS HP 9000 HP9000 HPUX Macintosh MAC MacOS IBM PC & compatibles PC MS Windows/MSDOS NeXT NeXT NeXTStep 1Since operating system release numbers often change, we have not listed them here. In general, HDF is supported on the most recent release of an operating system. Unofficial support for other platforms is available, thanks to HDF users' contributions. Contributed ports of HDF can be found in the "contrib" directory on NCSA's anonymous ftp server. * FORTRAN and C calling interfaces for storing and retrieving 8, and 24-bit raster images, palettes, scientific data and accompanying annotations * Utilities for editing and displaying HDF files, and converting raw data files to HDF files and vice versa * General purpose routines for creating HDF files Differences Between HDF 3.1 and HDF 3.2 HDF 3.2 represents a major increase in the functionality over HDF 3.1. In addition the addition of several new supported platforms (see list above), some of the interfaces have been expanded substantially. SDS Interface The Scientific Data Set (SDS) interface has undergone the most changes that affect typical users. Some of them may necessitate changes to your code. The three biggest changes are (1) the addition of support for new number types in SDS, (2) changes to SDS so that the code no longer transposes arrays, and (3) the addition of a tag for storing calibration information. These changes have necessitated some changes in the SDS interface. The following routines are no longer supported: DFSDsettype (dsstype) DFSDsetmaxmin (dssmaxm) DFSDgetmaxmin (dsgmaxm) The following new routines have been added: DFSDsetNT (dssnt) tells which number type is to be used for next DFSDadddata or DFSDputdata; replaces DFSDsettype1 DFSDgetNT (dsgnt) determines number type of current SDS DFSDsetrange (dssrang) specifies max and min values of next SDS to be written; replaces DFSDsetmaxmin 1DFSDsettype also was used to set the storage order of SDS data. This function is no longer needed, as explained in the section "Backward Compatibility" in Chapter 4. DFSDgetrange (dsgrang) determines max and min values of current SDS; replaces DFSDgetmaxmin DFSDpre32sdg (dsp32sd) tests if the current SDS was written by HDF3.2 or a previous release. These new routines are described in detail in Chapter 4. Annotations Interface A number of annotations routines require programmers to explicitly handle the opening and closing of files. In the past, this was accomplished by calling DFopen and DFclose, which use a structure called a DF. In HDF 3.2, DFopen and DFclose have been replaced by two new routines, Hopen and Hclose. Hopen takes the same arguments as DFopen, but returns an int32 file handle, which is suitable for use with the newly modified annotations routines and Hclose. Therefore programmers who previously used DFopen and DFclose in conjunction with the annotations interface should now use Hopen and Hclose to do file opening and closing. Vset Interface Previously, the Vset calling interface had its own library. It was not included as part of HDF 3.1. In HDF 3.2, the Vset calling interface is contained in the same library as the other HDF interfaces. In previous releases, the set of number types that Vdatas supported was different from the set supported in the SDS interface. In HDF 3.2, the two interfaces support the same number types. Although the Vset interface is part of the HDF 3.2 library, Vset documentation is not included in this document. The Vset documentation is currently being rewritten, and will eventually be merged with the documentation for the other interfaces. New General Purpose Interface The lower layer of HDF has been completely redesigned and reimplemented, and all application interfaces, such as RIS8 and SDS, have been reimplemented on this layer. The new lower layer incorporates the following improvements: * More consistent data and function types * More meaningful and extensive reporting of errors. * Simplification of key lower level functions * Improved techniques for facilitating portability * Support for alternate forms of physical storage, such as linked blocks storage, and storage of part of an object in an external file. * A version tag indicating which version of the HDF library last changed an HDF file. * Hooks to support simultaneous access to multiple files Since users do not normally access the general purpose layer of HDF, most changes will not affect users. However, a few changes should be noted: 1. Hopen should be used instead of DFopen 2. Hclose should now be used instead of DFclose 3. File handles are now of type int32, rather than pointers. Thus, Hopen returns a value of type int32, and all routines that previously had arguments of type "DF -" now take arguments of type int32 instead. The routines that this is most likely to affect for most users are DFfindnextref and the annotation routines that involve file labels and descriptions (see "Annotations Interface"). 4. There is no FORTRAN version of this interface. 5. Hnumber should now be used instead of DFnumber. 6. Hishdf should now be used instead of DFishdf. Details of the new interface are contained in the manual "HDF Specifications," January 1993, currently in draft form and available on the NCSA anonymous ftp server. Old General Purpose Interface Although the previous general purpose interface has been replaced by a new general purpose interface, backward compatibility is maintained by a set of routines that emulate the old routines. All of the old routines that begin with the letters "DF" (DFopen, DFclose, DFgetelement, etc.) have been rewritten on top of the new "H" layer (Hopen, Hclose, etc.). Users who currently use the "DF" routines should be able to continue to use them for some time. However, users are encouraged to switch to using the new "H" routines as soon as they reasonably can, since corresponding "DF" routines will be phased out in some future HDF release. Utilities HDF 3.2 contains the following new utilities, which are described in detail in the chapter "NCSA HDF Command Line Utilities." ristsds converts a series RIS8 HDF files into a single 3D SDS HDF file hdfcomp compresses 8-bit raster images from an HDF file, storing them in a new HDF file. hdf24hdf8 quantizes a HDF RGB 24 bit "pixel" image into a 8 bit image with RGB palette and stores it as a HDF 8-bit raster image file hdfpack condenses an HDF file by reading in all the objects in an HDF file and writing them out to a new file. New Test Suite The HDF 3.2 package distributed on NCSA's anonymous ftp server now contains a test suite that tests the basic functionality of all of the HDF interfaces. Future plans As of this writing (February 1993), substantial new enhancements to HDF are in the pipeline, including: * Support for the netCDF data model and interface within HDF, enabling programs to treat netCDF variables and SDSs interchangeably, where appropriate. * Support for JPEG compression of images, and other forms of compression for SDS storage. * Support for random hyperslab writing to SDS arrays, lifting the current restriction that slices must be written to SDSs in the order in which they are stored. * Support for simultaneous multiple HDF file access with high level interfaces (SDS, RIS, etc.), lifting the current restriction that only one HDF file can be accessed at a time. * New documentation, including a manual "Getting Started with HDF", an "HDF User's Guide," an "HDF Reference Manual," and a rewritten "HDF Specification." Use of This Manual This manual is designed for users who are working on an application that involves the use of HDF files. If you are completely unfamiliar with HDF, you should read or scan Chapter 1, "HDF Basics." The most important information about a particular interface is covered in the chapter that deals with that interface. Each chapter contains a description of the type of data with which the interface deals, descriptions of the routines that constitute the interface, and examples. Manual Contents The manual is organized into the following chapters: Chapter 1, "NCSA HDF Basics," describes HDF, explains the reasons for its creation, and gives a brief overview of available applications software. Chapter 2. "Storing Raster Images," describes the routines that are available for storing and retrieving raster images in HDF format, with or without compression, and with or without palettes. Chapter 3, "Storing Palettes," describes the routines that are available for storing and retrieving 8-bit palettes. Chapter 4, "Storing Rectangular Gridded Arrays of Scientific Data," describes the routines that are available for storing and retrieving scientific datasets. Chapter 5, "Annotating Data Objects and Files," describes the routines that are available for storing and retrieving data and file annotations. Chapter 6, "NCSA HDF Command Line Utilities," lists currently available utility routines and command line utilities that are available for working with HDF files. Appendix A, "NCSA HDF Tags,' presents a table of brief descriptions of most of the tags assigned at NCSA for general use. Appendix B, "Routine Lists," lists all the current routines available for use with NCSA HDF. Form of Presentation The material in this manual is presented in text, screen displays, or command line notation. Text In explaining various features and commands, this manual often presents a word within a paragraph in italics to indicate that the word is defined within the paragraph, or that it is a significant term that should be noted and/or is being mentioned for the first time. So that they are more easily identifiable within this manuscript, utilities, variables, integer arguments, tags, etc. have been presented in courier style. Portions of this manual refer to other portions of the manual where the other portions explain related topics. These cross references usually mention the title of sections or chapters enclosed in quotation marks, such as, See Chapter 1, "NCSA HDF Basics." Command Line Format Notation Throughout this manual, many explanations instruct you to make entries by typing on the keyboard. These entry instructions are printed in courier bold type and appear within a paragraph or on a separate line. The command lines in this manual are normally shown in lowercase, except in rare instances where uppercase is required. When it is necessary for you to press a key that is labeled with more than one character (such as the RETURN key), this manual identifies the key with all capital (uppercase) letters. Keys to be entered are printed in bold type. Keys that are pressed simultaneously or in succession are linked with a hyphen. For example, press CONTROL-A. The meaning of each special notation applied to format lines is listed in Table 1. Table I.1 Meaning of Entry Format Notations Appearance Example Entry Method On separate line; dothis Enter the keys for each lowercase, courier bold type character. Within a text line; RETURN Press the single key uppercase indicated. Within a text line; uppercase; CONTROL-A While holding down the hyphens between key names first one or two key(s) indicated, press the last key indicated. On separate line or within a filename This notation is a text line; italic, lowercase, variable, which courier bold type represents a certain kind of entry, but may consist of different characters every time you make the entry. Square brackets; bold type [] Do not enter the square brackets. Material presented between brackets is optional and should be entered only in certain cases. Ellipses; bold type ... Do not enter ellipses. Ellipses indicate you may enter more material similar to the material preceding the ellipses. Figure 1 shows you how to read and enter a command line. ED. NOTE: Figures are not available in this plain text version of the specification. Figure 1.1 Reading and Entering a Command Line Further Reading More detailed information about the basic HDF structure and how it works can be found in NCSA HDF Specifications, which you may download via FTP (refer to "How to Get HDF" in Chapter 1) or may request by writing NCSA at the addresses listed on the Bugs and Suggestions report form at the back of this manual. Chapter 1 NCSA HDF Basics Chapter Overview What Is Hierarchical Data Format? Why Was HDF Created? NCSA HDF Application Software NCSA Scientific Visualization Software HDF Calling Interfaces HDF Utilities Getting Started with HDF Examples Writing an HDF 8-Bit Raster Image Set Writing an HDF Scientific Dataset FORTRAN and C FORTRAN Stubs Atomic Data Type Specifications Array Specifications Case Sensitivity Name Length Header Files FORTRAN 77, ANSI C and K & R's C HDF Without FORTRAN Installing HDF Transferring HDF Files How to Get HDF FTP Archive Server U.S. Mail Chapter Overview This chapter provides a description of NCSA HDF, the reasons behind its creation, and a brief description of HDF application software. What Is Hierarchical Data Format? The Hierarchical Data Format (HDF) is a multi-object file structure that is designed to facilitate the sharing of data among people, projects, and machines on a network (Fig. 1.1). HDF was created at the National Center for Supercomputing Applications (NCSA) to serve the needs of diverse groups of scientists working on supercomputing projects of many kinds. Figure 1.1 HDF: A File Format for Scientific Data in a Distributed Environment *** INSERT FIGURE HERE *** Why Was HDF Created? Scientists commonly generate and process data files on several different machines, use various software packages to process files, and share data files with others who use different machines and software. Also, the mixture of information that scientists need to work with often varies from one file to another, even for the same application. Files may be conceptually related but physically separated; e.g., some data may be dispersed among different files, some in program code, and some in the minds of various users. HDF addresses these problems by providing a general purpose file structure that does the following: * Makes it possible for programs to obtain information about the data in a file from the file itself, rather than from another source * Lets you store different mixtures of data and related information in different files, even when the files are processed by the same application program * Standardizes the formats and descriptions of many types of commonly used datasets, such as raster images and scientific data * Encourages the use of a common data format by all machines and programs that produce files containing a specific dataset * Can be adapted to accommodate virtually any kind of data by defining new tags or new combinations of tags HDF files are self-describing. For each data object in an HDF file, there are predefined tags that identify such information as the type of data, the amount of data, its dimensions, and its location in the file. The self-describing capability of HDF files has important implications for processing scientific data. It makes it possible to fully understand the structure and contents of a file just from the information stored in the file itself. A program that has been written to interpret certain tag types can scan a file containing those tag types and process the corresponding data. Self- description also means that many types of data can be bundled in an HDF file. For example, it is possible to accommodate symbolic, numerical, and graphical data in one HDF file. Related items of information about a particular type of data are grouped into sets, such as the raster image sets (see Chapter 2, "Storing Raster Images") and scientific datasets (see Chapter 4, "Storing Rectangular Gridded Arrays of Scientific Data"). Each set defines an application area supported by HDF. Additional sets can be defined and added to HDF as the needs arise. Figure 1.2 shows a conceptual view of an HDF file containing a scientific dataset. The actual two-dimensional array of data is only one element in the set. Other elements include the number of dimensions (rank), the sizes of the dimensions, identifying information about the data and axes, and scales (ranges) for the axes. Figure 1.2 HDF File with Scientific Dataset *** INSERT FIGURE HERE *** NCSA HDF Application Software NCSA HDF application software currently comes in three forms: (1) NCSA scientific visualization tools that read and write HDF files, (2) calling interfaces that let you read and write HDF files from within a FORTRAN or C program, and (3) command-line utilities that operate directly on HDF files. The integration of these types of software in the computing environment at NCSA is illustrated in Fig 1.3. Visualization tools such as NCSA Collage and NCSA DataScope read and write HDF files. Calling interfaces in the HDF library (libdf.a) let you read and write HDF files from your programs. And utilities such as r8tohdf let you operate on HDF files at the command level. Figure 1.3 HDF Software in an Integrated Computing Environment *** INSERT FIGURE HERE *** NCSA Scientific Visualization Software The use of HDF files guarantees the interoperability of the scientific visualization tools at NCSA. Some tools operate on raster Software images, some operate on color palettes, some use images, color palettes, data and annotations, and so forth. HDF provides the range of data types that these tools need, in a format that lets different tools with different data requirements operate on the same files without confusion. HDF Calling Interfaces In order to minimize the amount of knowledge you need to have about HDF, calling interfaces have been developed for specific types of applications, such as the storage and display of raster images or scientific data archiving. A calling interface is a library of routines that can be called from an application program for storing and retrieving information, including raw data, from a particular type of HDF file. Different applications typically require different interfaces. Consequently, NCSA HDF provides FORTRAN and C calling interfaces for storing and retrieving 8- and 24-bit raster images, palettes, scientific data, and annotations. These interfaces, which are described in detail in chapters 2 through 5, are mutually compatible, and user programs can combine calls to routines in different interfaces when they need to store different kinds of data in the same file. HDF files tend to be used on several different machines, and HDF interfaces developed at NCSA are implemented on as many machines as possible. An important goal in the development of NCSA HDF user interfaces is to eliminate the necessity of changing program code when moving an application from one machine to another. HDF Utilities The HDF command line utilities are application programs that can be executed by entering them at the command level, just like other UNIX commands. They make it possible for you to perform, at the command level, common operations on HDF files for which you would normally have to write your own program. For example, the utility r8tohdf is a program that takes a raw raster image from a file and stores it in an HDF files in a raster image set. The HDF utilities provide capabilities for doing things with HDF files that would be very difficult to do under your own program control. For example, the utility hdfls lists the contents of an HDF file, including such things as the meanings of tags and the size and location of each data item. The HDF utilities are described in detail in Chapter , "NCSA HDF Command Line Utilities." Getting Started with HDF If you do not have access to a machine that already has an HDF library, you will need to install it yourself or have your system administrator install it. Although procedures for installing the HDF library vary from one system to another, the basic steps are the same in all cases. First, you need to get the software. The section, "How to Get HDF" tells you how to get the HDF software from NCSA. In some cases you can get the actual precompiled library, in which cases you can load it into your machine into an appropriate directory. If instead you get the source code for the HDF library, you first have to compile the source code into a linkable library. Detailed information on how to install and use HDF on specific systems can be found in the documentation that comes with the system-specific versions of the software. Examples Writing an HDF 8-Bit Raster Image Set A typical use of HDF involves preparation of scientific data for visualization as an 8-bit raster image. Using 8-bit raster imaging, the values on a grid of numbers can be represented by color values in a palette of 256 colors. The following code segments, the first in FORTRAN, the second in C, convert a 200 x 100 floating-point array to an 8-bit raster image, then store the image in an HDF file. FORTRAN: CHARACTER*1 image(100,200) INTEGER istat, d8aimg C Other Fortran code goes here C Convert values in array ivals to character (8-bit) data do 10 ix=1,100 do 10 iy=1,200 image(ix, iy) = char(ivals(ix, iy)) 10 continue C Write image to an HDF file istat = d8aimg(`myfile.hdf', image, 100, 200, 0) if (istat ne. 0) then write(*,*) `Error writing HDF file' endif C: char image[200][100]; int ix, iy, istat, DFR8addimage(); /* Other C code goes here */ for (ix=0; ix<200; ix++) for (iy=0; iy<100; iy++) image[ix][iy] = (char) (ivals[ix][iy]); istat = DFRBaddimage("myfile.hdf", image, 200, 100, 0); if (istat != 0) printf("Error writing HDF file\n"); NOTE: DFR8addimage writes the image stored in the array image to myfile.hdf. If myfile.hdf exists, the image is appended to the file. If myfile.hdf does not exist, a new file is created, and the image is written as the first image in the file. The variable istat is assigned the value 0 if DFR8addimage succeeds; , -1 is assigned if it fails. The routine DFR8getimage is available for retrieving images from HDF files. Routines are also available for storing color lookup tables (palettes) with raster images. Linking to the HDF library If your FORTRAN or C program makes a call to HDF, it must be linked to the HDF library. Normally the HDF library is in a file called libdf.a. You can indicate the linkage in your compile statement, or if a separate linkage step is used, it may be done at that time. Hence, if libdf.a is in your current directory, you compile and link your program to HDF with a statement such as the following. f77 -o myprog myprog.f libdf.a (FORTRAN) cc -o myprog myprog.c libdf.a (C) If your library is contained in a public library directory on a UNIX system, you can link it with a statement such as f77 -o myprog myprog.f -ldf (FORTRAN) cc -o myprog myprog.c -ldf (C) Writing an HDF An HDF Scientific Dataset An HDF scientific dataset (SDS) is a collection in an HDF file of information about scientific data stored as a multi-dimensional array of numbers. Each SDS must include the actual data array, its rank (number of dimensions), and its dimensions. Optionally, an SDS can also contain scales to be used along the different axes when interpreting the data, maximum and minimum values of the data, calibration information, and the coordinate system used to interpret the data. Labels, units, and format specifications for displaying and interpreting the data and dimensions may also be included. Below is code presented first in FORTRAN, then in C, that stores a 200 x 200 16-bit integer array called "pressure" in an SDS in the HDF file, Ex.hdf. It also stores labels, units, and formats as part of the same SDS. FORTRAN: INTEGER dssdims, dssnt, dssdast, dssdist, dsadata integer*2 pressure(200,200) INTEGER shape(2), ret, DFNT_INT16 DFNT INT16 = 22 shape(1) = 200 shape(2) = 200 ret = dssnt(DFNT_INT16) ret = dssdims(2, shape) ret = dssdast(`pressure 1', `Pascals', `E15.9', `cartesian') ret = dssdist(1, `x', `cm', `F10.2') ret = dssdist(2, `y', `cm', `F10.2') ret = dsadata(`Ex.hdf', 2, shape, pressure) C: #include "hdf.h" int16 pressure[200][200); int shape[2]; shape[1] = 200; shape[2] = 200; DFSDsetNT(DFNT_INT16); DFSDsetdims(2, shape); DFSDsetdatastrs("pressure 1", "Pascals", "E15.9", "cartesian"); DFSDsetdimstrs (1, "x", "cm", "F10.2"); DFSDsetdimstrs (2, "y", "cm", "F10.2"); DFSDadddata("Ex.hdf", 2, shape, pressure); NOTE: The "set" calls (DFSDsetdims(), etc.) indicate the ancillary information that is to be stored with the SDS. DFSDsetdims is required; the others are optional. DFSDsetNT indicates that the number type of the data is 16-bit integer. DFSDadddata writes the scientific dataset data to Ex.hdf. If Ex.hdf exists, the SDS is appended to the file. If Ex.hdf does not exist, a new file is created, and the SDS is written as the first in the file. FORTRAN and C Language Issues The necessity to support both FORTRAN and C interfaces for HDF inevitably leads to some difficulties in the design of the interfaces. What is natural to a C interface can be quite unnatural to a FORTRAN interface and vice versa. In order to make the FORTRAN and C versions of each routine as identical as possible, some compromises have often had to be made in the simplification of one or the other routine. FORTRAN Stubs Almost all of the actual code underlying the HDF interfaces is written in C. Every call to a FORTRAN routine ultimately makes access to a C routine that actually carries out the prescribed function. So, the FORTRAN routines might better be referred to as FORTRAN stubs rather than FORTRAN functions. When called, these stubs typically translate all parameter values immediately to a data type that is accessible to C, then call a corresponding C function to do the actual work. Atomic Data Type Specifications When mixing machines, compilers, and languages, it is difficult to keep straight differences in data types. For instance "integer" might be a 32-bit quantity on one machine, a 16-bit quantity on another, and 64-bits on a third. Differences between FORTRAN and C also leads to some difficulties in describing some of the data types in the argument lists of HDF routines. The help keep matters straight, special names have been given to all data types used in HDF routines. The following table shows the names used in all descriptions of C and FORTRAN routines for the data types used. C FORTRAN type of data name name 8-bit signed integer int8 CHARACTER*1 8-bit unsigned integer uint8 (not supported) 16-bit signed integer int16 CHARACTER*2 16-bit unsigned integer uint16 (not supported) 32-bit signed integer int32 ER*4 32-bit unsigned integer uint32 (not supported) 32-bit floating point number float32 REAL*4 64-bit floating point number float64 REAL*8 generic integer intn INTEGER The data types marked "NA" in the FORTRAN column are not generally available in FORTRAN, so no convention is indicated. In most cases, it is obvious how you should declare variables in you program to conform to these data types. For example a "uint16" in C would normally be "unsigned short." But in some cases the correspondence may be a little more difficult. When in doubt, consult the file hdfi.h, which contains type definitions for all of these types on all machines that HDF supports. To automatically define these data types, C programmers can use "#include" to include hdfi.h with their program. Array Specifications Some of the routines covered in this manual place no restrictions on the rank (number of dimensions) that a data array can have. This is perfectly legal in C, but unnatural in FORTRAN. Fortunately, since both C and FORTRAN pass arrays by reference, no problem arises in the actual interface between the FORTRAN calls and the corresponding stubs. The only real problem is in the notation used in this manual to describe the routines as if they were actual FORTRAN routines. As a result, in the declarations contained in the headers of FORTRAN functions, we use the following conventions: * CHARACTER*1 x(*) means that x refers to an array that contains an indefinite number of characters. It is the responsibility of the calling program to allocate enough space to hold whatever data is stored in the array. * REAL x(*) means that x refers to an array of reals of indefinite size and of indefinite rank. It is the responsibility of the calling program to allocate an actual array with the correct number of dimensions and dimension sizes. Case Sensitivity Another difference between FORTRAN and C is that FORTRAN identifiers, in general, are not case sensitive, whereas C identifiers are. Although all of the FORTRAN routines shown in this manual are written in lower case, FORTRAN programs that call them can use either upper or lower case without loss of meaning. Name Length Since some FORTRAN compilers can only interpret identifier names with seven or fewer characters, the names of the FORTRAN routines have been restricted to seven or fewer characters. Header Files If your program uses special HDF declarations or definitions, you may need to include a header file. The primary header file is hdf.h. It contains declarations and definitions that are used by the C routines. For example, prototypes for all C routines can be automatically included by including hdf.h. Also, if your program uses mnemonics for tags, the corresponding numerical values for the tags can be found in hdf.h. There is also a file called constants.f that contains FORTRAN parameter statements that declare the HDF constants that you are most likely to use in a FORTRAN program that invokes HDF routines. The inclusion of header files is not, in general, permitted by FORTRAN compilers. it is, however, sometimes available as an option. On UNIX systems, for example, the macro processors m4 and cpp let your compiler include and preprocess header files. If this or a similar capability is not available, you may have to copy whatever declarations, definitions, or values you need from constants.f into your program code. FORTRAN 77, ANSI C, and K&R C. As much as possible, we have tried to stick closely to those implementations of the two languages that are in most common use today, namely FORTRAN 77, ANSI C and K&R C. If your FORTRAN or C compiler understands FORTRAN 77, ANSI C, or K&R C, it should be able to link easily to the interfaces. Although we try to adhere to these standards, we must note that a primary objective of the HDF project is to support HDF on a variety of different machines, and this, in some cases, means accommodating some deviations. We are also aware of the fact that many potential users of HDF have compilers that our code does not accommodate. Let us know if your particular dialect does not work with HDF. We may or we may not be able to help. HDF Without FORTRAN If you do not use FORTRAN with HDF, you may want to compile the HDF library without any FORTRAN routines in it. Two instances in which you may choose to do so are the following: 1) If you want to reduce the size of the HDF library 2) If you don't have a FORTRAN compiler to use in compiling the library Details on how to compile HDF without FORTRAN are contained in the INSTALL file that is included with the anonymous ftp version of HDF. Installing HDF Details on how to install HDF are beyond the scope of this manual, but you can get quite a bit of information about the process from the readme files that come with the source code. See the section below "How to Get HDF" for information on how to get the actual HDF software. Transferring HDF Files HDF files are binary files, so any transfer protocol that transfers binary files without changing them can be used to transfer HDF files. Many HDF users use FTP to transfer HDF files. If you use FTP, switch to binary mode when transferring HDF files. If you use NCSA Telnet and you wish to transfer an HDF file to or from a Macintosh, you must pay special attention to whether or not to enable the "Macbinary" option. There are two case to consider: * If the HDF file is not from a Macintosh application (e.g., it is a normal HDF file generated by your FORTRAN or C program), then be sure to turn Macbinary mode off before performing the transfer. * If the HDF file corresponds to a Macintosh application (e.g., NCSA Layout, NCSA DataScope, etc.), and you want to transfer it so that it can be accessed from a Macintosh application on another Mac, then be sure to turn Macbinary mode on before performing the transfer. How to Get HDF You may obtain NCSA software via FTP, an archive server, or U.S. mail. Instructions for doing so are provided below. FTP If you are connected to Internet (NSFNET, ARPANET, MILNET, etc.) you may download NCSA HDF software, documentation, and source code, at no charge from an anonymous file transfer protocol (FTP) server at NCSA. The procedure you should follow to do so is presented below. If you have any questions regarding this procedure or whether you are connected to Internet, consult your local system administration or network expert. 1. Log on to a host at your site that is connected to the Internet and is running software supporting the FTP command. 2. Invoke FTP on most systems by entering the Internet address of the server: ftp ftp.ncsa.uiuc.edu or ftp 141.142.20.50 3. Log in by entering anonymous for the name. 4. Enter your e-mail address for the password. 5. Enter got README.FIRST to transfer the instructions (ASCII) to your local host. 6. Enter quit to exit FTP and return to your local host. 7. Review the README.FIRST file for complete instructions concerning the organization of the FTP directories and the procedure you should follow to download the README files that contain further information on how to get and compile the most recently released version of HDF for your machine and operating system and to determine which files to transfer to your home machine. Your login session should resemble the sample presented below, where the remote user's e-mail address is smith@xyz..univ.edu and user entries are indicated in boldface type. harriet_51% ftp ftp.ncsa.uiuc.edu Connected to zaphod. 220 zaphod FTP server (Version 4.173 Tue Jan 31 08:29:00 CST 1989) ready. Name (ftp.ncsa.uiuc.edu: smith): anonymous 331 Guest login ok, send ident as password. Password: smith@xyz.univ.edu 230 Guest login ok, access restrictions apply. ftp> get README.FIRST 200 PORT command successful. 150 Opening ASCII mode data connection for README.FIRST (10283 bytes). 226 Transfer complete. local: README.FIRST remote: README.FIRST 11066 bytes received in .34 seconds (32 Kbytes/s) ftp> quit 221 Goodbye. harriet_52% NCSA HDF documentation, program, and source code are now in the public domain. You may copy, modify, and distribute these files as you see fit. Archive Server To obtain NCSA software via an archive server: 1. E-mail a request to: archive-server@ncsa.uiuc.edu 2. Include in the subject or message line, the word "help." 3. Press RETURN. 4. Send another e-mail request to: archive-server@ncsa.uiuc.edu 5. Include in the subject or message line, the word "index." 6. Press RETURN. For example, if you use the UNIX mailing system, your login session should resemble the following sample, where user entries are indicated in boldface type. yoyodyne_51% mail archive-server@ncsa.uiuc.edu Subject: help . EOT Null message body; hope that's ok yoyodyne_52% mail archive-server@ncsa.uiuc.edu Subject: index . EOT Null message body; hope that's ok The information you receive from both the help and index commands will give you further instructions on obtaining NCSA software. This controlled-access server will e-mail the distribution to you one segment at a time. U.S. Mail Like other NCSA software, NCSA HDF is also available for purchase- -either individually or as part of the anonymous FTP reel or cartridge tapes--through the NCSA Technical Resources Catalog. Orders can only be processed if accompanied by a check in U.S. dollars made out to the University of Illinois. To obtain a catalog, contact: NCSA Documentation Orders 152 Computing Applications Building 605 East Springfield Avenue Champaign, IL 61820 (217) 244-0072 Chapter 2 Storing Raster Images Chapter Overview Header Files Raster Image Sets Reasons to Use Raster Image Sets 8-Bit Raster Image Sets Compression Schemes Writing 8-Bit Raster Images to a File Reading 8-Bit Raster Images from a File 24-Bit Raster Image Sets Interlace Schemes Compression Schemes Writing 24-Bit Raster Images to a File Reading 24-Bit Raster Images from a File Examples Involving 24-bit Raster Image Sets Sample Programs A C Program to Convert a Raw Palette and Raw Raster Image to HDF RIS8 Format C Functions to Convert Floating-Point Data to 8-Bit Raster Data Chapter Overview This chapter discusses the purposes and use of raster image sets, which allow you to store an image, together with its dimensions and a palette, in an HDF file. This chapter specifically introduces and describes the two raster image set interfaces currently contained in the HDF library: RIS8 and RIS24. Header Files The header file hdf.h contains the declarations and definitions that are used by the routines listed in this chapter. This file can, if needed, be included with your C source code, and in some cases also with FORTRAN code. Raster Image Sets A raster image set (RIS) is a set of tags and associated information required to store an image in an HDF file. In HDF, 8-bit raster image sets (RIS8) are used to store 8-bit raster images, and 24-bit raster image sets (RIS24) are used to store 24-bit raster images. The HDF library currently contains routines for storing raw raster images in RIS8 or RIS24 format and for retrieving raster images from files containing raster image sets. These routines are callable from C and FORTRAN programs that have access to the library. Table 2.1 lists the long and short names and the functions of the RIS8 and RIS24 routines currently contained in the HDF library. The following sections provide descriptions and examples of these calling routines. Table 2.1 Raster Image Set Routines in the HDF Library FORTRAN C Name Name Function DFR8setpalette d8spal sets the default palette to be used for subsequent images. DFR8addimage d8aimg appends the RIS8 for the image to the file. DFR8putimage d8pimg writes out the RIS8 for the image as the first image in the file. DFR8writeref d8wref sets the reference number of the image to write next DFR8getdims d8gdims retrieves the dimensions of the image and indicates whether a palette is associated and stored with the image. DFR8getimage d8gimg retrieves the image and any associated palette, and stores them in arrays. DFR8readref d8rref sets the reference number of the image to get next DFR8restart d8first causes the next get command to read from the first RIS8 in the file DFR8nimages d8nims counts the number of images stored in the file (FORTRAN version currently not available) DFR8lastref d8lref returns reference number of last RIS8 read or written DF24setil d2setil sets the interlace to be used when writing out the RIS24 for the image. DF24addimage d2aimg appends the RIS24 for the image to the file. DF24putimage d2pimg writes out RIS24 for the image to the file. DF24getdims d2gdims retrieves the dimensions and interlace of the image. DF24getimage d2gimg retrieves the image and stores it in an array. DF24readref d2rref sets the reference number of the image to get next DF24restart d2first causes the next get command to read from the first RIS24 in the file, rather than the next one. DF24reqil d2reqil specifies an interlace to be used in place of the interlace indicated in the file when the next raster image is read. Reasons to Use Raster Image Sets When raster images are stored in the form of HDF raster image sets, it becomes possible to use a variety of software tools for displaying and manipulating them. NCSA Image, for instance, can operate directly on images stored in HDF raster image format. Other software can display raster images in HDF format on a variety of different machines. A Raster Image Set (RIS) is a collection of information related to a raster image. A RIS can include such information as the dimensions, compression scheme, and interlacing scheme use for a particular image, along with the image itself. When this information is stored together in a file with an image, software does not have to search elsewhere for this pertinent information. This reduces the need to coordinate disparate pieces of information about a raster image, making the job of creating and running image-processing programs significantly easier. 8-Bit Raster Image Sets The phrase, 8-bit raster image set (RIS8), refers to the set of tags and associated information required to store an 8-bit raster image in an HDF file. An RIS8 contains at least the first three of the following components and may also contain a palette: * An image-here, a two-dimensional array of 8-bit numbers, one for each pixel in the raster image, where pixel values range from 0 to 255 (Pixel values indicate to the hardware which colors to use when drawing the corresponding pixels on the screen.) * Dimensions-two values that represent the x and y dimensions of the image, respectively * A compression scheme-a code that indicates if and how the image was compressed (See the following section, "Compression Schemes.") * A palette--a lookup table with 256 entries that tells the color to associate with each of the 256 possible pixel values (Each entry in the palette is chosen from a master palette of 224 RGB colors. Each palette entry consists of three bytes, one each for red, green, and blue. The first three bytes represent the R, G, and B values of the first color in the palette; the next three the R, G, and B values of the second color; and so forth. The total size of a palette is 768 bytes.) An example of an HDF file with two raster image sets is illustrated in Figure 2.1. Figure 2.1 Two Raster Image Sets in an HDF File *** INSERT FIGURE HERE *** Compression Schemes A compression scheme indicates if and how an image is compressed. Compression schemes currently supported by NCSA HDF are run length encoding and IMCOMP. The value of the integer argument compress in DFR8putimage and DFR8addimage determines which scheme, if any, is to be used, as shown in Table 2.2. Table 2.2 Compression Scheme Codes Value Compression Scheme 0 none DFTAG_RLE run length encoding (RLE) DFTAG_IMCOMP IMCOMP The HDF tags DFTAG_RLE and DFTAG_IMCOMP are defined as the values 11 and 12, respectively, in the file `hdf.h'. You can avoid using numbers for compression codes if you include this file in your program. A Note About RLE The run length encoding (RLE) method used in HDF works as follows: Each sequence of pixels begins with a count byte. The low seven bits of the count byte indicate the number of bytes in the sequence (n). The high bit of the count byte indicates whether the next byte should be replicated n times (high bit=1), or whether the next n bytes should be included as is (high bit=0). The amount of space saved by RLE depends upon how much repetition there is among the pixels in the rows. (Pixels are stored in rows.) If there is a great deal of repetition, much space is saved; if there is little repetition, the savings can be very small. In the worst case--when every pixel is different from the one that precedes it--an extra byte is added for every 127 bytes in the image. A Note About IMCOMP IMCOMP should be used with caution if you are concerned about losing information in your image. IMCOMP compression first breaks an image into 4 x 4 arrays of pixels, then for each array chooses two colors to distribute in the array. (These two colors are added to a 256-color palette that IMCOMP compression builds. This new palette is based on, but different from, the original palette assigned.) Each of the 16 pixels in the 4 x 4 array can now be represented by one bit (0=first color; 1=second color). In addition to these sixteen bits, there are two bytes that give the palette locations of the two colors that were assigned to the 4 x 4 array. Since each 4 x 4 array uses only 4 bytes, IMCOMP stores an image at a cost of 2 bits per pixel, which is 25% of the original storage requirement for the 8-bit image. The drawback of this savings is loss of information--only two colors are allowed to occupy each 4 x 4 array of pixels-whereas in the original image, 16 colors could occupy the same space. For many images this cost is bearable and hardly noticeable, but for some images, the results can be totally unrecognizable. Also note that IMCOMP is dependent on the existence of a palette. If you are going to use IMCOMP, you must include a palette with your image. Writing 8-Bit Raster Images to a File This section contains descriptions of routines supported in the HDF library for writing 8-bit RIS to HDF files. DFR8setpalette FORTRAN: INTEGER FUNCTION d8spal(palette) CHARACTER*1 palette(768) - palette to go with image C: int DFR8setpalette(palette) uint8 palette[768]; /* palette to go with image */ Purpose: To indicate what palette, if any, is to be used for subsequent images. Returns: 0 on success; -1 on failure. The palette that is set here continues as the default palette until it is changed by a new call to the routine. DFR8putimage FORTRAN: INTEGER FUNCTION d8pimg(filename, image, width, height, compress) CHARACTER*(*)filename - name of file to store RIS8 in INTEGER width, height - dimensions of image CHARACTER image(width, height) - array holding image to be put in file INTEGER compress - type of compression to use, if any C: int DFR8putimage(filename, image, width, height, compress) char *filename; /* name of file to store RIS8 in */ int32 width, height; /* dimensions of image */ char image [height] [width]; /* array with image */ /* to put in file */ uint16 compress; /* type of compression to use, if any Purpose: To write out the RIS8 for the image as the first image in the file, overwriting any information that was previously in the file. Returns: 0 on success; -1 on failure. The argument compress identifies the scheme to be used for compressing the data, if any. Refer to Table 2.2 for valid values of compress. If IMCOMP compression is used, the image must include a palette. (See the discussion of 8-bit compression schemes in the section, "Compression Schemes.") NOTE: DFR8putimage overwrites any information that exists in the HDF file. To write an image to a file by appending it, rather than overwriting it, use DFR8addimage (see below) . NOTE: In FORTRAN, the dimensions of the array image MUST be the same as the dimensions of the image itself. NOTE: The order in which you declare dimensions is different between C and FORTRAN. Ordering varies because FORTRAN arrays are stored in column-major order, while C arrays are stored in row-major order. (Row-major order implies that the horizontal coordinate varies fastest). When DFR8putimage writes an image to a file, it assumes row-major order. The FORTRAN declaration that causes an image to be stored in this way must have the width as its first dimension and the height as its second dimension, the reverse of the way it is done in C. To take this into account as you build your image in your FORTRAN program, you need to build the image "on its side." DFR8addimage FORTRAN: INTEGER FUNCTION d8aimg(filename,image,width,height, compress) CHARACTER*(*) filename - name of file to add RIS8 to CHARACTER image(width,height) - array holding image to be added to file INTEGER width, height - dimensions of the image INTEGER compress - type of compression to use, if any C: int DFR8addimage(filename,image,width,height,compress) char *filename; /* name of file to add RIS8 to */ char image(height)[width); /* array holding image to add to file */ int32 width, height; /* dimensions of the image */ uint16 compress; /* type of compression to use, if any */ Purpose: To append to the file the RIS8 for the image. Returns: 0 on success; -1 on failure. In all other respects, DFR8addimage is functionally equivalent to DFR8putimage. DFR8writeref FORTRAN: INTEGER FUNCTION DFR8wref(name, ref) character*(*) name - name of file containing image integer ref - reference number for next d8pimg or d8aimg C: int DFR8writeref(filename, ref) char *filename; /* file containing image */ uint16 ref; /* reference number for next DFR8putimage or DFR8addimage */ Purpose: To specify the reference number of the image to be written when DFR8addimage or DFR8putimage is next called. Returns: 0 on success; -1 on failure. CAUTION: It is unlikely that you will need this routine, but if you do, use it with caution. There is no guarantee that reference numbers appear in sequence in an HDF file; therefore, it is not safe to assume that a reference number is the sequence number for an image. Example: Writing a Palette and an Image in RIS8 Format Figure 2.2 demonstrates in FORTRAN how a palette stored in the array colors and a 400x600 (height=400, width=600) raw image stored in the array picture are written to a file in RIS8 format. The image is not compressed. Figure 2.2 Storing an RIS8 FORTRAN: INTEGER d8spal, d8pimg CHARACTER*1 colors(768) CHARACTER*1 picture(600,400) INTEGER ret ... ret = d8spal(colors) ret = d8pimg(`myfile.hdf',picture,600,400,0) if (ret ne. 0)write(*,*) `Error writing image.' ... The RIS8 with this palette and image is stored as the first image in `myfile.hdf'. Note that if something already existed in this file, it will be lost, because d8pimg recreates the file. If you simply want to append an image to the file, use d8aimg. Example: Writing a Series of RIS8 Images Figure 2.3 illustrates a series of FORTRAN calls in which four 800 x 1200 (height=800; width=1200) images are written to the same file. The first two use palette palA and are compressed using the run length encoding technique (DFTAG_RLE); the third and fourth use palette palB and are not compressed. Figure 2.3 Storing Multiple RIS8s in a Single File FORTRAN: INTEGER d8spal, d8pimg, d8aimg CHARACTER*1 palA(768), palB(768) CHARACTER*1 pic1(1200,800), pic2(1200,800) CHARACTER*1 pic3(1200,800), pic4(1200,800) INTEGER ret, DFTAG-RLE PARAMETER (DFTAG_RLE = 11) ret = d8spal(palA) ret = d8pimg(`myfile',pic1,1200,800,DFTAG_RLE) ret = d8aimg(`myfile',pic2,1200,800,DFTAG_RLE) ret = d8spal(palB) ret = d8aimg(`myfile',pic3,1200,800,0) ret = d8aimg(`myfile',pic4,1200,800,0) Reading 8-Bit Raster Images from a File The two routines, DFR8getdims and DFR8getimage, are sufficient to read raster images from a file. If enough is known about the images and palettes, only the latter routine is needed. DFR8getdims FORTRAN: INTEGER FUNCTION d8gdims(filename,width,height,ispalette) CHARACTER*(*) filename - name of file with RIS8 image INTEGER width, height - dimensions of next image in file INTEGER ispalette - 1 if there is a palette, else 0 C: int DFR8getdims(filename,width,height,ispalette) char *filename; /* name of file with RIS8 image */ int32 *width, *height; /* dimensions of next image in file */ int *ispalette; /* 1 if there is a palette, else 0 */ Purpose: To open the file with name filename, find the next image, retrieve the dimensions of the image in width and height, and tell, via ispalette, Whether there is a palette associated with the image. Returns: 0 on success; -1 on failure. If the file is being opened for the first time, DFR8getdims returns information about the first image in the file. If an image has already been read, DFR8getdims finds the next image. Thus, images are read in the same order in which they were written to the file. Normally, DFR8getdims is called before DFR8getimage So that if necessary, space allocations for the image and palette can be checked, and the dimensions can be verified. If this information is already known, DFR8getdims need not be called. DFR8getimage FORTRAN: INTEGER FUNCTION d8gimg(filename, image, bufwidth, bufheight, palette) CHARACTER*(*) filename - name of file with RIS8 image INTEGER bufwidth, bufheight; - dimensions of the buffer allocated to store image CHARACTER*1 image(bufwidth,bufheight) - array that will hold image CHARACTER*1 palette(768) - palette to go with image C: int DFR8getimage(filename, image, bufwidth, bufheight, palette) char *filename; /* name of file with RIS8 image */ int32 bufwidth, bufheight; /* dimensions of the buffer allocated to store image */ uint8 image(bufheight](bufwidth); /* array that will hold image */ uint8 palette[768); /* palette to go with image */ Purpose: To retrieve the image and its palette, if it is present, and store them in the specified arrays. Returns: 0 on success; -1 on failure. If palette is NULL, no palette is loaded, even if there is one stored with the image. If the image in the file is compreSSed, DFR8getimage automatically decompresses it. If DFR8getdims has not been called, DFR8getimage finds the next image in the same way that DFR8getdims does. NOTE: The variables bufwidth and bufheight give the number of columns and rows, respectively, in the array which you've allocated in memory to store the image. The image may actually be smaller than the allocated space. NOTE: The order in which you declare dimensions is different between C and FORTRAN. Ordering varies because FORTRAN arrays are stored in column-major order, while C arrays are stored in row-major order. (Row-major order implies that the horizontal coordinate varies fastest). When d8gimg reads an image from a file, it assumes row-major order. The FORTRAN declaration that causes an image to be stored in this way must have the width as its first dimension and the height as its second dimension. To take this .into account as you read image in your program, you need to read in the image "on its side." DFR8readref FORTRAN: INTEGER FUNCTION d8rref(name, ref) character*(*) name - name of file containing image integer ref - reference number for next d8gimg C: int DFR8readref(filename, ref) char *filename; /* file containing image */ uint16 ref; /* reference number for next DFR8getimage */ Purpose: To specify the reference number of the image to be read when DFR8getimage is next called. Returns: 0 on success; -1 on failure. This routine is most likely to be used in conjunction with DFANgetlablist1, which returns a list of labels for a given tag together with their reference numbers. It provides, in a sense, a random access to images. 1This routine is discussed in the chapter "Annotating Data Objects and Files." NOTE: There is no guarantee that reference numbers appear in sequence in an HDF file; therefore, it is not safe to assume that a reference number is the sequence number for an image. DFR8restart FORTRAN: INTEGER FUNCTION d8first() C: int DFR8restart() Purpose: To cause the next get to read from the first RIS8 in the file, rather than the RIS8 following the one that was most recently read. Returns: 0 on success; -1 on failure DFR8nimages (filename) FORTRAN: INTEGER FUNCTION d8nims(filename) CHARACTER*(*) filename - name of HDF file C: int DFR8nimages(filename) char *filename; /* name of HDF file */ Purpose: To count the number of 8-bit raster images contained in an HDF file. Returns: Number of images on success; -1 on failure. DFR8lastref FORTRAN: INTEGER FUNCTION d8lref() C: uint16 DFR8lastref() Purpose: To get last reference number written or read for an RIS8. Returns: Reference number on success; -1 on failure. This routine is primarily used for annotations. See Chapter 5, "Annotating. Data Objects and Files," for examples. Example: FORTRAN Program to Read and Write a Raster Image Set Figure 2.4 shows a FORTRAN program that reads in an image from a file called old.hdf into an array called image. It is known that a palette exists. The program also writes out the image to a file called new.hdf. Figure 2.4 Reading an RIS8: Dimensions and Presence of the Palette Known (FORTRAN) FORTRAN: program test_getimage C Program to illustrate use of d8gdims and d8gimg C C****| | *********************************************** INTEGER d8gdims, d8gimg, d8aimg INTEGER ispal, ret, width, height CHARACTER*1 image(200,150), pal(768) C****| |*********************** read in image *********** ret = d8gdims(`old.hdf', width, height, ispal) if (width.eq.200) and. (height.eq.150) ) then ret = d8gimg(`old.hdf',image,width,height,pal) else print *, `Wrong dimensions. Program aborted.' stop endif 0 C****| |***** write same image to different file ******* ret = d8aimg(`new.hdf',image, width,height, 0) stop end Remarks: The raster image stored in the file is known to be 200 bytes wide and 150 bytes high. Because of the storage order used by FORTRAN, the program is loading the image "on its side." Hence, the declaration "CHARACTER*1 image(200, 150)" rather than "CHARACTER*1 image(150, 200)." Example: C Program to Read and Write a Raster Image Set Figure 2.5 shows a C program that reads in an image from a file called old.hdf into an array called image. It is known that a palette exists. The program also writes out the image to a file called new.hdf. Figure 2.5 Reading an RIS8: Dimensions Palette Known C: /* ** Program to illustrate use of DFR8getdims and DFR8getimage */ #include "hdf.h" #define HEIGHT 150 #define WIDTH 200 main () { int DFR8getdims(), DFR8getimage(), DFR8putimage(); int32 width, height; intn ispal, ret; char image[HEIGHT] [WIDTH], pal[768]; /********************** read in image *********************/ DFR8getdims("old.hdf", &width, &height, &ispal); if ( (width==WIDTH) && (height==HEIGHT) ) ) { DFR8getimage("old.hdf",image,width,height,pal); } else { printf("Wrong dimensions. Program aborted."); exit(1); } /*************** write same image to different file ************/ DFR8addimage("new.hdf", image, width, height, 0); } Example: FORTRAN Program to Read in a Raster Image Set, where Allocated Space is Larger than Image Figure 2.6 shows a FORTRAN program that reads in an image whose size is different from the size of the space allocated to hold the image. It then moves the image to a array that is of the correct size, and outputs the new image to a new HDF file. Figure 2.6 Reading an RIS8: Dimensions Different from Allocated Space FORTRAN: program big_buffer C Program using d8gimg when buffers are larger than image C C****| |********************************************************* integer d8gdims, d8gimg integer width, height, ispal, ret character*1 image(500,400) character*1 newimage(200,150), pal(768) C****| |****** read in image into "too-large" array ******** ret = d8gdims (`old.hdf', width, height, ispal) ret = d8gimg (`old.hdf', image,500, 400, pal) print *, `width=', width,' height=', height C****| |*** copy image to an array that is the "right size" *** do 100 i=1,200 do 100 j=1,150 newimage(i,j) = image(i,j) 100 continue C****| |*** write newimage new file--same as original image *** ret = d8pimg(`new.hdf',newimage,200,150,0) stop end Remarks: * The RIS8, stored in a file called, old.hdf, is read into an array called image. This array is deliberately made larger than the expected image. Figure 2.7 shows how the image fits inside the buffer. Note that since FORTRAN is used, the image is stored "on its side." * The array newimage is an array that is exactly the size of the image. After it is written to new.hdf, you can view the image in that file and see that it is identical to the original image. * Although it is possible, as this example illustrates, to read an image into a buffer that is larger than the image, the reverse is not possible. The RIS8 interface only supports writing an image from an array that was allocated to be exactly the same size of the image. Figure 2.7 FORTRAN Image Stored in Oversized Buffer *** INSERT FIGURE HERE *** 24-Bit Raster Image Sets The phrase 24-bit raster image set (RIS24) refers to the set of tags and associated information required to store a 24-bit raster image in an HDF file. An RIS24 contains at least the following components: * An image--here, a two-dimensional array of 24-bit pixel representations, where each 24-bit pixel value has three 8-bit components: one each for the red, green, and blue (RGB) values of the pixel color. These RGB values may be arranged in the file in one of three different ways (see the following section, "Interlace Schemes"). * An interlace scheme--a code that describes the order in which the pixel components are physically stored in the file (see the following section, "Interlace Schemes"). * Dimensions-two values that represent the x and y dimensions of the image, respectively. Interlace Schemes An interLace scheme describes the way an image is stored in a file or in memory. NCSA HDF supports different interlace schemes because graphics applications and devices vary in the way they organize graphics images. By storing an image in a file using a scheme that is consistent with the expected application or device, you can achieve substantial improvements in performance. The value of the integer argument il determines which scheme is to be used, as shown in Table 2.3. The interlace schemes are described in the following sections. Table 2.3 Interlace Scheme Codes Value of il Interlace Scheme 0 pixel 1 scan-line 2 scan-plane Pixel Interlace Scheme The default interlace scheme describes an image pixel-by-pixel. This scheme is called pixel interlace. The code to specify the pixel interlace scheme in an RIS24 is 0 (zero). For example, by default, NCSA HDF assumes that a 100 x 200 image with three components (R, G, and B) is stored as an array of size 100 x 200 x 3, and that each element of this array is exactly one byte in size and contains an R, G, or B value. Specifically, an interlace code of 0 indicates that the bytes that describe the image are stored in the following order in the file or memory: R, G, and B values are stored, in that order, in the first three bytes of the array, corresponding to the first pixel in the first row of the image. R, G, and B values are stored in the second three bytes of the array, corresponding to the second pixel in the first row of the image. And so forth, until the RGB values for the 100 pixels of the first row of the image are stored. This process is repeated until the RGB values for each pixel in the 200 lines of the image are stored. Scan-Line Interlace Scheme The scan-line interlace scheme describes an image line-by-line. The code to specify the scan-line interlace scheme is 1. For example, an interlace scheme code of 1 for a 100 x 200 image with three components (R, G, and B) informs NCSA HDF to assume that the image is stored as an array of size 100 x 3 x 200. Specifically, an interlace code of 1 indicates that the bytes that describe the image are stored in the following order in the file or memory: 100 R values are stored consecutively in the first 100 bytes of the array for each of the pixels in the first line of the image, then 100 G values are stored in the second 100 bytes of the array for each of the pixels in the first line of the image, then 100 B values are stored in the third 100 bytes of the array for each of the pixels in the first line of the image, and so forth, until the RGB values for each of the 200 lines of pixels in the image are stored. Scan-Plane Interlace Scheme The scan-plane interlace scheme describes an image color component-by-color component. The code to specify the scanplane interlace scheme is 2. For example, an interlace scheme code of 2 for a 100 x 200 image with three components (R, 0, and B) informs NCSA HDF to assume that the image is stored as an array of size 3 x 100 x 200. Specifically, an interlace code of 2 indicates that the bytes that describe the image are stored in the following order in the file or memory: R values are stored in the first 100 x 200 bytes of the array for each of the pixels in the image; the G values for the image are stored in the second 100 x 200 bytes; and the B values in the third. Figure 2.8 illustrates how an RIS24 -- stored using the scan-plane interlace -- looks in an HDF file. Figure 2.8 Scan-Plane Interlace *** INSERT FIGURE HERE *** Compression Schemes As of this writing, image compression has not been implemented for 24-bit images in HDF. However, there are plans to implement a routine to cause 24-bit images to be stored in compressed mode. Writing 24-Bit Raster Images to a File DF24setil FORTRAN: INTEGER FUNCTION d2setil(il) INTEGER il; - interlace of image C: int DF24setil(il) int ii; /* interlace of image */ Purpose: To set interlace scheme to be used on subsequent writes. Returns: 0 on success; -1 on failure. If DF24setil is not called, the interlace code is assumed to be 0. Interlace codes: 0 = pixel interlacing; 1 = scan-line interlacing; 2 = scan-plane interlacing. DF24addimage FORTRAN: INTEGER FUNCTION d2aimg(name, image, width, height) CHARACTER*(*) name - name of HDF file CHARACTER*(*) image - array holding image to add to file INTEGER width, height - dimensions of array image C: int DF24addimage(filename, image, width, height) char *filename; /* name of HDF file */ void *image; /* pointer to the array holding image to add to file */ int32 width, height; /* dimensions of array image */ Purpose: To write out a 24-bit image. Returns: 0 on success; -1 on failure. Array image is assumed to be width x height x 3 bytes. NOTE: The order in which you declare dimensions is different between C and FORTRAN. Ordering varies because FORTRAN arrays are stored in column-major order, while C arrays are stored in row-major order. (Row-major order implies that the last coordinate varies fastest). When DF24addimage writes an image to a file, it assumes row-major order. The FORTRAN declaration that causes an image to be stored in this way must have the width as its first dimension and the height as its second dimension. To take this into account as you build your image in your FORTRAN program, you need to build the image "on its side." Hence, in FORTRAN programs, array image should be declared as: image(3, width, height) for interlace=0 image(width, 3, height) for interlace=1 image(width, height, 3) for interlace=2 In C programs, array image should be declared as: image[height][width] [3] for interlace=0 image[height][3)[width] for interlace=1 image[3][height)[width] for interlace=2 DF24putimage FORTRAN: INTEGER FUNCTION d2pimg(name, image, width, height) CHARACTER*(*) name - name of HDF file CHARACTER*(*) image - array holding image to add to file INTEGER width, height - dimensions of array image C: int DF24putimage(filename, image, width, height) char *filename; /* name of HDF file */ void *image; /* pointer to array for image to add to file */ int32 width, height; /* dimensions of array image */ Purpose: To write out a 24-bit image as the first image in the file, overwriting any information that was previously in the file.. Returns: 0 on success; -1 on failure. Array image is assumed to be width x height x 3 bytes. See DF24addimage for how to declare array image in FORTRAN and C programs. NOTE: DF24putimage overwrites any information that exists in the HDF file. To write an image to a file by appending it, rather than overwriting it, use DF24addimage (see above) Example: Writing a RIS24 with the Default Pixel Interlace The C code in Figure 2.9 demonstrates how the default pixel interlace is used when writing the 400 x 600 image stored in array picture in a file in RIS24 format. Figure 2.9 Storing an RIS24 Using Pixel Interlace C: char picture[400] [600] [3]; int ret; ret = DF24addimage("herfile.hdf",picture,600,400); if (ret != 0) printf("Error writing image to myfile.hdf."); ... Example: Writing Several 24-bit Images Figure 2.10 shows a series of C calls in which four 800 x 1200 images are written to the same file. The first two calls use the default interlace scheme; the second two calls use scan-line interlace. Figure 2.10 Storing Multiple RIS24s in a Single File C: int DF24addimage; char pic1[800] [1200] [3], pic2 [800] [1200] [3]; char pic3[800] [3] [1200], pic4 [800] [3] [1200]; DF24addimage("myfile",pic1,1200,800); DF24addimage("myfile",pic2,1200,800); DF24setil (1); DF24addimage("myfile",pic3,1200,800); DF24addimage("myfile",pic4,1200,800); Reading 24-Bit Raster Images from a File The two routines, DF24getdims and DF24getimage, are sufficient to read raster images from a file. If enough is known about the images and interlacing, only the latter routine is needed. DF24getdims FORTRAN: INTEGER FUNCTION d2gdims(name, width, height, il) CHARACTER*(*) name - name of HDF file INTEGER width, height - for returning dimensions INTEGER il - for returning interlace of image in file C: int DF24getdims(filename, pwidth, pheight, pil) char *filename; /* name of HDF file */ int32 *pwidth, *pheight; /* for returning dimensions */ int *pil; /* for returning interlace of image in file */ Purpose: To get dimensions and interlace storage scheme of next image RIS. Returns: 0 on success; -1 on failure. If the file is being opened for the first time, DF24getdims returns information about the first image in the file. If an image has already been read, DF24getdims finds the next image. Thus, images are read in the same order in which they were written to the file. If you know the dimensions of the image beforehand, there is no need to call DF24getdims. Simply allocate arrays with the proper dimensions for the image and let DF24getimage read in the images. If, however, you do not know the values of width and height, you must call DF24getdims to get them and then use them to determine the right amount of space needed for the array image. Successive additional calls to DF24getdims and DF24getimage, respectively, retrieve all of the images in the file in the sequence in which they were written. Interlace codes: 0 = pixel interlacing; 1 = scan-line interlacing; 2 = scan-plane interlacing. DF24getimage FORTRAN: INTEGER FUNCTION d2gimg(name, image, width, height) CHARACTER*(*) name - name of HDF file CHARACTER*(*) image - pointer to space to return image INTEGER width, height - dimensions of space to return image C: int DF24getimage(filename, image, width, height) char *filename; /* name of HDF file */ void *image; /* pointer to space to return image */ int32 width, height; /* dimensions of space to return image */ Purpose: To get image from next 24-bit RIS. Returns: 0 on success; -1 on failure. If DFR24getdims has not been called, DFR24getimage finds the next image in the same way that DFR24getdims does. The amount of space allocated for the image should be width x height x 3 bytes. To specify that the next call to DF24getimage should read the raster image from the RIS24 using a particular interlace, rather than the interlace used to store the image in the file, make a call to DF24reqil (see below). DF24readref FORTRAN: INTEGER FUNCTION d2rref(name, ref) character*(*) name - name of file containing image integer ref - reference number for next d2gimg C: int DF24readref(filename, ref) char *filename; /* file containing image */ uint16 ref; /* reference number for next DF24getimage */ Purpose: To specify the reference number of the image to be read when DF24getimage is next called. Returns: 0 on success; -1 on failure. You will most likely use this routine in conjunction with DFANgetlablist1, which returns a list of labels for a given tag together with their reference numbers. It provides, in a sense, a random access to images. NOTE: There is no guarantee that reference numbers appear in sequence in an HDF file; therefore, it is not safe to assume that a reference number is the sequence number for an image. DF24reqil FORTRAN: INTEGER FUNCTION d2reqil(il) INTEGER il - interlace to get next image with C: int DF24reqil(il) int il; /* interlace to get next image with */ Purpose: To cause next DF24getimage to store image in memory with the specified interlace. Returns: 0 on success; -1 on failure. Regardless of what interlace scheme is used to store the image, DF24reqil causes the image to be loaded into memory and be interlaced according to the specification of il. NOTE: Since a call to DF24reqil may require a substantial reordering of the data, I/O performance could be adversely affected; e.g. it could result in much slower I/O performance than would be achieved if no change in interlace were requested. Interlace codes: 0 = pixel interlacing; 1 = scan-line interlacing; 2 = scan-plane interlacing. DF24restart() FORTRAN: INTEGER FUNCTION d2first() C: int DF24restart() Purpose: To cause the next get to read from the first RIS24 in the file, rather than the RIS24 following the one that was most recently read. Returns: 0 on success; -1 on failure 1This routine is discussed in the chapter "Annotating Data Objects and Files." DF24lastref FORTRAN: uint16 yet implemented in FORTRAN) C: uint16 DF24lastref() Purpose: To get last reference number written or read for an RIS24. Returns: Reference number on success; -1 on failure. This routine is primarily used for annotations. See Chapter 5, "Annotating Data Objects and Files," for examples. Examples Involving 24-bit Raster Image Sets Example: Reading in a 24-bit Image Figure 2.11 shows a C call that reads in an image when the dimensions and interlace are already known and scan-plane interface is being used.. Figure 2.11 Reading an RIS24: Dimensions and Interlace Known C: char image[3] [256] [512]; DF24getimage("myfile.hdf",image,512,256); Example: Read an Image, Dimensions and Interlace Not Known Figure 2.12 illustrates a set of C calls that read in an image, where the dimensions of the image and interlace scheme are not known ahead of time. The data is stored in the array image as if the array were three planes of size width x height. Since no explicit declaration is given for image, it is the responsibility of the program to compute offsets in the array that correspond to particular elements. Figure 2.12 Reading an RIS24: Dimensions and Interlace Not Known C: #include "hdf.h" ... int32 width, height; intn il; char *image; /* pointer to space to return image */ DF24getdims("myfile.hdf",&width,&height,&il); DF24reqil(2); image = (char *) malloc(3*width*height); DF24getimage("myfile.hdf",image,width,height); Sample Programs A C Program to Convert a Raw Palette and Raw Raster Image to HDF RIS8 Format The example in Figure 2.13 shows a complete program for processing RIS8 data. Several features of HDF image storage are illustrated here. The program does the following, in order: * Reads into image1 a 256 x 512 8-bit raster image. * Reads into each of red, green, and blue 256 values representing the palette from a (non-HDF) file called ps.pal * Writes the palette and image as a run-length encoded raster image set to a file called testrig1.hdf * Reads the palette and image back in. * Compares the contents of image2 to the Contents of image2 to determine whether they are identical, as they should be Figure 2.13 C Program Dealing with Raster Image Sets main { char image1[131072], /* raw image to be read in then */ /* put into an RIS8 in testrig.df */ image2[131072], /* HDF image to be read from testrig.df */ palette[768], reds[256], greens[256], /* colors to load into palette */ blues[256], *p; /* pointer to palette */ int j, width, height, ispal; /* to tell if there is a palette */ FILE *fp; fp = fopen("denaa031","r"); /* read in raw 256x512 image fread(image1, 256, 512, fp); fclose(fp); fp = fopen("ps.pal","r"); /* read RGB values from palette file */ fread(reds,1,256,fp); fread(greens,1,256,fp); fread(blues,1,256,fp); fclose(fp); p = palette; /* reorganize palette so that */ for (j=0; j<256; j++) { /* RGB values are interleaved */ *p++ = reds[j]; *p++ = greens[j]; *p++ = blues[j]; } /* write out image with palette DFR8setpalette(palette); DFR8putimage("testrig1.df",image1,256,512,DFTAG_RLE); /* read in image with palette */ DFR8getdims("testrig1.df",&width, &height, &ispal); DFR8getimage("testrig1.df",image2,width,height,palette); if (memcmp(image1, image2, 131072) ==0) /* compare the images */ printf("identical\n"); else printf("different\n"); C Functions to Convert Floating-Point Data to 8-Bit Raster Data The function floattor8 shown in Figure 2.14 converts a floating-point data array into an 8-bit raster array. Once converted, this raw raster array is can be stored in RIS8 format. Figure 2.14 Converting Floating-Point Data to RIS8 C: /* * floattor8.c */ #define CHAR_MAX 255 /* * floattoR8 * Convert a data array into a raster array by dividing the * range in the data into 256 regions, numbering the regions * from zero to 255, and assigning to each position in the * raster array the number of the corresponding region. */ floattoR8(data, raster, size, max, min) float data[]; char raster[]; int size; float max, min; { int32 i; float32 step; if ((max == 0) && (min == 0)) findMaxMin(data, size, &max, &min); step = CHAR_MAX / (max - min); if (step == 0) return(-1); for(i=0;i data[i]) *min = data[i); } } Chapter 3 Storing Palettes Chapter Overview HDF 8-Bit Palettes Writing Palettes to a File Reading Palettes from a File Other Palette Routines Chapter Overview This chapter describes the routines that are available for storing and retrieving 8-bit palettes. HDF 8-Bit Palettes An 8-bit palette is a lookup table with 256 entries that tell the system hardware which color to associate with each of the 256 possible pixel values. Each entry in the palette is chosen from a master palette of 224 RGB colors. In HDF files, 8-bit palettes are assumed to be organized as follows. Each palette entry consists of 3 bytes-one each for red, green, and blue. The first three bytes represent the R, G, and B values of the first color in the palette; the next three the R, G, and B values of the second color; and so forth. The total size of a palette is 768 bytes. The HDF library contains routines for storing and retrieving palettes. These routines are callable from C and FORTRAN programs that have access to the library. All of the callable palette routines in the library begin with the letters DFP. The functions, DFPaddpal and DFPgetpal, are the primary routines for palette. I/O and should be sufficient for most palette I/O operations. The other six palette functions--DFPputpal, DFPnpals, DFPwriteref, DFPreadref, DFPrestart, and DFPlastref--provide greater control of the I/O process and are available to you if more control is needed. Table 2.1 lists the long and short names and the functions of the palette routines currently contained in the HDF library. The following sections provide descriptions and examples of these calling routines. Table 3.1 Palette I/0 Routines in the HDF Library C FORTRAN Name Name Function DFPaddpal dpapal appends a palette to a file. DFPgetpal dpgpal reads in the next palette in the file. DFPputpal dpppal writes a palette to a file. DFPnpals dpnpals indicates number of palettes in a file. DFPwriteref dpwref sets the reference number of the next palette to be written. DFProadref dprref sets the reference number of the next palette to be retrieved. DirPrestart dpreat specifies that the next call to DFPgetpal reads the first palette in the file, rather than the next. DFPlastref dplref returns the value of the reference number most recently read or written. Writing Palettes to a File DFPaddpal FORTRAN: INTEGER FUNCTION dpapal(filename, pal) CHARACTER*(*) filename - name of HDF file CHARACTER*(*) pal - 768-byte space with palette C: int DFPaddpal(filename, palette) char *filename; /* name of HDF file */ void *palette; /* 768-byte space with palette */ Purpose: To append the palette stored in the array palette to an HDF file. Returns: 0 on success; -1 on failure. If file does not exist, it is created and the palette written to it. DFPaddpal is often sufficient for adding the palette that you want to an HDF file. Other palette routines, which provide more refined access to palettes, are described next. Example: Writing a Palette to a File with DFPaddpal Figure 3.1 shows a C code segment for writing a palette to an HDF file. Figure 3.1 Writing a Palette to a File C : char pal [768]; DFPaddpal("myfile.hdf",pal); ... DFPputpal FORTRAN: INTEGER FUNCTION dpppal(filename, pal, ow, filemode) CHARACTER*(*) filename - name of HDF file CHARACTER*(*) pal - 768-byte space for palette INTEGER ow - if 1, overwrite last palette read or written if 0, write it as a fresh palette INTEGER filemode - if "a", append palette to file */ - if "w", create new file C: int DFPputpal (filename, palette, overwrite, filemode) char *filename; /* name of HDF file */ void *palette; /* 768-byte space for palette */ int overwrite; /* if 1, overwrite last palette read or written */ /* if 0, write it as a fresh palette */ char *filemode; /* if "a", append palette to file */ /* if "w", create new file */ Purpose: To write a palette to file. Returns: 0 on success; -1 on failure. This routine provides more control than DFPaddpal. Note that the combination filemode="w" and overwrite=1 has no meaning and will generate an error. Reading Palettes from a File DFPgetpal FORTRAN: INTEGER FUNCTION dpgpal(filename, pal) CHARACTER*(*) filename - name of HDF file CHARACTER*(*) pal - 768-byte space for palette C: int DFPgetpal (filename, palette) char *filename; /* name of HDF file */ void *palette; /* 768-byte space for palette */ Purpose: To get the next palette from file and store it in the array palette. Returns: 0 on success; -1 on failure. The array palette is assumed to be allocated at least 768 bytes. Successive additional calls to DFPgetpal retrieve the palettes in the file in the sequence in which they are stored. DFPgetpal iS Often sufficient for getting the palette that you want from an HDF file. Other palette routines, which provide more refined access to palettes are described below. Example: Reading the First Available Palette Figure 3.2 shows a C code segment that reads the first available palette from an HDF file. Figure 3.2 Reading the First Available Palette C: char pal[768]; DFPgetpal("myfile.hdf",pal); Other Palette Routines DFPnpals FORTRAN: INTEGER FUNCTION dpnpals(filename) CHARACTER*(*) filename C: int DFPnpals(filename) char *filename; /* name of HDF file */ Purpose: To tell how many palettes are present in a file. Returns: Number of palettes on success; -1 on failure. DFPwriteref FORTRAN: INTEGER FUNCTION dpwref(filename, ref) CHARACTER*(*) filename - name of HDF file INTEGER ref - ref number to be used in next palette write C: int DFPwriteref(filename, ref) char *filename; /* name of HDF file */ uint16 ref; /* ref number to be used in next palette write */ Purpose: To set the reference number of the next palette to be written. Returns: 0 on success; -1 on failure. DFPreadref FORTRAN: INTEGER FUNCTION dprref(filename, ref) CHARACTER*(*) filename - name of HDF file INTEGER ref - ref number to be used in next read C: int DFPreadref(filename, ref) char *filename; /* name of HDF file */ uint16 ref; /* ref number to be used in next DFPgetpal */ Purpose: To set the reference number of the palette that DFPgetpal will retrieve next. Returns: 0 on success; -1 if a palette with this reference number does not exist or if an error occurs. DFPrestart FORTRAN: INTEGER FUNCTION dprest() C: int DFPrestart() Purpose: To cause the next call tO DFPgetpal to read the first palette in the file. Returns: 0 on success. DFPlastref FORTRAN:. INTEGER FUNCTION dplref() C: int DFPlastref() Purpose: To determine the value of the reference number most recently read or written by a palette function call. Returns: The reference number on success;, -1 on failure. Chapter 4 Storing Rectangular Gridded Arrays of Scientific Data Overview Scientific Datasets Reasons to Use Scientific Datasets Header File Writing Scientific Datasets to a File Writing Entire Arrays to an HDF File The "Set" Routines: Preparing to Write an SDS Writing an SDS to a File Writing Parts of an SDS Reading Scientific Datasets from a File Reading and Entire Array Getting Other Information about an SDS Reading Parts of an SDS Other SDS Routines How SDS Routines Store and Convert Scientific Data How HDF Normally Stores Arrays How HDF Normally Represents Numbers Backward Compatibility Sample Programs A FORTRAN Program A C Program Chapter Overview This chapter describes the routines that are available for storing and retrieving scientific datasets. Scientific Datasets A Scientific Data Set (SDS) is an HDF set that stores rectangular gridded arrays of data, together with information about the data. Specifically, a SDS is a set of tags and associated information about scientific data. Assuming that a user of scientific data often will want information about the data, an SDS might contain the following information. (The first three items are required in every SDS; the rest are optional.) * The actual data values. * The number of dimensions (rank) and the size of each dimension * The number type of the data. * Scales to be used along the different dimensions when interpreting or displaying the data * Labels for all dimensions (each label can be thought of as the name of an independent variable) and for the data (the dependent variable) * Units for all dimensions and for the data * Format specifications to be used when displaying values for the dimensions and for the data * A range, attributing maximum and minimum values for the data set. * Calibration information including an offset and scale factor. * A fill value for representing missing data in a data set. * The coordinate system to be used when interpreting or displaying the data Figure 4.1 shows a conceptual view of a sample scientific dataset. The actual 2D array of data is only one element in the set. Other elements include the number of dimensions (rank), the sizes of the dimensions, identifying information about the data and axes, and scales for the axes. Figure 4.1 HDF File with Scientific Dataset *** INSERT FIGURE HERE *** A file can contain many SDSS. It can also contain other HDF objects, such as raster image sets and annotations, together in the same file with SDSS. The HDF library provides an SDS interface with routines for storing and retrieving scientific data sets. This user interface lets you (a) build an SDS and (b) extract data from an SDS. These routines can be called from C and FORTRAN programs that have access to the library. All routines are functions of type integer. Table 4.1 lists the C and FORTRAN names of SDS routines currently contained in the HDF library. The following sections provide descriptions and examples of these calling routines. Table 4.1 Scientific Dataset Routines in the HDF Library C FORTRAN Name Name Function DFSDadddata dsadata appends the data to the file, not overwriting other file contents. DFSDputdata dspdata writes the data to a new file, truncating old file if it exists. DFSDsetNT dssnt tells which number type is to be used for next DFSDadddata or DFSDputdata DFSDsetdims dssdims sets the rank and dimension sizes for succeeding SDSs DFSDclear dsclear clears all possible set values. DFSDsetdimstrs dssdist sets label, unit, and format specifications for a dimension and its scale. DFSDsetdimscale dssdisc sets the scale for a dimension. DFSDsetdatastrs dissdast sets label, unit, and format specifications for the data. DFSDsetlengths dsslens sets maximum lengths for strings that will hold labels, units, formats, and the name of the coordinate system. DFSDsetrange dssrang sets maximum and minimum data values. DFSDsetcal dsscal sets calibration information associated with data DFSDstartslice dssslc prepares system to write part of a dataset to a file. DFSDputslice dspslc writes part of a dataset to a file. DFSDendslice dseslc indicates write completion for all parts of a dataset. DFSDgetdata dsgdata reads the next dataset in the file. DFSDgetNT dsgnt gets the number type of data that will be read with DFSDgetdata. DFSDgetdims dsgdims gets the number of dimensions of the dataset and the sizes of the dimensions for the next SDS in the file. DFSDgetdimstrs dsgdist reads the label, unit, and format for a dimension and its scale. DFSDgetdimscale dsgdisc reads the scale for a dimension. DFSDgetdatastrs dsgdast reads the label, unit, and format specification for the data. DFSDgetrange dsgrang reads the maximum and minimum values. DFSDgetcal dsgcal gets calibration information associated with data DFSDreadref dsrref sets the reference number of the SDS to get next DFSDrestart dsfirst sets the next get command to read from the first SDS in the file, rather than the next. DFSDgetslice dsgslc reads part of a dataset. DFSDnumber returns the number of SDSs in the file. DFSDlastref dslref returns the value of the last reference number of an SDS read from or written to the file. DFSDpre32sdg dsp32sd tests whether the SDS with a given ref was created by an HDF library that precedes HDF3.2. Header File The header file dfsd.h contains the declarations and definitions that are used by the routines listed here. This file can, if needed, be included with your C source code. Writing Scientific Datasets to a File SDS information is written to a file in two steps. The first involves execution of a series of "set" calls, which put information about the actual data into a special structure in primary memory. To set information associated with an SDS, you usually first invoke DFSDsetdims, then execute whatever set routines you need. If you do not wish to specify a certain item, you need not invoke its corresponding set call. The second phase involves actually writing the data to a file, along with the information that has been set. You can write data arrays to an HDF file in two basic ways: 1. by writing out the entire array with a single call (DFSDadddata or DFSDputdata), and 2. by writing the array in smaller pieces, or "slices." Method #1 is covered in the section "Writing Entire Arrays to an HDF file. Method #2 is covered in the section "Writing Parts of a Scientific Dataset". In general. you perform these same two steps for each dataset you want to write to your file. However, it is not usually necessary to perform all set calls for every dataset you wish to write out. For example, if the rank and dimensions of all datasets are exactly the same, you only have to call the routine DFSDsetdims before writing out the first set. Thereafter, you need only call DFSDputdata/DFSDadddata to write out the different data sets. The HDF software remembers the rank and dimension values and associates them with all subsequent data arrays that are written to the same file, unless you change them. In other words, once an item has been set, it does not normally go away even after a DFSDputdata Or DFSDadddata call. The associated information is only cleared if the new dataset has different rank or dimensions, or if DFSDclear is called. (The only exception to this are that the values Set by DFSDsetrange and DFSDsetcal are cleared after they are written to a file.) A note about how to describe the dimensions of array: Most current Fortran compilers read/write data into/from an array in column major order. (See the section "How SDS Routines Store and Convert Scientific Data" for details.) For a given data stream, if the order of the dimensions of the Fortran array is the reverse of that of the C array, then the HDF files written by C or Fortran programs will be the same. All of the Fortran examples in this chapter use arrays with reversed dimensions from their counterparts in C. For instance, if a 3D Fortran array is declared as fdata(500, 800, 10), the corresponding C array would be declared as cdata[10] [800] [500]. The data can be imagined as 10 planes, each plane having 800 lines and each line having 500 points. We would denote this array as a 10x800x500 array. The size of the first dimension of fdata is 500 while that of cdata is 10. The size of the third dimension of .fdata is 10 while cdata is 500. The same interpretation applies for 2D arrays. Writing Entire Arrays to an HDF File DFSDadddata FORTRAN: INTEGER FUNCTION dsadata(filename, rank, dimsizes, data) CHARACTER*(*) filename - name of file to store SDS in INTEGER rank - number of dimensions of data array to be stored INTEGER dimsizes(rank) - array that holds sizes of dimensions of the data array data(*) array holding data to be stored C: int DFSDadddata(filename, rank, dimsizes, data) char *filename; /* name of file to store SDS in */ int rank; /* number of dimensions of data array to be stored */ int32 dimsizes[); /* array that holds sizes of dimensions of the data array */ *data; /* array holding data to be stored Purpose: To add to an HDF file the data in the multidimensional arraydata, as well as all other information that has previously been set (see "Set Routines" below.) Returns: 0 on success; -1 on failure. The argument rank gives the number of dimensions of the array data. The array dimsizes contains the sizes of the dimensions. The length of the array dimsizes is rank. The array "data" can be of any valid type (see discussion of DFSDsetNT below). If no number type has been set by DFSDsetNT, it is assumed that the data is.of type float32. The invocation of DFSDadddata triggers the writing of the entire SDS. That is, when DFSDadddata is called, all information that has been set by the "DFSDset..." calls (covered below) is written to the file, along with the data array itself. Example: Writing an Array as a Scientific Dataset. This example shows a call that stores a 5x20x5000 array of type float32s in an SDS. The SDS is to be stored in a file called "myfile.hdf", with no labels, scales, or other information. FORTRAN: INTEGER dsadata REAL points(5000,20,5) INTEGER dims(3), ret dims(1) = 5000 dims(2) = 20 dims(3) = 5 ret = dsadata(`myfile.hdf',3, dims, points) C: #include ".hdf.h" ... float32 points[5][20][5000]; int dims[3]; ... dims(0) = 5; dims(1) = 20; dims(2) = 5000; DFSDadddata("myfile.hdf",3, dims, points); DFSDputdata FORTRAN: INTEGER FUNCTION dspdata(filename, rank, dimsizes, data) CHARACTER*(*) filename - name of file to store SDS in INTEGER rank - number of dimensions of data array to be stored INTEGER dimsizes(rank) - array that holds sizes of dimensions data(*) - array holding data to be stored C: int DFSDputdata(filename, rank, dimsizes, data) char *filename; /* name of file to store SDS in */ int rank; /* number of dimensions of data array to be stored */ int32 dimsizes[]; /* array that holds sizes of dimensions */ *data; /* array holding data to be stored */ Purpose: DFSDputdata does the same thing that DFSDadddata does, except that it overwrites the previous contents of the file, whereas DFSDadddata appends the scientific dataset to the file. Returns: 0 on success; -1 on failure. NOTE: DFSDputdata destroys whatever was in the HDF file before it was called. Use it with caution. The "Set" Routines: Preparing to Write an SDS DFSDsetNT FORTRAN: INTEGER FUNCTION dssnt (numbertype) INTEGER*4 numbertype - number type of data to be written C: int DFSDsetNT(numbertype) int32 numbertype; /* number type of data to be written*/ Purpose: To set the number type to be used for data to be written out by the next DFSDadddata or DFSDputdata.. Returns: 0 on success; -1 on failure. DFSDsetNT must be called if a number type other than float32 is to be stored. DFSDsetNT and DFSDsetdims can be called in any order, but they should be called before any other "DFSDset" functions and before DFSDputdata or DFSDadddata. Valid parameter values for DFSDsetNT (e.g. DFNT_INT8) are of the general form "DFNT_", all capital letters. The following table gives the number types currently supported. If you include the file hdf.h in your program, you can use the symbolic names of the number types. Number types that are available in only the C-interface are marked with and asterisk (*). symbolic type name value 32-bit float DFNT_FLOAT32 5 64-bit float DFNT_FLOAT64 6 8-bit signed integer DFNT_INT8 20 8-bit unsigned integer* DFNT_UINT8 21 16-bit signed integer DFNT_INT16 22 16-bit unsigned integer* DFNT_UINT16 23 32-bit signed integer DFNT_INT32 24 32-bit unsigned integer* DFNT-UINT32 25 For other information how HDF stored numbers, see the section "How HDF Stores Numbers in SDSS" elsewhere. Example. Assuming that DFNT_INT8 has been defined and i8data is an array with 8-bit integer data, the following code fragments write out 8-bit integers to an SDS. FORTRAN: iret = dssnt(DFNT_INT8) iret = dsadata(`myfile.hdf', rank, dims, i8data) C: DFSDsetNT(DFNT_INT8); DFSDadddata("myfile.hdf", rank, dims, i8data); DFSDsetdims FORTRAN: INTEGER FUNCTION dssdims(rank, dimsizes) INTEGER rank - number of dimensions INTEGER dimsizes(rank) - dimensions of the SDS C: int DFSDsetdims(rank, dimsizes) int rank; /* number of dimensions */ int32 dimsizes[]; /* dimensions of the SDS */ Purpose: To set the rank and dimension sizes for subsequent scientific datasets that are written to the file. Returns: 0 on success; -1 on failure. This routine must be called before calling any other set routines, exCept DFSDsetNT. DFSDsetdims need not be called if other set routines are not called and the correct dimensions are supplied in DFSDputdata Or DFSDadddata. If rank or dimension sizes change, all previous set calls are cleared. Examples of the use of DFSDsetdims can be found in connection with other "set" routines described below. DFSDclear FORTRAN: INTEGER FUNCTION dsclear() C: int DFSDclear() Purpose: To cause all possible 'set' values to be cleared. Returns: 0 on success; -1 on failure. After a call to DFSDclear, numbertype, rank, dimensions and other values that were set by the "set" calls will not be written unless they have been set again. DFSDsetdimstrs FORTRAN: INTEGER FUNCTION dssdist(dim, label, unit, format) INTEGER dim - dimension this label, unit and format refer to CHARACTER*(*) label - label that describes this dimension CHARACTER* (*) unit - unit to be used with this dimension CHARACTER*(*) format - format to be used in displaying scale for this dimension C: int DFSDsetdimstrs(dim, label, unit, format) int dim; /* dimension this label, unit and format refer to*/ char *label; /* label that describes this dimension*/ char *unit; /* unit to be used with this dimension*/ char *format; /* format to be used to display scale */ Purpose: To set the items corresponding to dimension dim that are to be stored as strings in the SDS, namely label, unit, and format. Returns: 0 on success; -1 on failure. Note: In both C and Fortran programs, dim=1 for the first dimension; dim=2, for the second dimension, and so forth. Example: Writing a Scientific Dataset with Dimension Information. In this example a 200x300 data array is written to a file called `myfile.hdf', together with label, unit, and format information about each dimension. FORTRAN: INTEGER dssdims, dssdist, dsadata REAL press1 (300,200) INTEGER dims(2), ret ... dims(1) = 300 dims(2) = 200 ret = dssdims (2, dims) ret = dssdist (1, `position', `cm', `F10.2') ret = dssdist (2, `height', `m', `F10.3') ret = dsadata (`myfile.hdf', 2, dims, press1) C: float32 press1 [200] [300]; int dims[2]; ... dims[0] = 200; dims[1] = 300; DFSDsetdims(2, dims); DFSDsetdimstrs(1,"height", "m", "F10.3"); DFSDsetdimstrs(2,"position", "cm", "F10.2"); DFSDadddata("myfile.hdf", 2, dims, press1) ... DFSDsetdimscale FORTRAN: INTEGER FUNCTION dssdims(dim, dimsize, scale) INTEGER dim - dimension this scale goes with INTEGER dimsize - size of scale scale(dimsize) - the scale C: int DFSDsetdimscale(dim, dimsize, scale) int dim; /* dimension this scale corresponds to */ int32 dimsize; /* size of scale */ void *scale; /* the scale */ Purpose: To set the scale corresponding to dimension dim by taking it from the array scale. Returns: 0 on success; -1 on failure. A scale is a ID array whose values describe reference points along one of the dimensions of the SDS data. For example, a 2D SDS representing points on a map could have two scales, one representing points of latitude, and the other points of longitude.. Note: In both C and Fortran programs, dim=1 for the first dimension; dim=2, for the second dimension, and so forth. Example: Writing a Scientific Dataset with Dimension Scales. In this example a 200 x 300 data array is written to a file called `myfile.hdf', together with scales for each dimension. It is assumed that the arrays xscale and yscale have been assigned values that define the corresponding scales. FORTRAN: INTEGER dssdims, dssdisc, dsadata REAL press1(300,200) INTEGER dims(2), ret REAL latscale(200), lngscale(300) dims(1) = 300 dims(2) = 200 ret = dssdims(2, dims) ret = dssdisc(1, dims(1), lngscale) ret = dssdisc(2, dims(2), latscale) ret = dsadata(`myfile.hdf', 2, dims, press1) C: float32 press1[200][300]; float32 latscale[200], lngscale[300]; int dims[2]; ... dims[0] = 200; dims[1] = 300; DFSDsetdims(2, dims); DFSDsetdimscale(l, dims[0], latscale); DFSDsetdimscale(2, dims[1], lngscale); DFSDadddata("myfile.hdf", 2, dims, press1); DFSDsetdatastrs FORTRAN: INTEGER FUNCTION dssdast(label,unit,format,coordsys) CHARACTER*(*) label - label that describes the data CHARACTER*(*) unit - unit to be used with the data CHARACTER*(*) format - format to be used in displaying data CHARACTER*(*) coordsys - coordinate system C: int DFSDsetdatastrs(label, unit, format, coordsys) char *label; /* label that describes the data */ char *unit; /* unit to be used with the data */ char *format; /* format to be used in displaying the data */ char *coordsys; /* coordinate system */ Purpose: To set the items corresponding to the data that are to be stored as strings in the SDS, namely label, unit, format, and coordsys (coordinate system). Returns: 0 on success; -1 on failure. Note: HDF writes strings to an HDF file with a terminator appended at the end. The number of bytes in the HDF file is 1 byte more than the number of ASCII characters in the string. Example: Writing a Scientific Dataset with Data Information. In this example a 200 x 300 data array is written to a file called `myfile.hdf', together with label, unit, and format information about the data. In this example we assume that the coordsys parameter is of no interest to the user, so the empty string (` ') is given as the fourth argument to DFSDsetdatastrs. FORTRAN: INTEGER dssdims, dssdast, dsadata REAL press1(300,200) INTEGER dims(2), ret ... dims(1) = 300 dims(2) = 200 ret = dssdims(2, dims) ret = dssdast(`pressure 1', `Pascals', `E15.9,',") ret = dsadata(`myfile.hdf', 2, dims, press1); C: float32 press1[200] [300); int dims[21; ... dims[0] = 200; dims[1] = 300; DFSDsetdims(2, dims); DFSDsetdatastrs("Pressure 1","Pascals","E15.9",""); DFSDadddata("myfile.hdf", 2, dims, press1); DFSDsetlengths FORTRAN: INTEGER FUNCTION dsslens(maxlen_label, maxlen_unit,maxlen_format, maxlen_coordsys) INTEGER maxlen_label - max length of any label INTEGER maxlen_unit - max length of any unit INTEGER maxlen_format - max length of any format INTEGER maxlen_coordsys - max length of any coordsys C int DFSDsetlengths(maxlen_label, maxlen_unit,maxlen_format, maxlen_coordsys) int maxlen_label; /* max length of any label*/ int maxlen_unit; /* max length of any unit*/ int maxlen_format; /* max length of any format*/ int maxlen_coordsys; /* max length of any coordsys*/ Purpose: To set, optionally, the maximum lengths for the strings that will hold labels, units, formats, and the name of the coordinate system. Returns: 0 on success; -1 on failure. These lengths are used by the routines DFSDgetdimstrs and DFSDgetdatastrs to determine the maximum lengths of strings that they get from the file. Normally, DFSDsetlengths is not needed. If it is not called, default maximum lengths of 255 are used for all strings. DFSDsetrange1 FORTRAN: INTEGER FUNCTION dssrang(max, min) max - high value in range min - low value in range C: int DFSDsetrange(pmax, pmin) void *pmax; /* pointer to the high value in range */ void *pmin; /* pointer to the low value in range */ Purpose: To set maximum and minimum data values. Returns: 0 on success; -1 on failure. Since the max/min values are supposed to relate to the data itself, it is assumed that the type of max/min is the same as the type of the data. Note that in the C version of DFSDsetrange the arguments are pointers, rather than simple variables, whereas in the FORTRAN version they are simple variables of the same type as the data array. This routine does not compute the maximum and minimum values. It merely stores the values it is given. NOTE: When the maximum and minimum values are written to a file, the HDF element that holds these values is cleared, because it is assumed that subsequent datasets will have different values for max and min. 1The obsolete routines DFSDsetmaxmin and DFSDgetmaxmin assumed that all numbers were floating point. Now that the maximum and minimum values can be of different number types, it is necessary to redefine the parameters in the C versions of the routines from 'floats' to pointers. To avoid confusion, and possible undetected errors, these routines have been replaced by DFSDsetrange and DFSDgetrange. Example. In this example 16-bit data is written to an HDF file. Notice that max and min must be the same data type as the SDS array data. FORTRAN: integer*2 max, min, data(100,100) ... ret = dssdims(rank, dims) ret = dssrang(max, min) ret = dsadata(`myfile.hdf', rank, dims, data) C: int16 max, min, data[100] [100]; ... DFSDset dims (rank, dims); DFSDset range((void *)&max, (void *)&min); DFSDadddata("myfile.hdf", rank, dims, data); DFSDsetcal FORTRAN: INTEGER FUNCTION dsscal(cal, cal_err, ioff, ioff_err, cal_type) real*8 cal - calibration factor real*8 cal_err - calibration error (tolerance) real*8 ioff - uncalibrated offset real*8 ioff_err - uncalibrated offset error (tolerance) integer*4 cal-type - number type of uncalibrated data C: int DFSDsetcal(cal, cal_err, ioff, ioff_err, cal_type) float64 cal /* calibration factor */ float64 cal-err /* calibration error */ float64 ioff /* uncalibrated offset */ float64 ioff-err /* uncalibrated offset error */ int32 cal_type /* number type of uncalibrated data */ Purpose: To provide calibration information about the data in an SDS. Returns: 0 on success; -1 on failure. This routine attaches to the SDS a record containing four 64-bit floating point values followed by a 32-bit integer, to be interpreted as follows: cal calibration factor cal_err calibration error ioff uncalibrated offset ioff_err uncalibrated offset error cal_type numbertype of uncalibrated data The relationship between a value 'iy' stored in an SDS and the actual value `y' is defined as: y = cal.* (iy - ioff) The variable ioff_err contains a potential error of ioff, and cal_err contains a potential error of cal. Currently the calibration record is provided for information only. The SDS interface performs no operations on the data based on the calibration tag. DFSDsetcal works like other SDS 'set' routines, with one exception: calibration information is automatically cleared after call to DFSDputdata or DFSDadddata. Hence, DFSDsetcal must be called anew for each data set that is to be written. Example. In the following example a 100x100 array of 16-bit integers is stored in an SDS, together with calibration information indicating that the values in the array are intended to be converted to 32-bit floats, and each value should be multiplied by 10.0 then added to 16.75. FORTRAN: integer dsadata, dssnt, dsscal integer DFNT_FLOAT32, DFNT_INT16 real*8 cal, cale, ioff, ioffe integer*4 ctype integer*2 i16(100,100) DFNT_INT16 = 22 DFNT_FLOAT32 = 5 ... cal = 10.0 cale = 0.0 ioff = 16.35 ioffe = 0.8 ctype = DFNT_FLOAT32 C C Write array and calibration information to file C ret = dssnt(DFNT_INT16) ret = dsscal(cal, cale, ioff, ioffe, ctype) ret = dsadata(`of.hdf', rank, dims, i16) C: int16 i16[1001[1001; float64 cal, cale, ioff, ioffe; int32 calNT; ... cal = 10.0; cale = 0.0; ioff = 16.75; ioffe = 0.8; calNT = DFNT-FLOAT32; ... DFSDsetNT(DFNT_INT16); DFSDsetcal(cal, cale, ioff, ioffe, calNT); DFSDputdata("myfile.hdf", rank, dims, i16); Writing Parts of an SDS The routines DFSDstartslice, DFSDputslice, and DFSDendslice let you write an SDS in pieces, or slices. A slice is a rectilinear subarray of elements that can be stored in an SDS. In the current implementation of HDF (HDF 3.2) you are restricted in the order in which you write slices to an SDS: slices must be written in such a way that each slice is physically contiguous in the file to the slice that was written before it. For example, suppose you wish to write a 7 x 12 SDS in a series of slices as illustrated in Figure 4.2. You would write the slices in order, as follows: * The first slice to be written begins at the origin, writing all elements from (l,l) to (2,12). * The second slice extends from (3,I) to (6,12). * The third slice covers only part of row 7, from (7, 1) to (7,4). * The fourth slice covers the rest of row 7, filling out the array. Figure 4.2 A 7x12 SDS Divided into Slices of Varying Sizes *** INSERT FIGURE HERE *** Calling Sequence for the Slice-Writing Routines To store an array in slices, make calls to DFSDstartslice, DFSDputslice, and DFSDendslice in the following order: DFSDstartslice(filename) DFSDputslice(windims, data, dims) DFSDputslice(windims, data, dims) DFSDputslice(windims, data, dims) DFSDendslice() You must call DFSDstartslice before either of the other routines. Thereafter, DFSDputslice may be called many times to write several contiguous slices. DFSDendslice must be called to complete the write. Besides DFSDputslice, no other HDF routines may be called between the calls to DFSDstartslice and DFSDendslice. These three routines are discussed below in more detail. DFSDstartslice FORTRAN: INTEGER FUNCTION dssslc(filename) CHARACTER*(*) filename name of HDF file C: int DFSDstartslice(filename) char *filename; /* name of HDF file */ Purpose: To prepare the system to write a slice to a file. Returns: 0 on success; -1 on failure. Before calling DFSDstartslice, you must call DFSDsetdims to specify the dimensions of the dataset to be written to the file. DFSDstartslice always appends a new dataset to an existing file. Remember, you must call DFSDstartslice before calling DFSDputslice or DFSDendslice. DFSDputslice FORTRAN: INTEGER FUNCTION dspslc(windims, source, dims) INTEGER windims(*) - dimensions of slice source(*) - array containing slice INTEGER dims(*) - dimensions of array source C: int DFSDputslice(windims, source, dims) int32 windims[]; /* dimensions of slice*/ *source; /* array for storing slice*/ int32 dims[]; /* dimensions of array source*/ Purpose: To write a slice to an SDS. Returns: 0 on success; -1 on failure. DFSDputslice takes some part of an array in memory and stores it as part of the SDS array last specified by DFSDsetdims. Slices must be stored contiguously. Array windims ("window dimensions") specifies the size of the slice to be written. windims has as many elements as there are dimensions in the entire SDS array. source iS the array in memory containing the slice. dims is an array containing the dimensions of the array source. Notice that windims and dims need not be the same. Windims could refer to a subarray of source, in which case only a portion of source is written to the SDS array. See this chapter's section "Restrictions on Slices To Be Written" for restrictions on what may constitute a slice. DFSDendslice FORTRAN: INTEGER FUNCTION dseslc() C: int DFSDendslice() Purpose: To specify that the entire dataset has been written. Returns: 0 on success; -1 on failure. DFSDendslice must call after all the slices are written. It checks to ensure that the entire dataset has been written, and if it has not, returns an error code. Example: Writing Slices Suppose we want to create an 8x12 SDS array like the array in Fig. 4.3, and we want to write it using the slices shown in the figure. Suppose also that the arrays from which we get our data are 2x12 arrays in memory called source1, source2, source3 and source4. Fig. 4.4 contains sample C code to accomplish the write. Figure 4.3 Writing Slices form a four arrays to a 7x12 SDS *** INSERT FIGURE HERE *** Figure 4.4 Writing Slices from a four arrays to a 7x12 SDS /************************************************ * * Example C code: Write out slices of different sizes * from a four 2 x 12 arrays to one 8 x 12 * SDS. * ************************************************/ ... int rank; int SDSdims[2], sourcedims[2], windims[2); float source1 [2] [12]; float source2 [2] [12]; float source3 [2] [12]; float source4 [2] [12]; /* code that builds the array source goes here */ ... SDSdims[0]=8; SDSdims[1]=12; sourcedims[0]=2; sourcedims[1]=12; DFSDsetdims(2,SDSdims); /* write out scientific data set in slices */ DFSDstartslice("myfile.hdf"); windims[0]=2; windims[1]=12; /* { (1, 1) to (2, 12) } */ DFSDputslice (windims, source1, sourcedims); windims[0]=2; windims[1]=12; /* { (3,1) to (4,12) } */ DFSDputslice(windims, source2, sourcedims); windims[0]=2; windims[1]=12; /* { (5, 1) to (6, 4) } DFSDputslice(windims, source3, sourcedims); windims[01=2; windims[1]=12; /* { (7, 1) to (8, 12) } DFSDputslice(windims, source4, sourcedims); DFSDendslice(); ... Restrictions on Slices To Be Written It is important to recognize that you cannot write just any subarray from an SDS. The subarray must satisfy the restriction that when written to an SDS it must be written in a physically contiguous sequence in the file. This restriction must hold because the current implementation of HDF only allows data elements to be written in contiguous chunks. (Reading slices from SDSs is less restrictive.) For example, notice in the example in Figure 4.2 that the fourth slice extends from the middle of the tow to the end. The array windims iS (1,8). The slice does not, and cannot, extend to the eighth row. For example, windims could not be (2,7), for if it were, then there would have to be a pp between the places in the file where the two partial rows were stored, and this would violate the restriction that slices must be written in a physically contiguous chunk. So far we have assumed implicitly that the array is stored by row (row-major order), rather than by column. In C programs, SDS arrays are stored by row by default, but in FORTRAN programs, arrays are stored in column-major order. In that case, the roles of row and column are switched. If a slice starts in the middle of a column, then it may not spill over into the next column. As long as you write out data in chunks that encompass an entire dimension (a row or column at a time, a "slab" at a time, etc.), this restriction is unnecessary. It is when you have to write unequally-sized chunks that you have to worry about the restriction. For higher dimensional arrays, the same rule applies, although its implications may be more confusing. A more general statement of the restriction for higher dimensions can be described as follows: dimsizes = {n1, n2, ...ni, ... nrank) windims = {w1, w2, ...wi, ... wrank) startpos = {p1, p2, ...pi, ... prank) Case (a) Array elements are to be stored in row-major order (the default order; the last dimension varies fastest). If wi is the first element of windims that is greater than 1 but less than ni, then all previous elements of windims must be 1, and all succeeding elements of windims must be equal to their counterparts in dimsizes. That is, if there is an i such that 1i, wk=nk. In other words, in a multidimensional array with dimsizes={n1, n2, ... ni, ni+1, ...nrank) and 1i, wk=1 In other words, in a multidimensional array with dimsizes={n1, n2, ... ni, ni+1, ...nrank) and 1 data(*) - array for holding the data C: int DFSDgetdata(filename, rank, dimsizes, data) char *filename; /* name of file with SDS */ int rank; /* number of dimensions */ int32 dimsizes[]; /* array that holds dimensions of buffer that will hold the data */ data[]; /* array for holding the data */ Purpose: To get the dataset from the next SDS in the file and store it in the array data. Returns: 0 on success, -1 on failure. If you do not know the values of rank or dimsizes, you must call DFSDgetdims (described below) to get them and then use them to provide the right amount of space for the array data. If you do not know the number type of the data in an SDS, you can call DFSDgetNT (described below) to find it out. Each new call to DFSDgetdata (or to DFSDgetdims and DFSDgetdata) reads from the SDS that succeeds the last one read. For example, if DFSDgetdata is called three times in succession, the third call reads data from the third SDS in the file. Of course, if you do not know the values of rank or dimsizes, you must call DFSDgetdims to get them and then use them to provide the right amount of space for the array data. If DFSDgetdims or DFSDgetdata is called and there are no more scientific datasets left in the file, an error code is returned and nothing is read. DFSDrestart (see below) can be used to override this convention. Example: Reading in an SDS. The following is code to read in an array whose dimensions are known to be 100x200, and whose number type is known to be int16 (16-bit integer). FORTRAN: INTEGER dsgdata INTEGER*2 density(200, 100) INTEGER sizes(2), ret sizes(1) = 200 sizes(2) = 100 ret = dsgdata(`myfile.hdf', 2, sizes, density), ... C: unit16 density[100] [200] int32 sizes[2], ret; sizes[0] = 100; sizes[1] = 200; ret = DFSDgetdata("myfile.hdf", 2, sizes, density); ... Getting Other Information about an SDS DFSDgetNT FORTRAN: INTEGER FUNCTION dsgnt(numbertype) INTEGER*4 numbertype - number type of data in SDS C: int DFSDgetNT(pnumbertype) int32 *pnumbertype; /* number type of data in SDS */ Purpose: To get the number type of the current SDS. Returns: 0 on success; -1 on failure. Since DFSDgetNT gets the number type of the current data set, DFSDgetdims or DFSDgetdata should usually be called before calling DFSDgetNT. See the description of DFSDsetNT for a listing of the valid values for numbertype and their meanings. DFSDgetdims FORTRAN: INTEGER FUNCTION dsgdims(filename, rank, dimsizes, maxrank) CHARACTER*(*) filename - name of file with SDS INTEGER rank - number of dimensions INTEGER*4 dimsizes(maxrank) - array for holding dimensions of data set in the file INTEGER maxrank - size of array dimsizes C: int DFSDgetdims(filename, rank, dimsizes, maxrank) char *filename; /* name of file with SDS */ int *rank; /* number of dimensions */ int32 dimsizes[]; /* array for holding dimensions of data set in the file */ int maxrank; /* size of array dimsizes */ Purpose: To get the rank (number of dimensions) of the dataset and the dimsizes of each dimension in the next SDS in the file. Returns: 0 on success; -1 on failure. The input argument maxrank tells DFSDgetdims the size of the array that is allocated for storing the array of dimension sizes. The value of rank cannot exceed the value of maxrank. The allocation of space for reading in the SDS should correspond to the values read in by DFSDgetdims DFSDgetdimstrs FORTRAN: INTEGER FUNCTION dsgdist(dim, label, unit, format) INTEGER dim - dimension this label, unit and format refer to CHARACTER*(*) label - label that describes this dimension CHARACTER*(*) unit - unit to be used with this dimension CHARACTER*(*) format - format to be used in displaying scale for this dimension C: int DFSDgetdimstrs(dim, label, unit, format) int dim; /* dimension this label, unit and format refer to */ char *label; /* label that describes this dimension char *unit; /* unit to be used with this dimension char *format; /* format to be used in displaying scale for this dimension */ Purpose: To get the items corresponding to the dimension dim that are stored as strings in the SDS, namely label, unit, and format. Returns: 0 on success; -1 on failure. Note: dim=1 for the first dimension, dim=2 for the 2nd dimension, and so forth. The space allocated for label, unit and format should be at least 1 byte bigger than the length of the string. If the length is unknown when the program is written, one may declare the array size as 1+maxlen_label, _unit, or _format which were set by DFSDsetlengths. The default maxlength for those strings is 255. Example: Reading an SDS with dimension information. In this example a float32 array of size 800x500 is read from an HDF file, together with label, unit, and format information about each dimension. FORTRAN: INTEGER dsgdims, dsgdata, dsgdist INTEGER rank, dimsizes(2), ret CHARACTER*256 lnglabel, lngunit, lngfmt CHARACTER*256 latlabel, latunit, latfmt REAL*4 pressure(500,800) ret = dsgdims(`myfile.hdf', rank, dimsizes, 2) ret = dsgdist(1, lnglabel, lngunit, lngfmt) ret = dsgdist(2, latlabel, latunit, latfmt) ret = dsgdata(`myfile.hdf', rank, dimsizes, pressure) C: int rank; int32 dimsizes[2]; char lnglabel[256], lngunit[256], lngfmt[256], latlabel[256], latunit[256], latfmt[256]; float32 pressure[800][500]; DFSDgetdims("myfile.hdf", &rank, dimsizes, 2); DFSDgetdimstrs(1, latlabel, latunit, latfmt); DFSDgetdimstrs(2, lnglabel, lngunit, lngfmt); DFSDgetdati("myfile.hdf", rank, dimsizes, pressure); ... DFSDgetdimscale FORTRAN: INTEGER FUNCTION dsgdist(dim, size, scale) INTEGER dim - dimension this scale corresponds to INTEGER size - size of scale REAL scale(size) - the scale C: int DFSDgetdimscale(dim, size, scale) int dim; /* dimension this scale corresponds to */ int32 size; /* size of scale */ float32 scale[]; /* the scale */ Purpose: To get the scale corresponding to the dimension dim and store it in the floating-point array scale. Returns: 0 on success; -1 on failure. Note: dim=1 for the first dimension. In the current implementation of HDF, dimension scales must be of the same number type as the corresponding data. (There are plans to allow scales to have their own number type in a future release.) Example: Reading an SDS with dimension scales. In this example a 800 x 500 data array is read from `myfile.hdf', together with scales for each dimension. The scales are assumed to be of type float32. FORTRAN: INTEGER dsgdims, dsgdisc INTEGER rank, dimsizes(2), ret REAL lngscale(500), latscale(800) REAL*4 pressure(500,800) ret = dsgdims(`myfile.hdf', rank, dimsizes, 2) ret = dsgdisc(l, dimsizes(1), lngscale) ret = dsgdisc(2, dimsizes(2), latscale) ret = dsgdata(`myfile.hdf', rank, dimsizes, pressure) ... C: int rank; int32 dimsizes[2]; float32 latscale[800], lngscale[500]; float32 pressure[800][500]; DFSDgetdims("myfile.hdf", &rank, dimsizes, 2); DFSDgetdimscale(1, dimsizes[0], latscale); DFSDgetdimscale(2, dimsizes[1], lngscale); DFSDgetdata("myfile.hdf", rank, dimsizes, pressure); DFSDgetdatastrs FORTRAN: INTEGER FUNCTION dsgdast(label, unit, format, coordsys) CHARACTER*(*) label - label that Describes the data CHARACTER*(*) unit - unit to be used with the data CHARACTER*(*) format - format to be used in displaying data CHARACTER*(*) coordsys - coordinate system C: int DFSDgetdatastrs(label, unit, format, coordsys) char *label; /* label that describes the data */ char *unit; /* unit to be used with the data */ char *format; /* format to be used in displaying data */ char *coordsys; /* coordinate system */ Purpose: To get information about the data itself from all strings. Returns: 0 on success; -1 on failure. The parameter coordsys gives the coordinate system that is to be used for interpreting the dimension information. The space allocated for label, unit and format should be at least 1 byte bigger than the length of the string. If the length is unknown when the program is written, one may declare the array size as 1+maxlen_label, _unit, or _format which were set by DFSDsetlengths. The default maxlength for those strings is 255. Example: Reading a Scientific Dataset with Data Information. In this example a 800 x 500 data array is read from an HDF, together with label, unit, and format information about the data. FORTRAN: INTEGER dsgdims, dsgdata, dsgdast INTEGER rank, dimsizes(2), ret CHARACTER* 256 datalabel, dataunit, datafmt, coordsys REAL pressure(500,800), density(500,800) dimsizes(1) = 500 dimsizes(2) = 800 ret = dsgdims(`myfile.hdf', rank, dimsizes, 2) ret = dsgdata(`myfile.hdf', rank, dimsizes, pressure) ret = dsgdast(datalabel, dataunit, datafmt, coordsys) ... C: int rank; int32 dimsizes[2); char datalabel[256], dataunit[256), datafmt[256], coordsys[256]; float32 pressure[800][500); dimsizes[0] = 800; dimsizes[1] = 500; DFSDgetdims("myfile.hdf", &rank, dimsizes, 2); DFSDgetdata("myfile.hdf,', rank, dimsizes, pressure); DFSDgetdatastrs(datalabel, dataunit, datafmt, coordsys); ... DFSDgetrange FORTRAN: INTEGER FUNCTION dsgrang(max, min) max - high value stored with SDS min - low value stored with SDS C: int DFSDgetrange(pmax, pmin) *pmax; /, high value stored with SDS *pmin; f, low value stored with SDS Purpose: To get maximum and minimum values stored with the current SDS DFSDgetdims or DFSDgetdata should be called before DFSDgetrange is called. Returns: 0 on success, -1 on failure or if there are no max or min values. Since the max/min values are supposed to relate to the data itself, it is assumed that the type of max/min is the same as the type of the data. One implication of this is that in the C version of DFSDgetrange the arguments are pointers, rather than simple variables, whereas in the FORTRAN version they are simple variables of the same type as the data array. NOTE: These values need to have been set by a user via a call to DFSDsetrange.They are not automatically stored. Example. In this example 16-bit data is read from an HDF file. Notice that max and min must be the same data type as the SDS array data. FORTRAN: integer*2 max, min, data(100,100) ... ret = dsgdata(`myfile.hdf', rank, dims, data) ret = dsgrang(max, min) C: int16 max, min, data[100][100]; DFSDgetdata("myfile.hdf", rank, dims, data); DFSDgetrange(&max, &min); DFSDgetcal FORTRAN: INTEGER FUNCTION dsgcal(cal, cal_err, ioff, ioff_err, cal_type) real*8 cal - calibration factor real*8 cal_err - calibration error real*8 ioff - uncalibrated offset real*B ioff_err - uncalibrated offset error integer*4 cal-type - number type of uncalibrated data C: int DFSDgetcal(cal, cal_err, ioff, ioff_err, cal_type) float64 *cal calibration factor float64 *cal-err calibration error float64 *ioff uncalibrated offset float64 *ioff_err uncalibrated offset error int32 *caltype number type of uncalibrated data Purpose: To get calibration information about the data in the current SDS. Returns: 0 on success; -1 on failure. This routine reads the calibration record, if there is one, attached to an SDS. A calibration record contains four 64-bit floating point values followed by a 32-bit integer, to be interpreted as follows: cal calibration factor cal_err calibration error ioff uncalibrated offset ioff_err uncalibrated offset error cal-type numbertype of uncalibrated data The relationship between a value `iy' stored in an SDS and the actual value `y' is defined as: y = cal * (iy - ioff) The variable ioff_err contains a potential error of ioff, and cal_err contains a potential error of cal. Currently the calibration record is provided for information only. The SDS interface performs no operations on the data based on the calibration tag. DFSDreadref FORTRAN: INTEGER FUNCTION dsrref(name, ref) character*(*) name - name of file containing SDS integer ref - reference number for next dsgdata C: int DFSDreadref(filename, ref) char *filename; /* file containing SDS */ uint16 ref; /* reference number for next DFSDgetdata */ Purpose: To specify the reference number of the SDS to be read when DFSDgetdims or DFSDgetdata is next called. Returns: 0 on success; -1 on failure. This routine is most likely to be used in conjunction with DFANgetlablist, which returns a list of labels for a given tag together with their reference numbers. It provides a sort of random access to SDSS. NOTE: There is no guarantee that reference numbers appear in sequence in an HDF file, so it is not generally safe to assume that a reference number is a sequence number for an SDS. DFSDrestart FORTRAN: INTEGER FUNCTION dsfirst() C: int DFSDrestart() Purpose: To cause the next get to read from the first SDS in the file, rather than the SDS following the one that was most recently read. Returns: 0 on success; -1 on failure. Example: Reading Two SDSs from a File. In this example, two SDSs having the same dimension and other information, except the values of the data arrays, are read from a file. The interface remembers from the first call to the second that one SDS has already been accessed, so on the second call it gets the second SDS. FORTRAN: INTEGER dsgdims, dsgdata, dsgdast INTEGER rank, dimsizes(2), ret CHARACTER*256 datalabel, dataunit, datafmt, coordsys REAL pressure(300,200), density(300,200) ret = dsgdata(`myfile.hdf', rank, dimsizes, pressure) ret = dsgdast(datalabel, dataunit, datafmt, coordsys) ret = dsgdata(`myfile.hdf', 2, dimsizes, density) ret = dsgdast(datalabel, dataunit, datafmt,coordsys) ... C int rank; int32 dimsizes[21; char datalabel[256], dataunit[256], datafmt[256], coordsys(256]; float32 pressure[200] [300], density[200][300]; DFSDgetdata("myfile.hdf", rank, dimsizes, pressure); DFSDgetdatastrs(datalabel, dataunit, datafmt, coordsys); DFSDgetdata("myfile.hdf", 2, dimsizes, density); DFSDgetdatastrs(datalabel, dataunit, datafmt, coordsys); Reading Parts of an SDS The routine DFSDgetslice lets you read in a slice from the current SDS. A slice is any subarray, or "hypercube", of the SDS from which it is read. (Slices do not have to be read in contiguous order, so the rules for reading slices are more general than they are for writing slices.) A slice can be described with two one-dimensional arrays, one containing the coordinates of the comer that is nearest to the origin and the other containing the sizes of the slices dimensions. For example, suppose you wish to read the shaded slices from the 10 x 12 SDS shown in Figure 4.5. The relevant comer of the first slice is (3,4) and its dimensions are (4,6). The second slice begins at (1,10) and has dimensions (10,2). Figure 4.5 A 10 x 12 SDS Showing Two Slices That Are To Be Read. *** INSERT FIGURE HERE *** DFSDgetslice FORTRAN: INTEGER FUNCTION dsgslc(filename, winst, windims, dest, dims) CHARACTER*(*) filename - name of HDF file INTEGER winst - array with coordinates of start of slice INTEGER windims - array with dimensions of slice dest(*) - array for returning slice IINTEGER dims - dimensions of array dest C: int DFSDgetslice(filename, winst, windims, dest, dims) char *filename - name of HDF file int32 winst[] - array with coordinates of start of slice int32 windims[] - array with dimensions of slice *dest - array for returning slice int32 dims - dimensions of array dest Purpose: To read part of an SDS from a file. Returns: 0 on success; -1 on failure. DFSDgetslice accesses the dataset last accessed by DFSDgetdims. If DFSDgetdims has not been called, DFSDgetslice gets a slice from the next dataset in the file. Array winst specifies the coordinates of the start of the slice. Array windims gives the size of the slice. The number of elements in winst and windims must be equal to the rank of the dataset. For example, if the file contains a 3D dataset, winst may contain the values (2, 4, 3), while windims contains the values (3, 1, 4). This will extract a 3 x 4, two-dimensional slice, containing the elements between (2, 4, 3) and (4, 4, 6) from the original dataset. dest is the array into which the slice is read. It must be at least as big as the desired slice. dims is an array containing the actual dimensions of the array dest. The user assigns values to dims before calling DFSDgetslice. NOTE: In both C and Fortran the minimum value of winst[i] is 1. For example, if the 3D slice starts at the origin, winst has values [1,1,1] instead of [0,0,0] Example: Reading Slices The C code in Figure 4.6 shows how you could read the two slices shown in Figure 4.5. Figure 4.6 Reading Slices from a 10x12 SDS in Reverse Order /***************************************************** * Example C code: Read in slices from a 10 x 12 array. *****************************************************/ #include "hdf.h" main() { int i, rank; int32 dimsizes[21; DFSDgetdims("myfile.hdf", &rank, dimsizes, 2); /* starting at (3,4) read 4 x 6 window */ /* Note: (3,4) rather than (2,3) because */ /* FORTRAN-style indexing is used. */ /* getit("myfile.hdf", 3,4,4,6); /* starting at (1,10) read 8 x 2 window */ getit("myfile.hdf", 1,10,8,2); } getit(filename, st0, st1, rows, cols) int st0, st1, rows, cols; char *filename; { int i, j; int32 winst[2], windims[2], dims[2]; float32 data[500]; winst[0]=st0; winst[1]=st1; dims[0] = windims[0] = rows; dims[1] = windims[1] = cols; DFSDgetslice(filename, winst, windims, data,dims); for (i=0; i #define MAX_ROW 18 #define MAX-COL 36 #define SIZE_ARRAY (MAX_ROW * MAX_COL) main() { int ret, i, j; int rank, inrank; int32 shape[2], inShape[2]; float32 pressure[MAX_ROW][MAX_COL), latscales[MAX_ROW), lngscales[MAX_COL), inPressure[MAX_ROW][MAX_COL), inlatscales[MAX_ROW], inlngscales(MAX_COL), maxpressure, inMaxpressure, minpressure, inMinpressure, epsi; char *datalabel, *dataunit, *datafmt, inDatalabel[256], inDataunit[256], inDatafmt[256], *dimlabels(2), *dimunits[2), *dimfmts[21, inDimlabels[21[256], inDimunits[2] [256], inDimfmts[2] [256], inDummy[256]; epsi = 0.0001; rank = 2; shape[0] = MAX_ROW; shape[1] = MAX_COL; datalabel = "Pressure 1"; dataunit = "Pascals"; datafmt = "E15.9"; dimlabels[0] = "latitude_label"; dimlabels[1] = "longitude_label"; dimunits[0] = "km"; dimunits[1] = "km"; dimfmts[0] = "F10.2"; dimfmts[1] = "F10.2"; /* getpressure(pressure, SIZE_ARRAY); findMaxMin(pressure, SIZE_ARRAY, &maxpressure; &minpressure); for (i=0;i epsi) printf("Array position %d, %d is different\n", i, j); printf(" ITEM OUTPUT INPUT\n"); printf(" rank %-15d%d\n", rank, inrank); printf(" shape[0] %-15d%d\n", shape[0], inShape[0]); printf(" shape[1] %-15d%d\n", shape[1], inShape[1]); printf(" datalabel %-15s%s\n", datalabel, indatalabel); printf(" dataunit %-15s%s\n", dataunit, indataunit); printf(" datafmt %-15s%s\n", datafmt, indatafmt); printf(" dimlabels[0] %-15s%s\n", dimlabels[0], inDimlabels[0]); printf(" dimunits[0] %-15s%s\n", dimunits[0], inDimunits[0]); printf(" dimfmts[0] %-15s%s\n", dimfmts[0], inDimfmts[0]); printf(" dimlabels[1] %-15s%s\n", dimlabels[1], inDimlabels[1]); printf(" dimunits[1] %-15s%s\n", dimunits[1], inDimunits[1]); printf(" dimfmts[1] %-15s%s\n", dimfmts[1], inDimfmts[1]); printf(" maxpressure %-15f%f\n", maxpressure, inMaxpressure); printf(" minpressure %-15f%f\n", minpressure, inMinpressure); for(i=0;i epsi) printf("latscales is different at position %d\n", i); for(i=0;i epsi) printf("lngscales is different at position %d\n", i); printf("Check Completed\n\n"); Chapter 5 Annotating Data Objects and Files Chapter Overview Annotation Tags File Annotations HDF Object Annotations Tags and Reference Numbers DFTAG_NDG, DFTAG_SDG, and backward compatibility The Annotation Interface Writing Annotations for HDF Objects Reading Annotations for HDF Objects Listing All Labels for a Given Tag Writing Annotations for HDF Files Reading Annotations for HDF Files Getting Annotation Information from a File Chapter Overview This chapter describes the routines that are available for storing and retrieving data and file annotations. Annotation Tags It is often useful to associate information in text form about an HDF file and its data contents, and to keep that information in the same file that contains the data. HDF provides this capability in the form of annotations. An HDF annotation is a sequence of ASCII characters that is associated with one of three types of objects: 1. the file itself, 2. the individual HDF data objects in the file, or 3. the tags that identify the data elements. The current annotation interface supports only the first two types of annotation. HDF annotations can accommodate a wide variety of types of information, including titles, comments, variable names, parameters, formulas, and source code. Any textual information that a user might normally put into a notebook concerning the collection, meaning, or use of a file or data can be put into a file's annotations. Annotations are optionally supplied by a creator or user of an HDF file or data object. Annotations come in two forms: labels and descriptions, defined as follows: label a sequence of ASCII characters, except the character NULL (0).1 description any sequence of characters, including NULL. File Annotations Any HDF file can have labels (called file iDs) and descriptions stored in them. There are routines in the annotations interface specifically designed for reading and writing file IDs and file descriptions. HDF Object Annotations The annotation of HDF data objects is complicated by the fact that you have to uniquely identify the objects being annotated. In order to understand how annotations work for data objects, you need to know a little bit about how HDF data objects are structured and identified within an HDF file. HDF data objects are the basic building blocks of HDF files. An HDF data object has two parts: a 12-byte Data Descriptor (DD) and a data element. A DD has four fields: a tag, a reference number, a .32-bit data offset, and a 32-bit data length. (The latter two are unimportant here.) Taken together, the tag and reference number for a data object uniquely identify that object. Hence, the data object that a particular annotation refers to can be identified by storing the object's tag and reference number together with the annotation. 1Since NULL is used to terminate C strings, its use in the middle of a label has been rule out. Note that an HDF annotation is itself a data object, so it has its own DD. This DD has a tag and a ref number, and it points to the data element that constitutes the annotation. The data element that goes with an annotation contains three things: * the tag of the object that it is an annotation for. * the ref of the object that it is an annotation for, and * the annotation itself. For example, suppose you have an HDF file that contains three scientific datasets (SDS). Each SDS has its own DD consisting of the SDS tag DFTAG_NDG, and a unique reference number as illustrated in Figure 5.1. Figure 5.1 Three SDS Tags with Their Ref Numbers *** INSERT FIGURE HERE *** Suppose you wish to annotate the second SDS by storing the following annotation with it in the file: "Data from black hole experiment 8/18/87." This text would be stored in an HDF file as an annotation, and it would have stored with it the tag DFTAG_NDG and reference number 4. Figure 5.2 illustrates how the annotation would look in the file. Figure 5.2 Displayed Example of SDS, Ref #, and Annotation *** INSERT FIGURE HERE *** Tags and Reference Numbers Note that in order to use annotation routines, you need to know the tags and reference numbers of the objects you wish to annotate. Tag numbers are listed in Appendix A, "NCSA HDF Tags." Special routines are available for obtaining the reference numbers of certain tags, including tags for SDSS, Raster Image Sets, palettes, and annotations. These are: DFSDlastref, DFR8lastref, DFPlastref, and DFANlastref. They return the most recent reference number used in either reading or writing the corresponding data object. Reference numbers for objects other than these can be obtained with the routine DFfindnextref, a general purpose HDF routine. Usage of DFfindnextref is illustrated later in an example. See this chapter's section, "Example: Reading a Label and Description." DFTAG_NDG, DFTAG_SDG, and backward compatibility. In versions of HDF that predate HDF 3.2, an SDS could only support 32-bit floating point numbers, and native Cray floating point numbers. The HDF tag that identified the old (pre-HDF 3.2) SDS, was DFTAG_SDG. In order to support several new number types and at the same time make it easier to add new features (e.g. data compression) to future versions of SDS, a new structure for the SDG (scientific data group) was implemented. The new structure is called NDG, for "numeric data group", and has the tag DFTAG_NDG (720). (The NDG structure is described in detail in the manual "HDF Specifications.") If you have used the annotation interface to annotate Scientific Data Sets with versions of HDF that precede HDF 3.2, chances are you associated your annotation with DFTAG_SDG. DFTAG_SDG is still associated with all SDSs that contain 32-bit floats, but DFTAG_NDG is associated with all SDSS. Therefore, as illustrated in several examples in this chapter, it is now recommended that you use DFTAG-NDG for attaching annotations to SDSS. On the other hand, there may be situations in which you would want to have your program look for SDS annotations associated with DFTAG_SDG as well as DFTAG_NDG. These include: * if SDS annotations might have been written using an older version of the HDF library, or * if SDS annotations might have been written using a new version of the HDF library, but by a program that used DFTAG_SDG, rather than DFTAG_NDG. The Annotation Interface The HDF library provides two types of routines for storing and retrieving annotations: (1) routines for file IDs and file descriptions, and (2) routines for HDF data objects. Table 5.1 lists the C and FORTRAN names of annotation routines currently contained in the HDF library. The following sections provide descriptions and examples of these calling routines. Table 5.1 Long and Short Names for Annotation Routines C FORTRAN Name Name Purpose DFANputlabel daplab puts label of tag/ref. DFANputdesc dapdesc puts description of tag/ref. DFANgetlablen dagllen gets length of label of tag/ref. DFANgetlabel daglab gets label of tag/ref. DFANgetdesclen dagdlen gets length of description of tag/ref. DFANgstdone dagdeac gets description of tag/ref. DFANlablist dallist gets list of labels for a particular tag. DFANaddfid daafid adds file ID DFANaddfds daafds adds file description. DFANgetfidlen dagfidl gets file ID length. DFANgetfid dagfid gets file ID. DFANgetfdslen dagfdsl gets file description length. DFANgetfds dagfda gets file description. DFANlastref* dalrof returns ref of last annotation read or written. *DFANlastref is callable only by C routines. There is no equivalent FORTRAN routine in the HDF library. Writing Annotations for HDF Objects DFANputlabel FORTRAN: INTEGER FUNCTION daplab (filename, tag, ref, label) CHARACTER*(*) filename - name of HDF file to put label in CHARACTER*(*) label - label to write to the file INTEGER tag, ref - tag/ref of item whose label we want to store C: int daplab(filename, tag, ref, label) char *filename; /* name of ADF file to put label in */ uint16 tag, ref; /* tag/ref of item whose label you want to store*/ char *label; /* label to write to the file */ Purpose: To write out a label for the data object with the given tag/ref. Returns: 0 on success; -1 on failure. DFANputdesc FORTRAN: INTEGER FUNCTION dapdesc(filename, tag, ref, desc, desclen) CHARACTER*(*) filename - name of HDF file to put descr in CHARACTER*(*) desc - description to write to the file INTEGER tag, ref - tag/ref of item whose description you want to store INTEGER desclen - length of description C: int dapdesc(filename, tag, ref, desc, desclen) char *filename; /* name of HDF file descr stored in */ uint16 tag, ref; /* tag/ref of item whose descr you want to store */ char *desc; /* description to write to file */ int32 desclen; /* length of description */ Purpose: To write out a description for the data object with the given tag/ref. Returns: 0 on success; -1 on failure. The parameter desclen gives the length of the description that is to be written out. This is necessary because there is no simple way to tell the length of a description that can contain NULL characters without explicitly giving its length. Example: Adding Annotations to a Scientific Dataset The example in Figure 5.3 illustrates the use of DFANputlabel and DFANputdesc to write to a file a label and description for an SDS. The HDF object that contains a SDS is called a numeric data group (NDG). An NDG is a group of tag/refs that make up a SDS. The tag for an NDG is DFTAG-NDG; it's numeric value is 720. Figure 5.3 Adding Annotations to a SDS FORTRAN: integer dsadata, daplab, dapdesc, dslref integer ret, lref, shape(2), DFTAG_NDG real*4 dataset(2,5) parameter (DFTAG_NDG = 720) ... ret = dsadata(`myfile',2,shape,dataset) lref = dslref() ret = daplab(`myfile', DFTAG_NDG, lref, `testlab') ret = dapdesc(`myfile', DFTAG_NDG, lref, $ `This is a test', 14) C: #include "hdf.h" ... int lref, rank, dimsizes[21; char s[50); float *data; ... DFSDadddata("myfile",rank,dimsizes,data); ... sprintf(s,"Data from black hole experiment\n8/18/87"); lref = DFSDlastref(); DFANputlabel("myfile", DFTAG_NDG, i, "black hole"); DFANputdesc("myfile", DFTAG_NDG, i, s, strlen(s)); Remarks: * The call to dslref (DFSDlastref) returns the reference number of the SDS last written to the file. This call is needed to complete the tag/ref combination that uniquely identifies the desired scientific data group that is being annotated. * The file hdf.h that is included with the C source contains the number that corresponds to the tag for a SDS. In the FORTRAN program, this number is defined using a parameter statement. Tags and their numbers are listed in Appendix A, "NCSA HDF Tags." DFANgetlablen FORTRAN: INTEGER FUNCTION dagllen(filename, tag, ref) CHARACTER*(*) filename - name of HDF file label is stored in INTEGER tag, ref, - tag/ref of item whose label you want C: int32 DFANgetlablen(filename, tag, ref) char *filename; /* name of HDF file label is stored in */ uint16 tag, ref; /* tag/ref of item whose label you want */ Purpose: To get the length of a label of the data object with a given tag and reference number. This routine allows you to insure that there is enough space allocated for a label before actually loading it. Returns: The length of label on success; -1 on failure. DFANgetlabel FORTRAN: INTEGER FUNCTION daglab(filename, tag, ref, label, maxlen) CHARACTER*(*) filename, - name of HDF file label is stored in CHARACTER*(*) label - space to return label in INTEGER tag, ref - tag/ref of item whose label you want INTEGER maxlen - size of space to return label in C: int DFANgetlabel(filename, tag, ref, label, maxlen) char *filename; /* name of HDF file label is stored in */ uint16 tag, ref; /* tag/ref of item whose label you want */ char *label; /* space to return label in */ int32 maxlen; /* size of space to return label in */ Purpose: To read in the label of the data object with the given tag and reference number. Returns: 0 on success; -1 on failure. The parameter maxlen gives the amount of space that is available for storing the label. The length Of maxlen must be at least one greater than the anticipated length of the label, because a NULL byte is appended to the annotation. DFANgetdesclen FORTRAN: INTEGER FUNCTION dagdlen(filename, tag, ref) CHARACTER*(*) filename - name of HDF file descr is stored in INTEGER tag, ref - tag/ref of item whose descr you want C: int32 DFANgetdesclen(filename, tag, ref) char *filename; /* name of HDF file descr is stored in */ uint16 tag, ref; /* tag/ref of item whose descr you want */ Purpose: To get the length of a description of the data object with the given tag and reference number. This routine allows you to insure that there is enough space allocated for a description before actually loading it. Returns: The length of description on success; -1 on failure. DFANgetdesc FORTRAN: INTEGER FUNCTION dagdesc(filename, tag, ref, desc, maxlen) CHARACTER*(*) filename - name of HDF file descr is stored in CHARACTER*(*) desc - space to return description in INTEGER tag, ref - tag/ref of item whose descr you want INTEGER maxlen - size of space to return descr in C: int DFANgetdesc(filename, tag, ref, desc, maxlen) char *filename; /* name of HDF file descr is stored in */ uint16 tag, ref; /* tag/ref of item whose descr you want */ char *desc; /* space to return description in */ int32 maxlen; /* size of space to return descr in Purpose: To read in the description of the data object with the given tag and reference number. Returns: 0 on success; -1 on failure. The parameter maxlen gives the amount of space that is available for storing the description. The length of maxlen must be at least one greater than the anticipated length of the description, because a NULL byte is appended to the annotation. Example: Reading a Label and Description This example program (Figure 5.4) illustrates the use of DFANgetlabel, DFANgetdesclen, and DFANgetdesc to read from an HDF file a label and description for an SDS. Figure 5.4 Getting Annotations from a SDS FORTRAN: program getanntest C Program to test routines for reading a label and description integer daglab, dagdesc, dagdlen, dslref, dsgdata integer desclen, lastref, ret, dims(2), rank real*4 data(2,5) integer DFTAG_NDG character*20 label character*400 desc parameter (DFTAG_NDG = 720 ) C************** find ref of first SDS in file ************** ret = dsgdata(`myfile', rank, dims, data) lastref = dslref() C************** get label, then description **************** ret = daglab(`myfile', DFTAG_NDG, lastref, label, 20) print *,`Label: ', label desclen = dagdlen(`myfile', DFTAG_NDG, lastref) if (desclen .gt. 400) then print *, `Description too long. More than 400.' stop endif ret = dagdesc(`myfile',DFTAG_NDG, lastref, desc, desclen) print *, 'Description: ', desc stop end Figure 5.4 Getting Annotations from a SDS, Continued C: /* * Program to test routines for reading a label and description */ #include "hdf.h" main() { uint16 lastref; char label[20], *desc; float data[2] [5]; int rank, dims[2], desclen; DFSDgetdata("myfile", rank, dims, data); lastref = DFSDlastref(); /*** get label, then description ***/ DFANgetlabel("myfile", DFTAG_NDG, lastref, label, 20); printf("Label: %s\n", label); desclen = DFANgetdesclen("myfile", DFTAG-NDG, lastref); if (desclen < 0) { printf("Error reading description length.\n"); exit(1); } else { desc = (char *) malloc( desclen+1); } DFANgetdesc("myfile", DFTAG_NDG, lastref, desc, desclen); printf("Description: %s\n", desc); } Remarks: * Lower level routines F.open, Hclose, and DFfindnextref are used here to find the ref number of the first occurrence of the NDG (scientific data group) tag. * In the above example, DFANgetlabel assumes that the label is not more than 20 bytes long. If the program needs to know the length of a label, it can call DFANgetlablen to find this out. * Since the description could be very long, the routine DFANgetdesclen is called to find the space requirements for the description. In the C program, this space is allocated before calling DFANgetdesc to get the description. In the FORTRAN program, it is assumed to be 400 bytes or less. Listing All Labels for a Given Tag DFANlablist FORTRAN: INTEGER FUNCTION dallist(filename, tag, reflist, labellist, listsize, maxlen, startpos) CHARACTER*(*) filename - name of HDF file labels stored in CHARACTER*(*) labellist - array of strings to place labels in INTEGER tag - tag to find labels for INTEGER reflist(*) - array to place refs in INTEGER listsize - size of ref and label lists INTEGER maxlen - maximum length allowed for label INTEGER startpos - entries will be returned beginning from the startpos entry up to the listsize entry. C: int DEAN.ablist(filename, tag, reflist, labellist, listsize, maxlen, startpos) char *filename; /* name of HDF file labels stored in */ uint16 tag; /* tag to find labels for */ unit16 reflist[]; /* array to place refs in */ char *labellist; /* array of strings to place labels in */ int listsize; /* size of ref and label lists */ int maxlen; /* maximum length allowed for label */ int startpos; /* entries are returned starting from the startpos entry up to the listsize entry */ Purpose: To return a list of all reference numbers and labels (if labels exist) for a given tag. Returns: The number of reference numbers found on success; -1 on error. Listsize gives the number of available entries in the ref and label lists, maxlen is the maximum length allowed for a label, and startpos tells which label to start reading for the given tag. (If startpos is 1, for instance, all labels will be read; if startpos=4, all but the first 3 labels will be read.) Reflist contains a list of reference numbers of all objects with a given tag. Labellist contains a corresponding list of labels, where they exist. If there is no label stored for a given object, the corresponding entry in labellist is an empty string. Taken together, the reflist and labellist returned by DFANlablist constitute a directory of all objects and their labels (where they exist) for a given tag. The list, labellist, can be displayed to show all of the labels for a given tag. Or, it can be searched to find the ref of a data object with a certain label. Once the ref for a given label is found, the corresponding data object can be accessed by invoking other HDF routines. Hence, this routine provides you with a mechanism for direct access to data objects in HDF files. Example: Getting a List of Labels for Images in a File The example in Figure 5.5 illustrates the use Of DFANlablist to get a list of all labels used for SDSs in an HDF file. Figure 5.5 Getting a List of Labels from a File FORTRAN: program getlablist integer dallist integer i, nlabels, startpos, listlen, reflist(20) integer DFTAG_NDG, LISTSIZE, MAXLEN character*15 labellist(20) parameter (DFTAG_NDG = 720, * LISTSIZE = 20, * MAXLEN = 15 startpos = 1 nlabels = dallist(`myfile', DFTAG_NDG, reflist, * labellist, LISTSIZE, MAXLEN, startpos) do 100 i=1,nlabels print *,' Ref number: ',reflist(i), ' Label: ',labellist(i) 100 continue stop end program getlablist integer dallist integer i, nlabels, startpos, listlen reflist(20) integer DFTAG_NDG, LISTSIZE, MAXLEN character*15 labellist(20) parameter (DFTAG_NDG = 720, * LISTSIZE = 20, * MAXLEN = 15 startpos = 1 nlabels = dallist(`myfile',DFTAG_NDG, reflist, * labellist, LISTSIZE, MAXLEN, startpos) do 100 i=1,nlabels print *,' Ref number: ',reflist(i), Label: ',labellist(i) 100 continue stop end C: #include "hdf.h" #define LISTSIZE 20 #define MAXLEN 15 main() { int i, nlabels, startpos=1, listlen=10; uint16 reflist[LISTSIZE]; char labellist[MAXLEN*LISTSIZE+1]; nlabels = DFANlablist("myfile",DFTAG_NDG, reflist, labellist, listlen, MAXLEN, startpos); for (i=0; i= 0) { ret = DFANgetfid(fileid,inlabel, MAXLABLEN, NOTFIRST); printf("\nLabel: %s", inlabel); length = DFANgetfidlen(fileid, NOTFIRST); } /* read description length and description from file length = DFANgetfdslen(fileid, FIRST); ret = DFANgetfds(fileid,indescr, MAXDESCLEN, FIRST); printf("\n\nDescription: \n%s\n", indescr); Hclose(fileid); } Remarks: * These annotations are associated with the file, not with any particular object within the file. * We use the general purpose routines Hopen and Hclose. These routines do not open and close HDF files for you. You must do it explicitly. The value DFACC_READ is defined in hdf.h. Hence the "#include hdf.h" in the C program. It is assumed that the FORTRAN cannot perform such an include, so DFACC_READ is defined with a PARAMETER statement. Getting Annotation Information from a File DFANlastref FORTRAN: integer dalref() C: int DFANlastref() Purpose: To return the most recent reference number of a written or read annotation. Returns: The reference number on success; -1 on error. Chapter 6 NCSA HDF Command Line Utilities Chapter Overview Introduction hdfls: Listing Basic Information about an HDF file hdfed: Editing an HDF File Basics Tutorial Session hdfed with the -batch Option fptohdf: Converting Floating-Point Data to SDS and/or RIS8 Basics Notes Examples Utilities for Working with Raster Image Sets ristosds: Converting several RIS8 Images to one 3D SDS r8tohdf: Converting 8-Bit Raster Images to HDF hdftor8: Extracting 8-Bit Raster Images and Palettes from an HDF File hdfcomp: Compress 8-bit Raster Images in an HDF File. r24hdf8: Converting 24-Bit Raw Raster Images to HDF 8-Bit Raster Images hdf24hdf8: Converting 24-bit HDF image to HDF 8-Bit HDF Image paltohdf: Converting a Raw Palette to HDF hdftopal: Extracting a Palette from an HDF file hdfrseq/hdfseq: Displaying Images hdfpack: Compacting an HDF file Chapter Overview A number of HDF command line utilities are available for working with HDF files. Currently available programs are described below. Introduction The command line utilities are application programs that can be executed by entering them at the command level, just like other UNIX commands. These utilities serve two purposes: 1) They make it possible for you to perform, at the command level, common operations on HDF files for which you would normally have to write your own program. For example, the utility r8tohdf is a program that takes a raw raster image from a file and stores it in an HDF file in a raster image set. 2) They provide capabilities for doing things with HDF files that would be very difficult to do under your own program control. For example, the utility hdfseq takes a raster image from an HDF file and displays it immediately on a Sun console. In addition to the command utilities supported by NCSA, a number of user-contributed utilities have been contributed by HDF users. Although they are not supported by NCSA, these utilities are distributed by NCSA via anonymous ftp in a directory called HDF/contrib. These utilities will not be discussed in this manual. Table 6.1 lists the names and the functions of the utilities described in this chapter. The sections that follow provide descriptions and examples of these routines. Table 6.1 Scientific Dataset Routines in the HDF Library Name Function hdfls displays the tags, ref numbers, and (optionally) lengths of data elements. hdfed lets you browse in an HDF file and manipulate some of the data. fptohdf converts floating-point data to HDF floating-point data and/or 8-bit raster images. ristsds converts a series RIS8 HDF files into a single 3D SDS HDF file r8tohdf converts one or more raw 8-bit images to HDF RIS8 format and writes them to a file, possibly with palettes. hdftor8 converts images and or palettes from HDF format to raw format and stores them in two corresponding sets of files. hdfcoup compresses 8-bit raster images from an HDF file, storing them in a new HDF file. r24hdf8 converts a raw RGB 24-bit image to an 8-bit RIS8 with a palette. hdf24hdf8 quantizes a HDF RGB 24 bit "pixel' image into a 8 bit image with RGB palette and stores it as a HDF 8-bit raster image file paltohdf converts a raw palette to HDF format. hdftopal converts palette in an HDF file to raw format. hdfseq/hdfrseq displays sequences of images directly to the screen from HDF files containing raster images. hdfpack compacts and HDF file by reading in all the objects in an HDF file and writing them out to a new file. hdfls: Listing Basic Information about an HDF File The utility hdfls provides a quick look at the tags, reference numbers, and (optionally) lengths of the data elements. The syntax for hdfls is: hdfls [-o] [-1] filename -o Order: Indicates that the reference numbers are to be displayed in ascending order. -1 Long format: Displays more information about the file. Example 1 A file called aa.hdf contains three items associated with a raster image: (1) the image dimensions, (2) a palette, and (3) the raster image. To display information about the contents of the file, enter: hdfls aa.hdf The following is displayed: aa. hdf: Image Dimensions-8 (Raster-8) : (tag 200) Ref nos: 1 Image Palette-8 (Raster-8) : (tag 201) Ref nos: 3 Raster Image-8 (Raster-8) : (tag 202) Ref nos: 1 To display the same information together with the length of each data element, enter: hdfls -1 aa.hdf The resulting display is: aa.hdf: Image Dimensions-8 (Raster-8) : (tag 200) Ref no 1 4 bytes Image Palette-8 (Raster-8) : (tag 201) Ref no 3 768 bytes Raster Image-8 (Raster-8) : (tag 202) Ref no 1 120000 bytes hdfed: Editing an HDF File Basics The utility hdfed allows sophisticated HDF users to manipulate the elements in an HDF file. These manipulations include selecting groups and showing information about them, dumping them to the output, writing them to new files, deleting them, inserting them, replacing, say, the palette of an RIS8 group, and editing the text labels and descriptions of any element. hdfed is designed primarily for uses who need to know about HDF files at the level of individual data elements. It is not designed to provide a high-level view of the contents of an HDF file. Other tools and utilities should be used for that purpose. To use hdfed effectively you should probably have access the manual "HDF Specifications," which describes in detail the various components of an HDF file. hdfed is modeled to some extent after ed, the UNIX line editor. When you invoke hdfed, your terminal screen acts as a window into an HDF file. Your initial view of the file is as a set of tags and reference numbers. Each tag/ref combination uniquely identifies a data object in the file. hdfed will not allow the user to arbitrarily modify binary data in the file or any element, though it allows modification of tag and reference numbers within strict constraints. The user should not attempt to alter individual bytes. It is acceptable, however, to replace an element with another of the same type. For the sake of the following discussion, certain terms need to be clarified: data object or object refers to the data descriptor of an object (tag/ref/offset/length) plus the data itself. data or data element refers to the record that the data descriptor points to. For a precise description of the data that is associated with a given tag consult the manual "HDF Specifications ". group refers to a predefined collection of data objects that correspond to a particular application. For instance, a "raster image group' refers to the collection of objects that are used to store all of the information in a Raster Image Set. Once you have opened an HDF file with hdfed, you can do several things with the file, including the following: * Select an HDF object to examine more closely. * Move forward or backward among the objects in the file. * Get information about an object (tag, ref, size, label). * Display a raster image using the ICR protocol. * Display a dump of any object. * Delete an object. * Annotate an object with a label or description.. * Write an object to another HDF file. * Write a data element in its binary form to a non-HDF file. * Close the file and exit, or open a new file. hdfed also has a small set of special commands, including a conditional statement, a looping statement, an alias command and an unalias command. A simple 'help' feature also exists. The syntax for hdfed is: hdfed [filename] [-nobackup] [-batch) If filename is present, the corresponding file is opened, and a backup is made of the file. Otherwise, a file may be opened from within the editor. Options * -nobackup Specifies that no backup file is to be made. If this option is omitted, a backup file is automatically created. * -batch Specifies that input to hdfed is to be input via a stream of hdfed commands, rather than interactively. To receive usage information, as well as a quick list of the hdfed commands, type the command: hdfed -help While you are using the editor, you receive the prompt: he> You can now enter hdfed commands. Many hdfed Commands have qualifiers, or flags. For instance the info command may optionally be followed by -all, -long, -group, or -label. All of the commands and flags can be abbreviated to the extent that their abbreviations are unique. For example -he is an ambiguous abbreviation because it could stand for either the flag -hexadecimal Or the flag -help; on the other hand, the flag -hel is not ambiguous. Table 6.2 lists the hdfed COMMands. Table 6.2 hdfed Commands Name Function help Provide help on use of hdfed open Open HDF file close Close HDF file revert Revert to original file next Go to next object/group that satisfies predicate prev Go to previous object/group that satisfies predicate info Show information about data object dump Display data in binary, ASCII, etc. display Display a raster image using ICR put Put data element as binary into raw file putr8 Put an ris8 group into raw file getr8 Get an ris8 group from raw file delate Delete an object/group write Write an object/group to another HDF file annotate Annotate an object if Conditional statement select Loop for each object alias Set or show aliases unalias Remove an alias wait Message and wait for return To obtain information about the usage of a command, type: -help Do not just type the command and expect a help line. Some commands, such as delete, do not need any argument, so this could have unfortunate results. The commands are described in the text that follows. * help Print a help screen describing the basic purpose and functionality Of hdfed. * open This command is described above. * close [-keep] Close current open file. -keep: causes backup file not to be deleted. * revert Discard all changes made in hdfed. * next [] Move to the next object that satisfies the predicate. are of the general form: or simply group (as in "next group") can be any of the following: tag, ref, image_size, or label. can be any of the following: "=", "!=", "<", ">=", "<=" Examples: next group next (same as "next group". "group" is the default) next tag = 720 next ref = 2 next image_size < 1000 next label = "abc" * prev [] Move to the next object that satisfies the predicate. (See "next" for examples of predicates. * info [-all] [-long] [-group) [-label] Show information for one data object. -all Show info for all objects in file -long Show more info -group Organize info in group(s) -label Show label if any * dump [-offset ] [-length ] [-decimal|-short|-byte|-octal|-hexadecimal |-float|-double|-ascii] Display the contents of the current object in the format specified. -offset Start offset -length Length to look at -decimal Decimal format [32 bit integers] -short Decimal format [16 bit integers] -byte Decimal format [8 bit integers] -octal Octal format [Default] -hexadecimal Hexadecimal format -float Float format [32 bit floats] -double Float format [64 bit floats] -ascii Ascii format * display [-position ] [-expansion ] [-large] Display image on screen. (Only works where hdfseq works.) -position Image position on console screen -expansion Image expansion factor -large Make image as large as possible * put [-file ] [-verbose] Put the raw binary of this object in a file -file Out file name (default "elt#.@") -verbose Output diagnostic info * putr8 [-image ] [-palette ] [-verbose] Put an r8 group into raw image and palette files -image Image file name template (default "img#.@.%") -palette Palette file name template (default "pal#") -verbose To give output of steps taken * getr8 [-palette ] [- raster|-rle|-imcomp] Get a r8 group from raw files -palette Raw palette file -raster No compression (default) -rle Run-length compression * delete Delete this object or group. * write [-attachto ] Write an element or group into another HDF file - attachto What element to attach annotation to (ONLY for writing annotations) * annotate [-label|-descriptor] [-editor ] Edit an annotation - label Edit label (default) - descriptor Edit descriptor - editor Use editor (default EDITOR env value) * if [] * end Execute commands if predicates are satisfied by element * select [] * end Step through all elements in the file that satisfy the predicates and execute the commands on them. * wait [] Print message and then wait for user to hit return Tutorial Session Figure 6.1 contains a tutorial in the form of a session using hdfed. Examples of almost all of the commands are given below. In the following script, bold characters represent those typed by a user. Plain text characters represent characters that were printed by the computer. In some cases, ellipses ("..." ) are used to indicate that more information would normally be printed. Figure 6.1 Tutorial Session zaphod|2% he he> ! This is a script of a session on the RDF editor. he> ! It is meant as an example for anyone learning to he> ! use the editor. he> ! The exclamation mark in the FIRST column starts ! he> ! a comment he> help he> ! help command he> help hdfed allows sophisticated HDF users to manipulate the elements in an HDF file. These manipulations include selecting groups ... open he> ! opening a file he> he> ! open -help open [-nobackup) -nobackup Don't make a backup for this file. he> he> open h1 he> info he> ! look into the contents of this file he> ! using info he> he> info -all -label -long (1) Version Descriptor : (Tag 30) Ref: 1, Offset: 202, Length: 92 (bytes) (2) Scientific Data : (Tag 702) Ref: 2, Offset: 294, Length: 200 (bytes) (3) Number type : (Tag 106) Ref: 2 Offset: 494, Length: 4 (bytes) (4) SciData description : (Tag 701) Ref: 2, Offset: 498, Length: 22 (bytes) (5) SciData max/min : (Tag 707) Ref: 2, Offset: 520, Length: 4 (bytes) *(6) Numeric Data Group : (Tag 720) Ref: 2, Offset: 524, Length: 12 (bytes) Label: Experiment #1 (7) Data Id Label : (Tag 104) Ref: 3, Offset: 536, Length: 17 (bytes) (8) Scientific Data : (Tag 702) Ref: 4, Offset: 553, Length: 400 (bytes) (9) Number type : (Tag 106) Ref: 4, Offset: 953, Length: 4 (bytes) (10) SciData description : (Tag 701) Ref: 4, Offset: 959, Length: 22 (bytes) (11) Numeric Data Group : (Tag 720) Ref: 4, Offset: 979, Length: 8 (bytes) Label: Experiment #2 (12) Data Id Label : (Tag 104) Ref: 5, Offset: 987, Length: 17 (bytes) he> he> info -group -all **Group 1: Numeric Data Group : (Tag 720) Ref 2 Scientific Data : (Tag 702) Ref 2 SciData description : (Tag 701) Ref 2 SciData max/min : (Tag 707) Ref 2 **Group 2: Numeric Data Group : (Tag 720) Ref 4 Scientific Data : (Tag 702) Ref 4 SciData description : (Tag 701) Ref 4 **These do not belong to any group: Version Descriptor : (Tag 30) Ref 1 Number type : (Tag 106) Ref 2 Data Id Label : (Tag 104) Ref 3 Number type : (Tag 106) Ref 4 Data Id Label : (Tag 104) Ref 5 he> next he> ! Move in the file using next and prev. prev he> ! The move direction depends on the relative predicates he>! positions, so it is often necessary to do an `info -all' first he> he> info -all (1) Version Descriptor : (Tag 30) Ref 1 (2) Scientific Data : (Tag 702) Ref 2 (3) Number type : (Tag 106) Ref 2 (4) SciData description : (Tag 701) Ref 2 (5) SciData max/min : (Tag 707) Ref 2 *(6) Numeric Data Group : (Tag 720) Ref 2 (7) Data Id Label : (Tag 104) Ref 3 (8) Scientific Data : (Tag 702) Ref 4 (9) Number type : (Tag 106) Ref 4 (10) SciData description : (Tag 701) Ref 4 (11) Numeric Data Group : (Tag 720) Ref 4 (12) Data Id Label : (Tag 104) Ref 5 he> * he> ! The `*' in the first column marks the current he> ! position. he> ! The next and prev commands work on predicates. he> ! If I now want to move to the max/min element, he> ! I can use the `tag=' predicate he> he> prev tag=707 he> info (5) SciData max/min (SciData) : (Tag 707), Ref: 2 he> he> ! This predicate persist for next and prev. he> ! That means that if I now do another `next' he> ! it will look for a tag=707 he> he> next Reached end of file. Not moved. he> info (5) SciData max/min : (Tag 707) Ref 2 he> he> next group he> next group he> info (11) Numeric Data Group : (Tag 720) Ref 4 he> dump he> ! to see the binary representation of this element he> he> dump 0: 257400004 257200004 he> he> dump -short 0: 702 4 701 4 he> delete he> ! deleting groups he> he> ! If an element is required by other groups it is he> ! alone. However, this is not perfect since the way he> ! group membership in determined can be pretty ad hoc he> he> delete he> ! This deletes the Scientific Data Group. he> info -all (1) Version Descriptor : (Tag 30) Ref 1 (2) Scientific Data : (Tag 902) Ref 2 (3) Number type : (Tag 106) Ref 2 (4) SciData description : (Tag 701) Ref 2 (5) SciData max/min : (Tag 707) Ref 2 (6) Numeric Data Group : (Tag 720) Ref 2 (7) Data Id Label : (Tag 104) Ref 3 (8) Number type : (Tag 106) Ref 4 (9) Data Id Label : (Tag 104) Ref 5 he> he> ! Notice that the Numeric Data Group with he> ! ref #4 is missing, and now there are only he> ! 9 objects in the file. he> annotate he> ! Annotations are labels and descriptors. he> he> prev group he> info -label (6) Numeric Data Group : (Tag 720) Ref 2 Label: Experiment #1 he> annotate -editor /usr/ucb/ex "/tmp/he5091.1" 1 line, 14 characters :p Experiment #1 :s/$/ / Experiment #1 :wq "/tmp/he5091.1" 1 line, 27 characters he> info -label (6) Numeric Data Group : (Tag 720) Ref 2 Label: Experiment #1 he> write he> ! Write object or group to another EDF file he> he> write test he> he> ! Let's look at the file `test'. he> close;open test;info -all (1) Version Descriptor : (Tag 30) Ref 1 (2) Scientific Data : (Tag 702) Ref 2 (3) Number type : (Tag 106) Ref 2 (4) SciData description : (Tag 301) Ref 2 (5) SciData max/min : (Tag 707) Ref 2 *(6) Numeric Data Group : (Tag 320) Ref 2 he> he> close; he> display he> !We will open a file with some RIS8 images. he> he> open denm.HDF he> display he> he> ! display displays the current r8 group image via ICR. he> ! I.e. if you are using NCSA Telnet on a MacII, this he> ! would display the images from denm.hdf on your screen he> ! NOTE: not guaranteed to work otherwise. he> putr8 he> ! putr8 puts an r8 group into raw files he> he> putr8 _image my_image#.@.% -palette my_palette _verbose Writing to file: my_image8.10.10 Writing to file: my_palette he> select he> ! To stop through a file and, say, putr8 on all r8 he> ! groups we can use the select command he> he> select tag=306 >> putr8 -image testImages# -palette testPalettes# _verbose >> end Writing to file: testImages8 Writing to file: testPalettes8 Writing to file: testImages14 Writing to file: testPalettes14 Writing to file: testImages21 Writing to file: testPalettes21 he> he> ! select and if commands take the same predicates he> ! next and pref. There are also the predicates he> ! "succeed" and "fail" that tout the return status he> ! of the last command. he> put he> ! put puts an element into a binary file he> ! This in a dumb routine and does not know about the he> ! formats of the element he> he> put -file binary# he> put -file myBinary -verbose Writing to file: mybinary he> he> ! that's it for now he> revert;close;quit zaphod|2% hdfed with the -batch Option Sometimes we want to have hdfed execute a series of commands, hdfed with the without waiting for a prompt between commands. This process is -batch Option useful when a certain sequence of hdfed commands is commonly used, as illustrated in the following shell script: #!/bin/csh -f set file=$1 shift hdfed -batch $file -nobackup << EOF info -all -group $* close quit EOF echo "" This shell script lists information about the groups in an HDF file. The "-batch" in the line that calls hdfed tells hdfed that the edit commands are to be taken from the stream of characters beginning on the next line and ending with the EOF symbol. fptohdf: Converting Floating-Point Data to SDS and/or RIS8 Basics fptohdf is a utility that converts 32-bit floating-point arrays (from either text files or HDF float32 scientific datasets) to either HDF 8-bit raster image sets (RIS8) or HDF scientific float32 datasets (SDS), or both, and stores the results in an HDF file (see Fig. 6.2). The image can be scaled about a mean value that is provided by the user. Figure 6.2 The fptohdf Utility *** INSERT FIGURE HERE *** The syntax of fptohdf is as follows: fptohdf infile [infile ... ] -o outfile [out_opts] out_opts: [[-r] [[-e|-i] ] [-p palfile] -m ] [-f] * infile is a file containing a single floating-point array in either text or HDF SDS format. If the format is text, see "Notes" below on how it must be organized. * outfile is a file containing the converted dataset in HDF format. Depending on out_opts, outfile contains a scientific dataset and/or raster image set for each of the datasets in the input files. Options -r Stores as image in raster image set in output file -e Expands float data via pixel replication to produce image (default if -I option chosen) -i Applies bilinear interpolation when expanding float data to produce image when -e or -i options are chosen, these give the resolution in pixels of the desired image -p palfile Stores palette with image; get palene from HDF file palfile. -m Causes the data to be scaled around when generating the image, according to the following formulas: newmax = mean + 0.5*max(abs(max-mean),abs(mean-min)) newmin = mean - 0.5*max(abs(max-mean),abs(mean-min)) -f Stores as scientific dataset in output file (default) Notes * Image expansion is available via either pixel replication (-e) or bilinear interpolation (-i), but not both. * If the "-i" option is chosen, the expanded image must have dimensions that are greater than or equal to the dimensions of the original dataset * An optional palette can accompany the image by loading it from an HDF file that contains a palette. * Data from several input files (one set per input file) are stored as several datasets and/or images in one output file. Alternatively, a shell script can be used to call fptohdf repeatedly to convert data from several input files to several corresponding HDF files. * If an HDF file is used for input, it must contain an SDS. The SDS need only contain a dimension record and the data, but if it also contains max and min values and/or scales for the horizontal and vertical axes, these will be used also. ("Scales" are used for determining the gaps between the points on the axes. If the gaps are all the same, a uniform scale is given, such as "1.0 2.0 3.0 ... n". In an HDF file, scales may be omitted, but in a text file (see below) they must be included.) If a text file is used for input, it must follow the format shown in Figure 6.3. Figure 6.3 Format Used in a Text File for Input nrows ncols max_value min_value scale for vertical axis (an array with nrows entries) scale for horizontal axis (an array with ncols entries) data1 data2 data3 ... ...... NOTE: There are nrows vertical axis scale values and ncols horizontal axis scale values. datal, data2, etc., are the floating-point data and are assumed to be ordered by rows, left to right and top to bottom. Examples Example 1 Convert floating-point data in file fl.txt to a SDS format, and store it as an SDS in HDF file o1: fptohdf f1.txt -o o1 Example 2 Convert floating-point data in file f2 to an 8-bit raster image and store it as an RIS8 in file o2: fptohdf f2 -o o2 -r Example 3 Convert floating-point data in file f3 to RIS8 format and SDS format and store both the RIS8 and the SDS in file o3: fptohdf f3 -o o3 -r -f Example 4 Convert floating-point data in file f4 to a 500 x 600 raster image, storing the corresponding RIS8 in file o4. Also store a palette from palfile with the image: fptohdf f4 -o o4 -r -o 500 600 -p palfile Example 5 Convert floating-point data in all files whose names begin with the letter `f' to 500 x 600 images and store the corresponding RIS8s in file o5: fptohdf f* -o o5 -r -i 500 600 Utilities for Working with Raster Image Sets There are three utility programs for working with HDF files that contain raster image sets. These routines can be executed interactively at the command level without being embedded in programs. ristosds: Converting several RIS8 Images to one 3D SDS The utility converts a series of HDF raster image files into a single HDF 3D SDS file. Each input file contains one or more raster images. All images should be of the same dimension sizes. If there is a palette associated with the images, the palette should be included in the first HDF file. Any subsequent palettes will be ignored. The syntax for this utility is as follows: ristoods { } -o where infile is a file containing one or more 8-bit raster images sets (RIS8), and outfile is the file that will contain the 3D scientific data set (SDS). Example A directory contains 20 files named storm001, storm002, ... storm020. Each file contains a single RIS8 with a 100x200 image. We wish to combine these 20 images into a 32-bit float SDS with dimensions 100x200x20. We can do this by executing ristosds as follows: iristoods storm*.hdf -o stors.hdf r8tohdf: Converting 8-Bit Raster Images to HDF The utility rbtohdf converts one or more raw images to HDF RIS8 format and writes them to a file. The syntax for storing one RIS8 in to HDFa file using r8tohdf is as follows: r8tohdf rows cols outfile [-p palfile] {[-c],[-r], [-i]} imf1 imf2 ... where rows and cols are the number of rows and columns, respectively, of the raster image outfile is the file that will contain the raster image set, and imf1, imf2, and so forth, are files containing 8-bit raw raster images. Options -p palfile optionally stores a file containing a palette in the RIS8. If -p is not specified, no palette is stored in the RIS8. {[-c], [-r], [-i]} optionally chooses compression by run length encoding (-c), compression by the IMCOMP method (-i), or no compression (-r). The default is -r. Example 1 A 256 x 512 byte raw raster image is contained in a file called rawras, and the palette with which it is to be used is stored in a file called mypal. To convert this information to an RIS8 without compression and store the result in a file called ras.hdf , enter: r8tohdf 256 512 ras.hdf -p mypal rawras Example 2 A 800 x 1000 byte raw raster image is stored in a file called bigpic. You want to first convert this information to an RIS8 with run length encoding for compression and no palette, and then store the result in a file called bigpic.hdf. Enter: r8tohdf 800 1000 bigpic.hdf -c bigpic Example 3 Three 300 x 400 raw images are contained in files pic1, pic2, and pic3. To convert all three files to RIS8s with imcomp compression and store them in pic.hdf, enter: rStohdf 300 400 pic.hdf -i pic1 pic2 pic3 Example 4 A combination of different types of raster image sets is to be stored in a file called ras.hdf. The image from filerawrasi is to be stored without a palette. The images from rawras2 are to be stored with a palette that is copied from a file called mypal. The images from rawras1 and rawras2 are to be compressed using run length encoding, and rawras3 is not to be compressed. All images are 256 x 512. r8tohdf 256 512 ras.hdf -c rawras1 -p mypal rawras2 -r rawras3 hdftor8: Extracting 8-Bit Raster Images and Palettes from an HDF File The utility hdftor8 extracts the images and or palettes from an HDF file and stores them in two sets of files that contain only images and palettes, respectively. The syntax for hdftor8 is as follows: hdftor8 hdf_file [-i] [-v] [-r image_file] [-p palette_file] Where hdf-file is the file to extract images and-palettes from, and image_file and palette file are basic names of the files that will contain the images and palettes. These names are extended as follows: For each image file, the filename is given the extension ".#.@.% where # stands for the image number from the original file, @ is the x dimension of the image, and % is the y dimension of the image. For each palette file, the filename is given the extension ".#", where # stands for the palette number from the original file. If no names are given for the image file, the default name "img.#.@.%" is used, where #, @ and % are defined as in the preceding paragraph. Similarly the default name for a palette file is "pal.#". Options -i puts the program in interactive mode, so you can specify filenames interactively. -v specifies verbose mode, providing descriptive messages. -r indicates that the file whose name follows is to hold images. -p indicates that the file whose name follows is to hold palettes. Example 5 A file called denm.hdf contains three 512 x 256 images and three palettes. To extract these images and palettes and put them in separate files, enter: hdftor8 denm.hdf Six files are produced with the names img1.512.256, img2.512.256, img3.512.256, pal.1, pal.2, and pal.3. hdfcomp: Compress 8-bit raster images in an HDF file. The utility hdfcomp reads in raster-8 images from an HDF file and create a new HDF containing the images in a compressed format. Images will be appended to the output file, if it exists. The syntax for hdfcomp is: hdfcomp outfile {[-c], [-r0, [-i]} imagefile ... {[-c], [-r], imagefile outfile file to store compressed images in imagefile file containing an HDF image Options: - r Store without compression (default) - c Store using RLE compression - i Store using IMCOMP compression (requires a palette in the HDF file) Example, A directory contains 20 files named storm001, storm002, storm020. Each file contains a single 8-bit image.. We wish to compress these 20 images using run-length encoding and store them in a single file called allcomp.hdf. We can do this by executing hdfcomp as follows: hdfcomp allcomp.hdf -c storm*.hdf r24hdf8: Converting 24-bit Raw Raster Image to HDF 8-Bit Raster Images The command line utility r24hdfs quantizes a raw RGB 24-bit pixel image into an 8-bit image with a 256-color palette and stores both the palette and image in an HDF file. The syntax for r24hdf8 is: r24hdf8 x_dim y_dim r24_file hdf8_file x_dim is the x dimension of the image (horizontal size). y_dim is the y dimension of the image (vertical size). r24-file is a file containing the raw ROB 24 bit image. The order of pixels is left to right and top to bottom. Each pixel is three contiguous bytes: red byte, green byte, and blue byte. Use filter ptox to convert from "planar" to "pixel" where planar means all red bytes for the whole image, followed by all green bytes for the whole image, followed by all blue bytes for the whole image. hdf8_file is the output file, is an HDF file with one 8-bit raster image set; i.e. 8-bit image, dimensions, and RGB palette. hdf24hdf8: Converting 24-bit HDF image to HDF 8-Bit HDF Image The Utility hdf24hdf8 quantizes a HDF RGB 24 bit "pixel" image into a 8 bit image with 256-color palette and stores both the palette and image in an HDF file. The syntax for hdf24hdf8 is: hdf24hdf8 r24_file hdf8_file r24_file is the input file, an HDF file containing the HDF RGB 24-bit raster image hdf8_file is the output file containing one 8-bit RIS with an RGB palette. paltohdf: Converting a Raw Palette to HDF paltohdf converts a raw palette to HDF format. The incoming palette is assumed to have 768 bytes organized in the following order: 256 red values, 256 green values, and 256 blue values. The palette in the HDF file will have the RGB values interlaced: RGB RGB RGB... This is standard HDF format for 8-bit palettes. The syntax for paltohdf iS: paltohdf rawpalfile hdffile rawpalfile is a 768-byte input file with the red, green, and blue components stored as described above. hdffile, the output file, is an HDF file. If hdffile does not exist, it is created and the palette is added to it. If it already exists, the palette is appended to it. hdftopal: Extracting a palette from an HDF file hdftopal converts a palette from an HDF file to a raw palette in a raw palette file. The outgoing palette will have 768 bytes: first 256 red values, then 256 greens, then 256 blues. The syntax for hdftopal is: hdftopal hdffile rawpalfile hdffile, the input file, is an HDF file containing a palette. rawpalfile is a 768-byte output file with the red, green, and blue components stored as described above. hdfrseq and hdfseq: Displaying Images The utilities hdfseq and hdfrseq display sequences of images, one after the other, from files containing raster image sets. Although hdfseq and hdfrseq perform essentially the same function, the situations in which they are used are different. hdfseq displays images on a Sun-3 console and on any SGI IRIS that has gllib, when those images are stored at the Sun 3 or IRIS workstation. When hdfseq executes on a file, it causes the raster image to be displayed immediately on the console. hdfrseq performs the same operation, only remotely, via a terminal emulator such as Telnet 2.3 or Teltool 1.2. (The "r" in hdfrseq stands for remote.) When hdfrseq executes on a file, it converts the image from RIS8 format to NCSA's ICR (interactive color raster) format and sends it to the receiving terminal. ICR is a protocol that is easy to transmit to a remote display station. If the receiving software on the display station can decipher the ICR protocol, it displays the image on the console. In a typical application, the display station would be running a program, such as NCSA Telnet (Version 2.3 or later) or NCSA Teltool (Version 1.2 or later), that understands ICR and Immediately converts the incoming stream to a screen display. The syntax for hdfseq iS: hdfseq [-s) [-l] [-p xloc yloc] [-e expansion] file1 file2 ... file1, file2, and so forth are HDF files that contain raster image sets. Only one image is displayed per file. If more than one file is listed, the corresponding images are displayed in sequence, that is, in the order in which the files are listed. Each new image overwrites the preceding image on the screen. Options -s single step through the images. Once an image is displayed, it remains on the screen until you press RETURN to display the next image or enter Q to quit. -l makes the image as large as possible on the screen. -p xloc yloc places the upper left corner of the display at position (xloc, yloc) on the screen. -o expansion expands the image by the expansion factor specified by expansion. The syntax for hdfrseq is exactly the same. Example 6 Five images are stored in RIS8 form in the files star1, star2, star3, star4, and star5. You want to use hdfrseq to display all five images, in sequence, but with a pause between them. To make them appear on screen three times their normal size, enter: hdfrseq -s -e 3 star1 star2 star3 star4 star5 hdfpack: Compacting an HDF file The utility hdfpack compacts an HDF file by reading in all the data elements in the file and writing them out to a new file. hdfpack [-i | -b] [-d#] [-t#] -b Don't coalesce linked-block elements -i interactive mode; prompt for each linked-block element -d# Force the output file to have # DDs per DD block -t# Force output file to have # linked blocks per table entry Only one of options b and i can be specified. Examples To compact the file aa.hdf and put the result in aa.pck, enter hdfpack aa.cdf aa.pck Suppose the file bb.hdf contains elements stored as sequences of linked blocks. To compact this file but leave the linked-block elements intact, and put the result in bb.blk, enter hdfpack -b bb.hdf bb.blk Appendix A NCSA HDF Tags Overview This appendix includes a Table containing brief descriptions of most of the tags that have been assigned at NCSA for general use. The list is growing and will be expanded in future editions of the documentation. More detailed descriptions of the tags can be found in an appendix of the NCSA HDF Specifications. To obtain a draft of this document, contact the NCSA Software Development Group at the address listed on the README page at the beginning of this manual. Tag Types and Descriptions The following table has five columns. The Tag column gives the abbreviated symbolic name of the tag that is often used in an augmented form in HDF programs. The Name column gives a name commonly used in human-readable media. The Data Typecolumn describes the type of data that is associated with the tag, and where possible, gives the amount of data. The Number column gives the actual numeric value that is stored in the file to represent the tag. The Use column describes the tag in brief and general terms. In the table, the term string refers to a sequence of ASCII characters, with a zero byte possibly occurring at the end, but nowhere else. The term text likewise refers to a sequence of ASCII characters, but it may contain zero bytes elsewhere than at the end. Table A.1 HDF Tags Tag Name Data type Number Use Utility Tags DFTAG_NULL NoData none 001 used for place holding and filling up empty portions of the Data Descriptor Block. DFTAG_VERSION Library 4 bytes 030 specifies version of latest version number + string version of HDF library used to write to the file. DFTAG_NT Number Type 4 bytes 106 used by any other element in the file to indicate specifically what a numeric value looks like. DFTAG_MT Machine Type 0 bytes 107 specifies that all unconstrained or partially constrained values in this HDF file are of the default type for that hardware. DFTAG_FID File identifier string 100 points to a string that the user wants to associate with this file. It is intended to be user-supplied title for the file. DFTAG_FD File Descriptor text 101 points to a block of text describing the overall file contents. It is intended to be user-supplied comments about the file. DFTAG_TID Tag identifier string 102 provides a way for a human reader to tell the meaning of a tag stored in a file. DFTAG_TD Tag Descriptor text 103 similar to TID, but allows more extensive text to be included. DFTAG_DIL Data identifier Label string 104 associates the string with the Data identifier as a label for whatever that Data identifier points to in turn. By including DILs, any piece of data can be given a label for future reference. For example, this is often used to give titles to images. DFTAG_DIA Data Identifier text 105 associates the text block with the Data Annotation Identifier as annotation for whatever that Data identifier points to in turn. With DIA, any Data identifier can have a lengthy, user-written description of why that data is in the file. DFTAG-RLE Run length encoding 0 bytes 011 specifies that run length encoding is used to compress an image. DFTAG_IMC IMCOMP 0 bytes 012 specifies that IMCOMP compression is used Compression to compress an image. DFTAG_JPEG 24-bit JPEG ? bytes 013 provides header information Compression for 24-bit JPEG compressed images DFTAG_GREYPEG 8-bit JPEG ? bytes 014 provides header Compression information for 8-bit JPEG compressed images General Raster image Tags DFTAG_RIG Raster image Group n*4 bytes 306 lists the Data identifiers (tag/ref) that describe a raster image set. DFTAG_ID image Dimension 20 bytes 300 defines the dimensions of the 2D array that its corresponding Ri tag refers to. DFTAG_LD LUT Dimension 20 bytes 307 defines the dimensions of the 2D array that its corresponding LUT tag refers to. DFTAG_MD Matte Dimension 20 bytes 308 defines the dimensions of the 2D array that its corresponding MA tag refers to. DFTAG_RI Raster image x*y bytes 302 points to raster image data. DFTAG_CI Compressed Image n bytes 303 points to a compressed raster image. DFTAG-LUT Look Up Table n bytes 301 table to be used by hardware to assign RGB colors or HSV colors to data values. DFTAG_MA Matte Data n bytes 309 points to matte data. DFTAG_CCN Color Correction n bytes 310 specifies the Gamma correction for the image and color primaries for the generation of the image. DFTAG-CFM Color Format string 311 indicates what interpretation should be given to each element of each pixel in a raster image. DFTAG-AR Aspect Ratio 4 bytes 312 indicates the visual aspect ratio for this image. Composite Image Tag DFTAG_DRAW Draw n*4 bytes 400 identifies a list of Data Identifiers (tag/ref) which define a composite image. Tag for Composite or Raster Images DFTAG_XYP XY Position 8 bytes 500 indicates an XY position on the screen for composites and other groups. DFTAG_RUN Run n bytes 401 identifies code that is to be executed as a program or script. Vector image Tags DFTAG_T14 Tektronix 4014 n bytes 602 points to a Tek 4014 data stream. The bytes in the data field, when read and sent to a Tektronix 4014 terminal, will be displayed as a vector image. DFTAG_T105 Tektronix 4105 n bytes 603 points to a Tek 4105 data stream. The bytes in the data field, when read and sent to a Tektronix 4105 terminal, will be displayed as a vector image. Scientific Data Set Tags DFTAG_NDG Numeric Data Group n*4 bytes 720 lists the Data identifiers (tag/ref) that describe a scientific dataset. Supersedes DFTAG-SDG. DFTAG_SDD SDS n bytes 701 defines the rank and dimensions of the Dimension Record array that the corresponding SD refers to. DFTAG_SD Scientific Data reals 702 points to scientific data. DFTAG_SDS Scales reals 703 shows scales to be used when interpreting and displaying data. DFTAG_SDL Labels strings 704 labels for all dimensions and for the data. DFTAG_SDU Units strings 705 displays units for all dimensions and for the data. DFTAG_SDF Formats strings 706 formats for displaying axes and data. DFTAG_SDM Max/min 2 reals 707 shows maximum and minimum values for the data. DFTAG_SDC Coordinate system string 708 shows coordinate system to be used to interpret data. DFTAG_SDLNK SDS link 8 bytes 710 links together an old-style DFTAG_SDG and a DFTAG_NDG in cases where the NDG meets all criteria for an SDG.. DFTAG_CAL Calibration info 36 bytes 731 a calibration record for the associated DFTAG_SD. DFTAG_FV Fill value ? bytes 732 value which has been used to indicate unset values in the associated DFTAG-SD. VsetTags DFTAG_VG Vgroup 14+? bytes 1965 provides general purpose grouping structure DFTAG_VH Vdata description 22+? bytes 1962 provides information necessary to process a DFTAG_VS DFTAG_VS Vdata ? bytes 1963 contains a block of data that is to be interpreted according to the information in corresponding DFTAG_VH Obsolete Tags DFTAG_ID8 Image Dimension-8 4 bytes 200 two 16-bit integers representing the width and height of an 8-bit raster image in bytes. DFTAG-IP8 Image Palette-8 768 bytes 201 a 256 x 3 byte array representing the red, green and blue elements of the 256-color palette respectively. DFTAG-RI8 Raster Image-8 xy bytes 202 a row-wise representation of the elementary 8-bit image data. DFTAG_CI8 Compressed Image-8 n bytes 203 a row-wise representation of the elementary 8-bit image data. Each row is compressed using a form of run-length encoding. DFTAG_118 IMCOMP Image-8 n bytes 204 a 4:1 compressed 8-bit image, using the IMCOMP compression scheme. DFTAG_SDG Scientific Data Group n*4 bytes 700 lists the Data identifiers (tag/ref) that describe a scientific dataset. DFTAG_SDT Transpose 0 bytes 709 indicates that data is transposed in the file. Appendix B Routine Lists C FORTRAN Name Name Purpose Raster image Routines DFR8setpalette d8spal sets the default palette to be used for subsequent images. DFR8addimage dsaimg appends the RIS8 for the image to the file. DFRSputimage d8pimg writes out the RIS8 for the image as the first image in the file. DFRSwriteref d8wref sets the reference number of the image to write next DFROgstdims dsgdims retrieves the dimensions of the image and indicates whether a palette is associated and stored with the image. DFRSgetimage dsgimg retrieves the image and any associated palette, and stores them in arrays. DFRSreadref d8rref sets the reference number of the image to get next DFRBrestart d8first gets the next get command to read from the first RIS8 in the file, rather than the next. DFR8nimages d8nims counts the number of images stored in the file (FORTRAN version currently not available) DFR8lastref d8lref returns reference number of last RIS8 read or written DF24motil d2notil sets the interlace to be used when writing out the RIS24 for the image. DF24addimage d2aimg appends the RIS24 for the image to the file. DF24putimage d2aimg writes out RIS24 for the image to the file. Dr24getdims d2gdims retrieves the dimensions and interlace of the DF24getimage d2gimg retrieves the image and stores it in an array. DF24readref d2rref sets the reference number of the image to get next DF24restart d2first causes the next get command to read from the first RIS24 in the file, rather than the next one. DF24reqil d2reqil specifies an interlace to be used in place of the interlace indicated in the file when the next raster image is read. Palette Routines DFPaddpal dpapal appends a palette to a file. DFPgetpal dpgpal reads in the next palette in the file. DFPputpal dpppal writes a palette to a file. DFPnpals dpnpals indicates number of palettes in a file. DFPwriteref dpwref sets the reference number of the next palette to be written. DFPreadref dprref sets the reference number of the next palette to be retrieved. DFPrestart dprest specifies that the next call to DFPgetpal reads the first palette in the file, rather than the next. DFPlastref dplref returns the value of the reference number most recently read or written. Annotation Routines DFANputlabel daplab puts label of tag/ref. DFANputdesc dapdesc puts description of tag/ref. DFANgetlablen dagllen gets length of label of tag/ref. DFANgetlabel daglab gets label of tag/ref. DFANgetdesclen dagdlen gets length of description of tag/ref. DFANgetdesc dagdesc gets description of tag/ref. DFANlablist dallist gets list of labels for a particular tag. DFANaddfid daafid adds file ID. DFANaddfds daafds adds file description. DFANgetfidlen dagfidl gets file ID length. DFANgetfid dagfid gets file ID. DFANgetfdslen dagfdsl gets file description length. DFANgetfds dagfds gets file description. DFANlastref dalref returns ref of last annotation read or written. Scientific Dataset Routines DFSDadddata dsadata appends the data to the file, not overwriting other file contents. DFSDputdata dspdata writes the data to a new file, truncating old file if it exists. DFSDsetNT dssnt tells which number type is to be used for next DFSDadddata or DFSDputdata DFSDsetdims dssdims sets the rank and dimension sizes for succeeding SDSs DFSDclear dsclear clears all possible set values. DFSDsetdimstrs dssdist sets label, unit, and format specifications for a dimension and its scale. DFSDsetdimscale dssdisc sets the scale for a dimension. DFSDsetdatastrs dissdast sets label, unit, and format specifications for the data. DFSDsetlengths dsslens sets maximum lengths for strings that will hold labels, units, formats, and the name of the coordinate system. DFSDsetrange dssrang sets maximum and minimum data values. DFSDsetcal dsscal sets calibration information associated with data DFSDstartslice dssslc prepares system to write part of a dataset to a file. DFSDputslice dspslc writes part of a dataset to a file. DFSDendslice dseslc indicates write completion for part of a dataset. DFSDgetdata dsgdata reads the next dataset in the file. DFSDgetNT dsgnt gets the number type of data that will be read with DFSDgetdata. DFSDgetdims dsgdims gets the number of dimensions of the dataset and the sizes of the dimensions for the next SDS in the file. DFSDgetdimstrs dsgdist reads the label, unit, and format for a dimension and its scale. DFSDgetdimscale dsgdisc reads the scale for a dimension. DFSDgetdatastrs dsgdast reads the label, unit, and format specification for the data. DFSDgetrange dsgrang reads the maximum and minimum values. DFSDgetcal dsgcal gets calibration information associated with data DFSDreadref dsrref sets the reference number of the SDS to get next DFSDrestart dsfirst sets the next get command to read from the first SDS in the file, rather than the next. DFSDgetslice dsgslc reads part of a dataset. DFSDnumber returns the number of SDSs in the file. DFSDlastref dslref returns the value of the last reference number of an SDS read from or written to the file. Utility Routines hdfls displays the tags, ref numbers, and (optionally) lengths of data elements. hdfed lets you browse in an HDF file and manipulate some of the data. fptohdf converts floating-point data to HDF floating-point data and/or 8-bit raster images. ristsds converts a series RIS8 HDF files into a single 3D SDS HDF file r8tohdf converts one or more raw 8-bit images to HDF RIS8 format and writes them to a file, possibly with palettes. hdftor8 converts images and or palettes from HDF format to raw format and stores them in two corresponding sets of files. hdfcomp compresses 8-bit raster images from an HDF file, storing them in a new HDF file. r24hdf8 converts a raw RGB 24-bit image to an 8-bit RIS8 with a palette. hdf24hdf8 quantizes a HDF RGB 24 bit `pixel' image into a 8 bit image with RGB palette and stores it as a HDF 8-bit raster image file paltohdf converts a raw palette to HDF format. hdftopal converts palette in an HDF file to raw format. hdfseq/hdfrseq displays sequences of images directly to the screen from HDF files containing raster images. hdfpack compacts and HDF file by reading in all the objects in an HDF file and writing them out to a new file.