 |
Testimony Submitted by H. Thomas Hickerson to the Committee on Government
Reform's Subcommittee on Government Management, Information, and Technology
Oversight Hearing on the National Archives and Records Administration,
October 20, 1999
Subcommittee Chairman: Congressman Stephen Horn
H. Thomas Hickerson is President of the Society of American Archivists and
Associate University Librarian for Information Technology and Special Collections
at Cornell University
It is an honor and a pleasure to have this opportunity to appear here today
to provide testimony regarding issues critical to the National Archives and
Records Administration. While I will address several general issues, I will,
as requested, focus my remarks on the application of digital technologies and
particularly on the management of electronic records. My comments will reflect
three areas of experience:
(1) My experience of nearly thirty years' involvement in archival
practice and extensive professional leadership, including my present service
as President of the Society of American Archivists;
(2) My direction at Cornell University of the principal archival and rare book
programs, my development over the last eight years of an institute at Cornell
dedicated to the building of digital collections based on cultural and scientific
holdings, and my present responsibility for library information technologies
at Cornell; and
(3) My United States citizenship. While the first two of these provide the
basis for my authority and expertise in these proceedings, my citizenship also
provides a strong incentive for my the interest in the successful operation
of the National Archives and Records Administration (NARA).
In recognizing the importance of NARA for every U.S. citizen, we have only
to look at recent experience in Kosovo to see that invading forces sought to
systematically destroy land, financial, citizenship, and genealogical records
in an effort to destroy economic and political rights and community and cultural
identity. In the United States, responsibility for maintaining the archival
record is broadly distributed among state and municipal archives, university,
corporate, and religious repositories, research libraries, and historical societies
and museums, but no institution other than the National Archives is so central
and fundamental to the rights of every citizen and to the process of democratic
governance. That we dedicate only $200 million annually to this large, complex,
and vital undertaking is, on one hand, regrettable, and on the other, a rather
remarkable bargain. For I do feel that in spite of the extent of the challenge,
overall our National Archives has served us well. But it could have done more
in the past, and it must do more in the coming years. The future of the archival
record is at a critical juncture. Congressman Horn, I urge that you and your
colleagues vigorously assist in this critical transition.
We are living in a time of dramatic and continual change, both large and small.
Against a seemingly constantly evolving tableau, the dominant transformation
of our time, the information revolution, is taking place. This revolution focuses
on information creation, dissemination, use, management, storage, and preservation.
As a result, archivists are facing some of the most vexing challenges of modern-day
information management, confronting issues essential to government, corporations,
institutions, and organizations in operating effectively and in fulfilling
their legal requirements. At the same time, archivists continue to be dedicated
to preserving and supporting the use of large existing collections documenting
historic achievements, social and industrial development, and the experience
of everyday life.
As we enter the new century, the pace of change and a growing sense of the
seeming fragility of the human record have served to significantly expand the
appeal of original materials, as well as the interest in access to digital
facsimiles. Museums and other cultural repositories are enjoying extraordinary
growth in attendance. In Texas, for example, the Johnson Presidential Library
is second in number of visitors only to the Alamo, itself an historic site
and museum. Thus, we are faced with the paradox of society's increased interest
in historical documents, images, and objects, both in artifactual form and
in digital representations, while current records are increasingly generated
in electronic form, and e-mail, e-commerce, and Web sites are the predominant
means of written communication. In reviewing the mandate and priorities of
the National Archives, we must keep in mind this confluence of pervasive interest
in our documentary heritage and of transformative changes taking place worldwide.
Ideally, an integrated continuum will be established between the records and
services of this century and those of the next.
In my specific comments, I will concentrate on three aspects of NARA's mission
that I feel are central to their success in the next decade. The first is managing
records generated in electronic form. The second is leadership in innovative
applications of new technologies. The third is the need to extend services
to users and broaden the value of the nation's archives for the American public.
Although this third topic is more general, it is related to the other two.
While I will express significant concern regarding progress on these issues,
I will also emphasize my belief that significant change is underway in all
three areas.
Managing Electronic Records
In a report of the House Committee on Government Operations, "Taking a Byte
out of History: The Archival Preservation of Federal Computer Records," submitted
November 6, 1990, many of the difficulties inherent in the selection, preservation,
and use of electronic records over time were clearly identified. Though the
nature of the problem and its importance were perceptively stated, recommended
actions were explorative in approach rather than action-oriented. No new research
was funded; no new programs inaugurated. It is now nearly a decade later, and
there is not yet a scalable working model in place for realistically addressing
these issues.
While I feel strongly that NARA has been slow to dedicate the necessary resources
to this challenge, others have also lagged. We are all a decade behind, and
we are only now beginning to confront the issues of long-term preservation
and use of digital information in a serious fashion. It is not surprising that
the technology industry has not focused attention on the impermanence of digital
information, and when they have, they have talked principally about the lifespan
of particular media, such as magnetic tape or CD-ROMs. While media permanence
is important, it is not the principal challenge. If the bits survive, will
we continue to be able to read them when hardware and software generations
come and go with increasing speed? We must decide at the point of data generation
which information should be retained and usable over time and design a path
for migrating those records from one software and hardware generation to the
next. At this point, we should not feel secure that the necessary procedures
are in place.
The need for research and testing of methods of migrating information from
one technology generation to the next or the development of other means of
retaining the capacity to use today's information tomorrow is urgent. Government-funded
research has not yet made this issue a priority.
Of the six projects selected four years ago in the first phase of the National
Science Foundation's (NSF) Digital Library Initiative, none highlighted this
issue. In the recently completed Digital Library Initiative Phase 2 competition,
in which NSF was joined by several additional federal partners, including active
participation by the National Endowment for the Humanities (NEH) and the Library
of Congress, greater attention was directed to long-term viability. Nonetheless,
out of thirty-three projects funded, it appears that only two projects, those
at the University of Michigan and at Cornell University, are focused on preservation
issues. In describing the funding of the Cornell project, a joint project of
the Cornell Library and the Computer Science Department, William Ferris, Chairman
of the National Endowment for the Humanities, reported, "NEH, the National
Science Foundation and other federal agencies have begun the process by funding
a pioneering, $2.3 million preservation project at Cornell University. This
project will develop a standard way of organizing computerized collections,
preventing data loss in these collections by alerting managers to the periodic
need to upgrade aging CD-ROMs and tapes, and making the collections fully accessible
on the Internet. All Americans will benefit because the project will ensure
that computerized materials important for the study of America will be preserved
and accessible for generations to come." I would like to make two responses
to these inspiring remarks. First, while I appreciate Bill Ferris' words of
confidence, Cornell's project will only contribute in generating a viable solution
to this momentous problem, and second, while he is correct when he says that
it is pioneering effort, it should not be. We are all way behind the curve
on this issue.
Regrettably, the corporate and institutional sectors do not yet seem to have
made significant steps forward either. In part, this is due to the reluctance
of the technology industry to bring attention to this issue. Perhaps more importantly,
however, it is because of a division of responsibility between those responsible
for paper records, frequently corporate and institutional archivists, and those
responsible for computing systems, data processing professionals. System designers
and programmers have seldom reflected an archival viewpoint, and now that records
are frequently available in digital form only, this division of management
and perspective will have significant repercussions. I foresee the potential
for a 500,000-person sub-industry developing around this issue, and a significant
number of those will be archivists, equipped with new skills but embodying
traditional archival knowledge and values. The Society of American Archivists
has been offering electronic records workshops since the 1980s, and a new distance
learning course is so heavily subscribed that we are now taking applications
for next year.
While NARA has a long record of active involvement with the management of
electronic records, this responsibility must now become a priority in the allocation
of resources within the agency. This change and others basic to the new digital
environment may be traumatic, but they are necessary. Applied research will
be important within NARA, but viable solutions will only be developed and implemented
through partnerships with other agencies and with academic, corporate, and
professional partners. I am very impressed by recent NARA initiatives of this
nature. The Collection-Based Long-Term Preservation Project at the San Diego
Supercomputer Center is an outstanding example. Scientists at the Center are
working with several federal agencies to develop and test means of preserving
the organization of digital collections simultaneously with the digital objects
that comprise the collection. NARA is also actively involved in the InterPares
Project (International Research on Permanent Authentic Records Electronic Systems),
an international theoretical and methodological research project. This project
highlights the global nature of this issue. It is supported by funds from agencies
in several different countries, including the National Historical Publications
and Records Commission, a small funding agency located within NARA that has
proven vital to research and development in this area. NARA's involvement in
collaborative efforts is essential, and the National Archives should maintain
an open professional dialogue regarding successes and failures. Knowledge of
their experience will benefit archivists worldwide.
In closing my comments on this issue, I want to emphasize that solving this
issue is not just a technological one. It is also a political, social, organizational,
and economic issue as well. And it doesn't just require more and differently
allocated resources for NARA. An example is the planned retention of data in
the individual responses generated by the Year 2000 Census. My understanding
is that current plans are for the transfer through optical character recognition
(OCR) of the information on the forms at a 98% accuracy rate and storage of
that information in ASCII (a basic standard for recording electronic data)
on magnetic tapes for retention by NARA. Alternatives would be to create digital
image copies of each form and/or create microform copies. It seems that the
present choice is based on the project contractor's projected costs. While
the technical suitability of each option is deserving of professional investigation,
we must also factor usability by those citizens most affected by our choices
into such decisions.
I have raised the issue of the Census only as an example of the need for various
criteria to be considered in making these technical decisions. In developing
solutions to this critical challenge, we must balance our interest in technical
efficiency with the requirements necessary for effective governmental and citizen
use over time. Our models must incorporate these factors effectively, and we
are already a decade late in implementing them. We are losing valuable information
today, and more will be lost tomorrow. This is the Y2K that will not go away
next year.
Innovative Use of Technology
In the 1970s, when I first began to experiment in the use of computing technologies
for archival management, I employed a software package called SPINDEX II (Selective
Permutation INDEXing). This software, though based on a precursor created at
the Library of Congress, was developed and maintained by the National Archives.
A later version was used by Cornell to build a database describing archives
and manuscript held by some 1,100 repositories across New York state. In the
mid-80s, this information was transferred into the Research Libraries Information
Network, an online network that is now the principal international catalog
for archival holdings.
In developing and testing SPINDEX, NARA established a partnership of ten institutions,
including state, federal, university, and corporate repositories. For those
of us who began with SPINDEX, this experience and NARA's leadership were very
important. In the early 1980s, however, NARA turned inward in its systems development
efforts. At a time when many repositories were adopting common cataloging standards
that facilitated the use of existing systems and online access to research
information, NARA chose not to adopt these standards. Asserting the unique
requirements of the National Archives and refusing to modify existing practice,
NARA developed multiple, mutually incompatible systems in-house. To my knowledge,
none of these systems have survived, and the goal of SPINDEX development twenty-
five years earlier, providing automated access to summary descriptions of all
NARA holdings is yet to be realized.
The National Archives has now again embarked on an initiative to provide comprehensive
access to cataloging for all NARA holdings. While I still applaud this goal,
I am concerned that they have chosen a British system not widely employed in
this country. I am not presently able to evaluate the basis of their selection.
I have, however, just completed directing the last stages of a four-year selection
process to choose a new management system for the Cornell Library. It happens
that the system chosen by Cornell has also recently been chosen in exhaustive
competitions at the Library of Congress and the National Library of Medicine.
I am not suggesting that this system would be ideal for NARA, and I readily
acknowledge their differences from these other institutions. Nonetheless, in
this age of system interoperability and Internet access, I fear that unique
internal needs may be guiding their choices when the ability to provide easy
access for agency staff, researchers, and the public should be paramount. The
intent of my comments is not to urge use of particular software, but to emphasize
that common standards and solutions developed and applied in cooperation with
other agencies and institutions are required for success in today's information
environment.
My fear that paper-based management and service procedures are still dominating
strategic policy is further heightened by the present decision to delay further
production of digital copies of NARA holdings for public access via the Internet.
As John Carlin has explained to me personally, NARA has chosen to focus on
the complexities of electronic records, those originally generated in electronic
form, rather than to devote present resources to the creation of digital facsimiles
of existing materials via scanners or digital cameras. While I understand the
basis of his choice, I must emphasize that the fundamental nature of access
to information is changing, and that users expect the availability of both
information created in digital form and distinctive holdings copied digitally.
I am not suggesting that NARA will ever convert the majority of its existing
holdings to digital form, but the effort by the Library of Congress' American
Memory Project to build a virtual collection of 5,000,000 images is broadly
perceived as an outstanding success. The Cornell University Library has nearly
2,000,000 images from historical, artistic, and scientific collections available,
and a recent survey conducted by the Association of Research Libraries, some
120 of the largest research libraries in North America, found that over 90%
of their members were presently conducting or planning projects to digitally
convert unique holdings. I do not believe that NARA's decision to suspend conversion
efforts at roughly 122,000 documents and visual images is in the best interests
of the National Archives nor its global public. Our archives should be available
in classrooms, lecture halls, libraries, offices, and homes, as well as everywhere
else in a wireless world.
As the closing paragraph in the introduction of Newweek's October 11, 1999
section on "e-Life" explains, "We're at the beginning of a new way of working,
shopping, playing, and communicating. At Newsweek, we're calling this phenomenon
e-life, and it's just in time. Because the day is approaching when no one will
describe the digital, Net-based, computer-connected gestalt with such a transitory
term. We'll just call it life." Our nation's archives have to be part of that
life.
Extending Services and Broadening the Value of the Nation's Archives
This third topic is very closely related to the preceding discussion of broadening
use of NARA's holdings and services via new technologies. A similar interest
must be employed to make the experience of the National Archives compelling
for those onsite. I am absolutely thrilled by Congressional support for the
renovation of the National Archives Building on the Capitol Mall. The planned
renovation will dramatically improve storage conditions for records housed
there, and it will provide a state-of-the-art technology infrastructure for
both staff and researchers. Both of these improvements are critical and long,
long overdue, but I must admit that I am most excited by the plans to implement
a new concept in the display of the Charters of Freedom, the Declaration of
Independence, the Constitution, and the Bill of Rights, and in other exhibition
areas throughout the building. As designed, these displays will heightened
the significance and level of engagement of viewers through carefully realized
pictorial presentations. The planned theater offers the opportunity for new
multi-media presentations, similar to those that can be made available through
the Web. Although I am aware that private fundraising is required to complete
this effort, I urge your fullest support. I applaud the imagination and vision
of John Carlin and his colleagues in designing such a wondrous home for these
remarkable documents. I think that it will generate a new spirit among visitors,
researchers, and staff.
Concluding Remarks
In conclusion, I would like to express on behalf of the Society of American
Archivists my warm appreciation to John Carlin for his efforts to develop a
cordial and synergistic relationship between NARA and the Society. Mutually
beneficial collaborations have developed, and I am confident that our cooperation
will grow. The archival profession needs a strong National Archives. I believe
that John Carlin is providing effective leadership in confronting current challenges.
I urge your support of his efforts and greatly appreciate the opportunity to
address you today regarding the future of this distinguished institution so
important to each of us.
|
|
