Thanks to Carl Malamud for the heads-up about this new article by Fred Kaplan at Slate entitled "PowerPoint to the People: The urgent need to fix federal archiving policies." Basically, according to Kaplan, the National Archives (NARA) are a mess. "Electronic records ... are generally not disposed of in accordance with federal regulations," and NARA is "still unable to accept M$ Word docs and PowerPoint slides." This endemic problem has been known for a long time. Kaplan links to a 2003 NARA study titled "Records Maintenance and Disposition in Headquarters Air Force Offices," (report written in January 2005) that had been FOIA'd -- oddly all of the recommendations were blacked out?! This shows that government agencies are not following protocols for digital records management and the agencies charged with preservation are not doing enough to either build solid preservation systems or work with agencies to help them get into the 21st century. Is this really the "end of history?"
["retweeted" from Carl malamud!]
All the President's IMs: Are Federal Record-keeping Laws Out of Step With Modern Communications?, by MICHAEL C. DORF, FindLaw, Jan. 12, 2009.
Dorf argues that our federal record-keeping laws are out of step with the ways in which people now communicate.
NARA Cannot Assure Complete Transfer of Bush Records, by Steven Aftergood, Secrecy News, January 5, 2009.
Steven has a link to “National Archives Oversight: Protecting Our Nation’s History for Future Generations,” hearing before the Senate Committee on Homeland Security and Governmental Affairs, May 14, 2008 and comments on the the integrity of the process of transferring the records to the National Archives.
Although the President is supposed to obtain the written views of the Archivist prior to any proposed destruction of non-permanent records, "the final disposal authority rests with the incumbent president… regardless of the Archivist’s views."
The New York Times reports today on the problems the National Archives faces in acquiring, organizing, managing, preserving and making available the records of the Bush White House.
- Bush Data Threatens to Overload Archives, By Robert Pear And Scott Shane, New York Times, December 27, 2008.
The National Archives has put into effect an emergency plan to handle electronic records from the Bush White House amid growing doubts about whether its new $144 million computer system can cope with the vast quantities of digital data it will receive when President Bush leaves office on Jan. 20.
Among the problems NARA faces? Volume: NARA anticipates getting 100 terabytes of data 50 times the what they got from the Clinton White House. This is the equivalent of five times the contents of all 20 million catalogued books in the Library of Congress.
Cooperation: "Millions of White House e-mail messages created from 2003 to 2005 appear to be missing and may not be recoverable. And in September 2007, the top lawyer at the National Archives wrote in a memorandum that he had 'made almost zero progress' planning the transition because the White House had ignored repeated requests for information about the volume and formats of electronic records." In addition, Vice-President Cheney's lawyers claimed in a court filing that neither NARA nor the court "may supervise the vice president or his office" for compliance with the Presidential Records Act.
Formats: NARA says that there are a large numbers of White House records created with proprietary commercial software.
Access: Paul Brachfeld, the archives' inspector general, said "The electronic records archives system may be able to take in a tremendous amount of e-mail and other records.... But just because you ingest the data does not mean that people can locate, identify, recover and use the records they need."
Bush E-Mails May Be Secret a Bit Longer, by R. Jeffrey Smith, Washington Post, December 21, 2008; A01.
Legal Battles, Technical Difficulties Delay Required Transfer to Archives...
The required transfer in four weeks of all of the Bush White House's electronic mail messages and documents to the National Archives has been imperiled by a combination of technical glitches, lawsuits and lagging computer forensic work, according to government officials, historians and lawyers.
...The risks that the transfer may be incomplete are also pointed up by a continuing legal battle between a coalition of historians and nonprofit groups over access to Vice President Cheney's records. The coalition is contesting the administration's assertion in federal court this month that he "alone may determine what constitutes vice presidential records or personal records" and "how his records will be created, maintained, managed, and disposed," without outside challenge or judicial review.
...The National Archives and Records Administration is supposed to help monitor the completeness of the historical record but has no enforcement powers over White House records management practices.
Following up on the report about NARA's Executive Office of the President (EOP) Electronic Records Archives (ERA) (http://freegovinfo.info/node/2178), here is a story that goes in to more detail, much of it quite scary.
- Bush's exit to put new e-records system to the test, By Mary Brandel, Computerworld, November 21, 2008.
Deb Logan, an analyst at Gartner Inc., says in the article that the onerous task will be sorting through the unclassified and unprocessed data that the Bush administration will leave behind. "It would be one thing if the stuff had to be moved seamlessly to a records repository, but it's just eight years of stuff," she says. "It will be nearly impossible to get it under control without a massive expenditure of human resources because the technology is not there."
This is scary when one considers that it took NARA over a year to process the 2 Terabytes of data from the Clinton administration and NARA expects 140 Terabytes of data from the Bush administration. Even given that NARA now has a system in place that did not exist for processing the Clinton data, there is a big difference between the dealing with the technology issues and the information management issues:
Logan says part of the blame lies with federal agencies themselves, pointing to a GAO survey that concluded federal agencies have failed across the board to fulfill their records management obligations, "not out of malice or neglect but out of the nature of the volume of electronic communications and the time frame in which they have to do it," she says. "Anyone who's putting an optimistic face on the job is not being realistic."
Optimism may be relevant from a technology point of view, Thibodeau acknowledges, but not from an information management point of view. "From my side of NARA, I don't deal with what's in the records, just whether we can get them into the system," he notes. "We allow the library staff to deal with the content."
...Logan says the problem of managing electronic records won't be resolved until the government agencies themselves do a better job of electronic records management, including classifying, de-duplicating and purging data through the use of systems such as archiving, records and policy management, content monitoring/filtering, and content analytics tools.
A GAO report says, "NARA believes that if it cannot ingest the Bush records in a way that supports search and retrieval immediately after the transition, it may not be able to effectively respond to requests from the Congress, the new administration, and the courts for these records—a critical agency mission." (The National Archives and Records Administration’s Fiscal Year 2008 Expenditure Plan, September 2008, GAO-08-1105.)
The National Coalition for History has a report on the meeting of the Public Interest Declassification Board (PIDB) in November where there were discussions of the declassification, archival processing, and release of the 9/11 Commission records and the National Archives and Records Administration's Electronic Records Archives (ERA):
- Electronic Records Archives & 9/11 Commission Records Update, National Coalition for History, November 26th, 2008.
Two interesting tidbits:
Ken Thibodeau, Director of NARA’s Electronic Records Archive (ERA), reported that four federal agencies are currently using ERA system and that all federal agencies should be using the ERA by January 2011.
Thibodeau also reported on the upcoming transfer of the electronic records from the Bush administration to the National Archives. NARA is developing a separate system, known as the Executive Office of the President (EOP) ERA, to ingest an estimated 100 terabytes of electronic records from the Bush administration.
For those of you following the saga of Anthony Clark and his attempt to get access to NARA records about Presidential Libraries (See: How hard is it to get NARA records about NARA? and Getting Documents from NARA about NARA - UPDATE), our friend Kate over at ArchivesNext.com has researched the issue and written a good article about the issues. Thanks Kate!
- NARA's Management of Presidential Library Records: Revelations and Results, by Kate Theimer, The Records Manager: The Newsletter of The Society of American Archivists Records Management Roundtable. Vol 3, No. 1 (November 2008), pp 24-25.
[Cross posted on LegalResearchPlus]
About one month ago, I posted an item about the difficulty of getting documents about NARA from NARA -- the entry was based on an article written by Anthony Clark (Why Is It So Hard to Get Documents from the National Archives About the National Archives?, History News Network, July 21, 2008).
Here is an update to this very interesting story. On the Archivists' Forum, there is a recent entry from Anthony Clark detailing the latest ups and (mostly) downs of this saga.
"Some readers may know that I have had great difficulty accessing NARA's own records for my research into presidential libraries and NL, or NARA's Office of Presidential Libraries (see http://hnn.us/articles/52350.html) for more information). What you might not know is that in July NARA offered me a deal" - if I dropped all of my pending FOIA requests for NL's records, they would commit to systematically process all of NL's records - some 230 boxes - at a rate of nine boxes per month, until all boxes have been processed and made available. Just a few weeks later, not only did NARA "take back" part of that offer (while claiming it was never made), they have now reneged on it completely. I was so shocked by what NARA did today that I felt I had to make the list aware of what they had done. [Full details available on the Forum page.]"
As I hear more about this NARA-tive, I'll be sure to pass it along.
Anthony Clark, an independent researcher writing a book on the politics and history of presidential libraries, has written a provocative piece on access to National Archives and Records Administration (NARA) records about administration of Presidential Libraries:
- Why Is It So Hard to Get Documents from the National Archives About the National Archives?, By Anthony Clark, History News Network, July 21, 2008.
Clark claims that NARA is "improperly withholding its own records." He says that as part of NARA's job of overseeing the twelve presidential libraries, it has records that detail the development of the libraries through 1964, when NARA created the Office of Presidential Libraries (NL), but none of NL's records are available. NARA is calling these records "operational," which makes them available only through Freedom of Information Act (FOIA) requests.
Clark quotes Patrice McDermott, Director of OpenTheGovernment.org, as saying, "It is hard to understand how records that are old enough to have been destroyed if the records schedule had been followed can be considered 'operational.' Presidential libraries are an area of keen congressional and public interest and information about them held by NARA should be affirmatively disclosed to the greatest extent possible."
Kate at ArchivesNext has posted a thoughtful response after talking off the record to archives staff: Access to records of the National Archives, July 24th, 2008.