The U.S. National Archives joins the Commons!, Flickr blog, (February 1, 2010).

Please welcome the U.S. National Archives to The Commons, the world’s public photography archives on Flickr to which you can contribute information and knowledge.

With over 3,000 images in 49 sets uploaded already, perusing these important archival images should keep you entertained for a long time. Their four collections encompass important Americana, ranging from the famous Mathew Brady Civil War images to historical and iconic images of American history.

What are the impacts of the government using social networks and other commercial services to communicate with the public and for internal communications? What are the impacts of what is recorded at all when staff knows that their words will be publicly available later? These and other issues are covered in a fascinating overview of the Presidential Records Act:

• Why Did the Bush White House Copy and Print Every Email It Received? It's a Long Story, By Nneoma Amadi-obi, History News Network (Jan. 4, 2010).

There are interesting and informed comments to this article, including two (so far) by the always insightful Maarja Krusten, who worked for years in NARA's Office of Presidential Libraries.

Carl Malamud posed this question over on twitter: "What if our national cultural institutions all worked together on a common problem, attracted White House support?" In his post on the O'Reilly blog, "A National Scan Center: A Public Works Project", Malamud scopes out the issues and calls for Library of Congress, the Smithsonian Institution, the Government Printing Office, the National Archives and Records Administration, and the National Technical Information Service to come together and make the compelling case for funding a 5-year $500 million effort to create a National Scan Center. Here here Carl!

In the U.S., we face a similar deluge of paperwork that we faced in the 1930s. A huge backlog of paper, microfiche, audio, video, and other materials is located throughout the federal government. Little money has gone from Congress for digitization, and bureaucracies have resorted to a series of questionable private-public partnerships as a way of digitizing their materials. For example, the Government Accountability Office shipped 60 million pages of our Federal Legislative Histories (the record of each law from the initial bill through the hearings and conference reports) off to Thomson West, but didn't even get digital copies back. Another example is the recent failed effort by the Government Printing Office to digitize 60 million pages of the Federal Depository Library Program, an effort they tried to get through as a "zero dollar cost to the government" effort with the private sector.

There are no free lunches and there are no "no cost to the government" deals. The costs involve the government effort to supervise the contract, prepare the materials, and ship them, and in both the GAO and GPO cases, the government wasn't getting much back for its effort. What the government and the people usually get is a lien on the public domain, preventing the public from accessing these vital materials. Similar efforts are sprinkled throughout the government. I testified to Congress that I had learned that the National Archives was contemplating a scan of congressional hearings with LexisNexis under similar circumstances, and many may be aware of the questionable deal the Archives cut with Amazon where my favorite online superstore got de facto exclusive rights to 1,899 wonderful pieces of video.

Carl Malamud's FedFlix project is a joint venture with the National Technical Information Service (NTIS) whereby he takes NTIS videos, digitizes them and uploads them to the Internet Archive.

Well now he's expanding FedFlix to include public domain videos from the National Archives. He's released 41 videos into the public domain in this way, but has put together an Amazon Wish List in order to expand public access to public domain video content from the National Archives. If you see anything you'd like to buy the public domain, they'll take your DVD and upload the video to YouTube, the Internet Archive, and to public.resource.org's own rsync/ftp public domain stock footage library. So why not add a gift of the public domain to your favorite person's/people's stockings this year? We'll all be glad we did!

UPDATE 12/25/09: The wish list has been fulfilled. You can watch all of the donated NARA videos on YouTube, Internet Archive, or public.resource.org's bulk server. Thanks Carl!

[HT BoingBoing!]

NARA Nears Completion of Ingesting Bush Records, National Coalition for History (Sept 4, 2009).

The National Archives and Records Administration announced this week that the agency has nearly completed the process of loading the electronic records of President George W. Bush into the Electronic Records Archives (ERA) system. To date, more than 85% of the total volume has been ingested.

...The intake of these records is expected to be complete by the end of September, 2009. Currently, the use of the system is limited to Presidential Libraries staff...

See also: Archives reports progress on road to digital ERA, by Max Cacas, Federal News Radio, (Sept. 3, 2009).

The National Archives has started a blog, NARAtions, because, "we are hoping to talk with you about online public access to the records held by the U.S. National Archives and Records Administration (NARA)."

(Hat tip to Kate!)

President Obama Nominates David S. Ferriero To Be Archivist of the U.S. (updated), National Coalition for History (July 28, 2009).

On July 28, President Obama announced his intent to nominate David S. Ferriero to the position of Archivist of the United States. Mr. Ferriero currently serves as the Andrew W. Mellon Director of the New York Public Libraries (NYPL). Mr. Ferriero succeeds Professor Allen Weinstein who resigned as Archivist last December.

The National Archives (NARA) has just released a draft of their 2009 strategic plan entitled, Preserving the Past to Protect the Future. It details NARA's goals for the next 10 years and offers an interesting window into the agency's focus. They are asking for public feedback by August 5, 2009 via e-mail (vision@nara.gov) or by fax (301) 837-0319.

Especially read the section, "how will we know we have succeeded." Some of the benchmarks for success are achievable and admirable (i.e., "Within 30 days of the end of an administration, 100 percent of Presidential and Vice Presidential materials have been moved to NARA locations or NARA-approved facilities." and "By 2014, 100 percent of NARA records center holdings are stored in appropriate space.") but it's glaring that the first item of "success" is that "by 2016, 50 percent of agencies achieve passing scores for compliance with Federal records management policy." That seems like an extremely low benchmark to this non-archivist. Shouldn't ALL agencies already be compliant?

• Goal 1: Our Nation’s Record Keeper
• Goal 2: Preserve and Process
• Goal 3: Electronic Records
• Goal 4: Access
• Goal 5: Civic Literacy
• Goal 6: Infrastructure

...And the increasing demand by users for more online access to records has driven us to prioritize the processing of a backlog of records and to focus on increasing our digitization efforts.

Our holdings in all media—paper, electronic, film, and so on—are constantly growing as the proliferation of Government records continues. At the same time, the public demand for access is increasing. NARA has a significant backlog of unprocessed holdings that are therefore not yet readily available to the public. New records are arriving faster than they can be processed. Clearly, we must refocus attention and resources on making as many records as possible accessible to the public.

While we put great effort into readying records for public use, the expectation of easy online access to our holdings continues to grow. The American public expects information to be delivered almost instantly to their desktop with more and more information available via the Internet. However, we cannot provide online access to all our holdings in the next decade—the task is simply too big. Our focus must be on making our most requested holdings available online, and on providing researchers with online tools to help them in their work.

Because of work done every day in NARA facilities across the nation, the public can examine the records that document the actions of Government officials, the entitlements of individuals, the events that make up American history, and in some instances, how those events have affected the rest of the world. With new social media applications (Web 2.0), we have the opportunity to reach the public with tools they already use. These tools allow us to operate more transparently and to interact with our users in new, more informal ways.

Our primary response to the challenge of authentically preserving electronic records and providing access to these records in the future is the Electronic Records Archives (ERA). ERA will provide a comprehensive, systematic, and dynamic means of preserving and providing continuing access to authentic electronic records over time. When fully deployed, ERA will give us the means to preserve, process, and provide continuing access to Federal Government electronic records of archival value. Yet, at the same time, we must explore ways to provide economical storage and retrieval services for electronic records that remain under the legal control of the originating agencies. We must also continue to adapt our internal business processes and staff expertise to effectively use the ERA system and to ensure that it evolves over time to fit tomorrow’s technology.

The Internet has introduced countless researchers to the holdings of the National Archives. This thirst for online access to information is exciting, as it brings more and more people to our virtual doors, yet the task of building an “archives without walls” is daunting. Implicit in this task is the ability for archivists and the public to virtually interact. Our strategies in this area will focus on developing dynamic partnerships to digitize and deliver our most popular holdings and create interactive tools to help researchers find the information they seek. By putting our country’s records literally at the public’s fingertips, we will be able to share our rich resources with more people than ever before.

[Thanks to John Wonderlich and the Open House Project for pointing this one out!]

www.youtube.com/USNationalArchives

Thanks and a tip of the hat to Kate.

The National Security Archive at George Washington University has an overview article with good links on recent hearings focusing on the National Archives.

• Archive Testifies to House Oversight Committee About Challenges Facing National Archives, "Recommends New Archivist Have a Vision for Archives 2.0" (May 21, 2009)

Witnesses at the hearing included National Security Archive General Counsel Meredith Fuchs, Patrice McDermott, director of OpenTheGovernment.org, and Lee White, executive director of the National Coalition for History.

Update: More information on the hearings here: NARA's preservation practices for e-records called 'fatally flawed', By Jill R. Aitoro (05/21/2009).

Thanks to Carl Malamud for the heads-up about this new article by Fred Kaplan at Slate entitled "PowerPoint to the People: The urgent need to fix federal archiving policies." Basically, according to Kaplan, the National Archives (NARA) are a mess. "Electronic records ... are generally not disposed of in accordance with federal regulations," and NARA is "still unable to accept M$ Word docs and PowerPoint slides." This endemic problem has been known for a long time. Kaplan links to a 2003 NARA study titled "Records Maintenance and Disposition in Headquarters Air Force Offices," (report written in January 2005) that had been FOIA'd -- oddly all of the recommendations were blacked out?! This shows that government agencies are not following protocols for digital records management and the agencies charged with preservation are not doing enough to either build solid preservation systems or work with agencies to help them get into the 21st century. Is this really the "end of history?"

["retweeted" from Carl malamud!]

All the President's IMs: Are Federal Record-keeping Laws Out of Step With Modern Communications?, by MICHAEL C. DORF, FindLaw, Jan. 12, 2009.

Dorf argues that our federal record-keeping laws are out of step with the ways in which people now communicate.

NARA Cannot Assure Complete Transfer of Bush Records, by Steven Aftergood, Secrecy News, January 5, 2009.

Steven has a link to “National Archives Oversight: Protecting Our Nation’s History for Future Generations,” hearing before the Senate Committee on Homeland Security and Governmental Affairs, May 14, 2008 and comments on the the integrity of the process of transferring the records to the National Archives.

Although the President is supposed to obtain the written views of the Archivist prior to any proposed destruction of non-permanent records, "the final disposal authority rests with the incumbent president… regardless of the Archivist’s views."

The New York Times reports today on the problems the National Archives faces in acquiring, organizing, managing, preserving and making available the records of the Bush White House.

• Bush Data Threatens to Overload Archives, By Robert Pear And Scott Shane, New York Times, December 27, 2008.

The National Archives has put into effect an emergency plan to handle electronic records from the Bush White House amid growing doubts about whether its new $144 million computer system can cope with the vast quantities of digital data it will receive when President Bush leaves office on Jan. 20.

Among the problems NARA faces? Volume: NARA anticipates getting 100 terabytes of data 50 times the what they got from the Clinton White House. This is the equivalent of five times the contents of all 20 million catalogued books in the Library of Congress.

Cooperation: "Millions of White House e-mail messages created from 2003 to 2005 appear to be missing and may not be recoverable. And in September 2007, the top lawyer at the National Archives wrote in a memorandum that he had 'made almost zero progress' planning the transition because the White House had ignored repeated requests for information about the volume and formats of electronic records." In addition, Vice-President Cheney's lawyers claimed in a court filing that neither NARA nor the court "may supervise the vice president or his office" for compliance with the Presidential Records Act.

Formats: NARA says that there are a large numbers of White House records created with proprietary commercial software.

Access: Paul Brachfeld, the archives' inspector general, said "The electronic records archives system may be able to take in a tremendous amount of e-mail and other records.... But just because you ingest the data does not mean that people can locate, identify, recover and use the records they need."

Bush E-Mails May Be Secret a Bit Longer, by R. Jeffrey Smith, Washington Post, December 21, 2008; A01.

Legal Battles, Technical Difficulties Delay Required Transfer to Archives...

The required transfer in four weeks of all of the Bush White House's electronic mail messages and documents to the National Archives has been imperiled by a combination of technical glitches, lawsuits and lagging computer forensic work, according to government officials, historians and lawyers.

...The risks that the transfer may be incomplete are also pointed up by a continuing legal battle between a coalition of historians and nonprofit groups over access to Vice President Cheney's records. The coalition is contesting the administration's assertion in federal court this month that he "alone may determine what constitutes vice presidential records or personal records" and "how his records will be created, maintained, managed, and disposed," without outside challenge or judicial review.

...The National Archives and Records Administration is supposed to help monitor the completeness of the historical record but has no enforcement powers over White House records management practices.