Millions and Millions of Government and Military Web Pages Archived by NARA and The IA

Last year we posted a note on ResourceShelf about the “2004 Presidential Term Web Harvest” containing more than 75 million .Gov and .Mil web pages, equal to about 6.5 terabytes of data. It's a project of NARA and The Internet Archive. The archived sites can be browsed or keyword searched. Now available is the 109th Web Harvest. What does it contain? + More than four million pages (42 GB) crawled and archived between 11/11/06 and 12/11/06 + Browse by Members Name + Browse by Committee Name + Browse by Leadership + Browse by House or Senate Organizations Go to: http://www.webharvest.gov/collections/

The harvest produced a public reference copy of the web sites for the purpose of continual availability to the public, and also produced a record copy to be retained in the holdings of NARA…Web sites included in the harvest were identified from information provided by the Web Systems Branch of the House Information Resources staff and by Senate webmasters in the Offices of the Secretary of the Senate and the Sergeant at Arms.
A bit more on ResourceShelf including a comment by Librarian of Congress, James Billington, about the average lifespan of a web site. Continue reading

Continue Reading →

GPO, LOCKSS, IP Authentication, and the future of FDLP — more clarification needed

If you have not had a chance to read the message from Joseph P. Paskoski (Clarification on GPO LOCKSS report), I encourage you to do so. It does indeed help clarify GPO's intentions in ways that, I believe, seriously endanger long-term, free, public access to government information.

For those confused by the recent thread about GPO, LOCKSS, and IP authentication, allow me to try to summarize what we now know:

GPO is not "advocating" use of IP authentication for LOCKSS.

On the other hand, GPO is considering "an exclusive service for depository libraries" and is recommending exploring "other user authentication options" to implement such a system.

Further, GPO is only willing to "consider" (not guarantee) making content available without user authentication. This is evidently true of FDsys as well as any use of LOCKSS.

To me, this means that GPO is, indeed, planning a two-tier system of digital distribution: one exclusively for depository libraries (and, presumably, free) and, by implication, a second system presumably for the general public and based on cost recovery.

For this to work, GPO would have to do two things. First, it would have to restrict what FDLP libraries can do with the content they receive, either through technological locks or limitations, or licensing restrictions (including restrictions on re-distribution). Second, if GPO offered any content to the general public for free, it would have to offer similarly technologically dumbed-down, less-than-fully-functional, non-reusable content -- much the way Amazon offers one-page-at-a-time viewing of books as a teaser to get you to purchase the entire book. (For more on this see Why does GPO want to use IP Authentication?)

This model seems clear: distribution to depository libraries for free, but with limitations on use, location, and so forth; and distribution to the general public for a fee.

This sounds like an implementation of what GPO's strategic vision promised: a commitment to "free and ready public access to" government information "in partnership with Federal Depository libraries" while maintaining a separate, fee-based channel to meet its commitment to "distribute, on a cost recovery basis, copies of printed and electronic documents and other government information products to the general public," [emphasis added] (A strategic vision for the 21st century).

Why is this a threat to long-term, free, public access to government information? Imagine what such a system would look like to your users: They could use the net to get what they need, but they may have to pay or use a dumbed-down version. Or they could go offline, go to their library and use a "free" version, which would also have DRM or licensing restrictions, or both.

This is a far cry from DLC's underlying assumption that "much of the access to federal information resources is available 24/7 on the Internet" (Knowledge Will Forever Govern" A Vision Statement For Federal Depository Libraries In The 21st Century).

What this sounds like to me is a revival of the GPO bookstore concept for the digital age with the (fee-based) bookstore as the primary means of access to most government information and the go-to-the-library-building FDLP as the "free" path. This puts libraries and free access as second-tiers, non-networked alternatives for users. It would mean that libraries would be unable to participate in the open and free flow and re-use of government information (Web 2.0, Semantic Web, etc.). It is a vision of government information closer to Jack Valenti's vision of movie distribution than to Jefferson or Madison's visions of government information.

Imagine what this would mean to FDLP libraries and their ability to preserve access to information. Would systems like LOCKSS even be permitted? Or would locked-down-with-DRM or technologically-dumbed-down free versions made available to libraries be technically (or by license) un-preseravable?

I may be misinterpreting GPO's statements and I hope I am. I would welcome hearing further clarifications from GPO including that it does not intend to use DRM and that it does not intend to restrict what FDLP libraries or others can do with free content. I would welcome hearing from GPO that it does not intend to provide dumbed-down or technologically locked or functionally-disabled content for free while providing fully-functional content for a fee. I invite GPO to commit itself to open, free, reusable, preservable, distributable, unencumbered, fully-functional government information. I urge DLC to insist on digital distribution so that FDLP libraries can be fully functional online partners in the organization and preservation of government information and not by-standers who hope GPO will get funding to do so.

GPO could also show its good faith by continuing to study LOCKSS as one (not the exclusive) method of preservation and not just a method for distribution. The project could be expanded to include more than just e-journals. GPO could evaluate automated harvesting using tools that automatically create new directory structures and actively seek ways to help other depository libraries participate (e.g., reviewing automated harvesting).

Perhaps there can be some discussion of these issues at DLC.

Until we get further clarification, we'll all be left wondering.

Continue reading

Continue Reading →

100 Milestone Documents

One of my new favorite sites is 100 Milestone Documents This page from ourdocuments.gov is a compilation of 100 documents from the National Archives and Records Administration (NARA) that highlight key events in U.S. history (1776-1965). Each document includes an image, description and transcript. Continue reading

Continue Reading →

Librarian who received a “National Security Letter” testifies before the Senate

Warning against government secrecy, by Andrew Miga, Associated Press, April 11, 2007.

A Connecticut librarian who fended off an FBI demand for computer records on patrons said Wednesday the government's secret anti-terrorism investigations strip away personal freedoms. "Terrorists win when the fear of them induces us to destroy the rights that make us free," George Christian told a Senate panel.
Continue reading

Continue Reading →

Introducing Newkirk Barnes

April has certainly come in with a bang. I've fallen behind on my BOTM duties because of some computer issue (everyone take a minute and back up your data RIGHT NOW!). I'm back on the horse, and pleased to introduce Newkirk Barnes, our guest blogger for April, 2007. You can read more about Newkirk on her bio page. Newkirk comes to us from Mitchell Memorial Library at Mississippi State University (MSU). Take it away Newkirk!! Also, much thanks to Gary Price, our BOTM for March. Gary was one of our most prolific guest bloggers -- in fact he posted FGI item numbers 999, 1000, and 1001!!! -- and will continue to post to FGI when it strikes his fancy. In the meantime, don't forget to visit the ResourceShelf and DocuTicker! For those that may have missed some of Gary's posts, here's a list. Thanks again Gary!! We look forward to your informative posts in the future!!

Continue reading

Continue Reading →

Archives

Powered by WordPress / Academica WordPress Theme by WPZOOM