Home » Library (Page 24)
Category Archives: Library
Critical GPO systems and the FDLP cloud
[Update: 10/13/09: I've revised my thinking on the cloud as the term is loaded and doesn't really mean what I'm describing. A friend from the San Diego Supercomputer Center said, "some greybeards are going back to the original metaphor: the grid" and suggested the term "shared digital libraries" which is good. But what I'm describing is more like a biological ecosystem, the FDLP ecosystem. jrj] Last week's GPO purl server crash should be disconcerting to both the documents community and the public at large (in fact, although the hardware's been restored, resolution is ongoing as I write). I know GPO staff are just as worried about this and are doing everything they can to fix the purl server.
"The PURL Server is currently inaccessible. GPO is working with IT staff to restore service as soon as possible. We regret any inconvenience caused by the server problems. An updated listserv will be sent once service is restored."But in the meantime, there are 1250+ library catalogs and innumerable links to government documents that are not working. The crash of a critical piece of GPO's infrastructure brings a couple of things to mind: 1) What worries me about this is that FDsys and it's supposed upgrade in hardware/software/systems design is for all intents and purposes the same as GPOaccess. That is, FDsys is a monolith where the failure of one piece can cause the whole system to ground to a halt. As our readers know, we've been advocating for a long time for a distributed digital FDLP (a *true* "digital depository" system!). We're heartened by what we see of FDsys so far, but we need to be building a system with built-in redundancies. I envision a collaborative and distributed system of digital content, collaborative cataloging/metadata creation, as well as technical infrastructure. With this kind of system in place, a failed purl server will only cause a momentary blip in service as a backup purl server kicks on instead of a several week+ outage. How many system degradations (WAIS) and failures (purl server) until we shift our thinking from "[w:client-server]" (with libraries decidedly on the "client" side of the equation) to "[w:Peer-to-peer]" concepts and build systems with built-in redundancies that mirror what the FDLP has been for the last 150 years? How long before we build an FDLP cloud?
Less Access to Less Information By and About the U.S. Government
[UPDATE 3/25/10: Thanks to Bernadine Abbott Hoduski for sending me 1997-98 volumes, the complete chronology from 1981 - 98 is now digitized and hosted at the Internet Archive!] From 1981 until 1998, Anne Heanue and the fine folks at the Washington Office of the American Library Association (ALA) published an amazing series called Less Access to Less Information by and about the U.S. Government, a chronology of efforts to restrict and privatize government information. In 1986, the publication was listed in Project Censored's annual review, Top 25 censored stories for 1986. I recently had a nice email exchange with Emily Sheketoff, Associate Executive Director of ALA and manager of the Washington Office in which I suggested that Less Access to Less Information ought to be online for the world to see, read, share etc. Emily graciously gave me permission to digitize the series. So, with the help of Rick and Megan Prelinger, Robert Miller and others at the Internet Archive, I give you Less Access to Less information by and about the U.S. government in several formats including text, flip book, PDF, and DjVu.
- 1981-1987 chronology
- 1988-1991 chronology
- 1990-1996 chronology
- 1997 chronology: January – June
- 1997 chronology: June – December
- 1998 chronology: january – June
- 1998 chronology: June – December
FDLP: Services and Collections
Against the Grain has kindly permitted me to post this preprint of my new article.
- FDLP: Services and Collections [preprint] by James A. Jacobs, Against the Grain, 21(2) April/May 2009.
In the age of digital information, libraries and librarians are struggling to define their proper roles. In a time of financial uncertainty and economic crisis, many libraries are facing decisions that will have long-term implications and consequences. At a time like this it is particularly important that we have a clear vision of a sustainable role for libraries.And, don't forget Distributed Globally, Collected Locally: LOCKSS for Digital Government Information by Daniel Cornwall and James R. Jacobs. Against the Grain, 21(1) February, 2009. Continue reading
Distributed Globally, Collected Locally: LOCKSS for Digital Government Information
Since Daniel mentioned yesterday about LOCKSS and digital deposit as recession insurance (which BTW is a GREAT oogly hook for open govt!!) I thought I'd mention a hot new article that Daniel and I wrote for the February 2009 issue of Against the Grain about the new U.S. Government Documents Private LOCKSS Network (citation below). The issue has not officially been released, but we got permission to post to FGI as a preprint. The article describes the LOCKSS model of digital preservation and why that model is beneficial to apply to the realm of digital government information. We describe Carl Malamud's herculean efforts toward better access to government information; Then talk more specifically about the new USDOCS Private LOCKSS Network (USDocsPLN) using those documents harvested by Malamud. The paper concludes with a call to action. Let us know what you think. and by all means, help us move forward with the USDocs network by participating. LOCKSS is great recession insurance and SO much more! Citation: Distributed Globally, Collected Locally: LOCKSS for Digital Government Information. Daniel Cornwall and James R. Jacobs. Against the Grain, 21(1) February, 2009. p.42-44 (p.5-7 of the PDF)
The preservation of federal documents is too important to be left to the federal government alone; we have the makings of a viable system to preserve digital government publications. There are several ways you can help. Join our private LOCKSS Network. Join the LOCKSS alliance, get a server for under $1,000, and contact us. The more servers in the USDocsPLN, the merrier. Notify us of collections of electronic federal documents. LOCKSS staff can show you how easy it is to allow LOCKSS to ingest and preserve your materials. Attack the root problem. Demanding your Members of Congress legislate and FUND a system that will ensure that GPO proactively deposits publications and data through the FDLP and other interested partners. While the USDocsPLN project is a good start and an excellent ad-hoc effort, it should be the government's responsibility to put information in the hands of taxpayers. We should not have to be prying it out of the government’s hands. A distributed digital FDLP benefits everyone.Continue reading
Response to Public Printer
April 16, 2009 / Leave a comment
We at FGI would like to thank Robert C. Tapella, the Public Printer of the United States, for his response to our comments on his letter to President Obama regarding open government.
Mr. Tappella's response has some information that should be very encouraging and heartening to the depository library community. It also leaves some issues troublingly unaddressed.
Bulk Data Access to Legislative Information
First, it is wonderful to know that GPO is working with the Library of Congress, Congressional Research Service, the Law Library of Congress, and the Senate and House on the issue of access to bulk legislative data!
That news is important and significant. It is also very encouraging because it marks a new direction for dissemination of government information. Taken to its logical conclusion, this would mean that we will have a new route to obtaining government information. No longer will we be limited to information presented as web pages through government-built interfaces. No longer will we have to hope that web scraping will find all the information we want to gather or preserve. Raw information -- once locked in the dark web of government databases -- will be, potentially, available for libraries and others to download and repurpose.
Unfortunately, we can't look for this right away. Congress has only asked for a report, not action. The report itself is due "within 120 days of the release of Legislative Information System 2.0." Presumably that is a reference to a new version of the LIS that is currently only available within the legislative branch. I have not seen an announcement of a date for the release of a new version of the LIS, so it is not clear even when we can expect the report.
Nevertheless, it is certainly good to hear directly from Mr. Tapella that the task force working on this report will develop "a position on access to bulk data" and even intends to "work on making bulk data accessible."
It is somewhat ironic that this long, drawn-out process itself demonstrates the need for bulk data access. Although there have been calls for bulk data access for years, it literally took a legislative directive to get GPO and LOC and CRS to take the tentative steps they are taking now: to "develop a position" and "work on" the problem. Such passivity and long delays are, perhaps, inherent in a large, bureaucratic system, but they are crippling when it comes to keeping up with technological changes. This demonstrates why it is essential for the government to provide easy, free, reliable access to the raw information of government: doing so will enable others -- who can more quickly adopt new technologies -- to provide better access to that information faster than the government can.
What about Non-Legislative Data?
It is also unfortunate that the task force is only looking at bulk delivery of legislative information. Will it take another legislative directive to get GPO to "develop a position" on bulk access to other data? See Bulk Data Downloads: A Breakthrough in Government Transparency (by Tim O'Reilly, O'Reilly Radar, Mar 4, 2009) for a short list of other other data for which we need bulk access.
Will GPO Support Collections in FDLP Libraries or Just Backups?
Mr. Tapella's statement does not indicate that GPO has yet grasped the difference between 'backups' and digital deposit. GPO's focus is apparently still on making sure that its own collection is functional rather than facilitating digital collections in FDLP libraries. The "geographically dispersed content repository" described by Mr. Tapella is only "our backup" designed to ensure GPO's "continuity of operations" if GPO's own data repository becomes inoperable. This is a good and necessary feature but it is only a backup for GPO and has nothing to do with digital deposit.
Although Mr. Tapella points out that FDsys supports "repositories that can accept data much like libraries today accept tangible publications distributed from GPO," it seems clear that this generic design is intended as providing "backups" and would require "enhancements" to include bulk data access. This is a GPO-centric way of thinking. This is still a long way from GPO having a "position" on digital deposit and even further from "working on" making it possible.
Until GPO understands that it needs to support digital deposit so that FDLP libraries can build their own digital collections with their own functionality, FDLP libraries will not be partners in preservation and access; they will be, at best, little more than a backup for GPO.
APIs are not Digital Deposit
Mr. Tapella repeats the advantages of APIs, but fails to address the need for digital deposit. Providing APIs is not the same thing as providing digital deposit. As we have said in our original comment APIs are not magic. Each is a design for access and the product of choices made by the designer. Each has its own constraints built in. But don't take our word for it; read what developers say about the constraints of using existing government APIs:
We love APIs! We think they are great! We want more! We are so very glad that GPO will support them at last! But, please, Mr. Tapella, understand that APIs and a web site are only two of the three parts of a complete access system. Bulk data access is essential and we'd like to hear that GPO is planning for it now.
OAIS is not Digital Deposit
We are so very happy that FDsys is based on OAIS. It is something we have long advocated. But, again, Mr. Tapella, please understand that telling us about your preservation system and your intentions to preserve information does not reassure us that everything will be preserved and freely available to everyone forever. As we pointed out in our original comments, regardless of your intentions and the quality of your system, GPO may not always have the funding, resources, or mandate to provide free, permanent, public access to all government information and we therefore cannot rely on it alone to do so. And no single digital archive or repository can ever be as secure and safe as multiple archives. We need digital deposit to guarantee preservation and free access.
The GPO-centric approach to preservation and access is like a medieval town that stores all of its grain in one barn. When lightening strikes, the whole town goes hungry. In this day and age of $200 terabyte hard drives, peer-to-peer networks, and successful preservation systems like LOCKSS, it concerns us greatly that you still don't understand the need to have many collaborators working together to ensure long-term, free, public access.
Good News?
There are a couple of sentences in Mr. Tapella's reply that make me optimistic that GPO is on a path to change and does understand this need for collaborators. He says:
To me, this says two important things: First, "data distribution" is on the GPO agenda, at least nominally; second, APIs are just one part of a bigger, ultimate, solution. This gives me hope for more. I hope I'm not reading too much into this.
See also:
- Bulk data and Legislative Information 2.0.
- Congress’ legislative information systems: THOMAS and the LIS by Jeffrey C. Griffith, Government Information Quarterly 18.1 (2001): 43-60. Apr 16, 2009
- Congressional Research Service Products: Taxpayers Should Have Easy Access, Project on Government Oversight, February 10, 2003.)
- Comparison of Legislative Resources on GPO Access and Selected Government and Non-Government Web Sites
- Remixes: Creative uses of free government information
- OpenHouse Project Op-Ed on Databases
Continue reading →Continue Reading →