Maybe I should name this west coast lunchtime listen ;-) Be that as it may, Clay Shirky gave a talk last month (click on the image to get to the video) at the Berkman Center for internet and Society covering some of the ideas from his incredible new book Here Comes Everybody: The Power of Organizing Without Organizations. The focus of the talk is Shirky's notions about the enabling power of the Web and along the way he has a lot of interesting things to say about sharing, conversation, collaboration and collective action. There's a lot of power in sharing and Shirky points to several interesting examples of that power.

Here Comes Everybody: The Power of Organizing Without Organizations (Event Video/Audio)

Agencies not complying with record preservation policies, By Jill R. Aitoro, NextGov, April 24, 2008.

At the hearing, Linda Koontz, director of information management issues at the Government Accountability Office, released preliminary results from an ongoing GAO study of how four agencies managed e-mail and electronic records. ...Koontz said the agencies print and then file e-mails, but about half of senior officials were not following these procedures, and the e-mails for these officials were maintained in e-mail systems that lacked record-keeping capabilities, such as the ability to group the e-mails using a classification system.

The House is considering the Electronic Communications Preservation Act, which would strengthen policies for preservation of government records including White House e-mails.

Gary Stern, general counsel for NARA said that the legislation's potential cost to agencies could be "astronomical," and noted the bill's requirement that the National Archives would maintain authority over the White House's electronic records might be unconstitutional.

Patrice McDermott, director of OpentheGovernment.org, said:

"I understand the constitutional issues, and I don't have a good answer for that.... But one of the concerns is that there is no way to enforce accountability [of] records management in the White House. We understand it's a difficult dance [for NARA]. They're there at the invitation of the White House in many cases, but there needs to be some way for the outside community to hold the White House accountable."

All governments manipulate the media to garner favorable news coverage and spin the flow of information to put their actions in a positive light. But in a story in Sunday's NY Times (April 20, 2008) entitled "Behind TV Analysts, Pentagon’s Hidden Hand," David Barstow describes a concerted effort by the Bush Administration who used ostensibly objective military analysts to spread propaganda and dupe the American public in a campaign to generate favorable news coverage of the administration’s wartime performance in Iraq. It turns out that those "independent military experts" consisted of “more than 150 military contractors either as lobbyists, senior executives, board members or consultants.”

Once again, John Stewart describes this event with wit so that we'll laugh rather than scream. So I'll let him have the last word. And he mentions a GAO report called "Combating Terrorism: The United States Lacks Comprehensive Plan to Destroy the Terrorist Threat and Close the Safe Haven in Pakistan's Federally Administered Tribal Areas" that you can now get your hands on via the Internet Archive.

Five years into the Iraq war, most details of the architecture and execution of the Pentagon’s campaign have never been disclosed. But The Times successfully sued the Defense Department to gain access to 8,000 pages of e-mail messages, transcripts and records describing years of private briefings, trips to Iraq and Guantánamo and an extensive Pentagon talking points operation.

These records reveal a symbiotic relationship where the usual dividing lines between government and journalism have been obliterated.

Internal Pentagon documents repeatedly refer to the military analysts as “message force multipliers” or “surrogates” who could be counted on to deliver administration “themes and messages” to millions of Americans “in the form of their own opinions.”...

...Over time, the Pentagon recruited more than 75 retired officers, although some participated only briefly or sporadically. The largest contingent was affiliated with Fox News, followed by NBC and CNN, the other networks with 24-hour cable outlets. But analysts from CBS and ABC were included, too. Some recruits, though not on any network payroll, were influential in other ways — either because they were sought out by radio hosts, or because they often published op-ed articles or were quoted in magazines, Web sites and newspapers. At least nine of them have written op-ed articles for The Times.

Is there a cataloger in the house?! The fine folks over at radical reference are having a Library of Congress Subject Heading Suggestion Blog-a-Thon. Between now and Sunday, April 27, you can suggest subject headings and/or cross-references which will then be compiled and sent to the Library of Congress. Uber-cataloger Sandy Berman has been doing this for years, so it's great to see others taking on the challenge of collaborative subject description!

Some time between now and Sunday, April 27 at 6pm Eastern:

1. Select one or more subject headings or cross-references to suggest
2. Provide material to support your suggestion (in the form of a link and excerpted text/image)
3. Blog it somewhere (your own site; Radical Reference--if you're a registered and authenticated user on the site, you can create your own blog post, if not, just make it a comment to this post; an online file sharing service like Google Docs or Zoho)
4. Tag it for del.icio.us: rr_lcsh2008 and for:radical_reference. If you don't have a delicious account email me, and I'll tag it for you.
5. If you are suggesting a subject heading not previously submitted to LC (e.g. not on Sandy's scorecard), also submit your proposal to the Program for Cooperative Cataloging.
6. For discussion and help, join the Meebo and/or Skype chat,which will be active on Sunday from 4-6 ET for sure, and other times, as staffed.
7. If you are in the NYC area, you can come to the ABC No Rio Computer Center on Manhattan's Lower East Side for some in person collaboration.
8. We will email a link to the tagged items to LC, print out a copy of each blog post and mail it to Sandy, and we're kinda hoping that the members of the RADCAT (radical cataloging) discussion list will consider entering some of the suggested headings properly into the proposal form

Erik Ringmar, professor of social and cultural studies at the National Chiao Tung University, Hsinchu, Taiwan, wants others to join him in putting restricted government documents on the web.

I say this is awesome! There's certainly precedent for this kind of activism: Jared Benedict liberated a bunch of USGS maps and just last week, I uploaded the Iraqi Perspectives Report to the Internet Archive. Anyone else out there set free a government document? Leave us a comment.

So, I've taken it upon myself to start an organisation called MLOP, the "Movement for the Liberation of Old Papers". What I do is hack into restricted websites, download the documents I'm interested in, and then use my favourite open-source paint program to remove the copyright statements from each page. Next I assemble the pages into one single pdf file and upload it to the Internet Archive, where it will become universally available to both researchers and citizens. Yes, it does take a bit of time, but it's a very worthy cause (and I have a hardworking research assistant to help me).

I feel strongly about this, and I'm prepared to live with the legal consequences of my actions. This, after all, is the new frontier of civil rights - the right of access to information. How else can corruption be stopped and falsehoods exposed? How else can people in power be held accountable? I'd go to prison for the old parliamentary papers if I had to. Ever after I would proudly brag about having liberated an old House of Commons report from the clutches of market capitalism.

Mary Minow writes that a significant part of the Copyright Remedy Clarification Act of 1990 (CRCA) has been struck down by a California Southern District Court ruling.

In 2006, a marketing research firm sued the CSU system. It alleged that San Diego State University, which had been hired in 2004 to perform annual fiscal impact analyses for the Holiday Bowl games (SDSU had been hired because the marketing research firm, which had performed the analyses previously, had increased its fees), had misappropriated and plagiarized the marketing research firm's earlier reports.

The CRCA reads, in part, that "Any State, any instrumentality of a State, and any officer or employee of a State or instrumentality of a State ... shall not be immune, under the Eleventh Amendment ... from suit in Federal Court ... for a violation of any of the exclusive rights of a copyright owner ...."

In theory, this means that states are now in the clear from being targeted by the federal claims that the CRCA was worded to allow. As the District Court ruling states, "The CRCA was passed with the intent to subject states to liability for copyright infringement."

The major wrinkle is that the ruling appears to protect only state agents or employees who are acting in their "official capacity." As Minow's post points out, there are any number of steps that a plaintiff could take to establish legally that a state employee was not acting in his or her "official capacity." The most germane step would be that an individual sued under federal law (and the CRCA being federal law) can be classified as having acted in his or her "individual capacity" if the plaintiff can establish that an alleged violation was in contravention of protected federal copyright.

The ruling, Marketing Information Masters Inc. v. The Board of Trustees of the California State University System (.pdf), is here. More pleadings in the case are available here.

A new book is out entitled, "Access Denied: The Practice and Policy of Global Internet Filtering" edited by Ronald Deibert, John Palfrey, Rafal Rohozinski, and Jonathan Zittrain from the Berkman Center's OpenNet Initiative. This is a must-have for libraries -- many of whom deal with filtering at the personal computer level -- in order to inform the public on the more insidious filtering of internet traffic that happens at the country or backbone level. "Access Denied provides the definitive analysis of government justifications for denying their own people access to some information and also documents global Internet filtering practices on a country-by-country basis. (Jonathan Aronson, Annenberg School for Communication, USC)"

The site includes country profiles for those countries "in which it was believed that there was the most to learn about the extent and processes of Internet filtering." Read the BBC review and the Review in Nature.

In an investigation on how the Bush administration uses retired military officers to promote its message on the Iraq war, the New York Times successfully sued the Defense Department to gain access to 8,000 pages of e-mail messages, transcripts and records describing years of private briefings, trips to Iraq and Guantanamo and an extensive Pentagon talking points operation.

The story based on these documents (Behind Military Analysts, the Pentagon's Hidden Hand By David Barstow, New York Times, April 20, 2008) is supplemented online by "Audio, video and documents that show how the military’s talking points were disseminated" (How the Pentagon Spread Its Message and a "Document Archive," which allows users to read and download documents and parts of documents. Of the 8000 pages, only a few are available online, but these include emails, a "Talking Points Memo," excerpts from a Transcript of meeting with Mr. Rumsfeld, and a Pentagon document that reports "Monitoring of Analysts."

Together, the audio-visual presentation and the documents are a small model for how newspapers could be using the power of the web to enhance their coverage and utility. I would certainly like to see all 8000 pages online!

The story itself is a fascinating glimpse behind the scenes of the daily news.

Internal Pentagon documents repeatedly refer to the military analysts as "message force multipliers" or "surrogates" who could be counted on to deliver administration "themes and messages" to millions of Americans "in the form of their own opinions."

...Analysts have been wooed in hundreds of private briefings with senior military leaders, including officials with significant influence over contracting and budget matters, records show. They have been taken on tours of Iraq and given access to classified intelligence. They have been briefed by officials from the White House, State Department and Justice Department, including Mr. Cheney, Alberto R. Gonzales and Stephen J. Hadley.

In an odd addendum to the corruption case of (now former) Rep. Randy "Duke" Cunningham, prosecutors are arguing that once the executive branch says something is classified, courts are virtually powerless to review or disagree.

The arguments are in the case of Thomas Kontogiannis, a New York financier who admitted to one charge of laundering bribe money for former Rep. Cunningham.

• Cunningham figure's plea deal was sealed, By Greg Moran, San Diego Union-Tribune, April 20, 2008

While portions of the case remain secret, a batch of previously sealed court filings was released this week that show the government arguing what media law experts said was an astounding position....

In essence, prosecutors argued that once the executive branch says something is classified, courts are virtually powerless to review or disagree. That is true, they argued, even when the information is part of court records - which historically have been considered open under the First Amendment.

There is an excellent post relevant to government information over at ArchivesNext. I recommend this highly.

• NARA and the web harvest: a discussion of the issues, ArchivesNext, April 14, 2008.

Kate does an excellent job, in my opinion, of analyzing the NARA decision to not do another web harvest of agency web sites at the end of the current administration. For example, she says, "For archivists, these web harvests should be troubling because they dispense with the process of appraisal. In effect, anything on the top four levels of an agency’s web site was determined to be of permanent value." Kate also includes links to articles about the issue and the NARA response.

It has excellent and informative comments that include, but go beyond, the specific issue of NARA and web harvesting. I found these comments particularly useful because they are mostly from the perspective of archivists and give insight into long-term preservation issues. Some of those making comments are well known in archival circles and speak from experience and with authority. Christine says that "it is very difficult to do item-level appraisal of web files, because the pages are usually so interconnected."

Of the original blog posting at .govwatch that started off the controversy and its claim that NARA is "Quietly Destroying Millions of Documents," Thomas E. Brown says "Nothing could be father the truth" and backs up what he says with facts.

Maarja discusses information gaps created when dynamic records are overwritten and not preserved. I found this comment by Maarja particularly interesting:

Depending on the agency, decisions on how best to share information might have been driven initially by technological factors more so than long term capture of knowledge. From reading records managers’ forums, I gather that in some agencies IT more so than RM may have driven adoption of solutions for dealing with electronic records.

No two organizations are going to have exactly the same culture and organizational climate. So it’s hard to predict how preservation of electronic information is going to play out throughout the government.

While FGI normally focuses on US government information policy issues, there is a conflict going on in the UK that mirrors some of the recent stories about public data being used by private companies in a privileged way, forcing the taxpayer to pay twice for their data.

An April 17, 2008 Guardian article titled A costly 2008 Domesday Book details how not one, but two British agencies contracted with commercial companies to post government compiled data. The result:

After seven years of legal wrangling, an official, complete and constantly updated list of addresses in England and Wales is about to become available for commercial use. The National Land and Property Gazetteer (NLPG), compiled from data supplied by local councils, is being promoted as the best list of property addresses since the Domesday Book.

Free data it is not. Although prices have yet to be finalised, the commercial firm hosting the service said this week it will cost between £15,000 and £20,000 a year. Profits will be shared among local authorities to help them keep data up to date.

The gazetteer is not the only address database on the market. The state-owned Ordnance Survey also offers addresses as part of its MasterMap digital geographical database of Britain.

Most of the article is about campaigns to free the data. In analyzing the roadblocks, they talk about issues that will be familiar to US readers:

"We would like to give it away free," says Nicholson. However, he says, local authorities are not going to give their work away when they have to pay for the use of postcodes from the Royal Mail's Postcode Address File. Neither can Ordnance Survey, which is required by the Treasury to show a return on its activities, and regards MasterMap as a key part of its revenue-generating portfolio.

We wish the Free Our Data campaign well.

This is probably a good to time to mention that what FGI objects to isn't the selling of data per se, but the selling of data that has already been compiled at taxpayer expense. If a private company wanted to raise its own venture capital, compile its own address list completely independent of government sources, we'd be all for it charging whatever the market could bear. But a private entity should not be allowed to be the sole, fee-based dispenser of information that has been compiled by government agencies using money confiscated through taxation. THAT's what we're against.

According to an article in today's Los Angeles Times, the Department of Health and Human Services and the Department of Justice have appealed a ruling from the U.S. District Court for the District of Columbia that would give consumers more access to Medicare healthcare data.

Specifically, the August 2007 ruling, based on a FOIA request and then a subsequent lawsuit by the advocacy group Consumers' Checkbook/Center for the Study of Services, would have allowed disclosure of a subset of Medicare billing records for four states (Illinois, Maryland, Virginia, and Washington) and the District of Columbia. The information requested would not have contained any patient identifying information, but could have potentially allowed consumers to get more understanding of the operations of Medicare and the Center for Medicare and Medicaid Services (CMS), as well as make decisions about physician expertise and efficiency, according to the Times. As the judge's decision put it, "The public interest at stake is the interest in obtaining information that would help the public make more informed Medicare decisions and the interest in more information of how government funds are spent."

However, the American Medical Association opposed the ruling, and has also petitioned to join the appeal. The HHS appeal is based on a 1979 federal court ruling that blocked release of Medicare physician reimbursement data. HHS states that it shares the goals of Consumers' Checkbook in providing a transparent health care marketplace for consumers, but says that the 1979 ruling conflicts with the 2007 ruling. Observers quoted in the Times article said that the HHS was under pressure from the AMA to keep the data from being released and that it wasn't just a matter of conflicting legal opinions.

The HHS news release announcing its decision to join the DOJ appeal against release of Medicare data is here.

Last summer's ruling on the release of the data is here.

Well...this is a good sign. The EPA wants to know "what kind of environmental information you need, and how you want to get it". It's part of the National Dialogue on Access to Environmental Information and the EPA wants your input. You can contribute to their discussion board or submit a comment. I'm forwarding this link along to the Environmental Sciences Dept. faculty here where I work, among others. Spread the word!

The McClatchy Newspapers have reported on a new National Defense University report on the Iraq war.

• Pentagon institute calls Iraq war 'a major debacle' with outcome 'in doubt', Jonathan S. Landay and John Walcott, McClatchy Newspapers, April 17, 2008. "The war in Iraq has become "a major debacle" and the outcome "is in doubt" despite improvements in security from the buildup in U.S. forces, according to a highly critical study published Thursday by the Pentagon's premier military educational institute."
• Choosing War: The Decision to Invade Iraq and Its Aftermath, by Joseph J. Collins, Institute for National Strategic Studies, Occasional Paper 5, National Defense University Press, Washington, D.C. April 2008. "Measured in blood and treasure, the war in Iraq has achieved the status of a major war and a major debacle."

Dr. Joseph J. Collins has been Professor of National Security Strategy at the National War College since 2004. Prior to this assignment, he served for 3 years as the Deputy Assistant Secretary of Defense for Stability Operations. From 1998 to 2001, Dr. Collins was a Senior Fellow at the Center for Strategic and International Studies, where he researched economic sanctions, national security policy, and homeland security. In 1998, after nearly 28 years of military service, Dr. Collins retired from the U.S. Army as a colonel. His many publications include books and articles on war in Afghanistan, Operation Desert Storm, military culture, defense transformation, homeland defense, and the way ahead in Iraq. Dr. Collins holds a bachelor’s degree from Fordham University and two master’s degrees and a doctorate in political science from Columbia University. In 2004, he was awarded the Department of Defense Medal for Distinguished Public Service, its highest civilian award.

NDU Press publications are sold by the U.S. Government Printing Office. National Defense University Web site: http://www.ndu.edu.

See also: The Army Monograph that Predicted Just About Everything that's Happened in Iraq.

Over the past week, there have been some good conversations about government contracts to digitize government information and the National Archives decision to not conduct a web harvest or snapshot at the end of the current Administration. There is good news and bad news.

The good news
The good news is that NARA's decision was not nearly as bad as it appeared to be when it was first announced in a memo on March 27, 2008, which was circulated only to Federal records officers (see: The National Archives Is Quietly Destroying Millions of Documents). In a thoughtful post on its web site (National Archives and Records Administration Web Harvest Background Information, April 15, 2008, NARA; pdf version available), NARA outlines in detail the reasons why it would not conduct an end of administration web snapshot or harvest of Executive Branch websites nor require agencies to do so. The reasons, I think, are sound and in keeping with NARA's commitment to preserving information of historical value.

In addition, the NARA memo of April 15 makes explicit the fact that its decision and memo of March 27 do not apply to Presidential records or to records of the Congress. It says that "NARA will continue to conduct a web harvest of Congressional web sites" and that NARA "will also receive a snapshot of the White House website" noting that "Unlike Federal agencies governed by the Federal Records Act, the White House is governed by the Presidential Records Act, under which all Presidential records are treated as permanent and transferred to NARA for preservation at a Presidential Library."

The NARA "Background Information" document is also, I think, worth reading for its clear description of the shortcomings of web harvests in general. I think it is very useful for us to be reminded of these shortcomings to the extent that we believe we can rely on them as an adequate form of preservation.

In more good news, the NARA/TGN contract is not as bad as it could have been. I mentioned this in my earlier post here (The NARA/TGN contract as a bad precedent) and similar comments have been made in the useful and interesting thread over at ArchivesNext (NARA latest digitization agreement: One archivist's perspective). Merrilee Proffitt, of RLG, says in a comment there that the NARA model for contracts with third parties "actually comes out looking pretty good" when compared to the criteria described in the RLG paper Good Terms - Improving Commercial-Noncommercial Partnerships for Mass Digitization (by Peter B. Kaufman and Jeff Ubois, D-Lib Magazine, November/December 2007, Volume 13 Number 11/12).

The bad news
The bad news, as James pointed out this morning, is that the GAO contract for digitizing is very bad indeed (GAO *did* sell exclusive access to legislative history to Thomson West). Quoting Carl Malamud, James notes that GAO gets access to the digitized data but does not get a copy of its own; the rest of the government doesn't even get access to the data. The public is left with the option of going to GAO headquarters and paying 20 cents per page to copy paper! As Carl says, "This is one of those deals where the public domain got sold off."

This morning there was more bad news. Kate at ArchivesNext reports that the Citizens for Responsibility and Ethics in Washington (CREW) has a new report Record Chaos: The Deplorable State of Electronic Record Keeping in the Federal Government, that concludes "that the federal government is severely mismanaging its electronic records." CREW also says that a House Committee proposal to amend federal record keeping laws "is anemic and fails to make the substantial changes necessary to bring the federal government into the 21st century."

And even the good news is tempered by the fact that we have less than we could and are a long way from an even an adequate system of permanent preservation of digital information or a long-term solution to digitizing non-digital information. We will have to hope that the White House will deliver a snapshot of the White House web site and that the snapshot will be accurate and complete. The behavior of the White House with regard to electronic records and email does not make us optimistic. The NARA/TGN deal is better than the GAO/Thomson deal, but still leaves much to be desired and, as pointed out even by defenders of the deal, it is unlikely that we will ever have free, open, networked access to the digital information that TGN digitizes. That means the real effect of the deal is to privatize the information.

Comment
For me, the biggest disappointment in these latest developments is that librarians and archivists seem to be too willing to accept "good enough" and not willing enough to argue harder for "better." There are lots of people who have good reason to argue for less access, more fees, less privacy, and more control of information, but librarians and archivists should not be among them. I believe that we should not spend time making the case for the private sector; it is fully capable of making its own case. We should spend our time fighting for free, full, open, public access, usability of information, and long term preservation.

The primary mission of private sector companies is to make money, not to serve the public. They may serve the public as a by-product of making money, but no for-profit company will go to its owners and say "we are going to do the best thing for public access" without the qualification "that will make us money." Unfortunately "making money" often conflicts with public access. Politicians (and some bureaucrats) will argue for greater control of government information; some will argue for secrecy of government information on the one hand and privacy-invading policies on the other. Most government agencies do not have information access or long term preservation of their information as a primary mission and the exceptions are notable (e.g. LOC, NARA).

In contrast, the primary mission of many libraries and archives is to provide free public open access with long term preservation and usability. While others may have some of those pieces as secondary goals, few if any have them all. For many libraries and archives these goals are not just their primary mission but their defining characteristic.

While digitization and digital preservation are neither easy nor inexpensive, that doesn't mean that we have to pay any and all costs for them. The digital era should be making it possible to provide better access without giving up free use and reuse, without giving up open access, without turning over control to those whose primary mission is something other than free, open, public access and long term preservation. But increasingly we see a combination of politics and economics leaving us with contracts that trump copyright and fair use, with "access" being negotiated at almost any cost (including loss of control), with DRM technologies that prohibit easy (or any) reuse, and with privacy protections being deprecated or even ignored. Even in the case of the NARA/TGN contract that is legally "better" than the GAO/Thomson contract, we are left with the effect of two-tiers of access and network access being essentially privatized and fee-based.

I believe that librarians and archivists should be pushing the boundaries and insisting for more and better, not accepting some benefits by negotiating away the big benefits we could be getting in the digital age. This is particularly important for government information that is in the public domain. If we can't make this work for public information that is not copyrighted, how will we be able to do so for information that is?

I'm not arguing for a perfect, ideal world that is impractical to achieve. I am suggesting that we should fight for everything we can get. We should celebrate when we make inroads with a contract (like NARA/TGN) that is better than the others (like GAO/Thomson) but we should do so by committing to doing better next time. We should not accept this as "good enough" -- because it is not and we can do better next time. In fact, every time we accept a less-than-perfect deal as "good enough," we make it a little harder to make a better deal next time. We lower the bar if we accept "good enough" and stop trying to achieve better. We should not take the time to convince ourselves or the public that this is as good as we can get; we should take that time to admit to the limitations and trade offs and to commit to doing better next time.

There is lots written these days about "the future of libraries" and "the role of libraries in the digital age" and many people openly wonder if there is a place for libraries at all. I think there are several places where libraries have a unique role to play in society and the areas of digitization and digital access and preservation are important ones.

We need to make the case for the public; for free, open, public access; for long-term preservation and usability; for public accountability in the control of information; for reader privacy. Librarians and archivists have a unique role in doing that. In doing so, we will face an uphill battle and trade offs, but we should never lose sight of our unique role in society. We should never cheapen our professions by making the case for less (there are plenty of people to do that). We should always make the case for more. We will not always succeed and we will have to make trade offs. But we should always do so in the context of staking out a territory that is different from the private sector and those who are willing to get less. We should stand up for rights that others are not willing to fight for. We must fight for it when there are so many forces aligned against free, open access.

I'd like to see us emulate Carl Malamud and CREW and Brewster Kahle more and do less of making excuses for TGN and Thomson.

A few weeks ago, Daniel had a great post, "GAO/Thomson-West Contract Raises Questions" in which he expanded on a Boing Boing post "Did the US gov't sell exclusive access to its legislative history to Thomson West?" and analyzed the Thompson-West contract with the GAO for digitizing 20,597 legislative histories of most public laws from 1915-1995. Today, Carl Malamud got an answer to his FOIA request to the GAO seeking access to the digitized images of those legislative histories. I'll let Carl tell it in his own words:

Well, the answer is now a definitive yes, that data has been sold down the river and is out to sea.
Public.Resource.Org sent in a FOIA request to GAO on this topic seeking access to the scanned data. Today's letter answering our FOIA request spells out the bad news. Turns out the GAO doesn't even get the data, they simply are given an account on Thomson's service. The rest of the government doesn't get access to this data, and the public is invited to stop by the GAO headquarters and pay 20 cents per page to copy paper.

This is one of those deals where the public domain got sold off ... GAO gets a bit of convenience by having their stuff scanned for them, but they gave up way more than they got in the deal, and the public (including government workers and public interest groups who need to consult this data) lost big-time.

Carl has put up his paper trail explaining the story. Here's the link to the Scribd group with the full paper trail on this issue, and here's the link to last week's response from the GAO.

This perfectly exemplifies the problems we see with government agencies entering into contracts with private companies to digitize public domain materials (see for example "NARA/TGN contract as a bad precedent"). We have no problem with government agencies contracting with private companies to digitize government information. The problem as we see it is that so many agencies seem ignorant of the fact that privatizing access to said digitized public domain information actually limits access in the long run.

Last month we posted a story about the Iraqi Perspectives Report being available only by mail from the U. S. Joint Forces Command. Lo and behold, today I received a CDROM in the mail with the 5 volumes of the report in PDFs. I just uploaded the whole thing to the Internet Archive. Catalog away!!

Iraqi Perspectives Report Saddam And Terrorism: Emerging Insights From Captured Iraqi Documents. January 2007. IDA Paper P-4287. 5 volumes.

The Halfway House Between Science and Secrets; An Interview With Bruce Schneier on Science and Security By Jonathan Pfeiffer, interviewer Science Progress, March 19th, 2008.

"A recent, Congressionally-mandated, National Research Council report (Science and Security in a Post 9/11 World, by the Committee on a New Government-University Partnership for Science and Security, National Research Council) recognizes that the 9/11 attacks provoked counter-productive security measures that stifle access to fruitful scientific research.... The NRC warns that the widespread practice of labeling scientific research as "sensitive but unclassified" has had grave consequences for our security and our economy.... Security expert Bruce Schneier talks with Science Progress about the science that makes us smarter and the security that makes us safer."

Schneier says:

And a lot of scientific data, information, and knowledge-stuff that is used by the scientific community, used by businesses, used by everybody-gets stuck in this halfway house between secret and open. It's a form of secrecy, and it's a form of stifling information sharing. And where it affects scientists is that science thrives on information sharing. Science works because one person's research becomes another person's footnotes.

What is a Member of Congress to do? Embedding a YouTube video on on an official House or Senate web page is against the rules (See The Wrong Way to Talk About Member Web Sites), yet many do it anyway. The House and the Senate both have their own video technology for official sites but it is, uh.... slow..... And even pointing to YouTube is against the rules.

But recently, the Franking Commission has come up with a compromise idea. House Speaker Nancy Pelosi's office put out a request for a commercial-free video zone for Congress. YouTube was the only one that responded to the request, but it should be up and running within a month.

For details, see: Et Tu, YouTube? Lawmakers to Get Their Very Own Sites for Videos, By Jonathan Weisman, Washington Post, April 11, 2008; page A19.

What will that do to our view of "official government web sites"? Will videos be considered in the scope of Title 44 by GPO? Will they be scheduled for acquisition by NARA?

On April 9, 2008, Eamon Duffy posted the following announcement to govdoc-l that we at FGI felt was deserving of broader coverage:

The McGill Library Global Health Resource Wiki

wikisites.mcgill.ca/GlobalHealthGuide

The McGill Library has created a Global Health Resource Guide to promote collaboration and to share and organize knowledge about resources within the McGill community and beyond.

This resource guide was created as a wiki using MediaWiki software so that as many people as possible would be encouraged to both use the guide and add resources to it. All members of the McGill community - students, faculty, clinicians, researchers and staff - can add resources to the guide by logging in with their McGill user ID. Anyone in the international community with an interest in global health is welcome to use the guide for finding global health information and resources. If you are from an academic community outside of McGill, we encourage you to participate and contribute to the wiki. Contact us for a user ID if you would like to contribute resources or information to the guide.

To obtain more information, to comment or pass on suggestions for content, or to request a User ID to add resources and content, please contact one of these McGill librarians:

Eamon Duffy Government Information Service
Louisa Piatti Nahum Gelber Law Library
Jim Henderson Life Sciences Library
Deborah Meert Macdonald Campus Library
====================

Contact information for the above librarians can be found at http://wikisites.mcgill.ca/GlobalHealthGuide/index.php/McGill_Library_Global_Health_Resource_Guide:Contact_us

If you're affiliated with an academic community or library and have some time to assist building this resource, we encourage you to help. It is one more way the the librarian community can show how it adds value to information.

Our friends over at GovGab have started a conversation about web 2.0 tools at http://blog.usa.gov/roller/govgab/entry/government_in_the_web_2 we think you should join.

After some introduction that I encourage you to read on their site, they say:

So, America, I pose a question to you… What do you think of government agencies using social media tools like blogs, RSS feeds, podcasts, YouTube, Flickr, widgets, and microblogs to reach out and give you information? Is it good? Bad? Are there ways you’d like the government to provide information that we haven’t thought of or addressed yet? After all, we’re public servants and we’re here to serve, so let us know what you want and need.

So far they've got nine comments, all of them interesting. Some are from government employees explaining why their agencies aren't terribly social. Sometimes there good reasons. Read Ray's comment dated 4/11/2008, 2:05pm to see an example.

The thread has also shown that the GovGab folks are really listening. A poster called lentigogirl noted that GovGab's Flickr pictures were tagged "All Rights Reserved" when photographs produced by the government should be public domain.

Within a few hours, GovGab blogger Sommer came back with this response:

lentigogirl,

Thanks for the heads up on the Flickr photos. I hadn't noticed the photos were labeled "all rights reserved" before. Whenever I view our photos I'm logged into the account and that piece of information isn't displayed to you on your own account.

I've done some research on Flickr and they don't offer an option to set your photos for the public domain. The closest I can come for now is designating them "Attribution-NoDerivs Creative Commons" (for more info http://creativecommons.org/licenses/by-nd/3.0/). Also see the Flickr Help Forum for more information on this topic: http://www.flickr.com/search/forum/?q=photos%20in%20public
%20domain
Based on comments in the Help Forum I think the best solution might be to just tag all our photos with a "public domain" tag.

Thanks for teaching me more about Flickr -- it really is all about communicating with each other!

Have a great weekend!
Sommer

Now THAT is responsiveness. As regular FGI readers know, I've really enjoyed GovGab from day one. I think they do a terrific job. So now that they're asking for input on Web 2.0 stuff, I hope you'll take a moment to help them out by sharing your opinion. Then I hope that more agencies will use GovGab and the USA.gov as service benchmarks for responsiveness.

I just found this morning that Daniel Ellsberg, the former Marine and Department of Defense official who leaked the Pentagon Papers in 1971, is speaking at the Stanford Law School, along with Charles Nesson, his defense counsel, and Laurence Lessig. The flyer also says that the "discussion will also cover the prior restraint recently leveled against Wikileaks.org in a federal court in San Francisco and the judge's subsequent reversal of his own injunction." I'll try to live-blog what I can so keep checking back.

Here we go...

=======================

Lessig (LL), Ellsberg (DE) and Nesson (CN) on the dias. Ellsberg to tell the story of the PP and expound on free speech in the US.

CN: was a young law professor at Harvard in 1969. Vietnam was a widespread cause on campuses across the US as Iraq is not. Chicago 10 documentary was a great documentary about the 1968 Democratic convention that will give folks a feeling for the times that CN is speaking about. McNamara had generated a secret history/study (36 volume study starting in 1945) of Vietnam and SE Asia. This secret history showed all of the manipulation of the American public, the cynicism of the US govt. Ellsberg got a hold of this secret study, and tried to peddle it around. Finally, a DE was able to connect with a reporter at the NY Times. If Nixon administration had simply ignored it, it would've gone away. But Mitchell and other Nixon folks freaked out, tried to find the leaker and kept it in the news and in front of the public eyes. CN and DE would meet on bicycle in the Harvard tunnels so that the FBI couldn't follow them. DE distributed volumes to a bunch of different newspapers across the US.

DE's trial happened in L.A. prosecuted for theft of govt property. But he *copied* the papers, didn't steal them. Then the govt tried to get him on the espionage act. The govt had a bunch of expert witnesses, admirals, generals etc. DE's first witness was howard Zinn! Zinn told about the first volume like it was a bedtime story about Ho Chi Minh up in the hills, dreaming of freedom and democracy... Next witness was Naom Chomsky!! Eventually, the trial was thrown out.

DE: All charges were dismissed, but there was another case pending on the east coast. That case was eventually dropped because of how the L.A. trial went.

About 10 years after the trial, DE asked CN if he was guilty of the charges and CN said, "I don't know." Espionage laws (18 U.S.C. § 793 paragraphs D and E ? someone please fact check this!) are extremely confusing and contradicting and not understood by lawyers or judges around the country. DE speaks to many legal groups. DE's trial was the first trial for a leak of classified documents. So there was no law on the issue. There is no official secrets act that criminalizes the unauthorized exposure of classified information. So there's no basis in criminal law. An official secrets act would violate the 1st amendment. Congress DID pass an official secrets act in 2000, but Clinton vetoed it. It hasn't been reintroduced since. Very few people know these 4 points. DE assumed that there WAS an official secrets act and that he had violated it. Copying public domain govt information would not in 1971 have been against the law. DE thought he would go to jail for copying 7000 pages of secret information.

LL: what would you have said to your wife?! DE: never give a former wife information that could put you in prison!! LOL!

DE: at the time his wife didn't know that he might have been put in prison for his entire life. Whistle-blowing breaks up most marriages. There was a study done about this.

DE: I'm not a fan of APAC, they're leading the country toward the abiss. But the 2 AIPAC employees are facing the same charges that Tony Russo (DE's colleague) faced in 1971. AIPAC is effectively an Israeli lobby, a foreign agent. DE thinks the APAC case will be dismissed.

Our ability to be a real republican govt depends on unauthorized leaks! Torture went on against domestic and international laws and NSA domestic spying also went on for years and known by thousands of people. Neither of these were leaked for years. Bush administration kept this information from leaking because they told newspaper publishers that the next 9/11 would be on their hands. NYT by their reticence to leak the NSA story (they sat on it for a year!) gave the 2004 election to Bush.

We'll be able to find out about the NSA wiretapping by the civil suits against telecom companies. And that's why there's such a push by the administration for telecom immunity. NSA is our Stazi (east germany) now. The NSA has turned on the American public for the last 6 years; that's East Germany! When the NSA and the White House can know every private act, every journalist source, every political association, you don't have a democracy. The real question is if the NSA can be pulled back! We're in a very serious situation.

What should the law be? Should there be an official secrets act? No!! How many people should do what DE did and shouldn't go to prison for it? There should be a "Pentagon Papers" out every week or two. Don't wait until the bombs are dropping. Go to the press AND to Congress. Put out the documents that reveal that we're being lied into war, that the same thing is happening with Iran, that the administration knew at the time that Iraq didn't have WMD. Consider going to prison in order to stop a war. Give up your access to a president even if your name is Powell. Do this because the alternative is hundreds of thousands of deaths. Do the right thing! Changing of an administration is not going to change, we must change the law, change the climate, advocate to Congress.

UPDATE 2:

Questions and Answers:

How did you get 7000 pages out of your office? DE had a safe in his office and could easily take classified documents. Because of DE, Rand employees have to sign in and read classified documents in a clean room.

What do you see as the proper role for secrecy in the govt? What would be legitimate to protect secrets?
DE: Of course the govt has some legitimate secrets. Secrets that should be kept from an enemy (i.e., the place and time of the Normandy landing). A real expert on declassification, William Lawrence, estimated that the amount of material that deserved protection was in the area of 2% of the information that's classified. And .5% would need to be protected beyond a few months. The cost of the overbearing, pathalogical secrecy is Vietnam, Iraq, nuclear weapons... Most, 98% of secrets do not meet the requirements of damaging national security. Most of what the public needs to know is kept secret.

The United Nations High Commissioner for Refugees (UNHCR) has just announced the "Google Earth Outreach" partnership with google earth. The program will allow humanitarian agencies the ability to use Google Earth and Maps to "highlight their work on behalf of millions of refugees and other populations of concern in some of the world's most remote and difficult areas." All one needs is to download the google earth application and the UNHCR kml file, then launch the app and open the kml file (KML = Keyhole Markup Language). Navigate to a region -- like Chad, Iraq, Colombia or the Darfur region of Sudan -- and you'll get access to news, video and other information about the situation and daily life/struggles of refugees.

Check out the google earth gallery for other KML files that people have built -- from world energy consumption to Death valley driving tour to Yosemite 1/2 dome hikes to the Burke and Wills expedition of 1860! I haven't yet come across an FDLP KML file, but perhaps one of our readers would like to start one? It's as easy as creating a google map and clicking the button "view in google earth" at the top right of the map. This will automatically generate the KML file. You can then send it to admin AT freegovinfo DOT info and we'll post it to the site.

The New York Public Library has lots of free audio and video (and text!) on iTunes! See: NYPL on iTunes U., by Barbara Taranto, NYPL, April 8, 2008.

If you have iTunes, here are some direct links to get you started:

• NYPL at iTunes
• Live from the NYPL
• Slate At 10: Online Media and the Future of Journalism (text)

According to a post on .govwatch (The National Archives Is Quietly Destroying Millions of Documents April 08, 2008 by Coby Logen), a recent memo at the National Archives and Records Administration says:

After considering our other records management program priorities for FY 2008, availability of harvested web content at other "archiving" sites (e.g., www.archive.org), and the resources required for conducting and preserving a government-wide web snapshot, NARA has determined that we will not conduct a web harvest or snapshot at the end of the current Administration.

Logen says that "Not capturing federal web sites now may mean losing millions of web pages authored under the Bush administration when leadership changes in January 2009."

John Wonderlich at the Sunlight Foundations comments that "The fact that digital preservation is done by others outside NARA isn't an excuse for NARA to abdicate their responsibility, but an argument that they should be capable of fulfilling it." (Digital Preservation Under Threat? by John Wonderlich on April 9, 2008)

This seems yet another example of the government saying it cannot and therefore it won't. (The NARA/TGN contract as a bad precedent). Call it the Katrina of digital preservation?

The New York Times sums up the underlying issue nicely yesterday: "In Storing 1's and 0's, the Question Is $" (By John Schwartz, New York Times, April 9, 2008). It is not a technological issue; it is an issue of funding and policy and control. (See: The Technical is Political.)