For all you (de)classification geeks out there, here's an interesting new .gov blog to add to your blogrolls. NARA's Public Interest Declassification Board now has a blog called Transforming Classification. They'll be posting a bunch of white papers on various topics over the coming months. I'm looking forward to the conversation.

The Public Interest Declassification Board is an advisory board established by Congress to promote the fullest possible public access to a thorough, accurate, and reliable documentary record of significant U.S. national security decisions and activities. The Board’s mandate includes advising the President and other government officials on policies deriving from the issuance by the President of Executive orders regarding the classification and declassification of national security information...

...President Obama has charged the Board with designing a more fundamental transformation of the security classification system. In response to his request, we are proposing new solutions that address the shortcomings of the current system and tackle the challenges of digital records. By reducing inefficiencies and increasing public access, our proposals aim to improve the classification/declassification’s system capacity to protect and serve the American people.

Every other Wednesday over the next eight weeks, we will post either two or three “white paper” synopses to the blog describing an element of our proposed transformation.

[HT to Meredith Stewart]

Lunchtime Listen: Lawrence Lessig, speaking at CERN, talks about how hard, expensive, and limited our access is to scholarly information and how we have created this system.

• The architecture of access to scientific knowledge: just how badly we have messed this up, Lawrence Lessig CERN Colloquium and Library Science Talk, Streaming video (18 April 2011)

He mentions the problem of prohibiting access to an individual chart in an article posted for free on the internet. (Around the 16 minute mark: slide 189 et seq. [don't be intimidated by the number of slides; Lessig's presentation style is wonderful and clear and fast and enjoyable and no slide stays on the screen for more than a couple of seconds].)

We see this same problem in government information that incorporates copyrighted information with the results being anything from limited access to prohibited access to the non-copyrighted information. This is the poison pill problem of copyright.

David Ferriero, the Archivist of the U.S., will be testifying next week at a hearing that will address digital preservation of White House social media.

• White House officials to explain social-media policies next week, By Gautham Nagesh, The Hill (04/28/11).

  Senior White House officials will explain the Obama administration's policy for preserving tweets and other messages sent using social networks at a House Oversight hearing next Tuesday.

  The White House's official policy states that all messages between the president and his staffers and third parties on social-media sites like Twitter and Facebook must be preserved under the Presidential Records Act. That includes all tweets as well as direct messages and replies sent to official accounts.

  But panel Chairman Darrell Issa (R-Calif.) said it remains unclear what the administration's policies are for unofficial accounts still used for official communications, such as the personal accounts of some staff members....

Speaker Boehner and Majority Leader Cantor on Legislative Data Release, by John Wonderlich, Sunlight Foundation, (April 29, 2011).

Speaker Boehner and Majority Leader Cantor today sent a letter to the Clerk of the House calling for better access to the House's electronic data.

The letter asks that the House release its "legislative data" in "machine-readable formats" and establish standards for the House and its committees to include open data formats such as XML.

Obama orders streamlining of federal websites, By Joseph Marks, nextgov (04/27/2011)

President Obama issued an executive order on Wednesday tasking federal agencies with creating new technology-based plans to improve their customer service within six months.

The plan is also aimed at streamlining the clutter of more than 20,000 federal websites...

Executive Order--Streamlining Service Delivery and Improving Customer Service, The White House, Office of the Press Secretary (April 27, 2011).

Within 180 days of the date of this order, each agency shall develop, in consultation with the Office of Management and Budget (OMB), a Customer Service Plan (plan) to address how the agency will provide services in a manner that seeks to streamline service delivery and improve the experience of its customers.

Well, that headline is a bit misleading, but also a bit troubling. According to the Washington Post, the White House is ordering federal agencies to cancel print subscriptions to the Federal Register, the government’s own journal of official activities. "The move means that about 4,700 fewer editions of the Federal Register will be printed for executive branch agencies, saving the government at least 4 million annually, according to the White House."

But this is a little misleading on the cost-saving front, and may mean that in Congress' zeal to cut budgets, the GPO may not have the funding available to produce the Federal Register and other mandated and vital publications like the Congressional Record. Public Printer Bill Boarman, in a Mar. 17, 2011 Senate Appropriations hearing for the Government Printing Office, stated that 70% of the cost and work of publishing the Congressional Record is done in pre-press, and many of the same duties necessary to publish it in print are still necessary to put it out digitally.

In fact, there's a new bill working its way through Congress now to further defund GPO's ability to produce the record of the US government. The bill, H.R. 1626: Prevent the Reckless, Irresponsible, Needless Typography (PRINT) Act of 2011, was introduced 2 weeks ago by Congresswoman Candice Miller (R-MI). Stay tuned for more.

I guess this is a big CRS news day! Huffington Post reported on 4/26 that a newly released Congressional Research Service (CRS) Report requested by Senator Bernie Sanders (I-VT) found that the nation’s largest banks profited off the federal government’s bailout programs by borrowing cash for next to nothing, then lending it back to the federal government at substantially higher rates. The report has been posted at Sanders' Web site (PDF).

A newly-released study from the Congressional Research Service bolsters claims that the nation's largest banks profited off the Federal Reserve's financial crisis-era programs by borrowing cash for next to nothing, then lending it back to the federal government at substantially higher rates.

The report reinforces long-held beliefs that the banking system in essence engaged in taxpayer-financed arbitrage: They got money for free, then lent it back to Uncle Sam while collecting juicy returns. Left out of the equation are the millions of everyday borrowers, like households and small businesses, who were unable to secure loans needed to tide them over until the crisis ended.

The Fed released records under pressure in December and March that showed the extent of its largesse. The CRS study shows for the first time how some of the most sophisticated financial firms could have taken the Fed's money and flipped easy profits simply by lending it back to another arm of the government.

If you are a metadata librarian and would like to learn about metadata for social science data, check out the newest issue (April 2011) of DDI Directions, the newsletter of the Data Documentation Initiative. Then, browse the DDI website.

DDI is a specification for documenting social science data (e.g., census data, public opinion polls, economic time series, social surveys, government data, etc.). It enables you to document the entire lifecycle of data: conceptualization, collection, processing, distribution, discovery, analysis, repurposing, and archiving.

CRS Report Withheld By USTR Confirms That ACTA Language Is Quite Questionable, by Mike Masnick, TechDirt, April 26, 2011.

We're happy to announce that we've been able to get our hands on the -- until now -- secret Congressional Research Service analysis of ACTA.... [I]t shows that the language used by the USTR in ACTA has lots of weasel words that let them claim it doesn't impact US law, but the interpretations of the language could very much impact US law.

"...The Committee concludes that Congress intended to retain control over this document and that it is not an agency record subject to FOIA."

"Potential Implications For Federal Law Raised By The October 2010 Draft Of The Anti-Counterfeiting Trade Agreement - ACTA" CRS Memorandum To Honorable Ron Wyden From Brian T. Yeh, Legislative Attorney, American Law Division, 7-5182 October 29, 2010 7-5700 Redacted.

A new research paper that evaluated the information literacy skills of first-year dental students reports:

Nearly half of students missed one or more question components that required finding an evidence-based citation. Analysis of the survey revealed a significantly higher percentage of students who provided incorrect responses reported using Google as their preferred online search method. In contrast, a significantly higher percentage of students who reported using PubMed were able to provide correct responses.

Following a one-hour intervention by a health science librarian, virtually all students were able to find and retrieve evidence-based materials for subsequent coursework.

Kingsley, Karl, Gillian Galbraith, Matthew Herring, Eva Stowers, Tanis Stewart, and Karla Kingsley, Why Not Just Google It? An Assessment of Information Literacy Skills in a Biomedical Science Curriculum., BMC Medical Education, 11 (2011), 17 doi:10.1186/1472-6920-11-17.

Hat tip to InfoDocket!

From UNREDACTED (Decades Later, NARA Posts Documents on Guatemalan Syphilis Experiments, by Kate Doyle, April 25, 2011):

Between 1946 and 1948, U.S. public health researchers infected hundreds of Guatemalan prisoners, soldiers and mentally ill patients with syphilis and gonorrhea, without their knowledge or consent, in order to test the effectiveness of penicillin. The experiments were carried out in Guatemala under the cloak of confidentiality, and the results were never published in the United States. But after a scholar discovered archives chronicling the program at the University of Pittsburgh and published her findings last year, the U.S. National Archives and Records Administration (NARA) took custody of the documents and on March 29 made them publicly available.

From the NARA press release:

The National Archives at Atlanta announced that on March 29, 2011, it will release online the papers of Dr. John C. Cutler. Dr. Cutler, a former employee of the U.S. Public Health Service, 1942-1967, was involved in research on Guatemalan soldiers, prisoners, and mental health patients who were exposed to the syphilis bacteria. The collection is available online http://www.archives.gov/research/health/cdc-cutler-records and at the National Archives at Atlanta, located at 5780 Jonesboro Road, Morrow, Georgia, 30260.

A preview of the new House.gov site is now available:

• preview.house.gov

Gary has posted a useful summary overview at infodocket.

Sunlight Labs has a more in depth post here.

For libraries that rely on pointing to URLs rather than preserving information in their own digital libraries, the new report from the Chesapeake Project provides sobering, factual data on the reliability of that strategy.

In an examination of "link rot" the project found that 30.4% of URLs examined no longer provide access to their original information.

This study is particularly relevant to government information specialists because more than 90% of their sample URLs were from state governments (state.[state code].us), organizations (.org), and government (.gov) the top-level domains.

The Chesapeake Project Legal Information Archive, which harvests and preserves relevant digital legal information from the web, has been producing reports on "link rot" for several years. They define link rot as "a URL that no longer provides direct access to files matching the content originally harvested from the URL and currently preserved in the Chesapeake Group's digital archive."

Their new report is now available:

• "Link Rot" and Legal Resources on the Web: A 2011 Analysis, by the Chesapeake Digital Preservation Group, [April 2011].

In one interesting finding, the report says that the rate of loss of information slowed in the last year: "Whereas the prevalence of link rot among URLs in the sample nearly doubled every year during the first three years of the study, it slowed significantly in the fourth year." The report makes clear that although 30.4 percent, or nearly one-third, of the archived titles have disappeared from their original URLs since the beginning of the program in 2007, only 2.5 percent of URLs were lost to link rot within the past year.

Their data show that cumulative link rot frequency for .gov files was 10% in 2008, 13% in 2009, 25% in 2010, and 31% in 2011. There was an interesting development in that some state-level URLs that were inaccessible in 2010 were once again accessible when re-checked for the 2011 analysis. The cumulative link rot frequency for state level URLs was still almost as high as for the federal URLs: 10.8% in 2008, 15.8% in 2009, 32.1% in 2010, and 30.4% in 2011. Even with that slight improvement at the state level, the overall cumulative link rot percentage rose in 2011 (30.4%) over 2010 (27.9%). Another way of looking at this is that of the documents the Chesapeake Project has preserved, only 69.6% were still available at their original URL as of the 2011 study.

In an earlier study, the authors qualified their findings, noting that the findings are "not meant to be broadly applicable or to provide a representation of link rot throughout the universe of web resources" but only reports on those items in the Chesapeake Project archive. The studies do provide "insight into the vulnerability of law- and policy-related web resources selected by experienced law librarians from seemingly stable open-access web sites hosted by reputable organizations and state and federal governments."

Significantly, "All of the Web resources described in this report that have disappeared from their original locations on the Web remain accessible via permanent archive URLs here at legalinfoarchive.org, thanks to the Chesapeake Group's efforts. "

Nextgov reports that one Energy Department website, which is devoted to sharing clean energy practices with industry, was affected by the recent Amazon Cloud Failure.

State Department shifts digital resources to social media, By Alicia M. Cohn, The Hill (04/24/11).

With little fanfare, the State Department has abandoned America.gov -- an ambitious digital project launched three years ago to promote Democracy abroad -- and shifted its resources to social media projects.

The site, which provided original content translated into at least three languages besides English, "was meant to be a resource for cultural and policy information serving America’s interests abroad."

An announcement at http://America.gov says, "This site is being archived..." but provides no further information on what that means and if content that was on the site will be preserved or made available to the public.

Hat tip to Benton's Communications-Related Headlines!

From Slashdot:

don9030582 writes

"After Google announced it would permanently shutter its Google Videos collection, dozens of volunteers from around the world sprung into action in a massive effort to make a copy of the entire site. It was originally slated to go dark on April 29th, but now they have eliminated any such deadline and furthermore they will be migrating the collection to YouTube. We wish Google would have planned to do that from the beginning, but ultimately this is a victory for the preservation of user-generated content on the Internet."

Justia Opinion Summaries, Justia is providing FREE Daily & Weekly Opinion Summaries for all Federal Courts, and selected State Supreme Courts. U.S. Federal Courts, U.S. State Courts, Opinion Summaries by Practice Area.

Justia offers free case opinion summary newsletters for many US Appellate Courts. To sign up for the Case Summary Newsletters you first need to login to or create a Justia Account. Then you will be able to select the free newsletters you wish to subscribe to. Subscribe to summaries of US appellate court opinions at Daily.Justia.com. Practice area emails with summaries from all reviewed courts are sent weekly.

Hat tip to Sabrina I. Pacifici!

Random Samplings: The official blog of the U.S. Census Bureau.

Thanks to the Scout Report for pointing out this useful blog!

The U.S. Census Bureau provides a wealth of data about the population and economy of the United States, and certain aspects of their work are covered in this eminently engaging blog, "Random Samplings". The intent of this blog is "to describe the objective of their work and explain census and survey results". The blog began in September 2010, and visitors can search the archive of previous posts by category or date of publication. Some of the categories include "business ownership", "income", and "poverty".

-- U.S. Census Bureau: Random Samplings, Scout Report (2011-04-22)

The Scout Report has good links to CNET, Wired, and Wall Street Journal articles about Google's Map Maker project, as well as links to the American Memory Project and the David Rumsey Map Collection:

• With the release of Google Map Maker, users can contribute their own spatial knowledge, by Max Grinnell, The Scout Report (2011-04-22).

  In the previous millennium, those folks who wanted a high-quality map of their area might have had to go purchase or borrow an actual physical map. In recent years, online mapping tools and resources have sprouted like mushrooms after a hard rain. With all of that in mind, it is not so surprising that on Tuesday Google announced that it is allowing users to contribute changes to their very popular maps. This tool is called Google Map Maker...

As we look to relying increasingly on digital texts for discovery, access, and use of government information, it is worth understanding the issues (and difficulties) in accurately extracting text from a variety of sources. Here is a thirteen page paper that outlines the issues.

• Herceg, Paul M., and Catherine N. Ball, Reliable Electronic Text: The Elusive Prerequisite for a Host of Human Language Technologies, Mitre Technical Report (Mitre, 30 September 2010).

  Electronic text is a prerequisite for text-processing applications such as indexing and search, named entity recognition (NER), and machine translation (MT). For example, text that is printed on paper must first be converted to electronic form to make it searchable--typically, by scanning the pages and using Optical Character Recognition (OCR) to create electronic text.

  Not all electronic text is created equal, however. Electronic text comes in a wide variety of containers, from Microsoft Office documents to Oracle database records. Electronic text comes in a wide variety of character set encodings, such as Chinese "Big 5" or Unicode UTF-8. Electronic text may be written in any of the world's languages, including the special sublanguages of blogs, chat and forums. And from the end user's perspective, even formatting aspects of the electronic text may be important: for example, if the use case is translating a document for a customer, and the customer requires the translated document to be formatted to look like the original.

  From the application perspective, all aspects of electronic text potentially make a difference. Almost all products are language-specific, and will produce useful results only on supported languages. In terms of file types, some products accept Microsoft Word documents as input, while others may accept only plain text files. Some products may work best if all text is "normalized" to a single character set encoding such as UTF-8.

  The purpose of this paper is to explore factors to be considered when trying to match up text-containing files (in a variety of file formats, character set encodings, languages, etc.) with text-processing applications and use cases.

Hat tip to Info Docket!

"...and there is no foreseeable likelihood that it will do so..."

Secrecy Overwhelms U.S. Historical Record, by Steven Aftergood, Secrecy News (April 20th, 2011).

The [State] Department's "Foreign Relations of the United States" (FRUS) series is required to fully document the history of U.S. foreign policy no later than 30 years after the fact, but that’s not happening.

"No progress has been made toward bringing the [FRUS] series into compliance with the statutory requirement that volumes be published 30 years after the events they document," said the new annual report of the Advisory Committee on Historical Diplomatic Documentation.

So the CIA just got around to declassifying 6 of the U.S.'s oldest classified documents from WWI (1917 + 1918). They've posted them in their CIA FOIA reading room and the CIA Records Search Tool (CREST) at the National Archives (but to use CREST, a researcher must physically be present at the National Archives, College Park, Maryland :-|). That also means that the documents will also soon be available at the archive-it FOIA collection (I'm harvesting them as we speak ;-)).

These documents, which describe secret writing techniques and are housed at the National Archives, are believed to be the only remaining classified documents from the World War I era. Documents describing secret writing fall under the CIA's purview to declassify.

"These documents remained classified for nearly a century until recent advancements in technology made it possible to release them," CIA Director Leon E. Panetta said. "When historical information is no longer sensitive, we take seriously our responsibility to share it with the American people."

One document outlines the chemicals and techniques necessary for developing certain types of secret writing ink and a method for opening sealed letters without detection. Another memorandum dated June 14, 1918 - written in French - reveals the formula used for German secret ink.

"The CIA recognizes the importance of opening these historical documents to the public," said Joseph Lambert, the Agency's Director of Information Management Services. "In fiscal year 2010 alone, the Agency declassified and released over 1.1 million pages of documents."These documents, which describe secret writing techniques and are housed at the National Archives, are believed to be the only remaining classified documents from the World War I era. Documents describing secret writing fall under the CIA's purview to declassify.

"These documents remained classified for nearly a century until recent advancements in technology made it possible to release them," CIA Director Leon E. Panetta said. "When historical information is no longer sensitive, we take seriously our responsibility to share it with the American people."

One document outlines the chemicals and techniques necessary for developing certain types of secret writing ink and a method for opening sealed letters without detection. Another memorandum dated June 14, 1918 - written in French - reveals the formula used for German secret ink.

"The CIA recognizes the importance of opening these historical documents to the public," said Joseph Lambert, the Agency's Director of Information Management Services. "In fiscal year 2010 alone, the Agency declassified and released over 1.1 million pages of documents."

Declassified CIA documents (all pdf):

• Document 1
• Document 2
• Document 3
• Document 4 -- This is the one with the steps for checking for invisible ink.
• Document 5
• Document 6

This was such cool news that Rachel Maddow went gaga over the news!

[HT to Gary Price for posting CIA Declassifies Oldest Documents in U.S. Government Collection (1917 + 1918), View Them Online on InfoDocket. Thanks Gary!]

Sunlight has a story about the government's Federal Awardee Performance and Integrity Information System (FAPIIS) website that came online April 18th.

• The Worst Government Website We've Ever Seen?, by Tom Lee, Sunlight Foundation Blog (April 19, 2011).

  Contracting databases are part of the world of procurement, procurement is heavily influenced by the Defense Department, and DoD has a proud heritage of producing websites so ugly that they make you want to claw out your eyes. So FAPIIS has company. But if this was just a question of aesthetics, we wouldn't be complaining.

  Assuming you're using one of the few web browsers in which the site works at all (Chrome and Safari users are out of luck), the experience is off-putting from the start, as users are warned that their use of the site may be monitored, surveilled, or otherwise spied upon (you don't necessarily surrender your right to speak privately to your priest by using the website, though--thanks for clearing that up, guys!). Perhaps this is why their (arguably superfluous) SSL certificate is utterly broken....

Among other things, Tom notes that the site uses "CAPTCHA."

[T]he use of a captcha to gate government data is outrageous. Government should be making its data more accessible and more machine-readable. Captchas are designed to interfere with automated tools that facilitate malicious acts. But downloading government data is decidedly not a malicious act. Why are we trying to limit machines' ability to use this data?

Over at the Government Book Talk blog, Jim Cameron announced the publication of the first collection of President Obama's public papers.

The GPO bookstore page for the Public Papers of the Presidents of the United States, Barack Obama, 2009, Book 1 says that an eBook edition of this historic publication is coming soon and provides a link to request notification.

The full text is already available online as one large (9232 KB) PDF file, and as separate, individual PDF documents. (I hope NBC doesn't claim copyright to the interview on the Tonight Show!)

Kate Theimer has posted her presentation, Extinction or Evolution? (A slowed down version of my Smithsonian Ignite presentation on ArchivesNext. Although she addresses the future of archives, there are a lot of parallels to libraries. One point that I think is particularly relevant to government information is that if 10% of all information meets the needs of 90% of the public, what will happen to the rest of the information? Thanks, Kate, for a provocative presentation!

More presentations from Ignite:

• the Program section of the Ignite Smithsonian site
• selections at ArchivesNext