White House bars agencies from posting some statistics, by Aliya Sternstein, NextGov (01/27/2010).

According to this article, datasets posted to data.gov by the Nuclear Regulatory Commission, the Peace Corps, the Agriculture Department's Food Safety and Inspection Service, the Interior Department's Bureau of Reclamation, and the Social Security Administration have been removed by the Office of Management and Budget "because they raised privacy, security or other concerns."

The article is based on work done by OpenTheGovernment.org which is tracking agency participation with the Open Government Directive here.

You almost certainly have seen at least one story in the past week about "Open Government" and the release of new data. Reporters have slowly been picking up on a massive release of information spurred by President Obama's Open Government Directive. (See: New 'high value' data posted to data.gov.)

Below are a few announcements and stories that you may find of interest.

But, in addition to all the data released this week was a new policy that will, potentially, affect usability of government information in the future. In the December 8, 2009 memo (Open Government Directive [pdf] Memorandum For The Heads Of Executive Departments And Agencies, M-10-06, Peter R. Orszag Director, Office of Management and Budget) that implemented the President's Open Government Initiative, OMB specifically mandates open file formats.

To increase accountability, promote informed participation by the public, and create economic opportunity, each agency shall take prompt steps to expand access to information by making it available online in open formats.

And, OMB defines open formats as:

An open format is one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.

This is big news for two reasons. First, it should lead the government away from proprietary formats which are hard to preserve, hard to re-use, and typically require either proprietary software or only operate on specific platforms, or both. Think: documents in ODF format rather than Microsoft Word. Second, the directive mandates formats "without restrictions [on] re-use." Think: no DRM (and no licensing restrictions!).

As the ODF Alliance noted back in December when the OMB memo was released, much of government information is still released in "documents" which are not ideal for re-use of information even when the document formats are open. But, this is still an important, essential step:

Like it or not, government bureaucracies are still very document-centric and there is a lot of government “data” stored in documents, the challenge being how to provide easy access to this data.

...With today's announcement, the Obama Administration has taken an important step on open government data and acknowledged the role open formats play in this regard. For document-centric governments, an open document format remains essential to delivering on this promise.
-- Obama Administration To Require Government Agencies to Make Information Available in Open Formats. ODF Alliance, December 08, 2009.

Open formats will help libraries that want to preserve digital government information by making it easier and less costly to do so.

Here are some of the announcements about releases of new government data:

• Open Government Initiative White House.
• Another Milestone In Making Government More Accessible and Accountable. White House.
• U.S. Government, OSTP, Open New Troves of Data to the Public
• Justice Department Announces Release of New Information Online as Part of President’s Open Government Initiative
• How "Open Gov" Datasets Affect Parents and Consumers. White House.
• Open Government Initiative. White House.

Government Posting Wealth of Data to Internet, By THE ASSOCIATED PRESS, New York Times (January 22, 2010).

The Obama administration on Friday is posting to the Internet a wealth of government data from all Cabinet-level departments, on topics ranging from child car seats to Medicare services.

...Under a Dec. 8 White House directive, each department must post online at least three collections of "high-value" government data that never have been previously disclosed.

...All the new data collections will be added to the government's Web site, data.gov.

The United Kingdom has it's own version of data.gov and it has the added cachet of being promoted and advised by Sir Tim Berners-Lee.

• data.gov.uk

  This site seeks to give a way into the wealth of government data. [T]his means it needs to be: easy to find; easy to licence; and easy to re-use. We are drawing on the expertise and wisdom of Sir Tim Berners-Lee and Professor Nigel Shadbolt to publish government data as RDF – enabling data to be linked together.

• Tim Berners-Lee unveils government data project, BBC (21 January 2010).

  Web founder Sir Tim Berners-Lee has unveiled his latest venture for the UK government, which offers the public better access to official data.

  A new website, data.gov.uk, will offer reams of public sector data, ranging from traffic statistics to crime figures, for private or commercial use.

In this case, the Open Knowledge Foundation has gone a long way toward clarification…

See specifically SEE: http://opendefinition.org/

From the the “Open Knowledge Definition” home page:

“In the simplest form the definition can be summed up in the statement that ‘A piece of knowledge is open if you are free to use, reuse, and redistribute it’. “

In detail the definition suggests: [for sake of clarity, I have here deleted –- marked -- the “comments” appended to individual elements of the definition – please see the site for these…]

“The term knowledge is taken to include:

1. Content such as music, films, books
2. Data be it scientific, historical, geographic or otherwise
3. Government and other administrative information

“Software is excluded despite its obvious centrality because it is already adequately addressed by previous work.

“The term ‘work’ will be used to denote the item of knowledge at issue.

“The term ‘package’ may also be used to denote a collection of works. Of course such a package may be considered a work in itself.

“The term ‘license’ refers to the legal license under which the work is made available. Where no license has been made this should be interpreted as referring to the resulting default legal conditions under which the work is available.”

+++++++++++++++++++++++
The Definition

“A work is ‘open’ if its manner of distribution satisfies the following conditions:

1. Access: The work shall be available as a whole and at no more than a reasonable reproduction cost, preferably downloading via the Internet without charge. The work must also be available in a convenient and modifiable form.
2. Redistribution: The license shall not restrict any party from selling or giving away the work either on its own or as part of a package made from works from many different sources. The license shall not require a royalty or other fee for such sale or distribution.
3. Reuse: The license must allow for modifications and derivative works and must allow them to be distributed under the terms of the original work. The license may impose some form of attribution and integrity requirements: see principle 5 (Attribution) and principle 6 (Integrity) below.
4. Absence of Technological Restriction: The work must be provided in such a form that there are no technological obstacles to the performance of the above activities. This can be achieved by the provision of the work in an open data format, i.e. one whose specification is publicly and freely available and which places no restrictions monetary or otherwise upon its use.
5. Attribution: The license may require as a condition for redistribution and re-use the attribution of the contributors and creators to the work. If this condition is imposed it must not be onerous. For example if attribution is required a list of those requiring attribution should accompany the work.
6. Integrity: The license may require as a condition for the work being distributed in modified form that the resulting work carry a different name or version number from the original work.
7. No Discrimination Against Persons or Groups: The license must not discriminate against any person or group of persons.
8. No Discrimination Against Fields of Endeavor: The license must not restrict anyone from making use of the work in a specific field of endeavor. For example, it may not restrict the work from being used in a business, or from being used for military research.
9. Distribution of License: The rights attached to the work must apply to all to whom the work is redistributed without the need for execution of an additional license by those parties.
10. License Must Not Be Specific to a Package: The rights attached to the work must not depend on the work being part of a particular package. If the work is extracted from that package and used or distributed within the terms of the work's license, all parties to whom the work is redistributed should have the same rights as those that are granted in conjunction with the original package.
11. License Must Not Restrict the Distribution of Other Works: The license must not place restrictions on other works that are distributed along with the licensed work. For example, the license must not insist that all other works distributed on the same medium are open.”

In the sciences, the general case for sharing of all scientific knowledge (and knowledge resources) has long been clearly articulated. Robert K. Merton, sociologist of science at Columbia made the case in 1942 -- where he said: “The substantive findings of science are a product of social collaboration and are assigned to the community. They constitute a common heritage in which the equity of the individual producer is severely limited…”
[SEE: Robert K. Merton, “A Note on Science and Democracy,” Journal of Law and Political Sociology 1 (1942): 121.] at about the same time (1941) a "Declaration of Scientific Principles" appeared in Nature -- “7. The pursuit of scientific inquiry demands complete intellectual freedom. And unrestricted international exchange of knowledge…“ [SEE:“The Commonwealth of Science, ” Nature No.3753 October 4, 1941] These scientific values have been affirmed and re-affirmed many times. In fact, the predecessor to UNESCO was the League of Nations Committee on International Intellectual Cooperation... [SEE related: "The union of International Associations" and "The Mundaneum" http://www.uia.be/node/85 ]

The principle that access to knowledge is an essential human right (and fundamental to effective citizenship) has also been widely affirmed. The Universal Declaration of Human Rights, Article 19 declares: "Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers." [SEE: http://www.un.org/en/documents/udhr/index.shtml ]

In fact, the authors of the US Constitution realized that access to knowledge was essential to the public welfare -- the notion of limited monopoly on "intellectual property" -- as defined in the provisions for patent and copyright make this clear and by the recognition of the public domain. (Thomas Jefferson, corresponding with his Secretary of War in 1807, wrote: "The field of knowledge is the common property of all mankind.")

In 1954, when Edward R. Murrow asked him about patenting the polio vaccine, Jonas Salk famously commented" "That would be like patenting the sun..." 35 years later Nobel Laureate Joshua Lederberg was warning of the deterioration of the ethic of sharing [SEE: “Data Sharing: A Declining Ethic? -- Commercial pressures and heightened competition are testing the notion that scientific data and materials should be widely shared.” Science v. 248 p952- 957, 25 May 1990]. It seems more than a little ironic that now -- 50 years later -- "20% of human gene DNA sequences are patented" [Science Magazine Policy Forum: K.Jensen and F. Murray, "Intellectual Property: Landscape of the Human Genome," Science 14 October 2005: Vol. 310. no. 5746, pp. 239 - 240].

Thus sadly, despite clear and longstanding articulation of the principles of free and open access to knowledge, there has been a strong countervailing trend toward restriction of knowledge ["commoditization" -- SEE: J. Birkinshaw and T. Sheehan, “Managing the Knowledge Life Cycle,” MIT Sloan Management Review, 44 (2) Fall, 2002:77].

So, assuming (as in my previous post) that librarians are both stewards and advocates -- how do we make our case...? 1) We marshal all available historical and philosophical evidence for support of open access [in effect, I've cited just a few of the diverse sources for such a case...] 2) we organize and broadly disseminate by publishing, presenting, discussing, teaching 3) we insist upon evidence-based public policy with full transparency -- not only of data but of the logic that directs the definition of data as evidence 4) we insist upon transparent processes by which data can be transparently and effectively scrutinized -- this means specifying all forms of transformation to which data are subject and presenting the "chain of custody" / provenance of data thus certifying both logical validity and technical integrity 5) from a policy perspective, we begin with the "lowest hanging fruit", which politically means we start with bio-medicine (humans are naturally enough -- anthropocentric) -- thus NIH, CDC and UN/WHO and their evolving policies -- but we also push in other domains like conservation, agriculture and agrarian science, education... 6) we analyze carefully and advocate for the broad implementation of "qualified peer review mechanisms" (NSF provides an excellent model) 7) we aggressively advocate for K-gray science literacy...

More about all and each of these points will follow...

The University of Minnesota Libraries has taken a new approach to its planning process this year to help deal with seemingly conflicting realities. On the one hand, everything said publicly by University administration indicates that the U's financial future is Not Good. On the other, the Libraries has several projects in place that are innovative and many, many more on hold that would also be fabulous. These projects are in addition to the regular day-to-day work of a library. Something has to give somewhere, but the Libraries can't just metaphorically throw its hands in the air and say "the heck with this, I'm out".

So, the Libraries is hosting a speaker series with the goal of moving from lemons to lemonade. There have been two speakers so far - Lorcan Dempsey and Paul Courant. See https://wiki.lib.umn.edu/Staff/UniversityLibrariesSpeakerSeries for more information - future speakers will be Jim Neal and Clifford Lynch. While online access is limited during the talks, the future speakers will be recorded and the webcasts posted soon after for all to view. And, at the risk of sounding sycophantic, I believe our University Librarian's - Wendy P. Lougee - opening remarks are also worth a listen on their own merits.

Lorcan Dempsey - "Discovery and Delivery"
Dempsey began by describing levels of rarity of library collections based on OCLC data with the suggestion that where libraries should focus their expenditures (presumably on preservation, simply having the space to hold, doing really good digitization, etc) is on the rare items. Non-rare items could reasonably be entrusted to network-level services like the Hathi Trust. He then presented a typology of library collection types sorted by rarity and current levels of stewardship. Government publications fell into high stewardship, but low rarity. Dempsey acknowledged that this was a broad characterization and that there might be rare items within a category like government publications or maps. Also, the University of Minnesota is a partner in the Hathi Trust and has sent some of its government publications collection in for digitizing, so the Libraries are already on the path he's describing here. Caveats aside, I feel that he provides a well-reasoned and evidence-based rationale for shifting stewardship away from non-rare items and towards collections that are getting no real attention at all. This was only a tiny portion of his overall talk and I recommend going through the entire powerpoint or webcast to get the full presentation.

Presentation, Webcast, Related Readings: https://wiki.lib.umn.edu/Staff/UniversityLibrariesSpeakerSeries#dempsey

Paul Courant - "Scholarly Communications and Publishing"
Courant's talk can be best described as a reflection on just what is it that we'd like to pay for. He framed part of the problem in terms of the Parable of the Anarchist's Annual Meeting (see http://www.econ.ucsb.edu/~tedb/Journals/anarchists.pdf). In short: with coordination - either between libraries or between libraries and smaller publishers or both - we can take at least some control of the journal publishing arena. We already spend a fortune on a situation we don't like. Surely the logical thing is to begin to spend some money on creating a situation more to our liking. This includes taking on more of a publishing role and allying ourselves with societies and small publishers (including university presses) who might be more interested in the benefits of open access that the big vendors. However, when I asked if he was advocating canceling contracts with big vendors, he answered (I'm paraphrasing) "Well, probably not. Well, not entirely. Might want to pass on those Big Deals they offer though."

He also felt the library community should speak up loudly in favor of the recent RFI from the Office of Science and Technology Policy regarding increased access to the products of federally funded research. At the same time he reiterated that open access isn't exclusively a library issue. In fact, he said it's a faculty issue. Libraries need to keep pushing on the topic, but pushing faculty to understand that this is an arena they can control if the choose to do so.

Courant isn't a librarian - he's an economist by background and I found his application of an economics perspective refreshing. Again, like Dempsey's talk, there was no magic "the Libraries should do this" moment because we are in a tough spot without easy resolution. But, also like Dempsey's talk, he has a great way of expressing the issues facing libraries.

Presentation, Webcast, Related Readings: https://wiki.lib.umn.edu/Staff/UniversityLibrariesSpeakerSeries#courant

I don't know if these speakers really will lead to concrete ideas for coping with our budget problems, but I sure am glad we're having them - each one has been thought-provoking.

The Office of Science and Technology Policy (OSTP) is requesting input regarding enhanced access to federally funded science and technology research results, including the possibility of open access to them. Comments can be e-mailed to publicaccess@ostp.gov. The deadline for comments is January 7, 2010. For more, see the Federal Register announcement:

Federal Register: December 9, 2009 (Volume 74, Number 235) Page 65173-65175

On his first day in office, the President issued a Memorandum on Transparency and Open Government that called for an "unprecedented level of openness in government" and the rapid disclosure of one of our nation's great assets--information. Moreover, the Administration is dedicated to maximizing the return on Federal investments made in R&D. Consistent with this policy, the Administration is exploring ways to leverage Federal investments to increase access to information that promises to stimulate scientific and technological innovation and competitiveness. The results of government-funded research can take many forms, including data sets, technical reports, and peer-reviewed scholarly publications, among others. This RFI focuses on approaches that would enhance the public's access to scholarly publications resulting from research conducted by employees of a Federal agency or from research funded by a Federal agency.

[Thanks to Charles Bailey for originally posting to DIGLIB list!]

Even with this year's introduction of S.Res. 118 and H.R. 3762, it is still important to encourage our Government to make Congressional Research Service (CRS) Reports publicly accessible online!

Thus, I finally updated the latest list of Bills and contact information for the sponsoring Congressmen in the Delicious.com "CRS" tag Delicious.com "CRS" tag.

See also: CRS Reports to the People! Part 1 and Part 2 for more information on how to contact/write to your Congressmen.

Following up on Can Proprietary Formats make Government More Open? :

Josh Tauberer of govtrack.us, points us to The good and the bad of PDFs (OpenGovData.org wiki) in which Kevin Lyons, who works for the Nebraska legislature, wrote up some guidelines for PDF in government.

Lyons reminds us that not all PDF files are equal and he enumerates some of the advantages and disadvantages of encapsulating government information in PDFs.

Given how popular the PDF standard itself is, it shouldn't be a surprise that the term PDF actually covers a wide variety of different types of files. While all PDF files fit the PDF standard, there are several different subtypes of PDF that are helpful in the government world.

States urged to create data catalogs, by Joab Jackson, Federal Computer Week (Oct 07, 2009).

Federal agencies shouldn't be the only ones to open their data for the public — states and local governments should also be ramping up efforts to become more transparent, the National Association of State Chief Information Officers (NASCIO) contends in a newly published report.

The first thing a state should do is create a one-stop portal, or data catalog, for all its publicly-accessible data, along the lines of the White House's Data.Gov , the report states.

Law.Gov: America's Operating System, Open Source, by Carl Malamud, O'Reilly Radar (Oct 15, 2009).

Public.Resource.Org is very pleased to announce that we're going to be working with a distinguished group of colleagues from across the country to create a solid business plan, technical specs, and enabling legislation for the federal government to create Law.Gov. We envision Law.Gov as a distributed, open source, authenticated registry and repository of all primary legal materials in the United States. More details on the effort are available on our Law.Gov page.

Tim O'Reilly (O'Reilly & Associates) recently observed that Representative Jim Culberson (R-Texas) saw a demonstration of a product called SharedBook at the Gov 2.0 Summit and decided to use it to collect feedback from his constituents on the healthcare bill.

SharedBook is a publishing and annotation program advertised for a variety of purposes, including creation of dynamic documents:

Policy makers, nonprofits, educators, and special-interest communities can use SharedBook's platform to allow their members or constituents to engage in an online dialogue on bills, rules, research and other important documents. Starting with highlighted excerpts from the original content, a series of comments and replies can be posted and shared with any and all interested users to facilitate a pointed and detailed discussion. The source document is locked down and the community discussion is stored and presented back as footnotes at a granular level.

My first reaction to this was that opencongress.org already provides an excellent interface for viewing and commenting on bills before Congress, including the House's health care bill, why go to the trouble of setting it up for this one bill? The answer is that Mr. Culberson is using SharedBook because he wanted comments only from his own constituents.

Here's the press release.

220 Years Later, It’s Time to Publish the Constitution Annotated Online in XML, By Daniel Schuman, Sunlight Foundation, (09/17/09).

The Constitution Annotated has been written by the Library of Congress for nearly 100 years, and contains analysis of nearly 8,000 U.S. Supreme Court cases.

Over the decades, GPO has published print versions of this extraordinary resource every two years, with limited electronic versions available from 1992 edition onward. Although the Library of Congress has drafted the Constitution Annotated in XML for a number of years, that data is no longer present when it is published online by GPO.

From Slashdot:

• Professor Posts "Illegal Copy" of Guide To Oregon Public Record Laws Slashdot, September 16, 2009.

The Attorney General of Oregon is claiming copyright over a state-produced guide to using public-records laws and sells the 326-page book for $25. The AG's offices says, "that's how the AG's office makes back the cost of producing the book" (A smackdown over Oregon public records, by Jeff Mapes, The Oregonian, September 14, 2009).

Bill Harbaugh, Professor of Economics at the University of Oregon, has posted scans of the guide on his website and is daring the AG to respond. He notes that the manual includes on its cover the famous James Madison quote, "A popular Government, without popular information, or the means of acquiring it, is but a Prologue to a Farce or a Tragedy..." and he says that, "Given that this very quote is prominently posted on the cover of the same manual which AG John Kroger is trying to keep off the internet, I hold with those who favor farce."