Home » Posts tagged 'less_access' (Page 28)
Tag Archives: less_access
White House documents found to be altered
Researchers at the University of Illinois say they have found evidence on the Whitehouse Web site that suggests "a pattern of revision and removal from the public record that spans several years, from 2003 through at least 2005. Instead of issuing a series of revised lists with new dates, or maintaining an updated master list while preserving copies of the old ones, the White House removed original documents, altered them, and replaced them with backdated modifications that only appear to be originals."
- Airbrushing History, American Style, Scott Althaus and Kalev Leetaru, University of Illinois Urbana-Champaign, November 25, 2008
- Iraq Ally Lists Were Altered, Study Shows, by Thom Shanker, New York Times, November 24, 2008.
NYT: Federal Files Blip Into Oblivion
This is a pretty good popular-press overview of the problems of digital preservation of government information and some of the steps being taken to address the problems.
- In Digital Age, Federal Files Blip Into Oblivion, By Robert Pear, New York Times, September 13, 2008.
The Achilles' heel of record-keeping is people. In an effort to save money, federal agencies are publishing fewer reports on paper and posting more on the Web. The Web site of the Environmental Protection Agency lists more than 50 "broken links" that once connected readers to documents on depletion of the ozone layer of the atmosphere. At least 20 documents have been removed from the Web site of the United States Commission on Civil Rights. They include a draft report highly critical of the civil rights policies of the Bush administration. 93 percent of [top officials surveyed at NASA] were violating federal requirements for preserving e-mail correspondence. "Most Web records do not warrant permanent retention," because they do not have "long-term historical value," the [National] Archives said. Alarmed at the possible loss of White House e-mail messages, the House passed a bill in July that would require agencies to preserve more electronic records. ... Republican opponents said the requirements would be onerous and costly. Mr. Bush has threatened to veto the bill, saying it could "interfere with a president's ability to carry out his or her constitutional and statutory responsibilities."See also: Citizens in the Dark? Government Information in the Digital Age. Continue reading
GAO *did* sell exclusive access to legislative history to Thomson West
A few weeks ago, Daniel had a great post, "GAO/Thomson-West Contract Raises Questions" in which he expanded on a Boing Boing post "Did the US gov't sell exclusive access to its legislative history to Thomson West?" and analyzed the Thompson-West contract with the GAO for digitizing 20,597 legislative histories of most public laws from 1915-1995. Today, Carl Malamud got an answer to his FOIA request to the GAO seeking access to the digitized images of those legislative histories. I'll let Carl tell it in his own words:
Well, the answer is now a definitive yes, that data has been sold down the river and is out to sea. Public.Resource.Org sent in a FOIA request to GAO on this topic seeking access to the scanned data. Today's letter answering our FOIA request spells out the bad news. Turns out the GAO doesn't even get the data, they simply are given an account on Thomson's service. The rest of the government doesn't get access to this data, and the public is invited to stop by the GAO headquarters and pay 20 cents per page to copy paper. This is one of those deals where the public domain got sold off ... GAO gets a bit of convenience by having their stuff scanned for them, but they gave up way more than they got in the deal, and the public (including government workers and public interest groups who need to consult this data) lost big-time.Carl has put up his paper trail explaining the story. Here's the link to the Scribd group with the full paper trail on this issue, and here's the link to last week's response from the GAO. This perfectly exemplifies the problems we see with government agencies entering into contracts with private companies to digitize public domain materials (see for example "NARA/TGN contract as a bad precedent"). We have no problem with government agencies contracting with private companies to digitize government information. The problem as we see it is that so many agencies seem ignorant of the fact that privatizing access to said digitized public domain information actually limits access in the long run. Continue reading
Can we identify or verify or prevent government website scrubbing?
One issue we at FGI are concerned about is that, when government information is not officially distributed to depository libraries and when official digital government information is available only from government-controlled web servers, then that information can (intentionally or unintentionally) be deleted or altered leaving historians, journalists, economists and other citizens with no clear, complete record of government activities.
From time to time there are stories about government websites being "scrubbed," i.e., of information being removed from them, but it is often difficult to determine if these stories are accurate. Since stories like this are often (perhaps, usually) published to make a political point, discussion of them often revolves around the political issue rather than the issue of the integrity and permanence of government information in the larger sense.
One such story this week gives us an opportunity to at least quickly and superficially examine the existence of a problem, if not its extent:
- White House Scrubs Web Site on the Economy, Perrspectives [Jon Perr], March 20, 2008.
Perr says that a flash animation and a paragraph on tax cuts, which were on the White House Jobs and Economic Growth web page (also referred to as the "Economy & Budget Policies in Focus" web page) on March 16, 2008, were removed and no longer available on March 20. The animation said:
- "18,000 jobs created in December 2007,"
- "Over 8.3 million new jobs created since August 2003"
- "Unemployment rate remains low at 5%."
- "President Bush's actions are moving our economy forward"
And the deleted paragraph read:
President Bush Continues To Call On Congress To Further Reduce Economic Uncertainty By Making His Tax Relief Permanent.
President Bush believes the most important action to ensure the long-term health of our economy is to make sure the tax relief that is now in place is made permanent. The 2001 and 2003 tax cuts are set to expire in less than three years. If Congress allows that to happen, 116 million taxpayers will see their taxes go up by $1,800 on average, and we will see an end to many of the measures that have helped our economy grow – including the 10 percent individual income tax bracket, reductions in the marriage penalty, the expansion of the child tax credit, and reduced rates on regular income, capital gains, and dividends.
Perr discoverd that MSN has a cached copy of that page (dated 3/8/2008) that includes the animation and text. This morning, I used WebCite to make a copy of the MSN copy. (The WebCite copy does not do a good job of retaining the layout of the original, but the Flash animation is there and viewable as is the text paragraph and should remain there even after MSN removes its cached copy.)
I checked the Internet Archive, but the most recent snapshot of www.whitehouse.gov/infocus/economy/ as of this morning is June 7, 2007. I did some Google searching and was not able to locate the Flash animation, but I was able to locate a series, of nearly identical ones:
- 30 Straight Months of Job Gains Unemployment rates are below the ...
- 29 Straight Months of Job Gains Unemployment rates are below the ...
- 24 Straight Months of Job Gains Unemployment rate drops to lowest ...
- 20 Straight Months of Job Gains Unemployment rate drops to lowest ...
- 92000 Jobs Created in July 2007 Unemployment Rate Remains Low at ...
If Google is an accurate way to judge the content of whitehouse.gov, it would appear that, the White House has maintained earlier versions of this animation but has not preserved this more recent one. But, we do not know how accurate or comprehensive or current Google is.
I also browsed the White House News releases for March 2008 page, because it appeared that similar information had migrated to various "Fact Sheets." Indeed, the text paragraph is in the March 7, 2008 Fact Sheet: Taking Responsible Action to Keep Our Economy Growing. I was not able to find a link to the animations, however.
This brings me to the question: "Can we identify or verify or prevent government website scrubbing?" My own tentative conclusions are:
- We cannot prevent the government from changing its own websites, so we cannot prevent "scrubbing."
- We can verify that a site has changed, but currently our tools are limited to a) commercial web crawlers (like google, MSN and Internet Archive, and b) individuals who regularly monitor websites, and c) web crawlers created by libraries using their own tools or those provided by others (such as Archive-it).
- While tools exist to monitor changes in a web site (e.g., Change Detection), I don't believe that we can use these to look for significant (e.g., loss-of-information) alterations.
What conclusions can we draw from all this? Since we do not know how commercial indexers such as Google and MSN work and what their criteria are and since they do not have preservation as a mission, we can hardly rely on them. While this particular example may be trivial in itself, it demonstrates that government information in the digital age, the "e-government" age, is volatile and fragile and that we do not have a system in place that is as reliable for digital content as the FDLP libraries were for non-digital content. While it is hard to imagine a system that would be robust enough to catch every single digital bit of government information from every agency for all time, it is possible to imagine a system that would capture much more than we do now.
That leads me to a conclusion that we at FGI have long advocated: Libraries should be building collections of digital government information and GPO should facilitate this by depositing government information in FDLP libraries. If libraries created collections that could be text-mined by scholars and researchers, it would be possible to better audit, analyze, and preserve government information and make it more difficult for information to be scrubbed without being discovered and exposed. Indeed, it would remove, to some extent, the motivation to "scrub" if it was well know that the information was preserved and easily discoverable.
The question we should be asking ourselves is: How much are we losing every day? The task is too big for any one library or any one government agency (i.e., GPO). And it is not a task that commercial entities like Google and MSN are likely to take on.
Continue reading
A comment on government contracts and harvesting
April 16, 2008 / 2 Comments on A comment on government contracts and harvesting
Over the past week, there have been some good conversations about government contracts to digitize government information and the National Archives decision to not conduct a web harvest or snapshot at the end of the current Administration. There is good news and bad news.
The good news
The good news is that NARA's decision was not nearly as bad as it appeared to be when it was first announced in a memo on March 27, 2008, which was circulated only to Federal records officers (see: The National Archives Is Quietly Destroying Millions of Documents). In a thoughtful post on its web site (National Archives and Records Administration Web Harvest Background Information, April 15, 2008, NARA; pdf version available), NARA outlines in detail the reasons why it would not conduct an end of administration web snapshot or harvest of Executive Branch websites nor require agencies to do so. The reasons, I think, are sound and in keeping with NARA's commitment to preserving information of historical value.
In addition, the NARA memo of April 15 makes explicit the fact that its decision and memo of March 27 do not apply to Presidential records or to records of the Congress. It says that "NARA will continue to conduct a web harvest of Congressional web sites" and that NARA "will also receive a snapshot of the White House website" noting that "Unlike Federal agencies governed by the Federal Records Act, the White House is governed by the Presidential Records Act, under which all Presidential records are treated as permanent and transferred to NARA for preservation at a Presidential Library."
The NARA "Background Information" document is also, I think, worth reading for its clear description of the shortcomings of web harvests in general. I think it is very useful for us to be reminded of these shortcomings to the extent that we believe we can rely on them as an adequate form of preservation.
In more good news, the NARA/TGN contract is not as bad as it could have been. I mentioned this in my earlier post here (The NARA/TGN contract as a bad precedent) and similar comments have been made in the useful and interesting thread over at ArchivesNext (NARA latest digitization agreement: One archivist's perspective). Merrilee Proffitt, of RLG, says in a comment there that the NARA model for contracts with third parties "actually comes out looking pretty good" when compared to the criteria described in the RLG paper Good Terms - Improving Commercial-Noncommercial Partnerships for Mass Digitization (by Peter B. Kaufman and Jeff Ubois, D-Lib Magazine, November/December 2007, Volume 13 Number 11/12).
The bad news
The bad news, as James pointed out this morning, is that the GAO contract for digitizing is very bad indeed (GAO *did* sell exclusive access to legislative history to Thomson West). Quoting Carl Malamud, James notes that GAO gets access to the digitized data but does not get a copy of its own; the rest of the government doesn't even get access to the data. The public is left with the option of going to GAO headquarters and paying 20 cents per page to copy paper! As Carl says, "This is one of those deals where the public domain got sold off."
This morning there was more bad news. Kate at ArchivesNext reports that the Citizens for Responsibility and Ethics in Washington (CREW) has a new report Record Chaos: The Deplorable State of Electronic Record Keeping in the Federal Government, that concludes "that the federal government is severely mismanaging its electronic records." CREW also says that a House Committee proposal to amend federal record keeping laws "is anemic and fails to make the substantial changes necessary to bring the federal government into the 21st century."
And even the good news is tempered by the fact that we have less than we could and are a long way from an even an adequate system of permanent preservation of digital information or a long-term solution to digitizing non-digital information. We will have to hope that the White House will deliver a snapshot of the White House web site and that the snapshot will be accurate and complete. The behavior of the White House with regard to electronic records and email does not make us optimistic. The NARA/TGN deal is better than the GAO/Thomson deal, but still leaves much to be desired and, as pointed out even by defenders of the deal, it is unlikely that we will ever have free, open, networked access to the digital information that TGN digitizes. That means the real effect of the deal is to privatize the information.
Comment
For me, the biggest disappointment in these latest developments is that librarians and archivists seem to be too willing to accept "good enough" and not willing enough to argue harder for "better." There are lots of people who have good reason to argue for less access, more fees, less privacy, and more control of information, but librarians and archivists should not be among them. I believe that we should not spend time making the case for the private sector; it is fully capable of making its own case. We should spend our time fighting for free, full, open, public access, usability of information, and long term preservation.
The primary mission of private sector companies is to make money, not to serve the public. They may serve the public as a by-product of making money, but no for-profit company will go to its owners and say "we are going to do the best thing for public access" without the qualification "that will make us money." Unfortunately "making money" often conflicts with public access. Politicians (and some bureaucrats) will argue for greater control of government information; some will argue for secrecy of government information on the one hand and privacy-invading policies on the other. Most government agencies do not have information access or long term preservation of their information as a primary mission and the exceptions are notable (e.g. LOC, NARA).
In contrast, the primary mission of many libraries and archives is to provide free public open access with long term preservation and usability. While others may have some of those pieces as secondary goals, few if any have them all. For many libraries and archives these goals are not just their primary mission but their defining characteristic.
While digitization and digital preservation are neither easy nor inexpensive, that doesn't mean that we have to pay any and all costs for them. The digital era should be making it possible to provide better access without giving up free use and reuse, without giving up open access, without turning over control to those whose primary mission is something other than free, open, public access and long term preservation. But increasingly we see a combination of politics and economics leaving us with contracts that trump copyright and fair use, with "access" being negotiated at almost any cost (including loss of control), with DRM technologies that prohibit easy (or any) reuse, and with privacy protections being deprecated or even ignored. Even in the case of the NARA/TGN contract that is legally "better" than the GAO/Thomson contract, we are left with the effect of two-tiers of access and network access being essentially privatized and fee-based.
I believe that librarians and archivists should be pushing the boundaries and insisting for more and better, not accepting some benefits by negotiating away the big benefits we could be getting in the digital age. This is particularly important for government information that is in the public domain. If we can't make this work for public information that is not copyrighted, how will we be able to do so for information that is?
I'm not arguing for a perfect, ideal world that is impractical to achieve. I am suggesting that we should fight for everything we can get. We should celebrate when we make inroads with a contract (like NARA/TGN) that is better than the others (like GAO/Thomson) but we should do so by committing to doing better next time. We should not accept this as "good enough" -- because it is not and we can do better next time. In fact, every time we accept a less-than-perfect deal as "good enough," we make it a little harder to make a better deal next time. We lower the bar if we accept "good enough" and stop trying to achieve better. We should not take the time to convince ourselves or the public that this is as good as we can get; we should take that time to admit to the limitations and trade offs and to commit to doing better next time.
There is lots written these days about "the future of libraries" and "the role of libraries in the digital age" and many people openly wonder if there is a place for libraries at all. I think there are several places where libraries have a unique role to play in society and the areas of digitization and digital access and preservation are important ones.
We need to make the case for the public; for free, open, public access; for long-term preservation and usability; for public accountability in the control of information; for reader privacy. Librarians and archivists have a unique role in doing that. In doing so, we will face an uphill battle and trade offs, but we should never lose sight of our unique role in society. We should never cheapen our professions by making the case for less (there are plenty of people to do that). We should always make the case for more. We will not always succeed and we will have to make trade offs. But we should always do so in the context of staking out a territory that is different from the private sector and those who are willing to get less. We should stand up for rights that others are not willing to fight for. We must fight for it when there are so many forces aligned against free, open access.
I'd like to see us emulate Carl Malamud and CREW and Brewster Kahle more and do less of making excuses for TGN and Thomson.
Continue reading →Continue Reading →