Our mission

Free Government Information (FGI) is a place for initiating dialogue and building consensus among the various players (libraries, government agencies, non-profit organizations, researchers, journalists, etc.) who have a stake in the preservation of and perpetual free access to government information. FGI promotes free government information through collaboration, education, advocacy and research.

Celebrate Sunshine Week with new CRS content now online

March 11, 2019 / Leave a comment

Happy Sunshine Week, the week where we celebrate government transparency, FOIA and all things open government information! There’s lots happening this week including the upcoming 1/2 day live and streaming celebration at the National Archives and Records Administration (NARA). But there’s also work to be done. Evidently, appropriators are holding up the smart, pro-transparency “HR 736 Access to Congressionally Mandated Reports Act” scheduled for a floor vote on Tuesday. Contact your Representative today and tell them to pass this important act!

There’s too much news happening this week to list it all — make sure to subscribe to the First Branch Forecast weekly newsletter published by Daniel Schuman and his crack team at Demand Progress to keep up to date! — but I did want to highlight the good news coming out of the LIBRARY of Congress. Slowly but surely, they’re expanding the number of Congressional Research Service (CRS) Reports being published on their site crsreports.congress.gov. They still don’t have the coverage of EveryCRSReport.com which includes 14,742 CRS reports (and still growing) but LoC is getting there so good on them. Celebrate Sunshine Week by leaving LoC a comment and contacting your Representative to tell them to vote for “HR 736 Access to Congressionally Mandated Reports Act”. Sunshine is the best disinfectant!!

Since launching, we’ve added hundreds of new reports and are working hard to include the back catalog of older CRS reports – a process that is expected to be complete later this month. Today, you can access more than 2,300 reports on topics ranging from the Small Business Administration to farm policy. 
 
Starting this week, the Library is making additional product types available on the site. The site now includes In Focus products, which are two-page executive level briefing documents on a range of policy issues. For example, recent topics include military medical malpractice and the Temporary Assistance for Needy Families (TANF) Block Grant. Another newly-added product type is the Insight, which provides short-form analysis on fast moving or more focused issues. Examples of topics include volcano early warning systems and Congressional Member Organizations. Users can filter by product type using the faceted search on the left hand of the search results page.

via New CRS Content Now Online | Library of Congress Blog.

25-Year-Old FOIA Request Confirms FOIA Delays Continue Unabated

March 8, 2019 / Leave a comment

Happy Sunshine week (well, technically it’s next week, March 10-16, 2019)! The National Security Archive did a massive FOIA audit which showed that FOIA delays and backlogs continue across federal agencies. the most interesting/disturbing to me were the requests that fell into a FOIA “referral black hole” where agencies refer to or consult with other agencies on “any FOIA request in which it feels another agency or agencies may possibly claim ownership of, or “equity” in, the information within the records.” These referrals often result in massive delays.

One of the easiest ways to better deal with these referral delays is to allow FOIA.gov‘s request form to be submitted to multiple agencies (or multiple units within agencies) if the requester feels that the question overlaps agencies. but If anyone has other good ideas for how agencies can more quickly deal with the “referral black hole” please send me an email at freegovinfo AT gmail DOT com.

Washington, D.C. March 8, 2019 – Five federal agencies have FOIA requests more than a decade old and one, the National Archives and Records Administration, has a FOIA request more than 25 years old, this according to a National Security Archive Audit released today to mark the beginning of Sunshine Week. The survey also found there is a correlation between agencies with the oldest FOIA requests and those with the largest FOIA backlogs.

The Archive Audit team parsed through the annual FOIA reports federal agencies are required to submit to the Department of Justice’s Office of Information Policy and found that while many agencies appear to have used new reporting requirements as a tool to address the oldest agency FOIA requests, others have let decades-old requests linger. The Archive used the Fiscal Year 2017 reports because they were the most comprehensive collection available at the time of publication due to the delay caused by the government shutdown, and will update this posting once the complete set of FY 2018 reports are available.

The key driver for FOIA requests that could be renting cars by now and growing backlogs is the “referral black hole.” Agencies currently refer or consult on any FOIA request in which it feels another agency or agencies may possibly claim ownership of, or “equity” in, the information within the records. This daisy chain of referrals can often result in decades-long delay, and the re-review of the same document by multiple agencies is redundant, costly, and inefficient.

via 25-Year-Old FOIA Request Confirms FOIA Delays Continue Unabated | National Security Archive.

Reject GPO’s proposal to drop metadata from CGP

March 3, 2019 / Leave a comment

The Government Publishing Office has a brief proposal to omit some metadata in the Catalog of Government Publications (CGP). As written in the announcement, the proposed change in policy seems simple and obvious. GPO says that, for publications in govinfo, “historic URLs” (the original, source URL) and PURLs (the Persistent Uniform Resource Locators) are “identical.” GPO reasons that, since the the two fields are “redundant,” the historic URLs are unnecessary:

“LSCM proposes to cease the inclusion of historic URLs only in catalog records for resources in govinfo.”

The announcement ask for feedback on how the policy change would affect “cataloging/metadata and other operations and processes.”

The proposal should be rejected for three reasons.

1. The premise is wrong

GPO’s premise is flat out wrong. The information recorded in PURLs and “historic URLs” is neither identical nor redundant. PURLs change over time and “historic URLs” stay the same. The two fields record different information even when they are the same: PURLs record the current location of a resource and “historic URLs” record the original location of the resource. PURLs exist because URLs change.

(If GPO could guarantee that the public URL of govinfo items will never change, then it could just as easily advocate not including PURLs in CGP records. But that would, obviously, be crazy. PURLs are needed to account for the problem of “link rot.” Link rot is a well documented problem including for government information. GPO’s own digital repository has had already had three base addresses over the years requiring PURL redirects: access.gpo.gov, gpo.gov/fdsys, and govinfo.gov.)

Because PURLs are necessary, the issue GPO should be addressing is whether or not the original URL is a valuable piece of metadata — even after it “rots” (changes). And the question GPO should be asking is not how omitting that information will affect “cataloging/metadata and other operations and processes,” but how it will affect users and long-term access to digital resources.

2. The historic URL is valuable to users

The original URL of govinfo resources is a valuable piece of metadata for users. They can use it to locate a copy of a document in other archives (e.g., the Internet Archive, the End of Term Archive). They can use it to compare copies when more than one institution has archived the same content (perhaps at different times, reflecting changes over time).

The historic URL is essential to retrieving copies in other archives and this would be vital if technical, administrative, legal, or budgetary changes interfered with GPO keeping PURLs up to date, or keeping its PURL server online, or keeping its online copy available. It would also be vital if government information is intentionally withdrawn or privatized. There is no good reason to deny such essential information to users.

Some resources in govinfo (such as the Federal Register) are available in other versions from other government sources. The CGP records should record the historic URL so that users can understand which versions are described and to which version the PURL points.

The govinfo website provides users with the URLs of resources, but does not provide PURLs. Most users of resources in the TDR will have those public govinfo URLs and not PURLs. Users who cite those resources and provide links to them will likely use those URLs, not PURLs. Having those URLs in CGP would help users find the PURL when those URLs change (as they inevitably will).

3. The proposal ignores the evolution of govinfo

The proposed policy change is notable in what it leaves out: the future. At its inception, the policy would, apparently, affect only content in GPO’s Trusted Digital Repository (TDR).[1] The proposal should explicitly define the scope of the policy.

Currently, as far as we can tell[2], the TDR contains two kinds of resources with CGP records: Born-digital items that are first published by GPO in either fdsys or govinfo (example CGP record: America’s water infrastructure needs and challenges); and digitized paper items that GPO has ingested into fdsys or govinfo (example CGP record: Internal revenue cumulative bulletin.).

But CGP includes historic URLs for other kinds of resources. For example:

This prompts several questions that the policy proposal does not address: Will GPO’s TDR ever contain other resources? Will it evolve and expand over time? Will the policy affect other kinds of content if they become “resources in govinfo”?

This is important because the historic URLs in CGP records for resources that, apparently, are not affected by the new policy today, could be affected if they were ingested into the TDR (or if the policy is broader than we think it is, or if GPO interprets the policy differently in the future, etc.).

The “historic URLs” for that other content is even more important in some cases than it is for the content currently in the TDR.

The source URL of a harvested item establishes the provenance (origin) of the item archived. The standard for Trusted Digital Repositories requires TDRs to preserve information on the provenance of resources they archive (TDR 4.2.6.3). If GPO ever ingests harvested content into its TDR, it will need the historic URL. Having it in CGP records could provide a useful link between CGP and the GPO’s repository.

Digitized paper resources present a special problem for users. More than one digitization project may digitize the same paper document or different editions or versions of the same title. Differences can occur because of the use of different source copies and because of different standards and levels of accuracy of different digitization processes. This is further complicated in CGP by the fact that some CGP records point to non-government websites (example: Joan of Arc saved France), some point to the TDR (example: An account of the receipts and expenditures of the United States), and some point to harvested content (example: Pots and pans for your kitchen). There are probably other permutations and combinations that we have not discovered. The “historic” URL (e.g., its original location) would help the user identify the source of the digitization and would enable the user to obtain a copy from that source or from an archive of that source.

Conclusion

The “historic URLs” in CGP provide information to users that PURLs do not. That information is useful to users because it will help them identify, understand, and locate copies of resources. “Historic URLs” may seem unnecessary to GPO today, but they will increase in value to users over time. Making a decision for “resources in govinfo” today fails to take into account what resources may be in GPO’s TDR in the future (including harvested content and digitizations). The proposal to drop historic URLs is short-sighted. Dropping historic URLs today would be a mistake that users would resent in the future.

GPO should clarify the scope of the policy and how it would be applied in the future and evaluate its effects on users and long-term access.

Authors:
James A. Jacobs, University of California San Diego
James R. Jacobs, Stanford University

Endnotes

  1. The announcement mentions getting TDR certification and refers to “resources in govinfo.” GPO does not make it clear, however, if “govinfo” is a reference to its digital repository or the website www.govinfo.gov. Without a clarification, the scope of the policy proposal is unclear and ambiguous. The website runs using the content management system drupal. It appears that everything in the TDR is available through the website, but not everything listed on the website is available in the TDR (e.g., see: the browse page that points to GPO partner resources). For the most part, though, the website appears to be the public interface to the TDR. As far as we know, GPO has not said if that will always be the case. Presumably, the website could be used to point at content in more than one repository (permanent.gpo.gov, for example), or GPO might want to ingest into the TDR the content it has harvested at wayback.archive-it, or GPO might replace drupal with a different CMS. Any of these would result in a different scope of “resources in govinfo.” ↵
  2. Since GPO has not yet shared its TDR certification report publicly, our evaluation is based on what we can infer from using the govinfo website. ↵

The federal government’s responsibilities to provide online content in non-English languages

February 14, 2019 / Leave a comment

The Sunlight Foundation’s Web Integrity Project (WIP) has released a new report with an informal review of how federal agencies meet the needs of website users who lack English fluency.

The report reviews and explains the different regulations imposed on federal agencies to provide online content in non-English languages for individuals with limited English proficiency (LEP).

WIP found that there is little agreement about the duties of agencies. The approaches of various agencies vary widely — from full translation and taglines, to ad hoc translation using Google Translate.

Examples:

  • IRS.gov has largely complete copies of the English-language site in other languages with links to the non-English versions of the site available through a dropdown in the header of webpages throughout the website domain.

  • The websites for the Whitehouse (whitehouse.gov) and the National Park Service (nps.gov) contain no resources or links relating to content or information in non-English languages.

Terabytes of Enron data have quietly gone missing from the Department of Energy

February 1, 2019 / Leave a comment

This is yet another disturbing example of data loss documented by our friends at MuckRock. Evidently, a large amount of data from the Federal Energy Regulatory Commission (FERC) having to do with the infamous Enron Corporation has gone missing and even FERC staff do not know where it went.

How many examples will we need to post before libraries and archives get with the program and try and figure out ways to collect, archive, preserve and give access to born-digital information posted on .gov sites? And how does this particular example of data loss NOT happen again in the brave new world of open government data (aka H.R. 4174 the Foundations for Evidence-Based Policymaking Act of 2018 which was recently signed into law and described by Alex Howard)? If you’re as concerned as I am, you’ll contact FERC and request a copy of their data for your library.

Government investigations into California’s electricity shortage, ultimately determined to be caused by intentional market manipulations and capped retail electricity prices by the now infamous Enron Corporation, resulted in terabytes of information being collected by the Federal Energy Regulatory Commission. This included several extremely large databases, some of which had nearly 200 million rows of data, including Enron’s bidding and price processes, their trading and risk management systems, emails, audio recordings, and nearly 100,000 additional documents. That information has quietly disappeared, and not even its custodians seem to know why…

…While terabytes of information has disappeared, up to 4,516 documents remain available through a pair of predefined searches of FERC’s eLibrary. While FERC claims that they, not Lockheed Martin or CACI, do offer a trio of Enron datasets on CD, FERC has not responded to repeated requests for these datasets sent over the past two months.

via Terabytes of Enron data have quietly gone missing from the Department of Energy • MuckRock.

Post navigation

← Older posts

Blogroll

Archives