Treasury.IO: A daily data feed for the U.S. Treasury

March 18, 2017 by · Leave a Comment
Filed under: Doc of the day, post 

Ever wonder about the federal government’s checkbook? Well now you can take a peak inside for each day using Treasury.io. “Every day at 4pm, the United States Treasury publishes data tables summarizing the cash spending, deposits, and borrowing of the federal government.” Those data tables “catalog all the money taken in that day from taxes, the programs, and how much debt the government took out.”

One hitch: The Treasury’s data tables are (subjectively) ugly and (objectively) spreadsheet-unfriendly. So Treasury.io — an open-source civic project complete with a github repository! — continuously converts the files into good ol’ tabular data. You can download individual tables as CSVs, get the whole dataset as a big SQLite database, or query the API. There’s also a data dictionary and a Twitter bot.

HT to Jeremy Singer-Vine and his amazing Data Is Plural weekly newsletter of useful/curious datasets. If you haven’t subscribed, then you ought to go over there right now and do so post haste!

Every day at 4pm, the United States Treasury publishes data tables summarizing the cash spending, deposits, and borrowing of the Federal government. These files catalog all the money taken in that day from taxes, the programs, and how much debt the government took out to make it happen. It comes from a section of the U.S. Treasury called the Bureau of the Fiscal Service.

At a time of record fiscal deficits and continual debates over spending, taxation, and the debt, this daily accounting of our government’s main checking account is an essential data point that the public should have ready access to.

via Treasury IO: A daily data feed for the U.S. Treasury.

GPO releases digital Congressional Record for the 1970s

March 7, 2017 by · Leave a Comment
Filed under: Doc of the day, post 

The Government Publishing Office just announced that they’ve released another decade of historic bound Congressional Record, this time covering 1971 – 1980.

The U.S. Government Publishing Office (GPO) partners with the Library of Congress to release the digital version of the bound Congressional Record from 1971-1980 on GPO’s govinfo system.

This release covers debates and proceedings of the 92nd through the 96th Congresses. This era of Congress covers historical topics such as:

  • The Administrations of Presidents Richard Nixon, Gerald Ford, and Jimmy Carter
  • Passage/ratification of the 26th Amendment (allowing 18-year-olds to vote)
  • Watergate
  • The end of the Vietnam War
  • The Bicentennial
  • Civil Service Reform Act of 1978
  • The Iran Hostage Crisis
  • OPEC and the Oil Crises of the 1970s
  • Title IX Amendment of the Higher Education Act

via GPO Issues Digital Release of Historical Congressional Record for the 1970s.

Found and liberated: Deleted APHIS Annual Reports From Animal-Experimentation Facilities

February 8, 2017 by · Leave a Comment
Filed under: Doc of the day, post 

Last week, we posted that the USDA’s Animal and Plant Health Inspection Service had announced that it was removing from its Website “inspection reports, regulatory correspondence, research facility annual reports, and enforcement records that have not received final adjudication.”

Russ Kick of the MemoryHole blog has now published thousands of these reports, which he had downloaded last summer and deposited in the Internet Archive. These include Animal and Plant Health Inspection Service (APHIS) reports concerning animal welfare at zoos, circuses, aquariums, puppy mills, etc., as well as their archive of annual reports filed by facilities that experiment on animals.

There are still extant deleted APHIS files out there – including inspection reports and enforcement records. If found, please send them to Russ. More of the story is over at Motherboard this page, plus 2,600 individual annual report PDFs from a wide time period here and here.)

via Deleted APHIS Annual Reports From Animal-Experimentation Facilities — The Memory Hole 2.

Found and liberated: DHS Daily Open Source Infrastructure Report

February 8, 2017 by · Leave a Comment
Filed under: Doc of the day, post 

On January 25, we blogged that the Department of Homeland Security (DHS) Daily Open Source Infrastructure Report has been summarily discontinued. This report provided a daily curated selection of articles/links/summaries to open source articles about various areas of U.S. critical infrastructure.

Effective January 18, 2017, the Office of Infrastructure Protection (IP) is discontinuing the DHS Daily Open Source Infrastructure Report. The discontinuation of this report is part of broader efforts to more efficiently focus resources towards the highest priority needs of the critical infrastructure security and resilience community. IP is committed to working closely with our public and private sector partners in identifying innovative approaches to exchanging information in a timely and actionable manner to further support risk mitigation activities.

One reader, Dr. Megan Squire, a CS professor at Elon University, took it upon herself to harvest the reports (2,151 PDF files!) and deposit them in the Internet Archive. These are now part of the Internet Archive’s growing Government Documents collection. Thanks Megan for this work! I hope our readers will take up the “rogue internet archivist” mantle and collect and preserve digital government information in all its guises and at all levels!!

On January 18, 2017 the US Department of Homeland Security discontinued its Daily Open Source Infrastructure Report service which it had run since October 2006. To enable researchers to study the content of these reports, I collected as many as I could find (2,151 PDF files) and released them to the Internet Archive. You can find them here: DHS Daily Open Source Infrastructure Reports 2006-2017

The PDF files came from the following URLs:

https://www.dhs.gov/sites/default/files/publications/
https://www.dhs.gov/sites/default/files/publications/nppd/ip/daily-report/
https://www.dhs.gov/xlibrary/assets/

And when these yielded 404 errors (which they did for most pre-2013 files) I used the Internet Archive itself, with the following URL base:

http://web.archive.org/web/20061101153326/https://www.dhs.gov/xlibrary/assets/[filename]

via Found and liberated 2,151 missing DHS files – Megan’s Blog.

Document of the day: NSA’s guide to the internet

February 1, 2017 by · Leave a Comment
Filed under: Doc of the day, post 

This just came through my twitter feed from @MuckRock. Through a FOIA request which shook it loose from the notoriously difficult NSA, we now have access to NSA’s 2007 Untangling the Web: a guide to Internet research. It kind of reads like a Terry Pratchett novel if Terry was having a psychotic/psychedelic episode. As MuckRock notes, “you don’t have to go very far before this takes a hard turn into ‘Dungeons and Dragons campaign/Classics major’s undergraduate thesis’ territory.” Read on, you’ll thank me later!

And if you’re interested, I collected and cataloged a version for our library. The original NSA link to the document no longer resolves (and it was put up just last year!!), but there’s an archived copy in the WayBack Machine.

The NSA has a well-earned reputation for being one of the tougher agencies to get records out of, making those rare FOIA wins all the sweeter. In the case of Untangling the Web, the agency’s 2007 guide to internet research, the fact that the records in question just so happen to be absolutely insane are just icing on the cake – or as the guide would put it, “the nectar on the ambrosia.”

via The NSA’s guide to the internet is the weirdest thing you’ll read today.

Next Page »

  • Our mission

    Free Government Information (FGI) is a place for initiating dialogue and building consensus among the various players (libraries, government agencies, non-profit organizations, researchers, journalists, etc.) who have a stake in the preservation of and perpetual free access to government information. FGI promotes free government information through collaboration, education, advocacy and research.
  • Archives