This is yet another disturbing example of data loss documented by our friends at MuckRock. Evidently, a large amount of data from the Federal Energy Regulatory Commission (FERC) having to do with the infamous Enron Corporation has gone missing and even FERC staff do not know where it went.
How many examples will we need to post before libraries and archives get with the program and try and figure out ways to collect, archive, preserve and give access to born-digital information posted on .gov sites? And how does this particular example of data loss NOT happen again in the brave new world of open government data (aka H.R. 4174 the Foundations for Evidence-Based Policymaking Act of 2018 which was recently signed into law and described by Alex Howard)? If you’re as concerned as I am, you’ll contact FERC and request a copy of their data for your library.
Government investigations into California’s electricity shortage, ultimately determined to be caused by intentional market manipulations and capped retail electricity prices by the now infamous Enron Corporation, resulted in terabytes of information being collected by the Federal Energy Regulatory Commission. This included several extremely large databases, some of which had nearly 200 million rows of data, including Enron’s bidding and price processes, their trading and risk management systems, emails, audio recordings, and nearly 100,000 additional documents. That information has quietly disappeared, and not even its custodians seem to know why…
…While terabytes of information has disappeared, up to 4,516 documents remain available through a pair of predefined searches of FERC’s eLibrary. While FERC claims that they, not Lockheed Martin or CACI, do offer a trio of Enron datasets on CD, FERC has not responded to repeated requests for these datasets sent over the past two months.
On Jan 29, The Government Accountability Office (GAO) announced the launch of a new Science, Technology Assessment and Analytics (STAA) team to provide Congress with "thorough and balanced analysis of technological and scientific developments that affect our society, environment, and economy." GAO had announced its intention to set up this new STAA team back in December of 2018.
This new team will, apparently, serve some of the same functions that The Office of Technology Assessment (OTA) once served.
The Office of Technology Assessment was created by Congress in 1972 (2 USC 472) "within and responsible to the legislative branch." Its stated purpose was "to provide early indications of the probable beneficial and adverse impacts of the applications of technology and to develop other coordinate information which may assist the Congress." The idea was that Congress did not want to rely on think tanks or the Executive Branch agencies for an understanding of complex scientific issues. But, in 1995, Congress simply stopped funding OTA. (For background see, Science and Congress, by Adam Keiper.) OTA documents are archived by the University of North Texas Libraries at the CyberCemetery.
Since at least 2009, there have been attempts to fund OTA again. There has been some recent speculation that Congress might be more amenable to letting GAO take on the role that OTA once had. (GAO is "an independent, nonpartisan agency that works for Congress.") Indeed, GAO has been offering technology assessments since at least 2002. In GAO’s announcement, they say that they routinely provide analysis of how federal agencies manage and employ science and technology, such as regenerative medicine, 5G wireless communication, and quantum computing.
GAO says that the new STAA team will expand its support to Congress by:
- Conducting technology assessments and providing technical services
- Auditing science and technology programs and initiatives to assist in oversight of federal investments in research, development, and advanced manufacturing
- Compiling and utilizing best practices in engineering sciences, including cost, schedule, and technology readiness assessments
- Establishing an audit innovation lab to explore, pilot, and deploy new advanced analytic capabilities, conduct research in information assurance, and explore emerging technologies that will impact future audit practices.
- GAO expands and elevates tech assessment, by Adam Mazmanian FCW (Jan 29, 2019)
At launch, GAO combined existing in-house technology staffers and experts for STAA. But Persons said GAO will make outside hires, taking advantage of direct hire authority for technical positions and the Intergovernmental Personnel Authority, which allows government agencies to offer term appointments to academics and researchers at universities and nonprofits.
Russ Kick of AltGov2 is again on the FOIA case. This time he’s analyzed the Department of Interior’s proposed changes to their FOIA regulations (and helpfully cobbled together the current regulations with DoI’s proposed changes). Comments on the proposed changes can be submitted electronically by JANUARY 28, 2019. Here’s the current regulations and the proposed rules changes posted to regulations.gov.
Here’s my own take on this. Department of Interior’s reason for updating its FOIA regulations is that they’ve had “Exponential increases in requests and litigation.” From Fiscal Year (FY) 2016 to FY 2018, incoming FOIA requests to the Department increased 30 percent (from 6,428 to over 8,350). So it would make sense that they’d want to update their regulations to deal with the exponential increase in requests. But instead of requesting more staff to deal with the increase in requests or look for ways to proactively release more records, their proposed changes:
- look to expand the definition and reasoning of “burdensome” requests that can be denied. “The bureau will not honor a request that requires an unreasonably burdensome search or requires the bureau to locate, review, redact, or arrange for inspection of a vast quantity of material.”
- place monthly limits on the number of requests per month from frequent requesters.
- Make it more confusing and difficult to submit requests (eg it’s unclear whether requests will be accepted via email in the new regs).
- No longer refer or forward mistakenly directed requests. “A request to a particular bureau or a particular bureau component (for example, a request addressed to a regional or field office) will be presumed to seek only records from that particular bureau or particular component and will not be forwarded to another bureau or component.”
- Change “time limits” to “time frames,” making the time to fulfill requests more squishy and undefined.
There’s probably more buried in this request. Check out Russ’ analysis at MuckRock. And PLEASE send comments to Department of Interior about their proposed changes. Comments on the proposed changes can be submitted electronically by JANUARY 28, 2019.
The Department of the Interior wants to drastically change how it deals with Freedom of Information Act requests. To do that, it had to make a proposal, published in the Federal Register, that the public can comment on for 30 days. In theory, it has to consider this input before finalizing any changes to its FOIA regulations.
That proposal was published on December 28th, 2018, which is 1) a Friday 2) in the middle of the week between Christmas and New Year’s Day 3) during a government shutdown. Any one of those is a tried-and-true way to slip something past the public, but all three simultaneously? That is the trifecta of bureaucratic underhandedness.
So, now that the holidays are behind us for another year, let’s take a look at Interior’s FOIA wishlist, which is primarily designed to hobble requesters and solidify the department’s power. (To make it easier to see the many changes Interior wants, I’ve created a redline version of their currently FOIA regulations, with added language in bold and deletions in strikethrough, which you can see here.)
Two things to note this morning as the government shutdown continues.
- A lot of government information is being affected. Examples:
But, a lot of information was in the pipeline and is still being released. Gary Price has one of his excellent roundups of reference resources (Recently Published or Updated Data-Rich Reports Available on the Web) over at InfoDocket. Many of these are government documents including: Federal Justice Statistics, 2015-2016, Union Membership in the United States 2018, Separated Children Placed in Office of Refugee Resettlement Care, and more.
Congratulation to GPO for getting govinfo.gov certified as a Trusted Digital Repository! This is an important step for ensuring long term preservation and access to the contents of the GPO digital repository. The Government Publishing Office (GPO) announced today that it has received this certification.
(For those unfamiliar with certification, check out the “core criteria for digital preservation repositories” that four preservation organizations wrote in 2007. These are a consensus guide auditing and certifying repositories and will give you a general idea of the concepts of certification.)
PTAB used the international standard known as ISO 16363, Audit And Certification Of Trustworthy Digital Repositories. This is the newest and official version of the standard also known as OAIS ("The Reference Model for an Open Archival Information System"). (The standard is available for free from CCSDS and for a fee from the International Organization for Standards ISO.)
PTAB is accredited by the National Accreditation Board for Certification Bodies of India (NABCB) to conduct ISO 16363 audits worldwide utilizing ISO standard 17021 (Requirements for bodies providing audit and certification of management systems) (freely available from CCSDS).
PTAB uses a two stage ISO 16919/17021 audit process.