Home » Posts tagged 'Web archiving'
Tag Archives: Web archiving
Good Forbes piece on the End of Term crawl
Check out this recent piece in Forbes about the End of Term project (eotarchive.org). And if you’re so moved to help out, you can nominate any federal government url through our handy nomination tool. There’s still time to help us save democracy’s information!
Meet The Citizens Racing To Save Government Websites From Vanishing. Leslie Katz, Forbes, Oct 23, 2024
…With the Nov. 5 election just a week away, they’re harvesting vast amounts of government data before the White House welcomes new residents or former ones in January. The information will live on in the End of Term Web Archive, a giant repository of federal government websites preserved for the historical record as one administrative term ends and a new one begins. Librarians, archivists and technologists across the country join forces every four years to donate time, effort and resources to what they dub the end-of-term crawl, with the resulting datasets available to the public for free…
“Citizens have a right to access information about what their government is doing in their name,” says James Jacobs, a government information librarian at Stanford University, an End of Term Web Archive project partner. “That’s why libraries have long collected these materials to make sure they are organized, preserved and easily accessible for the long term.”
EDGI Releases Dataset of Federal Environmental Website Changes Under Trump
Thanks to the Environmental Data and Governance Initiative (EDGI) for releasing the Federal Environmental Web Tracker. This tool is a public dataset of searchable records of approximately 1,500 significant changes to federal agency environmental webpages under the Trump administration, these changes were almost always precursors or responses to policy changes. These changes came from a “list of 25,000 federal Web pages related to climate, energy, and the environment, including pages for 20 federal agencies such as EPA, NOAA, and NASA.” Here’s the Tracker’s explanatory page for more context and background.
EDGI continues to do important work in tracking the federal .gov Web domain. EDGI’s work goes hand in hand with the work of the End of Term Web Archive which has harvested the .gov/.mil Web space every 4 years since 2008 and is now deep into its 2020 harvest. And we’re still accepting nominations, so go to the End of Term Nomination Tool hosted by the University of North Texas (UNT) library. Help us collect a snapshot of the federal Web domain!
Today, the Environmental Data & Governance Initiative (EDGI) publishes searchable records of approximately 1,500 changes to federal agency environmental webpages under the Trump administration. For four years, EDGI’s website monitoring team has identified and catalogued significant changes to federal websites using their open source monitoring software. EDGI’s Federal Environmental Web Tracker makes records of significant changes publicly available.
The information that’s available on federal websites can have important policy implications. As EDGI has often reported over the past four years, changes to the information that’s available on federal websites are almost always precursors or responses to policy changes. Federal websites provide information that the public is likely to access before commenting on a proposed rule to learn about current regulatory efforts, the science underlying a new policy decision, or likely impacts of a proposed rule. The information found (or not found) on a federal website can impact public participation in regulatory processes.
In the weeks after Trump’s election in November 2016, newly-formed EDGI compiled a list of 25,000 federal web pages related to climate, energy, and the environment, including pages for 20 federal agencies such as EPA, NOAA, and NASA. First using proprietary software and then building and using novel open source software, EDGI has compared versions of these web pages weekly since January 2017. This new dataset represents the documented changes that EDGI’s website monitoring team flagged as significant in some way over the past four years.
EDGI’s Federal Environmental Web Tracker gives journalists, academic researchers, and the public data that can be used to provide insight, documentation, and analysis of the information policies and priorities of the Trump administration.
The Federal Environmental Web Tracker will be updated quarterly as EDGI continues to monitor federal environmental websites.
HHS Plans to Delete 20 Years of Critical Medical Guidelines on monday July 16!
According to the DailyBeast, the National Guideline Clearinghouse (NGC), a critical database of medical guidelines, is set to go dark on monday July 16 because “federal funding through Agency for Healthcare Research and Quality (AHRQ) will no longer be available to support the NGC as of that date.” For any questions, please contact Mary.Nix@ahrq.hhs.gov.
Of course, the Internet Archive has archived this site many times since 1998(!), but sadly, the Web archive hasn’t collected the most critical information because it’s hidden behind a database query.
This is completely unacceptable!
The Trump Administration is planning to eliminate a vast trove of medical guidelines that for nearly 20 years has been a critical resource for doctors, researchers and others in the medical community. Maintained by the Agency for Healthcare Research and Quality [AHRQ], part of the Department of Health and Human Services, the database is known as the National Guideline Clearinghouse [NGC], and it’s scheduled to “go dark,” in the words of an official there, on July 16.
via HHS Plans to Delete 20 Years of Critical Medical Guidelines Next Week.
The EPA’s Website after a year of climate change censorship
Here’s a good article from Time Magazine — “Here’s What the EPA’s Website Looks Like After a Year of Climate Change Censorship” — which accurately reports how the Trump Administration and EPA Administrator Scott Pruitt have changed, skewed or deleted government information from the EPA Website for crass political purposes. For more in-depth analysis of the issue of information scrubbing from federal websites, one should look to the work of the Environmental Data and Governance Initiative (EDGI) and especially their reports: “Changing the Digital Climate” and “The EPA Under Siege”.
According to former government officials and EPA staffers, the level of scrutiny is without precedent. In the hands of an administration that has eschewed facts for their alternative cousins, the agency’s site is increasingly unmoored from its scientific core.
“In my experience, new administrations might come in and change the appearance of an agency website or the way they present information, but this is an unprecedented attempt to delete or bury credible scientific information they find politically inconvenient,” Heather Zichal, a senior fellow at the Atlantic Council’s Global Energy Center, and previously President Barack Obama’s top White House adviser on energy and climate change, tells TIME.
The EPA’s site is now riddled with missing links, redirecting pages and buried information. Over the past year, terms like “fossil fuels”, “greenhouse gases” and “global warming” have been excised. Even the term “science” is no longer safe.
Christine Todd Whitman, the EPA Administrator under George W. Bush, says the overhaul is “to such an extreme degree that [it] undermines the credibility of the site”…
Of the more than 25,000 web pages tracked by the Environmental Data and Governance Initiative (EDGI) since Trump’s election, they say the EPA’s have been hit hardest. One section, which provided local communities with resources for combating climate change, disappeared for months only to resurface heavily redacted, including just 175 of its 380 pages.
via The EPA’s Website After a Year of Climate Change Censorship | Time.
These Advocates Want to Make Sure Our Data Doesn’t Disappear
Here’s another story about data rescue and the preservation of government information, this time from PC Magazine UK. Though the last data refuge event was in Denton, TX in May and the 2016 End of Term crawl has finished its collection work and will soon have its 200TB of data publicly accessible, there still remains much interest — and not a little bit of worry — about the collection and preservation of govt information and data. And with stories continuing to come out — eg this one from the Guardian entitled “Another US agency deletes references to climate change on government website” — about the US government agencies scrubbing or significantly altering their Websites, this issue will not be going away any time soon.
“Somewhere around 20 percent of government info is web-accessible,” said Jim (sic.) Jacobs, the Federal Government Information Librarian at Stanford University Library. “That’s a fairly large chunk of stuff that’s not available. Though agencies have their own wikis and content management systems, the only time you find out about some of it is if someone FOIAs it.”
To be sure, a great deal of information was indeed captured and now resides on non-government servers. Between Data Refuge events and projects such as the 2016 End-of-Term Crawl, over 200TB of government websites and data were archived. But rescue organizers began to realize that piecemeal efforts to make complete copies of terabytes of government agency science data could not realistically be sustained over the long term—it would be like bailing out the Titanic with a thimble.
So although Data Rescue Denton ended up being one of the final organized events of its kind, the collective effort has spurred a wider community to work in concert toward making more government data discoverable, understandable, and usable, Jacobs wrote in a blog post.
via Feature: These Advocates Want to Make Sure Our Data Doesn’t Disappear.
Latest Comments