Home » Posts tagged 'EOT'

Tag Archives: EOT

Our mission

Free Government Information (FGI) is a place for initiating dialogue and building consensus among the various players (libraries, government agencies, non-profit organizations, researchers, journalists, etc.) who have a stake in the preservation of and perpetual free access to government information. FGI promotes free government information through collaboration, education, advocacy and research.

End of Term crawl 2024 is now underway!

Well it’s that time again. The 2024 End of Term web crawl of the federal .gov/.mil web space (and other domains 🙂 ) has begun. We have just posted our first public announcement on the Internet Archive blog.

As we have done since 2008 (NARA did the first comprehensive crawl in 2004), a group of volunteers from the Internet Archive, GPO, Library of Congress, NARA, University of North Texas, and Stanford will be doing a “comprehensive” web harvest of the Federal government’s web space. For more information and background on the project, see our home page at https://eotarchive.org/. These archives can be searched full-text via the Internet Archive’s collections search (https://web.archive.org/) and also downloaded as bulk data for machine-assisted analysis from the project site.

But MOST IMPORTANTLY, we need YOUR help! We are currently accepting nominations for websites to be included in the 2024 End of Term Web Archive. Submit a url nomination by going to our nomination tool (hosted by University of North Texas!) and clicking the big yellow “add a url” button in the top right:

https://digital2.library.unt.edu/nomination/eth2024/

We encourage you to nominate any and all U.S. federal government websites that you want to make sure get captured. We’re also interested in any and all urls of federal sites that are NOT hosted on .gov/.mil (there are lots of federal government sites hosted on .edu, .org, and even .com! That includes social media but also research labs and other private/public partnerships). We already have a solid list of top level domains (eg epa.gov, congress.gov, defense.mil etc). Nominating urls deep within .gov/.mil websites helps to make our web crawls as thorough and complete as possible. Prizes will be awarded for most url nominations by individuals and institutions!

So get to it! Help us do the most complete crawl we can and also assure that the sites/publications/videos/data etc that are most important to YOU make it into the archive!!

The EPA’s Website after a year of climate change censorship

Here’s a good article from Time Magazine“Here’s What the EPA’s Website Looks Like After a Year of Climate Change Censorship” — which accurately reports how the Trump Administration and EPA Administrator Scott Pruitt have changed, skewed or deleted government information from the EPA Website for crass political purposes. For more in-depth analysis of the issue of information scrubbing from federal websites, one should look to the work of the Environmental Data and Governance Initiative (EDGI) and especially their reports: “Changing the Digital Climate” and “The EPA Under Siege”.

According to former government officials and EPA staffers, the level of scrutiny is without precedent. In the hands of an administration that has eschewed facts for their alternative cousins, the agency’s site is increasingly unmoored from its scientific core.

“In my experience, new administrations might come in and change the appearance of an agency website or the way they present information, but this is an unprecedented attempt to delete or bury credible scientific information they find politically inconvenient,” Heather Zichal, a senior fellow at the Atlantic Council’s Global Energy Center, and previously President Barack Obama’s top White House adviser on energy and climate change, tells TIME.

The EPA’s site is now riddled with missing links, redirecting pages and buried information. Over the past year, terms like “fossil fuels”, “greenhouse gases” and “global warming” have been excised. Even the term “science” is no longer safe.

Christine Todd Whitman, the EPA Administrator under George W. Bush, says the overhaul is “to such an extreme degree that [it] undermines the credibility of the site”…

Of the more than 25,000 web pages tracked by the Environmental Data and Governance Initiative (EDGI) since Trump’s election, they say the EPA’s have been hit hardest. One section, which provided local communities with resources for combating climate change, disappeared for months only to resurface heavily redacted, including just 175 of its 380 pages.

via The EPA’s Website After a Year of Climate Change Censorship | Time.

2016 End of Term Web Archive is now available

The 2016 end of term .gov/.mil web crawl is now available! We collected approximately 300TB of government websites which includes over “70 million html pages, over 40 million PDFs and, towards the other end of the spectrum and for semantic web aficionados, 8 files of the text/turtle mime type” as well as @100TB of public data via .gov FTP file servers! Thanks to everyone who participated on the project and the thousands(!) of seed nominators, both individuals and those that came in via DataRefuge and EDGI tools and public events.

The End of Term Web Archive contains federal government websites (.gov, .mil, etc) in the Legislative, Executive, or Judicial branches of the government. Websites that were at risk of changing (i.e., whitehouse.gov) or disappearing altogether during government transitions were captured. Local government websites, or any other site not part of the federal government domain were out of scope.

Lunchtime listen: “Storing Data Together” by Matt Zumwalt at Code4Lib2017

Drop everything and watch this presentation from the 2017 Code4Lib conference that took place in Los Angeles March 6-9, 2017. Heck, watch the entire proceedings because there is a bunch of interesting and thoughtful stuff going on in the world of libraries and technology! But in particular, check out Matt Zumwalt’s presentation “How the distributed web could bring a new Golden Age for Libraries” — after submitting his talk, he changed the new title to “Storing data together: the movement to decentralize data and how libraries can lead it” because of the DataRefuge movement.

Zumwalt (aka @FLyingZumwalt on twitter), works at Protocol Labs, one of the primary developers of IPFS, the Interplanetary File System (IPFS) — grok their tagline “HTTP is obsolete. It’s time for the distributed, permanent web!” He has spent much of his spare time over the last 9 months working with groups like EDGI, DataRefuge, and the Internet Archive to help preserve government datasets.

Here’s what Matt said in a nutshell: The Web is precarious. But using peer-to-peer distributed network architecture, we can “store data together”, we can collaboratively preserve and serve out government data. This resonates with me as an FDLP librarian. What if a network of FDLP libraries actually took this on? This isn’t some far-fetched, scifi idea. The technologies and infrastructures are already there. Over the last 9 months, researchers, faculty and public citizens around the country have already gotten on board with this idea. Libraries just have to get together and agree that it’s a good thing to collect/download, store, describe and serve out government information. Together we can do this!

Matt’s talk starts at 3:07:41 of the YouTube video below. Please watch it, let his ideas sink in, share it, start talking about it with your colleagues and administrators in your library, and get moving. Government information could be the great test case for the distributed web and a new Golden Age for Libraries!

This presentation will show how the worldwide surge of work on distributed technologies like the InterPlanetary File System (IPFS) opens the door to a flourishing of community-oriented librarianship in the digital age. The centralized internet, and the rise of cloud services, has forced libraries to act as information silos that compete with other silos to be the place where content and metadata get stored. We will look at how decentralized technologies allow libraries to break this pattern and resume their missions of providing discovery, access and preservation services on top of content that exists in multiple places.


FGI accidental docs librarian webinar: “Saving govt data: a conversation with the future”

Please tune in next Wednesday, March 29, 2017 from 9am – 10am Pacific / 12:00 – 1:00pm Eastern for the next Help! I’m an Accidental Government Information Librarian Webinar “Saving government data: A conversation with the future.” You’ll need to in order to get the link to the WebEx live session. “See” you there!

Help! I’m an Accidental Government Information Librarian presents … Saving government data: A conversation with the future, on Wednesday, March 29, 2017 from 12:00 – 1:00 p.m. (Eastern).

In recent months, the DataRefuge project has collaborated with hundreds of volunteers around the United States to collect, describe, and store federal data that support climate and environmental research and advocacy. This project, and others like it, works in conjunction with the End of Term Web Archive to capture and make available federal web content during administrative transitions.

Our discussion will explore the fragility of digital information, and expand on ideas about what data is. We’ll talk about current projects and efforts, and explore the future of this work. Finally, we’ll address the concept of sustainability, and propose a paradigm of empowered experimentation that aligns with our values and roles within libraries.

We will meet together for Session #69, online on Wednesday, March 29, 2017 from 12:00 – 1:00 p.m. (Eastern). Please RSVP for the session using this link:  http://bit.ly/GRS-Session69

We will use WebEx for the live session. Information on testing and accessing the session will be made available when you register.

The session will be recorded and available after the live session, linked from the NCLA GRS web page (http://www.nclaonline.org/government-resources).

Presenters:
Laurie Allen is the Assistant Director for Digital Scholarship in the Penn Libraries, where she leads a group working to expand the capacity of researchers at Penn to create and share scholarship in new forms. The group engages in digital project development, data management and curation, mapping, experimentations with emerging research methods, and open access publishing. In late 2016, Allen was part of the group that started Data Refuge, and has been involved in bringing together a group of collaborators to form a network of libraries, open data activists and open government efforts.

James A. Jacobs is Data Services Librarian Emeritus, University of California San Diego. He has more than 25 years experience working with digital information, digital services, and digital library collections. He is a technical consultant and advisor to the Center for Research Libraries in the auditing and certification of digital repositories using the Trusted Repository Audit Checklist (TRAC) and related CRL criteria. He served as Data Services Librarian at the University of California San Diego and co-taught the ICPSR summer workshop, “Providing Social Science Data Services: Strategies for Design and Operation”. He is a co-founder of Free Government Information.

James R. Jacobs is the US Government Information Librarian at Stanford University Libraries where he works on both collection development as well as digital projects like LOCKSS-USDOCS. He is a member of ALA’s Government Documents Roundtable (GODORT) and served a 3-year term on Depository Library Council to the Public Printer, including serving as DLC Chair. He is a co-founder of Free Government Information (freegovinfo.info) and Radical Reference (radicalreference.info) and is on the board of Question Copyright, a 501(c)(3) non-profit organization that promotes a better public understanding of the  effects of copyright, and encourages the development of alternatives to information monopolies.

Shari Laster is the Government Information Librarian and Data Services Librarian at the University of California, Santa Barbara. She currently serves as Assistant Chair/Chair-Elect for the Government Documents Round Table of the American Library Association, and is a past chair of the Depository Library Council, the advisory body for the Federal Depository Library Program.

via Help! I’m an Accidental Government Information Librarian Webinars | North Carolina Library Association.

Archives