Cory Doctorow says: "This morning, I found a an enormous, 30Lb box waiting for me at my post-office box. Affixed to it was a sticker warning me that by accepting this box into my possession, I was making myself liable for nearly $11 million in damages. The box was full of paper, and printed on the paper were US laws -- laws that no one is allowed to publish or distribute without permission. Carl Malamud, Boing Boing's favorite rogue archivist, is the guy who sent me this glorious box of weird. I was expecting it, because he asked me in advance if I minded being one of the 25 entities who'd receive this law-bomb on deposit. I was only too glad to accept -- on the condition that Carl write us a guest editorial explaining what this was all about. He was true to his word."

• Liberating America's secret, for-pay laws, By Carl Malamud, boingboing (Mar 19, 2012).

  Boing Boing Official Guest Memorandum of Law
  To: 	The Standards People
  Cc: 	The Rest of Us People
  From: 	Carl Malamud, Public.Resource.Org
  In Re: 	Our Right to Replicate the Law Without a License
  

Readers of Slashdot asked Carl Malamud about his experiences and hopes in his project to prod the U.S. government into scanning archived documents. They asked questions about metadata, digitizing rare books, what he thinks about corporate partnerships in the process to get public data released, other projects like Ancestry.com and PACER, and even "Which government agency is the worst to get information from?"

Malamud's answers are posted at the link below "with a mix of heartening and disheartening information about how the vast project is progressing."

• Carl Malamud Answers: Goading the Government To Make Public Data Public, Slashdot, Your Rights Online section, Posted by timothy on Monday January 23, 2012.

Earlier this month, we posted about the "Open letter and petition to President Obama to create a federal scanning commission and digitize all .gov publications". The petition closed on 1/20 and now David Ferriero, the Archivist of the US at the National Archives, has given the official NARA response. I'd say this is a positive first step, but much discussion is still needed. Please join the conversation over at the NARA Blog. I think documents librarians will be invaluable to this effort going forward!

Digitizing Federal Public Records
By David Ferriero

Thank you for signing a petition asking the Obama Administration to digitize all public records.

The Obama Administration believes increasing access to our collections by digitizing our records is a great idea. Our most recent efforts to do this ourselves as part of our OpenGov initiative, include the Citizen Archivist project, a Wikipedian in Residence, Tag it Tuesdays, and Scanathons. We are also moving forward on implementing the President’s recent Memorandum on Managing Government Records, which focuses on the need to update policies and practices for the digital age.

But all those things aren’t enough. Your petition, and the Yes We Scan effort broadly, calls for a national strategy, and even a Federal Scanning Commission, to figure out what it would take to digitize the holdings of many federal entities, from the Library of Congress to the Government Printing Office to the Smithsonian Institution.

These ideas bring up a host of questions that still need to be answered: What should the National Archives’ priorities be? Do we focus on preserving deteriorating paper records, still bound with red ribbons from two centuries ago? Do we make digital copies of Vietnam Era film footage? Should we focus on preserving those older paper records while citizens volunteer to digitize more recent, and better preserved, records?

The National Archives – which houses the Nation’s permanent records – is looking for your input to help answer these important questions on how we move forward. What are your thoughts on how the National Archives and other agencies should proceed? What questions should we be asking ourselves?

You can add your thoughts over on the National Archives blog, and I’m looking forward to having a longer discussion with the creators and signers of this petition on this important issue in the coming weeks– more details on that will follow.

Thank you again for your interest in this important issue. I’m looking forward to your ideas on how we can proceed with digitizing federal public records.

David Ferriero is the Archivist of the United States

Watching Them Watching: Issa Touts Video Archive of Oversight Hearings, by Nick Judd, TechPresident (January 6 2012).

As of today, the House Committee on Government Oversight under Rep. Darrell Issa has released 1,139 videos of hearings going back to the 103rd Congress of 1993-1994, committee staff announced today.

These videos, dusted off from the House committee's archives, join hundreds more going all the way back to 1987 on House.Resource.org, a repository for archived video and hearing transcripts gleaned from C-SPAN, the House and the Internet Archive as part of a collaboration between Carl Malamud's Public.Resource.org and House Speaker John Boehner. At the start of this Congress, Boehner asked Issa's Oversight committee — which had been recording its own video of hearings, doubling up on video already recorded by the House Broadcasting Studio, since the 2010-2011 session of Congress — to take on archiving and publicising video of committee hearings as a pilot project. The House this year also launched its own streaming of floor proceedings.

John Podesta and Carl Malamud have written an open letter to President Obama (text below) asking for the creation of a Federal Scanning Commission and to greatly increase the pace of digitization of federal resources. They need 25,000 signatures on their petition by January 20, so your help would be greatly appreciated!

While I have some reservations about wholesale digitization that are glossed over in the letter -- I worry for example about the process and how current digitization methods basically destroy documents, how current OCR software is less than perfect, and about only making a digital equivalent to a paper document, NOT the ability to extract and re-use data and statistics etc. (to read more, see "Achieving a collaborative FDLP future") -- as Malamud says:

"Just imagine ... what if we could scan the contents of the FDLP, back issues of the CFR, the briefs before the Supreme Court? We'll never know if we can scan .gov unless we start asking the questions. Please help us get started!"

For that, I'm asking readers to sign the petition and forward to your friends. A national effort is just what is needed. Librarians must advocate for and participate in this process!

December 21, 2011

The President
The White House
1600 Pennsylvania Avenue
Washington, D.C. 20500

Dear Mr. President:

Locked in our federal vaults is a tremendous storehouse of information that if digitized would form a core for our digital public libraries in America with huge benefit for our country: cutting costs in the Federal government, creating jobs throughout America, and revolutionizing how we educate our citizens, how we practice the law, and how we create news, art, and scholarly works.

Imagine if the riches contained in the National Archives, Library of Congress, Smithsonian Institution, Government Printing Office, National Library of Medicine, National Agricultural Library, National Technical Information Service, and scores of other federal organizations were made available, becoming the core of a national effort to make access to knowledge a right for all Americans. The dream is a big one, but if we do not begin the questions of what it would take to get there, we will never start down that road. Today, we don't know what it would take.

We are not necessarily suggesting that the federal government immediately undertake an ambitious effort to scan the holdings of .gov, but if we ever hope to begin even a small piece of making available our past for use by our future, we should at least begin to scope out the size of the problem. We believe it would require a decade-long commitment to digitization to make our nation's cultural, scientific, educational, and historical resources available, but we can't even begin that discussion unless we know how big the problem is. Such an effort is indeed ambitious to contemplate, but we can only ask if we were able to put a man on the moon, why can't we launch the Library of Congress into cyberspace?

Over the last year, a number of efforts have sprung up to create comprehensive digital libraries. The European Union has created Europeana with a goal to “make a large part of the world's cultural heritage available to a large part of the world's population.” In the United States, efforts have included Google Books, the Hathi Trust, the Internet Archive, and the recently announced Digital Public Library of America, a planning initiative with a goal of “creating a large-scale digital public library that will make the cultural and scientific record available to all.”

No matter what the eventual shape of these efforts, we know that the holdings of the U.S. government will play a crucial role, a central part of our public domain. While there have been many well-intentioned efforts to digitize federal holdings, those efforts have been preliminary and tentative. Our national cultural and scientific organizations have never worked together to develop a coherent digitization strategy to scan at scale.

The PCAST report on Designing a Digital Future hits the nail on the head on investing in Networking and Information Technology (NIT), but does not address squarely the question of what it would to take to digitize the holdings of our national institutions. The Presidential Memorandum on Managing Government Records discusses how to make record-keeping move into the modern age in the future, but does not address how to rescue the past and make it useful for Americans.

One way to begin is to convene governmental and non-governmental experts, perhaps in the form of a Presidential Commission, Interagency Task Force, or other mechanism. The “Federal Scanning Commission” would be tasked to answer 6 questions and deliver a report within a year:

1. What are the holdings of our national institutions? How many images, documents, videos, and other objects are there?
2. How long would it take to digitize these materials?
3. How much would it cost given current technology? Is there directed research or are there economies of scale that would bring those costs down?
4. What is the strategy for digital preservation of these materials? How will we avoid digital obsolescence?
5. What is the strategy for identifying restrictions on use of the material? How does one identify and safeguard materials that have copyright restrictions, contain personally identifiable information, or contain classified materials?
6. What are the economic and non-economic benefits of such an effort?
   • What are the cost savings to government?
   • What are the economic benefits? Would this effort enable industries that build on top of scientific and technical information, spur innovation in the legal marketplace, or enable our creative industries to create more effectively?
   • What are the non-economic benefits? Will such an effort lead to better STEM and other educational efforts? Will it promote a more informed citizenry and better access to justice?

To date, thinking about digitization has been piecemeal. Individual agencies have thought about the problem in terms of prototypes and pilots. Only the White House can bring these efforts together under one roof and begin to think in terms of a national digitization strategy for our federal government.

Bringing government agencies together with outside experts to solve a common problem related to our federal holdings has a precedent. When R. D. W. Connor was appointed as the first Archivist of the United States, he faced a herculean task, getting all the agencies of government to come together with a common vision of “safeguarding and preserving the records of our Government.” The idea of safeguarding and preserving the records of government was a new one, and Archivist Connor found “records mingled higgledy-piggledy with empty whiskey bottles.”

Archivist Connor appealed for help to President Roosevelt, asking for his assistance in forging a common vision among the agencies and for their cooperation. President Roosevelt formed a National Archives Council and convened the first meeting in the Cabinet Room, asking Secretary of State Cordell Hull to serve as chairman. By bringing the agencies together in one room, President Roosevelt made the dream of archiving the records of government a shared vision, and then made that vision real.

When Thomas Jefferson donated his books to create the cornerstone of the Library of Congress, his library contained a wealth of useful information, from an extensive collection on the law to books on agriculture, chemistry, surgery, and medicine. With this contribution, Jefferson saw to it that the government of the United States would play a central role in the increase and diffusion of knowledge. It is time now for us to lay the cornerstone for our own era, to anchor our digital age with the vast holdings of our government so that we may promote the useful arts and the progress of science.

We ask your help to achieve this 21st century dream, making the vast resources of our federal government available to all on the global Internet, making access to knowledge a right for all Americans and a defining contribution for our future.

Respectfully yours,

John D. Podesta, Chair
Center for American Progress

Carl Malamud, President
Public.Resource.Org

Happy 150th birthday US Government Printing Office! And to celebrate, here's a blast form the past: a 1979 report from the Public Interest Research Group entitled "The Peoples' Printer: A Report on the Government Printing Office" by Shawn kelly. This little known PIRG report was scanned and put online by Carl Malamud who said in a tweet that "it was handed to me in a brown paper wrapper at an event I was speaking at. Remarkable 1979 independent analysis." Thanks Carl for for scanning and tweeting about this!

The U.S. Government Printing Office (GPO) marks a milestone on March 4th when it celebrates 150 years of producing and delivering Government information for all three branches of the Federal Government and the public. GPO opened its doors on March 4, 1861, the same day President Abraham Lincoln took the oath of office. Throughout its history the agency has used constantly changing technologies to meet the needs of the Congress, Federal agencies, and the public. During GPO's early days, employees relied on ink and paper to set the text for The Emancipation Proclamation. Today, as another President from Illinois leads the Nation, GPO employees are using the latest digital technology to document the actions of our Government while carrying out its founding mission of Keeping America Informed.

While GPO's past has been about printing, its present and future are being defined by digital information technologies. In fact, GPO today is the product of more than a generation of investment in digital production and dissemination technologies, an investment that has yielded stunning improvements in productivity, capability, and savings for the taxpayers, savings of 66% on the cost of congressional printing alone. Employing just 2,200 staff, fewer than at any time in the past century, GPO now provides a range of products and activities that could only have been dreamed of 30 years ago: online databases of Federal documents with state-of-the-art search and retrieval capabilities available to the public without charge, Government publications available as e-Books, passports and smart cards with electronic chips carrying biometric data, print products on sustainable substrates using vegetable oil based inks, and a public presence not only on the Web but on Twitter, Facebook, and You Tube.

There is a good interview with Carl Malamud in Library Journal. Carl sees the intersection of legally public domain government information with orphan works, all legal information, and information we are losing because of lack of curation and control by the private sector.

• Public Information for All: An Interview with Carl Malamud, By Debbie Rabina, Library Journal (Nov 1, 2010).

  Do you see a role for libraries?

  The availability of materials and the parceling out in the public domain is a huge issue for libraries, not only for government documents generally and legal material specifically but everything from the letters of Ben Franklin to all this wonderful corpus of materials that has been issued but is no longer available, the problem of orphan and fallow works, for example, the problem of official vendors of materials that don’t allow knowledge to be spread. Librarians should and must be jumping up and down and pounding the table and saying, “This is a huge issue.”

Carl Malamud's at it again. In his rousing opening keynote titled "Currents of Our Time" at the 2010 Gov2.0 Summit in Washington DC, Malamud described both the horrendous current lay of the land in govt IT as well as several uplifting examples from the progressive era of government and activists at the forefront of positive public change. Then he defined his 3 steps for for transforming government as platform:

1. finish the opengov revolution (bulk data, strong FOIA, providing access to knowledge to all!)
2. spend a minimum of $250 million per year for a decade on a national scanning initiative.
3. open systems revolution: "a Computer Commission with the kind of authority the Civil Service Commission had to conduct agency-by-agency reviews and help us reboot .gov, flipping the bit from a reliance on over-designed custom systems to one based on open-source building blocks, judicious use of commercial off-the-shelf-components, and much tighter control of the beltway bandits."

"If we can land a man on the moon, surely we can launch the Library of Congress into cyberspace!"

Viva la open govt revolución!!

PS. Don't miss the other very fine speakers at the Summit as well!

Here's a way to spend an enjoyable lunchtime: watch Carl Malamud give his Keynote address "10 Rules for Radicals" to the WWW2010 Conference in Raleigh, NC on April 30, 2010 -- and if you've got more time, you can also watch all of the law.gov workshops over on Carl's Internet governance space at the Internet Archive! Certainly some great rules to live by!!

1. Call everything "an experiment."
2. When the authorities finally fire the starting gun, run as fast as you can.
3. Eyeballs rule.
4. When you achieve your objective, don't be afraid to turn on a dime and be nice.
5. Keep asking, keep rephrasing the question until they *can* say yes.
6. When you get the microphone, make sure you make your point clearly and succinctly.
7. Get standing. one can criticize all one wants, but if you can document malfeasance and wrongdoing, they have to talk to you.
8. Try to get the bureaucrats to threaten you (related to rule 7).
9. Look for over-reaching.
10. Don't be afraid to fail

[Thanks BoingBoing!]

Wow that was fast! The Gov2.0 Expo in Washington DC just wrapped up 2 days of speakers, panels, discussions etc and already the video from the Expo is up online (youtube and blip.tv channels). Here's just 2 of the many interesting talks that I've only begun to absorb. Enjoy!

Wish I could be in Washington DC next tuesday for CopyNight when Carl Malamud will speak at ALA Washington Office. If any of our readers go, please leave comments here on your thoughts/ideas/brainstorms/concerns etc. Thanks!

On Tuesday, May 25, the ALA Washington Office will host DC’s “CopyNight” group for an evening with special guest Carl Malamud about the future of public information.

Malamud is the founder of Public.Resource.Org, a foundation dedicated to making public information accessible. His latest project is an effort to bring all of the United States primary legal sources, such as legal codes and case law, online for free public access. Currently, access to many legal sources is only available through commercial databases that are extraordinarily expensive to use – making these materials inaccessible to most of the public. Malamud has been holding a series of public workshops and symposia, with help from a variety of thought leaders in law and technology, presenting the issues and challenges facing the project.

He’ll also talk about the International Amateur Scanning League, a group of DC-area volunteers digitizing government-produced DVDs currently only available from the National Archives in College Park, which he is making available through YouTube, the Internet Archive, and Public Resource’s own Public Domain Stock Footage Library.

Carl Malamud announced yesterday the inaugural meeting of the International Amateur Scanning League (IASL) (I'm already imagining cool swag!). Malamud is taking FedFlix program to the streets! Fedflix, a joint venture between the National Technical Information Service and Public.Resource.Org, digitizes NTIS video and makes them available on YouTube, the Internet Archive, and the public.resource.org Stock Footage Library.

Well now a gang of volunteers including members of DC CopyNight and Smithsonian employees working on their own time are going to the National Archives and Records Administration (NARA) and copying over 1,500 DVDs to be uploaded to the net.

Malamud said:

What makes this grassroots digitization effort so remarkable is that it has the full support of the government. Indeed, David Ferriero, the U.S. Archivist, joined me in the initial meeting where we taught volunteers how to rip DVDs!

Kudos to Malamud and the IASL!

And this makes me think that more libraries and librarians should be doing the same thing for govt documents. Why not set up your own scanning operations in your depository library (Book Liberator or DIY Book Scanner can show you how to digitize on the cheap!) and then deposit those scans into the Internet Archive's US Documents Collection (don't forget to follow FDLP digitization standards!). Scans could also be ingested into FDSys (when they've got that capability working ;-)). So get to it; what are you waiting for?!

Carl Malamud posed this question over on twitter: "What if our national cultural institutions all worked together on a common problem, attracted White House support?" In his post on the O'Reilly blog, "A National Scan Center: A Public Works Project", Malamud scopes out the issues and calls for Library of Congress, the Smithsonian Institution, the Government Printing Office, the National Archives and Records Administration, and the National Technical Information Service to come together and make the compelling case for funding a 5-year $500 million effort to create a National Scan Center. Here here Carl!

In the U.S., we face a similar deluge of paperwork that we faced in the 1930s. A huge backlog of paper, microfiche, audio, video, and other materials is located throughout the federal government. Little money has gone from Congress for digitization, and bureaucracies have resorted to a series of questionable private-public partnerships as a way of digitizing their materials. For example, the Government Accountability Office shipped 60 million pages of our Federal Legislative Histories (the record of each law from the initial bill through the hearings and conference reports) off to Thomson West, but didn't even get digital copies back. Another example is the recent failed effort by the Government Printing Office to digitize 60 million pages of the Federal Depository Library Program, an effort they tried to get through as a "zero dollar cost to the government" effort with the private sector.

There are no free lunches and there are no "no cost to the government" deals. The costs involve the government effort to supervise the contract, prepare the materials, and ship them, and in both the GAO and GPO cases, the government wasn't getting much back for its effort. What the government and the people usually get is a lien on the public domain, preventing the public from accessing these vital materials. Similar efforts are sprinkled throughout the government. I testified to Congress that I had learned that the National Archives was contemplating a scan of congressional hearings with LexisNexis under similar circumstances, and many may be aware of the questionable deal the Archives cut with Amazon where my favorite online superstore got de facto exclusive rights to 1,899 wonderful pieces of video.

Carl Malamud's FedFlix project is a joint venture with the National Technical Information Service (NTIS) whereby he takes NTIS videos, digitizes them and uploads them to the Internet Archive.

Well now he's expanding FedFlix to include public domain videos from the National Archives. He's released 41 videos into the public domain in this way, but has put together an Amazon Wish List in order to expand public access to public domain video content from the National Archives. If you see anything you'd like to buy the public domain, they'll take your DVD and upload the video to YouTube, the Internet Archive, and to public.resource.org's own rsync/ftp public domain stock footage library. So why not add a gift of the public domain to your favorite person's/people's stockings this year? We'll all be glad we did!

UPDATE 12/25/09: The wish list has been fulfilled. You can watch all of the donated NARA videos on YouTube, Internet Archive, or public.resource.org's bulk server. Thanks Carl!

[HT BoingBoing!]

We posted about Carl Malamud's address to the Gov2.0 Summit in september, but BoingBoing reminded us that there is now video of his address (below). Carl's speech is quite rousing and reminds all of us what we can and should be doing to facilitate access to government information. You can also get his pamphlet online to read along with the address.

And don't forget to read Appendix A: "29 things government could do today." One thing I would add to that list is that every witness statement inserted into the official record in the course of public Congressional hearings should be considered in the public domain regardless of its original copyright status (some witnesses submit published articles, book chapters and the like as part of their written statements which means that the Google Book Project *still* treats post-1923 scanned government publications as if they were in copyright and only shows snippets instead of full-text.)

“Government as platform” means exposing the core information that makes government function, information that is of tremendous economic value to society. Government information—patents, corporate filings, agriculture research, maps, weather, medical research—is the raw material of innovation, creating a wealth of business opportunities that drive our economy forward. Government information is a form of infrastructure, no less important to our modern life than our roads, electrical grid, or water systems. (p.21)

[Thanks for the reminder BoingBoing!]