For those of you that willl be in Washington DC next week, please consider attending the 2013 Legislative Data and Transparency Conference (RSVP required). There will be several interesting panels with House and external stakeholders like the Sunlight Foundation and the Cornell Legal Information Institute -- including a panel on electronic archiving and one on "missing data" and what to do about it ("missing" meaning not effectively on-line and digital, etc.).

The 2013 Legislative Data and Transparency Conference will take place on Wednesday, May 22, 2013, from 9 a.m. to 5 p.m. in the Capitol Visitor Center Auditorium. The conference brings together legislative branch agencies with data users and transparency advocates to discuss the use and future of legislative data. Topics include:

--Electronic legislative archiving
--XML and metadata standards
--Updates on beta.congress.gov

The Good, the Bad, and the Ugly in the Fiscal Cliff Package, "Savings and Costs in the Fiscal Cliff Package" [table] The Committee for a Responsible Federal Budget (January 1, 2013).

This is based in part on the document:

• Estimated Revenue Effects Of The Revenue Provisions Contained In An Amendment In The Nature Of A Substitute To H.R.8, The "American Taxpayer Relief Act Of 2012," As Passed By The Senate On January 1, 2013 Fiscal Years 2013 - 2022, JCX-1-13 Joint Committee On Taxation (January 1, 2013).

John Wonderlich does a good job of summing up how openness was a casualty of the so-called fiscal cliff drama.

• The Fiscal Cliff Process was an Atrocious, Secretive Mess, by John Wonderlich, Sunlight Foundation (Jan. 2, 2013).

  As we expected, the culmination of the "fiscal cliff" negotiations was a rush to the finish line, in which policies decided by a few men in a room were passed through the Congress without amendment.

Update:

Here is an interesting view of the bill that passed:

• 6 Things You Won't Believe That Are In The Fiscal Cliff Bill That The Senate Passed At 2 AM While Most Americans Were Drunk, by Joe Weisenthal, Business Insider (Jan. 1, 2013).

  ...why is the bill 157-pages long?

  There's a provision extending a tax policy related to Puerto Rican rum... a tax credit for 2- and 3-wheel electric vehicles... An extension of some special rules for the film and television business... A gift to the car-racing world... Help to asparagus farmers...

As Wonderlich said, "While Congress (and the rest of us) only just found out what was in the bill, a coterie of corporate lobbyists managed to get their profit-boosting tax expenditures included. It's hard to imagine how NASCAR and Hollywood had stronger negotiating positions than the House of Representatives, but in the end, they did."

The Library of Congress unveiled a new Web search tool for bills and other Congressional records Wednesday that will eventually replace the 17-year-old Thomas.gov website.

• Congress.gov. Also see: About page.

  Congress.gov makes federal United States legislative information freely available to the public. Launched Sept. 19, 2012, this version of the site is an initial beta release of Congress.gov, created as a successor to THOMAS.gov, the current public site for legislative information. The Congress.gov beta site contains legislation from the 107th Congress (2001) to the present, member of Congress profiles from the 93rd Congress (1973) to the present, and selected member profiles from the 80th through the 92nd Congresses (1947 to 1972). Over the next two years, Congress.gov will be adding information and features, eventually incorporating all of the information currently available on THOMAS.gov.

• Smartphone friendly, congressional search site unveiled, By Joseph Marks, NextGov (Sep 19, 2012).
   
• Congress launches THOMAS successor Congress.gov, by Daniel Schuman, Sunlight Foundation (Sept. 19, 2012)

  What's noticeable about this evolving beta website, besides the major improvements in how people can search and understand legislative developments, is what's still missing: public comment on the design process and computer-friendly bulk access to the underlying data.

Update:
Here is another story:

• What Congress.gov Means for a Congressional API, by Nick Judd and Miranda Neubauer TechPresident (September 19 2012)

  "I'm impressed," said Josh Tauberer, whose GovTrack scrapes data from THOMAS to provide it in a machine-readable form for other websites like OpenCongress, in an email. "From its new faceted search to its mobile-friendly HTML, they really hit the technology on the nail. And there's more explanation for people who aren't legislative pros. They may be slowly catching up to GovTrack.

  "This new site shows that the LOC actually has the technical chops to implement raw data properly, which was a serious concern of mine before," Tauberer also wrote.

  That said, Tauberer pointed out that the new site offers "no new actual information." House leadership has promised to offer access to the underlying data that fuels THOMAS and has repeatedly expressed a commitment to doing it. They just haven't committed to doing it during this Congress. And the lack of action on something that seems to them to be eminently doable has advocates kind of frustrated.

  Gayle Osterberg, Director of Communications for the Library of Congress, seemed to indicate in an email that the Library of Congress is ready to cooperate. They just need Congress — meaning the House and Senate both — to give them the go-ahead.

Another update:

• Congress.gov Beta: An Early Look at a New THOMAS, by Peggy Garvin, InfoToday, (September 27, 2012).

  The Congress.gov beta is still in the early stages of incorporating existing THOMAS content and implementing the improved search functions that THOMAS users have been waiting for. The Law Library of Congress, which is managing the transition, is anxious to get your feedback and suggestions via its form at http://beta.congress.gov/survey.

Looking Forward to the THOMAS Beta Website, by Daniel Schuman, Sunlight Foundation (Sept. 14, 2012).

In the near future, Congress is expected to release a major upgrade to its aging legislative information website THOMAS. The long-overdue update is part of a much larger effort to "enhance the effectiveness of mission-critical systems," a response to significant public and internal pressure to improve congressional efficiency and transparency. The launch of "THOMAS Beta" is the first step towards developing what the Library of Congress describes as a completely "modern legislative information system" that will replace THOMAS and Congress' more sophisticated internal legislative tracking website "LIS" in FY 2014. Both THOMAS and LIS will stay online alongside the beta website for several years.

While THOMAS Beta has been shown to stakeholders inside Congress, as far as I am aware there has been no formal engagement process with the public to identify specifications, discuss wireframes, or generally make sure the site meets the public's needs.

The Congressional Research Service has published an update to its handy guide for finding current legislation and regulations:

• Researching Current Federal Legislation and Regulations: A Guide to Resources for Congressional Staff. by Jerry W. Mansfield, Congressional Research Service, RL33895 (August 31, 2012). Available from Federation of American Scientists.

For those experienced in legislative and regulatory searching there won't be anything new or surprising here, but it is a handy introduction and reference.

One thing I particularly liked was the comparison on p. 13 of the "Legislative Information System," which provides access to legislative information to Members of Congress and their staff, and THOMAS, which makes information on federal legislation freely available to the public. That's right, one system for Congress and a separate system for us ordinary folk.

Here is a sample:

Again, this won't be news to most of you, but it is a nice summary of what we are missing.

This article reports on the importance of a bill that will enable Congress to provide bulk access to its legislative data. It also profiles one of the heroes of open-access to Congressional data, Josh Tauberer. As the Post says, Josh has prodded Congress and the result may be the "raw material for an Angie's List or a Yelp for Congress, a way for modern users to evaluate lawmakers with the same kind of crowdsourced help that they use to evaluate lunch."

This is a lot like how Carl Malamud got the SEC to put the EDGAR database online. (SEC'S EDGAR On Net, What Happened And Why, TAP-INFO, 30 Nov 1993).

Congressional data may soon be easier to use online, by David A. Fahrenthold, Washington Post, (June 8, 2012)..

Online, searching for a bill in Congress feels a little like time travel: Go looking for legislation, and you wind up in the Internet of 1995.

At Congress's '90s-vintage archive site, there's no way to compare bills side by side. No tool to measure the success rate of a bill's sponsor. And there's certainly no way to leave a comment. Congress makes it hard for outside sites to do any of this, either, by refusing to give out bulk data on its bills in a user-friendly form.

On Friday, that might start to change.

Rep. Crenshaw backs down, loses control over bulk data issue, by Josh Tauberer, GovTrack.us (June 7, 2012).

The government data that makes GovTrack go has been the center of what looks like a failed political power play over the last week. Rep. Crenshaw, whose appropriations subcommittee issued a draft report last week that nearly halted access to "bulk data downloads," now "agree[s] to free legislative information" according to a statement written jointly with House leaders yesterday.

Time to contact your representatives!

• #FreeTHOMAS, by Daniel Schuman, Sunlight Foundation (June 4, 2012)

  The better approach is for Congress to publish the data behind THOMAS. Government regularly does this elsewhere, and "bulk data" is responsible for clever new uses of information developed by citizens, journalists, and even the government itself.

  In upcoming days, the House is likely to pass legislative language that pays lip service to releasing THOMAS data while putting the idea in a deep freeze. This would be a disaster. But it's not too late. Tell your representative that you want Congress to publish legislative data now.

Daniel Schuman of Sunlight Foundation and Josh Tauberer of GovTrack.us give easy to understand explanations of the importance of a bill before Congress (H.R. 5882) that deals with open access to bulk Congressional data.

The committee report on the bill gives an embarrassing justification for denying bulk access to legislation:

...the Committee is also concerned that Congress maintains the ability to ensure that its legislative data files remain intact and a trusted source once they are removed from the Government's domain to private sites.

This is either disingenuous or badly informed.

• Bulk Access Developments after the H. Approps Hearing, by Daniel Schuman, Sunlight Foundation (June 1, 2012).

  Instead of striking a balance between the desire for openness with legitimate concerns about process, they fell victim to fears and misunderstanding about technology that resulted in a ham-fisted process that will likely freeze any forward momentum, or maybe even turn back the clock.

• Rep. Crenshaw thinks American public can’t be trusted with overseeing Congress, by Josh Tauberer, GovTrack.us Blog (June 4, 2012).

  ...for the last eight years I've been making this sort of information available on GovTrack, and last I checked that was a good thing. Even Congress's staff uses GovTrack: Crenshaw’s own staff has probably used GovTrack for their research.

Update:
I missed this piece which is another good explanation:

• Hill may freeze THOMAS in digital past, by Jennifer Peebles, Washington Examiner (May 31, 2012).

  A web interface that lets us call up and download one bill at a time was really innovative once -- say, 15 years ago. But that won't cut it anymore.

  Folks with computers -- notably, professional and citizen journalists -- would be able to take information about massive numbers of bills and analyze them in myriad ways -- if Congress would allow such information to be downloaded from THOMAS in bulk.

  It won't. And, according to a new draft report from the House Appropriations Committee, it won't be allowing bulk data downloads from THOMAS anytime soon.