Sunlight Foundation's OpenGov Champion of the month is Sandra Moscoso. Sandra is a mom of two public school students in Washington DC, and a member of the Capitol Hill Public School Parent Organization (CHPSPO) -- oh and she just happens to manage an open data portal at the World Bank’s financial sector.

...she and other CHPSPO members were able to collect data to show how the schools that had a full time librarian had better test score results than those who had lost theirs due to budget cuts. The group was able to use that figure as an effective basis for their request to the city to restore funding for librarians.

The Sunlight Foundation just put out their Open Legislative Data Report Card. California received a D grade :-| Find out how your state is doing. Below is the methodology that they used to grade state legislatures.

Methodology

Each state was evaluated in six categories based largely on the Ten Principles For Opening Up Government Information. Each score is based on at least two members of staff and a volunteer during our state survey. Additionally, state legislatures were contacted (unless noted in their score) to ensure that our information on bulk data availability and timeliness was as accurate as possible.

The specific criteria for each category are as follows:

COMPLETENESS

We evaluated each state on the data collected by Open States: bills, legislators, committees, votes and events. We also took note if a state went above and beyond to provide this information and other relevant contextual information such as supporting documents, legislative journals and schedules. Points were deducted for missing data, often roll call votes.

• 0 State provides full breadth of legislative artifacts Open States collects: bills, legislators, votes, and committees.
• -1 State does not provide stand-alone roll call votes.

TIMELINESS

Legislative information is most relevant when it happens, and many states are publishing information in real time. Unfortunately, there are also states where updates are more infrequent and showing up days after a legislative action took place. States were dinged if data took more than 48 hours to go online.

• 1 Multiple updates throughout the day, real time or as close to it as systems will allow.
• 0 Site updates once or twice daily, typically at the end of the legislative day.
• -1 Updates take longer than 24 hours to appear on the site, often up to a week.

EASE OF ACCESS

Common web technologies such as Flash or JavaScript can cause problems when reviewing legislative data. We found that the majority of sites work fairly well without JavaScript, but some received lower scores due to being extremely difficult to navigate, impossible to bookmark bills, and in extreme cases, completely unusable.

• 1 Site was considered exceptionally well layed out by multiple evaluators, no issues with Javascript.
• 0 Site was deemed average by those that evaluated it and/or had minor Javascript dependencies.
• -1 Site was considered more difficult than average to use by members of staff or volunteers or had more severe Javascript dependencies.
• -2 Site was considered extremely difficult to use with a heavy reliance on irregular browser behavior and Javascript.

MACHINE READABILITY

For many sites, the Open States team wrote scrapers to collect legislative information from the website code—a slow, tedious and error prone process. We collected data faster and more reliably when data was provided in a machine-readable format such as XML, JSON, CSV or via bulk downloads. If a state posted PDF image files or scanned documents, it received the lowest score possible.

• 2 Essentially all data can be found in machine-readable formats.
• 1 Lots of data in machine readable format but substantial portions that still required scraping HTML.
• 0 No machine readable data but standard screen scraping techniques applied.
• -1 Site had information that was much more difficult than average to collect. (Data only accessible via PDF or that required screen scraper to emulate Javascript.)
• -2 Site had information that was unaccessible to Open States due to use of scanned PDFs.

USE OF COMMONLY OWNED STANDARDS

Because our ability to access most of a state’s data is represented by the above “Machine Readability” metric, we decided to use this provision to measure how a state made their bill text available. Making text available in HTML or PDF is the norm, and was considered an acceptable commonly owned standard (PDFs are a commonly owned standard, but it would be certainly nice to see alternative options where bill text is only available via PDF). States that only make documents available in Microsoft Word or Wordperfect formats require an individual to purchase expensive software or rely on free alternatives that may not preserve the correct formatting. It is worth noting, all states except for two met the common criteria of providing HTML and/or PDF only, one state (Kansas) went above and beyond and another (Kentucky) did not even meet this threshold.

• 1 State made an effort to go above and beyond.
• 0 State provided bills in PDF and/or HTML format and nothing better (plaintext, ODT, etc.).
• -1 State only provided bills in a proprietary format.

PERMANENCE

Many states move or remove information when a new session starts, much to the dismay of citizens seeking information on old proposals and researchers that may have cited a link (e.g. http://somelegislature.gov/HB1 vs http://somelegislature.gov/2011/HB1) only to see it point to a different bill in the following session. Tim Berners-Lee, inventor of the World Wide Web, wrote an article declaring Cool URIs Don’t Change and we agree.

This poses a particular challenge to us since every page on OpenStates.org points to the page we collected data from, but if a state changes their site then users lose the ability to check us against the original source. Most (but not all) states are good about at least preserving bill information, but few were equally as good about preserving information about out-of-office legislators and historical committees, equally important parts of the legislative process.

• 2 All information is avaialble in a permanent location and data goes back a reasonable amount of time (a decade or so).
• 1 Almost all information has a permanent location but a single data set doesn't. (Or a recent change to the site has wiped out historical links but information appears to be preservable going forward.)
• 0 Legislator & committee information lacks a permanent location (such as committees and legislators) but most is acceptable.
• -1 Ability to link to old information is badly damaged and and/or there is less than a decade of historical information.
• -2 Vital information like bills or versions lack a permanent location.

The Sunlight Foundation has just released Open States for all 50 states, the District of Columbia and Puerto Rico. The site helps the public find their state legislators, review their votes, search upcoming legislation, and track bill progress. Open States gets their Bill, legislator, committee and event data from official sources, linked at the bottom of each legislator, bill, vote, committee or event page. Check out their methodology for more. They rely primarily on scraping data from sites. Wouldn't it be awesome of all state legislatures had bulk data feeds so that 1000 sites like Open States could bloom? Join the Webinar on February 22nd to learn more about Open States.

After more than four years of work from volunteers and a full-time team here at Sunlight we're immensely proud to launch the full Open States site with searchable legislative data for all 50 states, D.C. and Puerto Rico. Open States is the only comprehensive database of activities from all state capitols that makes it easy to find your state lawmaker, review their votes, search for legislation, track bills and much more.

If you're interested in your state lawmaker, you'll be able to get notifications for their actions, a map of their district, voting records, committee assignments, campaign finance records from Influence Explorer, local news articles and contact information. If you're curious about a particular piece of legislation, Open States allows you to check on its status, find the sponsors, break down votes, view bill text and all supporting documents. Our powerful search capabilities allow you to find similar topics across states and view overview pages for each state, chamber and committee.

The Sunlight Foundation recently named Liz Barry and her group at the Public Laboratory for Open Technology and Science (PLOTS) as OpenGov Champions. Sunlight highlights these champions for their work and ingenuity in furthering govt transparency.

Ms Barry and the PLOTS team is perhaps best known for using kites and helium balloons to map the BP Deepwater Horizon Oil Spill in 2010, the only high resolution images out in the media at the onset of the catastrophe. PLOTS uses "mapping and other scientific DIY methods to empower local residents and activists to issue their own data sets to better engage with their local governments in environmental and other issues in their communities."

Be sure to check out their many maps available in the PLOTS open data archive. And for all of you DIY scientists, you can chip in to the PLOTS DIY spectrometry kit kickstarter campaign and help them build a spectrum-sharing wiki.

Congratulations PLOTS!

Thanks Sunlight Foundation for creating Ad Hawk, the smartphone app that identifies political ads as they air and immediately tells you about who is behind them. I'm not so inundated with ads here in CA, but I bet this will be useful for all of our readers who are in swing states.

A friend just alerted me to the existence of the Sunlight Academy, "a collection of interactive tutorials for journalists, activists, researchers and students to learn about tools by the Sunlight Foundation and others to unlock government data." They've got training modules for tracking govt, unlocking data, lobbying, data analysis, and research tools. Very nicely done! I wish there was a site training.fdlp.gov for the Federal Depository Library Program ;-)

Whether you are an investigative journalist trying to get insight on a complex data set, an activist uncovering the hidden influence behind your issue, or a congressional staffer in need of mastering legislative data, Sunlight Academy guides you through how to make our tools work for you. Let’s get started!

HT to my friend Sue Lyons for alerting me to the Sunlight site and suggesting it as a model for the FDLP to follow!

[Editor's note: Adeeb Sahar, Stanford undergraduate student and Sunlight Foundation intern, asked me to post the following PSA about Sunlight's many projects of interest to students, researchers, and the public. FGI has no official connection to Sunlight Foundation. We just love what they're doing!]

The Sunlight Foundation has launched a campaign to partner with university libraries to provide easy access for students and researchers by cataloging as electronic resources its vast online databases of information on politics and government data.

Sunlight Foundation is a nonpartisan, nonprofit organization working to enhance government transparency through free online resources that track political contributions, follow federal regulations and bills and monitor Congressional activity.

Many universities have already let in the sunlight; Sunlight's projects are cataloged in university library databases including those at Stanford University, New York University and the University of Pennsylvania. In its ongoing effort to supply government information to students, the Sunlight Foundation is looking to partner with even more university libraries.

The following are the most commonly cataloged databases by university libraries and are geared toward university-level researchers and students interested in political science, public policy, and politics and government:

• Scout is the first free searchable database of regulations and bills from all fifty states and the federal government. This service searches through a variety of sources including the Congressional Record, THOMAS, and the Federal Register to produce curated legislative news alerts.
• Influence Explorer contains the most recent information on political contributions, lobbying information, contracts and other government data, allowing users to track and analyze influence by lawmaker, company or prominent individual.
• Clearspending is a scorecard that analyzes how well U.S. government agencies are reporting their spending data on USAspending.gov and provides insights to any descrepancies.
• Open Congress brings together official legislative data with news and blog coverage, social networking, public participation tools, and more to give users a comprehensive assessment of Congressional activity.
• Capitol Words makes searchable all Congressional records from 1996 to today by state, date or politician to uncover the most popular words and phrases used by legislators in the U.S. Congress.

If you are a subject specialist interested in including Sunlight Foundation's electronic databases on your university library website, contact Adeeb Sahar at asahar@sunlightfoundation.com or Amy Ngai at angai@sunlightfoundation.com. See the Sunlight Foundation site for more information about our projects.

Today, our pals at the Sunlight Foundation released Scout, a new tool that allows you to create customized keyword alerts to notify you whenever issues you care about are included in legislative or regulatory actions -- at both the state and federal level! They'll also soon release their Open States tool to target the legislative process of all 50 states.

Thanks Sunlight!

Start by entering a keyword or phrase you would like to get updates about, such as the vaguely defined "cyber threat" included in CISPA or any references to the "Digital Millennium Copyright Act." Scout then saves your subscriptions and sends notifications via email or text message whenever the subscribed issue or bill is talked about on the floor of Congress, mentioned in new regulations, appears in state and federal legislation or when Congress is moving forward for a vote. Through your profile you can create as many alerts as you'd like and group them by tags with the additional option to make them public for others to follow your issues. You can also complement a Scout subscription by adding optional external RSS feeds, such as press releases from a member of Congress or an issue-based blog.

[HT to BoingBoing!]

FGI just signed the letter below written by the Sunlight Foundation asking Congress to improve public access to legislative information by directing the Library of Congress to make their Thomas database accessible in bulk format. If you and/or your organization believe that free access to Congressional information is of critical importance, please please consider adding your name to the list of signatories on the letter. Daniel Schuman, Sunlight Foundation's policy counsel and director of the Advisory Committee on Transparency, requests that people sign on by COB on Monday April 2nd. Interested people may also email Daniel at dschuman@sunlightfoundation.com) with how they would like to be identified on the letter. Daniel thanks you and so do we!

Dear Congressman/Senator:

We are writing to ask you to improve public access to legislative information by directing the Library of Congress to publish the THOMAS database online. Congress created THOMAS with the mission of making federal legislation freely available to the public. While times have changed, and technologies have changed, THOMAS has not kept up.

As a result, millions of Americans access basic information about legislation and congressional actions through online information providers like GovTrack, OpenCongress, and Washington Watch. These free non-governmental websites are forced to rely on brittle programs to harvest information from THOMAS’s complex website. This harvesting is imperfect, expensive, and time consuming. The better approach -- which has been adopted by industry and many in government -- is to publish legislative information "in bulk" in addition to other means.

Bulk access would in essence make the entire legislative database available for download, instead of requiring users to gather information by visiting hundreds or thousands of web pages. It would make it easier for third parties to build innovative new tools, and ensure that Americans have the most accurate information at their fingertips. Congress already expressed its support for bulk access downloads in 2009, but the Library of Congress, which oversees THOMAS, has not acted. In the meantime, GPO, the executive branch, and the House of Representatives are already publishing information online in bulk.

The time has come for action. In this year's legislative branch appropriations bill, we urge you to direct the Library of Congress to implement bulk access to THOMAS within 120 days. The Library should also immediately create an advisory committee on improving public access to legislative information composed of people inside and outside of government. Congress should ensure that THOMAS lives up to its potential of making the legislative branch more open and transparent.

For more information, please contact Daniel Schuman, policy counsel, the Sunlight Foundation, at 202-742-1520 x 273 or dschuman@sunlightfoundation.com

Our Sunlight friend John Wonderlich provides some in-depth context for the Department of Justice's proposed new FOIA rules and from the blog title it's easy to guess what those new rules look like -- "Justice Department's New FOIA Regulations: Still Worse than Reported". Wonderlich's colleague Daniel Schuman created a side-by-side comparison of current and proposed new regulations, to help illuminate the differences.

Instead of obfuscating regulations and making them more restrictive, DoJ lawyers should be trying to simplify and expand the scope of FOIA in line with the Obama Administration's stated goals and Open Government Initiative. Otherwise, the rhetoric of "an unprecedented level of openness in government" and the establishment of "a system of transparency, public participation, and collaboration" will ring hollow or worse.

The DOJ sent a letter to respond to Congressional concerns about their lying about the existence of records. The letter hardly paints a clear picture, but basically says that the DOJ will withdraw a section of the proposed regulations, but that their conduct won't change, and that they'll continue to mislead requesters about whether records exist or not.

Unmentioned in the letter, however, are all the steps backward on FOIA that the DOJ is proposing in their rules. In a package completely at odds with President Obama and Attorney General Holder's public FOIA rhetoric, the new DOJ rules throw up new roadblocks and hurdles to requesters, and generally make it easier to deny requests. One has to wonder what possible motivation DOJ has for forcing elementary schools to pay for FOIA requests, where they used to qualify for fee waivers. Have elementary school students' FOI requests become a burden?