Home » Articles posted by James A Jacobs (Page 102)
Author Archives: James A Jacobs
Accuracy of Digitization
The Update on February 2012 Activities of the HathiTrust reports on research being done by the HathiTrust Research Center (HTRC) to quantify occurrences of Optical Character Recognition (OCR) errors in the HathiTrust corpus. OCR is the technology that converts a scanned image to text that can be searched and analyzed. Members of the HTRC examined 256,000 non-Google digitized volumes from HathiTrust using a clever algorithm that compared OCR text to a dictionary of known words. Using a supercomputer and a set of rules that were verified by a human expert, they found that 84.9 percent of the volumes examined (217,754 of the 256,416) had one or more OCR errors and 11% of the pages (7,745,034 of the 69,297,000) had one or more errors. The average number of errors per volume was 156. As we at FGI have argued here before, we believe that it essential to take into account OCR accuracy and error rates when digitizing paper collections. This is particularly important when digitizing books that contain statistical tables since it is harder to use current OCR technologies to accurately convert image scans to numbers than it is to convert scans to text. It is also harder to evaluate the accuracy of such conversions; you can't use a dictionary of known statistics the way you can use a dictionary of known words. (The HTRC study did not, apparently, examine accuracy of statistical table conversions.) Because of the large volume of such information in government publications, this is a very important issue as we collectively try to digitize our paper collections, evaluate their accuracy and usability, and determine how many paper copies we need to keep after digitization (see Achieving a collaborative FDLP future). Continue reading
Four Seriously Cool Information Resources
Our friend Gary Price has started a new series over at searchengineland on "incredibly useful online information resources that are most effectively searched using their own site search tools, rather than relying on general-purpose engines to surface their valuable content." This looks to be a series worth following. At least 3 of the 4 in his first entry should be of particular interest to government information specialists: the C-SPAN Video Library, Old Maps Online, and Archive-It. Gary's comments are always useful. Did you know that, "unlike the Internet Archive’s Wayback Machine, Archive-It collections are keyword searchable"?
- Four Seriously Cool Information Resources, by Gary Price, Search Engine Land, (March 9, 2012).
FRUS now available as e-books
Selected volumes of the eminent State Department series, Foreign Relations of the United States, are now available as e-books for reading on devices such as the Kindle and Nook.
- E-Books Initiative, U.S. Department of State, Office of the Historian. The Office of the Historian at the U.S. Department of State is pleased to announce the release of its Foreign Relations of the United States (FRUS) series in a new e-book format that is readable on popular electronic devices such as the Amazon Kindle and Apple iPad. The e-book edition combines many of the benefits of print and web publications in a new form that is portable and extremely convenient. During the pilot phase of the FRUS e-book initiative, five selected FRUS volumes are available here. The public is invited to download the new e-books and provide feedback to help improve the FRUS e-book edition. At the conclusion of the pilot phase later this year, the Office will work to offer e-book versions of many more FRUS volumes both through the Office website and on a wide array of e-bookstores. The Office will continue to expand and enhance its e-book offerings, as part of the ongoing FRUS digitization effort.
- Foreign Relations of the United States, 1964–1968, Volume XIX, Arab-Israeli Crisis and War, 1967.
- Foreign Relations of the United States, 1969–1976, Volume X, Vietnam, January 1973–July 1975.
- Foreign Relations of the United States, 1969–1976, Volume XXIV, Middle East Region and Arabian Peninsula, 1969–1972; Jordan, September 1970.
- Foreign Relations of the United States, 1969–1976, Volume XXXII, SALT I, 1969–1972.
- Foreign Relations of the United States, 1969–1976, Volume E–12, Documents on East and Southeast Asia, 1973–1976.
Providing better services by building digital collections
In an excellent article in the current issue of Library Hi Tech, Eric Morgan, the Digital Projects Librarian at the Hesburgh Libraries at the of the University of Notre Dame, provides examples of building services based on digital collections. As he says:
In our mind, the combination of digital humanities computing techniques - like all the services against texts outlined above - and the practices of librarianship would be a marriage made in heaven. By supplementing library collections with full text materials and then enhancing its systems to facilitate text mining and natural language processing, libraries can not only make it easier for readers to find data and information, but it can also make that data and information easier to use and understand.
- Use and understand: the inclusion of services against texts in library catalogs and "discovery systems." by Eric Lease Morgan, Library Hi Tech, Vol. 30 Iss: 1, (2012) pp.35 - 59. [subscription required]
The Impact of Disclosure on Public News & Knowledge
March 14, 2012 / Leave a comment
In honor of Sunshine Week the Sunlight Foundation has created Back to the Source: REDACTED in which they take original investigative articles and manually black out all the information that would not be known without existing transparency measures. It is worth taking a look at just how little we would know with the Freedom of Information Act and other public disclosure laws.
- Back to the Source REDACTED: The Impact of Disclosure on Public News & Knowledge, by Melanie Buck, Sunlight Foundation (March 14, 2012).
While journalistic skill and technique are essential for writing a good investigative article, we often take it for granted that journalists have access to the information they need to write complex news stories. Without publicly available data, much of our news would not be possible. We've been looking at investigative articles as part of an ongoing series called "Back to the Source" for the last several months. Now we've decided to amp it up a bit and make redacted visuals to explicitly demonstrate how little the public would know without laws and regulations that force the government to make the data it has publicly available.
Continue reading →Continue Reading →