Public comments and response to Ithaka S+R Models draft report

We have sent Ithaka S+R our comments for the draft directions and draft models documents (as well as comments on the other draft documents), but feel that a more in-depth analysis and response is warranted.

While there is much to like about this draft — as other commenters have rightfully pointed out — it has some serious gaps that could have potentially dire consequences for the future of the FDLP.

The main strength of the report is that it explicitly recognizes the need for digital preservation beyond the confines of GPO. The report says:

“Although GPO has taken on an important leadership role in preserving and maintaining the integrity of digital government information, GPO alone cannot effectively accomplish these goals; distributed responsibility for preserving and maintaining the integrity of government information is a long-standing value of the Program that must be maintained in the digital environment to provide users with confidence that the materials they use in digital form will remain available and unchanged over the long term.”

“Historically, government publications have been made tamper-resistant by the broad distribution of FDLP materials to a network of libraries, which maintain these collections independently of federal government control and can serve as an integrity check on the system. … [L]ibraries replicate this system in the digital world, ingesting and maintaining independent collections of digital FDLP materials.”

This recognition is the essential first step toward building an infrastructure of digital preservation for the long term that can withstand technological, economic, social, and political stress.

The report has several systemic weaknesses, however, that make its preliminary recommendations either inconsistent or too vague to be useful without explanations, modifications, and surveys of the community. We focus here on things we think need attention.

The Building Blocks, The Roles, The Models

To evaluate the Models presented by Ithaka S+R, one must first understand the relationship between “building blocks,” “roles,” and “models.”
The Models document presents five broad “building blocks” (sometimes called “components” or “categories”) for the program. Each of the building blocks consists of one or more “roles” for individual libraries.

The building blocks are broad categories that cover the service and collection functions of the FDLP. Together, these comprise the mission of the FDLP in aggregate.

The roles are distributed among the building blocks and are designed to give each library an opportunity to choose a role that matches its own level of commitment and participation within each building block. For example, the building block that deals with “tangible collections” has five roles ranging from having no tangible collections at all to preserving a comprehensive collection of all tangible materials.

The document also outlines some Short-Term Changes that will (a) allow Regionals to more easily become Selectives, (b) create a national needs-and-offers process, and (c) make it easier for libraries to deaccession large quantities of documents and others to collect those discarded materials.

The five building blocks and their associated roles are:

Building blocks	Roles
(1) Short-term changes (STC)	Regional role
(2) Information services	Three Service roles (S1-3) and one Training role (Tr)
(3) Digital materials	Four Digital collecting roles (D1-4)
(4) Tangible collections	Five Tangible collecting roles (T1-5)
(5) Page-validated tangible collections	One Page-validated print collecting role (T6)

The document offers five different Models as options; each would implement zero or more of the building blocks. Participating libraries would be expected to choose one role in each building block implemented. Some of the roles require little or no formal activity; some require significant commitment and resources.

Model	Building Block	Roles
0	[none]	[no change in roles]
1	1,2	STC, S1-3, Tr
2	1,2,3	STC, S1-3, Tr, D1-4
3	1,2,3,4	STC, S1-3, Tr, D1-4, T1-5
4	1,2,3,4,5	STC, S1-3, Tr, D1-4, T1-5, T6

This description of different models comprised of Program-oriented broad areas of activity (the building blocks) that are implemented through library-oriented roles is meant to create a sustainable, mission-oriented FDLP with each library fulfilling roles in each area but only at a level of participation to which the library is willing to commit.

Evaluating the Models: What would work and what would not?

It is difficult to evaluate the outcomes of implementing any of the five models because the report focuses more on descriptions of the components than it does on descriptions of the “big picture.” As written, the report makes it relatively easy for any given library to imagine choosing a particular role in any given category, but it makes it relatively difficult to imagine what the program as a whole would look like and whether or not it would be successful or sustainable.

Would the design of the models (i.e., the use of broad, program-level building blocks implemented through library-level participation in specific roles) result in a sustainable mission-oriented FDLP supported by individual libraries participating at locally-sustainable levels? Or would low levels of commitment by many libraries leave an FDLP with few active participants, heavy reliance on a few centralized institutions, and a greatly reduced number of trained staff nation-wide? The report does not address these questions.

The document does not predict the number of libraries that might choose those roles that require active, significant, formal participation. Neither does it predict the numbers of libraries that might choose those roles that require minimal or informal participation with little or no responsibility. It does, however, describe target minimums for some but not all roles (Tr, D3,D4,T3,T4,T5,T6); seven of the fourteen roles have no participation targets at all. Apparently, these minimum targets could result in an FDLP with as few as 20 or 30 active participants — perhaps even fewer.

The report writers do not tell us either what level of participation they would find ideal or what level of participation they anticipate as likely. Neither do they tell us (except for the specified target roles) what level of participation would be unacceptable. While we very much like the idea of flexibility, the draft Models report describes options without defining goals or outcomes and without tying the models to the FDLP Strategic Plan 2009 – 2014 quoted in the draft Directions document. This makes it very difficult to evaluate the potential effectiveness of the report’s recommendations.

In order to fully evaluate the proposed models, the consequences of different levels of participation should be included. Ideally, these would include the intended (ideal), anticipated (likely), and potential (worst case) consequences of different levels of participation in the different roles. These would explain both the benefits and the risks of adopting the models.

There are at least two ways the report could do this. First, it could describe how many libraries it intends to, and anticipates will, choose every role, and then describe the potential consequences of extreme cases of participation and non-participation. Second, it could describe the benefits of each role, not just to the participating library, but also to the FDLP Program, GPO, and, most importantly, to users.

Examining the Ithaka S+R vision

The requirements and options that Ithaka S+R has designed have two recurring themes. First, every model includes roles that will allow libraries to reduce or minimize their responsibilities. In some cases, this is quite explicit:

• “…allowing libraries to take on a services role independent of collections responsibilities.”
• “…enables libraries to take on flexible and minimal responsibilities within the Program.”
• “…a library may take on a relatively minimal role…”
• building digital collections that “have no formal status in securing long-term preservation or integrity.”
• building tangible collections that “have no formal role in the long-term preservation of FDLP materials”
• “we imagine moving into an environment in which some libraries may choose to deaccession large quantities of document[s]”
• allowing libraries to “freely deaccession any tangible materials once they become available in digital form.”
• empowering [Regionals] to “step down to Selective status”

Yes, we have taken the above excerpts out of context. We do so to highlight how Ithaka S+R envisions meeting the FDLP vision. The full context of the above quotes always includes the hope that allowing flexibility of participation will encourage some libraries to “stay in the program,” that minimal participation and responsibility is better than none, and that others will take on more responsibility. This could be one result of adopting one of the models, but there are other possible outcomes as well. What would happen if an inadequate number of libraries stepped up to the increased responsibilities? What would happen if a very large number of libraries decided to “step down” and take fewer or no responsibilities? We do not know from the report as written what the critical numbers are for participation.

Second, we know that the models clearly value two things: a more centralized, big-library approach to preservation of both digital and tangible materials, and a distributed, hierarchical approach to service. In the area of preservation, the report suggests relying almost exclusively on very few participants in significant roles. In the area of service, the report suggests some libraries will be able to do little, some a bit more, and some a lot. These concepts have two things in common: more libraries with fewer responsibilities and fewer libraries with more responsibilities. Will the models work because of this centralized design approach? We do not know. It is, of course, also possible that implementation of the models will result in a different result: perhaps many libraries will take on greater responsibilities and few libraries will “step down” to reduced or minimal responsibilities. But this leaves us with yet another unknown. Will the models work in spite of their centralized design?

What we do know is that the centralized/few-libraries model that Ithaka S+R suggests is a significant change from the decentralized, highly participatory traditional model of the FDLP. It deprecates the roles of most libraries and puts “more eggs in fewer baskets.”

This worries us for two reasons.

First, the more centralized Ithaka S+R approach reflects a hope that a system that consists of fewer institutions with more responsibility will be more sustainable and more effective. In our experience, larger organizations are less flexible and just as vulnerable to the problems of financial stress as smaller libraries. A larger institution may “survive” bad economic times, but at what cost to collections and services? On the other hand, a large system of diverse organizations with smaller, more specific user communities should provide a system that is more flexible (in terms of collections and services) and better able to effectively withstand economic stress than a few big institutions. What are the risks of adopting the Ithaka S+R model and substitute it for the current model? The report does not say.

Second, as noted above, the user perspective is almost completely absent from these models. The earlier Ithaka S+R reports and Findings suggest that users are satisfied with the current, largely disintermediated environment, and the current report does not explain how its models address this. It does not demonstrate that libraries or users will benefit from the hierarchical services-without-collections approach that it advocates.

We believe that many of the ideas in the Models document are good. We love the idea of a national needs-and-offers list. We are extremely happy to see the recognition of the need for multiple digital collections. We like the idea of cooperative arrangements and facilitating the building of collaborative projects. We like the idea of having new, “truly comprehensive” and “page validated “tangible” collections and big, comprehensive digital collections separate from GPO’s.

But we do not like being offered an either/or choice between traditional, distributed, user-focused collections and services, or big, centralized collections and services that minimize the roles of smaller, user-focused libraries. To us, the Models report seems to suggest a “race to the bottom.” It encourages libraries to reduce and minimize their participation and commitment and rely on a few libraries to shoulder the essential roles.
We think that a both/and approach would be better. Such an approach would build on existing strengths of a distributed system. It would encourage greater flexibility and more participation by individual libraries. And, it would offer incentives to build truly comprehensive digital AND “tangible” collections. Such an approach would benefit from the different strengths of distributed and centralized systems and would therefore be more sustainable, more effective, and provide greater benefits to users than either approach alone.

Where is Title 44?

The report does not describe how the “roles” are either consistent or inconsistent with Title 44. For example, role “T1” describes libraries that would not build or maintain any collections of tangible FDLP materials at all. The report does not say if that would require changes to Title 44.

There are several references to roles that will require GPO to select participating libraries and for those libraries to sign memoranda of understanding with GPO and to commit to fulfilling a role for a minimum period of time. Are these anticipated to be in addition to, or instead of, Title 44 commitments? Would GPO be able to select only from a pool of FDLP designated libraries, or would any library be eligible for selection?

Some of the wording in the report implies that libraries that are not current FDLP participants could take on the described roles without becoming designated as a Title 44 FDLP library; (e.g., “any library should be allowed to take on these roles with minimal formalities” and “encouraging current non-participating libraries to take on a services-only role within the Program”). Other wording specifically addresses eligibility for participation in FDLP as a prerequisite to performing a role; (i.e., “[W]ould it be possible for a 501(c)(3) digital library organization that makes freely available digital collections with no traditional, physical collections (such as the Internet Archive) to play a formal role in the FDLP?”). Are the roles described in the report open to FDLP libraries, to non-FDLP libraries, or both? What would be the impact on the outcomes of those different possibilities? Currently, we do not know the answers to these questions.

Where is the evidence?

The Models report builds on the drafts of earlier reports (Background, Environmental Scan, White Paper on Existing Library Networks, Findings, and Direction). Ithaka S+R has said that those were drafts and that final versions would be available later. Unfortunately, in our view, the “Findings” report in particular was incomplete and inadequately supported by the research described in the earlier reports. Since we have not seen a final version of the Findings or modifications of the other research reports, it is difficult to evaluate how closely the Models are justified by the earlier work. In our view, the Models document suffers from (apparently) using those earlier drafts and therefore presenting suggestions that are not supported by Ithaka S+R’s own research.

There is one particularly important example of this. One of the most important assumptions in the report is that libraries will discard print collections and rely on digitized versions of those publications. But the report does not justify digitization as an adequate replacement for discarded paper either in terms of usability or preservation. The report relies on a “slightly modified version” of the operations research model described in Ithaka S+R’s “What to Withdraw” report. It says that this model can be used for government publications, but it does not make it clear why this is a valid assumption. The “What to Withdraw” report is based on a study of digitizing scholarly journals, not government publications. Because government publications have significant differences from the scholarly journal literature, it is unclear that the “What to Withdraw” report is applicable in this instance.

Because the report’s models rely so heavily on digitization of paper, it is essential for the report to provide evidence of the effectiveness of this approach. The report does not deal with the significant differences between government documents and the scholarly journal literature. Specifically: 1) The age of documents may make scanning less accurate; 2) lack adequate bibliographic control especially for pre-1976 documents; 3) publications of odd sizes and formats may be difficult to scan at all; 4) many government documents contain statistical tables which are particularly difficult to accurately scan and convert to text, and 5) many government publications contain images, charts, photographs, and other graphics that are difficult to accurately scan. The “What to Withdraw” report itself notes that “too little is known” about digitizing images and tables and charts. The assumption that government documents can be accurately scanned overlooks (and, perhaps, underestimates) the challenges of digitization. Ithaka S+R puts the digitization cart before the deaccession horse.

We are particularly concerned about the accuracy and usability of statistical tables, which comprise a large and highly utilized segment of government publications. Until we have a verifiably reliable way of guaranteeing that numeric information can be accurately scanned, made human-legible, and OCR’d accurately, paper copies will continue to exceed the value of digitized copies for discovery, access, usability, and preservability. Such guarantees are still rare given the difficult and expensive process of digitizing such publications. OCR’d text is too often found unusable or unreliable and double/triple keying is the norm (For more on this topic, see Julie Linden’s d-lib article “Don’t leave the data in the dark” and Joseph, Lura E., “Image and Figure Quality: A Study of Elsevier’s Earth and Planetary Sciences Electronic Journal Back File Package,” Library Collections, Acquisitions, and Technical Services, 30, 162-168 [doi:10.1016/j.lcats.2006.12.002].)

If the report does not present adequate evidence for this point alone, the rest of the report’s suggestions, which are based on this assumption, become very questionable.

Where are the users?

As noted above, we believe the report would be much more valuable if it spelled out the benefits and risks of different models. In particular, we would like the final report to address how the different models and roles address the needs of and benefits to users.

While the report does deal with some big-issue general needs and implied benefits (e.g., long-term preservation), it either omits or gives little explicit attention to other needs (e.g., long-term free access and usability, discoverability, user-services). If the report addressed user benefits more explicitly, it would give a better sense of what Ithaka S+R is seeking to achieve and how likely it will be that those goals can be achieved.

The absence of a user-focus does more than make the report difficult to evaluate. It also results in omissions of important roles for the FDLP libraries and the community.

For example, the report does not consider the role of smaller collections except to assume that they have no important role in the Program. We suggest that the report should: a) Recognize that smaller, selective collections and larger, comprehensive collections are complementary, not incompatible; and b) Recognize that unique collections will, in some cases, be better able to ensure preservation and facilitate better services than comprehensive collections because the preservation activities and user services of such collections are based on the needs of specific, designated communities rather than on a generic mission to save and provide service for everything to everyone. (A “coordinated comprehensive” collection can too easily miss something that it should preserve simply because the scale of its mission makes 100% success difficult and because it has no designated community informing and verifying its activities — especially given that there is no comprehensive shelf list available to define what “comprehensive” is with any certainty). Such changes to the report would, we think, modify the roles and change the anticipated outcomes and potential consequences of the roles particularly with regard to benefits to users. This could also result in different levels of participation by libraries.

In a similar vein, the report does not suggest a role for what might be called “selective digital depositories.” By omitting this idea, roles D1-4 seem to unnecessarily limit the opportunities for digital preservation and digital service rather than create flexibility. The digital collection roles D1, D2, and D3 could embrace collaborative projects as well as projects of individual libraries. (Imagine, for example, an administrative group of libraries (for example ASERL libraries), a focus grouping such as several law libraries, a regional grouping such as border libraries, a subject or format grouping such as libraries with an interest in the environment, or GIS, or statistics, a grouping of libraries with similar users such as several undergraduate libraries or art libraries, or even children’s or school libraries, etc.)

The report also misses an opportunity to consider the needs of users and the advantages of the program to libraries when it focuses only on “local communities” and “local constituents.” By doing this, the report neglects to recognize the potential advantages of libraries providing services for non-geographically-based communities. While geographically local and nearby services are an important part of libraries’ service profile, in the digital age, libraries need no longer be limited only to geographically local constituents. By broadening their potential user-communities, libraries can at once increase and focus their user base, gain new kinds of support, and identify new sources for funding. By overlooking such opportunities, the report reduces both sustainability and effectiveness of participants.

We suggest that the report explicitly broaden the potential scope of FDLP libraries as each defines its services, collections, and constituents. Use of the terms “local” and “nearby” should be questioned and, when appropriate, changed to include services and users that are geographically nearby, or non-geographically based, or both. The report might give specific examples of how any given library might have several different communities: some geographically local, as well as some based on discipline or subject or agency content or community of interest. The examples above for digital collections would take on a broader context if considered as addressing a community of users rather than just a geographically bound community. Considering non-geographically based communities for services and even training will make the Program stronger and provide more flexibility to libraries while increasing benefits to users. Imagine a library providing services for a community of lawyers or physicians or demographers or even children. Imagine national-level training based for data and statistics, GIS, satellite imagery, API development, database preservation, and so forth.

Similarly, the report repeatedly uses the phrases “local priorities,” “program priorities” and “institutional priorities” but does not balance this focus with the priorities and needs of and benefits to the public at large. The report even suggests that libraries could make digital collections available only “for a local community” rather than “on the open web.” This turns the idea of FDLP on its head. FDLP materials, more so than any other library holdings, have importance beyond the local institution and the report should be more explicit about that. While the behaviors and choices of individual libraries are certainly affected by their own local priorities, it would be misleading to suggest to FDLP libraries that by addressing those local priorities alone they can necessarily address the wider obligation of the Program.

On one last note in this area, consider the report’s suggestion of a national needs and offers list. While we like that suggestion, we wonder why the report does not go beyond the discarding process and embrace better tools for identifying and managing collections for user discovery. The national needs-and-offers lists could be part of a larger project to build a better, more comprehensive and accurate inventory of government information. This could include digital and non-digital information and could facilitate discovery and preservation. With multiple digital collections and a role for the management of a flexible permanent link resolver, a user seeking a particular document should be able to easily get the document from any of a number of depositories without searching multiple catalogs.

In addition to facilitating tools for users and for regionals, the N&O list could be a tool for every library. As we noted in our comments on the draft models document, N&Os could be:

…more than just a shared space to upload N&O spreadsheets. It would need to be a database able to import spreadsheets and other doc formats, sortable by state/library, searchable by agency, connected to each library’s item selection (perhaps via documents data miner), have alerts so e.g. if I want to collect historically or more in-depth in a certain subject or agency, I’d get alerted whenever a new doc fitting that criteria was available. It’d also have to be able to track N&Os so we’d know to which library an offer was sent. In other words, it would need to be a tool for all libraries, not just regionals.

An alternate vision

The Ithaka S+R “Directions” document quotes the vision, mission, and values of the FDLP as described in the “Federal Depository Library Program Strategic Plan, 2009-2014” and says these are “not under reconsideration as part of this project.” Presumably, the Models report should support and further those values, or, at very least, not interfere with their accomplishment. At this point, it is difficult to evaluate the success of the report in meeting those criteria.

Given that, it is worth pausing and imaging a different set of goals — specifically driven to further the FDLP values and mission — and ask how well the report addresses those. This might give us a better “big picture” perspective on the potential outcome of adopting any of the report’s models and help identify gaps in those models.

Here is a list of some goals for a robust 21st century FDLP that we think are important:

• Collections:
• a large number of comprehensive analog/historic collections in various states/regions;
• systematic and distributed preservation of analog AND born-digital documents;
• analog AND digital “collections of excellence” (borrowed from ASERL draft proposal for managing FDLP collections);
• broad and expanded scope of FDLP collection parameters that include databases behind dynamic web sites (not just “documents” but “data”);
• facilitating the building of collections that go beyond the Title 44 scope of FDLP collections: collections that would include agencies and item types not traditionally collected as well as documents from local/regional offices of fed agencies.

• Technical AND public services:
• A documents librarian in every library;
• public service (real and virtual) across the country;
• Robust technical infrastructures to adequately deal with born-digital documents and open the possibility of enhanced services and collections;
• leveraging of the Web for access, services, and information;
• digitization AND enhanced digital access (i.e., machine readable data, georeferenced maps etc).

• Collaboration:
• Collaboration and communication between and among depository libraries, Federal agencies, and GPO;
• National Needs and Offers (N&O) database;
• Creation, maintenance and sharing of robust bibliographic metadata of collections;
• concerted and collective effort to collect fugitives that have historically fallen through the cracks;
• liaison program for all federal agencies as well as their local/regional offices;
• training of new librarians in collections AND services including creation and maintenance of training materials and wiki textbooks.

The proposed Ithaka S+R models address some, but not all, of these goals. The report does not describe adequately how the goals would be met or how many libraries would be needed to meet each one. How do Ithaka S+R’s proposed models meet the stated FDLP mission, values and goals? How many of the above goals would Ithaka S+R’s models address adequately or at all? We do not know.

We believe a vision of the future that encourages more participation (rather than less) and more collections (rather than fewer) with greater diversity of focus on specific user communities would be more sustainable, more effective, and provide more benefits to more users than the current models presented by Ithaka S+R. We should be envisioning an ideal FDLP and then creating the participatory roles to meet that goal. We should be creating space for all kinds and sizes of libraries to participate in a meaningful way. The current report seems to want to save the Program by minimizing and reducing participation. Although that might have the desired effect in the short term, will it build a better, more sustainable FDLP in the long run? Will it provide the benefits that smaller user-focused collections and services can provide? Or will it result in a smaller FDLP with a few monolithic collections and service points, incapable by design of addressing the needs of any but the large aggregate of users?

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.