Standards and Protocols for Implementing Digital Libraries

Abstract

The success of a digital library is a function of the combination of technologies’ standards and protocols used to develop it among others. Standards and protocols are used for determining content, defining structure, facilitating discovery and transmission of data, authentication and displaying information in a digital library. Some of the standards and protocols discussed in this chapter include Content standards which are specifications and guidelines for the creation, presentation and delivery of the digital resources in a digital library e.g. PDF, JPEG, TIFF; Metadata standards which present the rules and formats for creating and exchanging metadata among different systems and platforms e.g. Dublin Core, MARC, MODS; and Quality standards i.e. the criteria and measures for evaluating and improving the performance and effectiveness of a digital library e.g. ISO 9001 and ISO 16363. The chapter also discusses web environment and its characteristics as well as the web technologies that make digital library projects successful.

1. Introduction

Territorial boundaries, which hitherto constitute a major hindrance to information access, are gradually fading away due to developments in information and communications technology (ICT), while traditional constraints of time and space are also disappearing with the influx of digital libraries. Today, libraries all over the world are converting to digital libraries to enable them to provide borderless information services to their clients. A digital library is an electronic library that enables the access, storage and dissemination of information in a digitalised form and represents the meeting point of many disciplines and fields, including data management, information systems, information retrieval, library sciences, document management, the web, image processing, artificial intelligence (AI) and human-computer interaction. While there is no single definition for digital libraries, a typical digital library will have content, organisation, service, technology and people as its elements. One of the purposes of a digital library is to bring about the efficient and effective search in a network of organised digital collections; however, in a real digital library, searching is not enough, hence the main activities of users can be classified into five categories: locating and selecting among relevant sources, retrieving information from them, interpreting what was retrieved, managing the filtered-out information locally and sharing results with others [1]. These activities are not necessarily sequential, but are repeated and interleaved.

As a collection of digital resources, digital libraries can be accessed by users across different locations and devices. Some of the benefits of digital libraries include: preserving and disseminating cultural heritage, supporting education and research, and enhancing information literacy and access, just to mention but a few. There are certain standards and guidelines to be followed in order to ensure the quality, interoperability and sustainability of digital libraries. This chapter discusses some of the most important digital library standards and their usefulness in creating and developing a digital library. The chapter also x-rays web environment and the technologies that make digital libraries possible.

2. Digital library and the web environment

The web environment plays a very important role in building a digital library, hence understanding web environment is fundamental to understanding the technologies that work in such environment. Web environment is basically used as a computing platform that provides technical ideals for web technologies to thrive through standards and protocols that have made the web successful. It can also be described as a client/server computing architecture, in which the client (web browser) makes requests to the server (web server) for data. The server-side of the web computing architecture provides web users with access to very powerful computers, while the client-side of the web makes the web pages react to user input instantly [2]. These frameworks enable web technologies (which shall be discussed next) to operate in a seamless manner in the web environment. Web environment is characterised by Client/Server architecture; Server-side computing; Client-side computing; Hyperlinking between documents; Standards for creating web pages (HyperText Markup Language (HTML)) and accessing web pages (HyperText Transfer Protocol (HTTP)) that allow web browsers on any computing platform to use the web; Universality of access provided by the internet and internet naming convention; and the Ability to extend medium beyond HTML using browser plug-ins [3].

There are four basic technical ideals guiding the operation of technologies in the web environment and these include: platform independence, separation of contents from presentation, interoperability and conformity to standards [4]. Platform independence is premised on the emphasis that [5, 6, 7, 8]:

• websites should separate their content from their presentation as much as possible,

• data and functionality provided on website to users should also be available to external web applications preferably via web services interface and when possible,

• websites should conform to applicable standards, whether they are generic web standards like HTML or industry specific standards like OpenURL or SRU/W (library standards).

It is very important to state that building a digital library and integrating contents require a great deal of web design and development skill and competence because basic web pages support text as well as images and may be rendered differently depending on the hardware capabilities and preferences of the user. Digital Library creators and developers must, therefore, be mindful of the need to create and deliver digital libraries that are platform independent, so that library services provided can reach as many users as possible.

3. Overview of standards and protocols

Standard is a formalised protocol accepted by most of the parties that implement it, while protocol defines a set of rules used by two or more parties to interact between themselves. In the context of this chapter, Standards can be described as the protocols used for determining content, defining structure, facilitating discovery and transmission of data, authentication and displaying information in a digital library. Standards can be international i.e. standards set by bodies like the International Standards Organisation (ISO) and Internet Engineering Task Force (IEFT), it can also be national i.e. standards set by bodies like National Information Standards Organisation (NISO) in the United States or British Standards Authority (BSA) in the United Kingdom. Similarly, standards can also be set by a particular industry or by a single company and accepted by widespread usage [9]. Whatever form or dimension they take, standards are necessary for every aspect of the digital library. Some of the major digital library specific standards are discussed in the preceding section. Figure 1 shows the classification of digital library standards.

Figure 1.

Classification of digital library standards.

Each of the digital library standards shown in Figure 2 is described below [10]:

1. Content standards: are the specifications and guidelines for the creation, presentation and delivery of the digital resources in a digital library. They ensure the consistency, quality and usability of the resources, as well as their compatibility with different formats and devices. Content standards can include aspects such as file formats, encoding schemes, resolution, compression, colour, fonts, layout and accessibility. Some of the most common content standards for digital libraries are Portable Document Format (PDF), Joint Photographic Expert Group (JPEG), Tag Image File Format (TIFF), Extensible Markup Language (XML), HTML, Cascading Style Sheet (CSS) and Web Content Accessibility Guidelines (WCAG).

2. Metadata standards: are the rules and formats for creating and exchanging metadata among different systems and platforms. Some of the most widely used metadata standards for digital libraries are Dublin Core, Machine Readable Catalogue (MARC), Metadata Object Description Scheme (MODS), Metadata Encoding and Transmission Standard (METS) and Preservation Metadata Implementation Strategies (PREMIS). They cover different aspects of metadata, such as descriptive, structural, administrative, technical and preservation.

3. Protocol standards: are the rules and methods for the communication and interaction between different systems and components of a digital library. They enable the exchange of data, metadata and commands among different platforms and applications. Protocol standards also facilitate the integration and interoperability of digital libraries with other networks and services, such as the web, the cloud and the semantic web. Some of the most important protocol standards for digital libraries are HTTP, File Transfer Protocol (FTP), Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH), Z39.50, Search/Retrieval via URL (SRU) and Representational State Transfer (REST).

4. Quality standards: are the criteria and measures for evaluating and improving the performance and effectiveness of a digital library. They help to assess the relevance, accuracy, completeness, timeliness, reliability, usability and user satisfaction of the digital resources and services. They also help to identify the strengths, weaknesses, opportunities and threats of the digital library, as well as to plan and implement strategies for its development and enhancement. Some of the most relevant quality standards for digital libraries are ISO 9001, ISO 16363, ISO 16919, DLF DQM and DLF MQF.

5. Policy standards: are the principles and norms that guide the governance and management of a digital library. They define the roles and responsibilities of the stakeholders, such as the owners, creators, providers, users and regulators of the digital resources. They also establish the rules and procedures for the access, use, sharing, licencing, copyright, privacy, security and preservation of the resources. Policy standards can vary depending on the type, scope and purpose of the digital library, as well as the legal and ethical frameworks that apply to it.

6. Ethical standards: are the values and principles that inform the ethical conduct and decision-making of the digital library professionals and users. They reflect the respect for the rights, dignity, diversity and interests of the human and non-human actors involved in the digital library. They also promote the social responsibility, accountability, transparency and integrity of the digital library. Some of the most influential ethical standards for digital libraries are the International Federation of Library Associations and Institutions (IFLA) Code of Ethics, the American Library Association (ALA) Code of Ethics, United Nations Educational, Scientific and Cultural Organisation (UNESCO) Ethical Principles for Library and Information Services and COAR Values Statement (Figure 3).

Figure 2.

Basic HyperText Markup language (HTML) code.

Figure 3.

Output of the HyperText Markup language (HTML) code in Figure 1.

Protocols are predefined rules that govern communicating over a network. These rules must be followed by every system participating on a network. Protocol has been defined as a set of rules or conventions formulated to control the exchange of data between two entities desiring a connection [9]. They are required to define the exchange of control information between user device and the network. The basic elements of a protocol include data format and signal levels, control information coordination and error handling, and timing. Since digital libraries operate on the network, the internet or web to be specific, digital library creators or developers must take into consideration appropriate protocols for such project. Prominent among several protocols is the HyperText Transfer Protocol (HTTP) that defines two main concepts, namely; how client requests are relayed to servers and how servers respond to client requests. The request must specify the following four items [11]:

1. The Uniform Resource Locator (url) for the resource that the client wants.

2. A method, such as GET (which is used to fetch data from the server) or POST (which is used to send data that need to be updated on the server).

3. A list of headers (Request headers may contain miscellaneous information about the client or the resource that they have requested, such as the client browser name, operating system, message size, time and date of request, access control information/credentials, etc.).

4. The body which contains the information that the client wants to send to the server.

Some of these standards and protocols are specifically discussed in this section. Table 1 shows a list of some common web protocols and their uses.

Protocol		Use
Transmission Control Protocol	TCP	For establishing a connection between two devices on the web and managing the delivery of data packets
Internet Protocol	IP	For transferring data packets between two devices on the web
User Datagram Protocol	UDP	For establishing a connection between two devices on the web
File Transfer Protocol	FTP	For transferring files between the client and the server
Simple Mail Transfer Protocol	SMTP	For sending e-mail messages to the server
Post Office Protocol	POP	For fetching e-mail messages from the server
Simple Object Access Protocol	SOAP	For sending XML messages via HTTP

Table 1.

Common web protocols and their uses.

4. Web technologies for digital library implementation

Web technologies are the various tools and techniques that are utilised in the process of communication between different types of devices over the internet. It can also be defined as methods by which computers communicate with each other through the use of markup languages and multimedia packages [12]. Digital Library developers have utilised these technologies to build successful digital library projects that have stood the test of time for content integration service delivery. For instance, most of the successful digital and virtual libraries are built with combination of these web technologies and the same can also be said of engaging the library websites deployed by so many libraries around the world. Some of these web technologies and what they can be used to achieve are discussed below;

HyperText Markup Language (HTML): it is the web technology that describes the conceptual part of a web document or page i.e. the title, headings, lists, paragraphs, etc. It is a client-side language that is used to code the frontend of a website. It helps developers and content integrators to define how the webpage will be structured, and it does so with the help of elements that are identified by tags. Below is a very basic example of a piece of code written in HTML:

Each item enclosed in the <> brackets is an element. DOCTYPE, however, is just a declaration which communicates to the browser that what it is about to load is an HTML document. The <body> element defines all the content that forms the body of the document. <h1> and <p> indicate the start of a heading and a paragraph, respectively. </body>, </h> and </p> are closing tags. They indicate where the body, heading and paragraph end.

In this example, the style attribute is used to assign a particular background colour, font colour, family and size to the text. It can be extended further to include font style (bold, italic, etc.) and alignment (centre, left, etc.). Other attributes apart from style can also be used to modify elements in a similar manner. A solid background and knowledge of HTML and its variants remains an indispensable skill for digital library creators or developers.

Cascading Style Sheet (CSS): is one of the most fundamental web technologies used basically to beautify the appearance or look of a web page. To further beautify library contents on portals or websites, digital library creators or developers can integrate CSS into HTML code. Using CSS provides an elegant way to change the fonts, colours and layout of a website from a central location. It also allows more flexibility for displaying a web page on the part of the web browser. Instead of applying certain styles to certain elements of HTML code repeatedly, CSS can be used to streamline the process of styling pages across the entire site to save time and energy. Once a style sheet is prepared with all the default font styles, colours and other characteristics related to the overall layout of a webpage, it can thus be applied site-wide. Recently, CSS has also been combined with Bootstrap to design mobile-first web apps.

Web browser: a browser is an application used to access and view websites. It is a software program used to present and explore content on the World Wide Web (WWW). These pieces of content usually include pictures, videos and web pages and are connected using hyperlinks and classified with Uniform Resource Identifiers (URIs) [13, 14]. Web browsers are basically computer programs that make it possible for people to view all the resources that are part of the web. They are based on client-server architecture. Whenever a user enters a URL into a browser’s address bar, it relays the request to the server and then fetches and displays whatever the user requested for. Some popular web browsers include: Opera, Mozilla Firefox, Google Chrome and Safari. The two distinct parts of a website are the frontend and the backend. While the frontend refers to all those parts of a website that a user can see on their screen and interact with, the backend refers to the hidden mechanisms that make a webpage function. Typically, a user is generally unaware of what goes on at the backend. Table 2 gives a list of the two distinct parts of a website.

Frontend	Backend
Client-side	Server-side
Website design	Databases
UI/UX	Servers
Some UI technologies:	Some backend technologies:
HTML	PHP
CSS	Java
JavaScript	Python
AJAX	Ruby
	NET

Table 2.

Overview of the two distinct parts of a website.

Web and proxy servers: Web servers are computer programs that run on network operating systems which make web browsing possible. In a lay analogy, web servers can be described as the pots that serve plates requesting for meal. For instance, when users request web page from a web browser, the webserver delivers that file to them over the internet. In addition to the aforementioned basic function, the webserver organises files on a website; makes redirection from one url to another possible; provides support for uploading and modifying files; encryption of highly sensitive data; customisation of error messages; support for special webserver extensions or modules; authentication to sensitive areas of a website as well as support for web scripting languages such as PhP and ColdFusion. Library web servers are specially configured to authenticated users, as they access various areas of the virtual library and run many applications that are crucial to the delivery of services and digital resources [15].

Proxy server is a web server that acts as intermediary between the web browser and the World Wide Web (www). Proxy server can be defined as a validation tool that uses the Internet Protocol (IP) address of a computer or server at the library to serve as an intermediary between the remote users and the database servers that the library makes available [16]. Proxy servers can be used by library content integrators as well as digital library developers to provide remote access to IP restricted resources. They can also be used to filter access to the web in public libraries and provide a flexibility in logging activity on a library’s website. After the proxy server authenticates and authorises the user, it retrieves the restricted resources using the server’s IP address and sends it to the user who is browsing from an unauthenticated IP address. In the past, library patrons had to specially configure their web browser to use proxy servers, but URL-rewriting proxy servers, such as EZproxy that require no browser configuration, have now become the norm. In addition to providing remote access to scholarly information, proxy servers such as EZproxy also enrich the relationship between libraries and their users through their ability to assist in data collection that can be utilised in a variety of ways [17].

Relational database: A database is an organised collection of structured information or data stored electronically in a computer system. A relational database is a digital database that relies on the relational model of data [18]. The majority of web applications depend largely on relational databases for organisation, storage and retrieval of data using the Structured Query Language (SQL) for querying and maintaining the database. Relational database management systems (RDBMs) are available in several proprietary and open source manifestations. Oracle and Microsoft SQL Server are common propriety RDBMSs. MySQL and PostgreSQL are the most popular open source RDBMSs.

Common web applications, such as content management systems, wikis, and blogs, often keep their data in relational databases. Digital library applications, such as portals and digital collection systems, store metadata about the resources that they manage in relational databases.

5. Library specific standards and protocols

This section discusses some library specific standards and protocols vis-à-vis how they can and have been used to successfully deploy digital libraries, library websites, library portals and integrated library systems as the case may be Suffice to state that digital libraries are made possible by a combination or mix of numerous technologies supported by different standards and protocols to run seamlessly. Below are some of the library specific standards behind most success digital libraries, library portals, websites and integrated library systems;

1. Z39.50: Z39.50 is an international standard client-server and application layer communications protocol for searching and retrieving information from a database. It is a robust search and retrieve protocol that supports a rich set of functions, such as complex Boolean queries, results set that can be further searched, viewing the status of searches in progress, viewing holdings’ data, downloading records in different formats and authenticating users. Z39.50 is maintained by the Library of Congress Z39.50 is a complex standard and was not originally intended to run over TCP/IP (the protocol that runs the Internet).

2. SRU/W: The Search and Retrieve Web service (SRW) and Search Retrieve URL service (SRU) are collectively referred to as SRU/W. SRU/W was developed by the Z39.50 maintenance agency at the Library of Congress to address the shortcomings of Z39.50. Although the service differs from Z39.50 in a number of ways, SRU/W incorporates the lessons learned from Z39.50 to provide a powerful search and retrieve protocol specifically designed to work in a web environment. SRU/W relies on common web communication protocols and is much simpler to implement than Z39.50. The primary difference between SRW and SRU is that SRW communicates between the client and server using SOAP messages, while SRU uses a URL (i.e. an HTTP GET request) to issue a query and the response is in XML.

3. OpenSearch: OpenSearch is a standard used for finding materials that match a particular information needed by allowing clients to ask a server how queries can be constructed and passing searches using URLs. OpenSearch is similar to SRU but known for its simplicity to implement and returns the results in RSS or atom that can be read with a newsreader. A distinguishing feature of OpenSearch above SRU is that the former is intended for any kind of media, including still images, audio and video.

4. Open Archives Initiative: The Open Archives Initiative (OAI) is a standard for web content interoperability designed to aid and provide access to multiple digital collections by providing a framework that allows libraries and other service providers to harvest metadata about collections according to rules specified in the Protocol for Metadata harvesting (PMH). OAI is not designed to allow users to interactively search for items that meet a specific information need. Rather, it only provides a simple framework that allows organisations to download batches of records from an archive. Sufficing to state that despite its name, OAI only provides access to metadata about resources it does not provide the resources themselves.

5. OpenURL: OpenURL is a protocol that refers to an article, book or other resource rather than the physical location of the item on the Web. It was developed by the National Information Standards Organisation (NISO) for use in libraries and other repositories of information. It was turned over to the Online Computer Library Center (OCLC), Dublin, in 2006. Like SRU, OpenURL is a simple protocol that uses a URL to request items. Unlike SRU, the purpose of OpenURL is to supply information for a specific item, while SRU is designed to request an unknown number of items to meet an information need. OpenURL has widespread support in the vendor community and is used for tasks ranging from providing links in citation database, allowing users to request items via interlibrary loan with a single click, to connecting from catalogue records to articles in full text database. In essence, OpenURL defines a few fields that transmit bibliographic information to a web server. Open URL’s success is largely a function of its simplicity. However, it can only be used to transmit limited bibliographic information, though the standard is extensible and by transmitting a digital object identifier one can theoretically access any useful information about a resource and connect to it with proper access permissions. Programs that direct OpenURL requests can be directed to appropriate services (known as OpenURL resolvers) and can readily be purchased, though libraries often develop their own. OpenURL is so simple that a basic resolver can be designed in as little as a few hours.

6. Digital Object Identifiers: Digital object identifiers (DOIs) are strings of case insensitive characters that uniquely identify a resource online. It is a permanent identifier of a web document which consists of two parts: the prefix and the suffix. All resources, which are assigned DOIs by an agency, have the same prefix but the prefix does not change if ownership of the resource is transferred to another agency. All DOI prefixes begin with ‘10’, hence a typical DOI will look like this: 10.23456/1234567890ABCDEFG. It is important to note that DOIs and OpenURL are complementary technologies that can be used in conjunction with a local resolution services to allow users to access a resource. For example, a DOI could be embedded in an Open URL request and, similarly, a DOI resolution service could use a DOI to generate an OpenURL request to a library’s interlibrary loan system or a vendor database.

7. Lightweight Directory Access Protocol (LDAP): There are many authentication protocols, but LDAP is one of the best supported methods to control access. As its name implies, LDAP is a protocol that specifies how directory information can be queried and manipulated. LDAP is essentially a simple version of the complex X500 directory service (hence the ‘Lightweight’ component in the acronym LDAP). LDAP has become the standard for providing e-mail, phone and directory information. That main function of LDAP is to authenticate users.

8. Shibboleth: Shibboleth is an authentication standard that allows multiple institutions to share authentication information without revealing personal information. It allows users to access resources maintained by other institutions without the need for a different username or password or a proxy server and protects the anonymity of users preserving both privacy and reducing the risk of identity theft. Shibboleth works when a group of institutions agree to grant access to each other’s resources based on user attributes rather than personal information. When a restricted access website needs authentication information, it goes to Shibboleth, which then redirects the request to the user’s home institution. The home institution asks for a user ID and password, and Shibboleth generates a temporary name for the user. The restricted access website does not know who the temporary name represents, only that it was issued by a trusted institution. The website then asks the home institution for the unknown person’s attributes. If they are acceptable, access is provided. Although Shibboleth is designed to be relatively easy to implement, it has relatively little support from the vendor and library communities, and maximum interest in Shibboleth is from among a relatively small number of major academic institutions.

In summary, a digital library is a mix or a combination of different evolving technologies for seamless information service delivery. Figure 4 shows the combination of technologies that make up a digital library.

Figure 4.

Combination of technologies that make up digital libraries.

6. Conclusion

In this chapter, the importance of standards and protocols for implementing digital library was discussed. Standards and protocols ensure the effective and efficient utilisation of networked information resources, bibliographic description and interchange of cataloguing data as well as dictating the interplay in a networked information environment. It was established that the web environment also plays a significant role in building a digital library, hence there is the need to understand how the web environment works. A successful digital library would require skill in some of the technologies discussed in the chapter. For library managers and those at the management levels, an understanding of these standards and protocols (technologies) will guide and influence decision in favour of promising digital library projects. Stakeholders in Library and Information Services domain including content integrators and managers must understand the working of these technologies to enable them meet the information needs of users.