How did the UO Libraries come to have such a large and comprehensive collection of Oregon newspapers on microfilm? What did it take to preserve thousands of newspapers scattered around the state? Why does it still matter? And how does a 1952 Oldsmobile fit in?
Learn more about the history of the Libraries’ newspaper microfilming program and how it provided the basis for the current Oregon Digital Newspaper Program. ODNP Program Manager Elizabeth Peterson tells the story on the ODNP blog.
Oregon Digital is a digital asset management system which is shared jointly developed by staff at the Oregon State University Libraries and Press and the University of Oregon Libraries. It is built using the Samvera Hyrax digital repository framework, which was chosen (among other reasons) because of its support for Resource Description Framework (RDF) data. Uniform Resource Identifiers (URIs) can be recorded as values in Oregon Digital metadata descriptions, and collection metadata can be exported by administrators in the RDF N-Triples format. In this blog post I will share a look at the RDF metadata description sets which can be exported from Oregon Digital and share two SPARQL queries which can be run on this data.
You might ask why I’d use RDF and SPARQL when Solr queries can be run against all our metadata without any need to generate exports or manage individual RDF files for collections. I see the value of RDF and SPARQL in the ability to make use of other data sources. RDF – a foundational model for sharing linked open data – makes use of unambiguous identifiers for resources of interest, and these resources can be shared and reused across the web. So, for example, SPARQL queries can be run against Oregon Digital RDF and information in other linked open data repositories using federated queries, a powerful extension of the SPARQL query language.
The queries shown below use the SPARQL 1.1 query language. All of the queries and other code snippets shown here, as well as a Jupyter notebook which can be used to run the queries, are available in this GitHub Gist. To run queries on RDF data, a SPARQL endpoint or query interface is needed. The GitHub Gist where these queries are shared includes Python code for running these queries against the data, because this is the method I use. The materials don’t provide a detailed tutorial on setting up the software needed to use SPARQL for querying RDF data, but they do include some information and links to additional resources.
Even though I can include URIs in Oregon Digital metadata and export metadata as RDF, I don’t describe it as linked open data for a few reasons:
Another interesting point for me—someone still relatively new to the Samvera Hyrax user community—is that RDF exported from Oregon Digital seems very “noisy” from the perspective of an end-user interested in descriptive metadata. Many or most of the triples in each description set aren’t descriptive—technical-administrative metadata takes up a lot of space. For example, each resource is classed in six distinct ways (that is, has six distinct values for the rdf:type predicate), as can be seen in the snippet of Oregon Digital RDF available in the online materials—see od_rdf_excerpt.ttl.
To be clear, despite this, I think Hyrax’s implementation of elements of RDF and the ability to record URIs in metadata descriptions have benefits for both users and administrators!
When I’m investigating RDF from a source I’ve never used before, I often download some data in Turtle serialization just to look at it in a text editor. Sometimes I need to convert whatever serialization is available to Turtle—which is much easier for humans to read—myself, but the Python rdflib library and many other tools make this easy to do.
I knew from looking at the data that a portion of subject URIs—the nine-character persistent identifier (PID)—is present in the URL for viewing the object and metadata in a web browser. Viewing metadata in the browser helps me understand how metadata descriptions look and function for a user, so this seems like a valuable piece of information to create from the RDF. The following query can be run against RDF for a collection to yield a title, PID, and detailed web view URL for each object in it.
title_pid_showpage.rq
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX fedrep: <http://fedora.info/definitions/v4/repository#>
PREFIX ldp: <http://www.w3.org/ns/ldp#>
PREFIX pcdm: <http://pcdm.org/models#>
PREFIX dcterms: <http://purl.org/dc/terms/>
SELECT ?title ?pid ?showpage
WHERE {
?iri rdf:type fedrep:Container, fedrep:Resource, pcdm:Object, ldp:Container,
ldp:RDFSource, <http://projecthydra.org/works/models#Work> ;
<info:fedora/fedora-system:def/model#hasModel> ?model ;
dcterms:title ?title .
BIND (REPLACE(str(?iri),
"http://fcrepo\\.od2-prod\\.svc\\.cluster\\.local\\.:8080/fcrepo/rest/prod/([\\S]{2}\\/){4}(\\S{9})",
"$2") AS ?pid)
BIND (CONCAT("https://www.oregondigital.org/concern/", LCASE(?model), "s/", ?pid) AS ?showpage)
}
ORDER BY ?title
LIMIT 5 # (!) delete this line to see all query results
For this to function with RDF coming from a different Samvera Hyrax instance it would be necessary to change at least two components:
Now that I have the query syntax needed to distill a PID and create a web view URL for RDF resources, I can add some search terms to retrieve this information for the objects I’m interested in. In this query, I use the SPARQL UNION keyword to retrieve objects for which metadata contains either a particular string, or one of two subject URIs.
match_on_string_or_uri.rq
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX fedrep: <http://fedora.info/definitions/v4/repository#> PREFIX ldp: <http://www.w3.org/ns/ldp#> PREFIX pcdm: <http://pcdm.org/models#> PREFIX dcterms: <http://purl.org/dc/terms/> SELECT DISTINCT ?title ?pid ?showpage WHERE { ?iri rdf:type fedrep:Container, fedrep:Resource, pcdm:Object, ldp:Container, ldp:RDFSource, <http://projecthydra.org/works/models#Work> ; <info:fedora/fedora-system:def/model#hasModel> ?model ; dcterms:title ?title . {?iri ?p1 ?value1 . FILTER regex(?value1, "quilt(s)?", "i") } UNION {?iri ?p2 ?value2 . FILTER (?value2 IN (<http://id.loc.gov/vocabulary/ethnographicTerms/afset014799>, # quilting <http://id.loc.gov/vocabulary/ethnographicTerms/afset014804> )) } # quilts BIND (REPLACE(str(?iri), "http://fcrepo\\.od2-prod\\.svc\\.cluster\\.local\\.:8080/fcrepo/rest/prod/([\\S]{2}\\/){4}(\\S{9})", "$2") AS ?pid) BIND (CONCAT("https://www.oregondigital.org/concern/", LCASE(?model), "s/", ?pid) AS ?showpage) } ORDER BY ?title LIMIT 5 # (!) delete this line to see all query results
This query doesn’t come close to taking full advantage of SPARQL’s regex function, but this will match on “quilt” or “quilts” and the “i” flag allows matching regardless of capitalization.
This inaugural post describes some of what we do—a lot of different things! We hope you’ll return to learn more in future posts as we dive into different facets of digital libraries in greater detail.
The University of Oregon Libraries Digital Library Services (DLS) department was formed in 2023 as part of a strategic design process at the University of Oregon (UO) Libraries and brings together staff with expertise in multiple areas of digital stewardship to create, provide access to, and preserve digital objects to support learning at UO, in Oregon, and around the world. Our areas of specialization include digitization, digital object curation and management, project management, metadata, accessibility, discoverability, semantic web technologies, information architecture, user experience, digital preservation, copyright and intellectual property, and more. You can see the products of our work in various platforms on the web, whether you are a student or researcher at UO or halfway around the world.
Oregon Digital is a collaboration between the University of Oregon Libraries and Oregon State University Libraries and Press. Comprising more than 500,000 digitized and born-digital objects, this platform is a resource for scholarship and learning in diverse fields. One of the biggest drivers of growth for the UO collections in Oregon Digital is user requests for digitization of materials in the UO Libraries Special Collections and University Archives.
Historic Oregon Newspapers is a free online database where you can search and access the complete content of over 2.4 million pages from 375 Oregon newspapers published 1846 through the present. These newspapers include titles published in cities and small towns from the Oregon coast to the eastern border, at high schools and colleges, and by and for African Americans, Native Americans, migrant workers, labor unions, and many other Oregon communities, and we are continually adding new titles.
The historic newspapers are digitized through the Oregon Digital Newspaper Program, which also provides access to materials through a born-digital preservation program. More than 30 publishers participate in this program at present, and it has made nearly 200,000 pages dating from 2015 available online.
The Oregon Digital Newspaper Program is the newspaper digitization and digital preservation program for the state of Oregon. Based here at the University of Oregon Libraries in Eugene, the program has been digitizing Oregon newspapers since 2009 after receiving funding from the National Endowment for the Humanities and the Library of Congress, and an in-house, self-sustaining digitization program was created in 2015 with funding from the State Library of Oregon. We work with partners across the state including public libraries, historical societies, museums, local groups, and private donors to digitize content with support from private, local, and state-level funding sources.
Over the last five years, our digital newspapers program has garnered over 10 million page views from users around the state, country, and world, completed 65 digitization projects, raised over $115,000 in funding with partners, and added over 80 titles to the Historic Oregon Newspapers platform.
Scholars’ Bank is an open-access digital archive for the scholarly and creative output of the University of Oregon community. Its mission is to preserve and disseminate the intellectual and creative output of UO’s faculty, staff, and students. UO faculty, staff, and graduate students can deposit published and unpublished research and other scholarly output in Scholars’ Bank, and departments can distribute working papers, technical reports, and other research material. If you’d like to share your work, please contact us.
DLS also supports the creation of digital exhibits showcasing unique collections at the University of Oregon, with past support for exhibits funded by the Andrew W. Mellon Foundation and showcasing items from the UO Libraries and the Jordan Schnitzer Museum of Art. Current DLS exhibits focus on fields including sports, religious history, and labor history. We currently use the open-source Spotlight software platform for digital exhibits.
Our digitization services unit creates high-quality digital versions for a wide variety of library resources. These digital assets are published online in platforms like those described here and preserved securely for the long term. In fiscal year 2024, this unit generated more than 50,000 digital images from 56 archival collections and 72 bound volumes, growing collections in Oregon Digital as well as Scholars’ Bank, for which they digitized 49 theses and dissertations which had only been available in print. Many of the digital resources were also directly provided to more than 150 UO Libraries patrons, fulfilling requests for remote access to archival materials. Digitized content is created in compliance with the Federal Agencies Digital Guidelines Initiative four-star performance level and the Library of Congress Recommended Formats Statement.
Thank you to the following people who contributed to this post: