7.x to CLAW Migration Sprint - Complete!

The Islandora community has just wrapped up a very successful sprint dedicated to migrating from 7.x to Islandora CLAW. We at the Islandora Foundation want to give a big thanks to everyone who put in time during this sprint, as well as the organizations who lent us their talent on the company dime. We also want to give a special shout out to the Metadata Interest Group, who collectively put in a ton of time and tackled some intense questions for those who want to use a migration to Islandora CLAW as a chance to do metadata cleanup. During the course of two weeks, we managed to accomplish a lot. As of right now you can:
  1. Migrate over objects based on content type
  2. Migrate ALL the datastreams (except AUDIT, which is a special case)
  3. Extract metadata from any XML datastream and make it a Drupal field
  4. Model authorities such as people, organizations, and subjects
  5. Convert MODS to CSV using Cara Key's (LSU) XML2CSV tool
There's still some work left to do, though. On the horizon for the near term, be on the look out for:
  1. Migrating the AUDIT datastream
  2. Modeling more/different types of authorities
  3. Examples of extracting authorities from FOXML
  4. A workflow for those who want to use OpenRefine to reconcile linked data authorities during the migration process
Moving forward, this is an excellent chance for people to try out the tools we're developing and point them at their existing repositories. Our migration tool, originally developed by Jared Whiklo (University of Manitoba), is available on Github. And if you want to give modeling authorities a go, check out our new controlled_access_terms module, which was made by Seth Shaw (University of Nevada Las Vegas). If anyone has feedback/issues/questions, please feel free to create an issue or post a message on the mailing list. Here's a full list of all the people and organizations who helped make this once-considered-impossible feat a reality:
  • Benjamin Rosner - Barnard Collge, CU
  • Pat Dunlavey - Born-Digital
  • Andrija Sagic - Library "Milutin Bojic"
  • Ann McShane - Library Company of Philadelphia
  • Cara Key - Louisiana State University
  • Jason Peak - Louisiana State University
  • Jonathan Green - LYRASIS
  • Rachel Leach - Mount Holyoke College
  • Mark Jordan - Simon Fraser University
  • Adam Soroka - Smithsonian Institution
  • Rachel Tillay - Tulane University
  • Pete Clarke - University College Dublin
  • Jared Whiklo - University of Manitoba
  • Mike Bolam - University of Pittsburgh
  • Seth Shaw - University of Nevada Las Vegas
  • Paul Pound - University of Prince Edward Island
  • Rosie Le Faive - University of Prince Edward Island
  • Nat Kanthan - University of Toronto Scarborough
  • Marcus Barnes - University of Toronto Scarborough
  • Carolyn Moritz - Vassar College
Thanks to everyone involved! And if you missed out on this sprint, don't fret. We'll be holding another Islandora CLAW community sprint later this year after Islandora 7.x-1.12 is released.

iCampCA Instructors Announced

Islandora Camp is going to be in San Diego, CA from November 7 to 9th. We'll be holding our standard three day camp, with two days of sessions and one day of hands-on training from experienced Islandora instructors. You can register for the camp and find out more here. Both workshop tracks will contain content exploring Islandora's next major version, which pairs Drupal 8 and Fedora 4, alongside more traditional training in the current release with Drupal 7 and Fedora 3. We are very pleased to announce that the instructors for our training workshop will be:

Admin Track

Melissa Anez has been working with Islandora since 2012 and has been the Community and Project Manager of the Islandora Foundation since it was founded in 2013. She has been a frequent instructor in the Admin Track and developed much of the curriculum, refining it with each new Camp. Lately she has been expanding the iCamp workshop to cover new ground with Islandora CLAW.

Katie Romabiles began working with Islandora in 2017 when she took on the new position of Institutional Repository Specialist for San Diego State University. Since then, she has helped with the migration of the university's IR from DSpace to Islandora and has developed new repository collections in collaboration with SDSU faculty and staff. She attended iCampEU in 2018, and is excited to give back to the Islandora community by acting as host and support instructor for iCampCA.


Developer Track

Jared Whiklo began working with Islandora in 2012. After stumbling and learning for a year, he began to give back to the community in late 2013. He has since assisted in both Islandora and Fedora releases and (to his own disbelief) has become an Islandora 7.x-1.x, Islandora CLAW, and Fedora committer. His day job is Developer with Digital Initiatives at the University of Manitoba Libraries. His night job is at the Kwik-E-Mart.
Mark Jordan has taught at three other Islandora Camps and at the Islandora Conference. He is the developer of numerous custom Islandora modules, including Islandora Context, Islandora Themekey, Islandora Datastream CRUD, and the XML Solution Pack. Mark is also one of the co-developers of the the Move to Islandora Kit. He is also an Islandora committer and is currently serving as Chair of the Islandora Foundation Board of Directors. His day job is as Head of Library Systems at Simon Fraser University.

Islandora 250

The Islandora Installations list has hit 250* known sites! Let's have a look at some of the more interesting examples that brought us to this milestone:

California Revealed

California Revealed is a State Library initiative to help California’s public libraries, in partnership with other local heritage groups, digitize, preserve, and serve online historically significant Californiana (e.g., books, documents, audiovisual recordings).

Colleges Object Repository (CORe)

This eight-instutiton multi-site is a collaborative initiative aimed at creating access to unique digital content by Ontario colleges.

Frederick Douglass Papers

Managed by Indiana University-Purdue University Indianapolis (IUPUI), the Frederick Douglass Papers collects, edits, and publishes in books and online the speeches, letters, autobiographies, and other writings of abolitionist, statesan, and renowned orator Frederick Douglass.

Nova Scotia College of Art & Design (NSCAD)

Part of the CAUL-CBUA Atlantic Islandora Repository Network (CAIRN) multi-site, this repository features items of importance to the history of NSCAD University, including a collection of digitized scrapbooks containing NSCAD memorabilia dating back to 1921, a collection of books authored by past principal Elizabeth Styring Nutt, a digitized audio archive of lectures, performances, and sound art, and a collection of digital theses by MFA, MDes, and MA graduates dating from 1975 to present.

San Diego State University

The host of our next Islandora Camp digital collections, SDSU's Islandora repository covers theses, syllabi, and other materials produced by students, faculty, and staff.


Is your site on our list? Drop us a line to be added or updated.

* Yep, it's actually 254. Just like last time, the list grows  faster than the blog updates

Islandora and the COAR Next Generation Repositories Report

Late last year, a working group of the Confederation of Open Access Repositories (COAR) released a report with recommendations to adopt "new technologies, standards, and protocols that will help repositories become more integrated into the web environment and enable them ​to ​play ​a ​larger ​role ​in ​the ​scholarly ​communication ​ecosystem." Islandora's own Institutional Repository Interest Group took up the report and measured Islandora against it, looking at both the current functionality available in Islandora 7.x, and how we can best shape Islandora CLAW to meet these recommendations for the future (complete with issues in the CLAW GitHub so we can track our progress). They have shared their own results, written up by convenor Bryan Brown:


#1: Exposing Identifiers

The brunt of the recommendation here seems to be implementing best practices listed at http://signposting.org/ regarding typed HTTP links. I’m not sure what Islandora 7.x is doing in terms of typed HTTP links, but I’m assuming nothing beyond whatever Drupal 7 does by default. It could certainly be doing more, but there’s a lot to chew on in the best practices in terms of deciding what actually needs to be done, and how this should be done for different types of objects. CLAW, being a linked data application that operates primarily via HTTP, should definitely be doing these things. I’ve made a use case for this at https://github.com/Islandora-CLAW/CLAW/issues/860.


#2: Declaring ​Licenses ​at ​a ​Resource ​Level

Very similar to Behavior #1 (Exposing Identifiers), this recommends using best practices from http://signposting.org/ to use typed HTTP links to expose the URI for the license that best describes a resource. Good in theory, but not all licenses have machine-readable URIs, and would require either migrating existing free-text licenses to ones that have a URI, or in the case of special one-off licenses, creating URIs for local licenses (which wouldn’t be very interoperable). COAR recommends using Creative Commons licenses since they have readily available URIs, but CC licenses aren’t really a good fit for scholarly works since publishing introduces a lot of issues that CC licenses don’t cover. As for the human readable part, that’s just a matter of your metadata and your theming. 7.x and CLAW both should be able to display human-readable rights statements, but neither can do the HTTP link part currently. CLAW use case at https://github.com/Islandora-CLAW/CLAW/issues/860.


#3: Discovery ​through ​Navigation

Even more emphasis on using the best practices at http://signposting.org/. 7.x’s Islandora Google Scholar module adds a link to the PDF for citation/thesis objects as an HTML meta tag, but that’s it. Its easy to see how adding this as a typed HTTP link, especially for compound objects would be helpful to let a machine know about the different parts of a larger meta-object. This feature would be nice for 7.x, but as a Linked Data Application CLAW should definitely have it. Covered again by https://github.com/Islandora-CLAW/CLAW/issues/860.


#4: Interacting ​with ​Resources ​(Annotation, ​Commentary ​and ​Review)

Members of the IR IG are not sold on this one for use in university IRs. Perhaps there are very specific types of repo systems where peer review, comments and annotations are useful, perhaps for aggregators or publishing platforms. In a university IR, it seems like it could actually hinder adoption because faculty might not want folks interacting with their scholarship, and would request mediation for such things which would slow down already overworked IR staff. Drupal already has tons of modules for things like this, so you could probably modify one to work with Islandora objects in 7.x, and in CLAW you wouldn’t even have to write any code, just turn the module on and configure it. Turning those annotations into linked data on the object would be a bit more difficult, but that difficulty would be more in deciding how the metadata should look than how to implement.


#5: Resource ​Transfer

This seems to be suggesting a modern form of OAI-PMH, but in a way that includes assets in the transfer. Strong recommendation for ResourceSync, which we have no experience with, but looks like it would do the job. 7.x will probably never have this, but CLAW should focus on it. Use case at https://github.com/Islandora-CLAW/CLAW/issues/857.


#6: Batch ​Discovery

We aren’t really not sure how this differs from Behavior #5 (Resource Transfer) since this seems to be a use case where someone used “Resource Transfer” technology to put all of your repo’s stuff in an aggregator so that it could be found in multiple places. You take care of #5, you already take care of #6. Covered by use case https://github.com/Islandora-CLAW/CLAW/issues/857.


#7: Collecting ​and ​Exposing ​Activities

This seems to be a mash-up of #4 and #5: capture interactions, turn them into metadata that you expose, and then push that metadata along with the rest of your data with ResourceSync. There are a LOT of recommendations for possible ways to do this, which underscores the fact that there’s not a clear standard for this and probably not a lot of consumers for this kind of data either. This seems like a “nice to have”, not a “have to have”.


#8: Identification ​of ​Users

This seems like a good idea, and ORCID seems like the obvious best choice in a scholarly context. We don’t know much about the other two ID systems involved (Social Network Identities and WebID), perhaps they would be good for folks who don’t have an ORCID, but then again perhaps this could be a good way to get people to use/understand ORCID. Use of ORCID could potentially lock out non-academic users, which may be a bug or a feature depending on your goals. Whichever you pick, the problem is going to be getting something that people use across the web in order to deliver on the promises outlined in this section. In an age where people are wary about privacy and the web knowing too much about you, we don’t think this one would get as much broad adoption as COAR thinks.


#9: Authentication ​of ​Users

We don’t understand how this is different from #8, it seems like the two go together to such a degree that separating them is only confusing.


#10: Exposing ​Standardized ​Usage ​Metrics

This is a nice dream, but much harder than it sounds. Current generation repositories are pretty close to doing all they can in terms of capturing views/downloads on objects, although client-side triggers are better than server-side ones in order to avoid problems with caching, and Piwik seems to be a winner in the international community due to its focus on privacy and flexibility (although it does require setting up your own Piwik server). Standardizing the way usage stats are exposed from the same repo is a good idea as well, but none of us have experience with SUSHI or COUNTER.

All this can be done to perfect aggregation of usage stats on the same repo, but aggregating/summing stats from external sources is not going to be a practical option until there is a centralized source that does this with a solid API.


#11: Preserving ​Resources

While we agree with the sentiment here, we’re not sure they are saying anything new. Fedora should take care of the actual preservation bits, and Islandora has always requested least-common-denominator open format file types for archival master datastreams and used derivative processes to spin into other formats.

New Member: Amherst College

The Islandora Foundation is very pleased to welcome our newest member: Amherst College. Currently the owners of a custom Fedora 3 digital collection (ACDC), the team at Amherst College are explorers on the bleeding edge of Islandora, working with Islandora CLAW and Fedora 4. They have played an active role in development and discussions for the future of Islandora and we look forward to working more closely with them as members of the Islandora Foundation.

Islandora CLAW modules on Drupal.org

I've taken the liberty of putting CLAW's Drupal modules on drupal.org as sandbox projects.  It is my intention to promote these to full projects once CLAW is released so that our modules can be distributed through drupal.org and made available under the 'drupal' namespace on Packagist.  We've always been on the sidelines of the Drupal community, and this feels like a step in the right direction.  Not only will our modules be available somewhere other than just Github, but Islandora will also get exposure to the wider Drupal community.

This does not mean that we're adopting Drupal's workflow, as CLAW encompasses more than just Drupal modules.  As of now, there will be no impact on day to day development, which will continue as-is on Github.  However, the subtleties of its inclusion in the release process will need to get discussed and ironed out as we work through our initial release.

They're not much to look at, but here are the links if you're interested:

Islandora Show & Tell: Story City by Vancouver Public Library

This week we have a special two-part Show & Tell, looking at the remarkable Story City from Vancouver Public Library from the point of view of both the library behind it and the service company that helped with development.

Story City tells the stories of cultural life of the citizens of Vancouver and surrounding areas, through an extensive collection of audio recordings, photographs, videos, and scanned historical documents. Because these stories are built around the places where they happened, VPL decided to use a map interface as the point of entry for browsing these repository objects.

A map, plus a whole lot more. Story City allows visitors to filter stories on the map by neighbourhood, community, time period, and format, with visual markers for different kinds of content. Grouping pins keeps the interface neat in the face of multiple objects, and the power of Solr lets visitors share a ‘canned search’ with terms built into the url, such as showing just audio files featuring Chinese Canadians from 1971 - 2020.

The interface is sleek and intuitive, with viewers available for digital objects right on the map, so you can view an image or play a story about growing up in the West End just by clicking on a pin.

I spoke with Kay Cahill, Acting Director of Collections & Technology at VPL, about their Islandora Site This Vancouver, and the new Story City feature that prompted this Show & Tell:

Why did you choose Islandora?

We chose Islandora for our repository, This Vancouver, following a comprehensive evaluation of a variety of commercial and open source products. Islandora offered all of the features and flexibility that we were looking for, and we felt there was real value in choosing an open source product that would allow us to develop in-house expertise in configuration and development. Additionally, the concept of open source supports the value of resource sharing which is core to our mission as a public library. Undoubtedly there was a bit of an up-front staffing cost in the time it took to install and learn the software, but it has proven to be a robust tool that has served us well as we have developed our community collections.

We chose to develop the Story City site as an Islandora module for a number of reasons. Firstly, we wanted to build on the internal expertise we had developed over three years of working with Islandora for This Vancouver. Secondly, we wanted the map – which was developed through a Canada 150+ grant to showcase stories of journeys to and interactions with Vancouver - to serve as a discovery tool for the content in the broader repository. And thirdly, we wanted to develop something that could be released back to the open source community so that other organizations and individuals would be able to make use of it in the future.

What feature in your repository are you most proud of?

We are particularly proud of the Story City map. Digital Echidna did fantastic work bringing to life our vision of a map that would be fast, responsive, visually appealing, and provide a compelling discovery tool for the amazing content that we gathered from community members over the course of the project. We particularly like the way that pin clusters break into a spiral when a number of stories are linked to the same location on the map; this is a really user-friendly and effective presentation of something that could have been very confusing.

Do you have plans to expand your site in the future?

Absolutely. Providing venues for community stories to be shared, discovered and explored is a core part of VPL’s current strategic plan. The stories that we’ve gathered and shared through This Vancouver and Story City are a testament to how technology, which so often leads to an increased sense of isolation, can spark new dialogues and help build connections and understanding between community members. Our Community Digital Initiatives team continues to work on identifying new content for the repository,  and on sharing the stories and experiences of Vancouverites through this platform. Because we collect and curate new content for each project, there’s a lot of work that happens behind the scenes: developing community relationships, interviewing community members, and then editing the resulting files prior to adding them to the collection. To date we have mainly focused on audio recordings, but we are hoping to introduce our first video collection later this year.

What is your favourite object in the collection to show off?

I think I’d like to reframe this question slightly, and speak to the item that I think is the most important and impactful in the collection – the Women’s Memorial March Quilt. This, to me, is the perfect example of why we put the work into building our repository and developing tools like the map to make the content easier to discover and explore. The quilt was created to commemorate the murdered and missing women of Vancouver’s Downtown Eastside. Made by friends and family of the women, it’s carried through the streets once a year at the annual Women’s Memorial March. It is incredibly important to the community as one of the few places where the women, who are so often objectified by the press as drug addicts or sex workers, are remembered as their loved ones knew them: mothers, sisters, daughters, friends. By capturing the panels of the quilt in high-quality digital images and sharing them through the repository, they are not only preserved for the future but are available for anyone with access to the internet to view online.

Digital Echidna the company that worked with VPL to create the site,  shared further details about developing Story City and the Islandora Story Map module, which is available on GitHub:

Because this collection centres around places and the movement of people as it shapes a city over time, a map was decided upon to serve as the interface for the Islandora repository. This is where discovery is centered and is the landing page of the website. Pins and other visual identifiers on the map indicate the presence of objects -- audio recordings, photographs, video clips and scanned images -- that are in the digital asset collection.

The website has social sharing functionality and operates as an attractive and responsive online experience that is branded with Vancouver Public Library’s existing web presence. Users of the site can navigate both visually using pins on the map and with search facets, such as neighbourhood, background, and era filters. The site’s administrators can manage these assets and add new objects. Training, knowledge-sharing, and open communication were key components to this successful project.

Digital Echidna’s Role

In order for the mapping feature to work, we developed a new Islandora module (story_map). This module takes Islandora Content Objects and pins them to a map using the data from the Location fields within said Objects. Content Objects need to be geotagged with the

appropropriate latitude and longitude coordinates, as well as with the associated metadata and tags for search facets. The interface handles aggregation of multiple pins into clusters, which drills down to a more detailed view when the user clicks.

This is a very visual-dependent module. The pins for different types of content are visually distinct; when the user clicks on an individual pin, a high level description will be offered. Another click will open the media for the object in the appropriate viewer/player, lead to a more in-depth description, and easily return to the map to explore other pins or to explore another search facet.

Key Factors of Success

  • Taking Solr data and displaying it on a map
  • Map solution done with IP Geolocation Views & Maps and Leaflet. The IP Geolocation (a.k.a the mapping engine) takes the data from Islandora Solr Views and displays it on a Leaflet map (a.k.a the map renderer)
  • Displaying media specific pins that lead to pop ups that show the media in the appropriate viewer/player
  • Customizing Views plugins
  • Custom JS social sharing
    • Filter sharing using the Views AJAX History module
    • Popup sharing

Has your institution built an Islandora site or tool that you want to share with our community? Contact us to set up a Show & Tell of your own!

Islandora Camp in San Diego, CA

The last Islandora Camp of 2018 will be in sunny San Diego, CA from November 7 to 9, hosted by San Diego State University. Islandora Camp features:

  • Day One: General and introductory sessions about the softare and the community that uses and supports it
  • Day Two: Hand-on workshop training with tracks for Developers and front-end Admintrative users, featuring both Islandora 7.x and Islandora CLAW
  • Day Three: Sessions on specific sites, tools, and topics of interest to Islandora users

Registration will open soon at our usual USD rates of $450 Early Bird and $495 Regular, so stay tuned and save the date. #iCampCA


Looking back at Islandora Camp Halifax

The Islandora community wrapped up our first Nova Scotian iCamp last week, having spent three days learning and sharing with this amazing Mount Saint Vincent University view as a backdrop:

We opened bit with an update about the state of Islandora CLAW and all of the features it already has in place as it races towards a first release:

Followed by a day of hands-on workshop training. The Admin Track spent most of their day building an Islandora 7.x site from scratch, and we ended the track by debuting the first Islandora CLAW workshop and switching to build mode in the latest software, making objects and collections with CLAW's more Drupal-y interface. The Developer Track did their workshop Choose-Your-Own-Adventure style, building the lesson through questions and requests from the group.

The last day of camp was turned over to session from attendees, including gems like:

You can see slides from all of the iCampHRM sessions in the camp schedule.

Many thanks to our instrutors, our attendees, and to Mount Saint Vincent University for hosting us on their beautiful campus. Stay tuned for news about where Islandora Camp will go next...