Islandora 8 (formerly known as IslandoraCLAW) is the next generation of Islandora. This major upgrade is compatible with Drupal 8 and Fedora 5. For more details, please check out the following resources:
Latest Islandora 8 News
Open Repositories 2015 has always been bright line on our project calendar. It's been a deadline for most of our project goals because that's when we wanted to show the community what their funding, use cases, discussions, and development has wrought. Well, OR2015 has come and gone and there is plenty of good news. First, the official goals of the Islandora 7.x-2.x project that have been completed:
- Update Tuque
Tuque is the repository agnostic API that allows Islandora to interact with Fedora repositories. Tuque will need to be updated to connect to and work with the Fedora 4 APIs.
- Islandora (core)
Islandora (core) is the Drupal module that allows for browsing and managing assets in a Fedora repository. Islandora (core) provides core hook and API functionality for all Islandora solution pack and utility modules
- Islandora Solution Pack Collection
Islandora Solution Pack Collection allows Islandora to view to view and manipulate objects as a collection. It also allows collections to be created, viewable, and manageable. Along with Tuque and Islandora (core), it is a requirement for all other solution packs.
- Replace FedoraGSearch
Fedora 4 provides a pluggable framework for an external triplestore and search index. The same JMS indexer can be used to communicate with both the triplestore and search index (e.g. Solr). Any triplestore that supports SPARQL updates can be used with Fedora 4. Fuseki and Sesame have been tested.
- Community driven and lead - With some guidance and funding from the Islandora Foundation, Islandora7.x-2.x has been designed, built, tested, and documented by the Islandora Community as a whole. In particular, the efforts of the Fedora 4 Interest Group (an open group welcome to anyone who wants to participate) have been integral to shaping the project and continue to drive the work forward.
- Documentation from the onset - Rather than writing up documentation as a last step when Islandora 7.x-2.x is up and running, it has been a part of the process right from the design phase, making it easier for contributors to step in and make their own modifications to the heavily-commented code while its still being developed.
- Encouraging contributions - Hand in hand with the previous improvement, encouragement to contribute to the project has been a goal at all levels, from developers helping with the code to end-users providing use cases and requesting features. By drawing on the opinions of the community as a whole, and not just the developers among us, Islandora 7.x-2.x is made to reflect the needs of a broad group of users. Our call: “All contributions are welcome: use-cases, documentation, code, patches, bug reports, feature requests, etc. You do not need to be a programmer to speak up!”
- DevOps - A vagrant build was created to fire up a working copy of Islandora 7.x-2.x quickly and easily. Every update is reflected in the vagrant build. Anybody can run the environment and work with the stack, whether that be testing or developing.
- Everything is in a single Git repository!
- Portland Common Data Model - At long last, a common data model between Hydra and Islandora. Lots of collaboration between the Islandora, Hydra, and Fedora communities for the betterment of all our projects. Our Project Director, Nick Ruest, is a committer for PCDM and an active member of the Hydra Metadata Working Group, working with the Hydra community on base recommendations for technical metadata (Nick and Aaron Coburn co-lead that sub-group), structural metadata, rights metadata, and descriptive metadata.
- Working with the Fedora team - Contributing to Fedora development, attending Fedora Tech meetings, being a stakeholder for the audit service integration.
- migration-utils - The community is going to need a way to get content from Fedora 3 into Fedora 4. Our team collaborated with Mike Durbin (University of Virginia) on creating a migration utility, and with just a handful of minor changes, it will be ready for use very soon. Testing is even underway to adapt this tool to bring content out of legacy Fedora 2 repositories.
- Scalable, Distributed, Service Oriented Architecture - no assumptions on the location of any component of the stack.
- Apache Camel - adopted as a routing and mediation engine for integrating Drupal and Fedora. The fcrepo-camel component provided by the Fedora community has been utilized to replace Tuque as our primary means of interaction.
- True Drupal integration - Fedora content is modelled as nodes. Due to this, many modules in the codebase have been replaced with third party drupal.org modules. We can finally use that 'Add Content' button in Drupal!
- Asynchronous communication Between Drupal and Fedora.
- Bi-directional synchronization Between Drupal Fields and Fedora RDF.
- Drupal Field to RDF mappings - A handy user interface allowing end users to control field -> RDF mappings.
- Map XML with XPath - User interface controlling extraction of xml metadata using XPath, allowing fragments to get mapped to RDF. Similar to Hydra “Terminologies” except with a user interface.
- RDFa Enriched HTML Output - Google will love your site!
- Solr Solr Solr - The entire Solr suite of modules from drupal.org is available, giving the end user control over how and when content is indexed in Solr.
The next steps and goals for the project include:
- Refactoring the current Apache Camel processing implementation to utilize command-line PHP. This should be more comfortable for the Islandora community. The goal is to lower the barrier to contribution.
- Starting serious work to make the project easier to share. Tickets are coming, some marked for "newbies" to take on as an entry point to the project. The Basic Image Solution Pack will be the first target. Contributors are very welcome!
- WebAccessControl integration - The Fedora community has begun planning WebAccessControl integration. If you'd love to see XACML go away, we'd love to have you as part of the stakeholder team!
Islandora 7.x-2.x community sprints coordinated with Fedora community sprints.
As the project entered the fourth month, work continued on migration planning and mapping, migration-utils, and Drupal integration.
Migration work was split between working on migration-utils, migration mappings, data modeling (furthering Portland Common Data Model compliance), and working with the Islandora (Fedora 4 Interest Group), Fedora (Fedora Tech meetings), and Hydra (Hydra Metadata Working Group) communities on the preceding items. In addition, Audit Service-- a key requirement of an Islandora community fcrepo3 -> fcrepo4 migration -- finalized the second phase of the project. Community stakeholders are currently reviewing and providing feedback.
Work on migration-utils focused mainly applying a number of mappings (outlined here) to the utility, adding support for object-to-object linking, and providing documentation on how to use the utility. This work can be demonstrated by building the Islandora 7.x-2.x Vagrant Box, cloning the migration-utils repository, and pointing migration-utils at a fcrepo3 native filesystem or directory of exported FOXML.
As for object modeling and inter-community work, an example of this work is the below image of a sample Islandora Large Image object modeled in the Portland Common Data Model. This model will continue to evolve as the communities work together in the various Hydra Metadata Working Group sub-working groups.
On the Drupal side of things, work was started on Middleware Services, a middleware service that will utilize the Fedora 4 REST API and the Drupal Services modules, and create an API for the majority of interactions between the two systems. In addition, a few Drupal modules have been created to leverage this; islandora_basic_image, islandora_collection, islandora_dcterms.
In addition, the team has been exploring options with RDF integration and support in Drupal, as well as how to handle editing (Islandora XML Forms) the various descriptive metadata schemas the community uses. This is captured in a few issues in the issue queue; #27 & #28. Due to the importance of the issue, a special Fedora 4 Interest Group meeting was held to discuss how to proceed with this functionality in Islandora 7.x-2.x. The group's consensus was to solicit use cases from the community to better understand how to proceed with 7.x-2.x
Work will continue on the migration and Drupal sides of the project into May.
If you have been following events in the Islandora community lately, you probably know that we are in the midst of an ambitious project to build an Islandora that works on top of Fedora 4. The project is going great, but it has raised some questions about how to proceed with one of the most crucial yet under-appreciated tools in the Islandora stack: XML Formbuilder.
Largely the work of a single developer (discoverygarden's awesome Nigel Banks), XML Form Builder is a powerful (and sometimes difficult-to-master) tool that allows you to leverage the ease of Drupal forms to edit metadata for your Fedora objects. All Islandora Solution Packs come with standard MODS forms, but XML Form Builder lets you go far beyond those basic building blocks to meet almost any use case. A simple form for students to add objects with just a few fields; a brand new form to manipulate esoteric metadata standards; a tiny tweak to an existing Solution Pack form that brings it in line with your institution's specific needs.
The Fedora 4 project team recognizes that this functionality needs to exist in the future version of Islandora. The question now is: How? Do we port over XML Forms and untangle the legacy of compromises that allows it to work in the current environment? Do we sidestep the issue and build a new tool that more closely leverages the underlying structure of Fedora 4? Do we build a new tool and make it look like XML Form Builder so it's comfortable to use, but still works completely differently? XML, RDF, Xpath, etc. It's a big task and it comes with a lot of big questions.
So, we turn to you, the Islandora Community, to get some answers. What tools do you need to work with metadata for your collections? How are you using XML Form Builder now? What parts of it don't you use? What parts are critical? To that end, Nick Ruest has put together a template to collect use cases. Please add yours to the list and help us shape Islandora's future.
The Fedora 4 upgration is coming into its third month with a big focus on migration. Notes from the last project meeting are available here. Some highlights:
Jared Whiklo, Web Application Developer at University of Manitoba, has joined the project team and is working with Danny and Nick on code tasks.
Recent work has focused on a couple of areas. The first was collaborating with Mike Durbin (University of Virginia) on fcrepo4-labs/migration-utils, which will support an upgration in this order:
- traversing the objectStore file system
- archive export format
- migrate export format
In order to provide a large set of test fixtures for use with this tool, Nick updated YUDL’s Fedora to 3.8.1-SNAPSHOT.
The second focus on was data modelling. Specifically, mapping fcrepo3 object properties to fcrepo4 container properties, fcrepo3 datastream properties to fcrepo4 file properties, mapping RELS-EXT properties, identifying standard audit trail events, and working towards bringing Islandora into compliance with the Portland Common Data Model. This work was shared with the community via a Large Image Solution Pack example object modelled in Fedora 4.
Related to the migration, work has also been done around Audit Service design in Fedora 4. Nick participated in all of the Audit Service design meetings, and led a discussion around PROV and PREMIS ontology usage in the service. A code sprint led by Esme Coles and devoted to the Audit Service began on March 30th. That work is outlined here.
The migration work will most likely continue through April. If you want to attend future meetings and keep an eye on development, please join the Islandora Fedora 4 Interest Group. Your ideas and use cases are also very welcome as issues is Islandora Labs. For anyone planning to attend the Open Repositories conference in Indianapolis this June, the upgration team will be giving a presentation on the upgration project called Islandora and Fedora 4: The Atonement.
On Friday, February 27th, the Fedora 4 Interest Group met for the second time to discuss the progress of our big upgration (the first meeting was back at the end of January). The full notes from the meeting are here, but I'll summarize some of the highlights:
The project has entered its second month with plenty accomplished. Nick was sent to Code4Lib 2015 in Portland, Oregon to work with our Technical Lead, Danny Lamb. The two worked on the proof-of-concept, and it was presented as a lightning talk (video demo). Additionally, Nick and Danny worked with the Hydra and Fedora communities on a shared data model, Hydra Works, which evolved into the Fedora Community Data Model.
After Code4Lib 2015, Nick and Danny focused on updating the Technical Design document, that provides:
- an understanding of the Islandora 7.x-2.x design rationale
- the importance of using an integration framework
- the use of camel
- inversion of control and camel
- camel and scripting languages
- Islandora Sync
- Solr and Triple store indexing
- Islandora (Drupal).
Or, to sum up the new ways of Islandora in one imge:
Nick and Danny also focused on the development virtual environment (DevOps) for the project. Nick decided to move away from using Chef and Berkshelf due to dependency support. The DevOps setup was moved to basic bash scripts and Vagrant. Contributors to the project can now spin a virtual development environment (which includes the proof-of-concept) in about 5 minutes with a single command: vagrant up. Instructions here.
Nick also focused on project documentation and documentation deployment. All document for the project resides in the git repository for the project, in Markdown format. The documentation can be generated into a static site with MkDocs and thendeployed to GitHub Pages. The documentation for the project can be viewed here, and information about how the documentation is built and deployed can be found here. There is also an outline of how you can contribute to the project here (regardless of your background. We need far more than programmers).
Nick, Danny, and Melissa also did an interview for Duraspace.
The upgration portion of the project is dependant on a couple of sub-items of the project to play out, but continues in tandem.
The first sub-item is the Fedora Audit Service. The Islandora community make use of the audit service in Fedora 3.x for PREMIS and other provance services. It currently does not exist in Fedora 4.x, so the community has come together to plan our the service over two conference calls that will outline use cases and functional requirements, which will then translate to JIRA tickets for a Fedora code sprint in late March. Notes from the first meeting are here. Nick has been tasked with identifying if the community should use the PROV-O ontology, the PREMIS ontology, or a combination of both. The second item is bridging the work of Mike Durbin’s migration-utils and Danny’s Apache Camel work in the Islandora & Fedora 4 project. While Nick was working to create test fixtures for Mike and Danny, he discovered a bug in Fedora 3.8.0, which will need to be resolved before any test fixtures can come out of York University's upgration pilot.
Nick and Danny will most likely focus on migration work and community contributed developer tasks in March.
The Islandora Foundation is pleased to welcome Simon Fraser University as a Partner for their support of the Fedora 4 project. Longtime member PALS has also earmarked some of their membership dues to help out the upgration. If you or your instition are interested in being financial supporters, please drop me a line.
Contributor Kevin Bowrin wrote up an account of exprience installing and trying out the work our team has done so far. Check it out.