Islandora CLAW (formerly known as Islandora 7.x-2.x) is the next generation of Islandora. Still in development, this major upgrade will be compatible with Drupal 8 and Fedora 4. For more details, please check out the following resources:
Latest CLAW News
The Islandora Foundation is growing up. As a member-supported nonprofit, we have been very fortunate to have the support of more than a dozen wonderful universities, private companies, and like-minded projects - enough support that within our first year of operation, we were solvent. As of 2015, we now have a small buffer in our budget, which is a comfortable place to be.
But comfortable isn't enough. Not when our mission is to steward the Islandora project and ensure that it is the best software that it can be. With the launch of Fedora 4 last December, we started work on a version of Islandora that would work with this new major upgrade to the storage layer of our sites, recognizing that our community is going to want and need to move on to Fedora 4 someday and we had better be ready with a front-end for them when the time comes. Islandora 7.x-2.x was developed to the prototype stage with special funding from some of our supporters, and development continues by way of volunteer sprints. Meanwhile, Islandora 7.x-1.x (which works with Fedora 3) continues to be supported and improved - also by volunteers.
It's a lot to coordinate, and we have determined through consultation with our interest groups, committees, and the community in general that in order to do this right, we need to have someone with the right skill set dedicated to coordinating these projects. We need a Tech Lead.
Now to the point: we need money. We have a confirmed membership revenue of $86,000 per year*, which is plenty for one employee plus some travel and general expenses, but not enough to hire this second position that we need to get the project to the next level. About a month ago I contacted many of the institutions in our community to see if they could consider becoming members of the Islandora Foundation, and we had a gratifying number of hopeful responses (thank you to those folks!), but we're still short of where we need to be.
And so, the Funding Lobster (or Lobstometre). In the interest of transparency, and perhaps as motivation, this little guy is showing you exactly where things stand with our Tech Lead goal. If we get $160,000 in memberships we can do it (but we'll be operating without a net), $180,000 and we're solid, and if we hit $200,000 or above that's just unmitigated awesome (and would get turned into special projects, events, and other things to support the community). He's the Happy Lobster, and not the Sad Lobster, because we do believe we'll get there with your help, and soon.
How can you help? Become a member. While it would be great if we could frame this as a funding drive and take one-time donations, since the goal is to hire a real live human being who will want to know that they can pay their rent and eat beyond their first year of employment, we need to look for renewable commitments. Our membership levels are as follows:
- Member - $2000
- Collaborator - $4000
- Partner - $10,000
- $10 - $250+ (at your discretion)
There are many benefits to membership, including things like representation on governing committees and discounts at events. Check out the member page or drop me an email if you want to know more.
* some of our members were able to allocate more funding to support 7.x-2.x development than their typical membership dues. It is currently unknown how many will be able to maintain that funding level at renewal, but yearly membership revenue could be as high as $122,000. I went with the number we can be sure of.
The latest Islandora community sprint, announced back in October, is now underway. Sixteen wonderful people signed up for the two-week adventure into the Islandora 7.x-2.x codebase, and this morning the team met for the first time to take in a demo from Nick Ruest and Danny Lamb (recorded here for those who could not attend live). The sprinters then had their first "stand-up" where they went through existing tickets and picked tickets to work on.
We are tracking work through GitHub for this sprint, and you can see the list of issues here (you are very welcome to add your own!). We loaded the sprint with a lot of documentation tasks, with a special focus on basic user documentation to help new users get used to how to make their way around this new version.
Want to be a part of an Islandora sprint but couldn't make it this time? There will be another in December, and hopefully once a month thereafter, to keep development rolling on 7.x-2.x. Developers, documenters, and testers are all welcome. If you have the time and willingness to help, we will find an aspect of the project where you can work!
A new Islandora Community sprint is coming up November 2 - 14. In September we had great success working on maintenance tickets for Islandora 7.x-1.x. The focus this time will be Islandora 7.x-2.x and the future of the stack.
The goal for the sprint is to get sprinters up to speed on the project through knowledge sharing, project workflows, and documentation. It may be the case that little to no code is written during this two week sprint, and that is perfectly ok! We hope that this will be the first sprint of many on Islandora 7.x-2.x!
You do not have to be a developer to join and we have a number of non-dev or "newbie" tickets lined up for you to work on if you want to take part.
The sprint will kick off with a meeting on November 2 where Danny Lamb and Nick Ruest will provide a brief overview of the new stack, go over issues marked for the sprint, and work with sprinters on what they would like to do or get out of the sprint.
If you’d like to familiarize yourself with sprint issues, please check out this list of issues.
If you’d like to join the sprint, please add your information to this spreadsheet.
It would be great if we could also use this time to discuss some outstanding use-cases that will have architectural impacts. You don’t have to be a committer to take part in the discussions. All opinions are welcome. Please see the list of use cases here. Highest priority:
Open Repositories 2015 has always been bright line on our project calendar. It's been a deadline for most of our project goals because that's when we wanted to show the community what their funding, use cases, discussions, and development has wrought. Well, OR2015 has come and gone and there is plenty of good news. First, the official goals of the Islandora 7.x-2.x project that have been completed:
- Update Tuque
Tuque is the repository agnostic API that allows Islandora to interact with Fedora repositories. Tuque will need to be updated to connect to and work with the Fedora 4 APIs.
- Islandora (core)
Islandora (core) is the Drupal module that allows for browsing and managing assets in a Fedora repository. Islandora (core) provides core hook and API functionality for all Islandora solution pack and utility modules
- Islandora Solution Pack Collection
Islandora Solution Pack Collection allows Islandora to view to view and manipulate objects as a collection. It also allows collections to be created, viewable, and manageable. Along with Tuque and Islandora (core), it is a requirement for all other solution packs.
- Replace FedoraGSearch
Fedora 4 provides a pluggable framework for an external triplestore and search index. The same JMS indexer can be used to communicate with both the triplestore and search index (e.g. Solr). Any triplestore that supports SPARQL updates can be used with Fedora 4. Fuseki and Sesame have been tested.
- Community driven and lead - With some guidance and funding from the Islandora Foundation, Islandora7.x-2.x has been designed, built, tested, and documented by the Islandora Community as a whole. In particular, the efforts of the Fedora 4 Interest Group (an open group welcome to anyone who wants to participate) have been integral to shaping the project and continue to drive the work forward.
- Documentation from the onset - Rather than writing up documentation as a last step when Islandora 7.x-2.x is up and running, it has been a part of the process right from the design phase, making it easier for contributors to step in and make their own modifications to the heavily-commented code while its still being developed.
- Encouraging contributions - Hand in hand with the previous improvement, encouragement to contribute to the project has been a goal at all levels, from developers helping with the code to end-users providing use cases and requesting features. By drawing on the opinions of the community as a whole, and not just the developers among us, Islandora 7.x-2.x is made to reflect the needs of a broad group of users. Our call: “All contributions are welcome: use-cases, documentation, code, patches, bug reports, feature requests, etc. You do not need to be a programmer to speak up!”
- DevOps - A vagrant build was created to fire up a working copy of Islandora 7.x-2.x quickly and easily. Every update is reflected in the vagrant build. Anybody can run the environment and work with the stack, whether that be testing or developing.
- Everything is in a single Git repository!
- Portland Common Data Model - At long last, a common data model between Hydra and Islandora. Lots of collaboration between the Islandora, Hydra, and Fedora communities for the betterment of all our projects. Our Project Director, Nick Ruest, is a committer for PCDM and an active member of the Hydra Metadata Working Group, working with the Hydra community on base recommendations for technical metadata (Nick and Aaron Coburn co-lead that sub-group), structural metadata, rights metadata, and descriptive metadata.
- Working with the Fedora team - Contributing to Fedora development, attending Fedora Tech meetings, being a stakeholder for the audit service integration.
- migration-utils - The community is going to need a way to get content from Fedora 3 into Fedora 4. Our team collaborated with Mike Durbin (University of Virginia) on creating a migration utility, and with just a handful of minor changes, it will be ready for use very soon. Testing is even underway to adapt this tool to bring content out of legacy Fedora 2 repositories.
- Scalable, Distributed, Service Oriented Architecture - no assumptions on the location of any component of the stack.
- Apache Camel - adopted as a routing and mediation engine for integrating Drupal and Fedora. The fcrepo-camel component provided by the Fedora community has been utilized to replace Tuque as our primary means of interaction.
- True Drupal integration - Fedora content is modelled as nodes. Due to this, many modules in the codebase have been replaced with third party drupal.org modules. We can finally use that 'Add Content' button in Drupal!
- Asynchronous communication Between Drupal and Fedora.
- Bi-directional synchronization Between Drupal Fields and Fedora RDF.
- Drupal Field to RDF mappings - A handy user interface allowing end users to control field -> RDF mappings.
- Map XML with XPath - User interface controlling extraction of xml metadata using XPath, allowing fragments to get mapped to RDF. Similar to Hydra “Terminologies” except with a user interface.
- RDFa Enriched HTML Output - Google will love your site!
- Solr Solr Solr - The entire Solr suite of modules from drupal.org is available, giving the end user control over how and when content is indexed in Solr.
The next steps and goals for the project include:
- Refactoring the current Apache Camel processing implementation to utilize command-line PHP. This should be more comfortable for the Islandora community. The goal is to lower the barrier to contribution.
- Starting serious work to make the project easier to share. Tickets are coming, some marked for "newbies" to take on as an entry point to the project. The Basic Image Solution Pack will be the first target. Contributors are very welcome!
- WebAccessControl integration - The Fedora community has begun planning WebAccessControl integration. If you'd love to see XACML go away, we'd love to have you as part of the stakeholder team!
Islandora 7.x-2.x community sprints coordinated with Fedora community sprints.
As the project entered the fourth month, work continued on migration planning and mapping, migration-utils, and Drupal integration.
Migration work was split between working on migration-utils, migration mappings, data modeling (furthering Portland Common Data Model compliance), and working with the Islandora (Fedora 4 Interest Group), Fedora (Fedora Tech meetings), and Hydra (Hydra Metadata Working Group) communities on the preceding items. In addition, Audit Service-- a key requirement of an Islandora community fcrepo3 -> fcrepo4 migration -- finalized the second phase of the project. Community stakeholders are currently reviewing and providing feedback.
Work on migration-utils focused mainly applying a number of mappings (outlined here) to the utility, adding support for object-to-object linking, and providing documentation on how to use the utility. This work can be demonstrated by building the Islandora 7.x-2.x Vagrant Box, cloning the migration-utils repository, and pointing migration-utils at a fcrepo3 native filesystem or directory of exported FOXML.
As for object modeling and inter-community work, an example of this work is the below image of a sample Islandora Large Image object modeled in the Portland Common Data Model. This model will continue to evolve as the communities work together in the various Hydra Metadata Working Group sub-working groups.
On the Drupal side of things, work was started on Middleware Services, a middleware service that will utilize the Fedora 4 REST API and the Drupal Services modules, and create an API for the majority of interactions between the two systems. In addition, a few Drupal modules have been created to leverage this; islandora_basic_image, islandora_collection, islandora_dcterms.
In addition, the team has been exploring options with RDF integration and support in Drupal, as well as how to handle editing (Islandora XML Forms) the various descriptive metadata schemas the community uses. This is captured in a few issues in the issue queue; #27 & #28. Due to the importance of the issue, a special Fedora 4 Interest Group meeting was held to discuss how to proceed with this functionality in Islandora 7.x-2.x. The group's consensus was to solicit use cases from the community to better understand how to proceed with 7.x-2.x
Work will continue on the migration and Drupal sides of the project into May.