Islandora CLAW

Latest CLAW News

Fedora 4 Project Update III

manez's picture
Submitted by manez on

The Fedora 4 upgration is coming into its third month with a big focus on migration. Notes from the last project meeting are available here. Some highlights:

Jared Whiklo, Web Application Developer at University of Manitoba, has joined the project team and is working with Danny and Nick on code tasks. 

Recent work has focused on a couple of areas. The first was collaborating with Mike Durbin (University of Virginia) on fcrepo4-labs/migration-utils, which will support an upgration in this order:

  • traversing the objectStore file system
  • archive export format
  • migrate export format

In order to provide a large set of test fixtures for use with this tool, Nick updated YUDL’s Fedora to 3.8.1-SNAPSHOT.

The second focus on was data modelling. Specifically, mapping fcrepo3 object properties to fcrepo4 container properties, fcrepo3 datastream properties to fcrepo4 file properties, mapping RELS-EXT properties, identifying standard audit trail events, and working towards bringing Islandora into compliance with the Portland Common Data Model. This work was shared with the community via a Large Image Solution Pack example object modelled in Fedora 4.

Related to the migration, work has also been done around Audit Service design in Fedora 4. Nick participated in all of the Audit Service design meetings, and led a discussion around PROV and PREMIS ontology usage in the service. A code sprint led by Esme Coles and devoted to the Audit Service began on March 30th. That work is outlined here.

The migration work will most likely continue through April. If you want to attend future meetings and keep an eye on development, please join the Islandora Fedora 4 Interest Group. Your ideas and use cases are also very welcome as issues is Islandora Labs. For anyone planning to attend the Open Repositories conference in Indianapolis this June, the upgration team will be giving a presentation on the upgration project called Islandora and Fedora 4: The Atonement.

Islandora/Fedora 4 Project Update II

manez's picture
Submitted by manez on

On Friday, February 27th, the Fedora 4 Interest Group met for the second time to discuss the progress of our big upgration (the first meeting was back at the end of January). The full notes from the meeting are here, but I'll summarize some of the highlights:

Project Updates

The project has entered its second month with plenty accomplished. Nick was sent to Code4Lib 2015 in Portland, Oregon to work with our Technical Lead, Danny Lamb. The two worked on the proof-of-concept, and it was presented as a lightning talk (video demo). Additionally, Nick and Danny worked with the Hydra and Fedora communities on a shared data model, Hydra Works, which evolved into the Fedora Community Data Model.

After Code4Lib 2015, Nick and Danny focused on updating the Technical Design document, that provides:

  1. an understanding of the Islandora 7.x-2.x design rationale
  2. the importance of using an integration framework
  3. the use of camel
  4. inversion of control and camel
  5. camel and scripting languages
  6. Islandora Sync
  7. Solr and Triple store indexing
  8. Islandora (Drupal).

Or, to sum up the new ways of Islandora in one imge:

Nick and Danny also focused on the development virtual environment (DevOps) for the project. Nick decided to move away from using Chef and Berkshelf due to dependency support. The DevOps setup was moved to basic bash scripts and Vagrant. Contributors to the project can now spin a virtual development environment (which includes the proof-of-concept) in about 5 minutes with a single command: vagrant up. Instructions here.

Nick also focused on project documentation and documentation deployment. All document for the project resides in the git repository for the project, in Markdown format. The documentation can be generated into a static site with MkDocs and thendeployed to GitHub Pages. The documentation for the project can be viewed here, and information about how the documentation is built and deployed can be found here. There is also an outline of how you can contribute to the project here (regardless of your background. We need far more than programmers).

A new use case template makes bringing your ideas to the table much easier. Check out some of the existing use cases for examples - and add yours!

Nick, Danny, and Melissa also did an interview for Duraspace.

Upgration

The upgration portion of the project is dependant on a couple of sub-items of the project to play out, but continues in tandem.

The first sub-item is the Fedora Audit Service. The Islandora community make use of the audit service in Fedora 3.x for PREMIS and other provance services. It currently does not exist in Fedora 4.x, so the community has come together to plan our the service over two conference calls that will outline use cases and functional requirements, which will then translate to JIRA tickets for a Fedora code sprint in late March. Notes from the first meeting are here. Nick has been tasked with identifying if the community should use the PROV-O ontology, the PREMIS ontology, or a combination of both. The second item is bridging the work of Mike Durbin’s migration-utils and Danny’s Apache Camel work in the Islandora & Fedora 4 project. While Nick was working to create test fixtures for Mike and Danny, he discovered a bug in Fedora 3.8.0, which will need to be resolved before any test fixtures can come out of York University's upgration pilot.

Nick and Danny will most likely focus on migration work and community contributed developer tasks in March.

Funding

The Islandora Foundation is pleased to welcome Simon Fraser University as a Partner for their support of the Fedora 4 project. Longtime member PALS has also earmarked some of their membership dues to help out the upgration. If you or your instition are interested in being financial supporters, please drop me a line.

Other News

Contributor Kevin Bowrin wrote up an account of exprience installing and trying out the work our team has done so far. Check it out.

 

 

Islandora/Fedora 4 Project Update

manez's picture
Submitted by manez on

The Islandora 7.x/Fedora 4.x integration that we announced in December has officially begun. Work began on January 19th and our first team meeting was Friday, January 30th and we will be meeting on the 4th Friday of every month at 1:00 PM Eastern time. Here's what's going on so far:

Project Updates

The new, Fedora 4 friendly version of Islandora is being built under the working designation of Islandora 7.x-2.x (as oppose to the 7.x-1.x series that encompasses current Fedora 3.x updates to Islandora, which are not going away any time soon). A new GitHub organization is in place for development and testing, and the Islandora Fedora 4 Interest Group has been reconvened under new Terms of Reference to act as a project group for the Fedora 4 integration. If you want to participate, please sign up as part of this group. If you don't have time to participate in regular meetings, we would still love to hear your use case. You can submit it for discussion in the issue queue of the interest group. Need help getting into the GitHub of it all? Contact us and we'll get you there.

There is also a new chef recipe in the works to quickly spin up development and testing environments with the latest for 7.x-2.x. Special thanks to MJ Suhonos and the team at Ryerson University for Islandora Chef!

The project is under the direction of Project Lead Nick Ruest (York University) and Tech Lead Danny Lamb (discoverygarden, Inc.), with participation from:

  • The University of Toronto Scarborough
  • The University of Oklahoma
  • The University of Manitoba
  • The University of Virginia
  • The University of Prince Edward Island
  • The University of Limerick
  • Simon Fraser University
  • REUNA
  • LYRASIS
  • Common Media
  • The Colorado Alliance

Special thanks goes to Aaron Coburn, whose fcrepo Camel module is going to be an integral part of our own designs for Fedora 4 and Islandora.

If you would like to talk to Nick and Danny about the project, or even offer up some help while they code away on an unofficial 'sprint,' you can meet up with them at discoverygarden's table at Code4Lib 2015 in Portland, OR February 9 - 12.

Technical Planning

Danny Lamb has kicked off the design of the next stage of Islandora with a Technical Design Doc that you should definitely read and comment on if you have any plans to use Islandora with Fedora 4 in the future. We are still at the stage of hearing use cases and making plans, so now is the time to get your needs into the mix. The opening line sums up the basic approach: Islandora version 7.x-2.x is middleware built using Apache Camel to orchestrate distributed data processing and to provide web services required by institutions who would like to use Drupal as a frontend to a Fedora 4 JCR repository. 

Some preliminary Big Ideas:

  • No more Tuque. No more GSearch. No more xml forms. The Java middleware layer will handle many things that were previously done in PHP and Drupal.
  • It will treat Drupal like any other component of the stack. There will be indexing in Drupal for display using nodes, fields, and other parts of the Drupal ecosystem.
  • It will use persistent queues, so the middleware layer can exist on separate servers.
  • The Fedora-Drupal connection comes first. An admin interface will be developed later.

And some preliminary Wild Ideas (we'd love to hear your opinions):

  • Headless Drupal 7.x
  • Make the REST API endpoints the same for Drupal 7 and Drupal 8 so migration is easier.
  • Dropbox-style ingest.

Migration

Or rather, upgration (a portmanteau of upgrade and migration, and our new favourite word). Nick Ruest and York University are working through a Fedora 3.x -> 4.x upgration path. Because York's Islandora stack is as close to generic as you can reasonably get in in production, this should provide a model for a generic upgration path that others can follow - as well as keeping the needs of the Islandora community on the radar for the Fedora 4 development team, so that all of the pieces evolve to work together.

Funding

We launched the project with a funding goal of $100,000 to get a functioning prototype and Fedora 3.x -> 4.x migration path. We are very pleased to announce that we have achieved more than half of that funding goal and are well set to see things through to the end. 

Many, many thanks to our supporters, all of whom are now members the Islandora Foundation as Partners:

  • LYRASIS
  • York University
  • McMaster University
  • University of Prince Edward Island
  • University of Manitoba
  • University of Limerick

If your institution would like to join up, whether as a $10,000 Partner or at some other level of support, please contact us

 

Islandora/Fedora 4 Project and the Fedora 4 Interest Group

nruest's picture
Submitted by nruest on

Hi folks-

Today is the first official day of the Islandora/Fedora 4 Project! 

I'd really like to make sure communication is clear around this project, so we are going to reconvene the Fedora 4 Interest Group under a new terms of reference, and use this as the primary place for communication about the project. That is not to say that we will not be giving regular updates to the mailing lists :-)

If you have thoughts, opinions, use cases, functional requirements, ramblings, rants, etc., of how Islandora and Fedora 4 should work together, then this is *definitely* the place to share those.

If you were a part of the original Fedora 4 Interest Group, and you'd like to continue on, awesome!

If you would not like to continue on, feel free to remove yourself from the list of members of the ToR, or tell me to remove you.

If this is new to you, and you'd like to be apart of this, feel free to sign up; fork the repo, add your name, and issue a pull request. Or, tell me to add you. All that said, I'd like to schedule a call as soon as possible, so please add yourself to this Doodle poll.

cheers!

-nruest

Islandora and Fedora 4

manez's picture
Submitted by manez on

Now that Fedora 4 has a production release, we at the Islandora Foundation would like to share our plans to integrate so that Islandora users, new and existing, can take advantage of the expanded performance and functionality of this major update to Fedora.

The details of our proposed plan and budget can be found in our Fedora 4 Prospectus and Project Plan, prepared by the Fedora 4 Interest Group. In short, we have established Drupal 7.x as the front end for a Fedora 4 prototype, and will commence development in January 2015 with an update and demo of the new system to be ready in time for the Open Repositories conference in June. To reach this goal, we have identified funding needs and begun soliciting support from the Islandora community. The primary use for these funds will be a dedicated developer to serve as Technical Lead on the project and oversee our effort to get an initial product out to the community. The developer for this position was recently selected, and we are very pleased to announce that Daniel Lamb of discoverygarden will be spearheading the technical development of Fedora 4/Islandora 7 integration. 

Taking the lead on overall management of the project will be Project Director Nick Ruest, two-time Islandora Release Manager and heavy contributor to the Islandora stack, with project support from the Islandora Foundation. We will be calling for volunteers from the community to join in the effort and we plan to hold one or two hackfests in the coming year. While developers, both individual and as in-kind donations from supporter institutions, are vital to this project, we are also very much in need of non-developer volunteers to test, document, and provide use cases to determine the features and scope of the update. A call for volunteers will go out in late January or early February. In the mean time, if you have questions or would like to commit funds or developer time to support project, please contact us, or ask on the mailing list.

For more information about how Islandora will work with Fedora 4, please check out this recent webinar hosted by discoverygarden. In the coming months, Nick Ruest and Daniel Lamb will be providing more detailed plans and reports here, and the Fedora 4 Interest Group will move its mandate from being a group to get the community moving towards Fedora 4 integration to a group that will work through the integration by soliciting feedback on proposed use cases and implementations. In addition, the group will work with the greater Fedora community towards a generic Fedora 3.x to Fedora 4 migration scenario.

Pages

Subscribe to Latest CLAW News