MediaShelf Blog

This week the University of Prince Edward Island hosted the second installment of Red Island Repository Institute (RIRI).  Participants came from North America and Europe to get a week-long intensive immersion in all things Fedora.  The institute is organized by Mark Leggott and the team at UPEI who created Islandora, a plugin that integrates Fedora into Drupal.  Thorny Staples (Product Director for Fedora Commons), Chris Wilper (Technical Lead for Fedora Commons), and I were the primary instructors.

Due to budget cuts across the board, there were less people in attendance at RIRI this year.  Over the next few years, broad community collaboration is certain to involve much more remote interaction online and less face-to-face exchange at large conferences.  I have a hunch that this will actually be beneficial for the growth of technical communities as long as it doesn’t last too long or become too restrictive.  After all, there is something potent about the ethos of Less Talk, More Code.  In the meantime, those of us who were able to make it to PEI had a really productive week.

Though structured as an intensive week-long training exercise, RIRI has taken on an element of information exchange between all of the participants.  The predominant flow of information was, as you would expect, from the three instructors to the 15 attendees, but I think that Thorny and Chris will agree with me that we have all picked up some good information and ideas along the way.  If anything, the institute has started to migrate in the direction of that quasi-mythical ideal conference where you get the conference-type info about who is doing what in your field while also getting your hands dirty with workshops and in-depth detail and discussion of how they are doing it and how you might build on their efforts.

Learning Curve

Those of us who were attending RIRI for the second time were of the consensus that this year had a much more gratifying learning curve with a sufficient continuity carrying through all five days.  This was predominantly thanks to how much Islandora and ActiveFedora have matured in the past twelve months.  We had everyone up and running, writing and running code against Fedora, in the multiple hands-on sessions through the week.

In contrast with last year, where we spent quite a bit of time on installing and configuring Fedora, this year we were able to use a VirtualBox that the Islandora team set up.  The VirtualBox has a debian image with Fedora, Islandora and Drupal pre-installed.  On Tuesday I installed Ruby, ActiveFedora and Solr on there too.  By Wednesday, we were able to pass around a couple of USB sticks and have everyone running identical development environments for the hands-on sessions.

I’m sure we will soon post the VirtualBox image for anyone to download.  Next year, everyone will probably have it running on their laptops before they even arrive on the Island.

Islandora

The Islandora team at UPEI have been very busy setting up Fedora-based Virtual Research Environments for the variety of scholars at their University.  It seems that once they had hammered out the module that integrates Fedora into Drupal, they fearlessly dove into spinning off deployments for numerous disciplines and projects.  I was thoroughly impressed by how many VREs they have set up, the diversity of content they are collecting, and the range of tools that they are setting up to manage, display, and operate on that content.  Four years ago, I looked at Fedora and saw a tidal wave of potential uses behind it.  That wave has finally begun to break and it is as exciting as we had imagined.

ActiveFedora

We used ActiveFedora as the basis for the hands-on sections of the institute.  On Wednesday we basically ran through the console tour that you can find on the ActiveFedora project site.  In the developer breakout on Thursday, we defined some ActiveFedora models in a rails app, created some Fedora objects based on those models, and explored their RDF relationships.

“That really makes sense.” seemed like the main response to ActiveFedora.  I can’t think of a better confirmation that designing the Domain Specific Language (DSL) for ActiveFedora’s models was worth it.

As part of the workshops, I also ran through the improvements I have planned for ActiveFedora version 1.1.

Hydra

Thorny gave a really great presentation on the background, goals, and status of the Hydra project.    Hydra and Islandora stand as wonderful complements to each other.  Islandora, written in PHP, takes Drupal as the starting point from which it reaches into the wide open space of a Fedora repository.  Hydra takes nearly the opposite approach.  Hydra does not assume an overarching system as its operating context.  Instead it builds outwards from ActiveFedora, which in turn builds upon Fedora’s internal flexibilities and strengths.   Where Islandora functions as a component that you plug into Drupal, Hydra apps are free-standing solutions that in turn rely on a core whose functionality can be integrated into potentially any system.  One is a stalactite, the other is a stalagmite.  Both have grown naturally out of real-world needs.

FeSL

Projects seeking to adopt Fedora right now have great options before them.  Different technologies will suit different projects while the overall vision and best practices pollinate across solutions.  The best example of this cross pollination is the FeSL Project, which Islandora and Hydra are both participating in along with MediaShelf and DuraSpace.  This effort will result in a complete replacement of Fedora’s security implementation so that it can be used more effectively and more flexibly by any of the client applications we write — Hydra, ActiveFedora, Islandora, Muradora, or otherwise.  We are still seeking additional community funding for FeSL.  For more info, see the Fedora Enhanced Security Layer page on the Fedora wiki.

Workflow

One of the more exciting themes for me this year was workflow for Fedora Repositories.  I’ve been actively interested in the topic since Richard Green and Chris Awre presented RepoMMan at OpenRepositories in 2007.  Despite my interest, I have tread softly in that realm because the topic, and particularly technologies like BPEL, have always seemed dangerously askew from the actual problem(s) at hand.  I’ve always felt that we were framing the entire problem incorrectly.  In the past nine months, that exact line of thinking has come to a head, leading people like the Hydra project (including the former RepoMMan team) to explore a variety of approaches.  This resulted in a well-received presentation at OpenRepositories this year in Atlanta, which Thorny presented again for the RIRI participants this week.

The topic of workflow finally congealed for me in an actionable way here at RIRI when Mark Leggott started talking about supporting Taverna in the Islandora VREs.  Taverna is one of a handful of “workflow engines” designed to allow scientists to chain together batches of computational operations that they want to perform on data in their labs.  From an architectural perspective, Taverna isn’t all that different from BPEL tools.  The difference is that Taverna and its ilk are designed with the assumption that the scientist who owns the data will create the workflows.  Consider this in contrast to the idea of a developer or repository manager creating workflows on behalf of the end users.  This simple assumption, that workflows will be constructed and run by the people who own/create the underlying content, leads the technology down a very different path of development.  Until now, we have mostly thought of workflows as services that a repository provides for end users.  It’s time for us to flip that around and instead think of ways to expose repositories and their supporting services as nodes that end users can tie into their own workflows as they see fit.  This approach has the ring of good engineering.  It’s a simpler, loosely-coupled, user-centric solution to the problem.  It has the additional benefit of putting the repository engineer alongside the content owner/creator while they both create and share workflows to perform their corresponding tasks.

Solutions Integration Community

RIRI has also brought focus back onto the utility of the Solutions Integration Community that we have been gradually building this past year.  I’ve now set up a fedora-solutions google group.  Join up if you want to be a part of the stuff described on the solutions integration community page on the Fedora wiki.

A Great Year to Come

Over the next 12 months we are going to see a brilliant array of advances around Fedora Repositories.  MediaShelf will certainly touch many parts of those advances.  We’re here to help you make your mark in this space, and we’re here to make Fedora work for your users.  Contact us if you want to put MediaShelf to work on your repository efforts.

We have been working hard on creating a Domain Specific Language for declaring object models in ActiveFedora.  We settled on a syntax based on DataMapper.

Here are sample Model declarations for Audio Records and Oral Histories that we are using in a current project.  Keep in mind that this is just a teaser.  The syntax is likely to change over the next few months.

require 'active-fedora'

class AudioRecord

include ActiveFedora::Model

relationship "parents", :is_part_of, [nil, :oral_history]
# Also considering...
#has n, :parents, {:predicate => :is_part_of, :likely_types => [nil, :oral_history]}
# OR
# is_part_of [:oral_history]

property "date_recorded",   :date
property "file_name", :string
property "duration",  :string
property "notes", :text

datastream "compressed", ["audio/mpeg"], :multiple => true
datastream "uncompressed", ["audio/wav", "audio/aiff"], :multiple => true

end

Note that we are making it possible to inject custom methods into a class that search against RDF predicates.  This way, thanks to line 9 below, calling oral_history.parts will return everything pointing at the oral history object with info:fedora/isPartOf.  We are also thinking of supporting constraint paramaters like oral_history.parts(:type => AudioRecord), which would only return the parts that are of type AudioRecord.

require 'active-fedora'

class OralHistory 

    # Imitating DataMapper ...

    include ActiveFedora::Model

    relationship "parts", :is_part_of, [:audio_record], :inbound => true

    # These are all the properties that don't quite fit into Qualified DC

    # Put them on the object itself (in the properties datastream) for now.

    property "alt_title", :string

    property "narrator",  :string

    property "interviewer", :integer

    property "transcript_editor", :text

    property "bio", :string

    property "notes", :text

    property "hard_copy_availability", :text

    property "hard_copy_location", :text

 

    has_metadata "dublin_core", :type => ActiveFedora::MetadataDatastream::QualifiedDublinCore do |m|

      # Default :multiple => true, :refinements => :none

      #

      # on retrieval, these will be pluralized and returned as arrays

      # ie. subject_entries = my_oral_history.dublin_core.subjects

      #

      # aiming to use method_missing to support calling methods like

      # my_oral_history.subjects  OR   my_oral_history.titles  OR EVEN my_oral_history.title whenever possible

      m.field "identifier", :string, :refinements => ["info:fedora", "info:doi"]

      m.field "title", :text, {:multiple => false, :required => true}

      m.field "subject", :text, :refinements => ["dcterms:LCSH", :none]

      m.field "date", :date

      m.field "language", :text

      m.field "location", :text

      m.field "coverage", :text, :refinements => ["dcterms:TGN"]

      m.field "temporal", :text, :refinements => ["dcterms:Period"]

      m.field "abstract", :text

      m.field "rights", :text

      m.field "type", :text

      m.field "SizeOrDuration", :text

      m.field "format", :text

      m.field "medium", :text

    end

    has_metadata "significant_passages" do |m|

      m.field "significant_passage", :text

    end

    has_metadata "sensitive_passages" do |m|

      m.field "sensitive_passage", :text

    end

end

In recent months, I have increasingly found myself connecting different Fedora developers with each other. Due to the ongoing upsurge of interest in Fedora Commons, and the constant increase in the number of projects using Fedora, it’s difficult to keep track of who is doing what.  Last year, after a lively BoF at OpenRepositories, I created a page on the Fedora wiki listing Fedora User Interface Projects.This page is useful to read, but it doesn’t really answer the question “Where do I start if I want to create my own Fedora client app?”

The Fedora commons team is working hard to set up stable channels of information to help people answer questions like this.  In the meantime, here is my quick rundown.  This is not meant to be a definitive reference.  I’m just rattling off the most salient points from the top of my head in the hope that someone will find it useful.

Java Solutions

Fedora’s APIs are entirely webservice based, so Java is not actually a necessity.  Nonetheless many projects choose to implement their client applications in Java since they already have a Java stack in play on their servers.  

The Fedora Client (distro)

Fedora itself is distributed with a Java Swing GUI client called fedora-admin.  This is a good place to get raw components for a client app.  If you download the source code for the fedora distribution, you can re-use the SOAP stubs from the fedora-admin client to connect your applications to Fedora.  This pretty much gives you the same code that you would get by generating Java code from Fedora’s WSDL, but I hear that it has been tweaked by the Fedora dev team to work more smoothly.

Muradora

Muradora is a Java application created by the DRAMA team in Australia.  Muradora started out as a proof of concept for the DRAMA Authentication and Authorization middleware.  Since then, it has become a prominent end-user client for Fedora.  Muradora does a good job of using Fedora’s existing features as much as possible.  It creates really clean Fedora objects, and will also recognize new Fedora objects automatically as long as you use some very simple, re-usable RELS-EXT relationships to arrange your objects into collections.

If you want to create your own client application, Muradora is a good place to look for samples of best practices for creating and using Fedora objects. 

Struts & Spring

As the Fedora Wiki reflects, there are a number of projects using Struts and Spring to create Fedora client applications.  Muradora is one of those projects.

Grails, Wicket, etc.

I’ve been hearing a lot about the new RAD-inspired Java frameworks like Grails and Apache Wicket.  At the moment, I’m not aware of any Fedora projects that are using these frameworks.  I’m sure that will change soon. 

PHP Solutions

Fez

Fez is a prominent PHP-driven frontend solution for Fedora.  It is maintained by developers at the University of Queensland in Australia.  I haven’t looked at the Fez code in over a year, but that might be a good place to start if you want to create your own PHP client for Fedora. 

Drupal

Numerous projects are currently implementing Drupal modules for Fedora.  I’ve heard about most of them informally, so I can’t list the organizations, but I can tell you that the University of Prince Edward Island is one of them.  UPEI is hosting the Red Island Repository Institute this August.  I anticipate that there will be quite a lot of Drupal-centric skill sharing going on there. 

Python Solutions

Ben O’Steen’s work at Oxford

Ben is a rockstar.  I’ve already praised him in prior posts and, honestly, his work speaks for itself.  If you intend to do any work with Python and Fedora, definitely start by looking at his code.  I also recommend planning for the fact that you will almost definitely want to pull his innovations into your code on an ongoing basis. The best place to get up to date information about Ben’s work is on his blog Less Talk, More Code. 

Plone

I have heard about at least one substantial project doing work with Plone and Fedora.  They are currently in early stages of development. 

Django

I’m surprised that I still haven’t heard about anyone using Django with Fedora.  This nifty framework by Adrian Holovaty hails from the world of online publishing.  It was originally developed in-house by the Washington Post.  Django seems like a really great framework for doing rapid application development.  Due to its history in publishing, it seems like a perfect fit for quite a few Fedora use cases.  

Though I still have not heard about any active projects using Django with Fedora, I have inspired some local developers here in Minneapolis to push for it in their own organization.  Watch this space.

Ruby Solutions

RubyFedora

I’m pleased to let you know that the RubyFedora gem is available for download from RubyForge.  This library allows any Ruby application or script to use the full REST API of Fedora 3.0.  We here at MediaShelf wrote both Fedora’s REST API and the RubyFedora gem that consumes it, so the two work together really well.

ActiveFedora

ActiveFedora is a work in progress.  When it’s ready, we will post it on RubyForge alongside RubyFedora.  The intention of ActiveFedora is to allow Ruby developers to interact with Fedora repositories in the same ways that they currently interact with relational databases.  We want developers to think of Fedora as “just another component” in their user-driven applications.  We are achieving this by imitating Ruby on Rail’s ORM and database management features. 

The ActiveFedora gem will lay the foundation for true, sustainable, rapid development of Fedora client applications.    It will also provide a structure for figuring out and re-using best practices around Fedora’s Content Model Architecture.

acts_as_fedora

acts_as_fedora is a Rails plugin that ties Fedora into an existing Rails application.  Rather than replacing the database connection completely, it adds hooks into Fedora within your database objects’ lifecycle.  I’m not sure of the status of this gem.  It may not be open source, and may not be actively supported.  If anyone has more information, let me know.

ColdFusion Solution

At the recent JA-SIG conference, I learned about a project at Cornell University that is using ColdFusion to create a Fedora client application.   For more info, check out the page on the JA-SIG website.