MediaShelf Blog

Leading up to this year’s Open Repositories, it became clear that there was demand for a BOF (Birds of Feather) session focused on agile languages and Fedora.  I pitched the idea in an email to a couple colleagues beforehand and then announced the BOF at my presentation on Monday morning.  Rather than constricting it to Fedora projects, I billed it as Agile Languages and Repositories.  About 30 people showed up.  The split was pretty even between Ruby, Python, and PHP developers.  About a third seemed to be Java developers in the process of defecting.  In addition to people doing stuff with Fedora, there were a handful of DSpace developers and possibly a couple who maintain ePrints repositories.  

For the first half of the BOF we sat in mixed groups, eating our lunches and each talking about the work we do.  We then split up by language (Ruby, Python, PHP) and discussed language-specific topics.  For that second half I sat at the Ruby table where we talked about ActiveFedora, JRuby, RDF support for Ruby, MODS support for Ruby, Solr (solr-ruby and RSolr), and how Blacklight fits into the mix. 

I closed the conversation by asking if we should set up email lists for collaboration.  It seemed reasonable to set up a general mailing list for the solutions community as well as a list specifically for people doing stuff with Ruby, Fedora repositories, and (most likely) ActiveFedora.  I also resolved to encourage the creation of Python-oriented and PHP-oriented equivalents.  For now I have created two lists on Google Groups.  The first one, Fedora Commons Create, is for general discourse about creating client applications for Fedora.  The second, ActiveFedora / Ruby + Fedora Commons, is for Ruby-specific collaboration.

In the end, I was really pleased to realize that for the first time we had a substantial group of people interested in each of the main interpreted languages (Ruby, Python, PHP) and each group had at least one open source Fedora-based project to use as a starting point for their conversations.  The Ruby group had ActiveFedora, the Python group had Ben O’Steen’s work and Peter Herndon’s Django integration, and the PHP/Drupal people had Islandora & Fez to start from. 

This was a comfortable step forward from the scenario as it was a year ago.

Fedora Commons Create

Visit this group

ActiveFedora / Ruby + Fedora Commons

Visit this group

We have been working hard on creating a Domain Specific Language for declaring object models in ActiveFedora.  We settled on a syntax based on DataMapper.

Here are sample Model declarations for Audio Records and Oral Histories that we are using in a current project.  Keep in mind that this is just a teaser.  The syntax is likely to change over the next few months.

require 'active-fedora'

class AudioRecord

include ActiveFedora::Model

relationship "parents", :is_part_of, [nil, :oral_history]
# Also considering...
#has n, :parents, {:predicate => :is_part_of, :likely_types => [nil, :oral_history]}
# OR
# is_part_of [:oral_history]

property "date_recorded",   :date
property "file_name", :string
property "duration",  :string
property "notes", :text

datastream "compressed", ["audio/mpeg"], :multiple => true
datastream "uncompressed", ["audio/wav", "audio/aiff"], :multiple => true

end

Note that we are making it possible to inject custom methods into a class that search against RDF predicates.  This way, thanks to line 9 below, calling oral_history.parts will return everything pointing at the oral history object with info:fedora/isPartOf.  We are also thinking of supporting constraint paramaters like oral_history.parts(:type => AudioRecord), which would only return the parts that are of type AudioRecord.

require 'active-fedora'

class OralHistory 

    # Imitating DataMapper ...

    include ActiveFedora::Model

    relationship "parts", :is_part_of, [:audio_record], :inbound => true

    # These are all the properties that don't quite fit into Qualified DC

    # Put them on the object itself (in the properties datastream) for now.

    property "alt_title", :string

    property "narrator",  :string

    property "interviewer", :integer

    property "transcript_editor", :text

    property "bio", :string

    property "notes", :text

    property "hard_copy_availability", :text

    property "hard_copy_location", :text

 

    has_metadata "dublin_core", :type => ActiveFedora::MetadataDatastream::QualifiedDublinCore do |m|

      # Default :multiple => true, :refinements => :none

      #

      # on retrieval, these will be pluralized and returned as arrays

      # ie. subject_entries = my_oral_history.dublin_core.subjects

      #

      # aiming to use method_missing to support calling methods like

      # my_oral_history.subjects  OR   my_oral_history.titles  OR EVEN my_oral_history.title whenever possible

      m.field "identifier", :string, :refinements => ["info:fedora", "info:doi"]

      m.field "title", :text, {:multiple => false, :required => true}

      m.field "subject", :text, :refinements => ["dcterms:LCSH", :none]

      m.field "date", :date

      m.field "language", :text

      m.field "location", :text

      m.field "coverage", :text, :refinements => ["dcterms:TGN"]

      m.field "temporal", :text, :refinements => ["dcterms:Period"]

      m.field "abstract", :text

      m.field "rights", :text

      m.field "type", :text

      m.field "SizeOrDuration", :text

      m.field "format", :text

      m.field "medium", :text

    end

    has_metadata "significant_passages" do |m|

      m.field "significant_passage", :text

    end

    has_metadata "sensitive_passages" do |m|

      m.field "sensitive_passage", :text

    end

end

Picking up from the ideas in The Missing Sync for Fedora Commons, I’ve been talking with Thorny and Sandy at Fedora Commons about creating a Fedora Solutions Integration Council.  We haven’t quite figured out the structure of it, but the ideas are coming together pretty quickly. Bottom line, the council’s responsibility is to help everyone make informed decisions and support each other’s work.  

 As a first stab, I’m putting effort into three things:  

1. bring together the streams of communication (ie. blogs, irc, etc) 
2. help projects find and connect with others who are doing similar work
3. identify the major themes: problem areas, innovations, exciting solutions, etc.

Ultimately, I hope this will allow us to shed light on the various avenues of exploration in Fedora-centric application development.  So many people are doing such interesting and exciting work.  It’s time for us to talk more openly and enthusiastically about it.

The other Fedora Solutions Councils are organized around themes like eScience, Museums, and Education.   In contrast, the Integration Council is aimed at addressing the cross-cutting concerns of application development.  We all have to deal with things like access controls, scalability, and workflow.  The best solutions to these types of challenges are often applicable in many contexts, regardless of whether you are an eScience project or a small humanities archive.  Our aim is to get as much information flowing between developers as possible.  I want to let developers decide for themselves which ideas apply to their work.

Watch this space. 

Last month I picked up a Palm Centro. I quickly discovered that you can’t sync Apple’s calendar and address book applications to Palm OS without a $50 product called “The Missing Sync“. Within a week I had exchanged my Centro for a small, black Samsung dumbphone.

Since then, the topic of synchronization has come up repeatedly in my life.

On my laptop, I’m finally looking into synching Apple iCal with Google Calendar.

At home, I’ve started using Dopplr to figure out travel plans with my family and friends.

In my work life, I’ve started recognizing the fact that I actually play a sync role in the Fedora Commons community. I’m passionate about helping people use Fedora, so I’m constantly asking developers “How did you do that?” or “What went wrong? How did you fix it?”. This has naturally lead me to conversations where I find myself saying “Oh! You should talk to XXX project about the work that they’re doing. It’s right down your alley.” or “I think that someone has already solved that problem. Let’s ping the fedora-users list before we reinvent a wheel.”

I like this new theme. It fits with the way I want to operate in the world.

Fedora Commons is a community-driven project. The team in Ithaca has taken great strides to stabilize and facilitate community process. [In fact, the footwork and brainwork that Sandy Payette has done behind the scenes this year is facinating, but that's a topic for another post.] They now have a Chief Architect (Daniel Davis), a Director of Communications (Carol Minton Morris), and a Director of Community Strategy (Thornton Staples). When these three talented people joined Fedora Commons, I thought “Phew! Problem solved.” What I didn’t realize was that there is still a missing link.

I’ve learned that there is only so much that a centralized organization can do to synchronize community efforts. Ultimately, you still need people who slosh around in the morass of innovations, workarounds and hacks in order to find those gems of best practices and well designed solutions. More importantly, you need those people to put momentum behind the good ideas and ensure that they filter back into the common pool.

Until this month, I had not realized how important this is to community-driven open source software development. There are tons of projects out there who are more than happy to collaborate, to share ideas and solutions, and even to contribute code. However, one thing is consistently true about these projects: their hands are full. They rarely have time to look over each others’ shoulders and trade notes, let alone figuring out how to share their code.

There are, of course, notable exceptions to this rule. For example, Gert Pedersen has done an admirable job of maintaining GSearch and making it generally useful for everyone. Every time a new use case or problem crops up, he usually has a solution on SourceForge within a few weeks.

What about all of the other work that people are doing?

As of late, projects have started inviting me to play an advisory role in their Fedora work, to be their missing sync tool. I’m really excited about this because ultimately it means that I have an opportunity to help more people play to their strengths. I hope that by playing this role, I can help ensure that more great solutions find their way directly into Fedora itself while other solutions join the constellation of tools, services, and documentation that populate the Fedora Commons galaxy.

It’s amazing how quickly the OAI-ORE standard has caught on. I’m really excited to play around with the Foresite toolkit, which is the latest bit of lightweight goodness to come out of JISC. It seems that the first implementations have all been DSpace-oriented but Ben O’Steen is on the project to I expect to see some Fedora uses floating around soon enough.

In recent months, I have increasingly found myself connecting different Fedora developers with each other. Due to the ongoing upsurge of interest in Fedora Commons, and the constant increase in the number of projects using Fedora, it’s difficult to keep track of who is doing what.  Last year, after a lively BoF at OpenRepositories, I created a page on the Fedora wiki listing Fedora User Interface Projects.This page is useful to read, but it doesn’t really answer the question “Where do I start if I want to create my own Fedora client app?”

The Fedora commons team is working hard to set up stable channels of information to help people answer questions like this.  In the meantime, here is my quick rundown.  This is not meant to be a definitive reference.  I’m just rattling off the most salient points from the top of my head in the hope that someone will find it useful.

Java Solutions

Fedora’s APIs are entirely webservice based, so Java is not actually a necessity.  Nonetheless many projects choose to implement their client applications in Java since they already have a Java stack in play on their servers.  

The Fedora Client (distro)

Fedora itself is distributed with a Java Swing GUI client called fedora-admin.  This is a good place to get raw components for a client app.  If you download the source code for the fedora distribution, you can re-use the SOAP stubs from the fedora-admin client to connect your applications to Fedora.  This pretty much gives you the same code that you would get by generating Java code from Fedora’s WSDL, but I hear that it has been tweaked by the Fedora dev team to work more smoothly.

Muradora

Muradora is a Java application created by the DRAMA team in Australia.  Muradora started out as a proof of concept for the DRAMA Authentication and Authorization middleware.  Since then, it has become a prominent end-user client for Fedora.  Muradora does a good job of using Fedora’s existing features as much as possible.  It creates really clean Fedora objects, and will also recognize new Fedora objects automatically as long as you use some very simple, re-usable RELS-EXT relationships to arrange your objects into collections.

If you want to create your own client application, Muradora is a good place to look for samples of best practices for creating and using Fedora objects. 

Struts & Spring

As the Fedora Wiki reflects, there are a number of projects using Struts and Spring to create Fedora client applications.  Muradora is one of those projects.

Grails, Wicket, etc.

I’ve been hearing a lot about the new RAD-inspired Java frameworks like Grails and Apache Wicket.  At the moment, I’m not aware of any Fedora projects that are using these frameworks.  I’m sure that will change soon. 

PHP Solutions

Fez

Fez is a prominent PHP-driven frontend solution for Fedora.  It is maintained by developers at the University of Queensland in Australia.  I haven’t looked at the Fez code in over a year, but that might be a good place to start if you want to create your own PHP client for Fedora. 

Drupal

Numerous projects are currently implementing Drupal modules for Fedora.  I’ve heard about most of them informally, so I can’t list the organizations, but I can tell you that the University of Prince Edward Island is one of them.  UPEI is hosting the Red Island Repository Institute this August.  I anticipate that there will be quite a lot of Drupal-centric skill sharing going on there. 

Python Solutions

Ben O’Steen’s work at Oxford

Ben is a rockstar.  I’ve already praised him in prior posts and, honestly, his work speaks for itself.  If you intend to do any work with Python and Fedora, definitely start by looking at his code.  I also recommend planning for the fact that you will almost definitely want to pull his innovations into your code on an ongoing basis. The best place to get up to date information about Ben’s work is on his blog Less Talk, More Code. 

Plone

I have heard about at least one substantial project doing work with Plone and Fedora.  They are currently in early stages of development. 

Django

I’m surprised that I still haven’t heard about anyone using Django with Fedora.  This nifty framework by Adrian Holovaty hails from the world of online publishing.  It was originally developed in-house by the Washington Post.  Django seems like a really great framework for doing rapid application development.  Due to its history in publishing, it seems like a perfect fit for quite a few Fedora use cases.  

Though I still have not heard about any active projects using Django with Fedora, I have inspired some local developers here in Minneapolis to push for it in their own organization.  Watch this space.

Ruby Solutions

RubyFedora

I’m pleased to let you know that the RubyFedora gem is available for download from RubyForge.  This library allows any Ruby application or script to use the full REST API of Fedora 3.0.  We here at MediaShelf wrote both Fedora’s REST API and the RubyFedora gem that consumes it, so the two work together really well.

ActiveFedora

ActiveFedora is a work in progress.  When it’s ready, we will post it on RubyForge alongside RubyFedora.  The intention of ActiveFedora is to allow Ruby developers to interact with Fedora repositories in the same ways that they currently interact with relational databases.  We want developers to think of Fedora as “just another component” in their user-driven applications.  We are achieving this by imitating Ruby on Rail’s ORM and database management features. 

The ActiveFedora gem will lay the foundation for true, sustainable, rapid development of Fedora client applications.    It will also provide a structure for figuring out and re-using best practices around Fedora’s Content Model Architecture.

acts_as_fedora

acts_as_fedora is a Rails plugin that ties Fedora into an existing Rails application.  Rather than replacing the database connection completely, it adds hooks into Fedora within your database objects’ lifecycle.  I’m not sure of the status of this gem.  It may not be open source, and may not be actively supported.  If anyone has more information, let me know.

ColdFusion Solution

At the recent JA-SIG conference, I learned about a project at Cornell University that is using ColdFusion to create a Fedora client application.   For more info, check out the page on the JA-SIG website.