Skip navigation.
Home
Adopting Standards and Specifications for Educational Content
Home

Article: Lessons Learned by ASPECT content developers

by Lars Ingesman, UNI-C

As ASPECT comes to a close in February 2011, content providers have highlighted a number of issues related to the standards and specifications that have been investigated in the project. These include OAI-PMH, IEEE LOM and LRE MAP v.4, SCORM, IMS Common Cartridge and the IMS Question & Test Interoperability.

The project has included 12 content providers in 11 countries from both the public and private sectors with very different backgrounds and business models. However, although the views and experiences of organizations differ to some extent, it has been possible to identify a number of ‘lessons learned’ that can form the basis of best practices when implementing the above standards and specifications.

A more detailed account of these conclusions can be found in project deliverable D5.5 Report on the advantages and issues associated with the large-scale implementation of selected standards, which will be available on the project web site in early January 2011. In this article we highlight some of them.

Specifications and standards for content discovery

OAI-PMH

All content providers in ASPECT implemented OAI-PMH (Open Archives Protocol for Metadata Harvesting) in order to establish a connection to the ASPECT version of the Learning Resource Exchange (LRE) portal. Content developers not only found OAI-PMH a simple specification to read and understand but also thought the protocol was fairly simply to implement, not least because there are freely available libraries for a number of different development environments and programming languages.

One of the organizations that implemented an OAI-PMH connection from scratch, despite the fact that they had an existing LRE connection (based on the Simple Query Interface), was the University of Ljubljana. After having been through the process, they have no hesitation in recommending harvesting with the OAI-PMH specification as a best practice for connecting repositories:

“We are not really experienced in this area but we find this protocol simple and efficient and therefore we are recommending it as a best practice to connect your repositories.”

LOM and ILOX

At the start of the project the metadata format required by the LRE was a profile of the IEEE Learning Object Metadata (LOM) specification.

During ASPECT, the IMS Learning Object Discovery and Exchange (LODE) group set about trying to solve a number of different issues related to LOM; for example, among other things the problem of having multiple manifestations of the same learning object, for instance a SCORM version, a Common Cartridge version, and a web-based version. Building on the work of the LODE group on Information Learning Object eXchange (ILOX), a new LRE Metadata Application Profile v. 4 was developed during the project.

Initially there was some unease among content developers because, at first sight, the specification appeared rather complex and difficult to understand. However, after having implemented the specification, a number of content providers started to see that it may solve some of the problems they actually have with different versions of the same learning resource.

Specific problems that some partners experienced in implementing LRE MAP v.4 included: mapping their own LOM metadata to the LRE LOM profile; problems over choosing vocabulary entries that related to the LRE only rather than core LOM values; and the lack of change logs in updates of the specification. On the whole, however, content providers found it relatively easy to implement the ILOX wrapper around their existing LOM export format and the LRE MAP v4. In addition, the documentation has improved during the course of the project based on the input from ASPECT partners and other organizations.

CNDP in France was one of the content providers that saw positive elements in the new metadata application profile.

“It was very rich to discover and study the possibilities offered by the LODE ILOX model. The model can respond to some information management problems we face, as multiple metadata records for the same resource to indicate technical issues, multiple metadata records to manage the information proposed as facets in the LRE application profile.”

General issues concerning metadata standards

It is important to remember though that potential benefits are not the only thing that matters for the uptake of a specific standard or specification. ASPECT has shown that there are other factors that need to be taken into account, such as costs, organizational issues, different needs, the maturity of the specification in question and the level of support from experts, documentation and validation services.

For organizations that are starting to add metadata to their resources from scratch, the availability of a selection of best-practice guidelines and standards to implement may be of enormous benefit. Instead of having to go through a costly process of examining a number of different standards and specifications, one ASPECT partner was able to short-circuit the process and rely on the recommendations of ASPECT experts.

Others partners, however, were in a quite different situation. While some could see definite advantages to the new LRE MAP v4, they also anticipated that it would require a huge effort to implement this given their current practice or that it might not be feasible because neither strict LOM nor implementing the ILOX model would meet the business needs of the organization. Others also had issues relating to the stability of standards and were reluctant to adopt a specification until it had reached a certain level of maturity.

Specifications and standards for content use

In ASPECT we were primarily concerned with exploring how best practices could be developed as content developers in the project worked with SCORM, IMS Common Cartridge and QTI.

SCORM

SCORM has been around for a number of years. It is a relatively accepted and, in certain areas, widely used packaging standard; for example in computer-based training where the focus is on self-paced, individualised learning. Work in ASPECT involved converting over 200 SCORM packages using a utility from Icodeon, one of the ASPECT partners. This utility basically just re-packages the resources of a SCORM zip-file as a Common Cartridge zip-file, removing or disabling whatever SCORM-specific functionality the original package may contain.

Working with the conversion utility made it very clear how different SCORM-packaging tools created somewhat different and not always acceptable SCORM packages – and certainly not always SCORM packages that the conversion utility was capable of handling correctly. In this process, most of the content providers that were involved in the conversion also got a close look at the SCORM specifications as well as validation tools and different SCORM players.

A number of the problems we ran into were caused by the different implementations of the SCORM specification by different tools. One conclusion to draw from this may be that the complexity of the SCORM specification is one of the greatest issues when talking about standardization and general acceptance. If a specification is difficult to implement, different tools providers may very well end up implementing slightly different versions of the specification – and no one will actually reap the benefits of a working with an accepted standard.

One particularly interesting finding was that rather few of our content providers actually used SCORM-specific functionality such as sequencing, navigation and LMS interaction. Most simply used the SCORM specification as a form of IMS Content Packaging. One reason for this mentioned by some project partners is that utilizing some of the advanced features of SCORM requires a lot of work both in the design and the development phase. Another reason cited was that the tools available are not user-friendly enough to allow instructional designers to easily create packages that use these features. Also, there is a cost issue, in that large-scale implementation with different output formats is not easy to combine with highly tailored learning resources.

In other cases a number of content partners had very strong views concerning the use of content packages that were strongly influenced by the pedagogical models they had adopted. In these cases, SCORM features such as navigation and sequencing were seen as primarily supporting individualised instruction based on highly structured information presentation. While this could be appropriate for training highly motivated people in specific skills, it was not thought to be appropriate by those partners who had a more socio-constructivist view of learning.

The conclusions of the ASPECT content developers related to SCORM are that:

  • Many tools produce non-compliant SCORM packages.
  • A number of tools only implement parts of the SCORM specifications.
  • Few content providers use SCORM-specific functionality such as sequencing and navigation.
  • SCORM packages provide benefits if users rely on presentations and tracking by means of an LMS.
  • To many people SCORM is still closely associated with individualized, self-paced learning based on behaviourist instructional design principles.

IMS Common Cartridge

Given some of the perceived ‘issues’ with SCORM, in ASPECT we focused more on the newer IMS Common Cartridge specification.

The general view is that the Common Cartridge specification is an interesting specification. There are a number of reasons for this: there are simple tools available to help developers get started; it is easy to create packages using these tools; and there are validation tools available, such the IMS Online Validator service.

Of course, as IMS Common Cartridge is a relatively new specification, there are a number of challenges. For one partner the lack of standards’ compliant tools was the most serious issue, resulting in this organization having to do a lot of manual editing of the XML files generated by various tools to achieve complete compliance with the specification. Many authoring tools used by this and other partners also did not allow them to create the quality of content that they required as publishers. Restrictions placed on the use of supplementary standards (such as LOM and QTI) also made the standard more difficult to use.

Another key issue for a number of ASPECT content providers is the fact they do not consider content packaging relevant – to their users, to their view of learning, to the resources they provide, and the distribution models they have chosen so far.

Although some content providers recognize that IMS Common Cartridge is an ‘improvement’ over SCORM, the views relating to SCORM-packaged content still bias their attitude towards the Common Cartridge format. As expressed by ANSAS:

“Certainly, in our view, the Common Cartridge standard is more adequate to the education world compared to the SCORM one, but still it does not fit in our socio-constructivist view.”

Several others shared much the same view that packaged content is somehow more ‘closed’ and not quite relevant to their model of learning and distribution.

Non-adoption of or lack of interest in content packaging though is not just a question of learning models. Some see it primarily as a question of the distribution mechanism selected and the fact that currently in some countries the VLEs they have deployed in schools are not using packaged content.

The conclusions of ASPECT content developers related to IMS Common Cartridge are that:

  • The specification is relatively easy to understand.
  • It is relatively easy to create a script-based packaging process.
  • Currently there are not many tools that produce fully compliant Common Cartridge packages.
  • There is one simple drag-n-drop tool that is free and easy to use.
  • Not all Common Cartridge functionality is supported by the tools.
  • There is a freely available online validator for testing cartridges.
  • The Common Cartridge specification imposes restrictions on LOM and QTI used as part of a Common Cartridge package.
  • The user needs a Common Cartridge compliant LMS or environment in order to run packages.

QTI

Currently, the IMS Global Learning Consortium is working on a revision and update of the Question and Test Interoperability Specification (QTI), moving it from version 1.2 to a version 2.1. However, the Common Cartridge specification uses a subset of the older, and in the opinion of some, obsolete version of QTI. In the Common Cartridge specification only six of the question types specified in QTI 1.2 are available.

On the whole, rather few ASPECT content providers have shown much interest in QTI, but two in particular, Cambridge University Press and Young Digital Planet, have been looking very carefully at QTI and the Common Cartridge ‘integration’ of the QTI specification.

Problems identified are that only six of the question types specified in QTI 1.2 are available in Common Cartridge and again many of the tools that claim to be standards’ compliant are not compliant with the most recent version of the standard.

The conclusions of the ASPECT content developers related to the IMS QTI specification are:

  • The specification is fairly complex.
  • There are not many tools that are fully compliant with the specification.
  • There are two partly competing versions of the specification: 1.2, which is used in the Common Cartridge specification, and 2.x, which is currently being developed.
  • There are restrictions as to how questions can be presented. Some content providers want a better integration of questions, exercises and actual content presentation.
»