 |
Data Migration Tools Project Summary
The goal of this project is to propose a framework for data migration (importing legacy data into MIFOS) that includes a well defined data import format and to implement the import of one business object type (the MIFOS Center type) as a proof of concept. This project is a first step towards the Desired Feature item Improved Tools For Data Migration.
Project Artifacts
Final Project Status - 10 April 2007
Work on this project (implementing a proof of concept) has been completed. A project proposal for a full implementation has been submitted.
Work completed
Week ending 13 April 2007
- prepared project plan for a complete Mifos Data Exchange XML import implementation
- wrote a sketch of a data migration framework using data import-- the Data Migration Roadmap
- performed a quick survey of open source ETL tools
- final checkin of project code
Week ending 6 April 2007
- completed proof of concept import functionality for Mifos Center objects
- worked on project planning and estimation for a complete Mifos Data Exchange XML import implementation
- studied Mifos codebase to understand potential issues with a complete Mifos Data Exchange XML import implementation
- generalized import code to handle custom field entries
- revised schemas to handle custom field entries
- investigated XML comparison libraries to handle validating equality of unordered elements
- had a discussion with Terry Wong regarding prior work on a Mifos API and its relation to data migration
Week ending 30 Mar 2007
- initial check-in of schemas and code for reading and validating Mifos Data Exchange XML
- completed a first pass at persisting imported Mifos Center objects into the database and did a check-in
- continued refining the XML schema
- had a discussion with Arun Kanabar from MFTech about their experience as the Mifos specialist doing data migration for Grameen Koota (see DataMigrationMFTechDiscussion)
Week ending 23 Mar 2007
- preliminary decision taken to use RELAX NG (RNG) as the primary XML schema language for data migration and also support W3C? XML Schema (WXS) (see DataMigrationSchemaAndToolChoices)
- preliminary decision taken to use JAXB to generate java classes for reading,writing and validating XML
- defined and iterated on an RNG schema for the Mifos Center business object
- created an ant build file that includes targets for:
- RNG -> WXS conversion
- RNG -> RNC (RELAX NG compact syntax format)
- WXS -> java class generation using JAXB
- running tests to exercise XML validation, reading & writing
- sucessfuly converted the RNG schema to a WXS schema
- sucessfully generated and compiled java files from the WXS schema using JAXB
- created a sample Mifos Data Exchange XML file for Center objects
- wrote and successfully ran initial tests to validate, read and write Mifos Data Exchange XML for Center objects
Week ending 16 Mar 2007
|