MetaData Sync is an app to facilitate the synchronisation of metadata and data between DHIS2 instances, even between different versions. With a few clicks you can perform tasks that used to require many hours of manual work. In this post we tell you about its main functionalities, a bit of its history and some examples of use.
DHIS2 is an incredibly powerful data management system, but working with multiple instances can be a tedious process requiring manual actions, many scripts… MetaData Sync is designed to solve these problems. It is a versatile tool that is developed in response to the needs of different projects and use cases. Some of our partners use it to migrate data from local instances, sometimes in the field with poor connectivity, to a main instance at their headquarters. Others merge heterogeneous data collected from several countries so that it can be used for global reporting. It is also widely used to create metadata packages that can be easily installed on any instance, facilitating, for example, standardisation in disease data collection.
‘The most innovative thing about MetaData Sync is that it saves an incredible amount of manual time. Sending data from one instance to another, doing the necessary transformations to be able to send it, creating metadata packages that can be installed on other instances... until MetaData Sync all of these were very tedious manual tasks. For me, and I'm sure for hundreds of other people working with DHIS2, this application has been a huge time saver’.
What are MetaData Sync's key features?
With MetaData Sync you can:
- Synchronise data and metadata: send data of different types from one instance to another, automatically performing all the necessary transformations on the fly. You decide whether you want to send all available data or only some data.
- Metadata mapping: compare two or more DHIS2 instances, map their metadata and save the mapping for later use in synchronising these instances.
- Synchronisation history: check at any time the result of a synchronisation, who performed it and the status result.
- Metadata distribution: bundle metadata into modules, generate packages and upload them to a repository for public distribution, or use your instance as a package repository. You can also notify when there are changes to these packages and make metadata requests to other instances.
- Migration rules: Create rules for migrations, including metadata exclusion rules, or create packages from these rules.
- Metadata Custodians: Configure your instance so that only certain people can approve metadata requests from other instances.
"You can do everything, really everything, from MetaData Sync's graphical interface. It is very easy and intuitive to use. We are very grateful to our users for their feedback, which has made it possible for us to develop this versatile yet simple application."
What is the story behind MetaData Sync?
Lucia Fernández, who is now a senior technical advisor at the Health Clinton Initiative, was working at the World Health Organization Global Malaria Programme in 2019, where she was leading the development of digital tools for malaria control. Her role involved the creation of DHIS2 programmes for various countries, with the objective of facilitating the collection of data on insecticide resistance in mosquito populations.
“We were creating modules that each country would install in their own instance to collect the information. At the same time, at the global level, we needed this data every year to feed into the Malaria Threats Maps platform and to produce a section of the World Malaria Report. Up until then, country information was collected using four Excel forms. We spent a lot of time cleaning the data up and making it uniform so that we could use it. One day I thought, ‘Could we make a tool that would let countries send data straight from their own systems to our global one?’ At the time we were already working with EyeSeeTea and we had hours left on a first contract. They always had great solutions, so I decided to approach them with this problem. They suggested to improve an application they had already developed for synchronising some metadata objects, which was a very early version of what is now MetaData Sync”, explains Lucía Fernández.
"While working on MetaData Sync, I was really impressed by how efficient and flexible was EyeSeeTea. They consistently met deadlines and were always available to assist and adapt to our needs. Any issues that arose were addressed promptly. It was a very enjoyable and productive process. The conversations we had with EyeSeeTea were always stimulating and creative, and it was inspiring to see how they managed to find technical solutions to all the problems we discussed."
Ignacio Foche, director of EyeSeeTea, says that working with Lucia Fernandez was an incredible experience: “Lucia identified the problem, explained it to us, specifying exactly what the app would have to do, and accompanied us throughout the development process with regular testing. Not only that, but when we realised that more WHO departments had similar problems, Lucia was able to think beyond their needs as a department. She proposed to the others to join the development so that their needs could also be included in the app. Lucía’s ability to get different departments to join forces to develop the app was key to making MetaDatas Sync such a versatile tool.”
Why use MetaData Sync? Some use cases
One thing that shows how useful MetaData Sync is the fact that there are so many different ways to use it. Here at EyeSeeTea, we use it every day to keep development and production in sync. Almost every DHIS2 project has at least two instances, one for testing and one for production. Before MetaData Sync, syncing them up was really time-consuming and involved much scripting and manual tasks.
"This application has had an enormous impact on instance maintenance. MetaData Sync has saved us loads of time, work and headaches in keeping dev and prod instances in sync."
Sometimes, what we need is to change data types in the same instance. For example, for the same disease we may be collecting individual data through a programme in one district and aggregated data in another. Thanks to this application, we can send individual data to an aggregated dataset. Another case is when an international organisation works with data from different countries, and each country has an instance where they are collecting information with different types of data, but they have to be integrated in the same centralised instance. MetaData Sync lets you bring all this data together no matter how it’s been collected in each country.
“Another use case is to make metadata packages from one instance publicly shareable so that they can be installed on another instance, even with different versions of DHIS2. What’s more, every time a change is made to the packages, an update can be generated very quickly, notified and shared. Lucía Fernández explains an example of this functionality: “From the very beginning, the impact of MetaData Sync was very noticeable. I was able to generate metadata packages from our global instance to share with the ministries of health in the countries we worked with so that they could install them in their systems. With one click, the package was generated, and with another click, it appeared installed in the health ministry instance, with all the forms, indicators, dashboards, etc… It was ready to be used for data entry and visualisation. It was amazing. Before Metadata Sync, you had to prepare these packages manually, specifying many of the metadata dependencies. Now MetaData Sync managed the dependencies and connections between data elements and packaged everything important into a single metadata file that could be shared. This application saved me a lot of headaches.”
We may also want to transfer data between different DHIS2 instances. For example, an international organisation may have different instances implemented at different levels (regional, national) and need to transfer data between them. In these situations, differences in metadata and organisational units can make data exchange complicated. With MetaData Sync we can map the metadata in each instance and automatically apply the necessary transformations so that data can be sent from one to another.
"MetaData Sync is making DHIS2 much more accessible. Thanks to this app and other tools developed by EyeSeeTea, there are many things we can do without computer skills. We now have these user-friendly applications, where you can do very complex things with a few clicks on an interface. For people who don't have an IT background, this is very valuable, we are much less dependent on IT teams. And what's more, this translates into saving many thousands of dollars in development or consultancy."
How do you use MetaData Sync in your organisation? We’d love to hear about your experiences, ideas and suggestions. Your feedback helps us to improve MetaData Sync. And if you want to know how to get the most out of it, you can watch the tutorials, check out the documentation and sign up for our next webinar.
