OHTN Case Study
The Ontario HIV Treatment Network (OHTN) are doing something ingenious with Penelope smart forms.
The Toronto-based organization, a collaborative network of community-based agencies, people with HIV, health care providers, government policy makers, educators and researchers which supports more than 30 AIDS Services Organizations (ASOs) across Ontario, is in the midst of a transition that will see each ASO in the province exclusively use Penelope smart forms to document their programs and services, and track all activity by their workers.
Initial Penelope implementation
OHTN first purchased Penelope for Ontario ASOs in 2008, with the intention of allowing each agency the chance to configure and use Penelope their own way.
One of the team members at OHTN involved in the initial implementation of Penelope was Carlos Joseph, a development and training specialist with OHTN.
Formerly a software implementation specialist in the corporate banking sector, Carlos was hired to help implement and configure Penelope for each ASO.
Starting with a pilot implementation at two ASOs in downtown Toronto, the project expanded across Ontario, with all 31 implementations completed by 2010.
Benefits of Penelope
Given that some ASOs were coming from a paper-based system, the benefits to using Penelope were immediate, said Carlos.
"With everyone on the same database, for some it put structure into their day," he said. "(With Penelope), they had a way of recording all the work that they did more permanently, rather than just in a book or a notepad or in a file folder. And also for the managers or (executive directors), it gave them an idea of what work was being done, and the numbers of things that were being done. And then they could investigate the things that seemed to be reported incorrectly - some of it was that maybe we had something categorized incorrectly at the back end and some of it could be the workers recording something in a way that maybe they were mistaken and it was just being recorded in a different way, so we found and were able to streamline a lot of things that way."
The original implementation plan was to essentially customize the implementation for each agency, to "make it just like what they were doing currently, not change the way they were working," said Carlos.
Given that OHTN consists of over 30 community agencies, however, complications perhaps inevitably ensued. Dmitry Rechnov, the manager of the evidence-based practice unit at OHTN, picks up the story from there.
"We ended up, unfortunately, with bad data, simply because agencies called the same things differently, all the drop-down menu choices, wording of cart items they'd been using, it was the same but different," he said. "So we ended up having not an ideal result, because basically the system plays two roles for them - it is their case management software and, at the same time, it is the tool which provides them with the data they need for reporting."
Added Carlos: "A lot of things got recorded in slightly different ways. They'd be doing the same work, sitting side by side, but they'd be recording that same thing slightly differently. Or they'd be recording too many details about that thing, or not enough details about that thing, and so from agency to agency and even from worker to worker, the information was different."
Using Penelope smart forms for data collection
In order to ensure uniform data collection, Dmitry and Carlos have been heavily involved in overseeing a switch to using Penelope's innovative and highly configurable smart forms to log all activity at each ASO.
The transition means moving from using events and service units in Penelope to "completely running the whole system on the documents," said Dmitry. "We developed customized documents and this is the new way the system is being set up now."
Added Carlos: "We're making everything the same across the province. We're coming out with a set of definitions, for everyone to use the same definitions, and by doing that, it enables us to use a document and now the document also has the capability of guiding the user through what they need to complete. Depending on their earlier choices (when filling out the document), they can complete the necessary things later on."
The decision to use Penelope's powerful documents functionality to ensure uniform collection of data at ASOs across Ontario was the result of an extensive process analysis, Dmitry said.
"The way we started building the documents was actually us looking at the work process. We tried to put ourselves in the workers' shoes and we thought, 'OK, what happens when this person comes to work? What is this person actually doing? Which question is this person asking of their clients?' And we basically built the documents in the language which is currently being used by the agencies, with all the choices, drop-down selections, structures (and) elements reflective of actually the work process which is happening day by day," he said.
He added: "We completely, completely changed the system. This transformation which we're going through right now, to me, is equal to the initial setup."
End-to-end configuration of Penelope
OHTN employs two full-time staff to support the ASOs in their use of Penelope on a day-to-day basis, while also assigning staff to configure the system for each agency, from setting up services, configuring user-defined fields and setting up security settings, to training people on how to use the system on an initial and ongoing basis.
They also help with data extraction from the system in the form of custom reports, said Carlos.
"I think the original intention when they were shopping around was that each agency would be shown how to, for example, first and foremost, administer their own system as a system administrator - add new users, set up security classes, change passwords, reset passwords, all that stuff - and out of the 30, we didn't have too many that could do that. So a lot of my job ended up being a sys admin here for them. Then, too, only a couple were able to use the ODBC, download their data and work it up in an Excel file, so our office here ended up being the ones who would work the data and manipulate it for them, and set up charts and graphs for them as well, so that just became our function. I guess our expectation was that they would be doing certain things when we went into this originally, when it was proposed, but effectively when it went live, we had to change our expectations somewhat."
Working with Athena Software
This experience has come in useful because, as a result of moving to using documents exclusively, they have also been working closely with Athena staff on custom queries and reports to be able to extract massive amounts of data from the completed forms.
They currently have two large reports being worked on in collaboration with Athena's technical staff, and are using Athena's professional services to create custom reports that will become part of Penelope for each ASO.
Dmitry said the experience of working with the Athena technical team has been "very positive. With all this transition, we realized that we pushed you guys out of your comfort zone a little bit," he said with a laugh, "simply because of these things which we asked you to develop for us."
Carlos added: "I've learned a lot working with Athena. Every single staff member there has something new to teach me."
Dmitry said OHTN plan to roll out the document-based system across each ASO completely by the end of their next fiscal year, with each stage already scheduled in and planned.
The first phase of the rollout has already been completed, with non-client-related activities falling under the Education, Community Development and Outreach banner already being tracked in a new document within Penelope at all 31 agencies.
The next step is to roll out the organization's support services document, which will capture data on all client-related activities. Carlos said this form is currently being piloted at six agencies, ranging in size from two staff members to 25, with the pilot period ending in April 2015.
Though the transition is a long-term project, Dmitry said the benefits are already being realized for staff and funders alike.
"The concept of transforming the system and using the documents is basically a long-term thing, because right now I know that the system is capable of doing pretty much everything," he said.
"The funders are already asking, 'OK, can we bring the next bunch of users, another bunch of programs, can we start recording data from the Hepatitis C program, which is a separate branch of the Ministry of Health and Long-Term Care, shall we consider bringing anonymous HIV testing sites and offering them to use the system,' so the prospect for the funder has completely changed already. Even at this point (where) we haven't yet even produced a single report, just the user end which we demonstrated and the user interface which we built convinced the funder that the system is capable of doing pretty much everything."
Download a two-page summary of the OHTN case study to share with your colleagues!
Or visit the OHTN website to learn more about OHTN and ASOs.