In January 2020, I spent a week in Ethiopia with a team of medical professionals from my church and a sister church in Boston. It was an amazing week, and the environment was quite different from my “normal” in the States. On the return trip, as the plane was descending into Pittsburgh, I was shocked to see residential neighborhoods with large, beautiful houses, after a week of houses being very small and made of corrugated metal.

Anyway, our team was there to run a clinic. I had been told I might be able to work on a tech project, but had pretty low expectations. This post is about how that did, in fact, materialize.

Background

Ethiopia ACT serves the sick and poor in Addis Ababa. It was started in 2002, at the height of the HIV crisis. Today their ministry still helps with HIV/AIDS, but has expanded to include other social care as well: For instance, they have met the financial needs of hundreds of families and have helped hundreds of families become self-sufficient. When I met the founder, Andy Warren, I was impressed by the data-driven decision-making approach he champions. He told me that measurable impact is one of their ministry values.

Because data is so important to their ministry, Ethiopia ACT runs a database server in their main office with records for families. One of their staff members, Danny, does most of the database administration, and he had asked our team to work with him to enable social workers to collect data in the field with donated Android phones and sync that data to the database. Previously the social workers were using paper forms. So Bill, a Pittsburgh colleague, and I got to work thinking about how to make this possible.

The initial vision was simple enough: Capture the patient ID, a couple metrics related to HIV/AIDS, and the date. As I understood the problem domain better and sketched a solution, I suggested other changes to their data model to get more value from the work we were doing. I convinced Danny that in addition to tracking HIV metrics over time, we could get even more value by having a table of visits. Working together, we came up with the following additional fields:

• Date of visit
• Social worker
• Location (both latitude and longitude, and What3Words address)
• Follow-up actions required
• Notes on the visit

Since then, the project has expanded to capture additional data, but this illustrates some of the work in improving the data model.

One key detail I didn’t realize on my first day, that became apparent throughout the week: Supporting offline use would be critical. The electric and Internet infrastructure in Ethiopia is extremely unreliable, so a good solution would need to not only allow data to be collected offline, but also support the database server being offline frequently.

About the Tech

Ethiopia ACT uses FileMaker to run its database. Bill and I were unfamiliar with it, and briefly considered other options, as I wondered if we could find something open source that would be more robust and easier to integrate, but the ease of maintaining FileMaker by their staff and of creating UI’s to manipulate the data would be hard to replicate. We decided to leave it.

Ethiopia ACT also uses Fulcrum to gather data from the field. With Fulcrum, anyone can make a survey form, and push it to mobile devices. The app works great offline and can sync data to Fulcrum’s servers when it comes back online. This worked great for some projects, but for the data they track in FileMaker, there was no obvious way to connect the two systems.

Exploring the “How”

After a couple days of investigating, we not only decided to stick with FileMaker, but also decided that Fulcrum was probably the best option (or a good-enough option) for the mobile data collection. I did spend a hot minute tinkering with Expo, but that wouldn’t have supported easy maintenance. (Frontend friends: If you think running npm install is awful over high-speed Internet, you should really try it in Africa sometime. 😃)

Thus the high-level plan became:

• Use Fulcrum’s webhooks feature to watch for form submissions
• Use FileMaker’s REST API to add the data

With that plan, we decided to try integrating with FileMaker first: There were more moving pieces with that – authentication, pushing to multiple tables, figuring out how to even enable the API (we had to upgrade FileMaker to get support for it), etc.

For the FileMaker integration, I looked at two libraries primarily: fmrest (Python) and fms-api-client (JavaScript). Although I’m better at Python, the JavaScript package was much better documented, so I decided to go with it. It took a few tries to understand the API, but overall a proof-of-concept of pushing dummy data into the server was pretty simple.

Final Architecture

Here’s the data flow we landed on:

• Data is collected in a Fulcrum form.
• Once the data is submitted, it triggers a webhook that pushes the data to a Google Cloud Function.
• The Google Cloud Function transforms the data and saves it in Firestore. Running this Cloud Function would be basically free, but we decided to also integrate with What3Words to take the latitude and longitude of the form submission and look up the “address” of the beneficiary.¹ So the Cloud Function uses VPC access to talk to the What3Words API.
• A script is run on the database server to pull data from Firestore and push it into Fulcrum. It processes one document at a time, and pushes the data into FileMaker, deleting documents as they are successfully processed.

Tech Stack Summary:

• Google Cloud Platform: Google Cloud Function, Firebase, and Serverless VPC.
• Languages and Frameworks: Node.js for both the Google Cloud Function and the database server script, with Jest for unit testing, and fms-api-client to talk to FileMaker.

Battle Scars

Every tech project has at least a few foibles. Some of the ones we encountered:

• FileMaker doesn’t have a programmatic way to manage migrations. Migrations are literally just someone changing the columns through a UI. Between some cleanup I wanted to do (spelling standardization, for instance) and the changes needed for the project, we had quite a list, and no simple way to test it all, although we could (and did) make copies of the database file. Shout-out to Bill for managing this part of the project! 🙏
• I thought it might be a good idea to try running a reverse SSH proxy from the database server to a cloud server, and let the Fulcrum webhook push data directly through to the database server. I wanted to try it because it would have made the overall integration easier, but it proved infeasible with the Internet reliability issues, and I also knew it would be better to have a data queue in the cloud that we could pull from when we were online.
• Setting up CI/CD – the Cloud Function deploys, the test runners, it all takes a few tries – but that’s normal.

Setting up Bitbucket CI/CD for a side project, as seen by commit history:

- Start on Pipeline
- Continue working on deploy strategy [skip ci]
- Overhaul Pipeline build
- Fix pipeline
- Enable Docker for second step

... and it's probably not over yet.

— Alex Watt (@alexcwatt) December 21, 2020

Some Wins

• GitLab CI/CD: Having nearly 100% test coverage, and an automated test and deployment pipeline, was a big win overall. Danny has been able to make changes to the code with greater confidence, and has been able to write tests for new features modeled on the existing tests.
• VS Code’s Live Share is fantastic for collaboration. The audio quality was better than WhatsApp, and once we got setup, Danny and I were able to pair program like we were sitting next to each other, despite being 7000 miles apart.
• Google Cloud’s Firestore is a simple document database, and the Node SDK is a breeze. I was glad for the transaction support because as part of the data modeling, I decided to have the incoming Fulcrum records split into multiple documents in Firestore – basically one per object. I wanted to write all of these documents together, or none at all, and transactions enabled that.

Summary

What I thought would be a small project actually became a pretty mature software project. I needed some additional time after getting back to polish it, and since then Danny has also wanted to extend it to handle more fields. I’ve been mostly hands-off for this – I review his code, and we’ve had lots of conversations about programming, version control, testing, etc., but he has been the one making the changes.

As I think about the impact, I am really excited that Ethiopia ACT can use this project to better serve hundreds of families in Addis Ababa. It’s also been very exciting to see the skills Danny has developed from continuing on this project (e.g., Git, VS Code, Node.js, etc.) and to get to continue to work with him, including our occasional early morning calls over the past year (early morning in Pittsburgh is afternoon in Africa).

Danny also told me that this project has made the field workers more connected with the administrators and managers, which was encouraging to hear. I look forward to seeing how the Lord uses this in 2021 and beyond!

---

In the long tradition of writing a “Hello World!” program to learn a new language, I decided to title this post about writing a program in (and for) Africa, “Hello Africa!”