Software Development

HealthPulse AI Leverages MediaPipe to Improve Well being Fairness

Spread the love

A visitor publish by Rouella Mendonca, AI Product Lead and Matt Brown, Machine Studying Engineer at Audere

Please observe that the data, makes use of, and functions expressed within the under publish are solely these of our visitor authors from Audere.

About HealthPulse AI and its software in the actual world

Preventable and treatable illnesses like HIV, COVID-19, and malaria infect ~12 million per yr globally with a disproportionate variety of circumstances impacting already underserved and under-resourced communities1. Communicable and non-communicable illnesses are impeding human improvement by their detrimental affect on schooling, revenue, life expectancy, and different well being indicators2. Lack of entry to well timed, correct, and reasonably priced diagnostics and care is a key contributor to excessive mortality charges.

As a result of their low price and relative ease of use, ~1 billion speedy diagnostic checks (RDTs) are used globally per yr and rising. Nonetheless, there are challenges with RDT use.

  • The place RDT information is reported, outcomes are exhausting to belief as a result of inflated case counts, lack of reported anticipated seasonal fluctuations, and non-adherence to remedy regimens.
  • They’re utilized in decentralized care settings by these with restricted or no coaching, growing the danger of misadministration and misinterpretation of check outcomes.

HealthPulse AI, developed by a digital well being non-profit Audere, leverages MediaPipe to handle these points by offering digital constructing blocks to extend belief on the earth’s most generally used RDTs.

HealthPulse AI is a set of constructing blocks that may flip any digital answer right into a Fast Diagnostic Check (RDT) reader. These constructing blocks remedy distinguished world well being issues by enhancing speedy diagnostic check accuracy, lowering misadministration of checks, and increasing the supply of testing for situations together with malaria, COVID, and HIV in decentralized care settings. With only a low-end smartphone, HealthPulse AI improves the accuracy of speedy diagnostic check outcomes whereas mechanically digitizing information for surveillance, program reporting, and check validation. It offers AI facilitated digital seize and end result interpretation; high quality, accessible digital use directions for supplier and self-tests; and requirements based mostly real-time reporting of check outcomes.

These capabilities can be found to native implementers, world NGOs, governments, and personal sector pharmacies by way of an online service to be used with chatbots, apps or server implementations; a cell SDK for offline use in any cell software; or immediately by way of native Android and iOS apps.

It allows revolutionary use circumstances similar to quality-assured digital care fashions which allows stigma-free, handy HIV dwelling testing with linkage to schooling, prevention, and remedy choices.

HealthPulse AI Use Circumstances

HealthPulse AI can considerably democratize entry to well timed, high quality care within the personal sector (e.g. pharmacies), within the public sector (e.g. clinics), in group packages (e.g. group well being employees), and self-testing use circumstances. Utilizing solely an RDT picture captured on a low-end smartphone, HealthPulse AI can energy digital care fashions by offering beneficial determination assist and high quality management to clinicians, particularly in circumstances the place strains could also be faint and exhausting to detect with the human eye. Within the personal sector, it might automate and scale incentive packages so auditors solely must overview automated alerts based mostly on check anomalies; procedures which presently require human critiques of every incoming picture and transaction. In group care packages, HealthPulse AI can be utilized as a coaching instrument for well being employees studying how you can appropriately administer and interpret checks. Within the public sector, it might strengthen surveillance techniques with real-time illness monitoring and verification of outcomes throughout all channels the place care is delivered – enabling sooner response and pandemic preparedness3.

HealthPulse AI algorithms

HealthPulse AI offers a library of AI algorithms for the highest RDTs for malaria, HIV, and COVID. Every algorithm is a set of Laptop Imaginative and prescient (CV) fashions which are educated utilizing machine studying (ML) algorithms. From a picture of an RDT, our algorithms can:

  • Flag picture high quality points widespread on low-end telephones (blurriness, over/underexposure)
  • Detect the RDT sort
  • Interpret the check end result

Picture High quality Assurance

When capturing a picture of an RDT, it is very important be sure that the picture captured is human and AI interpretable to energy the use circumstances described above. Picture high quality points are widespread, notably when photos are captured with low-end telephones in settings which will have poor lighting or just captured by customers with shaky fingers. As such, HealthPulse AI offers picture high quality assurance (IQA) to establish adversarial picture situations. IQA returns issues detected and can be utilized to request customers to retake the picture in actual time. With out IQA, shoppers must retest as a result of uninterpretable photos and expired RDT learn home windows in telehealth use circumstances, for instance. With just-in-time high quality concern flagging, extra price and remedy delays may be prevented. Examples of some adversarial photos that IQA would flag are proven in Determine 1 under.

Images of malaria, HIV and COVID tests that are dark, blurry, too bright, and too small.
Determine 1: Photos of malaria, HIV and COVID checks which are darkish, blurry, too vibrant, and too small.


With simply a picture captured on a 5MP digicam from low-end smartphones generally utilized in Africa, SE Asia, and Latin America the place a disproportionate illness burden exists, HealthPulse AI can establish a selected check (model, illness), particular person check strains, and supply an interpretation of the check. Our present library of AI algorithms helps lots of the mostly used RDTs for malaria, HIV, and COVID-19 which are W.H.O. pre-qualified. Our AI is situation agnostic and may be simply prolonged to assist any RDT for a spread of communicable and non-communicable illnesses (Diabetes, Influenza, Tuberculosis, Being pregnant, STIs and extra).

HealthPulse AI is ready to detect the kind of RDT within the picture (for supported RDTs that the mannequin was educated for), detect the presence of strains, and return a classification for the actual check (e.g. constructive, detrimental, invalid, uninterpretable). See Determine 2.

Figure 2: Interpretation of a supported lateral flow rapid test.
Determine 2: Interpretation of a supported lateral circulate speedy check.

How and why we use MediaPipe

Deploying HealthPulse AI in decentralized care settings with unstable infrastructure comes with quite a lot of challenges. The primary problem is a scarcity of dependable web connectivity, typically requiring our CV and ML algorithms to run regionally. Secondly, telephones accessible in these settings are sometimes very outdated, missing the newest {hardware} (< 1 GB of ram and comparable CPU specs), and on totally different platforms and variations ( iOS, Android, Huawei; very outdated variations – probably not receiving OS updates) cell platforms. This necessitates having a platform agnostic, extremely environment friendly inference engine. MediaPipe’s out-of-the-box multi-platform assist for image-focused machine studying processes makes it environment friendly to satisfy these wants.

As a non-profit working in cost-recovery mode, it was necessary that options:

  • have broad attain globally,
  • are low-lift to keep up, and
  • meet the wants of our goal inhabitants for offline, low useful resource, performant use.

While not having to put in writing a variety of glue code, HealthPulse AI can assist Android, iOS, and cloud gadgets utilizing the identical library constructed on MediaPipe.

Our pipeline

MediaPipe’s graph definitions enable us to construct and iterate our inference pipeline on the fly. After a person submits an image, the pipeline determines the RDT sort, and makes an attempt to categorise the check end result by passing the detected result-window crop of the RDT picture to our classifier.

For good human and AI interpretability, it is very important have good high quality photos. Nonetheless, enter photos to the pipeline have a excessive degree of variability we’ve little to no management over. Variability elements embrace (however should not restricted to) various picture high quality as a result of a spread of smartphone digicam options/megapixels/bodily defects, decentralized testing settings which embrace differing and non-ideal lighting situations, random orientations of the RDT cassettes, blurry and unfocused photos, partial RDT photos, and lots of different adversarial situations that add challenges for the AI. As such, an necessary a part of our answer is picture high quality assurance. Every picture passes by way of quite a lot of calculators geared in direction of highlighting high quality issues which will stop the detector or classifier from doing its job precisely. The pipeline elevates these issues to the host software, so an end-user may be requested in real-time to retake a photograph when crucial. Since RDT outcomes have a restricted validity time (e.g. a time window specified by the RDT producer for the way lengthy after processing a end result may be precisely learn), IQA is important to make sure well timed care and save prices. A excessive degree flowchart of the pipeline is proven under in Determine 3.

Figure 3: HealthPulse AI pipeline
Determine 3: HealthPulse AI pipeline


HealthPulse AI is designed to enhance the standard and richness of testing packages and information in underserved communities which are disproportionately impacted by preventable communicable and non-communicable illnesses.

In direction of this mission, MediaPipe performs a crucial function by offering a platform that enables Audere to shortly iterate and assist new speedy diagnostic checks. That is crucial as new speedy checks come to market recurrently, and check availability for group and residential use can change ceaselessly. Moreover, the flexibleness permits for decrease overhead in sustaining the pipeline, which is essential for cost-effective operations. This, in flip, reduces the price of use for governments and organizations globally that present providers to individuals who want them most.

HealthPulse AI choices enable organizations and governments to profit from new improvements within the diagnostics area with minimal overhead. That is an integral part of the first well being journey – to make sure that populations in under-resourced communities have entry to well timed, cost-effective, and efficacious care.

About Audere

Audere is a worldwide digital well being nonprofit creating AI based mostly options to handle necessary issues in well being supply by offering revolutionary, scalable, interconnected instruments to advance well being fairness in underserved communities worldwide. We function on the distinctive intersection of worldwide well being and excessive tech, creating superior, accessible software program that revolutionizes the detection, prevention, and remedy of illnesses — similar to malaria, COVID-19, and HIV. Our various staff of passionate, revolutionary minds combines human-centered design, smartphone know-how, synthetic intelligence (AI), open requirements, and the most effective of cloud-based providers to empower innovators globally to ship healthcare in new methods in low-and-middle revenue settings. Audere operates primarily in Africa with tasks in Nigeria, Kenya, Côte d’Ivoire, Benin, Uganda, Zambia, South Africa, and Ethiopia.

1 WHO malaria truth sheets

Leave a Reply

Your email address will not be published. Required fields are marked *