Design Status

This is a prototype.

Purpose

This web app uses computer vision to identify the three highest probability diagnoses for a skin lesion. A more advanced version could be used to identify cancerous skin lesions early.

Lesion Types

The following descriptions were taken from this research paper.

nv
Melanocytic nevi are benign neoplasms of melanocytes and appear in a myriad of variants, which all are included in our series. The variants may differ significantly from a dermatoscopic point of view.

mel
Melanoma is a malignant neoplasm derived from melanocytes that may appear in different variants. If excised in an early stage it can be cured by simple surgical excision. Melanomas can be invasive or non-invasive (in situ). We included all variants of melanoma including melanoma in situ, but did exclude non-pigmented, subungual, ocular or mucosal melanoma.

bkl
"Benign keratosis" is a generic class that includes seborrheic ker- atoses ("senile wart"), solar lentigo - which can be regarded a flat variant of seborrheic keratosis - and lichen-planus like keratoses (LPLK), which corresponds to a seborrheic keratosis or a solar lentigo with inflammation and regression [22]. The three subgroups may look different dermatoscop- ically, but we grouped them together because they are similar biologically and often reported under the same generic term histopathologically. From a dermatoscopic view, lichen planus-like keratoses are especially challeng- ing because they can show morphologic features mimicking melanoma [23] and are often biopsied or excised for diagnostic reasons.

bcc
Basal cell carcinoma is a common variant of epithelial skin cancer that rarely metastasizes but grows destructively if untreated. It appears in different morphologic variants (flat, nodular, pigmented, cystic, etc) [21], which are all included in this set.

akiec
Actinic Keratoses (Solar Keratoses) and intraepithelial Carcinoma (Bowen’s disease) are common non-invasive, variants of squamous cell car- cinoma that can be treated locally without surgery. Some authors regard them as precursors of squamous cell carcinomas and not as actual carci- nomas. There is, however, agreement that these lesions may progress to invasive squamous cell carcinoma - which is usually not pigmented. Both neoplasms commonly show surface scaling and commonly are devoid of pigment. Actinic keratoses are more common on the face and Bowen’s disease is more common on other body sites. Because both types are in- duced by UV-light the surrounding skin is usually typified by severe sun damaged except in cases of Bowen’s disease that are caused by human papilloma virus infection and not by UV. Pigmented variants exists for Bowen’s disease [19] and for actinic keratoses [20]. Both are included in this set.

vasc
Vascular skin lesions in the dataset range from cherry angiomas to angiokeratomas [25] and pyogenic granulomas [26]. Hemorrhage is also included in this category.

df
Dermatofibroma is a benign skin lesion regarded as either a benign proliferation or an inflammatory reaction to minimal trauma. It is brown often showing a central zone of fibrosis dermatoscopically [24].

Image format

The app accepts images in jpg or png format.

Limitations

The Deep Learning model that powers this app is not skilled enough to consistently assign the highest score to the correct lesion. Although, the model is trained good enough to detect melanoma better than others, its still not that good to be used in real life. Also, during testing, the correct lesion was regularly among the top 3 predicted lesions.

Note: The model was not trained using photos that were taken with a mobile phone. Therefore, the model's prediction accuracy could be affected by variations in the quality of mobile phone images.

Published Design

The design code and the step-by-step process used to train and test the model has been published on Kaggle. You can find the open source notebook here. The test results are also available in the notebook.

Dataset Licence

The model was trained using image data that was released under a CC BY-NC-SA 4.0 licence. That means that this app cannot be used for commercial purposes.

---

Will I need to wait for the model to download every time I use the app?

No. Once the model downloads it will be cached in memory. The next time you visit this site you won’t need to wait.

That said, I suggest that you always use this tool with your web browser set to incognito mode. The model won’t be saved, but this will ensure that each time you visit this site you will always use the most up to date version of the model and not the old one that's stored in your device's memory.

What about patient data privacy?

Your images are never sent to an external server. Any image you submit stays on your computer or mobile phone. This is because the model is running on your device. This is also why a batch of images can be analyzed in less than two seconds.

The tech that makes this possible is called Tensorflow.js. It was created by Google.

What do the decimal numbers mean?

They are probability scores between 0 and 1. They show how confident the model is that a particular condition is present on the image.
For example, mel, Melanoma: 0.231 means that the model is 23% sure that the lesion shown in the image is melanoma.

How do I use this app on a computer?

- Go to this site on your computer.
- Click the orange button.
- Select one image or multiple images and click ‘Open’.
- The app will print results for each image.

How do I use this app on a mobile phone?

You can take a new photo or you can submit photos that are stored on your phone. The steps below apply to Android. iOS may be similar but I'm not sure.

[ 1 ] Take a photo

- Go to this site on your phone.
- Tap the orange button.
- Select “Camera”
- Take a photo
- Tap the "tick"
- The photo will be sent to the app for analysis.

[ 2 ] Use images from your photo folder

- Go to this site on your phone.
- Tap the orange button.
- Tap "Files"
- Tap the image name
- To submit a batch of images, tap each image (tick) to select it and then tap "Open".

What would it take to make this tool more trustworthy?

Three things are needed:
- Collaboration with a dermatologist/pathologist.
- More high quality labeled training data.
- Field testing.

AI-based Espert Systems can be brittle. Good training scores don't guarantee that a model will perform well under real-world conditions. This tool first needs to be field tested. Then the model needs to be re-trained to make it more robust.