What can artificial intelligence and image processing tell us about coins? Zachary Taylor of the American Numismatic Society gave a walkthrough in April of 'CADS'.
I was intrigued by the ideas of calibrating feature extraction by sizing a circle and throwing away everything outside the circle. I hadn’t thought of it myself — the die study I am attempting is on a series where the die is larger than the coins. It seems like an ingenious hack given how feature extraction works. Taylor also suggested that the images should be cleaned up — I think he mentioned removing backgrounds — but if the software removes everything except the center there is no need to remove backgrounds at all.
Greek coins, and many Roman coins, suffer from dies that degrade faster near the edges than the center. Only using the middle of the coin may help to keep degrading dies "together" in the tree (dendrogram) view later.
Taylor next showed setting the “blur radius” until the image looks “cartoonish”. He called the CADS kind of blurring “median blur”. This seems like a good idea. I suspect it could be done automatically with a small bit of work rather than requiring a human operator. Taylor said this was to remove circulation wear. I suspect this also makes the analysis more resistant to photographic and lighting concerns.
The next thing is what he calls “feature detection”. CADS uses OpenCV, which has been under development for a couple of decades. Taylor says using feature detection came from Huapeng Su’s earlier ANS CADS prototype. A 'feature' is just a contrast pattern of light and color, not something a human would call a 'feature' like a helmet decoration or nose shape. OpenCV tutorials suggest features represent “corners” — small regions on the image where the contrast changes a lot, and changes differently from nearby regions.
The coins CADS has been tested on seemed to be all VF-XF condition. I will be curious to see how it does with coins with a wider range of conditions. I was wondering if the blur could be different for each coin, with less blur being applied to lower grade coins.
The next step is calculating the “distance metric” between all coins. I suspect the computational complexity is N^2 to the number of coin images. He described this as “brute force Hamming method”, with the “20 best matches condensed into 1”, “agglomerative nesting” with “no risk of overfitting”. Taylor said CADS uses Oriented FAST and Rotated BRIEF (not SIFT). I am not a computer vision guy and only know the meaning of some of those terms … but OpenCV provides a library for all of these things: https://docs.opencv.org/master/d1/d89/tutorial_py_orb.html . Taylor’s work seems to be choosing to use that library and tuning the parameters rather than implementing a distance calculation from scratch.
Taylor said the algorithm to compute the distance metric takes “a few hours” for 3000 owls and he showed it taking less than a minute for about 100 Paphos tests. This also suggests the performance is N^2.
The next piece is the dendrogram graph, which looks nice. (It might just be the dendrogram feature of the Javascript library D3 https://www.d3-graph-gallery.com/dendrogram ?)
Taylor described the dendrogram sub-trees as representing dies if tuned correctly. I suspect his algorithm will also group coins from the same die by “die state”.
The “card view” letting the user quickly flip between all the children at a point in the dendrogram looked good. I suspect the card view would be greatly improved if the images were rotated and scaled so that the matched features of all coins on the card line up as close as possible.
After Taylor's demo, Alan Stahl showed Princeton’s work on the topic. He had previously shown this work at the New York Numismatic Club. Princeton’s approach has at least two key differences.
Stahl’s team used a complex physical technique to get the 3D height of the object. It is a great technique but it may not get traction because of the difficulty performing it and getting access to the coins. Yet it is worth pointing out that the CADS approach has only been tested on (over?)-cleaned silver coins. Bronze coins and coins with a lot of toning may not work directly with CADS — photographs of plaster casts may needed to get good results from CADS if patina variations overwhelm “feature detection”. If that is true then the Princeton approach not look so difficult compared to CADS for those series.
Stahl was concerned about the file size of his images. I suspect he could just use the calculated “height” as a greyscale pixel values and store his images as black-and-white JPEGs. That would reduce the image size to normal and I suspect it would let him use Taylor’s workflow directly.
Princeton had humans annotate manually some interesting points on the coins. This is labor intensive! Yet manual annotation might have some merit being used in conjunction with the Taylor approach. It might be that the CADS approach has a hard time “connecting” worn coins, partly reworked dies, or lighting techniques. Perhaps after CADS has reduced the 3000 obverses to a hundred “dies” the Princeton approach of manually marking points could be used on CADS-identified dies to find further matches that CADS missed.
I was disappointed that after all of these years the ANS and Mr. Taylor only managed to look at two series of ancient coin types. Taylor said this isn’t a problem because the “learning” is unsupervised.
