A program to play “Set”

index_1.gif

Justin Pearson
Sep 21, 2017

index_2.gif

The big function

index_3.gif

Demo

index_4.gif

index_5.gif

index_6.gif

index_7.gif

index_8.gif

index_9.gif

import 0.393928
parse 0.073856
find sets 0.002175
display 0.413222

index_10.gif

Demo 2

index_11.gif

index_12.gif

index_13.gif

index_14.gif

index_15.gif

index_16.gif

import 0.262394
parse 0.076477
find sets 0.005584
display 0.53886

index_17.gif

How it works

Table of contents

1. Import image
2. Parse image
   2a. Select cards from image
   2b. From each card, parse its blobs:
       Count blobs
       Color
       Shape
3. Find Sets
4. Display Sets

Import Image

We shrink the image to make it run faster.

index_18.gif

index_19.gif

Parse image

Select cards from image

A Label Matrix for an image is a matrix of non-negative integers the same size as the image, where each integer represents a different blob, and 0 represents the background.

For example, here is the label matrix for an image with two blobs. (We have to negate the color because white means blob and black means background.)

index_20.gif

index_21.gif

You can use ArrayPlot or Colorize to display a label matrix as a picture where colors represent different integers:

index_22.gif

index_23.gif

index_24.gif

index_25.gif

Here is the label matrix for our image.

index_26.gif

index_27.gif

It is a big matrix, same size as the image:

index_28.gif

index_29.gif

index_30.gif

Use Colorize to see the different regions:

index_31.gif

index_32.gif

We use the label matrix to indicate regions of interest in an image. We call these regions “components”.

The ComponentMeasurements function provides information about an image's components.

For each component, we retrieve its picture (“MaskedImage”) and the coordinates of the corners of its bounding box (“BoundingBox”).

The 3rd argument is a function which serves to keep only large and circular components.
The 4th argument specifies how we want the output formatted.
The function is quite fast.

index_33.gif

index_34.gif

From each card, parse its blobs

Next we parse the blobs out of each card and look at the things important for Set:
(1) how many blobs
(2) what color are the blobs
(3) what shape are the blobs

Here is an example card:

index_35.gif

index_36.gif

For each card we make an association (dictionary) to contain information about it:

index_37.gif

index_38.gif

We compute a Label Matrix for this card to analyze its blobs.

index_39.gif

index_40.gif

For each component (symbol on the card), we compute:
(a) its picture (“MaskedImage”),
(b) its median color (“Median”),
(c) the fraction of the component that covers its bounding disk (“BoundingDiskCoverage”).

Property (c) will be useful for figuring out the shape; fraction near 1 means very circular, fraction near 0 means very spiky.

We're careful to only consider components that don't touch a border, because we don't want to include the components on the 4 edges of the image.

index_41.gif

index_42.gif

Count blobs

We add to our dictionary the number of blobs:

index_43.gif

index_44.gif

Color

For each blob on the card, classify its color as Red, Dark Green, or Purple -- whichever its median color is closest to.
Then pick the commonest color over all the blobs as the color of the whole card. (Essentially allow the blobs to “vote” on the color.)

index_45.gif

index_46.gif

index_47.gif

index_48.gif

index_49.gif

Shape

It turns out the bounding-disk coverage is a good predictor of a blob’s shape.
With a little trial and error, I found some thresholds.
Again, we have them vote on which shape.

index_50.gif

index_51.gif

index_52.gif

index_53.gif

index_54.gif

index_55.gif

Blobs’ BoundingDiskCoverage

Save the pictures of the blobs and the BoundingDiskCoverage, for later analysis.

index_56.gif

index_57.gif

End result of analyzing one card

index_58.gif

index_59.gif

Analyze all cards

Roll this up and run it on all cards:

index_60.gif

It’s quite fast:

index_61.gif

index_62.gif

Here’s the data we calculated for each card:

index_63.gif

card count color shape box blobCoverages
index_64.gif 3 index_65.gif oval {{258.,263.},{325.,330.}} index_66.gif
index_67.gif 3 index_68.gif squiggly {{168.,263.},{234.,329.}} index_69.gif
index_70.gif 2 index_71.gif oval {{348.,260.},{416.,328.}} index_72.gif
index_73.gif 2 index_74.gif oval {{77.,255.},{141.,320.}} index_75.gif
index_76.gif 2 index_77.gif diamond {{165.,171.},{230.,237.}} index_78.gif
index_79.gif 2 index_80.gif squiggly {{78.,170.},{141.,235.}} index_81.gif
index_82.gif 3 index_83.gif diamond {{263.,168.},{330.,234.}} index_84.gif
index_85.gif 3 index_86.gif oval {{353.,161.},{421.,228.}} index_87.gif
index_88.gif 1 index_89.gif oval {{167.,81.},{232.,146.}} index_90.gif
index_91.gif 3 index_92.gif oval {{80.,75.},{143.,140.}} index_93.gif
index_94.gif 1 index_95.gif squiggly {{260.,72.},{326.,138.}} index_96.gif
index_97.gif 1 index_98.gif diamond {{353.,60.},{421.,127.}} index_99.gif

Real quick, let’s analyze how well “BoundingDiskCoverage” can predict shape.
You can see in the graph that a BoundingDiskCoverage of 0.4 differentiates well between “diamond” and “squiggly”, and 0.5 differentiates well between “squiggly” and “oval”, although one green oval is mis-classified due to a brightness problem.

index_100.gif

Graphics:\"BoundingDiskCoverage\" predicts shape well.

Find Sets

Here are 3 cards. Are they a Set?

index_102.gif

index_103.gif
index_104.gif
index_105.gif

This triplet is a Set if, for each category of ‘count’, ‘color’, ‘shape’, they all share the same value, or all have different values.

Group by value:

index_106.gif

index_107.gif

Tally up each category:

index_108.gif

index_109.gif

For each category, there should be either 3 entries or just 1.

index_110.gif

index_111.gif

So it’s not a set.

Roll it up into a function:

index_112.gif

Find all sets:

index_113.gif

index_114.gif
index_115.gif
index_116.gif
index_117.gif
index_118.gif
index_119.gif
index_120.gif
index_121.gif

Display Sets

Use the bounding boxes of each set’s cards to highlight each set:

index_122.gif

index_123.gif

Created with the Wolfram Language