How I Use AI Software to Determine What to WearI built a model to detect color trends from runway

Source: Malicki M Beser via Unsplash
For nearly two years now, I’ve been experimenting with image recognition models as a means of detecting and predicting fashion trends. The largest roadblock in this nerdy-yet-fashionable endeavor is training these AI models, to a high degree of accuracy, to detect a vast amount of items in photos.
Why so difficult? In order to train an image recognition model on Google’s user-friendly, AutoML tool, a minimum of ~100 images is best. However, what if the trend you’re trying to detect is new, or even micro, seen in just a couple of looks coming down the Versace runway? Suddenly, image recognition training becomes harder than manually going through and labeling images yourself.
Thankfully, there is one domain that does not require excessive ML training, and whose detection software is reusable regardless of the runway collection: colors.
So, I thought to myself, could there be an easy, ML-automated way to detect the values and quantities of colors in each runway collection? Is there an objective way to talk about colors in clothing?

The Software

Eager to find out, I put my hands down on the keyboard, and got to work.
One by one, I began sending images through Google’s Color Vision API through Python code developed in a Jupyter Notebook. I specifically extracted around 3–5 colors present in each image, as well as their prominence.
I immediately passed this data through some NumPy libraries and generated bar charts. Each color was represented as a bar with a height proportional to its frequency in an outfit. For example, take a look at the following nine looks from Saint Laurent’s Fall 2020 collection — completely represented as graphs of their colors’ data:
Looks from Saint Laurent’s Fall 2020 runway collection, represented by each outfit’s color & color frequency.
Detecting prominent color palettes, or determining what fashion trends are being pushed out each season, suddenly becomes something quantifiable, data-driven, and — dare I say — nerdy.

The Data

The difficulty with detecting colors in an image of a runway look is, of course, making sure the computer is only looking at the colors of the clothing. In each image, there is a model, there are accessories, and, in some cases, there is a colorful runway or setup. How can we filter out all of this background noise?
Let’s take a look at the Tommy Hilfiger TommyNow Fall 2020 collection — notably the “Tommy x Lewis Hamilton x H.E.R” collection. Models walked down a deep, mahogany-shade runway, which took up prominent space in each look’s photo. The first time I ran this collection’s images through my color recognition API, this strong, wooden runway color dominated the data.
So, how did I get rid of these unwanted data points? How did I ensure the computer was only looking at the colors of the clothing itself?
Look 3 from TommyNow’s Fall 2020 collection, via Vogue Runway, shown with a data graph representing the outfit’s color & color frequency.
If you have the time or the monthly subscription, you could, of course, manually remove all of the image backgrounds via Adobe Photoshop. For a small dataset, perhaps this is preferable. However, as your dataset grows, automation is key.
The great thing about computers is that they think very objectively. As in, when they detect colors, they assign each shade a quantifiable, numerical, RGB value. So, it’s a fairly simple task to detect which colors are, mathematically, in close proximity with each other.
To filter out Tommy’s runway from being captured in my data, I first obtained its RGB value. Then, in a couple of lines of code, I filtered out any colors the AI tool detected that were, RGB-wise, within 10% of this figure (mainly, to accommodate for small changes in its lighting and hues). Within seconds, my data was clean.

The Point

So, with my software model, I can go from a folder of hundreds of runway photos to quantifying the outfits’ colors, detecting palettes, and aggregating fashion trend insights, in solely a matter of seconds. I mean, why spend time wondering if your outfits are in style when you can just build software to tell you?

I’m a NYC-based software engineer, researching the intersection between fashion and technology. My roots are in Spain & Cuba. Read more of my stuff here, or check out my pride & joy, The Little Book of Big Fashion Data.