Blog

How Do The World's Largest Companies Design Machine Learning Applications?

I'm often asked "what kind of machine learning project should I be working on?"

And I usually respond with "follow your curiosity."

Why?

Because because machine learning is so experimental, you might as well figure things out by tinkering. Testing things that might not work.

However, machine learning projects are no longer works of magic. The device you're reading this on is probably using machine learning in a number of different ways that you're not aware of (see Apple's implicit machine learning below).

That said, this issue of ML Monthly (April 2021 issue) collects different design best practices from companies using machine learning on a global scale.

And after reading them, you will begin to notice that there are many coincidences in how things are done. This is a good thing. Because overlaps are what you can use for your own projects.

As machine learning code and models become more and more reproducible, you'll notice a general theme here: Machine learning is an infrastructure problem.

Which is something you already knew all along: "How can I get data from one place to another as quickly and efficiently as possible?"

If you are thinking of working on your own machine learning projects, read each of the guidelines below and try the materials in the bonus section, but remember that none of them will replace the knowledge you gain by experimenting yourself (guidelines, schmuidelines).

Apple’s Human Interface Guidelines for Machine Learning

I am writing these lines on an Apple MacBook in a library where I can see at least 6 other Apple logos. This morning I saw two people in front of me pay for their coffee with their iPhones.

Apple devices are everywhere.

And they all use machine learning in many different ways, to improve photos, to preserve battery life, to enable voice searches with Siri, to suggest words for quick typing.

Apple's Human Interface Guidelines for Machine Learning share how they think and how they encourage developers to think about using machine learning in their applications.

They start with two high-level questions and break them down from there:

What is the role of machine learning in your application?

What are the inputs and outputs?

As for the role of machine learning in their application, they wonder if it is fundamental (necessary) or complementary (nice). Is it private or public? Is it visible or invisible? Is it dynamic or static?

For the ins and outs (I'm a big fan of this analogy because it's similar to the ins and outs of an ML model) they discuss what a person will put into their system and what their system will show them.

Does a person give the model explicit feedback? I mean, does it tell your model if it's right or wrong? Or does the system collect implicit information (information that does not require the person to do any additional work other than using the application)?

Google People and AI Research (PAIR)

Google's design principles for AI can be found in its People and AI Research (PAIR) guide.

The PAIR guide is also accompanied by a great glossary with the different machine learning terms that you will find in this field (there are many). It breaks down the design of an AI project into six sections.

User needs + Defining success

Where is the intersection between what AI is capable of and what the people who use your service need?

Should you automate (eliminate a painful task) or augment (improve) with AI?

What is the ideal result?

Data collection and evaluation

Convert a person's requirements into data requirements (it all starts with data)

Where does the data come from? (Are they sourced responsibly?)

Build, fit and tune your model (good models start with good data)

Mental models (setting expectations)

What does one person think their ML system can accomplish?

Explanation + confidence

AI systems are based on probability (and can give strange results), how can this be explained?

What information should a person know about how an artificial intelligence model made a decision? (confidence levels, "we showed you this because you liked that ...")

Feedback and control

How can a person give feedback to help their system improve?

Errors + Funny Failure

What is a "mistake" and what is a "failure"? (a self-driving car stopping at a green light could be a mistake, but running a red light could be a mistake)

ML systems are not perfect and your system will end up failing, what do you do when it does?

Each section comes with a worksheet to practice what you have learned.

One trend you'll notice after reviewing the guidelines (especially PAIR) is setting expectations. You have to be very frank about what the system is capable of. If a person expects their system to be magical (as ML is often presented) but is not aware of its limitations, they may be disappointed.

Microsoft Design Guidelines for Human-Artificial Intelligence Interaction

Microsoft's design guidelines for human-artificial intelligence interaction address the problem in four stages:

Initially (what should a person know when using their system for the first time?)

During the interaction (what should happen while a person uses your service?)

When you're wrong (what happens when your system is wrong?)

Over time (how does your system improve over time?)

You will see that Microsoft's guidelines lead you to put yourself in the shoes of a person who uses their ML system. And again we see a trend.

Problem → Create solution (ML or not) → Set expectations → Allow feedback → Have a mechanism for when it is wrong → Improve over time (go back to the beginning).

Facebook Field Guide to Machine Learning

While the previous resources have taken the approach of a global ML system, Facebook's Field Guide to Machine Learning focuses more on the modeling aspect.

Their video series divides a machine learning modeling project into six parts:

Definition of the problem: what problem are you trying to solve?

Data: what data is available?

Evaluation: what defines success?



Characteristics: What characteristics of the data best fit your measure of success? Model: Which model is best suited to the problem and the data you have?

Experimentation: how can the previous steps be repeated and improved?

But as the modeling aspect of machine learning becomes more accessible (thanks to pre-trained models, existing code bases, etc.), it's important to consider all the other parts of machine learning.

The 3 principles of Spotify to design products with ML technology

How do you build a service that provides music to 250 million users around the world?

You start by being manual before being magic (principle 3) and continually ask the right questions (principle 2) to identify where the frictions of your service users lie (principle 1).

The sentence above is a pun on Spotify's three principles for designing products based on machine learning.

Principle 1: Identify friction and automate it

Any point where a person has a hard time reaching their goals while using your service can be considered friction.

Imagine a person searching for new music on Spotify and finding nothing to suit their tastes. This could hurt someone's experience.

Spotify realized this and used machine learning-based recommendation systems to create Discover Weekly (which I'm currently listening to), a playlist that gets refreshed with new music every week.

And in my case, it seems like they must have followed their other two principles while creating it, because these songs that I'm listening to are a hit.

Principle 2: Ask the Right Questions

Question. To ask. To ask. If you don't know, you may end up designing a product in the wrong direction.

Like many of the other steps in the guidelines above that challenge you to think from the point of view of the person using your service, this is the point of asking the right questions: find out what problems your customers are having and see if you can solve them. using machine learning.

Principle 3: Go manual before going magic

Have you found a source of friction?

Can you solve it without machine learning?

How about you start with a heuristic (an idea of how things should work)?

For example, if you were Spotify and you tried to create a new music playlist that someone would be interested in, how would you classify something as new?

Your starting heuristic could be that anything older than 30 days would not be classified as new.

After testing multiple heuristics and hypotheses (a manual process), you could revisit whether or not machine learning might help. And thanks to your experiments, you would do it from a very well-informed point of view.