Federated edge learning can outperform the cloud in terms of privacy, speed, and cost

In 2000, the “cloud” began to rise. Developers and companies have started acquiring on-demand virtual computing resources to run their software and applications.

Over the past two decades, developers have become accustomed to and reliant on instant infrastructure managed and maintained by someone else. And it’s no surprise. Moving away from hardware and infrastructure allows developers and companies to focus primarily on product innovation and user features.

Amazon Web Services, Microsoft Azure, and Google Cloud have made storage and computing ubiquitous, on-demand, and easy to deploy. These hyperscalers have built solid, high-margin businesses based on this approach. Organizations using the cloud have traded capital expenditures (servers and hardware) for operational expenditures (pay-as-you-go compute and storage resources).

Enter federated learning

While the ease of use of the cloud is a boon for any budding team trying to innovate at any cost, cloud-centric architecture is a significant cost of revenue as the business scales. In fact, 50% of the revenues of large SaaS companies go to cloud infrastructure.

As machine learning (ML) grows in popularity and usability, organizations are storing more and more data in the cloud and training larger and larger models in search of greater model accuracy and greater user benefits. This further exacerbates dependence on cloud service providers, and organizations struggle to repatriate workloads to on-premise solutions. In fact, it would require them to hire an excellent infrastructure team and completely re-architect their systems.

Organizations are looking for tools that enable them to innovate new products and offer high accuracy with low latency while being cost effective.

Step to the edge of federated science (FL).

What is Federated Learning (FL) on the Edge?

FL, or collaborative learning, takes a different approach to data storage and processing. For example, while popular cloud-oriented ML methods send data from your phone to centralized servers and store that data in a silo, FL on the edge stores data on the device (i.e. mobile phone or tablet). This works as follows:

Step 1: Your edge device (or mobile phone) fetches the starting model from the FL server.

Step 2: After that, on-device training is carried out; the data on the device improves the model.

Step 3: Encrypted training results are sent back to the server to improve the model, while basic data is stored securely on the user’s device.

Step 4: With an on-device model, you conduct training and inference at the edge in a completely distributed and decentralized way.

This loop continues iteratively and the accuracy of the model increases.

Federated User Training Benefits

When data centralization does not rely on you or limit you to a bottleneck, there are huge benefits for the user. With FL on the edge, developers can improve latency, reduce network connections, and increase energy efficiency while promoting user privacy and better model accuracy.

FL on the edge is made possible by the ever-growing hardware capabilities of the phones in our pockets. Device computing performance and battery life improve every year. As the smartphone processor and the hardware in our pocket improve, FL techniques unlock more and more complex and personalized use cases.

Imagine, for example, privacy-oriented software installed on your phone that can automatically redact replies to incoming emails with individual tone, punctuation style, slang, and other hyper-personalized attributes – all you have to do is click send.

The pull of businesses is strong

In my conversations with many Fortune 500 companies, it became obvious to me how much demand there is for FL on the edge in various sectors. CTOs say they were looking for a solution to bring FL techniques to life on the edge. CFOs refer to the millions of dollars spent implementing the infrastructure and model that could otherwise be saved with the FL approach.

In my opinion, the three industries with the greatest potential to benefit from federated learning are finance, media, and e-commerce. I will explain why.

Use Case #1: Finance – Better Latency and Security

Many large international financial companies (Mastercard, PayPal) are eager to use FL on the edge to help them identify account takeovers, money laundering and fraud detection. More accurate models are on the shelf and have not been approved for release.

Why? These models increase latency enough to negatively affect the user experience – we can all think of apps that we no longer use because they took too long to open or crashed. Companies cannot afford to lose users for these reasons.

Instead, they accept a higher rate of false negatives and suffer from excessive account hijacking, money laundering and fraud. FL on the edge enables businesses to simultaneously reduce latency, demonstrating a relative increase in model performance over traditional cloud-centric deployments.

In the media sector, companies like Netflix and YouTube want to improve the accuracy of their suggestions on what movies or videos are worth watching. The Netflix award is famous for awarding $1 million for a 10% performance increase over its own algorithm.

FL on the edge can offer a similar effect. Today, when a new show starts or a popular sporting event (such as the Superbowl) is broadcast, companies limit the signals they collect from their users.

Otherwise, the sheer amount of data (at a rate of millions of requests per second) creates a network bottleneck that prevents them from recommending content at scale. With edge computing, companies can use these signals to suggest personalized content based on insights into individual users’ tastes and preferences.

Use Case #3: Ecommerce – More up-to-date and relevant suggestions

Finally, e-commerce companies and marketplaces want to increase click-through rates (CTRs) and increase conversions based on real-time feature stores. This enables them to re-rank customer recommendations and deliver more accurate predictions without delay over traditional cloud-based recommendations.

Imagine, for example, opening the Target app on your phone and receiving highly personalized product recommendations in a completely privacy-centric way – no identifying information would leave your phone. Federated learning can increase CTR with a more efficient privacy-aware model that offers users more up-to-date and relevant suggestions.

Market landscape

Thanks to technological advancements, both large corporations and start-ups are working to make FL more ubiquitous so that both businesses and consumers can take advantage of it. For companies, this probably means lower costs; for consumers, this can mean a better user experience.

There are already some early players in this space: Amazon SageMaker enables developers to deploy ML models primarily on edge devices and embedded systems; Google Distributed Cloud extends its infrastructure to the edge; and fledgling Nimbleedge companies are reimagining the infrastructure stack.

While we’re in the early rounds, FL on edge is here and hyperscalers are in the incumbent’s dilemma. Cloud providers’ revenues from computing, storage and data are at risk; Modern vendors that have adopted the Edge Computing architecture can offer customers the highest accuracy of the ML model and lower latency. This improves the user experience and increases profitability – a value proposition that cannot be ignored for long.

Neeraj Hablani is a partner at Neotribe Ventures focusing on early-stage companies creating breakthrough technologies.