Published on Thu Sep 17 2020

Large-Scale Intelligent Microservices

Mark Hamilton, Nick Gonsalves, Christina Lee, Anand Raman, Brendan Walsh, Siddhartha Prasad, Dalitso Banda, Lucy Zhang, Lei Zhang, William T. Freeman

Deploying Machine Learning (ML) algorithms within databases is a challenge. This article introduces an Apache Spark-based micro-service orchestration framework. This allows users to integrate ready-to-use intelligence into any datastore with an Spark connector.

0
0
0
Abstract

Deploying Machine Learning (ML) algorithms within databases is a challenge due to the varied computational footprints of modern ML algorithms and the myriad of database technologies each with its own restrictive syntax. We introduce an Apache Spark-based micro-service orchestration framework that extends database operations to include web service primitives. Our system can orchestrate web services across hundreds of machines and takes full advantage of cluster, thread, and asynchronous parallelism. Using this framework, we provide large scale clients for intelligent services such as speech, vision, search, anomaly detection, and text analysis. This allows users to integrate ready-to-use intelligence into any datastore with an Apache Spark connector. To eliminate the majority of overhead from network communication, we also introduce a low-latency containerized version of our architecture. Finally, we demonstrate that the services we investigate are competitive on a variety of benchmarks, and present two applications of this framework to create intelligent search engines, and real-time auto race analytics systems.

Tue May 26 2015
Machine Learning
MLlib: Machine Learning in Apache Spark
MLlib is Apache Spark's open-source distributed machine learning library. MLlib provides efficient functionality for a wide range of learning settings. Shipped with Spark, MLlib supports several languages and includes several underlying primitives.
0
0
0
Wed May 22 2019
Machine Learning
The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development
Machine Learning Bazaar is a new framework for developing machine learning and automated machine learning systems. We use these components to create a general-purpose, multi-task, end-to-end AutoML system. We demonstrate 5 real-world use cases and 2 case studies of our approach.
0
0
0
Thu Jan 21 2021
Machine Learning
Clairvoyant Prefetching for Distributed Machine Learning I/O
I/O is emerging as a major bottleneck for machine learning training, especially in distributed environments. At large scale, I/O takes as much as 85% of training time. NoPFS uses clairvoyance to predict when and where a sample will be accessed.
2
1
6
Tue Apr 04 2017
Machine Learning
AMIDST: a Java Toolbox for Scalable Probabilistic Machine Learning
The AMIDST Toolbox is a software for scalable probabilistic machine learning with a focus on streaming data. The toolbox supports a flexible modeling language based on Probabilistic graphical models with latent variables and temporal dependencies.
0
0
0
Tue Sep 16 2014
Artificial Intelligence
ICE: Enabling Non-Experts to Build Models Interactively for Large-Scale Lopsided Problems
The human teacher guides the machine towards accomplishing the task of interest. The machine can access, process, andsummarize more examples than the teacher can see in a lifetime. Both the teacher and the machine continuously learn and benefit from the interaction.
0
0
0
Sat Oct 20 2018
Artificial Intelligence
MMLSpark: Unifying Machine Learning Ecosystems at Massive Scales
Microsoft Machine Learning for Apache Spark (MMLSpark) is a system of enhancements that expand the Apache Spark distributed computing library. It tackles problems in Deep Learning, Micro-Service Orchestration,Gradient Boosting, Model Interpretability, and other areas of modern computation.
0
0
0