Agenda

Predictive modelers love building the models and then comparing them to determine the best model to deliver to the stakeholder. For Regression, the common metrics are R^2, mean squared error, root mean squared error, or mean absolute error. For classification, we usually see the confusion matrix as the basis for accuracy: Precision/Recall, Specificity/Sensitivity, and percent correct classification. These all have their place in our toolbox.

However, in many projects, if not in the majority of products, the business doesn’t care about any of these. The model is intended to increase revenue or minimize churn. If the analyst uses a standard metric, that modeler may optimize the standard metric but miss out on better models for the business. In this talk, alternative metrics will be explored that improve the effectiveness of the models operationally for the business.

While machine learning algorithms can produce valuable results, evaluating the performance of the resulting exercise can be a bit confusing - to the layman, that is. Often the recipient of the results will hear about Receiver Operator Characteristic curves or a KS statistic or a means squared error, or confusion matrices or some other foreign associated term. Frequently, managers and other users of predictive algorithms are not fluent in these concepts, and they might find themselves as confused as a confusion matrix might be to them. This session will focus on practical measures of performance evaluation designed for the non-quantitative executive, as well as the data scientist who needs to present the results of his efforts to less analytically inclined audiences. The end results are a simple scorecard measuring the ML algorithm performance.

Product recommendation is at the heart of Personalization group’s efforts to help Albertsons customers. Deep Learning has become the go-to approach for recommendation. Therefore, the group has begun to put efforts into applying Deep Learning to enhance new product recommendations. First, leveraging transaction data and product catalog, we built Customer DNA and Product DNA models. Customer DNA model captures customer characteristics such as purchase behavioral pattern, dietary preference, e-com affinity, customer location, etc. and embeds into a list (vector) of numbers. Similarly, Product DNA model captures product characteristics (e.g., is product organic and/or sugar-free?) and product-product associations—e.g., bread and peanut butter are usually purchased together. Second, we leverage these models to build a next generation recommendation system—inspired by the Wide and Deep recommendation model architecture. Our experiments building the framework have generated favorable results and we will share our journey from model conception to putting it in production to better serve our customers.

As John Deere continues to evolve as a technology company we must also look at bringing along existing systems and legacy data into this new world.  With a fresh slate we can re-evaluate "the way we've always done it" and look to enhance the data experience for users of all skill levels.  As we embrace cloud solutions for our data we can focus on things such as who should really govern data and why, how can we enable more users to load data that's important to them, and how can we reduce the barriers to entry for employees who are new to Data and Analytics and allow them to be successful.

At Northwestern we have developed a system that is built to consume and capitalize on IoT infrastructure by ingesting device data and employing modern machine learning approaches to infer the status of various components of the IoT system. It is built on several open source components with state-of-the-art artificial intelligence. We will discuss its distinguishing features of being Kubernetes native and by employing our work that enables features to be specified through a flexible logic which is propagated throughout the architectural components. We will discuss select use cases implemented in the platform and the underlying benefits. The audience will learn how to build or use a streaming solution based on Kubernetes and open source components.

MLOps provides a managed and optimized workflow for training, deploying and operating machine learning models and applications. More and more, ML models are now deployed at the Edge for use cases in manufacturing, logistics, healthcare and smart homes and cities. Edge deployments bring in unique challenges like resource constraints, limited bandwidth and unreliable networks. Existing cloud & enterprise MLOps needs to be adapted for Edge to overcome these constraints and maximize efficiency. This presentation will cover the unique challenges for machine learning and inference at the Edge, techniques and processes to overcome these challenges and how the MLOps workflow can be modified to adapt them. Data Scientists, MLOps Engineers and architects will benefit from understanding the unique challenges and best practices for ML at the Edge.

By 2025 more than 465,000 petabytes of data will be collected on a daily basis across the globe, however, only a fraction of a percent of this data is considered to be useful for analysis and models. In order to locate the most useful attributes, feature engineering is a vital skill for all data scientists and analysts. But how does a data scientist decide how to engineer the right features? 

In this session, Brandon Southern explores:

• A framework for thinking like a business owner
• Developing a product and customer mindset
• Building new features that have saved millions of dollars in annual costs for top organizations

Did my data change after a certain intervention? This is a common question with data observed over time. Classical statistical and engineering approaches include control charts to see if the series falls outside of the normal boundaries of expected data. A Bayesian approach to this problem calculates the probability that the data series changes at every point along the series. Bayesian change point analysis allows the analyst to evaluate a whole series and look where the highest probability of change occurred. Has the financial asset lost value after the recent financial report? Are the healthcare outcomes at this hospital better after our new process to help patients? Did the manufacturing process improve after upgrading the machinery? All these questions and more can be answered with these techniques.

The Data-verse is an ever expanding space. It is the task of data science to explore the fundamentals of this space. The tools and techniques being employed are having incredible impact across many different business verticals. One interesting area of exploration is to take concepts  from Physics and apply them to data science problems. Physics at its essence focuses on reducing the complex physical world into a set of fundamental laws or axioms that govern it. In this talk, I will explore examples from 3 different business use cases where Physics served as inspiration in solving complex problems. The first example comes from marketing and product recommendations and adopts the concept of time dilation from relativity to solve a low recommendation problem. The second comes from IOT GPS transportation devices and uses the concept of spacetime and equivalence classes to characterize motion behavior. The last concept comes from logistics and developing an estimated time of arrival model using a Gradient Boosting Regressor. This focuses on using a “predictive motion” for model convergence.

Today's AI tools can produce highly accurate results, yet advanced machine learning models often remain a black box. Transparency is essential when businesses are relying on AI more than ever. Explainability needs to be part of the equation when building AI systems we can trust. Organizations should start by including explainability as one of the key principles within their responsible AI guidelines by applying FEAT (Fairness, Ethics, Accountability and Transparency) for building responsible AI products. Through this talk, I will discuss how we can benefit from AI and minimize risk by making ML models explainable for businesses.