Full Machine Learning Week 7-Track Agenda 2022 – Detailed Session Descriptions

In today's hyper-digital world, the data contained in documents often represent significant business value. The application of machine learning to extracting information from these sources is becoming big business. However, each use case represents different challenges in data extraction.This talk examines the application of intelligent document processing in the health insurance space. BCBS Tennessee Director of Data Science & AI Brandon Cosley will discuss how their Data Science Center of Excellence deployed an ensemble of machine learning techniques (e.g. DL, open-source, and NLP libraries) to extract information from documents in different business contexts. He will highlight the successes and challenges of each implementation while focusing on key findings associated with business success.

Blood, platelets and other transferable fluids are critical for patient health.  At PAW-2020 we described OneBlood’s use of analytics to optimize blood donor recruitment, to forecast hospital needs and to manage the blood supply chain during the Covid pandemic.  Now we provide an update with a focus on platelets.  We built and deployed three predictive models.  Marketing campaigns use these models, and dashboards enable campaign tracking and iterative improvements.  Inventory monitoring and hospital demand forecasting are the remaining solution components.  With all of these components working together, we have dramatically increased the number of platelet donors, stabilized inventory to match demand, and dramatically increased platelet availability in Florida hospitals.

Business partners are inundated with a non-stop barrage of how they need to capitalize on data and it's uses in order to stay relevant. As such data scientists and business partners spend ample time discussing the value of predictive modeling, use cases and potential ROI. Engaging and aligned as those early conversations can be, it is often a single conversation once the model is built and ready for deployment that can be the most problematic. In short...Just how accepting are business partners of the deployment changes necessary to a process for a predictive model to deliver the promised quantifiable value?Resistance to these changes ultimately turn into a self-fulfilling prophecy causing the appearance of failure to many modeling efforts. Which is why ensuring model deployment acceptance conversations occur throughout a model delivery life cycle is critical to overcoming the distrust, disinformation and general dislike of change a model deployment can create. Let's discuss several examples from a Fortune 500 financial services company's a call center's predictive modeling efforts, where model deployment acceptance impacted not only process integration, model development and benefit realization and the areas of opportunity that could have driven a different result.

In January 2020, just after the first case of COVID-19 was discovered in the US, NYU Computer Science Prof. Anasse Bari led with infectious disease medical doctor Prof. Megan Coffee a multidisciplinary team of AI and medicine experts both in the US and China to develop the first COVID-19 Clinical Severity Predictive Tool. The tool aimed to help medical doctors triage and provide care effectively during the incoming surges of cases by using algorithms that can predict which mildly ill patients were likely to become severely ill. In July 2021, the team developed another tool named COVID-19 Early-alerts Signals built on a digital epidemiology framework that analyzes alternative data sources to discover predictors of the pandemic curve, which could supplement traditional predictive models and inform early warning systems and public health policies. The research finds that online google searches can predict major regional increases and decreases in COVID-19 cases.  After the vaccine rollout, Prof. Bari and Coffee led a team that developed a Vaccine Hesitancy Analytics Tool which is a real-time big data analytics cloud application to track misinformation and extract themes and topics related to vaccine hesitancy. The platform was based on natural language processing and sentiment analysis predictive algorithms. The tool was deployed using Amazon Web Services. 

 
In this talk Prof. Bari will outline the experimental research results from the three tools his team developed: (1) COVID-19 Clinical Severity Predictor, (2) Pandemics Early-alert Signals Tool based on alternative data, and (3) Vaccine Hesitancy Analytics Tool. This talk will also highlight the analytics lessons learned and how we can better prepare for future pandemics using predictive analytics and algorithms.
 
* Prof. Anasse Bari led these projects and teams with medical doctor Prof. Megan Coffee, Dr. Matthias Heymann and other researchers from the NYU Courant Institute of Mathematical Sciences, the NYU Computer Science Department and the NYU Grossman School of Medicine. 

Additive manufacturing aka 3D Printing is fast becoming a viable option for final part manufacturing as material choices grow along with advancements in core printing technology. One of the key challenges is the ability to produce high quality parts with repeatability and to address this challenge there is a need for an automated part quality monitoring and prediction system.

In this session we present a system we developed that is designed to aid process engineers from designing a process, conducting Design Of Experiments(DOE), verifying the process, to deploying and monitoring the process in real/near real time.

Real/near real time monitoring of a process is based on the concept of a process digital twin. The process digital twin is a collection of machine learning models (supervised and unsupervised) working in unison to detect anomalies and generate alerts during the manufacturing process. Each of these models that make up the digital twin are developed based on the data generated (process parameters, telemetry and metrology) during the DOE and process verification phases and may be targeted at a subsystem. The models are continuously updated based on new data collected during the production phases. The system has been deployed internally for testing purposes.

Sales forecasting is a key process in defining a realistic revenue target for public and private companies and it’s a backbone of short term and long-term strategic planning. Despite the rapid growth of AI application across many business processes, Sales forecasting to a large extent has still been driven by human intelligence, a time-consuming effort with high likelihood of human error and significant inaccuracy. Many prior efforts in developing large-scale sales forecasting engines have not been successful mainly due to the lack of clear definition on what machines can tackle vs human. This presentation demonstrates the strategy and process that led to the development of a large scale AI driven sales forecasting engine in practice that impacted many business processes including Revenue Recognition, Commercial Planning, Product Marketing, Supply Chain, and Strategic Planning. 

It takes 10-13 years to design, manufacture and deliver a new aerospace product, which can inhibit growth for the industry as a whole. With a strained global supply chain, it’s more important than ever to make design and manufacturing processes more efficient in order to keep pace with forecasted demand in commercial travel. In this session, Guillermo Jenaro Rabadan, Project Executive of Project ADAM (Advanced Digital Design and Manufacturing) at Acubed, the Silicon Valley Innovation Center of Airbus, will discuss how to apply model-based engineering and digital technologies to manufacturing processes in order to  reduce lead times, production costs and improve workflows dynamically.

The size of the credit market in US Dollars is in the tens of trillions, providing credit to people, business, and governments. Traditional credit scoring has been the primary tool for assessing the risk of default and is an indispensable tool for lenders. Due to the decentralized, pseudonymous nature of cryptocurrency, the same credit scoring models used by traditional lenders aren't useful. There is, however, a wealth of data available, and an opportunity to leverage that data to better assess the risk of borrowers. This talk will explore that data, the challenges, and the opportunities for both lenders and borrowers.

Building and deploying predictive models for the COVID-19 pandemic was challenging and most of the models have not performed as well as hoped.  I cover five lessons learned from analyzing data by the Pandemic Response Commons, a not-for-profit that collects, analyzes and shares COVID-19 related data in the Chicago region.  I also look at the challenges of understanding COVID-19 health disparities and present the results of models showing the unequal impact of COVID-19 on different populations.  We conclude by discussing how regions can prepare for the future by putting in place persistent infrastructure for regional data collection, analysis and sharing.

In this session, an overview of Statistics and Machine Learning Algorithms with Supervised Learning (Logistic Regression, GLM Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, Neural Networks) and Unsupervised Learning (Z-Score, IQR, DBSCAN Clustering, Principal Component Analysis, etc.) will be provided.  Then, we will have a holistic comparison of each method at main analytics stages via two use cases. An insurance predictive analytics use case will be employed for the supervised machine learning comparison and a banking outlier detection analytics use case will be employed for the unsupervised machine learning comparison. Finally, the analytics result validation, implementation, and interpretability will be discussed and compared. Sample Python code will be shared.

Timely capacity planning of cardiothoracic ICU relies on early doctor’s recommendations on the type of bed a patient will need after surgery. A predictive model might be used to potentially help doctors make more accurate recommendations. Using the data of around 1500 cardiothoracic surgeries we built a gradient boosting model which predicts whether a patient will need a PACU (i.e. fast track trajectory) or an ICU bed with AUC=0.8 (precision=0.59, recall=0.84). In a hybrid scenario where the recommendation for PACU trajectory is made combining doctor’s recommendations and model’s predictions, the number of patients misclassified to PACU would be 1-in-10, which would be a significant improvement over the current 1-in-5.

Learn why digital transformation is an important revenue opportunity. The Obstacles in Operationalizing ML Workflow, the stability & scalability in Pipelines. Model Lineage, Versions of Libraries & Docker Images and the dependency based on roles such as Data Engineers & Data Scientists.

Certain types of insurance that are legally required for consumers, such as auto and home insurance, typically face a high amount of regulatory scrutiny. Insurers in the United States often must obtain state regulatory approval of their pricing models before being able to sell new insurance products or change pricing of existing products.  Beyond that, model users must feel assured that the variables and the relationships between variables used in the models are logical and intuitive, both to themselves, as well as to stakeholders affected by the model results. In this session, we’ll discuss best practices and case studies for analyzing and building regulator and user understanding and trust in machine learning models developed for insurance applications. We will also discuss how the data science industry is leveraging the rating and advisory organization model to obtain streamlined regulatory review and approval of complex models for use by insurance carriers.

Robust estimation of when, or if, a patient will refill a prescription requires accounting for intrinsic patient-level factors and environmental conditions, plus specific interventions focused on adherence. Bayesian multi-factor models provide a general means of incorporating data across all scales, accounting for the group-wise variation of arbitrary combinations of factors. Fitting robust and reliable Bayesian models, however, is difficult in practice, and much has been done recently to describe a "Bayesian workflow." We will demonstrate one approach to developing a Bayesianmulti-factor model using best-practice workflows to regularize our exploration and provide a concise and expressive model.

Long short term memory neural networks (LSTMs) originally rose to prominence in natural language processing, but have also shown value in time series. One drawback of LSTMs, and neural networks more generally, is that it can be difficult to derive effective confidence intervals. Two primary methods for calculating confidence intervals in LSTMs have been proposed: dropout-based simulation and estimating probability distributions.

In this case study we look at dropout-based simulation, which we found to be effective and more flexible than distributional estimation. However, added flexibility came at the cost of elevated computational burden.

Can automl be used to forecast the price of S&P 500 futures? Can various stock technical analysis indicators like moving averages, exponential moving averages, Bollinger bands, relative strength index etc. be used to forecast the S&P futures price for the next day?  In this session, Jiwani will share the results of case study that used various autoML tools like H2O automl, H2O Driverless AI, Rapidminer, and other automl software packages. He will examine the results from them all and go over the performance of each one. He will also reveal that some of these are overfitting.

Long-term health consequences of COVID-19 are symptoms that continue weeks or months after first diagnoses. Symptoms span respiratory, neurological, psychological, and cardiac problems and range from mild to debilitating.  Little is known about the risk factors contributing to long COVID, whether vaccines play a role or the best treatment options. UnitedHealth Group data represents millions of COVID-19 patients – some fully recovered and others that suffer from continued health consequences. Machine learning provides the opportunity to characterize these risk factors and predict probability that future disease will occur.

Value chains have been evaluated for decades. In this era of digital transformation, understanding the Data Science Value Chain is critical, but seldom is it examined as a system, nor re its component parts subject to systematic study. In this session, it will be shown how ML exists as a component within the value chain along with data acquisition, cleansing, formatting and accessibility. A conceptual case study aggregated from several non-specific sources shows the path from good data to the benefits of ML over traditional methods such as Designed Experiments. An overview of the 4.0 culture is integrated for a broader view of the benefit of ML within the value chain. Digital transformation's effects on the value chain are also integrated within the 4.0 culture. This presentation will highlight the myth that "everything data belongs to IT" by showing management and non-IT professionals the need for more knowledge about the data science value chain and where they fit within its constructs. Proposing a collaborative activity among the non-IT parts of the organization and an analytics maturity model expands where and how ML benefits decision making throughout the organization. The collaborative process also enhances communication of ML results to assist management in seeing beyond IT as a sole resource.

Safety National Casualty Corporation is the leader in Excess Workers' Compensation Insurance. Premium audit of Excess Workers Comp policies requires considerable resources and time at the end of each policy period, especially when the audit is conducted physically. To optimize the premium audit process, in collaboration with the audit and underwriting department, our data analytics team developed a set of predictive models, which leverages historical audit data and account information to predict future premium audit results. The prediction results have been applied to optimize the ordering of audits to collect more premium faster, selectively waive audits based on expected additional premium, and more efficiently allocate premium audit resources.

A key to meaningfully and sustainably accelerating patient  flow, improving quality, and saving caregiver time, is having the ability to spot situations and risks early, so caregivers and expediters can intervene in the moment.  Real-time, contextual information that is simple to digest and easy to access, is the currency with which to make this happen.  By combining clinical expertise, real-time and predictive analytics, and pre-defined action sets, Humber River Hospital in Toronto, Canada, is unlocking capacity, improving protocol compliance and reducing patients sent to ICU, while at the same time improving care team communication and reducing caregiver stress.

Water is fundamental to an effective society.  Individuals and organizations throughout society, including drinking water and wastewater systems, possess multifaceted relationships to water. Broadly, society seeks safe and abundant supplies of water while avoiding modern challenges of aging infrastructure, emerging contaminants, cybersecurity threats, legal compliance, customer satisfaction, climate change, ESG commitments, among other matters.  Machine learning (ML) applications continue to offer meaningful solutions for drinking water and wastewater systems and other organizations with a relationship to water.  The panel articulates the state of ML and its future implications in meaningfully addressing the challenges facing the water sector.

Effective capacity management for surgical departments includes models for prediction of surgery duration, ICU-bed type and length-of-stay. We used deidentified data from 3,000 cardio-thoracic surgeries to develop and validate predictive models of surgery duration that are based on random forest, extreme gradient boosting and linear regression.  The ensembled predictive model of acute cardio-thoracic surgery duration reduced the number of surgeries “behind-the-schedule” by 28% (from 60% to 32%) and boosted the surgery “on-time” by 15% (from 30% to 45%). Surgery planners could benefit from the predictive models by creating an optimized surgery schedule as a prerequisite to effective capacity management and improved patient and staff experience.

This case study presents how the biggest building material company worldwide increased sales and reduced inventory by predicting 80% of its stockouts a week ahead using ML. The Company faced high stockout levels in Brazil due to limited warehouse space against high safety stock requirements affected by COVID. No historic data would explain the new pandemic-influenced demand pattern. The Company gained operational flexibility by creating a reaction process to mitigate out-of-stock risks using stockout’s forecasts in a pragmatic, simple to run and highly comprehensible approach.A scalable data structure and pipeline was set up to give full operational visibility of the previous two years, such as daily stock levels, transit stocks, production plans, schedules, demand forecasts/actuals, committed order books, stockouts history and more. The team trained a gradient-boosting algorithm that uses tree-based learning for the supervised problem of classifying whether there will be a following week stockout for a given warehouse-product combination. To have consistent results across regions and products the algorithm was trained with different time horizons, comparing accuracy and identifying new variables to explain deviations. After two months, the initiative released an 80% accuracy algorithm and F1 over 0.7 for top 5 warehouses in volume and 87 products.

Transaction data has immense potential to go beyond traditional data aggregation by banks, to connecting the dots and providing valuable customer insights across industries. By acquiring financial data and then cleansing and enriching it, organizations can derive insights into customer needs and behavior to provide more meaningful interactions, identify lending opportunities, uncover current risks, provide competitive analysis, improve marketing efforts of a retail giant, and identify growth opportunities for clients. In this session, we will explore the importance of utilizing transaction data and applying machine learning algorithms to datasets to clarify and categorize the transactional data. Institutions can leverage this customer data to provide personal experience and advice.

We explored the feasibility of deep learning algorithms to improve the accuracy of predicting daily emergency hospital visits by tracking their spatiotemporal association with PM concentrations. We compared predictive accuracy of the models based on PM datasets from a single but more accurate air monitoring station in each district and multiple but less accurate monitoring sites within a district in Seoul, South Korea. We used MLP (multilayer perceptron) to integrate PM data from multiple locations and then LSTM (long short-term memory) models to incorporate the intrinsic temporal PM trends into the learning process. The results reveal evidence that predictive accuracy is improved from 1.67 to 0.79 in RMSE when spatial variations of air pollutants from multi-point stations are incorporated in the algorithm as a 9-day time window. The findings suggest guidelines on how environmental and health policymakers can arrange limited resources for emergency care and design ambient air monitoring and prevention strategies.

The rate of adoption for AutoML and MLOps solutions is incredible. Despite all of the great work being done to operationalize ML across industry there are two areas which still require custom work: feature engineering and product integration. The AutoML we run is only as good as the data it has to learn from. We'll be discussing a Spark based approach to automating the feature engineering portion of any MLOps solution. The result is an abstracted, extensible solution for the feature engineering portion of your AutoML or MLOps solution.

In today’s social environment, where responsibility to justice and fairness is being reconsidered vigorously, the insurance industry finds itself in the middle of the debate. For those who say insurers should do more to help balance society’s scales, the industry’s reply has been an insistence that actuarial science is colorblind. Underwriting and pricing are built only on socially appropriate factors that are predictive of loss. Factors such as race, ethnicity, religious practice, sexual preference, or national origin are directly excluded from consideration. This industry position has been challenged for many years as the use of territory and credit have been directly criticized as proxies for race and/or income. Factors such as credit based insurance score, gender, occupation and education, all very predictive of loss, have likewise found challenges and restrictions growing. Adding to the intensity of today’s debate is a growing insistence by many that being “colorblind,” on social justice issues is no longer enough. Some critics insist that insurers, along with the rest of society, must be actively involved in promoting justice and fairness.

We have reached a point in this debate where the status quo will no longer suffice without significant support. A new law in Colorado will require insurance companies to demonstrate that their prices and practices are not unfairly discriminatory, and more regulatory action is expected in other states. Given this is a new requirement, the discussions have begun to move from should we do this to how do we do this. This presentation will discuss various definitions of bias and discrimination in rating and insurance practices, methods and techniques that have been developed in data science to uncover unintentional bias, and how they techniques can be applied to insurance industry practices.

The ultrasound examination is one of the most common techniques of medical imaging. FOLLISCAN is an analytical and predictive system based on interpretable deep learning algorithms to support healthcare practitioners in ultrasound ovaries diagnostics - antral follicles examination. The counting of follicles is of major importance, e.g., it helps estimate the ovarian reserve. Moreover, a large number of antral follicles indicates polycystic ovarian morphology.

FOLLISCAN is designed for fertility clinics, diagnostic centers and, hospitals and will respond to their well-identified need, i.e. time and cost savings - through wider access to ultrasound diagnostics to make better and more informed clinical decisions. The results of the project address the need to perform ultrasound examinations in a faster, easier, more objective and accurate manner. 
In this talk, we will describe the process of building this solution and show the most important issues for creating a solution based on deep learning.
FOLLISCAN is currently tested and implemented in all INVICTA fertility clinics across Poland.

Ask our “Rockstars” anything about predictive analytics! Curious about machine learning tips, tricks, best practices and more? This is your opportunity to hear advice directly from the experts. The hardest part of machine learning can be getting models deployed into production. Our panel has done it, and is willing to spill the tea on how to make it happen. You’ll want to stick around for this ultimate session of the conference.