13 Chatbot trends and statistics you can’t afford to miss in 2021!

Machine learning is the technology behind the development of chatbots which involves:

• A computer program that learns and evolves
• Natural language processing (NLP) to mimic human-generated text and language

Before the advent of the internet, face-to-face meetings and phone calls dominated the communication landscape. Years later, online forms, mobile apps, social media, and emails have taken over as modern forms of communication.

A multitude of industries employ chatbots that help visitors navigate through a website or answer their questions. Given the highly competitive market, customers have increased expectations from brands.

Incorporating chatbots help meet those expectations. Also, AI is constantly evolving, which means that chatbots will become more sophisticated in the future. Hence, we will discuss AI chatbot trends in 2021 that you should consider adopting in your strategy. 

Let’s brush up on your knowledge of chatbot statistics 2021.

Key Chatbot Statistics

Before going into the chatbot statistics for 2021 and beyond, it’s essential to look at the state of the conversational AI market overall.

Conversational AI Market 

Research suggests that the spending on will surpass $77 Bn by 2022. The forecast for 2019 was $35.8 Bn, which means that it will likely double by next year. A third of that spending will be incurred on software.

Besides, a significant chunk of software spending will go towards AI applications such as chatbots and personal assistants (nearly $14.1 billion). 

Apart from that, the market is anticipated to experience a stellar growth at 30.5% CAGR (compound annual growth rate) between 2017-2022. In terms of market size, North America leads the global conversational AI market.  

However, when it comes to CAGR, Asia Pacific (APAC) will experience the fastest growth during the forecast period. Besides, the adoption of conversational AI software is widespread in North America.

An increasing government spending, large number of chatbot companies, and rising investments in artificial intelligence and machine learning are responsible for the adoption.

Chatbot Forecasts & Predictions

Listed below are key chatbot forecasts and predictions, divided by year:

2021

• About 50% of businesses will increase their annual expenditure on Chabot creation compared to conventional mobile app development.
• Chatbots are expected to see a whopping investment of $4.5 billion by 2021.
• The implementation of artificial intelligence in customer service interactions is anticipated to rise by 400% between 2017-2021.
• AI will manage approximately 85% of customer interactions by 2021. 

2022

• Businesses across different domains would save nearly $0.70 per customer interaction, thanks to chatbots (CNBC). 
• About 90% of customer interactions with banks would be automated using chatbots by 2022 (Juniper Research). 
• Chatbots are projected to save almost $8 billion annually for businesses by 2022. 
• About 70% of white-collar employees would regularly interact with chatbots.

2023

• Businesses and customers will save 5 billion hours on interactions because of chatbots by 2023. 
• Chatbot eCommerce transactions are projected to surpass a value of $112 billion by 2023.
• The banking, retail, and healthcare sectors will save nearly $11 billion a year by employing chatbots in customer service.
• The global chatbot market is expected to observe a double-digit growth at 34.64% CAGR between 2017-2023. 

2024

• The global conversational AI market is forecasted to reach $15.7 billion at 30.2% CAGR by 2024. 
• AI will redefine 50% of user experiences using natural language, computer vision, augmented reality, and virtual reality. 
• The global chatbot market is predicted to observe an impressive growth at 34.75% CAGR between 2019-2024. Moreover, it will cross $7.5 billion by the end of 2024.

2025

• Nearly 95% of customer interactions (online conversations and live telephone) will be taken over by artificial intelligence by 2025. However, at this point, it will be pretty tricky for customers to differentiate a chatbot from a human agent. 
• Businesses that leverage AI to automate their customer engagement will observe a 25% increase in their operational efficiency. 
• The annual revenue of the global AI software market will cross $118 billion by 2025. 

2026

• By 2026, the global chatbot market will touch $10.8 billion with an impressive CAGR of 30.9% during the 2018-2026 forecast period. 

These statistics indicate that chatbots are the future. However, with further advancements in AI, only capable and intelligent conversational AI platforms shall persist. 

In the next section, we will look at the chatbot trends more closely.

Chatbot Trends in 2021

Given the many advantages of chatbots, rising chatbot trends, and the ever-increasing popularity of machine learning (ML) and artificial intelligence, adopting new technologies has become business-critical.

Here are the top 13 chatbot trends in 2021 that you should consider embracing:

1. Payments via Chatbots

First on our list of chatbot trends is payment gateways. Several payment services, including Paypal, have incorporated chatbots in their payment gateways and digital wallets in 2020.

This shows that the application of chatbots is no longer limited to just customer service. For example, users simply have to type “pay electricity bill” in this application.

The  would walk them through the process until the final payment is made. 

2. Voice Bots

Voice-driven search, thanks to the advent of conversational AI, is taking the world by storm. It is the next big thing. According to a report by Accenture, most consumers prefer voice-based interfaces over text-based interfaces on messaging platforms.

The trend of utilizing conversational bots to assist consumers over both voice and text is on the rise. In sectors such as education, travel, and insurance, voice bots would be of great help. 

It’s all about providing a seamless user experience!

3. Chatbots with Human Touch

Chatbots use artificial intelligence to suggest options when a customer types in their inquiry. Year after year, they are becoming increasingly sophisticated in terms of mimicking human conversation. 

They will identify the right intent behind a query and provide an accurate response to that query. By learning from interactions, chatbots are beginning to pick up patterns in user behavior. 

Hence, chatbots are becoming indispensable for high-impact conversations.

4. Chatbots with Emotional Intelligence

This is slowly becoming a reality with newer, groundbreaking technology. For instance, facial feature detection AI software can detect the feelings of a person.

Similarly, chatbots with emotional intelligence can figure out your mood (happy, sad, or angry) by looking for patterns in your text. It could be capitalization or punctuation or your voice to predict your emotions.

Conversational AI with soft skills will further humanize the interaction between businesses and their customers. 

5. Chatbots based on Natural Language Processing

Chatbots driven by NLP are groundbreaking for businesses that have more excellent customer service scenarios. They can determine the user intent and generate responses accordingly.

They learn from past conversations with the customer to provide accurate answers. The Royal Bank of Scotland (RBS), for instance, has already incorporated NLP-powered chatbots to enhance their customer experience.

Check out this blog to learn more about how custom chatbots can help you improve customer satisfaction.

In 2021, you will observe more and more organizations adopting  to improve conversations. 

6. Analytics and Insights with Chatbot

To measure your performance, you need to track and evaluate data. And for that, you need chatbot analytics and insights. The good news is most chatbots today offer analytics so that you can continue to improve your strategies.

They keep a record of:

• every question asked
• every answer delivered
• every query transferred to agents

Use this information to improve your product or service. Therefore, chatbots allow you to foster strong customer relationships. 

7. Multilingual Capabilities

Next on our list of chatbot trends is multilingual capabilities. Did you know that only 20% of the world’s population speaks English? So if you plan to expand your business globally or already cater to a global customer base, you need chatbots with multilingual capabilities. 

This will not only offer a boost to your localization efforts but also amplify your reach. But, again, it’s because customers prefer to use their native tongue to communicate with their trusted brands. 

Multilingual bots enable you to tap into new markets and personalize the experience for your audience.

8. Supporting Remote Workforce

A recent report by Gartner says that about 75% of business leaders would allow 5% of their employees to work from home post-pandemic permanently.

Apart from that, 25% agreed to permanently shift 10% of their employees to the remote work setting. Hence, it’s safe to say that such a work culture is here to stay.

In such a scenario, chatbots would become pivotal in answering typical employees’ concerns. 

They can:

• answer FAQs on remote work policies
• track employee health
• notify employees of the latest changes in policies
• offer work from home tips

9. Chatbots on Social Media

Social media is the hub of much social interaction in present times. We have moved on from just making friends on social media to voicing opinions, ordering products and services, offering reviews, and even getting in touch with businesses. Thus, it becomes a necessity for businesses to use chatbots to facilitate interaction on these platforms.

Many industry leaders in various sectors have already employed chatbots to use this vital resource to better understand the customer needs and even improve ways that the business can help the consumers.

Facebook already has a chatbot feature, but it is very limited in its capabilities; perhaps it was only a test to whether chatbots would fare well on the platform. The answer is a resounding yes. Facebook has now become a trendsetter in equipping businesses with the ability to use customized chatbots made by other parties to help in this process. Every social media platform is likely to follow suit.

10. Chatbots Built with Low Code Platforms

The pandemic forced companies to become more agile in their processes. This, in turn, led to the development of low-code chatbot platforms that allow businesses to deploy apps at a much faster rate.

Even less experienced users can build chatbots using these low-code development platforms.

Many organizations have deployed chatbots for sales support, customer support, service desk management, and more with this approach.

11. Recruiting and HR Chatbots

A recruiting chatbot can filter candidates, answer their basic questions, and schedule them for interviews. As a result, they speed up the entire recruitment process.

These chatbots are helpful even after the hiring process is over. During onboarding, they can answer the most basic HR questions. This is undoubtedly a boon for large organizations with massive workforces.

12. Self-Learning Chatbots

To stay ahead in the race, it is of utmost importance to train bots with new data and keep them up-to-date. In 2021, you can expect companies to make chatbots that are self-learning.

This means companies do not have to spend time feeding new data to bots. They will analyze the pattern in every interaction and train themselves to keep the users or customers engaged. In other words, bots will learn to improve their response capabilities based on user feedback.

13. Text Messaging with Chatbot

While every other mode of communication, like email and phone calls, still holds ground, the most personal still remains – a personal message. SMS and WhatsApp are the go-to apps that people regularly check and are comfortable conversing over.

In 2021 and beyond, chatbots are going to be leaning onto this opportunity to better connect with audiences. Many companies like Yatra and MakeMyTrip already use certain chatbot features to send flight tickets directly to Whatsapp and the details via SMS. This has made the process convenient for users, and any progress would only make things easier.

In 2021, you will see SMS and create a personalized experience and facilitate open-ended conversations.

The Future of Chatbots

In a rapidly maturing conversational AI market, haphazardly placed chatbots will vanish. Only the ones that are strategically developed and implemented across channels shall survive.

Across all digital platforms, millions of users will prefer voice-enabled conversational AI to interact with an enterprise. Such is the future of chatbots. Moreover, various chatbot applications will no longer remain siloed.

Businesses would be able to create an intranet of chatbots that can share information and work together seamlessly. In the meantime, chatbots will continue to boost user engagement and enhance customer support. 

This article was originally published here

Source Prolead brokers usa

The risk of using ‘half-baked’ data to address the challenges posed by COVID-19

As COVID-19 rampages across the globe it is altering everything in its wake. For example, we are spending on essentials and not on discretionary categories; we are saving more and splurging less; work-life balance has a deeper focus on mental health; we are staying home more and traveling less. Our priorities have changed. If you look at this unfolding scenario wearing a data hat, the facts and knowledge we relied upon to forecast markets have been rendered practically useless.

We saw evidence of this in late March when the pandemic took root in western nations. There was a surge in demand for toilet paper, fueled by panic-buying, leading to an 845% increase in sales over last year.[i] The most powerful analytic engines belonging to the world’s largest retailers could not forecast the demand. Reason: models used by analytical engines are trained on existing data and there are no data points available to say, “Here is COVID-19; expect a manic demand for toilet paper.” Businesses know that their investments in digital technology turned out to be the silver lining in the new normal, but they also learnt that depending on the current stockpile of data can lead to blind spots, skewed decisions and lost opportunities.

While the pandemic will leave a profound impact on how the future shapes up, it is providing data scientists with plenty to think about. They know that the traditional attributes of data need to be augmented to deliver dependable and usable insights, to deliver personalization and to forecast the future with confidence.

When the underlying data changes, the models must change. For example, in the wake of a crisis, consumers would normally choose more credit lines to tide over the emergency. But they aren’t doing that. This is because they know that their jobs are at risk. They are instead reducing spends and dipping into their savings. Here is another example—supply chain data is no longer valid, and planners know the pitfalls of using existing data. “It is a dangerous time to depend on (existing) models,” cautions Shalain Gopal, Data Science Manager at ABSA Group, the South Africa-based financial services organization. She believes that organizations should not be too hasty to act on information (data) that could be “half-baked”.

There is good reason to be wary of the data organizations are using. Models are trained on normal human behavior. Given the new developments, it must be trained on data that reflects the “new” normal to deliver dependable outcomes. Gopal says that models are fragile, and they perform badly when they have to handle data that is different from what was used to train them. “It is a mistake to assume that once you set it up (the data and the model) you can walk away from it,” she says.

There are 5 key steps to accelerating Digital Transformation in the “new normal” which dictates how an organization sources and uses data. These provide a way to reimagine data and analytics that lays the foundation for an intelligent enterprise and helps derives maximum insights from data:

• Build a digitally enabled war room for real-time transparency, responsiveness and decision-making
• Overhaul forecasting to adapt to the rapidly changing environment with intelligent scenario-planning
• Rebuild customer trust with personalized digital experiences
• Invest in technology for remote working, operational continuity and security
• Accelerate intelligent automation using data

Events like the Great Depression, 9/11, Black Monday, the 2008 financial crisis, and now the COVID-19 pandemic, are opportunities to create learning models. Once the Machine Learning system ingests what the analytical models should see, forecasting erratic events becomes easier. This implies that organizations must build the ability to maintain and retrain the models and create the right test data with regularity.

ITC Infotech recommends 6 steps to reimagine the data and analytics approach of an organization in the new normal:

1. Harmonize & Standardize the quality of data
2. Enable Unified data access across the enterprise
3. Recalibrate data models on a near real-time basis
4. Amplify data science
5. Take an AI-enabled platform approach
6. Adopt autonomous learning

The ability to make accurate predictions and take better decisions does not depend solely on connecting the data dots—it depends on the quality, accuracy and completeness of the data. Organizations that bring data to the forefront of their operations also know that it is important to understand the right dataset, what the data is being used to solve. In effect, data and analytics have many moving parts. These have become especially important in the light of the changes being forced by COVID-19. Now, there is a rare window of opportunity in which organizations can rapidly adjust their approach to data—and gain an advantage that conventional business wisdom cannot match.

[i] https://www.chron.com/business/article/Toilet-paper-demand-shot-up-…

Co-Authored by :

Shalain Gopal

Data Science Manager, ABSA Group

Kishan Venkat Narasiah

General Manager, DATA, ITC Infotech

Source Prolead brokers usa

Cluster sampling: A probability sampling technique

Source Prolead brokers usa

Chatbot Market and Current Important Statistics

Source Prolead brokers usa

Detect natural disasters with the help of predictive analytics tools

Predictive analytics tools are a key asset in detecting natural disasters. With higher accuracy than other weather detection sensors, they can detect early signs of an oncoming calamity to prevent mistakes like the one that happened in 2016.

On 28th September 2016, weather sensors picked up a storm moving westward of the Lesser Antilles island chain. The storm was a category one threat, which was concerning but manageable. 

Within 24 hours, the threat level increased from category one to category five, turning a storm into a hurricane. 

The hurricane, known as Hurricane Matthew, would go on to inflict immense damage across the Caribbean and the southeastern United States, making its final landfall on October 8th in South Carolina, leaving untold damage in its wake. (The name “Matthew” would eventually be retired because of the damage incurred). 

The National Center for Environmental Information (NCEI) estimated that over $10.3 billion were lost in property damage. While the World Vision assessment team estimated that southwestern Haiti lost over 80-90% of its housing, the damage done to staple food crops would take (at the time) five years to recover from. The devastation inflicted by the hurricane was immense.

However, what is particularly worrying for local and national governments is the growing number of natural disasters.

Research shows that the number of natural disasters is growing—Statistica indicates that the number of natural disasters that took place in 2020 was 416, while there were 411 natural disasters in 2016 (the year Hurricane Matthew devastated Haiti and much of the Caribbean). 

More concerning than this number, is the frequency of natural disasters. Over 207 disasters were recorded from across the world, in the first six months of 2020 alone. 

These disasters are incredibly costly (global costs are estimated at $71 billion), and as Hurricane Matthew shows, can be difficult to predict. 

Amidst such a turbulent environment, what local and national governments need are tools that could help them better anticipate these devastating events. 

The ability to anticipate natural disasters will give them the ability to plan emergency responses and procedures that could mitigate the damage from these events. 

This is where predictive analytics tools play a crucial role. 

Predictive data tools can refine the so meteorologists can make more accurate predictions when a natural weather phenomenon turns into a disaster. 

The science behind predictive analytics tools and natural disasters

The secret is in big data. Previous natural disasters have generated plenty of information, like rainfall, wind levels, and weather patterns, that can be extracted.

Predictive analytics software tools can collect, clean, and analyse this data to gain useful insights into natural disasters. This allows weather departments to better detect the early warning signs in any weather phenomenon, so they will know if a category one rainfall will turn into a category five storm. 

Machine learning algorithms within the analytics platform can collect and analyse the data. The more data it is fed, the deeper the level of understanding the system builds, on the difference between natural disasters and normal weather. 

When a normal weather phenomenon occurs, data analytics platforms can study the weather patterns and compare them against data on previous natural disasters. If current weather patterns match previous data findings, it is a sign that a disaster is impending. 

This is invaluable because meteorologists can use predictive analytics tools to refine the process of detecting natural disasters. They can detect the early warning signs a lot sooner, allowing them to make accurate calls on time, preventing errors similar to what happened with Hurricane Matthews.

Analytics tools can improve weather detection in other areas, for example, refining early warning systems for better accuracy. 

Better natural disaster practices benefit other areas related to disaster management, like emergency response and disaster relief. Local and national governments can improve relief measures and response protocols because they will know what constitutes an emergency and what doesn’t.

Preparing for the future with analytics platforms

Research shows a disturbing trend where natural disasters are growing in frequency due to climate change. While local and national governments cannot undo the causes behind this development overnight, they can improve their response procedures and disaster relief measures to mitigate the damage from these disasters. 

Predictive analytics tools can help in this goal by improving detection methods as analytics platforms pull from previous data sources to improve the detection process and remove uncertainties that could compromise accuracy in natural disaster detection. 

Source Prolead brokers usa

What is data analysis and why is it important?

Data analysis is the process of evaluating data using analytical and statistical tools to discover useful information and help you make business decisions. There are several methods for analyzing data, including data mining, text analysis, business intelligence, and data visualization.Not only does the most complex application data needs analysis but even a simple email database needs analysis and cleansing.

How is data analysis done?
Data analysis is part of the broader process of getting business intelligence. The process includes one or more of the following steps:

Defining Goals: Any research should start with a set of well-defined business goals. Most of the decisions made in the rest of the process depend on how clearly the research objectives are formulated.

Asking questions: An attempt was made to ask a question in a problem area. For example, do red sports cars crash more often than others?

Gathering information: Data relevant to this issue should be obtained from appropriate sources. In the example above, data can be obtained from a variety of sources, including: DMV or police incidents, insurance claims, and hospitalization records. When data is collected through surveys, a questionnaire for subjects must be submitted. Questions should be modeled appropriately for the statistical method used.

Data Processing: Raw data can be collected in several different formats. The collected data must be cleaned and transformed so that the data analysis tools can import it. In our example, we can retrieve DMV crash reports as text files, claims from a relational database, and hospital admissions as APIs. The data analyst must combine these different forms of data and transform them into a form suitable for analysis tools.

Data Analysis: In this step, the cleaned and aggregated data is imported into the analysis tools. These tools allow you to explore your data, find patterns in it, and ask and answer what-if questions. It is the process by which the data gathered in research is made meaningful through the correct application of statistical methods.

Overall, data analysis provides an understanding that businesses need to make effective and efficient decisions. Combined with Data Analytics, they have a good understanding of the needs and capabilities of the company. Data analysis applications are very broad. Big data analytics can optimize the performance of verticals across industries.

Increased productivity enables companies to thrive in an ever-evolving competitive environment and to be successful in many areas. Environmental protection and crime prevention are some of the applications of data analysis. Data analysis apps now look endless.

Every day more and more data is collected, which opens up new opportunities for applying data and improving society and the world.

Reference: www.mastersindatascience.org/learning/what-is-data-analytics/

Source Prolead brokers usa

Explainable AI for Medical Images

Most of what goes by the name of Artificial Intelligence (AI) today is actually based on training and deploying Deep Learning (DL) models. Despite their impressive achievements in fields as diverse as image classification, language translation, complex games (such as Go and chess), speech recognition, and self-driving vehicles, DL models are inherently opaque and unable to explain their predictions, decisions, and actions.

This is not a critical issue for several applications (such as movie recommendation systems or news/social media feed customization, for example) where the end user will evaluate the quality of the AI based on the results it produces, make occasional adjustments to help it improve future results (e.g., by rating additional movies), or move away from that product/app. There is rarely a need to require an explanation for the AI’s decisions when there is very little at stake.

However, for high-stakes situations and mission-critical applications – such as self-driving vehicles, criminal justice decisions, financial systems, and healthcare applications – explainability might be considered crucial. It is not enough to have an AI provide a solution, decision, or diagnosis; it is often necessary (and in some cases, absolutely essential) to explain the process behind such decisions. The emerging field of XAI (eXplainable Artificial Intelligence) addresses this need and offers several methods to provide some level of explanation to deep learning AI solutions.

In this blog post, we focus primarily on the human factors of XAI: What needs to be explained and how can this be communicated to the (human) user of an AI solution?

Fig 1 – Human factors of XAI: an explainable model requires a suitable interface

Recognizing that explainability is desirable when deploying AI solutions in some areas of human activity is only the beginning. In order to understand the why and how behind an AI model’s decisions and get a better insight into its successes and failures, an explainable model must explain itself to a human user through some type of explanation interface (Figure 1). Ideally such interface should be rich, interactive, intuitive, and appropriate for the user and task.

In the field of image classification, a common interface for showing the results of XAI techniques consists of overlaying the “explanation” (usually in the form of a heat map or saliency map) on top of the image. This can be helpful in determining which areas of the image the model deemed to be most relevant for its decision-making process. It can also assist in diagnosing potential blunders that the DL model might be making, producing results that are seemingly correct but reveal that the model was “looking at the wrong places.” Classical examples include the husky vs. wolf image classification algorithm that was, in fact, a “snow detector”, and an image classification solution where the tench, a large fish species, is often identified by human fingers.

In radiology, there is a well-known case where models used to detect pneumonia in chest X-rays had learned to detect a metal token that radiology technicians place on the patient in the corner of the image field of view at the time they capture the image, which in turn identified the source hospital, causing the models to perform well in images from the hospital they were trained on, and poorly in images from other hospitals with different markers.

XAI is often presented as an “all-or-nothing” addition to regular AI, leading to potentially false dichotomies – such as the trade-offs between accuracy and explainability (which suggests that in order to get more of one you must sacrifice the other, which is not universally true) – or vague questions based on abstract scenarios, such as “Would you pay more for AI that explains itself (and performs at the same level as the baseline solution)?”

Fig 2 – XAI as a gradual approach: in addition to the model’s prediction, different types of supporting information can be added to explain the decision.

We choose to see explainability as a gradual process instead (Figure 2), where an AI system that predicts a medical condition from a patient’s chest x-ray might use gradually increasing degrees of explainability: (1) no explainability information, just the outcome/prediction; (2) adding output probabilities for most likely predictions, giving a measure of confidence associated with them; (3) adding visual saliency information describing areas of the image driving the prediction; (4) combining predictions with results from a medical case retrieval (MCR) system and indicating matched real cases that could have influenced the prediction; and (5) adding computer-generated semantic explanation [5].

There is no universally accepted taxonomy of XAI techniques. In this blog post we use the categorization of the field adopted by the authors of a recent survey paper on the topic, which breaks down the field into scope, methodology and usage, as follows:

• Scope: Where is the XAI method focusing on?
  • The scope could be local (where the focus is on individual data instances) or global (where the emphasis is on trying to understand the model as a whole)
• Methodology: What is the algorithmic approach?
  • The algorithm could focus on: (a) gradients back-propagated from the output prediction layer to the input layer; or (b) random or carefully chosen changes to features in the input data instance, also known as perturbations.
• Usage: How is the XAI method developed?
  • Explainability might be intrinsic to the neural network architecture itself (which is often referred to as interpretability) or post-hoc, where the algorithm is applied to already trained networks.

Some post-hoc XAI algorithms have become popular in recent years, among them Grad-CAM (Gradient-weighted Class Activation Mapping) and LIME (Local Interpretable Model-agnostic Explanation). Grad-CAM is gradient-based, has local scope, and can be used for computer vision tasks. LIME is perturbation-based, has both local and global scope, and can be used with images, text, or tabular data.

Here is an example of how to use MATLAB to produce post-hoc explanations (using Grad-CAM and image LIME) for a medical image classification task, defined as follows:

Given a chest x-ray (CXR), our solution should classify it into Posteroanterior (PA) or Lateral (L) view.  

The dataset (PadChest) can be downloaded from https://bimcv.cipf.es/bimcv-projects/padchest/ .

Both methods (gradCAM and imageLIME) are available as part of the MATLAB Deep Learning toolbox and require a single line of code to be applied to results of predictions made by a deep neural network (plus a few lines of code to display the results as a colormap overlaid on the actual images).

Figures 3 and 4 show representative results. The Grad-CAM results (Fig 3) suggest that the network is using information from the borders and corners of the image to make a classification decision. The LIME results (Fig 4) tell a rather different story. In both cases, we are left wondering: to what extent did either method help explain the model’s decision?

Fig 3 – Example of result: Grad-CAM for CXR classification task.

Fig 4 – Example of result: image LIME for CXR classification task.

Here is another example where we applied Grad-CAM and image LIME to results of a binary skin lesion classifier.

Figures 5 and 6 show representative results. They are organized in a confusion matrix-like fashion (with one case each of true positive (TP), true negative (TN), false positive (FP), and false negative (FN)). They reveal fascinating aspects (and limitations) of these post-hoc techniques:

• For easy images (such as the true positive on the bottom-right corner), both techniques perform extremely well and suggest that the model was “looking at the right place within the image”
• For the false positive on the bottom-left corner, both techniques show that the model “learned” the ruler on the left-hand-side of the image (in addition to taking into account aspects from the lesion region as well), which might have explained the incorrect prediction
• For the images on the top row, both techniques suggest that the insights are not as clear (at least to a layperson) as the images on the bottom row; one of the problematic aspects of these post-hoc techniques is actually exemplified here, since they are equally comfortable explaining correct as well as incorrect decisions in a similar way.

Fig 5 – Example of result: Grad-CAM for skin lesion classification task.

Fig 6 – Example of result: image LIME for skin lesion classification task.

Deep Learning models are opaque and, consequently, have been deemed “inadequate” for several applications (since usually they can make predictions without explaining how they arrived at the result, or which factors had greater impact). Explainable AI (XAI) techniques are possible ways to inspire trust in an AI solution.

In this blog post we have used MATLAB to show how post-hoc explanation techniques can be used to show which parts of an image were deemed most important for medical image classification tasks. These techniques might be useful beyond the explanation of correct decisions, since they also help us identify blunders, i.e., cases where the model learned the wrong aspects of the images. 

XAI is an active research area, and new techniques and paradigms are likely to emerge in the near future. If you’re interested in learning more about XAI and related issues, see the  recommended resources below.

Source Prolead brokers usa

It’s Insights Monetization, Not Data Monetization

Okay, this is on me.  I own it.  My blog “Why Data Monetization is a Waste of Time” created quite a stir.  I loved it.  I’d like to take this “data monetization” conversation one step further:

Most organizations should not be focused on data monetization; instead, they should be focused on insights monetization

When I published my first book “Big Data” in September, 2013, I referred to the fourth stage of the “Big Data Business Model Maturity Index as the “Data Monetization” phase.  However, it wasn’t until my latest book “The Economics of Data, Analytics, and Digital Transformation” that I published the end of last year, that I finally got the concept right.  For most organizations, it shouldn’t be “Data Monetization.”  Instead, “Insights Monetization” – customer, product, and operational insights or predicted propensities – is more appropriate (Figure 1).

Figure 1: Data & Analytics Business Maturity Index

The “Insights Monetization” phase of the Data & Analytics Business Maturity Index is described as such:

The Insights Monetization phase requires business leadership to envision (using design thinking) how the organization can leverage and blend their wealth of customer, product, and operational insights (predicted propensities) to create new monetization opportunities including new markets and audiences, new products and services, new channels and partnerships, new consumption models, etc.

That doesn’t mean to preclude data monetization – or the direct selling of one’s data – as an option, but data monetization has a very different context than insights monetization.  Let me explain.

The problem with the term “Data Monetization” is that for many organizations, this implies the direct selling of the organization’s data.  And there are certainly companies out there that do sell data.  Nielsen, Acxiom, Experian, Equifax and CoreLogic are companies whose business model is the acquisition, aggregation, and selling of third-party data.  For example, Figure 2 shows the personal data that one can buy from Acxiom.  Yea, sort of scary.

Figure 2:  Source: “Here are the data brokers quietly buying and selling your personal …”

Selling data requires a significant organization to acquire, cleanse, align, package, market, sell, support, and manage the data for external consumption.  And there is a myriad of growing legal and privacy concerns to navigate, so a pretty decent legal team will be required as well.

For other organizations, data monetization amounts to creating data services that facilitate the exchange of an organization’s data in exchange for something of value from another organization. Walmart’s Retail Link® is an example of this sort of “data monetization.”

Walmart’s Retail Link® gives Walmart suppliers – think Consumer Packaged Goods (CPG) companies like Procter & Gamble, PepsiCo, and Unilever – access to Walmart’s “point of sale” (POS) data. Retail Link provides suppliers access to the supplier’s product sell-through (sales) data by SKU, by hour, by store. Suppliers can also get on-hand inventory by SKU, as well as gross margin achieved, inventory turns, in-stock percentages, and Gross Margin Return on Inventory Investment (Figure 3).

Figure 3: Sample report courtesy of  Trend Results[1]

This is a great example of Data Monetization. Unfortunately, not all organizations have the clout and financial and technology resources of a Walmart to dictate this sort of relationship.  Walmart invests a significant amount of time, money and people resources to on-board, support, maintain, and upgrade Retail Link.  In that aspect, Walmart looks and behaves like an enterprise software vendor.

But for organizations that lack the clout, finances, and technology expertise of a Walmart, there are other “monetization” options.

Insights Monetization is about leveraging the customer, product, and operational insights (predicted propensities) buried in your data sources to optimize and/or reengineer key business and operational processes, mitigate (compliance, regulatory, and business) risks, create new revenue opportunities (such new products, services, audiences, channels, markets, partnerships, consumption models, etc.), and construct a more compelling, differentiated customer experience.

To drive “Insights Monetization” requires some key concepts and techniques.

(1) Nanoeconomics.  Nanoeconomics is the economics of individualized human and/or device predicted propensities.  Nanoeconomics helps organizations transition from overly generalized decisions based upon averages to precision decisions based upon the predicted propensities, patterns, and trends of individual humans or devices (Figure 4).

Figure 4:  Nanoeconomics to Transform Organizational Decision Making

Remember, making decisions based on averages at best yield average results.  We can do better than average by leveraging nanoeconomics to make precision policy and operational decisions.

(2) Analytic Profiles provide a model for capturing and codifying the organization’s customer, product, and operational analytic insights (predicted propensities) in a way that facilities the sharing and refinement of those analytic insights across multiple use cases (Figure 5).

Figure 5: Analytic Profiles

An Analytic Profile capture of metrics, predictive indicators, segments, scores, and business rules that codify the behaviors, preferences, propensities, inclinations, tendencies, interests, associations and affiliations for the organization’s key business entities such as customers, patients, students, athletes, jet engines, cars, locomotives, CAT scanners, and wind turbines.

(3) Use Cases are clusters of Decisions around a common Key Performance Indicator (KPI) where Decisions are a conclusion or resolution reached after analysis that leads to an informed action.  Sample use cases include reduce customer attrition, improve operational uptime, and optimize asset utilization. And the application of the Analytic Profiles can be used to help optimize or address the organization’s top priority use cases (Figure 6).

Figure 6: Role of Analytic Profiles in Optimizing Use Cases

The power of use cases are:

• Readily identifiable by the business stakeholders (because they live them every day)
• Resolution or optimization is a source of quantifiable business and operational value (improving, reducing, or optimizing the use cases can have material business and operational value)
• Encourages tight collaboration between business stakeholders and the data & analytics teams to properly define the objectives, benefits, value statements, and potential impediments of the use case, identify the metrics and KPIs against which use case progress and success will be measured, and codify the associated costs of use case False Positives and False Negatives (see “The Art of Thinking Like a Data Scientist” methodology).
• Provides the basis for a use case-by-use case development and implementation approach – driven by Return on Investment (ROI) – to building the organization’s data and analytic assets (see the “Schmarzo Economic Digital Asset Valuation Theorem“).

The good news:  organizations have a bounty of use cases.  The bad news: organizations don’t fail due to a lack of use cases; they fail because they have too many. 

Organizations tend to fail because they peanut butter their precious data and analytics resources across too many poorly defined use cases, and don’t get the prerequisite business stakeholder buy-in to actually deploy or use the resulting analytics.  Focus baby, focus!

Most organizations should not be focused on data monetization; instead, they should be focused on insights monetization

While there will be examples of companies who can successfully “sell their data” (Nielsen, Walmart), for most organizations their monetization journey will be more about applying their customer, product, and operational insights (or predicted propensities) to optimize their top priority business and operational use cases.

Insights Monetization is about leveraging the customer, product, and operational insights (predicted propensities) buried in your data sources to optimize and/or reengineer key business and operational processes, mitigate (compliance, regulatory, and business) risk create new revenue opportunities (such new products, services, audiences, channels, markets, consumption models, etc.) and create a more compelling, differentiated customer experience.

Heck, it only took me about 7 years to get the distinction between data monetization and insights monetization right.  As Homer would say, DOH!

[1] Trend Results is in no way associated with or endorsed by Wal-Mart Stores, Inc. All references to Wal-Mart Stores, Inc. trademarks and brands are used in strict accordance with the Fair Use Doctrine and are not intended to imply any affiliation of Trend Results with Wal-Mart Stores, Inc. Retail Link is a registered trademark of Wal-Mart Stores, Inc.

Source Prolead brokers usa

Intro to the E-R Diagram

• E-R diagrams capture meaning in your data.
• This high-level visual tool identifies core components of an enterprise
• Simple steps to follow to create the diagram.

Entity-Relationship (E-R) Modeling is one approach to visualize what story your data is trying to tell. This goal of this predecessor to object modeling (e.g. UML or CRC cards) is to give you a high-level, graphical view of the core components of an enterprise—the E-R diagram. An E-R diagram (sometimes called a Chen diagram, after its creator, Peter Chen) is a conceptual graph that captures meaning rather than implementation [1]. Once you have the diagram, you can convert it to a set of tables.

Entity Relationship Diagrams can quickly become very complex and can seem overwhelming to look at for the first time. However, the diagram is built block by block, based on elements which you define. If you’re familiar with your data, and you know some basic E-R diagram symbols (more on that below), you can build an E-R diagram following a few simple rules.

The starting point for making an E-R diagram is to identify a few key items from users. Some key questions to ask [2]:

• What data needs to be kept?
• What queries do we need to ask?
• What business rules should we build in? For example, if the BANK ACCOUNT table has one column for owner, then you have committed to having one owner per bank account.

Next, you want to identify entities. An entity is something about which data is collected, stored or maintained [3]. Look for concrete objects in the problem domain that you can describe with a noun. For example, your core components for a telecommunications company might be WORKERS, CUSTOMERS, CELL PHONES and SERVICE PLANS. You’re looking to describe objects that are clearly distinguishable from other objects, like people, places, and things. “Things” can also be an abstraction like project names or departments.

List the attributes of each entity.  An attribute is a characteristic of an entity—usually a noun. Attributes describe parts of an entity and should be indivisible [ideally] single-valued. For example, TEACHER might have the attributes “name”, “title” and “specialty”. These tend to (more or less) become the fields. Although these should ideally be single (like social security number), they can be layered/composite (like city/state/zip). The domain tells us what values are permissible for an attribute. For example, (John, 123 Front Street, 9999).

Putting the above two components together, each ENTITY is described by a set of attributes. For example:

TEACHER (entity): Id, Name, Address, Parking Decal Number (attributes).

Build Relationships: A relationship is an association between two or more entities. What relationships are present in your data? Employee-supervisor? Teacher-student? Product-Consumer? Sometimes you may have to make a choice if entities and relationships aren’t clear cut. For example, “customer orders” could be an entity, or it could be a relationship.

From this basic information, you can create a simple E-R diagram.  First, you need to be familiar with the meaning of a few basic shapes. Like flow charts, each shape in the E-R diagram has a specific meaning:

• Rectangle: Entity sets.
• Double rectangle: weak entity: one that exists only as the result of another entity. Examples dependents, invoices, transactions.    
• Diamond: Relationship.
• Oval: Attribute.
• Double Oval: attributes that can have multiple values.
• Dashed oval: An attribute that can be derived from others. In other words, it can be calculated from other “stored” attributes in the database. For example, “age” can be derived from “date of birth” and the current date.

The above diagram shows the basic shapes. Start with a few core entities, and build out. As you should follow a few placement rules, like keeping attributes above entities, you’ll want to use software (even something simple like PowerPoint will work) to create one. A few tips for making the E-R diagram:

• Common practice is for entities to be described by a single noun.
• Attributes should be kept above the entity if possible.
• Connect relationship diamonds from the left and right points.
• Use straight connecting lines: do not use arrows.
• Underline key attributes–unique identifiers like social security number or driver license number.

References

Image of Basic E-R diagram: By Author

[1]  Database Design   http://jcsites.juniata.edu/faculty/rhodes/dbms/ermodel.htm

[2] Entity-Relationship modeling http://pld.cs.luc.edu/database/ER.html

[3] Drawing the Entity-Relationship Diagram https://www.csee.umbc.edu/portal/help/oracle8/java.815/a64686/05_de…

Source Prolead brokers usa

AI powered cyberattacks – threats and defence

Source Prolead brokers usa