Ways Artificial Intelligence Impacts the Banking Sector

“75 percent of respondents at banks with over $100 billion in assets say they’re currently implementing AI strategies, compared with 46 percent at banks with less than $100 billion in assets,” UBS Evidence Lab reports.

Artificial Intelligence (AI) has become an integral part of the most demanding and fast-paced industries. The impact of AI in the investment banking industry and financial sector has been phenomenal and it is completely redefining the way they function, create products and services, and how they transform the customer experience. 

In this article, let’s explore a few key ways AI is impacting the investment banking sector.

Improving customer support

Customer satisfaction directly impacts the performance of any enterprise, including those in the investment banking industry. It directly shapes people’s perceptions of the financial institution’s brand. It also influences banks’ client targeting and retention efforts. One of the major issues users face is that financial institutions never seem to be open when they need them most.

For example, what if a customer’s account gets blocked during the holidays? Or, what if a customer wants to learn more about the bank loans later in the day when the employees have already clocked off? The customers’ money never sleeps. Therefore, financial institutions must focus on offering their clients the right services when they need them most.

The AI chatbots and voice assistants are the best for offering customer support. These sophisticated tools are available 24/7 irrespective of their time zone or location, customers can use chatbots for any task that doesn’t require human interaction, such as familiarizing themselves with the services, solving problems, and seeking answers to any question they may have. Most importantly, AI chatbots are constantly learning more about the customers by observing their previous interactions and browsing history in order to serve them with highly personalized user experiences.

Minimizing operating costs

Even though the investment banking industry and financial institutions already use the latest technologies to make their jobs safer and simpler, their employees still need to manage loads of paperwork daily. These kinds of time-intensive and repetitive tasks can cause an increase in operational costs and harm overall employee productivity, which might result in human error. AI eliminates these error-prone human processes.

For example, machine learning, automation tools, AI assistants, and handwriting recognition can streamline several aspects of human jobs. These tools can collect, classify, and enter customer data directly from their contracts and forms. This is a great opportunity for banks to leave manual and repetitive tasks to AI-backed machines and spend more time on creative, high-value works like serving customers with better, highly personalized services or finding new methods of enhancing client satisfaction.

Supporting customers to choose their credit and loans

Financial institutions still depend on factors like one’s credit score, credit history, revenue, and banking transactions to determine whether they’re creditworthy. This is exactly where AI can help, as its analysis goes far beyond the customer data. AI loan decision systems are taking help from ML to observe the patterns and behaviors that help a bank to determine whether a user can really be a good credit customer or not. AI makes credit decision systems more accurate and reliable.

Better regulatory compliance

AI also enhances the way banks impose their regulatory controls. The investment banking industry is one of the biggest regulated industries globally. All banks are required to have reasonable risk profiles to prevent major problems, offer good customer support, and identify the patterns in customer behavior. They rely on tools to identify and prevent the risk of financial crimes like money laundering.

With the growth of AI tools, investment banking has experienced a revolution in its efforts to offer safer and more reliable customer experiences. These pieces of software usually depend on cognitive fraud analytics that observes customer behaviors, track transactions, identify suspicious activities, and assess the data of various compliance systems. Even though these tools haven’t reached their full potential yet, they are already helping banks enhance their regulatory compliance and minimize unnecessary risk.

Enhances risk management

In banking, AI is a major game-changer when it comes to risk management. Financial institutes like the IB sector are prone to risk due to the type of data they handle every day. For instance, banks employ AI-powered solutions, which have the ability to analyze data in huge volumes and can quickly spot patterns from many channels. This helps predict and prevent credit risks and can identify individuals and businesses who might default on their obligation to repay their loans. It can also identify malicious acts such as identity theft and money laundering. AI tools and algorithms have revolutionized risk management in offering a safer and more trusted banking experience. Thus, it is clear that the impact of AI in the IB sector has enhanced risk management.

Wrapping Up      

As the world is pacing briskly towards complete digital transformation, advanced technologies like AI will have a greater impact on the banking sector in the future. The AI will offer more flexible and agile business models for the growing requirements in this digital world.

Source Prolead brokers usa

Enhance Cognitive Bandwidth with Outsourced Web Research Services

The web research process is essential for businesses irrespective of the industry verticals they deal in. From global Corporates to small and medium-sized enterprises to aggregator startups, organizations need online research to take the data-driven approach. This helps them in making informed decisions and scale new heights.

Efficiently performed internet research enables stakeholders to perform competitor analysis. They can learn about the evolving market demographics, shifting consumer preferences, and other factors impacting business growth. Above all, the insights derived after analyzing data help the leaders to understand the demand and supply chain in the market.

This enables the businesses to ace their peers and gains a competitive advantage in the industry. They can identify the gaps and uncover unique opportunities for themselves. Last, but not least, companies can devise effective strategies as well as chalk out roadmaps that bring in growth. This consequently helps them to gain big wins in profit and get incremental ROI.

However, the web research function is a significant undertaking. It is a time-consuming and resource-intensive process that requires dedicated efforts. Otherwise, the analysis can be delayed leading to a slowed-down decision-making process. Managing it along with other core competencies becomes challenging for a majority of organizations.

On the other hand, hiring an in-house team is not always a feasible option. It not only involves a tiring recruitment process, but it also adds to operational expenditures substantially—in terms of technology implementation, employee salaries, infrastructural investments, etc. Instead, a smart option is to engage professional web research services.

Benefits of Engaging Professional Web Research Services

Apart from being a cost-efficient alternative, associating with outsourcing companies enables businesses to utilize their resources strategically. They can enhance the cognitive bandwidth of their employees as well as increase productivity. In addition to this, they can reap a host of other advantages as mentioned here.

• Professional Excellence

The offshoring vendors have a pool of competent professionals hired from around the world. They have hands-on experience in the web research process and know what it takes to cater to the client’s needs. These professionals work as an extended in-house team to help businesses achieve the set goals and strive to deliver excellence in every endeavor.

• Technological Competence

Equipped with the latest technologies, streamlined processes, and a time-tested blend of manual workflows, the external vendors can efficiently scrape out large volumes of data from any number of resources. They leverage the right-fit tools to exceedingly meet the client’s requirements. If needed, the professional providers can also alter their operational approach.

• Industry Compliant Practices

There are numerous data-related laws that have to be taken care of while dealing with such tasks. Business as usual, they stay updated with all the latest norms, rules, and legislation. So, they abide by all such regulations including GDPR, HIPAA, ADA, DDA, CGPA, and so on. All their practices are industry compliant and follow stringent data security protocols to ensure its confidentiality.

• Quality with Accuracy

These two are the most important factors that are considered by businesses before outsourcing such ancillary tasks. The fact is acknowledged by the professional providers and lays focus on quality as well as the accuracy of the outcomes. Having in-built quality check systems, their QA team ensures that the results are error-free. Also, they assure up to 99.99% accuracy.

• Scalability and Flexibility

A conspicuous advantage of engaging professional services is the versatility they provide. This means that the offshoring companies offer the ease of scaling their operations upwards or downwards based on the client’s needs. They have flexible delivery models to ensure that the outcomes are efficient and thorough across different industry verticals.

• Customized Solutions

Every business has distinct requirements. The fact is well addressed by the professional providers. After properly assessing and understanding the client’s pain points, the outsourcing companies offer them a customized and comprehensive suite of offerings including online research services, market research services, internet research services, etc.

Wrapping Up

If you are looking to prospect web research services, you can find a good bunch of options. Make sure that the outsourcing company understands business requirements and project’s needs as well as aligns their outcomes with your unique goals. Therefore, your major goal should be finding the right partner!

Source Prolead brokers usa

DSC Weekly Digest 21 June 2021

One of the more underappreciated problems of working with big data systems, machine learning systems, or knowledge graphs is the fact that the number of classes (types of things) can very quickly number in the hundreds or even thousands. If the data involved in these systems comes primarily from external data stores, this can be problematic even with service interfaces, but where the issue becomes quickly unmanageable is in the realm of user interfaces and user experience.

To give an example, a typical ERP such as Salesforce may contain data for people, locations, transactions, accounts, products, and so on, often numbering in the dozens of different kinds of entities being tracked. All too often, the attitude is that this information comes from databases, but the reality is that somewhere along the lines, someone – a data entry person, an account manager, a customer, a shipper, somebody – will need to enter that information into the computer in the first place. This is actually a source of one of the biggest problems in enterprise data management today.

Why? All too often, the data that goes into one data store represents a very narrow (and almost always programmer-designed) view of various kinds of data. In practice, people do not have access to the underlying data but rather are now reliant upon data services – web services, mainly – that transform an existing record into a frequently lossy JSON, XML or JSON file, losing metadata along the way. This means that keys often become poorly transcribed or lose their context, it means that the ability to add additional properties becomes a major, expensive headache. What’s worse, this process when repeated across dozens of systems, creates an impenetrable thorny wall that reduces or even eliminates any kind of flexibility.

One of the benefits of semantic-based systems (knowledge graphs) is that you can solve several data engineering problems at once. Such knowledge graphs are highly connected, but those connections can be traversed with query languages (SPARQL, GQL, GraphQL, Tinkerpop, and so forth) and can be extended with very little effort. Moreover, it becomes possible to infer structure from data, to hold data connected in multiple ways, and to readily handle true temporality, not just the transactional log focus that’s typical with SQL databases.

Additionally, because of this, systems can use inferential reasoning to be able to “ask” the user to provide just the information that is needed, through generated UI elements. A person in your system changes addresses? Intelligent interface design can bring up just the information needed for updating (or adding) that address, and can even tell by UI cues whether to create new address records or update existing ones – without the need for a programmer to create such a UI screen for this. Similarly, machine learning applications can take a hand-drawn sketch (perhaps with dragged elements) and turn it into a user interface trivially.

This approach has many key benefits – consolidation and significant reduction in duplicate (or near-duplicate) data, major reductions in the time and cost of building applications, reducing or even eliminating the need for complex forms (and perhaps the need for everyone to constantly re-enter resume information), among many others. Additionally, such efforts can create large-scale datastores that dynamically respond to changes in models without overt and expensive software production efforts.

It is the ability to better control the entry of data into the data ecosystem in the first place, not just clever chatbots or expensive data-mining efforts, that will enable data-driven companies to succeed. Control the input, the ingestion, of data so that it is consistent with the underlying models early in the process, and you are well on your way to reducing everything from data cleanliness, master data management, feature engineering and even data analysis systems. However, to do that, it’s time to move beyond SQL and start embracing the graph.

In media res,
Kurt Cagle
Community Editor,
Data Science Central

Source Prolead brokers usa

Stroke Prediction using Data Analytics and Machine Learning

• Data-based decision making is increasing in medicine because of its efficiency and accuracy.
• One branch of research uses Data Analytics and Machine Learning to predict stroke outcomes.
• Models can predict risk with high accuracy while maintaining a reasonable false positive rate.

Stroke is the second leading cause of death worldwide. According to the World Health Organization [1], 5 million people worldwide suffer a stroke every year. Of these, one third die and another third are left permanently disabled. In the United States, someone has a stroke every 40 seconds and every four minutes, someone dies [2]. The aftermath is devastating, with victims experiencing a wide range of disabling symptoms including sudden paralysis, speech loss or blindness due to blood flow interruption in the brain [3]. The economic burden to the healthcare system in the United States amounts to about $34 billion per year in the US [4]. An additional $40 per year is spend on care for elderly stroke survivors [5]. 

Despite these alarming statistics, there is a glimmer of hope: strokes are highly preventable if high risk patients can be identified and encouraged to make lifestyle modifications. While some risk factors like family history, age, gender, and race cannot be modified, it is estimated that 60 to 80% of strokes could be prevented through healthy lifestyle changes like losing weight, smoking cessation, and controlling high cholesterol and blood pressure levels [6, 7].  However, despite the wealth of data on these risk factors, traditional, non-DS methods typically perform quite poorly at predicting who will have a stroke and who will not; Identifying high risk patients is a challenge  because of the complex relationship between contributory risk factors.

Predicting Stroke Outcome with Data Analytics and Machine Learning

Various DA and ML models have been successfully applied in recent years to assess stroke risk factors and outcomes. They include evaluating a mixed-effect linear model to predict the risk of cognitive decline poststroke [8] and developing a deep neural network (DNN) model, applying logistic regression and random forest to predict poststroke motor outcomes [9].  In another study, researchers created a model capable of predicting stroke outcome with high accuracy and validity; The research applied an unbalanced dataset containing information for several thousand individuals with known stroke outcomes. Various algorithms including decision trees, Naïve Bayes and Random Forest were assessed, with random Forest classifier the most promising, predicting stroke outcome with 92% accuracy. The following table shows results for various methods employed in the study [10]:

Improving the False Negative Rate with ML

One of the major challenges with attempting to predict any major disease like stroke is the high cost of false negatives. A false negative result is where the patient has the disease, but the test (or predictive tool) does not identify the patient as having the disease (or risk for the disease). Unlike false negatives in a business setting, false negatives in medicine can have deadly consequences. If someone gets a false negative and is told they are not at risk for a major disease, then they are not in a position to make informed lifestyle choice–putting their life in danger. Historically, false negative rates from traditional approaches exceeds 50%, but this has been reduced to less than 20% by applying machine learning tools [11].

The Future of Stroke Prediction

Data Science is the optimal solution to dealing with the approximately 1.2 billion clinical documents being produced in the United States every year [12]. The results from DS based stroke prediction research look promising. The tools are more reliable and valid than traditional methods; They can also be acquired conveniently and at a low cost [11]. As data science continues to grow within medicine, it  will open new opportunities for more informed healthcare and prevention of deaths from major diseases like stroke.

References

Image: mikemacmarketing / original posted on flickrLiam Huang / clipped and posted on flickr, CC by 2.0 via Wikimedia Commons

[1]  Stroke, Cerebrovascular accident | Health topics – WHO EMRO

[2] Heart Disease and Stroke Statistics—2020 Update: A Report From the …

[3] The science of stroke: Mechanisms in search of treatments

[4] American Heart Association Statistics Committee and Stroke Statisti…

[5] Care received by elderly US stroke survivors may be underestimated.

[6] Preventing Stroke: Healthy Living Habits | cdc.gov.

[7] The science of stroke: Mechanisms in search of treatments,

[8] Risk prediction of cognitive decline after stroke.

[9] Prediction of Motor Function in Stroke Patients Using Machine Learn…,

[10] Stroke prediction through Data Science and Machine Learning Algorithms 

[11] A hybrid machine learning approach to cerebral stroke prediction ba… dataset

[12] Health story project

Source Prolead brokers usa

Top Eleven Skills to Look Out For While Hiring DevOPS Engineer

Companies that incorporate DevOps practices get more done. It is as simple as that. The technical benefits include continuous delivery, easier management, easier to manage, and faster problem-solving. In addition to this, there are cultural benefits like more productive teams, better employee engagement, and better development opportunities.

Source Prolead brokers usa

Why Data Monetization is a Waste for Most Companies

Here, let’s start this blog with some controversy:

“For most organizations, data monetization is a total waste of time”

I’ve been having lots of conversations recently about where the Data and Analytics organization should report.  Good anecdotal insights, but I wanted to complement those conversations with some raw data.  So, I ran a little LinkedIn poll (thanks to the nearly 2,000 people who responded to the poll) that asked the question: “Based upon your experience across different organizations, where does the Data & Analytics organization typically report TODAY?”  The results are displayed in Figure 1.

Figure 1: “Where does the Data & Analytics organization typically report TODAY?”

The poll results in Figure 1 were incredibly disappointing but at least they help me understand why a data monetization conversation for most organizations is a total waste of my time.

From the poll, we learn that in 54% of organizations, the Data & Analytics organization reports to the Chief Information Officer (CIO). The data monetization conversation is doomed when the Data & Analytics organization reports to the CIO. Why? Because the Data & Analytics initiatives are then seen as technology efforts, not business efforts, by the business executives.  And if data and analytics are viewed as technology capabilities and not directly focused on deriving and driving new sources of customer, product, and operational value, then there is no data monetization conversation to be had.  Period.

Arguments for why the Data and Analytics function SHOULD NOT report to the Chief Information Officer (CIO) include:

• The CIO’s primary focus is on keeping the operational systems (ERP, HR, SFA, CRM, BFA, MRM) up and running. If one of these systems go down, then the business grinds to a halt – no orders get taken, no products get sold, no supplies get ordered, no components get manufactured, etc.  Ensuring that these systems never go down (and stay safe from hackers, cyberattacks, and ransomware) is job #1 for the CIO.  Unfortunately, that means that data and analytics are second class citizens in the eyes of the CIO as data and analytics are of less importance to the critical operations of the business.
• And while everyone is quick to point out that the CIO typically has responsibility for the data warehouse and Business Intelligence, the Data Warehouse and BI systems primarily exist to support the management, operational, and compliance reporting needs of the operational systems (see SAP buying Business Objects).
• Finally, the head of the Data & Analytics organization (let’s call them the Chief Data & Analytics Officer or CDAO) needs to be the equal to the CIO when it comes to the senior executive discussions and decisions about prioritizing the organizations technology, data, and analytics investments. If the CDAO reports to the CIO, then the data and analytics investments could easily take a back seat to the operational system investments.

Let’s be very honest here, packaged operational systems are just sources of competitive parity.  I mean, it’s really hard to differentiate your business when everyone is running the same SAP ERP, Siebel CRM, and Salesforce SFA systems.  Plus, no one buys your products and services because you have a better finance or human resources system. 

So, organizations must elevate the role of the data and analytics organization if they are seeking to leverage their data to derive and drive new sources of customer, product, and operational value.  Consequently, the arguments for why the Data and Analytics function (or CDAO) SHOULD report to the CEO, General Manager, or Chief Operating Officer include:

• In the same way that oil was the fuel that drove the economic growth in the 20th century, data will be the driver of economic growth in the 21st century. Data is no longer just the exhaust or byproduct from the operations of the business.  In more and more industries, data IS the business. So, the CDAO role needs to be elevated as an equal in the Line of Business executives to reflect the mission critical nature of data and analytics.
• One of the biggest challenges for the Data and Analytics function is to drive collaboration across the business lines to identify, validate, value, and prioritize the business and operational use cases against which to apply their data and analytics resources. Data and analytics initiatives don’t fail due to a lack of use cases, they fail because they have too many.  As a result, organizations try to peanut butter their limited data and analytic resources across too many use cases leading to under-performance in each of them.  The Data and Analytics function needs to stand as an equal in the C-suite to strategically prioritize the development and application of the data and analytic assets.
• A key priority for the Data and Analytics function is to acquire new sources of internal (social media, mobile, web, sensor, text, photos, videos, etc.) and external (competitive, market, weather, economic, etc.) data that enhances the data coming from the operational systems. These are data sources that don’t typically interest the CIO. The Data and Analytics function will blend these data sources to uncover, codify, and continuously-enhance the customer, product, and operational insights (predicted propensities) across a multitude of business and operational use cases (see the Economics of Data and Analytics).
• It is imperative that all organizations develop a data-driven / analytics-empowered culture where everyone is empowered to envision where and how data and analytics can derive and drive new sources of value. That sort of empowerment must come from the very top of the organization.  Grassroots empowerment efforts are important (see Catalyst Networks), but ultimately it is up to the CEO and/or General Manager to create a culture where everyone is empowered to search for opportunities to exploit the unique economic characteristics of the organization’s data and analytics.

To fully exploit their data monetization efforts, leading-edge organizations are creating an AI Innovation Office that is responsible for:

• Testing, validation, and training on new ML frameworks,
• Professional development of the organization’s data engineering and data science personnel
• “Engineering” ML models into composable, reusable, continuously refining digital assets that can be re-used to accelerate time-to-value and de-risk use case implementation.

The AI Innovation Office typically supports a “Hub-and-Spoke” data science organizational structure (see Figure 2) where:

• The centralized “hub” data scientist team collaborates (think co-create) with the business unit “spoke” data scientist teams to co-create composable and reusable data and analytic assets. The “Hub” data science team is focused on the engineering, reusing, sharing, and the continuous refinement of the organization’s data and analytic assets including the data lake, analytic profiles, and reusable AI / ML models.
• The decentralized “spoke” data science team collaborates closely with its business unit to identify, define, develop, and deploy AI / ML models in support of optimizing the business unit’s most important use cases (think Hypothesis Development Canvas). They employ a collaborative engagement process with their respective business units to identify, validate, value, and prioritize the use cases against which they will focus their data science capabilities.

Figure 2:  Hub-and-Spoke Data Science Organization

The AI Innovation Office can support a data scientist rotation program where data scientists cycle between the hub and the spoke to provide new learning and professional development opportunities. This provides the ultimate in data science “organizational improv” in the ability to move data science team members between projects based upon the unique data science requirements of that particular use case (think Teams of Teams).

Finally, another critical task for the AI Innovation Office is to be a sponsor of the organization’s Data Monetization Council that has the corporate mandate to drive the sharing, reuse, and continuous refinement of the organization’s data and analytic assets. If data and analytics are truly economic assets that can derive and drive new sources of customer, product, and operational value, then the organization needs a governance organization with both “stick and carrot” authority for enforcing the continuous cultivation of these critical 21st century economic assets (see Figure 3).

Figure 3:  Role of Data Monetization Governance Council

A key objective of the Data Monetization Governance Council is to end data silos, shadow IT spend, and orphaned analytics that create a drag on the economic value of data and analytics. And for governance, to be successful, it needs teeth.  Governance must include rewards for compliance (e.g., resources, investments, budget, and executive attention) as well as penalties for non-compliance (e.g., withholding or even clawing back resources, investments, budget, and executive attention).  If your governance practice relies upon cajoling and begging others to comply, then your governance practice has already failed.

So, in summary, yes, for most organizations (54% in my poll), the data monetization conversation is a total waste of time because data monetization conversation doesn’t start with technology but starts with the business. That means that the Data and Analytics function must have a seat in the C-suite, otherwise the data monetization conversation truly is a waste of time.

By the way, I strongly recommend that you check out the individual comments from the nearly 2,000 folks who responded to my LinkedIn poll.  Lots of very insightful and provocative comments.  Yes, that is the right way to leverage social media!!

Source Prolead brokers usa

Good source of coding puzzles for programming interviews

Here is a paper which gives a set of coding puzzles which could be useful for technical interviews in data science.

The paper introduces a new type of programming challenge called programming puzzles, as an objective and comprehensive evaluation of program synthesis, and release an open-source dataset of Python Programming Puzzles (P3).

Each puzzle is defined by a short Python program f, and the goal is to find an input x which makes f output True.

Paper: https://bit.ly/3cQcSFj
Problems: https://bit.ly/2THhBCd
Dataset: https://bit.ly/3zAjLEg

Thanks to Dennis Bakhuis (where I say the paper as a LinkedIn post)

List of puzzles is as below

algebra

• Quadratic Root
• All Quadratic Roots
• Cubic Root
• All Cubic Roots

basic     

• Sum Of Digits
• Float With Decimal Value
• Arithmetic Sequence
• Geometric Sequence
• Line Intersection
• If Problem
• If Problem With And
• If Problem With Or
• If Cases
• List Pos Sum
• List Distinct Sum
• Concat Strings
• Sublist Sum
• Cumulative Sum
• Basic Str Counts
• Zip Str
• Reverse Cat
• Engineer Numbers
• Penultimate String
• Penultimate Rev String
• Centered String

chess    

• Eight Queens Or Fewer
• More Queens
• Knights Tour
• Uncrossed Knights Path
• UNSOLVED_Uncrossed Knights Path

classic_puzzles  

• Towers Of Hanoi
• Towers Of Hanoi Arbitrary
• Longest Monotonic Substring
• Longest Monotonic Substring Tricky
• Quine
• Rev Quine
• Boolean Pythagorean Triples
• Clock Angle
• Kirkman
• Monkey And Coconuts
• No Colinear
• Postage Stamp
• Squaring The Square
• Necklace Split
• Pandigital Square
• All Pandigital Squares
• Card Game
• Easy
• Harder
• Water Pouring
• Verbal Arithmetic
• Sliding Puzzle

codeforces        

• Is Even
• Abbreviate
• Square Tiles
• Easy Twos
• Decreasing Count Comparison
• Vowel Drop
• Domino Tile
• Inc Dec
• Compare In Any Case
• Sliding One
• Sort Plus Plus
• Capitalize Firs tLetter
• Longest Subset String
• Find Homogeneous Substring
• Triple
• Total Difference
• Triple Double
• Repeat Dec
• Shortest Dec Delta
• Max Delta
• Common Case
• Five Powers
• Combination Lock
• Combination Lock Obfuscated
• Invert Permutation
• Same Different
• Ones And Twos
• Min Consecutive Sum
• Max Consecutive Sum
• Max Consecutive Product
• Distinct Odd Sum
• Min Rotations

compression     

• LZW
• LZW_decompress
• Packing Ham

conways_game_of_life 

• Oscillators
• Spaceship

games  

• Nim
• Mastermind
• Tic Tac Toe X
• Tic Tac Toe O
• Rock Paper Scissors

game_theory    

• Nash
• ZeroSum

graphs 

• Conway
• Any Edge
• Any Triangle
• Planted Clique
• Shortest Path
• Unweighted Shortest Path
• Any Path
• Even Path
• Odd Path
• Zarankiewicz
• Graph Isomorphism

ICPC     

• Bi Permutations
• Optimal Bridges
• Checkers Position

IMO      

• Exponential Coin Moves
• No Relative Primes
• Find Repeats
• Pick Near Neighbors
• Find Productive List
• Half Tag

lattices 

• Learn Parity
• Learn Parity With Noise

number_theory

• Fermats Last Theorem
• GCD
• GCD_multi
• LCM
• LCM_multi
• Small Exponent Big Solution
• Three Cubes
• Four Squares
• Factoring
• Discrete Log
• GCD
• Znam
• Collatz Cycle Unsolved
• Collatz Generalized Unsolved
• Collatz Delay
• Lehmer

probability         

• Birthday Paradox
• Birthday Paradox Monte Carlo
• Ballot Problem
• Binomial Probabilities
• Exponential Probability

  Image source walmart jigsaw puzzle

Source Prolead brokers usa

The Lost Art of Decile Analysis

                                                                  Image Source: Author

“Logistic Regression is not Regression but a Classification Algorithm”.

You might have seen this in latest popular machine learning books, blogs or you might have heard *Data Science Gurus* utter the same in their highly subscribed YouTube channels.

Machine learning has usurped and renamed many statistical techniques. Often to the extent that they now disbelieve and reject its statistical origins. Case in point is “Logistic regression is not Regression”

However, nothing can be further from the truth than this assertion. We live in a meme culture, memes even become cryptocurrency.

So why not use a meme to drive home the point that Logistic Regression is indeed Regression.

                                                                Image Source: Author

This meme would have made things clearer for most of you. However, let me unpack it for more clarity.

Logistic Regression models a continuous outcome (the probabilities). The probabilities range between 0 and 1.

Binary Logistic Regression is used as a Classification algorithm when we want the response variable to be dichotomous (Churn/Not Churned, Pass/Fail, Spam/No spam etc.)

Usually, we make Logistic Regression into a classification algorithm by setting an appropriate probability cut-off or threshold (0.4, 0.5, 0.6 etc.).

The problem of classifying using a threshold value

Fixing the probability threshold is purely a business call and not a statistical one. The threshold could vary from one domain/industry to another.

Frank Harrell in his blog¹ aptly makes the point “classification is a forced choice”.

Now consider this example, you choose a threshold value of 0.5. Now the ML algorithm outputs the probability of default or no default (1- default, 0 — no default) for 4 customers as 0.51, 0.49, 0.23 and 0.92. Based on the threshold, 2 are classified as ‘default’ and 2 are classified as ‘no default’. However, ask yourself this, isn’t it too close to call for the customers with probability 0.51 and 0.49? 0.51 is surely closer to 0.49 (which is classified as no default) than it is to 0.92 (which is classified as default).

Some popular machine learning packages and low code tools do not depict the predicted probabilities to the user explicitly. The user is hence oblivious to what predicted probabilities are gotten. He /she simply gets the decision — default or no default (1 or 0). In the case of 0.49 and 0.51, the user happily made the decision that the person will not default and will default respectively. But a peek inside the predicted probabilities reveals that it was too close to call!

The other problem with thresholds is that, when we use an improper scoring rule like classification accuracy, it can be easily gamed. For e.g., if out of 100 people, 95 of them default on loan, and 5 don’t. If the classifier classifies everyone to default on loan, then it would have an accuracy of 95% !!

So, is there a better way to use Logistic Regression?

The answer is Yes.

Industries such as Finance and Marketing use Logistic Regression in a more appropriate way for Credit risk modelling and marketing campaign targeting respectively.

A real use case

Let us say, you are a CMO in charge for sales and marketing of a product in your organization. You plan to launch a marketing campaign to increase sales of that product. You have been given a fixed budget for this. Now you would like to get the best ROI possible i.e., spend the exact fixed budget or even lesser and get the maximum sales possible. Here is what you have.

You have the data for 10,000 customers who had either purchased or not purchased a similar product in the past.

You would like to understand which customers should be targeted to increase the likelihood of purchase this time.

You would obviously like to target those people who are more likely to buy the product since you have a fixed budget for the campaign. How to go about it?

What is a Decile Analysis?

Decile analysis was once a popularly used technique, however the convention of teaching and bucketing machine learning problems into either ‘classification’ or ‘Regression’ types, lead people to forget Decile analysis type analyses. I am pretty sure, most freshly minted data scientists would not have even heard of Decile analysis. So, coming back to what is Decile Analysis.

Decile Analysis is used to categorize dataset from highest to lowest values or vice versa. (Based on predicted probabilities)

As obvious from the name, the analysis involves dividing the dataset into ten equal groups. Each group should have the same no. of observations/customers.

It ranks customers in the order from most likely to respond to least likely to respond.

How is Decile Analysis done?

Step 1: Build a Logistic Regression model. In this case the dependent variable was ‘probability of buying the product’. With 1 denoting purchased, 0 denoting not purchased. Also, the relevant independent variables were selected.

Step 2: Get the predicted probabilities from Logistic Regression algorithm. Arrange the probabilities in descending order.

Step 3: Divide the whole dataset into 10 groups, each group should contain equal no. of observations. So, if there 10,000 records, each group would have 1000 records/customers.

Step 4: Compute % of responders for each decile

Step 5: Compute response rate for each decile

Step 6: Compute lift for each decile

The top decile would have customers who are most likely to respond followed by the decile 2 which would have customers who are next most likely to respond and so on.

One major advantage of the decile analysis is that the probabilities and probability range are their own error measures. That is, if the top decile has the probability range 0.75–0.81, then then the probability of this person not buying the product i.e., the error would be (1- [0.75–0.81], here ‘[]’ denotes the interval of values between 0.75–0.81 including 0.75 and 0.81)

What does a Decile Analysis output look like?

The below table depicts a typical decile analysis output.

                                                     Table 1 – Source: Author

Let’s unpack the output table.

As stated earlier, each decile has equal number of customers (1000 in each decile).

% of responders for each decile = No. of responders in that decile/ No. of total responders for all 10 deciles

Refer to Table 1:

Here, % of responders for Decile 1 = 224/984 = 22.8%

984 is the total no. of responders in all 10 deciles

Similarly, % of responders for Decile 2 = 16.5% (162/984)

And cumulative % of responders for top 2 deciles = 39.2%

Gains and Gain Chart

From table 1, Decile 1 contains top 10% of the customers who are most likely to buy. Decile 1 has the highest no. of responders (224 responders).

So out of the total responders in all deciles, 22.8% of the responders are in Decile 1. So, for 10% of the customer base, there are 22.8% responses in Decile 1.

Similarly for 20% of the customer base in decile 1 and decile 2, there are 39.2% responses.

Gain Chart below explains this better.

                                                                              Image Source: Author

Takeaway from Gain Chart: Gain Chart can be used to assess what % of customers respond in each decile. So, rather than targeting customers from lower deciles, customers can be picked up from the top deciles only.

The dashed line is the Baseline. Baseline tells how much % of customers would respond if we target customers randomly without a model.

Response Rate

Response Rate tells what is the percentage of customers who responded in each decile. The response % is highest in Decile 1 followed by Decile 2 and so on.

Response Rate for each decile = No. of respondent in that decile/ No. of customers in that decile

Refer to Table 1:

Here, Response Rate for Decile 1 = 224/ 1000 = 22.4%

Takeaway from Response Rates comparison: Response Rate of each decile is depicted in the chart below. The average response rate of all deciles is 9.8%. So, customers from Decile 1 to Decile 4 lie above the average response rate and should be targeted for the campaign.

                                                            Image Source: Author

Lift and Lift Curve

Lift = cumulative % of responders / customers % at each decile

Refer to Table 1:

Lift for Decile 1 = 22.8%/10% = 2.28

Lift for Decile 2 = 39.2%/20% = 1.96

How to interpret: If we target top two deciles, then we would target 20% of the customers. In the same deciles, the cumulative % of responders are 39.2%. Hence, there is a lift of 1.96.

A lift of 1 means there is no gain compared with the no. of customers targeted at random. Lift greater than 1 means, model approach is better than selecting the customers at random.

Takeaway from Lift Chart: Can be used to identify the deciles which have a higher Lift.

                                                                 Image Source: Author

How to use Decile Analysis in business decision making:

Now that we have done building Decile Analysis, the next pertinent question is how we use it to make effective business decisions:

Let us revisit our Decile Analysis Table

                                                                             Image Source: Author

References:

1. https://www.fharrell.com/post/classification/

Source Prolead brokers usa

More Fun Math Problems for Machine Learning Practitioners

This is part of a series featuring the following aspects of machine learning:

This issue focuses on cool math problems that come with data sets, source code, and algorithms. See previous article here. Many have a statistical, probabilistic or experimental flavor, and some are dealing with dynamical systems. They can be used to extend your math knowledge, practice your machine learning skills on original problems, or for curiosity. My articles, posted on Data Science Central, are always written in simple English and accessible to professionals with typically one year of calculus or statistical training, at the undergraduate level. They are geared towards people who use data but are interesting in gaining more practical analytical experience. The style is compact, geared towards people who do not have a lot of free time. 

Despite these restrictions, state-of-the-art, of-the-beaten-path results as well as machine learning trade secrets and research material are frequently shared. References to more advanced literature (from myself and other authors) is provided for those who want to dig deeper in the interested topics discussed. 

1. Fun Math Problems for Machine Learning Practitioners

These articles focus on techniques that have wide applications or that are otherwise fundamental or seminal in nature.

1. New Mathematical Conjecture?
2. Cool Problems in Probabilistic Number Theory and Set Theory
3. Fractional Exponentials – Dataset to Benchmark Statistical Tests
4. Two Beautiful Mathematical Results – Part 2
5. Two Beautiful Mathematical Results
6. Four Interesting Math Problems
7. Number Theory: Nice Generalization of the Waring Conjecture
8. Fascinating Chaotic Sequences with Cool Applications
9. Representation of Numbers with Incredibly Fast Converging Fractions
10. Yet Another Interesting Math Problem – The Collatz Conjecture
11. Simple Proof of the Prime Number Theorem
12. Factoring Massive Numbers: Machine Learning Approach
13. Representation of Numbers as Infinite Products
14. A Beautiful Probability Theorem
15. Fascinating Facts and Conjectures about Primes and Other Special Nu…
16. Three Original Math and Proba Challenges, with Tutorial
17. Challenges of the week

2. Free books

• Statistics: New Foundations, Toolbox, and Machine Learning Recipes

  Available here. In about 300 pages and 28 chapters it covers many new topics, offering a fresh perspective on the subject, including rules of thumb and recipes that are easy to automate or integrate in black-box systems, as well as new model-free, data-driven foundations to statistical science and predictive analytics. The approach focuses on robust techniques; it is bottom-up (from applications to theory), in contrast to the traditional top-down approach.

  The material is accessible to practitioners with a one-year college-level exposure to statistics and probability. The compact and tutorial style, featuring many applications with numerous illustrations, is aimed at practitioners, researchers, and executives in various quantitative fields.

• Applied Stochastic Processes

  Available here. Full title: Applied Stochastic Processes, Chaos Modeling, and Probabilistic Properties of Numeration Systems (104 pages, 16 chapters.) This book is intended for professionals in data science, computer science, operations research, statistics, machine learning, big data, and mathematics. In 100 pages, it covers many new topics, offering a fresh perspective on the subject.

  It is accessible to practitioners with a two-year college-level exposure to statistics and probability. The compact and tutorial style, featuring many applications (Blockchain, quantum algorithms, HPC, random number generation, cryptography, Fintech, web crawling, statistical testing) with numerous illustrations, is aimed at practitioners, researchers and executives in various quantitative fields.

To receive a weekly digest of our new articles, subscribe to our newsletter, here.

About the author:  Vincent Granville is a data science pioneer, mathematician, book author (Wiley), patent owner, former post-doc at Cambridge University, former VC-funded executive, with 20+ years of corporate experience including CNET, NBC, Visa, Wells Fargo, Microsoft, eBay. Vincent is also self-publisher at DataShaping.com, and founded and co-founded a few start-ups, including one with a successful exit (Data Science Central acquired by Tech Target). He recently opened Paris Restaurant, in Anacortes. You can access Vincent’s articles and books, here.

Source Prolead brokers usa

In a Cloud-Native World, It’s Time to Rethink Data Storage

Digital transformation has created new product and service capabilities and untold additional yottabytes of data. It has become increasingly clear that data is a key creator of value. Take, for example, the realm of digital entertainment. For proof, just scan your monthly credit card bill for streaming service subscriptions. Also, take a moment to think of truly impactful digital content — the MRI that aids a doctor in early disease detection for a patient, the genome data that helps unlock a cure and the convenience of planning our daily lives online for work, family, travel and entertainment.

There has been a corresponding change in how data is created, stored and consumed. People generate data both in their business and personal lives, but we now also see machine-generated data being created at a massive pace in manufacturing locations, utilities, vehicles and so on. Data lives in our homes, cars, on cruise ships, in airplanes, hospitals, sports stadiums and many more places.

Consequently, organizations need to create a plan for infrastructure to consume, manage, store and protect data anywhere. This now translates into data everywhere, from the data center to the cloud and to the emerging “edge” — and this edge is a dramatically growing area of technology innovation and consumption.

Data storage’s level playing field

A decade or two ago, the storage administrator was the employee who managed storage within the enterprise data center. These deeply knowledgeable and technical professionals understand that protecting data is key to their business success and making it consumable to the right people (and only the right people) is the primary objective of their jobs. Understanding how data is stored, its formats and how it is accessed and consumed gave rise to a specialized world of users who understand the speeds and feeds of storage and fluently speak the language of technical data storage acronyms.

As change continues at record pace, it’s no longer just the enterprise IT staff who have the responsibility of capturing, protecting and giving access to data storage. It has become the domain of a broad range of application owners and technical architects as well as highlighted the role of development operations or “DevOps” teams. This collection of people now makes critical decisions within enterprises for solutions — which encompass applications, people, processes and infrastructure — and all of these decisions are made in a more independent manner than before.

Cloud-native shakes things up

Whereas we used to hear about enterprise resource planning (ERP) and business process re-engineering (BPR), we now hear about business applications, data lakes, big data analytics, artificial intelligence and machine learning. These workloads are driving major changes in data, how much of it needs to be stored and how it gets consumed.

Workloads of this type welcome modern design methodologies and principles in application development, design and deployment. This new wave, termed cloud-native, includes the use of distributed software services packaged and deployed as containers and orchestrated on Kubernetes. The promises of this new approach include efficiency, scalability and — very importantly — portability. The latter aspect will allow software applications and infrastructure to support the new dynamic described earlier: data is created and lives everywhere.

That’s the technical aspect of the change. The storage aspect sees that cloud-native applications will also change how storage is accessed, provisioned and managed. This is a world of software services and interactions between services through well-defined interfaces or APIs. Storage has historically been an area where standard interfaces have been adopted. In the realm of file systems, specifically, there are well-known SMB and NFS protocols. 

For cloud-native applications, there is a natural fit of API-based access to storage, which object storage supports naturally through its RESTful APIs. The popular Amazon S3 API is now fully embraced by independent software vendors (ISVs) and storage vendors alike for the cloud, data center and the edge. APIs also apply to storage management and monitoring, and API-based automation is another central theme in this cloud-native wave.

Future-proofing storage

Object storage brings all the right ingredients together – offering portability, API-based access, automation and scalability to effectively unbounded levels – to be the optimal storage model for the new cloud-native world. Next-generation object storage solutions can and will go further in providing higher levels of performance for new applications and workloads and will also provide simplicity of operations to ensure that wider ranges of users will be able to fully exploit them.

Data storage and management have become increasingly complex in the age of apps. Demands shift with the technology, mandating a new method of data management and delivery. Lightweight, cloud-native object storage is what’s needed to power this next generation of cloud-native applications throughout their entire lifecycle – no matter where your data resides.

Source Prolead brokers usa